kolla項目是為了容器化openstack，目標是做到100個節點的開箱即用，所有的組件的HA都具備。kolla是一個革命性的項目，我們以前積累的安裝部署經驗，全部都報廢。使用kolla可以快速部署可擴展，可靠的生產就緒的openstack環境。
基本環境
操作系統：CentOS Linux release 7.2.1511 (Core)
內核版本：3.10.0-327.28.3.el7.x86_64
docker版本：Docker version 1.12.1, build 23cf638
部署kolla
1. 安裝依賴
yum install epel-release python-pip
yum install -y python-devel libffi-devel openssl-devel gcc
pip install -U pip |
2. 修改docker啟動文件
# Create the drop-in unit directory for docker.service
mkdir -p /etc/systemd/system/docker.service.d
# Create the drop-in unit file
tee /etc/systemd/system/docker.service.d/kolla.conf <<-'EOF'
[Service]
MountFlags=shared
EOF |
3. 重啟docker
systemctl daemon-reload
systemctl restart docker |
4. 安裝docker的python庫
yum install python-docker-py
or
pip install -U docker-py |
5. 配置時間同步（此處略）
6. 禁用libvirt
systemctl stop libvirtd.service
systemctl disable libvirtd.service |
7. 安裝ansible
這里需要注意的是如果安裝stable版的kolla需要Ansible < 2.0，master版需要Ansible > 2.0。默認yum安裝ansible版本>2.0，因為我要安裝stable/mitaka版，所有指定安裝版本。
pip install -U ansible==1.9.4 |
8. 安裝stable版kolla
-
下載源碼
git clone https://git.openstack.org/openstack/kolla -b stable/mitaka |
-
安裝依賴
pip install -r kolla/requirements.txt -r kolla/test-requirements.txt |
-
源碼安裝
9. 安裝tox，生成配置文件
pip install -U tox
cd kolla/
tox -e genconfig
cp -rv etc/kolla /etc/ |
10. 安裝python client
yum install python-openstackclient python-neutronclient |
11. 本地docker倉庫
all-in-one環境中本地倉庫不是必須的這里沒有配置
編譯鏡像
更多的編譯選項可以參看：Building Container Images
如果個別鏡像編譯失敗可以重新執行以上操作，因為docker的容器緩存，重新編譯會很快
編譯成功后生成的鏡像如下所示：
# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
kolla/centos-binary-heat-engine 2.0.3 28956cc878d3 20 hours ago 571.4 MB
kolla/centos-binary-heat-api-cfn 2.0.3 d69858fd13fa 20 hours ago 571.4 MB
kolla/centos-binary-heat-api 2.0.3 90a92ca6b71a 20 hours ago 571.4 MB
kolla/centos-binary-heat-base 2.0.3 8f1cf8a1f536 21 hours ago 551.6 MB
kolla/centos-binary-neutron-openvswitch-agent 2.0.3 e7d0233ca541 21 hours ago 822.3 MB
kolla/centos-binary-neutron-base 2.0.3 8767569ca9b3 21 hours ago 796.7 MB
kolla/centos-binary-openvswitch-vswitchd 2.0.3 6867586ae335 21 hours ago 330.6 MB
kolla/centos-binary-openvswitch-db-server 2.0.3 3c692f316662 21 hours ago 330.6 MB
kolla/centos-binary-openvswitch-base 2.0.3 c3a263463f8f 21 hours ago 330.6 MB
kolla/centos-binary-cron 2.0.3 d16d53e85ed9 26 hours ago 317.5 MB
kolla/centos-binary-kolla-toolbox 2.0.3 1fd9634b88ee 26 hours ago 568.4 MB
kolla/centos-binary-heka 2.0.3 627a3de5e91c 26 hours ago 371.1 MB
kolla/centos-binary-neutron-metadata-agent 2.0.3 aad43ed7a5a1 42 hours ago 796.7 MB
kolla/centos-binary-neutron-server 2.0.3 bc1a7c0ec402 42 hours ago 796.7 MB
kolla/centos-binary-nova-compute 2.0.3 619344ac721b 42 hours ago 1.055 GB
kolla/centos-binary-nova-libvirt 2.0.3 6144729fff5f 42 hours ago 1.106 GB
kolla/centos-binary-neutron-linuxbridge-agent 2.0.3 720c9c5fa63d 42 hours ago 822 MB
kolla/centos-binary-neutron-l3-agent 2.0.3 3a82df7cb9c2 42 hours ago 796.7 MB
kolla/centos-binary-glance-api 2.0.3 fb67115357d5 42 hours ago 673.8 MB
kolla/centos-binary-neutron-dhcp-agent 2.0.3 8c6fa56497ca 42 hours ago 796.7 MB
kolla/centos-binary-nova-compute-ironic 2.0.3 6f235dc430e5 43 hours ago 1.019 GB
kolla/centos-binary-glance-registry 2.0.3 f4cf7bc1536f 43 hours ago 673.8 MB
kolla/centos-binary-cinder-volume 2.0.3 0197cc13468d 43 hours ago 788.4 MB
kolla/centos-binary-cinder-api 2.0.3 ed7c623e7364 43 hours ago 800.4 MB
kolla/centos-binary-cinder-rpcbind 2.0.3 75466dc5a3ba 43 hours ago 790.2 MB
kolla/centos-binary-horizon 2.0.3 92c7ea9fc493 43 hours ago 703.1 MB
kolla/centos-binary-cinder-backup 2.0.3 e3ee19440831 43 hours ago 761.3 MB
kolla/centos-binary-cinder-scheduler 2.0.3 e3ee19440831 43 hours ago 761.3 MB
kolla/centos-binary-nova-consoleauth 2.0.3 96a9638801cd 43 hours ago 609.6 MB
kolla/centos-binary-nova-api 2.0.3 eff73f704a90 43 hours ago 609.4 MB
kolla/centos-binary-nova-conductor 2.0.3 6016ae01a60d 43 hours ago 609.4 MB
kolla/centos-binary-nova-scheduler 2.0.3 726f100a5533 43 hours ago 609.4 MB
kolla/centos-binary-nova-spicehtml5proxy 2.0.3 c6a1a49e4226 43 hours ago 609.9 MB
kolla/centos-binary-glance-base 2.0.3 1e4efa0f6701 43 hours ago 673.8 MB
kolla/centos-binary-nova-network 2.0.3 87f6389dd11a 43 hours ago 610.4 MB
kolla/centos-binary-ironic-pxe 2.0.3 82f25f73c28f 43 hours ago 574.2 MB
kolla/centos-binary-nova-novncproxy 2.0.3 4726875ed228 43 hours ago 610.1 MB
kolla/centos-binary-nova-ssh 2.0.3 51c70b9e9c47 43 hours ago 610.4 MB
kolla/centos-binary-cinder-base 2.0.3 7c2d031be713 43 hours ago 761.3 MB
kolla/centos-binary-keystone 2.0.3 c51a93cc9e2e 43 hours ago 585.2 MB
kolla/centos-binary-ironic-api 2.0.3 b1771f5cc27f 43 hours ago 570.6 MB
kolla/centos-binary-ironic-inspector 2.0.3 32f4e33e1037 43 hours ago 576.2 MB
kolla/centos-binary-ironic-conductor 2.0.3 d552c64f3a08 43 hours ago 599 MB
kolla/centos-binary-nova-base 2.0.3 8f077fafc5d8 43 hours ago 588.7 MB
kolla/centos-binary-rabbitmq 2.0.3 d9e543e4f179 43 hours ago 370.3 MB
kolla/centos-binary-ironic-base 2.0.3 6c4c453ddbce 43 hours ago 550.8 MB
kolla/centos-binary-openstack-base 2.0.3 cf48d5b3f3ee 43 hours ago 518.2 MB
kolla/centos-binary-mariadb 2.0.3 cd9b363fe034 43 hours ago 630.5 MB
kolla/centos-binary-memcached 2.0.3 49c536466427 43 hours ago 354.6 MB
kolla/centos-binary-base 2.0.3 d04ac1ecd01a 43 hours ago 300 MB
centos latest 980e0e4c79ec 2 days ago 196.7 MB |
部署容器
1. 生成密碼
openstack環境的密碼等變量可以在 /etc/kolla/passwords.yml 中指定，為了方便可以使用kolla-genpwd工具自動生成復雜密碼。
為了方便，我們修改其中的管理員登陸密碼
vim /etc/kolla/passwords.yml
keystone_admin_password: admin |
2. 修改部署配置文件
修改/etc/kolla/globals.yml 文件，指定部署的一些信息
vim /etc/kolla/globals.yml
kolla_base_distro: "centos"
kolla_install_type: "binary"
enable_haproxy: "no"
#kolla_internal_vip_address: "10.10.10.254"
kolla_internal_address: "192.168.2.120"
network_interface: "ens160"
neutron_external_interface: "ens192"
neutron_plugin_agent: "openvswitch"
openstack_logging_debug: "True" |
3. 檢查配置
4. 開始部署
5. 部署成功后查看容器
# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3938136934cf kolla/centos-binary-horizon:2.0.3 "kolla_start" 17 hours ago Up 17 hours horizon
cc68cb8d96e4 kolla/centos-binary-heat-engine:2.0.3 "kolla_start" 17 hours ago Up 17 hours heat_engine
96c94995ef7c kolla/centos-binary-heat-api-cfn:2.0.3 "kolla_start" 17 hours ago Up 17 hours heat_api_cfn
cb8ae3afb767 kolla/centos-binary-heat-api:2.0.3 "kolla_start" 17 hours ago Up 17 hours heat_api
e8f98659e03f kolla/centos-binary-neutron-metadata-agent:2.0.3 "kolla_start" 17 hours ago Up 17 hours neutron_metadata_agent
d326fa732c2b kolla/centos-binary-neutron-l3-agent:2.0.3 "kolla_start" 17 hours ago Up 17 hours neutron_l3_agent
4b1bbbe4fe5b kolla/centos-binary-neutron-dhcp-agent:2.0.3 "kolla_start" 17 hours ago Up 17 hours neutron_dhcp_agent
88b2afbba5d9 kolla/centos-binary-neutron-openvswitch-agent:2.0.3 "kolla_start" 17 hours ago Up 17 hours neutron_openvswitch_agent
b73d52de75b2 kolla/centos-binary-neutron-server:2.0.3 "kolla_start" 17 hours ago Up 17 hours neutron_server
1c716402d95f kolla/centos-binary-openvswitch-vswitchd:2.0.3 "kolla_start" 17 hours ago Up 17 hours openvswitch_vswitchd
176e7ee659f1 kolla/centos-binary-openvswitch-db-server:2.0.3 "kolla_start" 17 hours ago Up 17 hours openvswitch_db
457e0921c61a kolla/centos-binary-nova-ssh:2.0.3 "kolla_start" 17 hours ago Up 17 hours nova_ssh
b02acebb3dc3 kolla/centos-binary-nova-compute:2.0.3 "kolla_start" 17 hours ago Up 17 hours nova_compute
59be78a597d8 kolla/centos-binary-nova-libvirt:2.0.3 "kolla_start" 17 hours ago Up 17 hours nova_libvirt
668ad8f91920 kolla/centos-binary-nova-conductor:2.0.3 "kolla_start" 17 hours ago Up 17 hours nova_conductor
34f81b4bc18b kolla/centos-binary-nova-scheduler:2.0.3 "kolla_start" 17 hours ago Up 17 hours nova_scheduler
eb47844e6547 kolla/centos-binary-nova-novncproxy:2.0.3 "kolla_start" 17 hours ago Up 17 hours nova_novncproxy
93563016cf21 kolla/centos-binary-nova-consoleauth:2.0.3 "kolla_start" 17 hours ago Up 17 hours nova_consoleauth
cc8a1cca2e98 kolla/centos-binary-nova-api:2.0.3 "kolla_start" 17 hours ago Up 17 hours nova_api
40db89e89758 kolla/centos-binary-glance-api:2.0.3 "kolla_start" 17 hours ago Up 17 hours glance_api
4fa5f0f38f0d kolla/centos-binary-glance-registry:2.0.3 "kolla_start" 17 hours ago Up 17 hours glance_registry
f05120c95a9f kolla/centos-binary-keystone:2.0.3 "kolla_start" 17 hours ago Up 17 hours keystone
149a49d57aa6 kolla/centos-binary-rabbitmq:2.0.3 "kolla_start" 17 hours ago Up 17 hours rabbitmq
5f4298c3821e kolla/centos-binary-mariadb:2.0.3 "kolla_start" 17 hours ago Up 17 hours mariadb
64f6fbb19892 kolla/centos-binary-cron:2.0.3 "kolla_start" 17 hours ago Up 17 hours cron
4cab0e756b61 kolla/centos-binary-kolla-toolbox:2.0.3 "/usr/local/bin/dumb-" 17 hours ago Up 17 hours kolla_toolbox
293a7ccaab52 kolla/centos-binary-heka:2.0.3 "kolla_start" 17 hours ago Up 17 hours heka
6dcf3a2c12cc kolla/centos-binary-memcached:2.0.3 "kolla_start" 17 hours ago Up 17 hours memcached |
6. 修改虛擬化類型
因為是在虛擬機中安裝，不支持kvm，需要修改虛擬類型為qemu
vim /etc/kolla/nova-compute/nova.conf
[libvirt]
...
virt_type=qemu |
然后就可以通過 kolla_internal_address 訪問openstack環境
一些有用的工具
-
部署完成后，運行以下命令可以生成一個openrc文件(運行openstack CLI所需的環境變量):
kolla-ansible post-deploy |
-
openrc文件生成之后，使用以下命令可以幫你做一下openstack的初始化工作，包括上傳一個glance鏡像以及創建幾個虛擬網絡:
source /etc/kolla/admin-openrc.sh
kolla/tools/init-runonce |
-
由於錯誤的出現，可能需要多次的部署，而有些錯誤重新部署是不會進行修正的，所以需要將整個環境進行清理:
tools/cleanup-containers #可用於從系統中移除部署的容器
tools/cleanup-host #可用於移除由於殘余網絡變化引發的docker啟動的neutron-agents主機
tools/cleanup-images #可用於從本地緩存中移除所有的docker image |
日志查看
kolla通過heka容器來收集所有容器的日志
docker exec -it heka bash |
所有的容器都可以從這個目錄中獲取服務日志：/var/log/kolla/SERVICE_NAME。
如果需要輸出日志，請運行：
大多數容器不會stdout，上面的命令將不會提供信息。
出錯處理
deploy時遇到以下錯誤:
TASK: [rabbitmq | fail msg="Hostname has to resolve to IP address of api_interface"] ***
failed: [localhost] => (item={'cmd': ['getent', 'ahostsv4', 'localhost'], 'end': '2016-06-24 04:51:39.738725', 'stderr': u'', 'stdout': '127.0.0.1 STREAM localhost\n127.0.0.1 DGRAM \n127.0.0.1 RAW \n127.0.0.1 STREAM \n127.0.0.1 DGRAM \n127.0.0.1 RAW ', 'changed': False, 'rc': 0, 'item': 'localhost', 'warnings': [], 'delta': '0:00:00.033351', 'invocation': {'module_name': u'command', 'module_complex_args': {}, 'module_args': u'getent ahostsv4 localhost'}, 'stdout_lines': ['127.0.0.1 STREAM localhost', '127.0.0.1 DGRAM ', '127.0.0.1 RAW ', '127.0.0.1 STREAM ', '127.0.0.1 DGRAM ', '127.0.0.1 RAW '], 'start': '2016-06-24 04:51:39.705374'}) => {"failed": true, "item": {"changed": false, "cmd": ["getent", "ahostsv4", "localhost"], "delta": "0:00:00.033351", "end": "2016-06-24 04:51:39.738725", "invocation": {"module_args": "getent ahostsv4 localhost", "module_complex_args": {}, "module_name": "command"}, "item": "localhost", "rc": 0, "start": "2016-06-24 04:51:39.705374", "stderr": "", "stdout": "127.0.0.1 STREAM localhost\n127.0.0.1 DGRAM \n127.0.0.1 RAW \n127.0.0.1 STREAM \n127.0.0.1 DGRAM \n127.0.0.1 RAW ", "stdout_lines": ["127.0.0.1 STREAM localhost", "127.0.0.1 DGRAM ", "127.0.0.1 RAW ", "127.0.0.1 STREAM ", "127.0.0.1 DGRAM ", "127.0.0.1 RAW "], "warnings": []}}
msg: Hostname has to resolve to IP address of api_interface
FATAL: all hosts have already failed -- aborting
PLAY RECAP ********************************************************************
to retry, use: --limit @/root/site.retry
localhost : ok=87 changed=24 unreachable=0 failed=1 |
解決辦法：
vim /etc/hosts
127.0.0.1 localhost
192.168.2.120 localhost |
