https://www.cnblogs.com/diantong/p/11911503.html 推薦
https://blog.csdn.net/networken/article/details/106745167 學習
轉載：https://blog.csdn.net/networken/article/details/106728002
安裝環境准備
官方部署文檔：
https://docs.openstack.org/kolla-ansible/train/user/quickstart.html
kolla安裝節點要求：
2 network interfaces
8GB main memory
40GB disk space
本次部署train版all-in-one單節點，使用一台centos7.8 minimal節點進行部署，該節點同時作為控制節點、計算節點、網絡節點和cinder存儲節點使用，同時也是kolla ansible的部署節點。
kolla的安裝要求目標機器至少兩塊網卡，本次安裝使用2塊網卡對應管理網絡和外部網絡兩個網絡平面，在vmware workstation虛擬機新增一塊網卡ens37：
ens33，NAT模式，管理網絡，正常配置靜態IP即可。租戶網絡與該網絡復用，租戶vm網絡不單獨創建網卡
ens37，橋接模式，外部網絡，無需配置IP地址，這個其實是讓neutron的br-ex 綁定使用，虛擬機通過這塊網卡訪問外網。
網卡配置信息，大部分為默認參數
[root@kolla ~]# cat /etc/sysconfig/network-scripts/ifcfg-ens33 TYPE=Ethernet PROXY_METHOD=none BROWSER_ONLY=no BOOTPROTO=static DEFROUTE=yes IPV4_FAILURE_FATAL=no IPV6INIT=yes IPV6_AUTOCONF=yes IPV6_DEFROUTE=yes IPV6_FAILURE_FATAL=no IPV6_ADDR_GEN_MODE=stable-privacy NAME=ens33 UUID=a41355ae-f475-39d7-9e61-eb5f8f19f881 DEVICE=ens33 ONBOOT=yes IPADDR=192.168.93.30 PREFIX=24 GATEWAY=192.168.93.2 DNS1=114.114.114.114 DNS2=8.8.8.8 IPV6_PRIVACY=no [root@kolla ~]# cat /etc/sysconfig/network-scripts/ifcfg-ens37 TYPE=Ethernet PROXY_METHOD=none BROWSER_ONLY=no BOOTPROTO=none DEFROUTE=yes IPV4_FAILURE_FATAL=no NAME=ens37 UUID=553a2dd0-b53e-417e-98a9-9a7a6a44a53c DEVICE=ens37 ONBOOT=yes
如果啟用cinder還需要額外添加磁盤，這里添加一塊sdb盤並創建為pv和vg，使用lvm作為cinder的后端存儲：
pvcreate /dev/sdb vgcreate cinder-volumes /dev/sdb
注意卷組名稱為cinder-volumes，默認與后面的globals.yml中定義一致。
[root@kolla ~]# cat /etc/kolla/globals.yml | grep cinder_volume_group #cinder_volume_group: "cinder-volumes"
部署kolla ansible
配置主機名,kolla預檢查時rabbitmq可能需要能夠進行主機名解析
hostnamectl set-hostname kolla
安裝依賴
yum install -y python-devel libffi-devel gcc openssl-devel libselinux-python
安裝 Ansible，注意版本，默認2.9應該可以滿足要求
yum install -y ansible
配置阿里雲pip源，否則pip安裝時會很慢
mkdir ~/.pip cat > ~/.pip/pip.conf << EOF [global] trusted-host=mirrors.aliyun.com index-url=https://mirrors.aliyun.com/pypi/simple/ EOF
安裝 kolla-ansible
kolla版本與openstack版本對應關系：https://releases.openstack.org/teams/kolla.html
yum install -y epel-release yum install -y python-pip pip install -U pip pip install kolla-ansible==9.1.0 --ignore-installed PyYAML
復制 kolla-ansible配置文件到當前環境
mkdir -p /etc/kolla chown $USER:$USER /etc/kolla ##Copy globals.yml and passwords.yml cp -r /usr/share/kolla-ansible/etc_examples/kolla/* /etc/kolla ##Copy all-in-one and multinode inventory files cp /usr/share/kolla-ansible/ansible/inventory/* .
修改ansible配置文件
$ vim /etc/ansible/ansible.cfg [defaults] host_key_checking=False pipelining=True forks=100
默認有all-in-one和multinode兩個inventory文件，這里使用all-in-one，來規划集群角色，可以看到所有節點都是同一個節點kolla。
# sed -i 's#localhost ansible_connection=local#kolla#g' all-in-one #查看修改后的配置，其他默認即可 # cat all-in-one | more [control] kolla [network] kolla [compute] kolla [storage] kolla [monitoring] kolla [deployment] kolla ...
配置主機名解析，實際在環境預配置時kolla會自動添加解析到/etc/hosts
cat >> /etc/hosts <<EOF 192.168.93.30 kolla EOF
配置ssh免密
ssh-keygen ssh-copy-id root@kolla
檢查inventory配置是否正確，執行：
ansible -i all-in-one all -m ping
生成openstack組件用到的密碼，該操作會填充/etc/kolla/passwords.yml，該文件中默認參數為空。
kolla-genpwd
如果報錯執行
pip uninstall cryptography
pip install "cryptography==2.7"
修改keystone_admin_password，可以修改為自定義的密碼方便后續horizon登錄，這里改為kolla。
$ sed -i 's#keystone_admin_password:.*#keystone_admin_password: kolla#g' /etc/kolla/passwords.yml $ cat /etc/kolla/passwords.yml | grep keystone_admin_password keystone_admin_password: kolla
修改全局配置文件globals.yml，該文件用來控制安裝哪些組件，以及如何配置組件，由於全部是注釋，這里直接追加進去，也可以逐個找到對應項進行修改。
cp /etc/kolla/globals.yml{,.bak}
cat >> /etc/kolla/globals.yml <<EOF
#version
kolla_base_distro: "centos"
kolla_install_type: "binary"
openstack_release: "train"
#vip
kolla_internal_vip_address: "192.168.93.100"
#docker registry
docker_registry: "registry.cn-shenzhen.aliyuncs.com"
docker_namespace: "kollaimage"
#network
network_interface: "ens33"
neutron_external_interface: "ens37"
neutron_plugin_agent: "openvswitch"
enable_neutron_provider_networks: "yes"
#storage
enable_cinder: "yes"
enable_cinder_backend_lvm: "yes"
#virt_type
nova_compute_virt_type: "qemu"
EOF
參數說明：
kolla_base_distro: kolla鏡像基於不同linux發型版構建，主機使用centos這里對應使用centos類型的docker鏡像即可。
kolla_install_type: kolla鏡像基於binary二進制和source源碼兩種類型構建，實際部署使用binary即可。
openstack_release: openstack版本可自定義，會從dockerhub拉取對應版本的鏡像
kolla_internal_vip_address: 單節點部署kolla也會啟用haproxy和keepalived，方便后續擴容為高可用集群，該地址是ens33網卡網絡中的一個可用IP。
docker_registry: 默認從dockerhub拉取鏡像，這里使用阿里雲鏡像倉庫，也可以本地搭建倉庫，提前推送鏡像上去。但該倉庫目前只有train和ussuri版本的鏡像，如何自己推送鏡像參考該博客的其他文章。
docker_namespace: 阿里雲kolla鏡像倉庫所在的命名空間，dockerhub官網默認是kolla。
network_interface: 管理網絡的網卡
neutron_external_interface: 外部網絡的網卡
neutron_plugin_agent: 默認啟用openvswitch
enable_neutron_provider_networks: 啟用外部網絡
enable_cinder: 啟用cinder
enable_cinder_backend_lvm: 指定cinder后端存儲為lvm
nova_compute_virt_type: 由於使用vmware安裝，要改為qemu，否則創建虛擬機失敗，生產部署默認使用kvm。
上面部分參數可能有默認配置，也可以不用明確開啟，比如neutron_plugin_agent。
有些參數也可以在部署后配置，比如nova_compute_virt_type，找到配置文件修改，並重啟對應組件容器即可：
[root@kolla ~]# cat /etc/kolla/nova-compute/nova.conf |grep virt_type #virt_type = kvm virt_type = qemu [root@kolla ~]# docker restart nova_compute
修改docker官方yum源為阿里雲yum源，另外配置docker鏡像加速，指定使用阿里雲鏡像加速。
$ vim /usr/share/kolla-ansible/ansible/roles/baremetal/defaults/main.yaml
docker_yum_url: "https://mirrors.aliyun.com/docker-ce/linux/{{ ansible_distribution | lower }}"
docker_custom_config: {"registry-mirrors": ["https://uyah70su.mirror.aliyuncs.com"]}
部署openstack組件
部署openstack
#預配置，安裝docker、docker sdk、關閉防火牆、配置時間同步等 kolla-ansible -i ./all-in-one bootstrap-servers #部署前環境檢查 kolla-ansible -i ./all-in-one prechecks #拉取鏡像，也可省略該步驟，默認會自動拉取 kolla-ansible -i ./all-in-one pull #執行實際部署，拉取鏡像，運行對應組件容器 kolla-ansible -i ./all-in-one deploy
報錯
1、no test named 'equalto'
pip install -U Jinja2
pip install Jinja2===2.10.2 （成功）
2、另外關閉systemctl stop NetworkManager
3、計算節點安裝neutron組件時驗證sysctl值的時候報錯
sysctl net.bridge.bridge-nf-call-ip6tables sysctl net.bridge.bridge-nf-call-iptables 報錯sysctl: cannot stat /proc/sys/net/bridge/bridge-nf-call-iptables: No such file or directory 用以下命令解決 [root@computer ~]# modprobe br_netfilter [root@computer ~]# ls /proc/sys/net/bridge/ bridge-nf-call-arptables bridge-nf-call-iptables bridge-nf-filter-vlan-tagged bridge-nf-call-ip6tables bridge-nf-filter-pppoe-tagged bridge-nf-pass-vlan-input-dev [root@computer ~]# sysctl -p [root@computer ~]# sysctl net.bridge.bridge-nf-call-ip6tables net.bridge.bridge-nf-call-ip6tables = 1 [root@computer ~]# sysctl net.bridge.bridge-nf-call-iptables net.bridge.bridge-nf-call-iptables = 1
以上部署沒有報錯中斷說明部署成功，所有openstack組件以容器方式運行，查看容器
[root@kolla ~]# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 325c17a52c79 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-chrony:train "dumb-init --single-…" 36 hours ago Up 25 hours chrony 6218d98755ee registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-cron:train "dumb-init --single-…" 36 hours ago Up 25 hours cron 02b6598c1089 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-kolla-toolbox:train "dumb-init --single-…" 36 hours ago Up 25 hours kolla_toolbox 8572e445abad registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-fluentd:train "dumb-init --single-…" 36 hours ago Up 25 hours fluentd f11a103c5ade registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-openstack-base:train "dumb-init --single-…" 44 hours ago Up 25 hours client 5c91def3c963 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-horizon:train "dumb-init --single-…" 44 hours ago Up 25 hours horizon e024bd4f5dd3 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-heat-engine:train "dumb-init --single-…" 44 hours ago Up 25 hours heat_engine 2d1491bd9e1a registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-heat-api-cfn:train "dumb-init --single-…" 44 hours ago Up 25 hours heat_api_cfn eeefcfb31a61 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-heat-api:train "dumb-init --single-…" 44 hours ago Up 25 hours heat_api 9b51b53448fc registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-neutron-metadata-agent:train "dumb-init --single-…" 44 hours ago Up 25 hours neutron_metadata_agent 9f88a6c0cf31 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-neutron-l3-agent:train "dumb-init --single-…" 44 hours ago Up 25 hours neutron_l3_agent a419cb3270a6 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-neutron-dhcp-agent:train "dumb-init --single-…" 44 hours ago Up 25 hours neutron_dhcp_agent 959f6faba972 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-neutron-openvswitch-agent:train "dumb-init --single-…" 44 hours ago Up 25 hours neutron_openvswitch_agent cc1b081cf876 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-neutron-server:train "dumb-init --single-…" 44 hours ago Up 25 hours neutron_server eea1a87feb43 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-openvswitch-vswitchd:train "dumb-init --single-…" 44 hours ago Up 25 hours openvswitch_vswitchd 376f81bf75a2 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-openvswitch-db-server:train "dumb-init --single-…" 44 hours ago Up 25 hours openvswitch_db c68fd9a92d73 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-nova-compute:train "dumb-init --single-…" 44 hours ago Up 25 hours nova_compute 2492e2a32c80 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-nova-libvirt:train "dumb-init --single-…" 44 hours ago Up 25 hours nova_libvirt 3802d199b29f registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-nova-ssh:train "dumb-init --single-…" 44 hours ago Up 25 hours nova_ssh 1281c311ecd4 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-nova-novncproxy:train "dumb-init --single-…" 44 hours ago Up 25 hours nova_novncproxy 2e8c8478116b registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-nova-conductor:train "dumb-init --single-…" 44 hours ago Up 25 hours nova_conductor 950feb59b549 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-nova-api:train "dumb-init --single-…" 44 hours ago Up 25 hours nova_api 49497e664922 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-nova-scheduler:train "dumb-init --single-…" 44 hours ago Up 25 hours nova_scheduler f5eb37b48f7d registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-placement-api:train "dumb-init --single-…" 44 hours ago Up 25 hours placement_api 54cd0e3be101 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-cinder-backup:train "dumb-init --single-…" 44 hours ago Up 25 hours cinder_backup b4efa4449e7f registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-cinder-volume:train "dumb-init --single-…" 44 hours ago Up 25 hours cinder_volume 159b669d2fd3 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-cinder-scheduler:train "dumb-init --single-…" 44 hours ago Up 25 hours cinder_scheduler 9fc7e6a4cb25 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-cinder-api:train "dumb-init --single-…" 44 hours ago Up 25 hours cinder_api b3f8f711f2b1 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-glance-api:train "dumb-init --single-…" 44 hours ago Up 25 hours glance_api 760e92d698e2 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-keystone-fernet:train "dumb-init --single-…" 44 hours ago Up 25 hours keystone_fernet 95f235c4ac10 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-keystone-ssh:train "dumb-init --single-…" 44 hours ago Up 25 hours keystone_ssh 03306334ce19 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-keystone:train "dumb-init --single-…" 44 hours ago Up 25 hours keystone 5173d4191567 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-rabbitmq:train "dumb-init --single-…" 44 hours ago Up 25 hours rabbitmq eb6bca26f6ce registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-tgtd:train "dumb-init --single-…" 44 hours ago Up 25 hours tgtd 79fac2ca1b19 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-iscsid:train "dumb-init --single-…" 44 hours ago Up 25 hours iscsid 4a3fcefc7009 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-memcached:train "dumb-init --single-…" 44 hours ago Up 25 hours memcached 0773eaf446e4 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-mariadb:train "dumb-init -- kolla_…" 44 hours ago Up 25 hours mariadb 77f0beaa28e5 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-keepalived:train "dumb-init --single-…" 44 hours ago Up 25 hours keepalived b02b744d2da3 registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-haproxy:train "dumb-init --single-…" 44 hours ago Up 25 hours haproxy
確認沒有Exited等異常狀態的容器
[root@kolla ~]# docker ps -a | grep -v Up
本次部署運行了39個容器
[root@localhost kolla-env]# docker ps -a | wc -l 39
查看拉取的鏡像，發現鏡像數量與容器數量是一致的。
[root@kolla ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-glance-api train aec757c5908a 2 days ago 1.05GB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-keystone-ssh train 2c95619322ed 2 days ago 1.04GB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-keystone-fernet train 918564aa9c01 2 days ago 1.04GB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-keystone train 8d5f3ca2a73c 2 days ago 1.04GB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-cinder-api train 500910236e85 2 days ago 1.19GB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-cinder-volume train f76ebe1e133d 2 days ago 1.14GB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-cinder-backup train 19342786a92c 2 days ago 1.13GB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-cinder-scheduler train 920630f0ea6c 2 days ago 1.11GB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-heat-api train 517f6a0643ee 2 days ago 1.07GB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-heat-api-cfn train 2d46b91d44ef 2 days ago 1.07GB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-heat-engine train ab570c135dbc 2 days ago 1.07GB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-horizon train a00ddb359ea5 2 days ago 1.2GB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-fluentd train 6a5b7be2551b 2 days ago 697MB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-cron train 0f784cd532e2 2 days ago 408MB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-chrony train 374dabc62868 2 days ago 408MB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-iscsid train 575873f9e4b8 2 days ago 413MB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-haproxy train 9cf840548535 2 days ago 433MB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-keepalived train b2a20ccd7d6a 2 days ago 414MB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-openstack-base train c35001fb182b 3 days ago 920MB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-nova-compute train 93be43a73a3e 5 days ago 1.85GB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-placement-api train 26f8c88c3c50 5 days ago 1.05GB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-nova-api train 2a9d3ea95254 5 days ago 1.08GB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-nova-novncproxy train e6acfbe47b2b 5 days ago 1.05GB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-nova-conductor train 836a9f775263 5 days ago 1.05GB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-nova-ssh train f89a813f3902 5 days ago 1.05GB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-nova-scheduler train 8061eaa33d21 5 days ago 1.05GB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-openvswitch-vswitchd train 2b780c8075c6 5 days ago 425MB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-openvswitch-db-server train 86168147b086 5 days ago 425MB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-rabbitmq train 19cd34b4f503 5 days ago 487MB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-mariadb train 882472a192b5 6 days ago 593MB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-neutron-dhcp-agent train a007b53f0507 7 days ago 1.04GB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-neutron-metadata-agent train 8bcff22221bd 7 days ago 1.04GB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-nova-libvirt train 539673da5c25 7 days ago 1.25GB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-kolla-toolbox train a18a474c65ea 7 days ago 842MB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-tgtd train ad5380187ca9 7 days ago 383MB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-memcached train 1fcf18645254 7 days ago 408MB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-neutron-server train 539cfb7c1fd2 8 days ago 1.08GB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-neutron-openvswitch-agent train 95113c0f5b8c 8 days ago 1.08GB registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-neutron-l3-agent train fbe9385f49ca 8 days ago 1.08GB
查看cinder使用的卷，自動創建了lvm
[root@kolla ~]# lsblk | grep cinder
├─cinder--volumes-cinder--volumes--pool_tmeta 253:3 0 20M 0 lvm
│ └─cinder--volumes-cinder--volumes--pool 253:5 0 19G 0 lvm
└─cinder--volumes-cinder--volumes--pool_tdata 253:4 0 19G 0 lvm
└─cinder--volumes-cinder--volumes--pool 253:5 0 19G 0 lvm
[root@kolla ~]# lvs | grep cinder
cinder-volumes-pool cinder-volumes twi-a-tz-- 19.00g 0.00 10.55
查看網卡狀態
[root@kolla ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:0c:4e:fe brd ff:ff:ff:ff:ff:ff
inet 192.168.93.30/24 brd 192.168.93.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet 192.168.93.100/32 scope global ens33
valid_lft forever preferred_lft forever
inet6 fe80::7a6c:d06c:ee49:4cd5/64 scope link noprefixroute
valid_lft forever preferred_lft forever
3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master ovs-system state UP group default qlen 1000
link/ether 00:0c:29:0c:4e:08 brd ff:ff:ff:ff:ff:ff
inet6 fe80::20c:29ff:fe0c:4e08/64 scope link
valid_lft forever preferred_lft forever
4: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 02:42:2a:d9:93:52 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
inet6 fe80::42:2aff:fed9:9352/64 scope link
valid_lft forever preferred_lft forever
6: veth0c46c6a@if5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default
link/ether 1a:ce:d7:61:d0:cc brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet6 fe80::18ce:d7ff:fe61:d0cc/64 scope link
valid_lft forever preferred_lft forever
7: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether de:e5:b7:4d:e8:b8 brd ff:ff:ff:ff:ff:ff
11: br-int: <BROADCAST,MULTICAST> mtu 1450 qdisc noop state DOWN group default qlen 1000
link/ether 52:14:05:ba:ce:4c brd ff:ff:ff:ff:ff:ff
13: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether d2:5b:76:f5:01:49 brd ff:ff:ff:ff:ff:ff
14: br-ex: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:0c:29:0c:4e:08 brd ff:ff:ff:ff:ff:ff
22: qbr2749f64b-1f: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default qlen 1000
link/ether 3a:0d:ad:56:9d:9d brd ff:ff:ff:ff:ff:ff
23: qvo2749f64b-1f@qvb2749f64b-1f: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master ovs-system state UP group default qlen 1000
link/ether c2:c5:8b:a6:72:8b brd ff:ff:ff:ff:ff:ff
inet6 fe80::c0c5:8bff:fea6:728b/64 scope link
valid_lft forever preferred_lft forever
24: qvb2749f64b-1f@qvo2749f64b-1f: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master qbr2749f64b-1f state UP group default qlen 1000
link/ether 3a:0d:ad:56:9d:9d brd ff:ff:ff:ff:ff:ff
inet6 fe80::380d:adff:fe56:9d9d/64 scope link
valid_lft forever preferred_lft forever
25: tap2749f64b-1f: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast master qbr2749f64b-1f state UNKNOWN group default qlen 1000
link/ether fe:16:3e:94:b5:71 brd ff:ff:ff:ff:ff:ff
inet6 fe80::fc16:3eff:fe94:b571/64 scope link
valid_lft forever preferred_lft forever
26: qbr0a14e63d-2e: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default qlen 1000
link/ether 02:f9:32:c0:f4:b7 brd ff:ff:ff:ff:ff:ff
27: qvo0a14e63d-2e@qvb0a14e63d-2e: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master ovs-system state UP group default qlen 1000
link/ether 76:86:46:4c:4f:61 brd ff:ff:ff:ff:ff:ff
inet6 fe80::7486:46ff:fe4c:4f61/64 scope link
valid_lft forever preferred_lft forever
28: qvb0a14e63d-2e@qvo0a14e63d-2e: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master qbr0a14e63d-2e state UP group default qlen 1000
link/ether 02:f9:32:c0:f4:b7 brd ff:ff:ff:ff:ff:ff
inet6 fe80::f9:32ff:fec0:f4b7/64 scope link
valid_lft forever preferred_lft forever
29: tap0a14e63d-2e: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast master qbr0a14e63d-2e state UNKNOWN group default qlen 1000
link/ether fe:16:3e:ee:08:6b brd ff:ff:ff:ff:ff:ff
inet6 fe80::fc16:3eff:feee:86b/64 scope link
valid_lft forever preferred_lft forever
另外需要注意，不要在該節點安裝libvirt等工具，這些工具安裝后可能會啟用libvirtd和iscsid.sock等服務，kolla已經在容器中運行了這些服務，這些服務會調用節點上的sock文件，如果節點上也啟用這些服務去搶占這些文件，會導致容器異常。默認kolla在預配置時也會主動禁用節點上的相關服務。
安裝OpenStack客戶端
執行openstack相關命令和操作，需要本地安裝openstack客戶端，但是本次在kolla節點安裝客戶端報錯，所以直接啟動一個官方的base容器，該容器默認帶有客戶端命令，使用時將admin-openrc.sh掛載進容器即可。
安裝OpenStack CLI客戶端(可能報錯，略過該步驟)
pip install python-openstackclient
kolla-ansible post-deploy cat /etc/kolla/admin-openrc.sh
kolla ansible提供了一個快速創建cirros demo實例的腳本(可能報錯，略過該步驟)
source /etc/kolla/admin-openrc.sh /usr/share/kolla-ansible/init-runonce
訪問openstack horizon
訪問openstack horizon需要使用vip地址，節點上可以看到由keepalived容器生成的vip
[root@kolla ~]# ip a |grep ens33
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
inet 192.168.93.30/24 brd 192.168.93.255 scope global ens33
inet 192.168.93.100/32 scope global ens33
瀏覽器直接訪問該地址即可登錄到horizon
我這里的用戶名密碼為admin/kolla，信息可以從admin-openrc.sh中獲取
[root@kolla ~]# cat /etc/kolla/admin-openrc.sh
# Clear any old environment that may conflict.
for key in $( set | awk '{FS="="} /^OS_/ {print $1}' ); do unset $key ; done
export OS_PROJECT_DOMAIN_NAME=Default
export OS_USER_DOMAIN_NAME=Default
export OS_PROJECT_NAME=admin
export OS_TENANT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=kolla
export OS_AUTH_URL=http://192.168.93.100:35357/v3
export OS_INTERFACE=internal
export OS_ENDPOINT_TYPE=internalURL
export OS_IDENTITY_API_VERSION=3
export OS_REGION_NAME=RegionOne
export OS_AUTH_PLUGIN=password
容器運行openstack客戶端
由於在kolla主機節點沒有成功安裝openstack客戶端，這里在容器里使用客戶端
拉取官方鏡像地址是kolla/centos-binary-openstack-base:train，這里從阿里雲拉取
docker pull registry.cn-shenzhen.aliyuncs.com/kollaimage/centos-binary-openstack-base:train
進入容器就可以正常執行openstack相關命令了
[root@kolla ~]# docker exec -it client bash ()[root@f11a103c5ade /]# source /admin-openrc.sh ()[root@f11a103c5ade /]# openstack service list +----------------------------------+-------------+----------------+ | ID | Name | Type | +----------------------------------+-------------+----------------+ | 2aed09dc3dbd450599042edd9badcc17 | nova_legacy | compute_legacy | | 2c26e8f09c20455bb67e1df58e7f5ab5 | nova | compute | | 2ec7dd7cd3ce4298931e7272a6e0abd4 | glance | image | | 47062da43fd644eabaa21ae3ec3189da | keystone | identity | | 567057b208ae4a3bb2e3e8e3e7b80bd8 | neutron | network | | 63418bb02ffd449f940c886e640162a1 | heat | orchestration | | 652da566d85c47eb8d38465fe54c232e | cinderv2 | volumev2 | | 9c8acd17ecbf457fb8b4f29cfc7859da | heat-cfn | cloudformation | | d1ef13894f2e44688a1bb117e64d8715 | placement | placement | | d35e629c03794b4c87c6dc2670f3f00a | cinderv3 | volumev3 | +----------------------------------+-------------+----------------+
示例demo腳本也被掛載到了容器，修改init-runonce示例腳本外部網絡部分的配置，然后執行該shell腳本
()[root@f11a103c5ade /]# cat init-runonce
# This EXT_NET_CIDR is your public network,that you want to connect to the internet via.
ENABLE_EXT_NET=${ENABLE_EXT_NET:-1}
EXT_NET_CIDR=${EXT_NET_CIDR:-'192.168.1.0/24'}
EXT_NET_RANGE=${EXT_NET_RANGE:-'start=192.168.1.200,end=192.168.1.250'}
EXT_NET_GATEWAY=${EXT_NET_GATEWAY:-'192.168.1.1'}
$ bash init-runonce
參數說明：
EXT_NET_CIDR 指定外部網絡，由於使用橋接模式，直接橋接到了電腦的無線網卡，所以這里網絡就是無線網卡的網段。
EXT_NET_RANGE 指定從外部網絡取出一個地址范圍，作為外部網絡的地址池
EXT_NET_GATEWAY 外部網絡網關，這里與wifi網絡使用的網關一致
腳本會創建一些資源，如下載cirros鏡像並上傳，創建外部和內部網絡等，另外腳本執行過程中會創建ssh key，直接回車即可，默認保存在容器中的/root/.ssh目錄下，其中的id_rsa私鑰可以用來遠程連接實例使用。
該腳本首先會從github下載cirros鏡像，如果網絡較慢可以提前下載到cache目錄
docker exec -it client mkdir -p /opt/cache/files/ wget https://github.com/cirros-dev/cirros/releases/download/0.4.0/cirros-0.4.0-x86_64-disk.img docker cp cirros-0.4.0-x86_64-disk.img client:/opt/cache/files/
最后根據提示手動運行一個實例
openstack server create \
--image cirros \
--flavor m1.tiny \
--key-name mykey \
--network demo-net \
demo1
在horizion查看創建的網絡和實例
登錄實例控制台，驗證實例與外網的連通性，cirros用戶密碼在初次登錄時有提示：
在kolla節點上ssh連接實例浮動IP，cirros鏡像默認用戶密碼為cirros/gocubsgo，該鏡像信息官網有介紹：
https://docs.openstack.org/image-guide/obtain-images.html#cirros-test
[root@kolla ~]# ssh cirros@192.168.1.248 cirros@192.168.1.248's password: $ $
運行CentOS實例
centos官方維護有相關cloud image，如果不需要進行定制，可以直接下載來運行實例。
參考：https://docs.openstack.org/image-guide/obtain-images.html
CentOS官方維護的鏡像下載地址：
http://cloud.centos.org/centos/7/images/
也可以使用命令直接下載鏡像，但是下載可能較慢，建議下載好在進行上傳。以centos7.8為例：
wget http://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud-2003.qcow2c
下載完成后上傳鏡像到openstack，直接在horizon上傳即可。也可以使用命令上傳。
注意：默認該鏡像運行的實例只能使用ssh key以centos用戶身份登錄，如果需要使用root遠程ssh連接到實例需要在上傳前為鏡像配置root免密並開啟ssh訪問。
參考：https://blog.csdn.net/networken/article/details/106713658
另外我們的命令客戶端在容器中，所有這里有些不方便，首先要將鏡像復制到容器中，然后使用openstack命令上傳。
這里復制到client容器的根目錄下。
[root@kolla ~]# docker cp CentOS-7-x86_64-GenericCloud-2003.qcow2c client:/ [root@kolla ~]# docker exec -it client bash ()[root@f11a103c5ade /]# ()[root@f11a103c5ade /]# source /admin-openrc.sh ()[root@f11a103c5ade /]# ls | grep CentOS CentOS-7-x86_64-GenericCloud-2003.qcow2c
執行以下openstack命令上傳鏡像
openstack image create "CentOS78-image" \ --file CentOS-7-x86_64-GenericCloud-2003.qcow2c \ --disk-format qcow2 --container-format bare \ --public
如果實例創建失敗可以查看相關組件報錯日志
[root@kolla ~]# tail -100f /var/log/kolla/nova/nova-compute.log
未配置root密碼連接實例
如果沒有提前定制鏡像修改root密碼，只能使用centos用戶及sshkey登錄，由於是在容器中運行的demo示例，ssh私鑰也保存在容器的默認目錄下，在容器中連接實例浮動IP測試
[root@kolla ~]# docker exec -it client bash ()[root@f11a103c5ade /]# ssh -i /root/.ssh/id_rsa centos@192.168.1.105 Last login: Sat Jun 13 05:47:49 2020 from 192.168.1.100 [centos@centos78 ~]$ [centos@centos78 ~]$
運行Ubuntu實例
下載鏡像
wget https://cloud-images.ubuntu.com/bionic/current/bionic-server-cloudimg-amd64.img
上傳鏡像
openstack image create "Ubuntu1804" \ --file bionic-server-cloudimg-amd64.img \ --disk-format qcow2 --container-format bare \ --public
按照正常流程創建實例即可，ubuntu鏡像默認用戶為ubuntu，首次登陸使用sshkey方式，然后執行以下命令即可直接切換到root用戶（centos無法使用該方式）
$ sudo -i
kolla配置和日志文件
- 各個組件配置文件目錄： /etc/kolla/
- 各個組件日志文件目錄：/var/log/kolla/
清理kolla ansilbe集群
kolla-ansible destroy --include-images --yes-i-really-really-mean-it #重置cinder卷，謹慎操作 vgremove cinder-volume
配置docker阿里雲加速
[root@openstack ~]# cat /etc/docker/daemon.json
{
"registry-mirrors": ["https://k9e55i4n.mirror.aliyuncs.com"]
}
# systemctl daemon-reload
# systemctl restart docker
查看ip是不是阿里的
