OpenStack在線調整虛擬機的大小報錯
報錯日志
2021-06-02 10:15:19.142 844211 INFO nova.compute.manager [req-13a6e58c-9826-49fc-81cb-0221a4febe18 083a5c802fc748658db1821f3f778ddf dd3bc80d13fb431bacc466cbaaf7dbd7 - - -] [instance: ceed63dc-8cdd-43f5-95d8-3f7de86051e9] Setting instance back to ACTIVE after: Instance rollback performed due to: Resize error: not able to execute ssh command: Unexpected error while running command.
Command: ssh -o BatchMode=yes 10.89.30.32 mkdir -p /var/lib/nova/instances/ceed63dc-8cdd-43f5-95d8-3f7de86051e9
Exit code: 255
Stdout: u''
Stderr: u'Host key verification failed.\r\n'
2021-06-02 10:15:19.551 844211 INFO nova.compute.manager [req-13a6e58c-9826-49fc-81cb-0221a4febe18 083a5c802fc748658db1821f3f778ddf dd3bc80d13fb431bacc466cbaaf7dbd7 - - -] [instance: ceed63dc-8cdd-43f5-95d8-3f7de86051e9] Successfully reverted task state from None on failure for instance.
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher [req-13a6e58c-9826-49fc-81cb-0221a4febe18 083a5c802fc748658db1821f3f778ddf dd3bc80d13fb431bacc466cbaaf7dbd7 - - -] Exception during message handling: Resize error: not able to execute ssh command: Unexpected error while running command.
Command: ssh -o BatchMode=yes 10.89.30.32 mkdir -p /var/lib/nova/instances/ceed63dc-8cdd-43f5-95d8-3f7de86051e9
Exit code: 255
Stdout: u''
Stderr: u'Host key verification failed.\r\n'
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher incoming.message))
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 185, in _dispatch
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args)
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/exception.py", line 110, in wrapped
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher payload)
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher self.force_reraise()
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/exception.py", line 89, in wrapped
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher return f(self, context, *args, **kw)
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 359, in decorated_function
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher LOG.warning(msg, e, instance=instance)
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher self.force_reraise()
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 328, in decorated_function
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 409, in decorated_function
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 316, in decorated_function
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher migration.instance_uuid, exc_info=True)
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher self.force_reraise()
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 293, in decorated_function
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 387, in decorated_function
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher kwargs['instance'], e, sys.exc_info())
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher self.force_reraise()
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 375, in decorated_function
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3933, in resize_instance
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher self.instance_events.clear_events_for_instance(instance)
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib64/python2.7/contextlib.py", line 35, in __exit__
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher self.gen.throw(type, value, traceback)
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 6643, in _error_out_instance_on_exception
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher raise error.inner_exception
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher ResizeError: Resize error: not able to execute ssh command: Unexpected error while running command.
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher Command: ssh -o BatchMode=yes 10.89.30.32 mkdir -p /var/lib/nova/instances/ceed63dc-8cdd-43f5-95d8-3f7de86051e9
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher Exit code: 255
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher Stdout: u''
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher Stderr: u'Host key verification failed.\r\n'
2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher
2021-06-02 10:15:46.889 844211 INFO nova.compute.resource_tracker [req-2adbb715-5cef-4e83-ac9f-41a39c554fa9 - - - - -] Auditing locally available compute resources for node st-computer12
在日志中錯誤提示還是挺明顯的,根據錯誤提示,解決相應的故障就行了。
注意一點:
openstack的虛擬機在線調整大小的原理:
其實就相當於做了一個雲主機在不同宿主機(計算節點)之間的遷移,所以前提是至少需要有兩個計算節點。
如果是單機部署的openstack(即控制節點和計算節點都在一台機器上),有且只有一個計算節點,那么是無法完成在線調整虛擬機大小的。
同時要注意的是:
要在相關遷移雲主機間進行無密碼訪問,由於OpenStack是由Nova組件來管理雲主機,所以需要對Nova用戶進行無密碼訪問。
如上圖中的centos-004這台雲主機的規格采用的是名為kvm001類型,即2核CPU、4G內容,現在需要將這台雲主機的規格降為1核2G,即kvm002類型。操作流程如下:
1)修改控制節點和節點節點的nova.conf文件,添加下面兩行內容:
# 可以參考最原始的nova.conf
[defaults]
allow_resize_to_same_host=True
scheduler_default_filters=RetryFilter,AvailabilityZoneFilter,RamFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter
2)控制節點上重啟nova相關服務
[root@linux-node1 src]# systemctl restart openstack-nova-api.service openstack-nova-cert.service openstack-nova-consoleauth.service openstack-nova-scheduler.service openstack-nova-conductor.service openstack-nova-novncproxy.service
3)計算節點上重啟nova服務
[root@linux-node1 nova]# systemctl restart openstack-nova-compute
4)nova賬號的ssh雙向認證關系
下面的操作在雲主機所在計算節點和其他相關遷移雲主機的計算節點上操作。
要清楚的是:
計算節點可能有多台,但是我們只需要在要調整大小的雲主機所在的計算節點和其他的另外一台或多台計算幾點上操作就行,並不是要求所有的計算節點都要操作(全部計算節點都操作也是可以的)
將nova賬號的shell類型改為/bin/bash,即可以使用nova賬號登陸
[root@linux-node1 ~]# cat /etc/passwd|grep nova
nova:x:162:162:OpenStack Nova Daemons:/var/lib/nova:/bin/bash
然后在nova賬號下產生公私鑰
[root@linux-node2 ~]# su - nova
Last login: Thu Nov 17 17:32:54 CST 2016 from linux-node2.openstack on pts/11
-bash-4.2$ ssh-keygen -t rsa
-bash-4.2$ pwd
/var/lib/nova
-bash-4.2$ cd .ssh/
-bash-4.2$ ls
id_rsa id_rsa.pub
將id_rsa.pub拷貝為authorized_keys
-bash-4.2$ cp id_rsa.pub authorized_keys
```
然后將id_rsa.pub公鑰內容拷貝到其他相關遷移雲主機的計算節點上的/var/lib/nova/.ssh/authorized_keys文件內,同時也要講對方的公鑰內容拷貝過來,做成雙向信任關系(即雙方在nova賬號下ssh登陸時都不需要輸入密碼)
同時,還要在各計算節點上做nova和root賬號的雙向信任關系。
修改權限
```bash
[root@linux-node1 ~]# cd /var/lib/nova/.ssh/
[root@linux-node1 .ssh]# ll
total 16
-rw-------. 1 nova nova 816 Nov 17 17:32 authorized_keys
-rw-------. 1 nova nova 1679 Nov 17 17:12 id_rsa
-rw-r--r--. 1 nova nova 408 Nov 17 17:12 id_rsa.pub
修改sudo權限
[root@linux-node1 ~]# visudo
.....
nova ALL=(ALL) NOPASSWD: ALL
最后測試nova賬號間的信任連接,確認ssh無密碼連接!
5)接下來登陸dashboard界面,在線調整雲主機的大小:
選擇目標flavor
點擊確認修改尺寸
待調整后,再查詢虛擬機新的大小規格,就會發現已經更新了。