塊存儲服務部署相關
塊存儲服務(cinder)為實例提供塊存儲。存儲的分配和消耗是由塊存儲驅動器,或者多后端配置的驅動器決定的。還有很多驅動程序可用:NAS/SAN,NFS,ISCSI,Ceph等。
典型情況下,塊服務API和調度器服務運行在控制節點上。取決於使用的驅動,卷服務器可以運行在控制節點、計算節點或單獨的存儲節點。
1.塊存儲服務概覽
OpenStack塊存儲服務(cinder)為虛擬機添加持久的存儲,塊存儲提供一個基礎設施為了管理卷,以及和OpenStack計算服務交互,為實例提供卷。此服務也會激活管理卷的快照和卷類型的功能。
塊存儲服務通常包含下列組件:
cinder-api
接受API請求,並將其路由到``cinder-volume``執行。
cinder-volume
與塊存儲服務和例如``cinder-scheduler``的進程進行直接交互。它也可以與這些進程通過一個消息隊列進行交互。``cinder-volume``服務響應送到塊存儲服務的讀寫請求來維持狀態。它也可以和多種存儲提供者在驅動架構下進行交互。
cinder-scheduler守護進程
選擇最優存儲提供節點來創建卷。其與``nova-scheduler``組件類似。
cinder-backup daemon
``cinder-backup``服務提供任何種類備份卷到一個備份存儲提供者。就像``cinder-volume``服務,它與多種存儲提供者在驅動架構下進行交互。
消息隊列
在塊存儲的進程之間路由信息。
節點配置信息說明:
控制節點:controller:
IP:192.168.164.128
hostname&hosts:likeadmin
計算加點:Nova:
IP:192.168.164.129
hostname&hosts:likenode
塊存儲節點:cinder:
IP:192.168.164.136
hostname&hosts:likeblock
1.安裝並配置控制節點
這個部分描述如何在控制節點上安裝和配置塊設備存儲服務,即 cinder。這個服務需要至少一個額外的存儲節點,以向實例提供卷。
先決條件
在你安裝和配置塊存儲服務之前,你必須創建數據庫、服務證書和API端點。
1)完成下面的步驟以創建數據庫:
用數據庫連接客戶端以 root 用戶連接到數據庫服務器:
# mysql -u root -p
創建 cinder 數據庫:
MariaDB [(none)]> CREATE DATABASE cinder;
允許 cinder 數據庫合適的訪問權限:
MariaDB [(none)]> GRANT ALL PRIVILEGES ON cinder.* TO 'cinder'@'localhost' \
IDENTIFIED BY 'djl18001';
MariaDB [(none)]> GRANT ALL PRIVILEGES ON cinder.* TO 'cinder'@'%' \
IDENTIFIED BY 'djl18001';
用合適的密碼替換 CINDER_DBPASS,此試驗環境配置的密碼為:djl18001
退出數據庫客戶端。
2)獲得 admin 憑證來獲取只有管理員能執行的命令的訪問權限:
[root@likeadmin conf.d]# source /root/.admin-openrc
3)要創建服務證書,完成這些步驟:
創建一個 cinder 用戶:
[root@likeadmin conf.d]# openstack user create --domain default --password-prompt cinder
User Password: djl18001
Repeat User Password:
+---------------------+----------------------------------+
| Field | Value |
+---------------------+----------------------------------+
| domain_id | default |
| enabled | True |
| id | 5cb9dc4f00f54ac083b9f6a20702d5f2 |
| name | cinder |
| options | {} |
| password_expires_at | None |
+---------------------+----------------------------------+
[root@likeadmin conf.d]#
添加 admin 角色到 cinder 用戶上。
#openstack role add --project service --user cinder admin
4)創建cinderv2和cinderv3服務實體:
[root@likeadmin ~]# openstack service create --name cinderv2 \
> --description "OpenStack Block Storage" volumev2
+-------------+----------------------------------+
| Field | Value |
+-------------+----------------------------------+
| description | OpenStack Block Storage |
| enabled | True |
| id | bebac42bee1240d88185e391c9db0e4b |
| name | cinderv2 |
| type | volumev2 |
+-------------+----------------------------------+
[root@likeadmin ~]# openstack service create --name cinderv3 \
> --description "OpenStack Block Storage" volumev3
+-------------+----------------------------------+
| Field | Value |
+-------------+----------------------------------+
| description | OpenStack Block Storage |
| enabled | True |
| id | 51d66eede82042c889a19100bf33f421 |
| name | cinderv3 |
| type | volumev3 |
+-------------+----------------------------------+
[root@likeadmin ~]#
注解:塊設備存儲服務要求兩個服務實體。
5)創建塊設備存儲服務的 API 入口點
[root@likeadmin ~]# openstack endpoint create --region RegionOne \
> volumev2 public http://likeadmin:8776/v2/%\(project_id\)s
+--------------+-----------------------------------------+
| Field | Value |
+--------------+-----------------------------------------+
| enabled | True |
| id | 41588a56fbb34e818d544ba88bbc6e7b |
| interface | public |
| region | RegionOne |
| region_id | RegionOne |
| service_id | bebac42bee1240d88185e391c9db0e4b |
| service_name | cinderv2 |
| service_type | volumev2 |
| url | http://likeadmin:8776/v2/%(project_id)s |
+--------------+-----------------------------------------+
[root@likeadmin ~]# openstack endpoint create --region RegionOne \
> volumev2 internal http://likeadmin:8776/v2/%\(project_id\)s
+--------------+-----------------------------------------+
| Field | Value |
+--------------+-----------------------------------------+
| enabled | True |
| id | 102356c41d704315b05cb7e0eb14deca |
| interface | internal |
| region | RegionOne |
| region_id | RegionOne |
| service_id | bebac42bee1240d88185e391c9db0e4b |
| service_name | cinderv2 |
| service_type | volumev2 |
| url | http://likeadmin:8776/v2/%(project_id)s |
+--------------+-----------------------------------------+
[root@likeadmin ~]# openstack endpoint create --region RegionOne \
> volumev2 admin http://likeadmin:8776/v2/%\(project_id\)s
+--------------+-----------------------------------------+
| Field | Value |
+--------------+-----------------------------------------+
| enabled | True |
| id | aa3964f87d394e48bfd79ab9675d6a5d |
| interface | admin |
| region | RegionOne |
| region_id | RegionOne |
| service_id | bebac42bee1240d88185e391c9db0e4b |
| service_name | cinderv2 |
| service_type | volumev2 |
| url | http://likeadmin:8776/v2/%(project_id)s |
+--------------+-----------------------------------------+
[root@likeadmin ~]# openstack endpoint create --region RegionOne \
> volumev3 public http://likeadmin:8776/v3/%\(project_id\)s
+--------------+-----------------------------------------+
| Field | Value |
+--------------+-----------------------------------------+
| enabled | True |
| id | 1b3c4b6d51d643dcb1b4e0156dc189d5 |
| interface | public |
| region | RegionOne |
| region_id | RegionOne |
| service_id | 51d66eede82042c889a19100bf33f421 |
| service_name | cinderv3 |
| service_type | volumev3 |
| url | http://likeadmin:8776/v3/%(project_id)s |
+--------------+-----------------------------------------+
[root@likeadmin ~]# openstack endpoint create --region RegionOne \
> volumev3 internal http://likeadmin:8776/v3/%\(project_id\)s
+--------------+-----------------------------------------+
| Field | Value |
+--------------+-----------------------------------------+
| enabled | True |
| id | 2085d8a66ee24645ba26f071c3b1e24f |
| interface | internal |
| region | RegionOne |
| region_id | RegionOne |
| service_id | 51d66eede82042c889a19100bf33f421 |
| service_name | cinderv3 |
| service_type | volumev3 |
| url | http://likeadmin:8776/v3/%(project_id)s |
+--------------+-----------------------------------------+
[root@likeadmin ~]# openstack endpoint create --region RegionOne \
> volumev3 admin http://likeadmin:8776/v3/%\(project_id\)s
+--------------+-----------------------------------------+
| Field | Value |
+--------------+-----------------------------------------+
| enabled | True |
| id | dd51fadba29647eea46c19db330f4522 |
| interface | admin |
| region | RegionOne |
| region_id | RegionOne |
| service_id | 51d66eede82042c889a19100bf33f421 |
| service_name | cinderv3 |
| service_type | volumev3 |
| url | http://likeadmin:8776/v3/%(project_id)s |
+--------------+-----------------------------------------+
[root@likeadmin ~]#
2.安全並配置組件
1)安裝軟件包:
# yum install openstack-cinder
2)編輯 /etc/cinder/cinder.conf,同時完成如下動作:
在 [database] 部分,配置數據庫訪問:
[database]
# ...
connection = mysql+pymysql://cinder:djl18001@likeadmin/cinder
用你為塊設備存儲數據庫選擇的密碼替換 CINDER_DBPASS。
3)在``[DEFAULT]``部分,配置``RabbitMQ``消息隊列訪問權限:
[DEFAULT]
# ...
transport_url = rabbit://openstack:djl18001@likeadmin
用你在 “RabbitMQ” 中為 “openstack” 選擇的密碼替換 “RABBIT_PASS”。
4)在 “[DEFAULT]” 和 “[keystone_authtoken]” 部分,配置認證服務訪問:
[DEFAULT]
# ...
auth_strategy = keystone
[keystone_authtoken]
# ...
auth_uri = http://likeadmin:5000
auth_url = http://likeadmin:35357
memcached_servers = likeadmin:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = cinder
password = djl18001
將 CINDER_PASS 替換為你在認證服務中為 cinder 用戶選擇的密碼。
5)在 [DEFAULT 部分,配置``my_ip`` 來使用控制節點的管理接口的IP 地址。
[DEFAULT]
# ...
my_ip = 10.0.0.11
6)在 [oslo_concurrency] 部分,配置鎖路徑:
[oslo_concurrency]
# ...
lock_path = /var/lib/cinder/tmp
初始化塊設備服務的數據庫:
# su -s /bin/sh -c "cinder-manage db sync" cinder
3.配置計算節點以使用塊設備存儲
1)編輯文件 /etc/nova/nova.conf 並添加如下到其中:
[cinder]
os_region_name = RegionOne
4.完成安裝
1)重啟計算API 服務:
# systemctl restart openstack-nova-api.service
2)啟動塊設備存儲服務,並將其配置為開機自啟:
# systemctl enable openstack-cinder-api.service openstack-cinder-scheduler.service
# systemctl start openstack-cinder-api.service openstack-cinder-scheduler.service
5. 安裝並配置一個存儲節點
這個部分描述怎樣為塊存儲服務安裝並配置存儲節點。為簡單起見,這里配置一個有一個空的本地塊存儲設備的存儲節點。
這個向導用的是 /dev/sdb,但是你可以為你特定的節點中替換成不同的值。
該服務在這個設備上使用:term:‘LVM<Logical Volume Manager (LVM)>’提供邏輯卷,‘ISCSI<iSCSI Qualified Name (IQN)>’提供實例使用協議。
你可以按照這些提示對你的其他存儲節點的環境對水平環境稍作修改。
在你安裝和配置塊存儲服務之前,你必須准備好存儲設備。
注解:在存儲節點實施這些步驟。
1)安裝支持的工具包:
安裝 LVM 包:
# yum install lvm2
啟動LVM的metadata服務並且設置該服務隨系統啟動:
# systemctl enable lvm2-lvmetad.service
# systemctl start lvm2-lvmetad.service
2)在你安裝和配置塊存儲服務之前,你必須准備好存儲設備。
下面是如何進行磁盤管理,在進行磁盤管理前,我已對該虛擬機添加了一塊硬盤。
[root@likeblock ~]# fdisk -l
Disk /dev/sda: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000b011c
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 2099199 1048576 83 Linux
/dev/sda2 2099200 20971519 9436160 8e Linux LVM
Disk /dev/sdb: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mapper/centos-root: 8585 MB, 8585740288 bytes, 16769024 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mapper/centos-swap: 1073 MB, 1073741824 bytes, 2097152 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
[root@likeblock ~]#
[root@likeblock ~]# fdisk /dev/sdb
Welcome to fdisk (util-linux 2.23.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x78a4d600.
Command (m for help): m
Command action
a toggle a bootable flag
b edit bsd disklabel
c toggle the dos compatibility flag
d delete a partition
g create a new empty GPT partition table
G create an IRIX (SGI) partition table
l list known partition types
m print this menu
n add a new partition
o create a new empty DOS partition table
p print the partition table
q quit without saving changes
s create a new empty Sun disklabel
t change a partition's system id
u change display/entry units
v verify the partition table
w write table to disk and exit
x extra functionality (experts only)
Command (m for help): n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): p
Partition number (1-4, default 1):
First sector (2048-20971519, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-20971519, default 20971519): 10485708
Partition 1 of type Linux and of size 5 GiB is set
Command (m for help): p
Disk /dev/sdb: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x78a4d600
Device Boot Start End Blocks Id System
/dev/sdb1 2048 10485708 5241830+ 83 Linux
Command (m for help): t
Selected partition 1
Hex code (type L to list all codes): 8e
Changed type of partition 'Linux' to 'Linux LVM'
Command (m for help): p
Disk /dev/sdb: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x78a4d600
Device Boot Start End Blocks Id System
/dev/sdb1 2048 10485708 5241830+ 8e Linux LVM
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@likeblock ~]#
[root@likeblock ~]#
[root@likeblock ~]# fdisk -l
Disk /dev/sda: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000b011c
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 2099199 1048576 83 Linux
/dev/sda2 2099200 20971519 9436160 8e Linux LVM
Disk /dev/sdb: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x78a4d600
Device Boot Start End Blocks Id System
/dev/sdb1 2048 10485708 5241830+ 8e Linux LVM
Disk /dev/mapper/centos-root: 8585 MB, 8585740288 bytes, 16769024 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mapper/centos-swap: 1073 MB, 1073741824 bytes, 2097152 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
[root@likeblock ~]#
3)創建LVM 物理卷 /dev/sdb:
# pvcreate /dev/sdb
Physical volume "/dev/sdb" successfully created
創建 LVM 卷組 cinder-volumes:
# vgcreate cinder-volumes /dev/sdb
Volume group "cinder-volumes" successfully created
塊存儲服務會在這個卷組中創建邏輯卷
只有實例可以訪問塊存儲卷組。但是,底層的操作系統管理着與這些卷相關聯的設備。
默認情況下,LVM卷掃描工具會掃描``/dev`` 目錄,查找包含卷的塊存儲設備。如果項目在他們的卷上使用了LVM,掃描工具便會在檢測到這些卷時嘗試緩存它們,
這可能會在底層操作系統和項目卷上產生各種問題。所以您必須重新配置LVM,讓它掃描僅包含``cinder-volume``卷組的設備。編輯``/etc/lvm/lvm.conf``文件並完成下面的操作:
在``devices``部分,添加一個過濾器,只接受``/dev/sdb``設備,拒絕其他所有設備:
devices {
...
filter = [ "a/sdb/", "r/.*/"]
每個過濾器組中的元素都以``a``開頭,即為 accept,或以 r 開頭,即為**reject**,並且包括一個設備名稱的正則表達式規則。過濾器組必須以``r/.*/``結束,
過濾所有保留設備。您可以使用 :命令:`vgs -vvvv` 來測試過濾器。
如果您的存儲節點在操作系統磁盤上使用了 LVM,您還必需添加相關的設備到過濾器中。例如,如果 /dev/sda 設備包含操作系統:
filter = [ "a/sda/", "a/sdb/", "r/.*/"]
類似地,如果您的計算節點在操作系統磁盤上使用了 LVM,您也必需修改這些節點上 /etc/lvm/lvm.conf 文件中的過濾器,將操作系統磁盤包含到過濾器中。例如,如果``/dev/sda`` 設備包含操作系統:
filter = [ "a/sda/", "r/.*/"]
5.安全並配置組件
安裝軟件包:
# yum install openstack-cinder targetcli python-keystone
編輯 /etc/cinder/cinder.conf,同時完成如下動作:
在 [database] 部分,配置數據庫訪問:
[database]
# ...
connection = mysql+pymysql://cinder:djl18001@likeadmin/cinder
用你為塊設備存儲數據庫選擇的密碼替換 CINDER_DBPASS。
在``[DEFAULT]``部分,配置``RabbitMQ``消息隊列訪問權限:
[DEFAULT]
# ...
transport_url = rabbit://openstack:djl18001@likeadmin
用你在 “RabbitMQ” 中為 “openstack” 選擇的密碼替換 “RABBIT_PASS”。
在 “[DEFAULT]” 和 “[keystone_authtoken]” 部分,配置認證服務訪問:
[DEFAULT]
# ...
auth_strategy = keystone
[keystone_authtoken]
# ...
auth_uri = http://likeadmin:5000
auth_url = http://likeadmin:35357
memcached_servers = likeadmin:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = cinder
password = djl18001
將 CINDER_PASS 替換為你在認證服務中為 cinder 用戶選擇的密碼。
在 [DEFAULT] 部分,配置 my_ip 選項:
[DEFAULT]
# ...
my_ip = 92.168.164.136
將其中的``MANAGEMENT_INTERFACE_IP_ADDRESS``替換為存儲節點上的管理網絡接口的IP 地址,
例如樣例架構 <overview-example-architectures>中所示的第一台節點 10.0.0.41 。
在``[lvm]``部分中,配置LVM后端,包括LVM驅動,``cinder-volumes``卷組 ,iSCSI 協議和適當的 iSCSI服務。如果``[lvm]``部分不存在,則創建它:
[lvm]
volume_driver = cinder.volume.drivers.lvm.LVMVolumeDriver
volume_group = cinder-volumes
iscsi_protocol = iscsi
iscsi_helper = lioadm
在 [DEFAULT] 部分,啟用 LVM 后端:
[DEFAULT]
# ...
enabled_backends = lvm
在 [DEFAULT] 區域,配置鏡像服務 API 的位置:
[DEFAULT]
# ...
glance_api_servers = http://likeadmin:9292
在 [oslo_concurrency] 部分,配置鎖路徑:
[oslo_concurrency]
# ...
lock_path = /var/lib/cinder/tmp
完成安裝
啟動塊存儲卷服務及其依賴的服務,並將其配置為隨系統啟動:
# systemctl enable openstack-cinder-volume.service target.service
# systemctl start openstack-cinder-volume.service target.service
驗證塊設備存儲服務的操作。
注解:在控制節點上執行這些命令。
獲得 admin 憑證來獲取只有管理員能執行的命令的訪問權限:
[root@likeadmin ~]# source /root/.admin-openrc
列出服務組件以驗證是否每個進程都成功啟動:
[root@likeadmin ~]# openstack volume service list
+------------------+---------------+------+---------+-------+----------------------------+
| Binary | Host | Zone | Status | State | Updated At |
+------------------+---------------+------+---------+-------+----------------------------+
| cinder-scheduler | likeadmin | nova | enabled | up | 2019-04-09T06:57:36.000000 |
| cinder-volume | likeblock@lvm | nova | enabled | up | 2019-04-09T06:57:49.000000 |
+------------------+---------------+------+---------+-------+----------------------------+
[root@likeadmin ~]#
注:至此我們已經完成塊存儲的配置,可以通過塊存儲啟動實例了