Tgtd+Ceph部署
一、yum安裝tgt
[root@c720181 ~]# yum --enablerepo=epel -y install scsi-target-utils libxslt
（1）安裝 scsi-target-utils
安裝 scsi-target-utils
# yum --enablerepo=epel -y install scsi-target-utils libxslt查看版本
# tgtadm --version 1.0.55（2）啟動 tgtd 服務
//設置開機啟動 systemctl enable tgtd //啟動 systemctl start tgtd（3）查看是否支持 rbd
# tgtadm --lld iscsi --mode system --op show | grep rbd 如果有類似於上面的輸出結果，說明你的tgt加載了rbd模塊。該模塊實際上是一個名為bs_rbd.so的動態鏈接庫。如果命令沒有輸出，則說明你的tgt還不支持rbd，你需要安裝最新的tgt，或者直接下載編譯最新的源碼。
二、編譯安裝tgt
(1) 准備編譯環境
[root@c720181 ~]# yum install git install librados2-devel librbd1-devel gcc rpm-build
(2)獲取Tgt
[root@c720181 ~]# git clone https://github.com/fujita/tgt.git
(3) 進入目錄后編輯
[root@c720181 tgt]# vim Makefile
export ISCSI_RDMA
export CEPH_RBD = 1 #修改為=1
export GLFS_BD
export SD_NOTIFY
[root@c720181 tgt]# vim scripts/tgtd.spec
...
%{_mandir}/man5/*
%{_mandir}/man8/*
%{_initrddir}/tgtd
/usr/lib/tgt/backing-store/bs_rbd.so
/etc/bash_completion.d/tgt
%attr(0600,root,root) %config(noreplace) /etc/tgt/targets.conf
...
(4)編譯
[root@c720181 tgt]# make rpm
Building version: 1.0.79-v1.0.79
Creating rpm build dirs under /root/tgt/pkg
Creating tgz scsi-target-utils-1.0.79-v1.0.79.tgz
Creating rpm
Wrote: /root/tgt/pkg/SRPMS/scsi-target-utils-1.0.79-v1.0.79.src.rpm
Failed to build rpm. LOG: /tmp/build-pkg.sh-386879.log
make: *** [rpm] Error 1
[root@c720181 tgt]# vim /tmp/build-pkg.sh-386879.log
error: Failed build dependencies:
libibverbs-devel is needed by scsi-target-utils-1.0.79-v1.0.79.x86_64
librdmacm-devel is needed by scsi-target-utils-1.0.79-v1.0.79.x86_64
libaio-devel is needed by scsi-target-utils-1.0.79-v1.0.79.x86_64
docbook-style-xsl is needed by scsi-target-utils-1.0.79-v1.0.79.x86_64c
解決辦法：安裝依賴包
[root@c720181 tgt]# yum install -y libibverbs-devel librdmacm-devel libaio-devel docbook-style-xsl
重新編譯：
[root@c720181 tgt]# make rpm
Building version: 1.0.79-v1.0.79
Creating rpm build dirs under /root/tgt/pkg
Creating tgz scsi-target-utils-1.0.79-v1.0.79.tgz
Creating rpm
Wrote: /root/tgt/pkg/SRPMS/scsi-target-utils-1.0.79-v1.0.79.src.rpm
Wrote: /root/tgt/pkg/RPMS/x86_64/scsi-target-utils-1.0.79-v1.0.79.x86_64.rpm
Wrote: /root/tgt/pkg/RPMS/x86_64/scsi-target-utils-debuginfo-1.0.79-v1.0.79.x86_64.rpm
Done.
(5)生成安裝文件
[root@c720181 tgt]# ls pkg/RPMS/x86_64/
scsi-target-utils-1.0.79-v1.0.79.x86_64.rpm scsi-target-utils-debuginfo-1.0.79-v1.0.79.x86_64.rpm
[root@c720181 tgt]# cp pkg/RPMS/x86_64/scsi-target-utils-1.0.79-v1.0.79.x86_64.rpm /root/
(6)在3個MON節點安裝Tgt rpm包
[root@c720181 ~]# rpm -ivh scsi-target-utils-1.0.79-v1.0.79.x86_64.rpm --force
Preparing... ################################# [100%]
Updating / installing...
1:scsi-target-utils-1.0.79-v1.0.79 ################################# [100%]
[root@c720182 ~]# rpm -ivh scsi-target-utils-1.0.79-v1.0.79.x86_64.rpm --force
Preparing... ################################# [100%]
Updating / installing...
1:scsi-target-utils-1.0.79-v1.0.79 ################################# [100%]
[root@c720183 ~]# rpm -ivh scsi-target-utils-1.0.79-v1.0.79.x86_64.rpm --force
Preparing... ################################# [100%]
Updating / installing...
1:scsi-target-utils-1.0.79-v1.0.79 ################################# [100%]
三、使用 rbd 作為 target
1. 創建 rbd 池和 image
[ceph-admin@c720181 my-cluster]$ ceph osd pool create tgtpool 8
pool 'tgtpool' created
[ceph-admin@c720181 my-cluster]$ rbd create --size 10240 tgtpool/image01
[ceph-admin@c720181 my-cluster]$ rbd info tgtpool/image01
rbd image 'image01':
size 10GiB in 2560 objects
order 22 (4MiB objects)
block_name_prefix: rbd_data.2a8276b8b4567
format: 2
features: layering, exclusive-lock, object-map, fast-diff, deep-flatten
flags:
create_timestamp: Fri Aug 30 18:57:33 2019
2、 通過配置文件來創建 target
[root@c720181 ~]# vim /etc/tgt/targets.conf
include /etc/tgt/conf.d/*.conf
<target iqn.c720181.kevin.cn:cephtgt.target0>
bs-type rbd
backing-store tgtpool/image01
initiator-address 172.22.69.0/24
</target>
[root@c720181 ~]# systemctl start tgtd
3、查看激活的 target
[root@c720181 ~]# tgtadm --lld iscsi --mode target --op show
Target 1: iqn.c720181.kevin.cn:cephtgt.target0
System information:
Driver: iscsi
State: ready
I_T nexus information:
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00010000
SCSI SN: beaf10
Size: 0 MB, Block size: 1
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
SWP: No
Thin-provisioning: No
Backing store type: null
Backing store path: None
Backing store flags:
LUN: 1
Type: disk
SCSI ID: IET 00010001
SCSI SN: beaf11
Size: 10737 MB, Block size: 512
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
SWP: No
Thin-provisioning: No
Backing store type: rbd
Backing store path: tgtpool/image01
Backing store flags:
Account information:
ACL information:
172.22.69.0/24
四、客戶端安裝iSCSI Initor程序並掛載
(1)安裝iSCSI Initor程序
[root@client ~]# yum -y install iscsi-initiator-utils
(2)掃描iSCSI Target
[root@client ~]# iscsiadm -m discovery -t sendtargets -p c720181（tgt服務端的IP或主機名）
172.22.69.63:3260,1 iqn.c720181.kevin.cn:cephtgt.target0
(3)掛載目標設備
如果是掛載這個網絡上所有的節點，使用以下命令即可
iscsiadm -m node --login
如果是掛載指定的一個存儲設備，則需要使用以下命令
iscsiadm -m node -T <target_name>:<protocol> -p <server_ip>:<port> -l
使用示例
[root@client ~]# iscsiadm -m node -T iqn.c720181.kevin.cn:cephtgt.target0 -p 172.22.69.63:3260 --login
Logging in to [iface: default, target: iqn.c720181.kevin.cn:cephtgt.target0, portal: 172.22.69.63,3260] (multiple)
Login to [iface: default, target: iqn.c720181.kevin.cn:cephtgt.target0, portal: 172.22.69.63,3260] successful.
取消掛載目標設備
iscsiadm -m node –T iqn.c720181.kevin.cn:cephtgt.target0 -p 172.22.69.63:3260 -u
其中末尾的u為unlogin的縮寫
(4)驗證本地掛載是否成功
[root@client ~]# lsscsi
[2:0:0:1] disk IET VIRTUAL-DISK 0001 /dev/sdb
[root@client ~]# fdisk -l
Disk /dev/sdb: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4194304 bytes
I/O size (minimum/optimal): 4194304 bytes / 4194304 bytes
五、windows客戶端連接
打開軟件
首先打開控制面板，然后搜索管理工具，雙擊打開Iscsi發起程序
掛載設備
在下圖中的目標中輸入iscsi所在的服務器IP地址，點擊快速連接，就可以發現該服務器上可用的Iscsi硬盤，選擇后即可點擊連接。如果搜索服務器沒搜索到，需在服務端的/etc/tgt/targets.conf的需要用到的硬盤的target中加入windows客戶端的IP地址。
initiator-address <windows_client_ip_address >
初始化磁盤
右鍵我的電腦，單擊管理，選擇磁盤管理，即可看到以下提示
點擊確定進行下一步，此時就能看到多出一個100GB未分配的磁盤
右鍵選中磁盤1，新建簡單卷，根據提示新創建一個磁盤。等待其格式化完成后，就可以看到一個新的可用磁盤
錯誤排除
如果在連接目標設備時發現以下錯誤
問題肯定處在目標設備名字帶中文，如下圖所示
需要在配置文件中把target的名字全部改成小寫即可，如下圖所示
改完后需保存，然后重啟tgt服務
service tgt restart
原文鏈接：https://blog.csdn.net/u010317005/article/details/80322855