在上一篇中我們介紹了Glusterfs在CentOS7上的安裝，並且提到Glusterfs client端有三種：Native client,NFS,Samba， 今天我們就來一起學習下這三種方式。
在開始之前，我已經做好了一個Volume:
[root@node01 ~]# gluster volume create repvol replica 2 > node01.lab.example.com:/bricks/thinvol1/brick > node02.lab.example.com:/bricks/thinvol1/brick volume create: repvol: success: please start the volume to access data [root@node01 ~]# gluster volume start repvol volume start: repvol: success [root@node01 ~]# gluster volume info repvol Volume Name: repvol Type: Replicate Volume ID: 7015b7fe-8039-4d5d-b698-e8321df5289c Status: Started Snapshot Count: 0 Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: node01.lab.example.com:/bricks/thinvol1/brick Brick2: node02.lab.example.com:/bricks/thinvol1/brick Options Reconfigured: transport.address-family: inet performance.readdir-ahead: on nfs.disable: on
1 Native mount client
1). server 端無需做額外設置，只需正常安裝，配置glusterfs服務即可
2). Client 端需要安裝glusterfs client,這里我們還是把node02看做client
[root@node02 ~]# yum install -y glusterfs glusterfs-fuse [root@node02 ~]# mount -t glusterfs node01.lab.example.com:repvol /mnt [root@node02 ~]# df -h | grep mnt node01.lab.example.com:repvol 2.0G 33M 2.0G 2% /mnt
Native mount的一大優勢是支持“高可用”，雖然我們掛載的時使用的命令是指定的node01.lab.example.com, 但是當node01不可用時，glusterfs volume 還是可以work的。我們可以stop node01上的glusterd service模擬node01出現故障，看是否影響volume repvol的使用。
[root@node01 ~]# systemctl stop glusterd
[root@node02 /]# df -h | grep mnt node01.lab.example.com:repvol 2.0G 33M 2.0G 2% /mnt [root@node02 /]# cd /mnt [root@node02 mnt]# touch test.file [root@node02 mnt]#
從上面可以看出volume repvol在node01 不可用時，還依然可用。
2 NFS
在查看volume詳細信息時，我們會發現有一個選項“nfs.disable: on”,從字面上就可以理解默認是禁用了nfs的
[root@node01 /]# gluster volume info repvol Volume Name: repvol Type: Replicate Volume ID: 7015b7fe-8039-4d5d-b698-e8321df5289c Status: Started Snapshot Count: 0 Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: node01.lab.example.com:/bricks/thinvol1/brick Brick2: node02.lab.example.com:/bricks/thinvol1/brick Options Reconfigured: transport.address-family: inet performance.readdir-ahead: on nfs.disable: on
手動放開nfs,並重啟volume
[root@node01 ~]# gluster volume set repvol nfs.disable off volume set: success [root@node01 ~]# gluster volume stop repvol Stopping volume will make its data inaccessible. Do you want to continue? (y/n) y volume stop: repvol: success [root@node01 ~]# gluster volume start repvol volume start: repvol: success [root@node01 ~]# gluster volume info repvol | grep nfs.disable nfs.disable: off [root@node01 ~]#
同時server上會啟動這樣一個進程做一個nfs共享以便提供nfs掛載
[root@node01 ~]# ps -aux | grep nfs | grep -v grep root 20064 0.1 6.5 581640 65028 ? Ssl 04:23 0:00 /usr/sbin/glusterfs -s localhost --volfile-id gluster/nfs -p /var/lib/glusterd/nfs/run/nfs.pid -l /var/log/glusterfs/nfs.log -S /var/run/gluster/d8b498939d669fdc978b5b3d96d5aa9d.socket
放開防火牆限制
[root@node01 ~]# firewall-cmd --add-service=nfs --add-service=rpc-bind success
在node02上使用nfs方式掛載volume repvol
[root@node02 /]# mount -t nfs node01.lab.example.com:repvol /mnt [root@node02 /]# df -h | grep mnt node01.lab.example.com:repvol 2.0G 33M 2.0G 2% /mnt
這里可以看出nfs掛載似乎也不復雜，而且client上不需要安裝額外的glusterfs組件。
3 Samba
將option "nfs.disable"恢復默認設置，並重啟volume
[root@node01 glusterfs]# gluster volume reset repvol nfs.disable volume reset: success: reset volume successful
[root@node01 ~]# gluster volume stop repvol Stopping volume will make its data inaccessible. Do you want to continue? (y/n) y volume stop: repvol: success
[root@node01 ~]# gluster volume start repvol volume start: repvol: success
[root@node01 ~]# gluster volume info repvol | grep nfs.disable nfs.disable: on
3.1在node01上安裝samba,啟動samba service並放開防火牆限制
[root@node01 ~]# yum install -y samba [root@node01 ~]# systemctl start smb [root@node01 ~]# firewall-cmd --add-service=samba
3.2修改volume option並重啟volume, glusterd service
編輯/etc/glusterfs/glusterd.vol 文件加入下面一行
option rpc-auth-allow-insecure on
[root@node01 ~]# gluster volume set repvol server.allow-insecure on [root@node01 ~]# gluster volume set repvol storage.batch-fsync-delay-usec 0
[root@node01 ~]# gluster volume set repvol stat-prefetch off
[root@node01 ~]# systemctl restart glusterd [root@node01 ~]# gluster volume stop repvol [root@node01 ~]# gluster volume start repvol
---查看一下samba配置文件，可以看到最下面多出一個關於repvol的共享：
[root@node01 ~]# cat /etc/samba/smb.conf
......
[gluster-repvol]
comment = For samba share of volume repvol
vfs objects = glusterfs
glusterfs:volume = repvol
glusterfs:logfile = /var/log/samba/glusterfs-repvol.%M.log
glusterfs:loglevel = 7
path = /
read only = no
guest ok = yes
......
3.3創建一個samba user
[root@node01 ~]# adduser -s /sbin/nologin smbuser
[root@node01 ~]# smbpasswd -a smbuser New SMB password: Retype new SMB password: Added user smbuser.
在node01 server 上掛載volume repvol到/mnt目錄下，並設修改目錄權限如下
[root@node01 ~]# mount -t glusterfs node01.lab.example.com:repvol /mnt [root@node01 ~]# chown smbuser: /mnt [root@node01 ~]# umount /mnt
3.4 在node02上安裝samba-client,cifs-utils
[root@node02 ~]# yum install -y cifs-utils [root@node02 ~]# yum install -y samba-client*
驗證samba共享目錄
[root@node02 ~]# smbclient -L node01.lab.example.com -U%
Domain=[SAMBA] OS=[Windows 6.1] Server=[Samba 4.4.4]
Sharename Type Comment
--------- ---- -------
print$ Disk Printer Drivers
IPC$ IPC IPC Service (Samba 4.4.4)
gluster-repvol Disk For samba share of volume repvol
Domain=[SAMBA] OS=[Windows 6.1] Server=[Samba 4.4.4]
Server Comment
--------- -------
Workgroup Master
--------- -------
使用smbuser進行掛載
[root@node02 ~]# mount -t cifs -o user=smbuser,pass=redhat //node01.lab.example.com/gluster-repvol /mnt Retrying with upper case share name mount error(6): No such device or address Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
查看samba日志：/var/log/samba/log.smbd發現了如下一些messages:
[2017/07/25 08:39:56.358336, 0] ../lib/util/modules.c:48(load_module)
Error loading module ‘/usr/lib64/samba/vfs/glusterfs.so‘: /usr/lib64/samba/vfs/glusterfs.so: cannot open shared object file: No such file or directory
google了很久，得到提示需要安裝samba-vfs-glusterfs.x86_64，然后yum進行安裝
再次嘗試掛載
[root@node02 ~]# mount -t cifs //node01.lab.example.com/gluster-repvol /mnt -o username=smbuser,password=redhat mount error(5): Input/output error Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
發現錯誤已經不同了，再次找到node01下面的samba日志，發現
[root@node01 samba]# cd /var/log/samba/ [root@node01 samba]# ls cores glusterfs-repvol.192.168.133.135.log log.smbd old
192.168.133.135即為node02的ip，打開這個glusterfs-repvol.192.168.133.135.log后發現如下錯誤：
[2017-07-25 13:09:46.473070] E [socket.c:3097:socket_connect] 0-gfapi: connection attempt on failed, (Permission denied)
[2017-07-25 13:09:46.473307] I [MSGID: 104025] [glfs-mgmt.c:752:mgmt_rpc_notify] 0-glfs-mgmt: Exhausted all volfile servers [Transport endpoint is not connected]
[2017-07-25 13:09:46.474733] I [MSGID: 101191] [event-epoll.c:659:event_dispatch_epoll_worker] 0-epoll: Exited thread with index 1
[2017-07-25 13:09:46.473303] E [MSGID: 104024] [glfs-mgmt.c:735:mgmt_rpc_notify] 0-glfs-mgmt: failed to connect with remote-host: localhost (Transport endpoint is not connected) [Transport endpoint is not connected]
權限有問題！！！而權限無外乎就是目錄本身權限和selinux權限，於是關閉了node01上的selinux，然后嘗試掛載，這次掛載成功。哈哈。。。不過selinux這里的設置還沒有研究出該怎么配置，先記在這里。
[root@node01 samba]# setenforce 0
[root@node02 ~]# mount -t cifs //node01.lab.example.com/gluster-repvol /mnt -o username=smbuser,password=redhat [root@node02 ~]# df -h | grep mnt //node01.lab.example.com/gluster-repvol 2.0G 33M 2.0G 2% /mnt [root@node02 ~]#
本文出自 “不積跬步，無以至千里” 博客，請務必保留此出處http://jiaxiaolei.blog.51cto.com/3117381/1950981