Kerbernetes的StatefulSet與Operator
作者：尹正傑
版權聲明：原創作品，謝絕轉載！否則將追究法律責任。
一.StatefulSet概述
1>.StatefulSet與Operator的關系
管理一組有狀態pod服務的部署(deployment)和擴展(scaling)，並保證這些pod的順序和唯一性。 與部署一樣，StatefulSet管理基於相同容器規范的pod。 與部署不同，statefulSet為每個pod維護一個粘性標識。這些pod是從同一個規范創建的，但不能互換：每個pod都有一個peristent標識符，可以在任何重新調度中維護。 事實上，使用StatefulSet可以很輕松的幫咱們完成部署，但是擴展的話需要我們自己編寫封裝擴容集群或者縮容集群腳本代碼的程序,這個程序我們統稱為Operator。而且我們編寫的腳本還需要進行大量測試，否則出現任意一個bug，將會在線上帶來不堪設想的后果。 於是,有一家公司叫CoreOS的公司對StatefulSet進行二次封裝開源了Operator SDK(Software Development Kit)接口，用戶可以借助於Operator SDK的API比較輕松定制出Operator控制器。 這意味着第三方的程序員再去開發雲原生應用不是針對Kubernetes雲原生API,而是針對CoreOS公司的Operator SDK。換句話說，Kubernetes的大本營被人開了一道后門,從后方侵入快速拿下一半的雲原生領土，而這個領土的所屬方叫CoreOS。而CoreOS已經是Redhat旗下的產品，而Redhat又是IBM旗下的產品。 在國外,微軟(Microsoft)，亞馬遜(Amazon)已經是雲計算的領頭羊了，而IBM作為一個老牌的基礎設施公司雖說也有自己的雲計算產品但影響力卻遠遠不如前兩者公司。因為大公司通常是很難接收革新的,它們有自己成熟的產品也有充分的市場份額，有一個其它產品時競爭對手時，它們通常把競爭對手視為眼中釘肉中刺，而不是擁抱和開發它們，IBM就錯失了很多良機。IBM之所以願意買RedHat，原因之一可能是因為紅帽公司旗下的CoreOS公司。 IBM公司地位發展也給咱們運維人員一定的啟發，這就好比你已經輕車熟路的使用K8S，突然間又出現了一個顛覆性的產品的苗頭之后,你一定不是第一個願意接收它的人，會有一種抵觸心理，此時若很多人抱着開放的心胸去學習了該產品，恍然之間覺得自己的技能要被淘汰了不得不去學習的時候，那些先行者早已在這個行業稱為巨頭了。 目前大部分應用也有自己的Operator開源項目，比較火熱的應用在Github有對應的開源地址。博主推薦閱讀:https://github.com/operator-framework/awesome-operators。
2>.StatefulSet的應用場景
statefulset對於需要以下一項或多項的應用程序很有價值: (1)有唯一固定的網絡標識符; (2)固定唯一的持久存儲; (3)有序優雅的部署和擴展; (4)有序右眼的刪除和終止; (5)有序自動的執行滾動更新； StatefulSet的限制條件(Limitations): (1)各Pod用到的存儲卷必須使用由StorageClass動態供給給或由管理事先創建好的PV; (2)刪除StatefulSet或縮減其規模導致Pod被刪除時不應該自動刪除(這個需要我們定義pvc指定其回收策略為"Retain")其存儲卷以確保數據安全; (3)StatefulSet控制器依賴於一個事先存在的Headless Serivce對象實現Pod對象的持久，唯一的標識符配置;次Headless Service需要由用戶手動配置。
3>.StatefulSet的Pod 標識符(Identity)
有序索引(Ordinal Index): 對於一個有N個副本的StatefulSet，StatefulSet中的每個Pod將被分配一個整數序數，從0到N-1，在這個集合上是唯一的。 穩定網絡ID(Stabel Network ID): StatefulSet中的每個Pod都從StatefulSet的名稱和Pod的序號派生其主機名。 StatefulSet可以使用無頭服務(Headless Service)來控制其Pods的域。此服務管理的域的任務格式為:"$(service name).$(namespace).svc.cluster.local",其中“cluster.local”是群集域。 創建每個Pod時，它將獲得一個匹配的DNS子域，其格式為:"$(podname).$(governing service domain)",其中governing service由StatefulSet上的serviceName字段定義。
4>.Pod管理策略(Management Policies)
在Kubernetes 1.7及更高版本中，StatefulSet允許您放寬其排序保證，同時通過其.spec.podManagementPolicay字段保持其唯一性和標識保證。 有序(OrderedReady)Pod管理(Management): OrderedReady pod management是statefulset的默認設置。換句話說,就是當一個Pod創建成功后再回去創建下一個Pod，即有序創建Pod。
並行的(Parallel)Pod管理(Management): 並行Pod管理告訴StatefulSet控制器並行啟動或終止所有Pod，不要等到Pod運行並准備就緒或完全終止后再啟動或終止另一個Pod。
5>.更新策略(Update Strategies)
在Kubernetes 1.7及更高版本中，StatefulSet的.spec.updateStrategy字段允許您配置和禁用StatefulSet中Pods的容器、標簽、資源請求/限制和注釋的自動滾動更新。 ON Delete: 由用戶手動刪除現有的Pod對象從而觸發其更新過程。 Rolling Updates: 默認策略,它通過自動更新機制完成更新過程，啟動更新過程時,它自動刪除每個Pod對象並以新配置進行重建,更新順序同刪除StatefulSet時的逆向操作機制，一次刪除並更新一個Pod對象。
可以通過指定.spec.update strategy.RollingUpdate.partition對RollingUpdate更新策略進行分區。
如果指定了分區，則當StatefulSet的.pec.template被更新時，序號大於或等於分區的所有pod都將被更新。
序號小於分區的所有pod都將不會更新，即使它們被刪除，也將在以前的版本中重新創建。
二.StatefulSet實戰案例
1>.構建NFS網絡文件系統
[root@test110.yinzhengjie.org.cn ~]# yum -y install nfs-utils Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirror.bit.edu.cn * extras: mirrors.tuna.tsinghua.edu.cn * updates: mirrors.tuna.tsinghua.edu.cn base | 3.6 kB 00:00:00 docker-ce-stable | 3.5 kB 00:00:00 extras | 2.9 kB 00:00:00 updates | 2.9 kB 00:00:00 updates/7/x86_64/primary_db | 6.7 MB 00:00:01 Resolving Dependencies --> Running transaction check ---> Package nfs-utils.x86_64 1:1.3.0-0.65.el7 will be installed --> Processing Dependency: libtirpc >= 0.2.4-0.7 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: gssproxy >= 0.7.0-3 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: rpcbind for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: quota for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libnfsidmap for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: keyutils for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libtirpc.so.1()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libnfsidmap.so.0()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Running transaction check ---> Package gssproxy.x86_64 0:0.7.0-26.el7 will be installed --> Processing Dependency: libini_config >= 1.3.1-31 for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libverto-module-base for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libref_array.so.1(REF_ARRAY_0.1.1)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.2.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.1.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libref_array.so.1()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libcollection.so.2()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libbasicobjects.so.0()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 ---> Package keyutils.x86_64 0:1.5.8-3.el7 will be installed ---> Package libnfsidmap.x86_64 0:0.25-19.el7 will be installed ---> Package libtirpc.x86_64 0:0.2.4-0.16.el7 will be installed ---> Package quota.x86_64 1:4.01-19.el7 will be installed --> Processing Dependency: quota-nls = 1:4.01-19.el7 for package: 1:quota-4.01-19.el7.x86_64 --> Processing Dependency: tcp_wrappers for package: 1:quota-4.01-19.el7.x86_64 ---> Package rpcbind.x86_64 0:0.2.0-48.el7 will be installed --> Running transaction check ---> Package libbasicobjects.x86_64 0:0.1.1-32.el7 will be installed ---> Package libcollection.x86_64 0:0.7.0-32.el7 will be installed ---> Package libini_config.x86_64 0:1.3.1-32.el7 will be installed --> Processing Dependency: libpath_utils.so.1(PATH_UTILS_0.2.1)(64bit) for package: libini_config-1.3.1-32.el7.x86_64 --> Processing Dependency: libpath_utils.so.1()(64bit) for package: libini_config-1.3.1-32.el7.x86_64 ---> Package libref_array.x86_64 0:0.1.5-32.el7 will be installed ---> Package libverto-libevent.x86_64 0:0.2.5-4.el7 will be installed ---> Package quota-nls.noarch 1:4.01-19.el7 will be installed ---> Package tcp_wrappers.x86_64 0:7.6-77.el7 will be installed --> Running transaction check ---> Package libpath_utils.x86_64 0:0.2.1-32.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ============================================================================================================================================================================================================================================================================== Package Arch Version Repository Size ============================================================================================================================================================================================================================================================================== Installing: nfs-utils x86_64 1:1.3.0-0.65.el7 base 412 k Installing for dependencies: gssproxy x86_64 0.7.0-26.el7 base 110 k keyutils x86_64 1.5.8-3.el7 base 54 k libbasicobjects x86_64 0.1.1-32.el7 base 26 k libcollection x86_64 0.7.0-32.el7 base 42 k libini_config x86_64 1.3.1-32.el7 base 64 k libnfsidmap x86_64 0.25-19.el7 base 50 k libpath_utils x86_64 0.2.1-32.el7 base 28 k libref_array x86_64 0.1.5-32.el7 base 27 k libtirpc x86_64 0.2.4-0.16.el7 base 89 k libverto-libevent x86_64 0.2.5-4.el7 base 8.9 k quota x86_64 1:4.01-19.el7 base 179 k quota-nls noarch 1:4.01-19.el7 base 90 k rpcbind x86_64 0.2.0-48.el7 base 60 k tcp_wrappers x86_64 7.6-77.el7 base 78 k Transaction Summary ============================================================================================================================================================================================================================================================================== Install 1 Package (+14 Dependent packages) Total download size: 1.3 M Installed size: 3.6 M Downloading packages: (1/15): keyutils-1.5.8-3.el7.x86_64.rpm | 54 kB 00:00:00 (2/15): libcollection-0.7.0-32.el7.x86_64.rpm | 42 kB 00:00:00 (3/15): libini_config-1.3.1-32.el7.x86_64.rpm | 64 kB 00:00:00 (4/15): gssproxy-0.7.0-26.el7.x86_64.rpm | 110 kB 00:00:00 (5/15): libnfsidmap-0.25-19.el7.x86_64.rpm | 50 kB 00:00:00 (6/15): libref_array-0.1.5-32.el7.x86_64.rpm | 27 kB 00:00:00 (7/15): libpath_utils-0.2.1-32.el7.x86_64.rpm | 28 kB 00:00:00 (8/15): libverto-libevent-0.2.5-4.el7.x86_64.rpm | 8.9 kB 00:00:00 (9/15): libtirpc-0.2.4-0.16.el7.x86_64.rpm | 89 kB 00:00:00 (10/15): libbasicobjects-0.1.1-32.el7.x86_64.rpm | 26 kB 00:00:00 (11/15): quota-4.01-19.el7.x86_64.rpm | 179 kB 00:00:00 (12/15): quota-nls-4.01-19.el7.noarch.rpm | 90 kB 00:00:00 (13/15): tcp_wrappers-7.6-77.el7.x86_64.rpm | 78 kB 00:00:00 (14/15): nfs-utils-1.3.0-0.65.el7.x86_64.rpm | 412 kB 00:00:00 (15/15): rpcbind-0.2.0-48.el7.x86_64.rpm | 60 kB 00:00:00 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Total 2.9 MB/s | 1.3 MB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : libbasicobjects-0.1.1-32.el7.x86_64 1/15 Installing : libref_array-0.1.5-32.el7.x86_64 2/15 Installing : libcollection-0.7.0-32.el7.x86_64 3/15 Installing : libtirpc-0.2.4-0.16.el7.x86_64 4/15 Installing : rpcbind-0.2.0-48.el7.x86_64 5/15 Installing : 1:quota-nls-4.01-19.el7.noarch 6/15 Installing : tcp_wrappers-7.6-77.el7.x86_64 7/15 Installing : 1:quota-4.01-19.el7.x86_64 8/15 Installing : keyutils-1.5.8-3.el7.x86_64 9/15 Installing : libnfsidmap-0.25-19.el7.x86_64 10/15 Installing : libpath_utils-0.2.1-32.el7.x86_64 11/15 Installing : libini_config-1.3.1-32.el7.x86_64 12/15 Installing : libverto-libevent-0.2.5-4.el7.x86_64 13/15 Installing : gssproxy-0.7.0-26.el7.x86_64 14/15 Installing : 1:nfs-utils-1.3.0-0.65.el7.x86_64 15/15 Verifying : libtirpc-0.2.4-0.16.el7.x86_64 1/15 Verifying : libverto-libevent-0.2.5-4.el7.x86_64 2/15 Verifying : 1:quota-4.01-19.el7.x86_64 3/15 Verifying : gssproxy-0.7.0-26.el7.x86_64 4/15 Verifying : libpath_utils-0.2.1-32.el7.x86_64 5/15 Verifying : libnfsidmap-0.25-19.el7.x86_64 6/15 Verifying : keyutils-1.5.8-3.el7.x86_64 7/15 Verifying : 1:nfs-utils-1.3.0-0.65.el7.x86_64 8/15 Verifying : tcp_wrappers-7.6-77.el7.x86_64 9/15 Verifying : libcollection-0.7.0-32.el7.x86_64 10/15 Verifying : libref_array-0.1.5-32.el7.x86_64 11/15 Verifying : libbasicobjects-0.1.1-32.el7.x86_64 12/15 Verifying : rpcbind-0.2.0-48.el7.x86_64 13/15 Verifying : libini_config-1.3.1-32.el7.x86_64 14/15 Verifying : 1:quota-nls-4.01-19.el7.noarch 15/15 Installed: nfs-utils.x86_64 1:1.3.0-0.65.el7 Dependency Installed: gssproxy.x86_64 0:0.7.0-26.el7 keyutils.x86_64 0:1.5.8-3.el7 libbasicobjects.x86_64 0:0.1.1-32.el7 libcollection.x86_64 0:0.7.0-32.el7 libini_config.x86_64 0:1.3.1-32.el7 libnfsidmap.x86_64 0:0.25-19.el7 libpath_utils.x86_64 0:0.2.1-32.el7 libref_array.x86_64 0:0.1.5-32.el7 libtirpc.x86_64 0:0.2.4-0.16.el7 libverto-libevent.x86_64 0:0.2.5-4.el7 quota.x86_64 1:4.01-19.el7 quota-nls.noarch 1:4.01-19.el7 rpcbind.x86_64 0:0.2.0-48.el7 tcp_wrappers.x86_64 0:7.6-77.el7 Complete! [root@test110.yinzhengjie.org.cn ~]#
[root@test110.yinzhengjie.org.cn ~]# mkdir -pv /yinzhengjie/data/volume{1,2,3,4,5} mkdir: created directory ‘/yinzhengjie/data’ mkdir: created directory ‘/yinzhengjie/data/volume1’ mkdir: created directory ‘/yinzhengjie/data/volume2’ mkdir: created directory ‘/yinzhengjie/data/volume3’ mkdir: created directory ‘/yinzhengjie/data/volume4’ mkdir: created directory ‘/yinzhengjie/data/volume5’ [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# ll /yinzhengjie/data/ -R /yinzhengjie/data/: total 0 drwxr-xr-x 2 root root 6 Feb 10 06:04 volume1 drwxr-xr-x 2 root root 6 Feb 10 06:04 volume2 drwxr-xr-x 2 root root 6 Feb 10 06:04 volume3 drwxr-xr-x 2 root root 6 Feb 10 06:04 volume4 drwxr-xr-x 2 root root 6 Feb 10 06:04 volume5 /yinzhengjie/data/volume1: total 0 /yinzhengjie/data/volume2: total 0 /yinzhengjie/data/volume3: total 0 /yinzhengjie/data/volume4: total 0 /yinzhengjie/data/volume5: total 0 [root@test110.yinzhengjie.org.cn ~]#
[root@test110.yinzhengjie.org.cn ~]# vim /etc/exports [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# cat /etc/exports /yinzhengjie/data/volume1 172.200.0.0/21(rw,no_root_squash) /yinzhengjie/data/volume2 172.200.0.0/21(rw,no_root_squash) /yinzhengjie/data/volume3 172.200.0.0/21(rw,no_root_squash) /yinzhengjie/data/volume4 172.200.0.0/21(rw,no_root_squash) /yinzhengjie/data/volume5 172.200.0.0/21(rw,no_root_squash) [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# ll /yinzhengjie/data/ total 0 drwxr-xr-x 2 polkitd root 22 Feb 10 06:40 volume1 drwxr-xr-x 2 root root 6 Feb 10 06:04 volume2 drwxr-xr-x 2 root root 6 Feb 10 06:04 volume3 drwxr-xr-x 2 root root 6 Feb 10 06:04 volume4 drwxr-xr-x 2 root root 6 Feb 10 06:04 volume5 [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# exportfs -rav exporting 172.200.0.0/21:/yinzhengjie/data/volume5 exporting 172.200.0.0/21:/yinzhengjie/data/volume4 exporting 172.200.0.0/21:/yinzhengjie/data/volume3 exporting 172.200.0.0/21:/yinzhengjie/data/volume2 exporting 172.200.0.0/21:/yinzhengjie/data/volume1 [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]#
[root@test110.yinzhengjie.org.cn ~]# systemctl status nfs ● nfs-server.service - NFS server and services Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; disabled; vendor preset: disabled) Active: inactive (dead) [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# systemctl start nfs [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# systemctl enable nfs Created symlink from /etc/systemd/system/multi-user.target.wants/nfs-server.service to /usr/lib/systemd/system/nfs-server.service. [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# systemctl status nfs ● nfs-server.service - NFS server and services Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; enabled; vendor preset: disabled) Drop-In: /run/systemd/generator/nfs-server.service.d └─order-with-mounts.conf Active: active (exited) since Mon 2020-02-10 06:08:52 CST; 6s ago Main PID: 6116 (code=exited, status=0/SUCCESS) CGroup: /system.slice/nfs-server.service Feb 10 06:08:52 test110.yinzhengjie.org.cn systemd[1]: Starting NFS server and services... Feb 10 06:08:52 test110.yinzhengjie.org.cn systemd[1]: Started NFS server and services. [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# ss -ntl | grep 2049 LISTEN 0 64 *:2049 *:* LISTEN 0 64 :::2049 :::* [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# iptables -vnL Chain INPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# iptables -vnL -t nat Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain INPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination [root@test110.yinzhengjie.org.cn ~]#
2>.在K8S集群的每一個節點安裝nfs客戶端驅動
[root@master200.yinzhengjie.org.cn ~]# yum -y install nfs-utils Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirror.bit.edu.cn * extras: mirror.bit.edu.cn * updates: mirror.bit.edu.cn base | 3.6 kB 00:00:00 docker-ce-stable | 3.5 kB 00:00:00 extras | 2.9 kB 00:00:00 kubernetes | 1.4 kB 00:00:00 updates | 2.9 kB 00:00:00 Resolving Dependencies --> Running transaction check ---> Package nfs-utils.x86_64 1:1.3.0-0.65.el7 will be installed --> Processing Dependency: libtirpc >= 0.2.4-0.7 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: gssproxy >= 0.7.0-3 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: rpcbind for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: quota for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libnfsidmap for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: keyutils for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libtirpc.so.1()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libnfsidmap.so.0()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Running transaction check ---> Package gssproxy.x86_64 0:0.7.0-26.el7 will be installed --> Processing Dependency: libini_config >= 1.3.1-31 for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libverto-module-base for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libref_array.so.1(REF_ARRAY_0.1.1)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.2.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.1.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libref_array.so.1()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libcollection.so.2()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libbasicobjects.so.0()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 ---> Package keyutils.x86_64 0:1.5.8-3.el7 will be installed ---> Package libnfsidmap.x86_64 0:0.25-19.el7 will be installed ---> Package libtirpc.x86_64 0:0.2.4-0.16.el7 will be installed ---> Package quota.x86_64 1:4.01-19.el7 will be installed --> Processing Dependency: quota-nls = 1:4.01-19.el7 for package: 1:quota-4.01-19.el7.x86_64 --> Processing Dependency: tcp_wrappers for package: 1:quota-4.01-19.el7.x86_64 ---> Package rpcbind.x86_64 0:0.2.0-48.el7 will be installed --> Running transaction check ---> Package libbasicobjects.x86_64 0:0.1.1-32.el7 will be installed ---> Package libcollection.x86_64 0:0.7.0-32.el7 will be installed ---> Package libini_config.x86_64 0:1.3.1-32.el7 will be installed --> Processing Dependency: libpath_utils.so.1(PATH_UTILS_0.2.1)(64bit) for package: libini_config-1.3.1-32.el7.x86_64 --> Processing Dependency: libpath_utils.so.1()(64bit) for package: libini_config-1.3.1-32.el7.x86_64 ---> Package libref_array.x86_64 0:0.1.5-32.el7 will be installed ---> Package libverto-libevent.x86_64 0:0.2.5-4.el7 will be installed ---> Package quota-nls.noarch 1:4.01-19.el7 will be installed ---> Package tcp_wrappers.x86_64 0:7.6-77.el7 will be installed --> Running transaction check ---> Package libpath_utils.x86_64 0:0.2.1-32.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ============================================================================================================================================================================================================================================================================== Package Arch Version Repository Size ============================================================================================================================================================================================================================================================================== Installing: nfs-utils x86_64 1:1.3.0-0.65.el7 base 412 k Installing for dependencies: gssproxy x86_64 0.7.0-26.el7 base 110 k keyutils x86_64 1.5.8-3.el7 base 54 k libbasicobjects x86_64 0.1.1-32.el7 base 26 k libcollection x86_64 0.7.0-32.el7 base 42 k libini_config x86_64 1.3.1-32.el7 base 64 k libnfsidmap x86_64 0.25-19.el7 base 50 k libpath_utils x86_64 0.2.1-32.el7 base 28 k libref_array x86_64 0.1.5-32.el7 base 27 k libtirpc x86_64 0.2.4-0.16.el7 base 89 k libverto-libevent x86_64 0.2.5-4.el7 base 8.9 k quota x86_64 1:4.01-19.el7 base 179 k quota-nls noarch 1:4.01-19.el7 base 90 k rpcbind x86_64 0.2.0-48.el7 base 60 k tcp_wrappers x86_64 7.6-77.el7 base 78 k Transaction Summary ============================================================================================================================================================================================================================================================================== Install 1 Package (+14 Dependent packages) Total download size: 1.3 M Installed size: 3.6 M Downloading packages: (1/15): keyutils-1.5.8-3.el7.x86_64.rpm | 54 kB 00:00:00 (2/15): gssproxy-0.7.0-26.el7.x86_64.rpm | 110 kB 00:00:00 (3/15): libini_config-1.3.1-32.el7.x86_64.rpm | 64 kB 00:00:00 (4/15): libbasicobjects-0.1.1-32.el7.x86_64.rpm | 26 kB 00:00:00 (5/15): libpath_utils-0.2.1-32.el7.x86_64.rpm | 28 kB 00:00:00 (6/15): libcollection-0.7.0-32.el7.x86_64.rpm | 42 kB 00:00:00 (7/15): libref_array-0.1.5-32.el7.x86_64.rpm | 27 kB 00:00:00 (8/15): libtirpc-0.2.4-0.16.el7.x86_64.rpm | 89 kB 00:00:00 (9/15): libverto-libevent-0.2.5-4.el7.x86_64.rpm | 8.9 kB 00:00:00 (10/15): quota-4.01-19.el7.x86_64.rpm | 179 kB 00:00:00 (11/15): libnfsidmap-0.25-19.el7.x86_64.rpm | 50 kB 00:00:00 (12/15): rpcbind-0.2.0-48.el7.x86_64.rpm | 60 kB 00:00:00 (13/15): nfs-utils-1.3.0-0.65.el7.x86_64.rpm | 412 kB 00:00:00 (14/15): tcp_wrappers-7.6-77.el7.x86_64.rpm | 78 kB 00:00:00 (15/15): quota-nls-4.01-19.el7.noarch.rpm | 90 kB 00:00:00 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Total 1.4 MB/s | 1.3 MB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : libbasicobjects-0.1.1-32.el7.x86_64 1/15 Installing : libref_array-0.1.5-32.el7.x86_64 2/15 Installing : libcollection-0.7.0-32.el7.x86_64 3/15 Installing : libtirpc-0.2.4-0.16.el7.x86_64 4/15 Installing : rpcbind-0.2.0-48.el7.x86_64 5/15 Installing : 1:quota-nls-4.01-19.el7.noarch 6/15 Installing : tcp_wrappers-7.6-77.el7.x86_64 7/15 Installing : 1:quota-4.01-19.el7.x86_64 8/15 Installing : keyutils-1.5.8-3.el7.x86_64 9/15 Installing : libnfsidmap-0.25-19.el7.x86_64 10/15 Installing : libpath_utils-0.2.1-32.el7.x86_64 11/15 Installing : libini_config-1.3.1-32.el7.x86_64 12/15 Installing : libverto-libevent-0.2.5-4.el7.x86_64 13/15 Installing : gssproxy-0.7.0-26.el7.x86_64 14/15 Installing : 1:nfs-utils-1.3.0-0.65.el7.x86_64 15/15 Verifying : libtirpc-0.2.4-0.16.el7.x86_64 1/15 Verifying : libverto-libevent-0.2.5-4.el7.x86_64 2/15 Verifying : 1:quota-4.01-19.el7.x86_64 3/15 Verifying : gssproxy-0.7.0-26.el7.x86_64 4/15 Verifying : libpath_utils-0.2.1-32.el7.x86_64 5/15 Verifying : libnfsidmap-0.25-19.el7.x86_64 6/15 Verifying : keyutils-1.5.8-3.el7.x86_64 7/15 Verifying : 1:nfs-utils-1.3.0-0.65.el7.x86_64 8/15 Verifying : tcp_wrappers-7.6-77.el7.x86_64 9/15 Verifying : libcollection-0.7.0-32.el7.x86_64 10/15 Verifying : libref_array-0.1.5-32.el7.x86_64 11/15 Verifying : libbasicobjects-0.1.1-32.el7.x86_64 12/15 Verifying : rpcbind-0.2.0-48.el7.x86_64 13/15 Verifying : libini_config-1.3.1-32.el7.x86_64 14/15 Verifying : 1:quota-nls-4.01-19.el7.noarch 15/15 Installed: nfs-utils.x86_64 1:1.3.0-0.65.el7 Dependency Installed: gssproxy.x86_64 0:0.7.0-26.el7 keyutils.x86_64 0:1.5.8-3.el7 libbasicobjects.x86_64 0:0.1.1-32.el7 libcollection.x86_64 0:0.7.0-32.el7 libini_config.x86_64 0:1.3.1-32.el7 libnfsidmap.x86_64 0:0.25-19.el7 libpath_utils.x86_64 0:0.2.1-32.el7 libref_array.x86_64 0:0.1.5-32.el7 libtirpc.x86_64 0:0.2.4-0.16.el7 libverto-libevent.x86_64 0:0.2.5-4.el7 quota.x86_64 1:4.01-19.el7 quota-nls.noarch 1:4.01-19.el7 rpcbind.x86_64 0:0.2.0-48.el7 tcp_wrappers.x86_64 0:7.6-77.el7 Complete! [root@master200.yinzhengjie.org.cn ~]#
[root@node201.yinzhengjie.org.cn ~]# yum -y install nfs-utils Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.aliyun.com * updates: mirror.bit.edu.cn base | 3.6 kB 00:00:00 docker-ce-stable | 3.5 kB 00:00:00 extras | 2.9 kB 00:00:00 kubernetes | 1.4 kB 00:00:00 updates | 2.9 kB 00:00:00 updates/7/x86_64/primary_db | 6.7 MB 00:00:00 Resolving Dependencies --> Running transaction check ---> Package nfs-utils.x86_64 1:1.3.0-0.65.el7 will be installed --> Processing Dependency: libtirpc >= 0.2.4-0.7 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: gssproxy >= 0.7.0-3 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: rpcbind for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: quota for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libnfsidmap for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: keyutils for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libtirpc.so.1()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libnfsidmap.so.0()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Running transaction check ---> Package gssproxy.x86_64 0:0.7.0-26.el7 will be installed --> Processing Dependency: libini_config >= 1.3.1-31 for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libverto-module-base for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libref_array.so.1(REF_ARRAY_0.1.1)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.2.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.1.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libref_array.so.1()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libcollection.so.2()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libbasicobjects.so.0()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 ---> Package keyutils.x86_64 0:1.5.8-3.el7 will be installed ---> Package libnfsidmap.x86_64 0:0.25-19.el7 will be installed ---> Package libtirpc.x86_64 0:0.2.4-0.16.el7 will be installed ---> Package quota.x86_64 1:4.01-19.el7 will be installed --> Processing Dependency: quota-nls = 1:4.01-19.el7 for package: 1:quota-4.01-19.el7.x86_64 --> Processing Dependency: tcp_wrappers for package: 1:quota-4.01-19.el7.x86_64 ---> Package rpcbind.x86_64 0:0.2.0-48.el7 will be installed --> Running transaction check ---> Package libbasicobjects.x86_64 0:0.1.1-32.el7 will be installed ---> Package libcollection.x86_64 0:0.7.0-32.el7 will be installed ---> Package libini_config.x86_64 0:1.3.1-32.el7 will be installed --> Processing Dependency: libpath_utils.so.1(PATH_UTILS_0.2.1)(64bit) for package: libini_config-1.3.1-32.el7.x86_64 --> Processing Dependency: libpath_utils.so.1()(64bit) for package: libini_config-1.3.1-32.el7.x86_64 ---> Package libref_array.x86_64 0:0.1.5-32.el7 will be installed ---> Package libverto-libevent.x86_64 0:0.2.5-4.el7 will be installed ---> Package quota-nls.noarch 1:4.01-19.el7 will be installed ---> Package tcp_wrappers.x86_64 0:7.6-77.el7 will be installed --> Running transaction check ---> Package libpath_utils.x86_64 0:0.2.1-32.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ============================================================================================================================================================================================================================================================================== Package Arch Version Repository Size ============================================================================================================================================================================================================================================================================== Installing: nfs-utils x86_64 1:1.3.0-0.65.el7 base 412 k Installing for dependencies: gssproxy x86_64 0.7.0-26.el7 base 110 k keyutils x86_64 1.5.8-3.el7 base 54 k libbasicobjects x86_64 0.1.1-32.el7 base 26 k libcollection x86_64 0.7.0-32.el7 base 42 k libini_config x86_64 1.3.1-32.el7 base 64 k libnfsidmap x86_64 0.25-19.el7 base 50 k libpath_utils x86_64 0.2.1-32.el7 base 28 k libref_array x86_64 0.1.5-32.el7 base 27 k libtirpc x86_64 0.2.4-0.16.el7 base 89 k libverto-libevent x86_64 0.2.5-4.el7 base 8.9 k quota x86_64 1:4.01-19.el7 base 179 k quota-nls noarch 1:4.01-19.el7 base 90 k rpcbind x86_64 0.2.0-48.el7 base 60 k tcp_wrappers x86_64 7.6-77.el7 base 78 k Transaction Summary ============================================================================================================================================================================================================================================================================== Install 1 Package (+14 Dependent packages) Total download size: 1.3 M Installed size: 3.6 M Downloading packages: (1/15): gssproxy-0.7.0-26.el7.x86_64.rpm | 110 kB 00:00:00 (2/15): keyutils-1.5.8-3.el7.x86_64.rpm | 54 kB 00:00:00 (3/15): libcollection-0.7.0-32.el7.x86_64.rpm | 42 kB 00:00:00 (4/15): libnfsidmap-0.25-19.el7.x86_64.rpm | 50 kB 00:00:00 (5/15): libbasicobjects-0.1.1-32.el7.x86_64.rpm | 26 kB 00:00:00 (6/15): libini_config-1.3.1-32.el7.x86_64.rpm | 64 kB 00:00:00 (7/15): libtirpc-0.2.4-0.16.el7.x86_64.rpm | 89 kB 00:00:00 (8/15): libref_array-0.1.5-32.el7.x86_64.rpm | 27 kB 00:00:00 (9/15): libverto-libevent-0.2.5-4.el7.x86_64.rpm | 8.9 kB 00:00:00 (10/15): libpath_utils-0.2.1-32.el7.x86_64.rpm | 28 kB 00:00:00 (11/15): nfs-utils-1.3.0-0.65.el7.x86_64.rpm | 412 kB 00:00:00 (12/15): quota-4.01-19.el7.x86_64.rpm | 179 kB 00:00:00 (13/15): quota-nls-4.01-19.el7.noarch.rpm | 90 kB 00:00:00 (14/15): tcp_wrappers-7.6-77.el7.x86_64.rpm | 78 kB 00:00:00 (15/15): rpcbind-0.2.0-48.el7.x86_64.rpm | 60 kB 00:00:00 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Total 2.2 MB/s | 1.3 MB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : libbasicobjects-0.1.1-32.el7.x86_64 1/15 Installing : libref_array-0.1.5-32.el7.x86_64 2/15 Installing : libcollection-0.7.0-32.el7.x86_64 3/15 Installing : libtirpc-0.2.4-0.16.el7.x86_64 4/15 Installing : rpcbind-0.2.0-48.el7.x86_64 5/15 Installing : 1:quota-nls-4.01-19.el7.noarch 6/15 Installing : tcp_wrappers-7.6-77.el7.x86_64 7/15 Installing : 1:quota-4.01-19.el7.x86_64 8/15 Installing : keyutils-1.5.8-3.el7.x86_64 9/15 Installing : libnfsidmap-0.25-19.el7.x86_64 10/15 Installing : libpath_utils-0.2.1-32.el7.x86_64 11/15 Installing : libini_config-1.3.1-32.el7.x86_64 12/15 Installing : libverto-libevent-0.2.5-4.el7.x86_64 13/15 Installing : gssproxy-0.7.0-26.el7.x86_64 14/15 Installing : 1:nfs-utils-1.3.0-0.65.el7.x86_64 15/15 Verifying : libtirpc-0.2.4-0.16.el7.x86_64 1/15 Verifying : libverto-libevent-0.2.5-4.el7.x86_64 2/15 Verifying : 1:quota-4.01-19.el7.x86_64 3/15 Verifying : gssproxy-0.7.0-26.el7.x86_64 4/15 Verifying : libpath_utils-0.2.1-32.el7.x86_64 5/15 Verifying : libnfsidmap-0.25-19.el7.x86_64 6/15 Verifying : keyutils-1.5.8-3.el7.x86_64 7/15 Verifying : 1:nfs-utils-1.3.0-0.65.el7.x86_64 8/15 Verifying : tcp_wrappers-7.6-77.el7.x86_64 9/15 Verifying : libcollection-0.7.0-32.el7.x86_64 10/15 Verifying : libref_array-0.1.5-32.el7.x86_64 11/15 Verifying : libbasicobjects-0.1.1-32.el7.x86_64 12/15 Verifying : rpcbind-0.2.0-48.el7.x86_64 13/15 Verifying : libini_config-1.3.1-32.el7.x86_64 14/15 Verifying : 1:quota-nls-4.01-19.el7.noarch 15/15 Installed: nfs-utils.x86_64 1:1.3.0-0.65.el7 Dependency Installed: gssproxy.x86_64 0:0.7.0-26.el7 keyutils.x86_64 0:1.5.8-3.el7 libbasicobjects.x86_64 0:0.1.1-32.el7 libcollection.x86_64 0:0.7.0-32.el7 libini_config.x86_64 0:1.3.1-32.el7 libnfsidmap.x86_64 0:0.25-19.el7 libpath_utils.x86_64 0:0.2.1-32.el7 libref_array.x86_64 0:0.1.5-32.el7 libtirpc.x86_64 0:0.2.4-0.16.el7 libverto-libevent.x86_64 0:0.2.5-4.el7 quota.x86_64 1:4.01-19.el7 quota-nls.noarch 1:4.01-19.el7 rpcbind.x86_64 0:0.2.0-48.el7 tcp_wrappers.x86_64 0:7.6-77.el7 Complete! [root@node201.yinzhengjie.org.cn ~]#
[root@node202.yinzhengjie.org.cn ~]# yum -y install nfs-utils Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirror.bit.edu.cn * extras: mirror.bit.edu.cn * updates: mirror.bit.edu.cn base | 3.6 kB 00:00:00 docker-ce-stable | 3.5 kB 00:00:00 extras | 2.9 kB 00:00:00 kubernetes | 1.4 kB 00:00:00 updates | 2.9 kB 00:00:00 updates/7/x86_64/primary_db | 6.7 MB 00:00:01 Resolving Dependencies --> Running transaction check ---> Package nfs-utils.x86_64 1:1.3.0-0.65.el7 will be installed --> Processing Dependency: libtirpc >= 0.2.4-0.7 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: gssproxy >= 0.7.0-3 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: rpcbind for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: quota for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libnfsidmap for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: keyutils for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libtirpc.so.1()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libnfsidmap.so.0()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Running transaction check ---> Package gssproxy.x86_64 0:0.7.0-26.el7 will be installed --> Processing Dependency: libini_config >= 1.3.1-31 for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libverto-module-base for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libref_array.so.1(REF_ARRAY_0.1.1)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.2.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.1.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libref_array.so.1()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libcollection.so.2()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libbasicobjects.so.0()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 ---> Package keyutils.x86_64 0:1.5.8-3.el7 will be installed ---> Package libnfsidmap.x86_64 0:0.25-19.el7 will be installed ---> Package libtirpc.x86_64 0:0.2.4-0.16.el7 will be installed ---> Package quota.x86_64 1:4.01-19.el7 will be installed --> Processing Dependency: quota-nls = 1:4.01-19.el7 for package: 1:quota-4.01-19.el7.x86_64 --> Processing Dependency: tcp_wrappers for package: 1:quota-4.01-19.el7.x86_64 ---> Package rpcbind.x86_64 0:0.2.0-48.el7 will be installed --> Running transaction check ---> Package libbasicobjects.x86_64 0:0.1.1-32.el7 will be installed ---> Package libcollection.x86_64 0:0.7.0-32.el7 will be installed ---> Package libini_config.x86_64 0:1.3.1-32.el7 will be installed --> Processing Dependency: libpath_utils.so.1(PATH_UTILS_0.2.1)(64bit) for package: libini_config-1.3.1-32.el7.x86_64 --> Processing Dependency: libpath_utils.so.1()(64bit) for package: libini_config-1.3.1-32.el7.x86_64 ---> Package libref_array.x86_64 0:0.1.5-32.el7 will be installed ---> Package libverto-libevent.x86_64 0:0.2.5-4.el7 will be installed ---> Package quota-nls.noarch 1:4.01-19.el7 will be installed ---> Package tcp_wrappers.x86_64 0:7.6-77.el7 will be installed --> Running transaction check ---> Package libpath_utils.x86_64 0:0.2.1-32.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ============================================================================================================================================================================================================================================================================== Package Arch Version Repository Size ============================================================================================================================================================================================================================================================================== Installing: nfs-utils x86_64 1:1.3.0-0.65.el7 base 412 k Installing for dependencies: gssproxy x86_64 0.7.0-26.el7 base 110 k keyutils x86_64 1.5.8-3.el7 base 54 k libbasicobjects x86_64 0.1.1-32.el7 base 26 k libcollection x86_64 0.7.0-32.el7 base 42 k libini_config x86_64 1.3.1-32.el7 base 64 k libnfsidmap x86_64 0.25-19.el7 base 50 k libpath_utils x86_64 0.2.1-32.el7 base 28 k libref_array x86_64 0.1.5-32.el7 base 27 k libtirpc x86_64 0.2.4-0.16.el7 base 89 k libverto-libevent x86_64 0.2.5-4.el7 base 8.9 k quota x86_64 1:4.01-19.el7 base 179 k quota-nls noarch 1:4.01-19.el7 base 90 k rpcbind x86_64 0.2.0-48.el7 base 60 k tcp_wrappers x86_64 7.6-77.el7 base 78 k Transaction Summary ============================================================================================================================================================================================================================================================================== Install 1 Package (+14 Dependent packages) Total download size: 1.3 M Installed size: 3.6 M Downloading packages: (1/15): keyutils-1.5.8-3.el7.x86_64.rpm | 54 kB 00:00:00 (2/15): libbasicobjects-0.1.1-32.el7.x86_64.rpm | 26 kB 00:00:00 (3/15): libnfsidmap-0.25-19.el7.x86_64.rpm | 50 kB 00:00:00 (4/15): libini_config-1.3.1-32.el7.x86_64.rpm | 64 kB 00:00:00 (5/15): libref_array-0.1.5-32.el7.x86_64.rpm | 27 kB 00:00:00 (6/15): libpath_utils-0.2.1-32.el7.x86_64.rpm | 28 kB 00:00:00 (7/15): libverto-libevent-0.2.5-4.el7.x86_64.rpm | 8.9 kB 00:00:00 (8/15): gssproxy-0.7.0-26.el7.x86_64.rpm | 110 kB 00:00:00 (9/15): libcollection-0.7.0-32.el7.x86_64.rpm | 42 kB 00:00:00 (10/15): nfs-utils-1.3.0-0.65.el7.x86_64.rpm | 412 kB 00:00:00 (11/15): libtirpc-0.2.4-0.16.el7.x86_64.rpm | 89 kB 00:00:00 (12/15): quota-4.01-19.el7.x86_64.rpm | 179 kB 00:00:00 (13/15): tcp_wrappers-7.6-77.el7.x86_64.rpm | 78 kB 00:00:00 (14/15): quota-nls-4.01-19.el7.noarch.rpm | 90 kB 00:00:00 (15/15): rpcbind-0.2.0-48.el7.x86_64.rpm | 60 kB 00:00:00 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Total 1.9 MB/s | 1.3 MB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : libbasicobjects-0.1.1-32.el7.x86_64 1/15 Installing : libref_array-0.1.5-32.el7.x86_64 2/15 Installing : libcollection-0.7.0-32.el7.x86_64 3/15 Installing : libtirpc-0.2.4-0.16.el7.x86_64 4/15 Installing : rpcbind-0.2.0-48.el7.x86_64 5/15 Installing : 1:quota-nls-4.01-19.el7.noarch 6/15 Installing : tcp_wrappers-7.6-77.el7.x86_64 7/15 Installing : 1:quota-4.01-19.el7.x86_64 8/15 Installing : keyutils-1.5.8-3.el7.x86_64 9/15 Installing : libnfsidmap-0.25-19.el7.x86_64 10/15 Installing : libpath_utils-0.2.1-32.el7.x86_64 11/15 Installing : libini_config-1.3.1-32.el7.x86_64 12/15 Installing : libverto-libevent-0.2.5-4.el7.x86_64 13/15 Installing : gssproxy-0.7.0-26.el7.x86_64 14/15 Installing : 1:nfs-utils-1.3.0-0.65.el7.x86_64 15/15 Verifying : libtirpc-0.2.4-0.16.el7.x86_64 1/15 Verifying : libverto-libevent-0.2.5-4.el7.x86_64 2/15 Verifying : 1:quota-4.01-19.el7.x86_64 3/15 Verifying : gssproxy-0.7.0-26.el7.x86_64 4/15 Verifying : libpath_utils-0.2.1-32.el7.x86_64 5/15 Verifying : libnfsidmap-0.25-19.el7.x86_64 6/15 Verifying : keyutils-1.5.8-3.el7.x86_64 7/15 Verifying : 1:nfs-utils-1.3.0-0.65.el7.x86_64 8/15 Verifying : tcp_wrappers-7.6-77.el7.x86_64 9/15 Verifying : libcollection-0.7.0-32.el7.x86_64 10/15 Verifying : libref_array-0.1.5-32.el7.x86_64 11/15 Verifying : libbasicobjects-0.1.1-32.el7.x86_64 12/15 Verifying : rpcbind-0.2.0-48.el7.x86_64 13/15 Verifying : libini_config-1.3.1-32.el7.x86_64 14/15 Verifying : 1:quota-nls-4.01-19.el7.noarch 15/15 Installed: nfs-utils.x86_64 1:1.3.0-0.65.el7 Dependency Installed: gssproxy.x86_64 0:0.7.0-26.el7 keyutils.x86_64 0:1.5.8-3.el7 libbasicobjects.x86_64 0:0.1.1-32.el7 libcollection.x86_64 0:0.7.0-32.el7 libini_config.x86_64 0:1.3.1-32.el7 libnfsidmap.x86_64 0:0.25-19.el7 libpath_utils.x86_64 0:0.2.1-32.el7 libref_array.x86_64 0:0.1.5-32.el7 libtirpc.x86_64 0:0.2.4-0.16.el7 libverto-libevent.x86_64 0:0.2.5-4.el7 quota.x86_64 1:4.01-19.el7 quota-nls.noarch 1:4.01-19.el7 rpcbind.x86_64 0:0.2.0-48.el7 tcp_wrappers.x86_64 0:7.6-77.el7 Complete! [root@node202.yinzhengjie.org.cn ~]#
[root@node203.yinzhengjie.org.cn ~]# yum -y install nfs-utils Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.aliyun.com * updates: mirror.bit.edu.cn base | 3.6 kB 00:00:00 docker-ce-stable | 3.5 kB 00:00:00 extras | 2.9 kB 00:00:00 kubernetes | 1.4 kB 00:00:00 updates | 2.9 kB 00:00:00 updates/7/x86_64/primary_db | 6.7 MB 00:00:00 Resolving Dependencies --> Running transaction check ---> Package nfs-utils.x86_64 1:1.3.0-0.65.el7 will be installed --> Processing Dependency: libtirpc >= 0.2.4-0.7 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: gssproxy >= 0.7.0-3 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: rpcbind for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: quota for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libnfsidmap for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: keyutils for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libtirpc.so.1()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libnfsidmap.so.0()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Running transaction check ---> Package gssproxy.x86_64 0:0.7.0-26.el7 will be installed --> Processing Dependency: libini_config >= 1.3.1-31 for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libverto-module-base for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libref_array.so.1(REF_ARRAY_0.1.1)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.2.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.1.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libref_array.so.1()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libcollection.so.2()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libbasicobjects.so.0()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 ---> Package keyutils.x86_64 0:1.5.8-3.el7 will be installed ---> Package libnfsidmap.x86_64 0:0.25-19.el7 will be installed ---> Package libtirpc.x86_64 0:0.2.4-0.16.el7 will be installed ---> Package quota.x86_64 1:4.01-19.el7 will be installed --> Processing Dependency: quota-nls = 1:4.01-19.el7 for package: 1:quota-4.01-19.el7.x86_64 --> Processing Dependency: tcp_wrappers for package: 1:quota-4.01-19.el7.x86_64 ---> Package rpcbind.x86_64 0:0.2.0-48.el7 will be installed --> Running transaction check ---> Package libbasicobjects.x86_64 0:0.1.1-32.el7 will be installed ---> Package libcollection.x86_64 0:0.7.0-32.el7 will be installed ---> Package libini_config.x86_64 0:1.3.1-32.el7 will be installed --> Processing Dependency: libpath_utils.so.1(PATH_UTILS_0.2.1)(64bit) for package: libini_config-1.3.1-32.el7.x86_64 --> Processing Dependency: libpath_utils.so.1()(64bit) for package: libini_config-1.3.1-32.el7.x86_64 ---> Package libref_array.x86_64 0:0.1.5-32.el7 will be installed ---> Package libverto-libevent.x86_64 0:0.2.5-4.el7 will be installed ---> Package quota-nls.noarch 1:4.01-19.el7 will be installed ---> Package tcp_wrappers.x86_64 0:7.6-77.el7 will be installed --> Running transaction check ---> Package libpath_utils.x86_64 0:0.2.1-32.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ============================================================================================================================================================================================================================================================================== Package Arch Version Repository Size ============================================================================================================================================================================================================================================================================== Installing: nfs-utils x86_64 1:1.3.0-0.65.el7 base 412 k Installing for dependencies: gssproxy x86_64 0.7.0-26.el7 base 110 k keyutils x86_64 1.5.8-3.el7 base 54 k libbasicobjects x86_64 0.1.1-32.el7 base 26 k libcollection x86_64 0.7.0-32.el7 base 42 k libini_config x86_64 1.3.1-32.el7 base 64 k libnfsidmap x86_64 0.25-19.el7 base 50 k libpath_utils x86_64 0.2.1-32.el7 base 28 k libref_array x86_64 0.1.5-32.el7 base 27 k libtirpc x86_64 0.2.4-0.16.el7 base 89 k libverto-libevent x86_64 0.2.5-4.el7 base 8.9 k quota x86_64 1:4.01-19.el7 base 179 k quota-nls noarch 1:4.01-19.el7 base 90 k rpcbind x86_64 0.2.0-48.el7 base 60 k tcp_wrappers x86_64 7.6-77.el7 base 78 k Transaction Summary ============================================================================================================================================================================================================================================================================== Install 1 Package (+14 Dependent packages) Total download size: 1.3 M Installed size: 3.6 M Downloading packages: (1/15): libcollection-0.7.0-32.el7.x86_64.rpm | 42 kB 00:00:00 (2/15): gssproxy-0.7.0-26.el7.x86_64.rpm | 110 kB 00:00:00 (3/15): libpath_utils-0.2.1-32.el7.x86_64.rpm | 28 kB 00:00:00 (4/15): libref_array-0.1.5-32.el7.x86_64.rpm | 27 kB 00:00:00 (5/15): keyutils-1.5.8-3.el7.x86_64.rpm | 54 kB 00:00:00 (6/15): libnfsidmap-0.25-19.el7.x86_64.rpm | 50 kB 00:00:00 (7/15): libini_config-1.3.1-32.el7.x86_64.rpm | 64 kB 00:00:00 (8/15): libtirpc-0.2.4-0.16.el7.x86_64.rpm | 89 kB 00:00:00 (9/15): quota-nls-4.01-19.el7.noarch.rpm | 90 kB 00:00:00 (10/15): quota-4.01-19.el7.x86_64.rpm | 179 kB 00:00:00 (11/15): libbasicobjects-0.1.1-32.el7.x86_64.rpm | 26 kB 00:00:00 (12/15): libverto-libevent-0.2.5-4.el7.x86_64.rpm | 8.9 kB 00:00:00 (13/15): rpcbind-0.2.0-48.el7.x86_64.rpm | 60 kB 00:00:00 (14/15): tcp_wrappers-7.6-77.el7.x86_64.rpm | 78 kB 00:00:00 (15/15): nfs-utils-1.3.0-0.65.el7.x86_64.rpm | 412 kB 00:00:00 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Total 2.6 MB/s | 1.3 MB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : libbasicobjects-0.1.1-32.el7.x86_64 1/15 Installing : libref_array-0.1.5-32.el7.x86_64 2/15 Installing : libcollection-0.7.0-32.el7.x86_64 3/15 Installing : libtirpc-0.2.4-0.16.el7.x86_64 4/15 Installing : rpcbind-0.2.0-48.el7.x86_64 5/15 Installing : 1:quota-nls-4.01-19.el7.noarch 6/15 Installing : tcp_wrappers-7.6-77.el7.x86_64 7/15 Installing : 1:quota-4.01-19.el7.x86_64 8/15 Installing : keyutils-1.5.8-3.el7.x86_64 9/15 Installing : libnfsidmap-0.25-19.el7.x86_64 10/15 Installing : libpath_utils-0.2.1-32.el7.x86_64 11/15 Installing : libini_config-1.3.1-32.el7.x86_64 12/15 Installing : libverto-libevent-0.2.5-4.el7.x86_64 13/15 Installing : gssproxy-0.7.0-26.el7.x86_64 14/15 Installing : 1:nfs-utils-1.3.0-0.65.el7.x86_64 15/15 Verifying : libtirpc-0.2.4-0.16.el7.x86_64 1/15 Verifying : libverto-libevent-0.2.5-4.el7.x86_64 2/15 Verifying : 1:quota-4.01-19.el7.x86_64 3/15 Verifying : gssproxy-0.7.0-26.el7.x86_64 4/15 Verifying : libpath_utils-0.2.1-32.el7.x86_64 5/15 Verifying : libnfsidmap-0.25-19.el7.x86_64 6/15 Verifying : keyutils-1.5.8-3.el7.x86_64 7/15 Verifying : 1:nfs-utils-1.3.0-0.65.el7.x86_64 8/15 Verifying : tcp_wrappers-7.6-77.el7.x86_64 9/15 Verifying : libcollection-0.7.0-32.el7.x86_64 10/15 Verifying : libref_array-0.1.5-32.el7.x86_64 11/15 Verifying : libbasicobjects-0.1.1-32.el7.x86_64 12/15 Verifying : rpcbind-0.2.0-48.el7.x86_64 13/15 Verifying : libini_config-1.3.1-32.el7.x86_64 14/15 Verifying : 1:quota-nls-4.01-19.el7.noarch 15/15 Installed: nfs-utils.x86_64 1:1.3.0-0.65.el7 Dependency Installed: gssproxy.x86_64 0:0.7.0-26.el7 keyutils.x86_64 0:1.5.8-3.el7 libbasicobjects.x86_64 0:0.1.1-32.el7 libcollection.x86_64 0:0.7.0-32.el7 libini_config.x86_64 0:1.3.1-32.el7 libnfsidmap.x86_64 0:0.25-19.el7 libpath_utils.x86_64 0:0.2.1-32.el7 libref_array.x86_64 0:0.1.5-32.el7 libtirpc.x86_64 0:0.2.4-0.16.el7 libverto-libevent.x86_64 0:0.2.5-4.el7 quota.x86_64 1:4.01-19.el7 quota-nls.noarch 1:4.01-19.el7 rpcbind.x86_64 0:0.2.0-48.el7 tcp_wrappers.x86_64 0:7.6-77.el7 Complete! [root@node203.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# mount -t nfs 172.200.1.110:/yinzhengjie/data/volume1 /mnt #在K8S node節點測試NFS服務是否可以成功掛載 [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# mount | grep mnt 172.200.1.110:/yinzhengjie/data/volume1 on /mnt type nfs4 (rw,relatime,vers=4.1,rsize=524288,wsize=524288,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=172.200.1.200,local_lock=none,addr=172.200.1.110) [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# df -h | grep mnt 172.200.1.110:/yinzhengjie/data/volume1 1.6T 416M 1.6T 1% /mnt [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
3>.創建nfs對應的pv
[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/statefulset/pv-nfs-demo.yaml [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/statefulset/pv-nfs-demo.yaml apiVersion: v1 kind: PersistentVolume metadata: name: pv-nfs-v3 labels: storsys: nfs spec: accessModes: ["ReadWriteOnce","ReadOnlyMany","ReadWriteMany"] capacity: storage: 5Gi volumeMode: Filesystem persistentVolumeReclaimPolicy: Retain nfs: server: 172.200.1.110 path: /yinzhengjie/data/volume3 --- apiVersion: v1 kind: PersistentVolume metadata: name: pv-nfs-v4 labels: storsys: nfs spec: accessModes: ["ReadWriteOnce","ReadOnlyMany","ReadWriteMany"] capacity: storage: 5Gi volumeMode: Filesystem persistentVolumeReclaimPolicy: Retain nfs: server: 172.200.1.110 path: /yinzhengjie/data/volume4 --- apiVersion: v1 kind: PersistentVolume metadata: name: pv-nfs-v5 labels: storsys: nfs spec: accessModes: ["ReadWriteOnce","ReadOnlyMany","ReadWriteMany"] capacity: storage: 5Gi volumeMode: Filesystem persistentVolumeReclaimPolicy: Retain nfs: server: 172.200.1.110 path: /yinzhengjie/data/volume5 [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/statefulset/pv-nfs-demo.yaml persistentvolume/pv-nfs-v3 created persistentvolume/pv-nfs-v4 created persistentvolume/pv-nfs-v5 created [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pv-nfs-v3 5Gi RWO,ROX,RWX Retain Available 12s pv-nfs-v4 5Gi RWO,ROX,RWX Retain Available 11s pv-nfs-v5 5Gi RWO,ROX,RWX Retain Available 11s [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get pv -o wide NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE VOLUMEMODE pv-nfs-v3 5Gi RWO,ROX,RWX Retain Available 16s Filesystem pv-nfs-v4 5Gi RWO,ROX,RWX Retain Available 15s Filesystem pv-nfs-v5 5Gi RWO,ROX,RWX Retain Available 15s Filesystem [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
4>.基於statefulset部署Pod
[root@master200.yinzhengjie.org.cn ~]# kubectl get pv -o wide NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE VOLUMEMODE pv-nfs-v3 5Gi RWO,ROX,RWX Retain Available 16s Filesystem pv-nfs-v4 5Gi RWO,ROX,RWX Retain Available 15s Filesystem pv-nfs-v5 5Gi RWO,ROX,RWX Retain Available 15s Filesystem [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/statefulset/statefulset-demo.yaml [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/statefulset/statefulset-demo.yaml apiVersion: v1 kind: Namespace metadata: name: yinzhengjie-sts --- apiVersion: v1 kind: Service metadata: name: myapp-sts-svc namespace: yinzhengjie-sts labels: app: myapp controller: mystatefulset spec: ports: - port: 80 name: web clusterIP: None selector: app: myapp-pod --- apiVersion: apps/v1 kind: StatefulSet metadata: name: statefulset-demo namespace: yinzhengjie-sts spec: selector: matchLabels: app: myapp-pod controller: mystatefulset serviceName: "myapp-sts-svc" replicas: 2 template: metadata: namespace: yinzhengjie-sts labels: app: myapp-pod controller: mystatefulset spec: terminationGracePeriodSeconds: 10 containers: - name: myapp image: ikubernetes/myapp:v1 ports: - containerPort: 80 name: web volumeMounts: - name: myapp-pvc mountPath: /usr/share/nginx/html volumeClaimTemplates: - metadata: name: myapp-pvc namespace: yinzhengjie-sts spec: accessModes: [ "ReadWriteOnce" ] resources: requests: storage: 2Gi [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/statefulset/statefulset-demo.yaml namespace/yinzhengjie-sts created service/myapp-sts-svc created statefulset.apps/statefulset-demo created [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get all -n yinzhengjie-sts NAME READY STATUS RESTARTS AGE pod/statefulset-demo-0 1/1 Running 0 56s pod/statefulset-demo-1 1/1 Running 0 55s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/myapp-sts-svc ClusterIP None <none> 80/TCP 56s NAME READY AGE statefulset.apps/statefulset-demo 2/2 56s [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get pvc -n yinzhengjie-sts NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE myapp-pvc-statefulset-demo-0 Bound pv-nfs-v5 5Gi RWO,ROX,RWX 73s myapp-pvc-statefulset-demo-1 Bound pv-nfs-v3 5Gi RWO,ROX,RWX 72s [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get pv -o wide NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE VOLUMEMODE pv-nfs-v3 5Gi RWO,ROX,RWX Retain Bound yinzhengjie-sts/myapp-pvc-statefulset-demo-1 6m34s Filesystem pv-nfs-v4 5Gi RWO,ROX,RWX Retain Available 6m33s Filesystem pv-nfs-v5 5Gi RWO,ROX,RWX Retain Bound yinzhengjie-sts/myapp-pvc-statefulset-demo-0 6m33s Filesystem [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get svc -n yinzhengjie-sts NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE myapp-sts-svc ClusterIP None <none> 80/TCP 4m40s [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n yinzhengjie-sts -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES statefulset-demo-0 1/1 Running 0 5m21s 10.244.1.35 node201.yinzhengjie.org.cn <none> <none> statefulset-demo-1 1/1 Running 0 5m20s 10.244.3.25 node203.yinzhengjie.org.cn <none> <none> [root@master200.yinzhengjie.org.cn ~]#
5>.擴容Pod的副本數
[root@master200.yinzhengjie.org.cn ~]# kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pv-nfs-v3 5Gi RWO,ROX,RWX Retain Bound yinzhengjie-sts/myapp-pvc-statefulset-demo-1 18m pv-nfs-v4 5Gi RWO,ROX,RWX Retain Available 18m pv-nfs-v5 5Gi RWO,ROX,RWX Retain Bound yinzhengjie-sts/myapp-pvc-statefulset-demo-0 18m [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get all -o wide -n yinzhengjie-sts NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES pod/statefulset-demo-0 1/1 Running 0 13m 10.244.1.35 node201.yinzhengjie.org.cn <none> <none> pod/statefulset-demo-1 1/1 Running 0 13m 10.244.3.25 node203.yinzhengjie.org.cn <none> <none> NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR service/myapp-sts-svc ClusterIP None <none> 80/TCP 13m app=myapp-pod NAME READY AGE CONTAINERS IMAGES statefulset.apps/statefulset-demo 2/2 13m myapp ikubernetes/myapp:v1 [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl scale sts statefulset-demo --replicas=3 -n yinzhengjie-sts statefulset.apps/statefulset-demo scaled [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pv-nfs-v3 5Gi RWO,ROX,RWX Retain Bound yinzhengjie-sts/myapp-pvc-statefulset-demo-1 19m pv-nfs-v4 5Gi RWO,ROX,RWX Retain Bound yinzhengjie-sts/myapp-pvc-statefulset-demo-2 19m pv-nfs-v5 5Gi RWO,ROX,RWX Retain Bound yinzhengjie-sts/myapp-pvc-statefulset-demo-0 19m [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get all -o wide -n yinzhengjie-sts NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES pod/statefulset-demo-0 1/1 Running 0 15m 10.244.1.35 node201.yinzhengjie.org.cn <none> <none> pod/statefulset-demo-1 1/1 Running 0 14m 10.244.3.25 node203.yinzhengjie.org.cn <none> <none> pod/statefulset-demo-2 1/1 Running 0 14s 10.244.2.30 node202.yinzhengjie.org.cn <none> <none> NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR service/myapp-sts-svc ClusterIP None <none> 80/TCP 15m app=myapp-pod NAME READY AGE CONTAINERS IMAGES statefulset.apps/statefulset-demo 3/3 15m myapp ikubernetes/myapp:v1 [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
6>.升級Pod版本
[root@master200.yinzhengjie.org.cn ~]# kubectl get all -o wide -n yinzhengjie-sts NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES pod/statefulset-demo-0 1/1 Running 0 19m 10.244.1.35 node201.yinzhengjie.org.cn <none> <none> pod/statefulset-demo-1 1/1 Running 0 19m 10.244.3.25 node203.yinzhengjie.org.cn <none> <none> pod/statefulset-demo-2 1/1 Running 0 4m29s 10.244.2.30 node202.yinzhengjie.org.cn <none> <none> NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR service/myapp-sts-svc ClusterIP None <none> 80/TCP 19m app=myapp-pod NAME READY AGE CONTAINERS IMAGES statefulset.apps/statefulset-demo 3/3 19m myapp ikubernetes/myapp:v1 [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl set image sts statefulset-demo myapp=ikubernetes/myapp:v2 -n yinzhengjie-sts statefulset.apps/statefulset-demo image updated [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get all -o wide -n yinzhengjie-sts NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES pod/statefulset-demo-0 1/1 Running 0 69s 10.244.1.36 node201.yinzhengjie.org.cn <none> <none> pod/statefulset-demo-1 1/1 Running 0 90s 10.244.3.26 node203.yinzhengjie.org.cn <none> <none> pod/statefulset-demo-2 1/1 Running 0 103s 10.244.2.31 node202.yinzhengjie.org.cn <none> <none> NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR service/myapp-sts-svc ClusterIP None <none> 80/TCP 22m app=myapp-pod NAME READY AGE CONTAINERS IMAGES statefulset.apps/statefulset-demo 3/3 22m myapp ikubernetes/myapp:v2 [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl describe sts statefulset-demo -n yinzhengjie-sts Name: statefulset-demo Namespace: yinzhengjie-sts CreationTimestamp: Wed, 12 Feb 2020 10:40:46 +0800 Selector: app=myapp-pod,controller=mystatefulset Labels: <none> Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"apps/v1","kind":"StatefulSet","metadata":{"annotations":{},"name":"statefulset-demo","namespace":"yinzhengjie-sts"},"spec":... Replicas: 3 desired | 3 total Update Strategy: RollingUpdate Partition: 824644603528 Pods Status: 3 Running / 0 Waiting / 0 Succeeded / 0 Failed Pod Template: Labels: app=myapp-pod controller=mystatefulset Containers: myapp: Image: ikubernetes/myapp:v2 Port: 80/TCP Host Port: 0/TCP Environment: <none> Mounts: /usr/share/nginx/html from myapp-pvc (rw) Volumes: <none> Volume Claims: Name: myapp-pvc StorageClass: Labels: <none> Annotations: <none> Capacity: 2Gi Access Modes: [ReadWriteOnce] Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal SuccessfulCreate 25m statefulset-controller create Claim myapp-pvc-statefulset-demo-0 Pod statefulset-demo-0 in StatefulSet statefulset-demo success Normal SuccessfulCreate 25m statefulset-controller create Claim myapp-pvc-statefulset-demo-1 Pod statefulset-demo-1 in StatefulSet statefulset-demo success Normal SuccessfulCreate 10m statefulset-controller create Claim myapp-pvc-statefulset-demo-2 Pod statefulset-demo-2 in StatefulSet statefulset-demo success Normal SuccessfulDelete 4m45s statefulset-controller delete Pod statefulset-demo-2 in StatefulSet statefulset-demo successful Normal SuccessfulCreate 4m33s (x2 over 10m) statefulset-controller create Pod statefulset-demo-2 in StatefulSet statefulset-demo successful Normal SuccessfulDelete 4m23s statefulset-controller delete Pod statefulset-demo-1 in StatefulSet statefulset-demo successful Normal SuccessfulCreate 4m20s (x2 over 25m) statefulset-controller create Pod statefulset-demo-1 in StatefulSet statefulset-demo successful Normal SuccessfulDelete 4m10s statefulset-controller delete Pod statefulset-demo-0 in StatefulSet statefulset-demo successful Normal SuccessfulCreate 3m59s (x2 over 25m) statefulset-controller create Pod statefulset-demo-0 in StatefulSet statefulset-demo successful [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
7>.縮容Pod的副本數(縮容時pvc對應pv數據並不會被刪除)
[root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n yinzhengjie-sts NAME READY STATUS RESTARTS AGE statefulset-demo-0 1/1 Running 0 19m statefulset-demo-1 1/1 Running 0 19m statefulset-demo-2 1/1 Running 0 5s [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl exec -it -n yinzhengjie-sts statefulset-demo-2 -- sh / # / # cd /usr/share/nginx/html/ #進入到存儲卷掛載路徑 /usr/share/nginx/html # /usr/share/nginx/html # ls /usr/share/nginx/html # /usr/share/nginx/html # echo "https://www.cnblogs.com/yinzhengjie/" > blog.txt #在存儲卷路徑寫入測試數據。 /usr/share/nginx/html # /usr/share/nginx/html # ls -l total 4 -rw-r--r-- 1 root root 37 Feb 12 2020 blog.txt /usr/share/nginx/html # /usr/share/nginx/html # exit [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n yinzhengjie-sts NAME READY STATUS RESTARTS AGE statefulset-demo-0 1/1 Running 0 19m statefulset-demo-1 1/1 Running 0 19m statefulset-demo-2 1/1 Running 0 5s [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl exec -it -n yinzhengjie-sts statefulset-demo-2 -- sh / # / # cd /usr/share/nginx/html/ /usr/share/nginx/html # /usr/share/nginx/html # ls /usr/share/nginx/html # /usr/share/nginx/html # echo "https://www.cnblogs.com/yinzhengjie/" > blog.txt /usr/share/nginx/html # /usr/share/nginx/html # ls -l total 4 -rw-r--r-- 1 root root 37 Feb 12 2020 blog.txt /usr/share/nginx/html # /usr/share/nginx/html # exit [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl scale sts statefulset-demo --replicas=2 -n yinzhengjie-sts statefulset.apps/statefulset-demo scaled [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n yinzhengjie-sts NAME READY STATUS RESTARTS AGE statefulset-demo-0 1/1 Running 0 24m statefulset-demo-1 1/1 Running 0 24m [root@master200.yinzhengjie.org.cn ~]#
8>.刪除statefulset
[root@master200.yinzhengjie.org.cn ~]# kubectl get sts -n yinzhengjie-sts NAME READY AGE statefulset-demo 3/3 55m [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl delete sts statefulset-demo -n yinzhengjie-sts statefulset.apps "statefulset-demo" deleted [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get sts -n yinzhengjie-sts No resources found in yinzhengjie-sts namespace. [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
三.使用StatefulSet部署etcd服務
StatefulSet(簡稱sts)可以做到簡單的部署，擴容等功能，但是要涉及到比較復雜的功能比如數據備份等功能建議大家使用Operator。
1>.創建etcd的service資源
[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/statefulset/etcd-services.yaml [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/statefulset/etcd-services.yaml apiVersion: v1 kind: Service metadata: name: etcd labels: app: etcd annotations: # Create endpoints also if the related pod isn't ready service.alpha.kubernetes.io/tolerate-unready-endpoints: "true" spec: ports: - port: 2379 name: client - port: 2380 name: peer clusterIP: None selector: app: etcd-member --- apiVersion: v1 kind: Service metadata: name: etcd-client labels: app: etcd spec: ports: - name: etcd-client port: 2379 protocol: TCP targetPort: 2379 selector: app: etcd-member type: NodePort [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/statefulset/etcd-services.yaml service/etcd created service/etcd-client created [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE etcd ClusterIP None <none> 2379/TCP,2380/TCP 14s etcd-client NodePort 10.107.187.160 <none> 2379:32360/TCP 14s kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7d16h [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get svc -o wide NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR etcd ClusterIP None <none> 2379/TCP,2380/TCP 31s app=etcd-member etcd-client NodePort 10.107.187.160 <none> 2379:32360/TCP 31s app=etcd-member kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7d16h <none> [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
2>.確認nfs有足夠的空間及掛載目錄並創建pv
[root@test110.yinzhengjie.org.cn ~]# vim /etc/exports [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# cat /etc/exports /yinzhengjie/data/volume1 172.200.0.0/21(rw,sync,no_root_squash) /yinzhengjie/data/volume2 172.200.0.0/21(rw,sync,no_root_squash) /yinzhengjie/data/volume3 172.200.0.0/21(rw,sync,no_root_squash) /yinzhengjie/data/volume4 172.200.0.0/21(rw,sync,no_root_squash) /yinzhengjie/data/volume5 172.200.0.0/21(rw,sync,no_root_squash) /yinzhengjie/data/volume6 172.200.0.0/21(rw,sync,no_root_squash) /yinzhengjie/data/volume7 172.200.0.0/21(rw,sync,no_root_squash) /yinzhengjie/data/volume8 172.200.0.0/21(rw,sync,no_root_squash) [root@test110.yinzhengjie.org.cn ~]#
[root@test110.yinzhengjie.org.cn ~]# mkdir /yinzhengjie/data/volume{6,7,8} [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# ll /yinzhengjie/data/ total 0 drwxr-xr-x 2 polkitd root 22 Feb 10 06:40 volume1 drwxr-xr-x 2 polkitd root 6 Feb 10 20:04 volume2 drwxr-xr-x 2 root root 6 Feb 10 06:04 volume3 drwxr-xr-x 2 root root 22 Feb 13 03:11 volume4 drwxr-xr-x 2 root root 6 Feb 10 06:04 volume5 drwxr-xr-x 2 root root 6 Feb 13 03:42 volume6 drwxr-xr-x 2 root root 6 Feb 13 03:42 volume7 drwxr-xr-x 2 root root 6 Feb 13 03:42 volume8 [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# exportfs -rav exporting 172.200.0.0/21:/yinzhengjie/data/volume8 exporting 172.200.0.0/21:/yinzhengjie/data/volume7 exporting 172.200.0.0/21:/yinzhengjie/data/volume6 exporting 172.200.0.0/21:/yinzhengjie/data/volume5 exporting 172.200.0.0/21:/yinzhengjie/data/volume4 exporting 172.200.0.0/21:/yinzhengjie/data/volume3 exporting 172.200.0.0/21:/yinzhengjie/data/volume2 exporting 172.200.0.0/21:/yinzhengjie/data/volume1 [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/statefulset/pv-nfs-demo.yaml [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/statefulset/pv-nfs-demo.yaml apiVersion: v1 kind: PersistentVolume metadata: name: pv-nfs-v3 labels: storsys: nfs spec: accessModes: ["ReadWriteOnce","ReadOnlyMany","ReadWriteMany"] capacity: storage: 5Gi volumeMode: Filesystem persistentVolumeReclaimPolicy: Retain nfs: server: 172.200.1.110 path: /yinzhengjie/data/volume3 --- apiVersion: v1 kind: PersistentVolume metadata: name: pv-nfs-v4 labels: storsys: nfs spec: accessModes: ["ReadWriteOnce","ReadOnlyMany","ReadWriteMany"] capacity: storage: 5Gi volumeMode: Filesystem persistentVolumeReclaimPolicy: Retain nfs: server: 172.200.1.110 path: /yinzhengjie/data/volume4 --- apiVersion: v1 kind: PersistentVolume metadata: name: pv-nfs-v5 labels: storsys: nfs spec: accessModes: ["ReadWriteOnce","ReadOnlyMany","ReadWriteMany"] capacity: storage: 5Gi volumeMode: Filesystem persistentVolumeReclaimPolicy: Retain nfs: server: 172.200.1.110 path: /yinzhengjie/data/volume5 --- apiVersion: v1 kind: PersistentVolume metadata: name: pv-nfs-v6 labels: storsys: nfs spec: accessModes: ["ReadWriteOnce","ReadOnlyMany","ReadWriteMany"] capacity: storage: 5Gi volumeMode: Filesystem persistentVolumeReclaimPolicy: Retain nfs: server: 172.200.1.110 path: /yinzhengjie/data/volume6 --- apiVersion: v1 kind: PersistentVolume metadata: name: pv-nfs-v7 labels: storsys: nfs spec: accessModes: ["ReadWriteOnce","ReadOnlyMany","ReadWriteMany"] capacity: storage: 5Gi volumeMode: Filesystem persistentVolumeReclaimPolicy: Retain nfs: server: 172.200.1.110 path: /yinzhengjie/data/volume7 --- apiVersion: v1 kind: PersistentVolume metadata: name: pv-nfs-v8 labels: storsys: nfs spec: accessModes: ["ReadWriteOnce","ReadOnlyMany","ReadWriteMany"] capacity: storage: 5Gi volumeMode: Filesystem persistentVolumeReclaimPolicy: Retain nfs: server: 172.200.1.110 path: /yinzhengjie/data/volume8 [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pv-nfs-v3 5Gi RWO,ROX,RWX Retain Bound yinzhengjie-sts/myapp-pvc-statefulset-demo-1 23m pv-nfs-v4 5Gi RWO,ROX,RWX Retain Bound yinzhengjie-sts/myapp-pvc-statefulset-demo-2 23m pv-nfs-v5 5Gi RWO,ROX,RWX Retain Bound yinzhengjie-sts/myapp-pvc-statefulset-demo-0 23m [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/statefulset/pv-nfs-demo.yaml persistentvolume/pv-nfs-v3 unchanged persistentvolume/pv-nfs-v4 unchanged persistentvolume/pv-nfs-v5 unchanged persistentvolume/pv-nfs-v6 created persistentvolume/pv-nfs-v7 created persistentvolume/pv-nfs-v8 created [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pv-nfs-v3 5Gi RWO,ROX,RWX Retain Bound yinzhengjie-sts/myapp-pvc-statefulset-demo-1 3h52m pv-nfs-v4 5Gi RWO,ROX,RWX Retain Bound yinzhengjie-sts/myapp-pvc-statefulset-demo-2 3h52m pv-nfs-v5 5Gi RWO,ROX,RWX Retain Bound yinzhengjie-sts/myapp-pvc-statefulset-demo-0 3h52m pv-nfs-v6 5Gi RWO,ROX,RWX Retain Available 11s pv-nfs-v7 5Gi RWO,ROX,RWX Retain Available 11s pv-nfs-v8 5Gi RWO,ROX,RWX Retain Available 11s [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
3>.創建etcd的實例StatefulSet配置文件
[root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/statefulset/etcd-statefulset.yaml apiVersion: apps/v1 kind: StatefulSet metadata: name: etcd labels: app: etcd spec: serviceName: etcd # changing replicas value will require a manual etcdctl member remove/add # # command (remove before decreasing and add after increasing) replicas: 3 selector: matchLabels: app: etcd-member template: metadata: name: etcd labels: app: etcd-member spec: containers: - name: etcd image: "quay.io/coreos/etcd:v3.2.16" ports: - containerPort: 2379 name: client - containerPort: 2380 name: peer env: - name: CLUSTER_SIZE value: "3" - name: SET_NAME value: "etcd" volumeMounts: - name: data mountPath: /var/run/etcd command: - "/bin/sh" - "-ecx" - | IP=$(hostname -i) PEERS="" for i in $(seq 0 $((${CLUSTER_SIZE} - 1))); do PEERS="${PEERS}${PEERS:+,}${SET_NAME}-${i}=http://${SET_NAME}-${i}.${SET_NAME}:2380" done # start etcd. If cluster is already initialized the `--initial-*` options will be ignored. exec etcd --name ${HOSTNAME} \ --listen-peer-urls http://${IP}:2380 \ --listen-client-urls http://${IP}:2379,http://127.0.0.1:2379 \ --advertise-client-urls http://${HOSTNAME}.${SET_NAME}:2379 \ --initial-advertise-peer-urls http://${HOSTNAME}.${SET_NAME}:2380 \ --initial-cluster-token etcd-cluster-1 \ --initial-cluster ${PEERS} \ --initial-cluster-state new \ --data-dir /var/run/etcd/default.etcd volumeClaimTemplates: - metadata: name: data spec: accessModes: - "ReadWriteOnce" resources: requests: storage: 1Gi [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
四.使用StatefulSet部署zookeeper服務參考配置文件
[root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/statefulset/zk-sts.yaml apiVersion: v1 kind: Service metadata: name: zk-hs labels: app: zk spec: ports: - port: 2888 name: server - port: 3888 name: leader-election clusterIP: None selector: app: zk --- apiVersion: v1 kind: Service metadata: name: zk-cs labels: app: zk spec: ports: - port: 2181 name: client selector: app: zk --- apiVersion: policy/v1beta1 kind: PodDisruptionBudget metadata: name: zk-pdb spec: selector: matchLabels: app: zk maxUnavailable: 1 --- apiVersion: apps/v1 kind: StatefulSet metadata: name: zk spec: selector: matchLabels: app: zk serviceName: zk-hs replicas: 3 updateStrategy: type: RollingUpdate podManagementPolicy: Parallel template: metadata: labels: app: zk spec: affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: "app" operator: In values: - zk-hs topologyKey: "kubernetes.io/hostname" containers: - name: kubernetes-zookeeper image: gcr.io/google-containers/kubernetes-zookeeper:1.0-3.4.10 resources: requests: memory: "1Gi" cpu: "0.5" ports: - containerPort: 2181 name: client - containerPort: 2888 name: server - containerPort: 3888 name: leader-election command: - sh - -c - "start-zookeeper \ --servers=3 \ --data_dir=/var/lib/zookeeper/data \ --data_log_dir=/var/lib/zookeeper/data/log \ --conf_dir=/opt/zookeeper/conf \ --client_port=2181 \ --election_port=3888 \ --server_port=2888 \ --tick_time=2000 \ --init_limit=10 \ --sync_limit=5 \ --heap=512M \ --max_client_cnxns=60 \ --snap_retain_count=3 \ --purge_interval=12 \ --max_session_timeout=40000 \ --min_session_timeout=4000 \ --log_level=INFO" readinessProbe: exec: command: - sh - -c - "zookeeper-ready 2181" initialDelaySeconds: 10 timeoutSeconds: 5 livenessProbe: exec: command: - sh - -c - "zookeeper-ready 2181" initialDelaySeconds: 10 timeoutSeconds: 5 volumeMounts: - name: data mountPath: /var/lib/zookeeper securityContext: runAsUser: 1000 fsGroup: 1000 volumeClaimTemplates: - metadata: name: data spec: accessModes: [ "ReadWriteOnce" ] storageClassName: gluster-dynamic resources: requests: storage: 5Gi [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
五.基於Operator部署etcd
博主推薦閱讀: https://github.com/coreos/etcd-operator https://github.com/coreos/etcd-operator/blob/master/example/deployment.yaml https://github.com/coreos/etcd-operator/blob/master/example/example-etcd-cluster.yaml
