k8s使用nfs動態存儲

1、Kubernetes集群管理員通過提供不同的存儲類，可以滿足用戶不同的服務質量級別、備份策略和任意策略要求的存儲需求。動態存儲卷供應使用StorageClass進行實現，其允許存儲卷按需被創建。如果沒有動態存儲供應，Kubernetes集群的管理員將不得不通過手工的方式類創建新的存儲卷。通過動態存儲卷，Kubernetes將能夠按照用戶的需要，自動創建其需要的存儲。

 

基於StorageClass的動態存儲供應整體過程如下圖所示：

1）集群管理員預先創建存儲類（StorageClass）；

2）用戶創建使用存儲類的持久化存儲聲明(PVC：PersistentVolumeClaim)；

3）存儲持久化聲明通知系統，它需要一個持久化存儲(PV: PersistentVolume)；

4）系統讀取存儲類的信息；

5）系統基於存儲類的信息，在后台自動創建PVC需要的PV；

6）用戶創建一個使用PVC的Pod；

7）Pod中的應用通過PVC進行數據的持久化；

8）而PVC使用PV進行數據的最終持久化處理。

 

2、example

0) 首先創建nfs服務，參考另外一篇文章：http://www.cnblogs.com/cuishuai/p/7798154.html

1）創建rbac授權，（如果集群開啟了rbac）,非default需要授權

apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: default
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: default
roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner
  apiGroup: rbac.authorization.k8s.io

 

 

2)創建nfs的nfs-client-provisioner

# cat deployment-nfs.yaml

kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: nfs-client-provisioner
spec:
  replicas: 1
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: docker.io/xhuaustc/nfs-client-provisioner:latest
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: fuseim.pri/ifs
            - name: NFS_SERVER
              value: 1.14.0.4  # nfs服務ip
            - name: NFS_PATH
              value: /data/nfs/file # nfs路徑
      volumes:
        - name: nfs-client-root
          nfs:
            server: 1.14.0.4 # nfs服務ip
            path: /data/nfs/file # nfs路徑

 

！！PS：10.10.229.43是nfs服務的監聽地址，/data/opv是nfs共享的目錄。

3)創建storageclass

# cat storageclass-nfs.yaml

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: managed-nfs-storage
  annotations:
    storageclass.kubernetes.io/is-default-class: "true" # 設置該storageclass為PVC創建時默認使用的存儲機制
provisioner: fuseim.pri/ifs # 匹配deployment中的環境變量'PROVISIONER_NAME'
parameters:
  archiveOnDelete: "true" # "false" 刪除PVC時不會保留數據，"true"將保留PVC數據
reclaimPolicy: Delete

 

！！PS：fuseim.pri/ifs為上面deployment上創建的PROVISIONER_NAME。

4)創建PersistenetVolumeClaim

在存儲類被正確創建后，就可以創建PersistenetVolumeClaim來請求StorageClass，而StorageClass將會為PersistenetVolumeClaim自動創建一個可用PersistentVolume。PersistenetVolumeClaim是對PersistenetVolume的聲明，即PersistenetVolume為存儲的提供者，而PersistenetVolumeClaim為存儲的消費者。

# cat pvc.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  annotations:
    volume.beta.kubernetes.io/storage-class: managed-nfs-storage
    volume.beta.kubernetes.io/storage-provisioner: fuseim.pri/ifs
  name: nfs-claim-pvc
spec:
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 1Gi

 

！！PS：nfs-class為上面創建的storageclass的name，10Gi是設置的目錄的大小。

5)執行命令

NAMESPACE=`oc project -q`
sed -i'' "s/namespace:.*/namespace: $NAMESPACE/g" rbac.yaml
oc create -f rbac.yaml
oc adm policy add-scc-to-user hostmount-anyuid system:serviceaccount:$NAMESPACE:nfs-client-provisioner

oc create -f nfs-client-provisioner-deployment.yaml
oc create -f strageclass.yaml
oc create -f nfs-pvc.yaml

# 應用配置

spec:
  containers:
  - name: test-pod
    volumeMounts:
      - name: nfs-claim-pvc
        mountPath: "/mnt"
  volumes:
    - name: nfs-pvc
      persistentVolumeClaim:
        claimName: nfs-claim-pvc