k8s創建StorageClass資源掛載報錯：waiting for a volume to be created, either by external provisioner "wangzy-nfs-storage" or manually created by system administrator

背景：

創建StorageClass存儲類資源時，掛載pvc時一直顯示pending狀態

 

報錯信息：

waiting for a volume to be created, either by external provisioner "wangzy-nfs-storage" or manually created by system administrator

 

翻譯結果：正在等待外部供應器“wangzy nfs storage”或系統管理員手動創建的卷

初步判斷是存儲供應的問題，也有些網上資料說是selfLink導致，因為kubernetes 1.20版本 禁用了 selfLink導致，本次分別從這兩處解決。

 一：設置selfLink參數

 

1.1 添加參數

spec:
  containers:
  - command:
   ...
   ...
    - kube-apiserver
    - --feature-gates=RemoveSelfLink=false　　#手動添加

 

 

1.2 重啟api-server

kubectl apply -f /etc/kubernetes/manifests/kube-apiserver.yaml

 1.3 再次查看，發現 pvc 狀態依舊是pending

於是查看供應鏈是否出問題

二：供應鏈查看

2.1 查看 nfs-client-provisioner，發現報錯

[root@master ~]# kubectl get pods -n dev
NAME                                      READY   STATUS    RESTARTS   AGE
nfs-client-provisioner-68c7ddcdc7-nwx2w   1/1     Running   0          18m
[root@master ~]# kubectl logs -f  nfs-client-provisioner-68c7ddcdc7-nwx2w  -n dev

報錯內容如下：

E0414 06:13:35.631549       1 leaderelection.go:234] error retrieving resource lock dev/wangzy-nfs-provisioner: endpoints "wangzy-nfs-provisioner" is forbidden: User "system:serviceaccount:dev:nfs-client-provisioner" cannot get resource "endpoints" in API group "" in the namespace "dev"

在日志中可以得到信息： default的namespace下，default 賬戶serviceaccount 不能在API group "" 獲取endpoints 資源。
因此我們需要創建一個角色，使其擁有對endpoint資源操作的權限，並且角色與賬戶進行綁定。

 

2.2 修改rbac，添加權限，網上很多，我這邊列出其中權限部分

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole    # 創建集群角色
metadata:
   name: nfs-client-provisioner-runner
# 角色權限
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["watch", "create", "update", "patch"]
  - apiGroups: [""]
    resources: ["services"]
    verbs: ["get"]
  - apiGroups: ["extensions"]
    resources: ["podsecuritypolicies"]
    resourceNames: ["nfs-provisioner"]
    verbs: ["use"]
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]

2.3 重啟服務，發現狀態變為bound

 

 

 

 

附加：

此處還碰到一個權限問題，也記錄一下，報錯信息如下

I0414 06:46:50.366392       1 event.go:221] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"dev", Name:"test-claim", UID:"d4d65bfc-3452-4eca-8629-da1aac58550b", APIVersion:"v1", ResourceVersion:"10396861", FieldPath:""}): type: 'Warning' reason: 'ProvisioningFailed' failed to provision volume with StorageClass "managed-nfs-storage": unable to create directory to provision new pv: mkdir /persistentvolumes/dev-test-claim-pvc-d4d65bfc-3452-4eca-8629-da1aac58550b: permission denied

 

 

原因是因為共享目錄沒有權限，給其添加權限，然后重啟provisioner服務

[root@master wangzy]# chmod -R 777 /root/data/　　#/root/data/為我的共享目錄

[root@master wangzy]# kubectl apply -f provisioner-02.yaml