Kubernetes升級為1.20版本后,原有的后端nfs存儲storageclass無法自動創建pv。
查看PVC狀態一直為pending狀態,查看nfs-provisioner日志
kubectl logs nfs-client-provisioner-5f696dc8bb-qhmsn E0118 03:01:07.352670 1 streamwatcher.go:109] Unable to decode an event from the watch stream: http2: server sent GOAWAY and closed the connection; LastStreamID=3737, ErrCode=NO_ERROR, debug="" E0118 03:01:07.353951 1 reflector.go:322] github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:668: Failed to watch *v1.StorageClass: Get https://10.96.0.1:443/apis/storage.k8s.io/v1/storageclasses?resourceVersion=604432&timeoutSeconds=387&watch=true: dial tcp 10.96.0.1:443: connect: connection refused W0118 03:01:07.366606 1 reflector.go:341] github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:665: watch of *v1.PersistentVolume ended with: too old resource version: 11565 (605173) W0118 03:01:07.367679 1 reflector.go:341] github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:662: watch of *v1.PersistentVolumeClaim ended with: too old resource version: 11565 (605173) I0118 03:08:28.340240 1 controller.go:987] provision "default/auth-platorm-redis-data-auth-platorm-redis-cluster-0" class "course-nfs-storage": started E0118 03:08:28.343582 1 controller.go:1004] provision "default/auth-platorm-redis-data-auth-platorm-redis-cluster-0" class "course-nfs-storage": unexpected error getting claim reference: selfLink was empty, can't make reference I0118 03:16:08.373590 1 controller.go:987] provision "default/auth-platorm-redis-data-auth-platorm-redis-cluster-0" class "course-nfs-storage": started E0118 03:16:08.382178 1 controller.go:1004] provision "default/auth-platorm-redis-data-auth-platorm-redis-cluster-0" class "course-nfs-storage": unexpected error getting claim reference: selfLink was empty, can't make reference I0118 03:30:41.647626 1 controller.go:987] provision "default/test-pvc" class "course-nfs-storage": started E0118 03:30:41.658419 1 controller.go:1004] provision "default/test-pvc" class "course-nfs-storage": unexpected error getting claim reference: selfLink was empty, can't make reference I0118 03:31:08.373713 1 controller.go:987] provision "default/auth-platorm-redis-data-auth-platorm-redis-cluster-0" class "course-nfs-storage": started I0118 03:31:08.373969 1 controller.go:987] provision "default/test-pvc" class "course-nfs-storage": started E0118 03:31:08.382279 1 controller.go:1004] provision "default/test-pvc" class "course-nfs-storage": unexpected error getting claim reference: selfLink was empty, can't make reference E0118 03:31:08.382791 1 controller.go:1004] provision "default/auth-platorm-redis-data-auth-platorm-redis-cluster-0" class "course-nfs-storage": unexpected error getting claim reference: selfLink was empty, can't make reference
報錯信息“selfLink was empty”,於是上網查詢相關內容,在官方1.20的變更說明中看到其中一條說明為:
Stop propagating SelfLink (deprecated in 1.16) in kube-apiserver
selfLink在1.16版本以后已經棄用,在1.20版本停用。
而由於nfs-provisioner的實現是基於selfLink功能(同時也會影響其他用到selfLink這個功能的第三方軟件),需要等nfs-provisioner的制作方重新提供新的解決方案。
目前可用的臨時方案是:
修改/etc/kubernetes/manifests/kube-apiserver.yaml文件,找到如下內容后,在最后添加一項參數
spec: containers: - command: - kube-apiserver
- --advertise-address=192.168.210.20
- --....... #省略多行內容
- --feature-gates=RemoveSelfLink=false #添加此行
如果是高可用的k8s集群,則需要在所有master節點上進行此操作。
添加后需要刪除apiserver的所有pod進行重啟
kubectl delete pod kube-apiserver-master01 -n kube-system kubectl delete pod kube-apiserver-master02 -n kube-system kubectl delete pod kube-apiserver-master03 -n kube-system
三台apiserver被kubelet重啟拉起后,再次查詢PVC,可以看到PVC狀態都為Bound,可以正常被PV綁定了