Kubernetes 中進行 PVC 的擴容

允許卷擴展

FEATURE STATE: Kubernetes v1.11 [beta]

PersistentVolume 可以配置為可擴展。將此功能設置為 true 時，允許用戶通過編輯相應的 PVC 對象來調整卷大小。

當下層 StorageClass 的 allowVolumeExpansion 字段設置為 true 時，以下類型的卷支持卷擴展。

掛載選項

卷類型	Kubernetes 版本要求
gcePersistentDisk	1.11
awsElasticBlockStore	1.11
Cinder	1.11
glusterfs	1.11
rbd	1.11
Azure File	1.11
Azure Disk	1.11
Portworx	1.11
FlexVolume	1.13
CSI	1.14 (alpha), 1.16 (beta)

說明： 此功能僅可用於擴容卷，不能用於縮小卷。

案例一：

先查看storageclass是否配置了動態擴容，主要看storageclass是否存在allowVolumeExpansion字段

[root@192 ~]# kubectl  get storageclass default -oyaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: default
parameters:
  archiveOnDelete: "false"
provisioner: nfs-client-provisioner-default
reclaimPolicy: Delete
volumeBindingMode: Immediate

可以看到並沒有allowVolumeExpansion，此時是不支持動態擴容的，可以擴一下測試看看

[root@192 ~]# kubectl edit pvc -n leijun-mysql comom-store-column-03-24 
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: comom-store-column-03-24
  namespace: default
spec:
  accessModes:
  - ReadWriteMany
  dataSource: null
  resources:
    requests:
      storage: 50Gi
  volumeName: pvc-24fc01a0-6d9b-11ea-9107-b8ca3a62236c
status:
  accessModes:
  - ReadWriteMany
  capacity:
    storage: 10Gi
  phase: Bound
"/tmp/kubectl-edit-fkm33.yaml" 38L, 1180C written
error: persistentvolumeclaims "comom-store-column-03-24" could not be patched: persistentvolumeclaims "comom-store-column-03-24" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
You can run `kubectl replace -f /tmp/kubectl-edit-fkm33.yaml` to try this update again.

給storageclass添加allowVolumeExpansion字段

[root@192 ~]# kubectl  edit  storageclass default    

# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: default
parameters:
  archiveOnDelete: "false"
provisioner: nfs-client-provisioner-default
reclaimPolicy: Delete
volumeBindingMode: Immediate
allowVolumeExpansion: true   #增加該字段表示允許動態擴容
~                                                                                                                                                
~                                                                                                                                                
~                                                                                                                                                
~                                                                                                                                                
~                                                                                                                                                                                                                                                                                            
~                                                                                                                                                
"/tmp/kubectl-edit-x0fyj.yaml" 24L, 738C written
storageclass.storage.k8s.io/default edited

再次修改pvc   spec.resources.requests.storage字段

[root@192 ~]# kubectl edit pvc comom-store-column-03-24
spec:
  accessModes:
  - ReadWriteMany
  dataSource: null
  resources:
    requests:
      storage: 30Gi
  volumeName: pvc-24fc01a0-6d9b-11ea-9107-b8ca3a62236c
status:
  accessModes:
  - ReadWriteMany
  capacity:
    storage: 10Gi
  phase: Bound
"/tmp/kubectl-edit-mibh7.yaml" 38L, 1180C written
persistentvolumeclaim/comom-store-column-03-24 edited

案例二：

首先當然是要有一個 Kubernetes 1.11 版本的集群。並且提供了支持 Auto provision 的存儲。下面的實驗是基於 Azure 的 ACS-Engine 集群。

創建 StorageClass

接下來准備兩個 Storage Class 對象，分別命名為 common 和 expend，二者主體基本一致，文件名分別為 sc-common.yaml 以及 sc-exp.yaml：

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
 name: exp  # sc-common.yaml 中這里的值為 common
parameters:
 cachingmode: None
 kind: Managed
 storageaccounttype: Standard_LRS
provisioner: kubernetes.io/azure-disk
reclaimPolicy: Delete
volumeBindingMode: Immediate
allowVolumeExpansion: true # sc-common.yaml 中刪掉這一行

創建一個 PVC

我們接下來創建一個 PVC，初始首先測試一下 common 這個 Storageclass，后續的 PVC 操作都從這一個 YAML 中修改而來。

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
 name: myclaim
spec:
 accessModes:
   - ReadWriteOnce
 volumeMode: Filesystem
 resources:
   requests:
     storage: 2Gi
 storageClassName: common

同樣使用 kubectl 創建這個 PVC：

$ kubectl apply -f pvc.yaml
persistentvolumeclaim/myclaim created
$ kubectl get pvc -w
NAME      STATUS    VOLUME    CAPACITY   ACCESS MODES   STORAGECLASS   AGE
myclaim   Pending                                       common         9s
myclaim   Pending                                 common    10s
myclaim   Pending   pvc-e3f8e886-8776-11e8-b82d-000d3aa2ebc3   0                   common    11s
myclaim   Bound     pvc-e3f8e886-8776-11e8-b82d-000d3aa2ebc3   2Gi       RWO       common    11s

第一次擴容嘗試

PVC 進入 Bound 狀態之后，我們編輯 pvc.yaml，將容量改成 3Gi，並重新 Apply：

$ cat pvc.yaml | sed  "s/2Gi/3Gi/" | kubectl apply -f -
Error from server (Forbidden): error when applying patch:
...
for: "STDIN": persistentvolumeclaims "myclaim" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize

結果表明，這次擴容失敗了，失敗的原因是 Storageclass 不支持擴容

使用新的 Storageclass 創建 PVC

接下來我們將這個 PVC 刪除，使用 exp 這個 Storageclass 重建 PVC：

$ kubectl delete -f pvc.yaml
persistentvolumeclaim "myclaim" deleted
$ sed -i .bak s/common/exp/ pvc.yaml
$ kubectl apply -f pvc.yaml
persistentvolumeclaim/myclaim created
$ kubectl get pvc -w
NAME      STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
myclaim   Bound     pvc-37eb6014-8778-11e8-b82d-000d3aa2ebc3   2Gi        RWO            exp            11s

創建之后，我們可以再次嘗試對 PVC 進行擴容：

$ cat pvc.yaml | sed  "s/2Gi/1Gi/" | kubectl apply -f -
The PersistentVolumeClaim "myclaim" is invalid: spec.resources.requests.storage: Forbidden: field can not be less than previous value
$ cat pvc.yaml | sed  "s/2Gi/3Gi/" | kubectl apply -f -
persistentvolumeclaim/myclaim configured

這里兩次執行命令：

• 縮容是不允許的
• 擴容成功

接下來我們再次獲取 PVC 信息：

$ kubectl describe pvc myclaim
Name:          myclaim
Namespace:     default
StorageClass:  exp
Status:        Bound
Volume:        pvc-37eb6014-8778-11e8-b82d-000d3aa2ebc3
...
Capacity:      2Gi
Access Modes:  RWO
Conditions:
...
FileSystemResizePending
...
Waiting for user to (re-)start a pod to finish file system resize of volume on node.

綁定 Pod

新建一個 Deployment 來使用前面創建的 PVC：

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
 name: sleep
spec:
 replicas: 1
 template:
   metadata:
     labels:
       app: sleep
       version: v1
   spec:
     containers:
     - name: sleep
       image: dustise/sleep:v0.5
       imagePullPolicy: IfNotPresent
       volumeMounts:
       - name: data
         mountPath: "/data"
     volumes:
     - name: data
       persistentVolumeClaim:
         claimName: myclaim

再次查看 PVC 的情況：

$ kubectl describe pvc myclaim
...
Capacity:      3Gi

容量的修改的確生效了。

綁定之后的 PVC 擴容

再次對這個 PVC 進行擴容，我們這次從 3Gi 擴容到 4Gi：

$ cat pvc.yaml | sed  "s/2Gi/4Gi/" | kubectl apply -f -
persistentvolumeclaim/myclaim configured

然后獲取一下 PVC 的情況：

$ kubectl describe pvc myclaim
...
Capacity:      3Gi
...
Events:
 Warning  VolumeResizeFailed     31s (x2 over 56s)  volume_expand
 Original Error: failed request: autorest/azure: Service returned an error. Status=<nil> Code="OperationNotAllowed" Message="Cannot resize disk k8s-5b49c85f-dynamic-pvc-37eb6014-8778-11e8-b82d-000d3aa2ebc3 while it is attached to running VM /subscriptions/6d9be255-d214-4502-a51d-08e1d9c4a7fb/resourceGroups/k8s/providers/Microsoft.Compute/virtualMachines/k8s-agentpool1-17067717-0."

這一情況看來，這次擴容仍然沒有生效，錯誤信息中有提示，無法在已經成功掛載的卷上進行擴容，因此我們清除所有 Pod：

$ kubectl scale deployment sleep --replicas 0
deployment.extensions/sleep scaled

在相關 Pod 消失之后，我們可以再次 describe pvc myclaim，發現這個 PVC 又一次處於等待綁定的狀態中。使用 scale 指令恢復 Deployment 運行：

$ kubectl scale deployment sleep --replicas 1
deployment.extensions/sleep scaled
$ kubectl describe pvc myclaim
...
Capacity:      4Gi
...

PVC 的擴容再次成功了。

參考：

https://kubernetes.io/zh/docs/concepts/storage/storage-classes/

https://www.cnblogs.com/zphqq/p/13330732.html

https://cloud.tencent.com/developer/article/1469648