以下部署過程基於官方手冊,稍作調整以便理解。
若還未搭建環境,可參考CentOS7 Kubernetes minikube本地環境搭建一文搭建環境。
本示例包含以下內容:
由於本篇內容是按實際操作順序編排,如要實踐本篇中的事例,推薦從頭到尾按順序一步步執行,以免“跳戲”。
創建Deployment
$ vim nginx-deployment.yaml
創建nginx-deployment.yaml
文件,聲明deployment的各項參數,文件內容如下:
apiVersion: extensions/v1beta1 kind: Deployment metadata: #指定deployment的名稱 name: nginx-deployment labels: app: nginx spec: #期望創建3個nginx實例(3個pod) replicas: 3 selector: #選擇label:app=nginx的pod來創建實例 matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.7.9 ports: - containerPort: 80
該文件表示要創建3個pod,每個pod運行nginx1.7.9鏡像。
執行創建
$ kubectl create -f nginx-deployment.yaml $ deployment.apps "nginx-deployment" created
創建成功,查看deployment和pod的運行狀況:
$ kubectl get deploy NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE nginx-deployment 3 3 3 3 57s $ kubectl get po -o wide NAME READY STATUS RESTARTS AGE IP NODE nginx-deployment-75675f5897-jgl27 1/1 Running 0 12s 172.17.0.5 minikube nginx-deployment-75675f5897-rrnsk 1/1 Running 0 12s 172.17.0.7 minikube nginx-deployment-75675f5897-txlkv 1/1 Running 0 12s 172.17.0.6 minikube
可以看到deployment已經創建成功,並創建了3個pod,IP地址分別是172.17.0.5,172.17.0.6,172.17.0.7
。
分別對三個IP地址使用curl命令,均能返回nginx的歡迎頁內容,表示創建成功。
$ curl http://172.17.0.5 <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> <style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } </style> </head> <body> <h1>Welcome to nginx!</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p> <p><em>Thank you for using nginx.</em></p> </body> </html>
更新Deployment
我們將nginx:1.7.9
版本更新到nginx:1.9.1
。
手動設置升級的nginx版本:
$ kubectl set image deployment/nginx-deployment nginx=nginx:1.9.1 --record deployment.apps "nginx-deployment" image updated
查看rollout,pod和rs狀態:
$ kubectl rollout status deploy nginx-deployment Waiting for rollout to finish: 2 out of 3 new replicas have been updated... $ kubectl get po NAME READY STATUS RESTARTS AGE nginx-deployment-75675f5897-rrnsk 1/1 Terminating 0 1m nginx-deployment-75675f5897-txlkv 0/1 Terminating 0 1m nginx-deployment-c4747d96c-bf97v 1/1 Running 0 4s nginx-deployment-c4747d96c-ctpl2 1/1 Running 0 5s nginx-deployment-c4747d96c-rthg5 1/1 Running 0 6s $ get rs NAME DESIRED CURRENT READY AGE nginx-deployment-75675f5897 0 0 0 1m nginx-deployment-c4747d96c 3 3 3 9s
可以看到舊的pod正在被終止,新的pod被創建出來,這是滾動升級的過程。
等升級完成再來看看rollout和pod的狀態:
$ kubectl rollout status deploy nginx-deployment deployment "nginx-deployment" successfully rolled out $ kubectl get po NAME READY STATUS RESTARTS AGE nginx-deployment-c4747d96c-bf97v 1/1 Running 0 4m nginx-deployment-c4747d96c-ctpl2 1/1 Running 0 4m nginx-deployment-c4747d96c-rthg5 1/1 Running 0 4m
可以看到已經升級完成了,依舊是3個pod。
我們可以通過describe
命令查看deployment的升級過程:
$ kubectl describe deployments Name: nginx-deployment Namespace: default CreationTimestamp: Mon, 31 Dec 2018 16:52:13 +0800 Labels: app=nginx Annotations: deployment.kubernetes.io/revision=2 kubernetes.io/change-cause=kubectl set image deployment/nginx-deployment nginx=nginx:1.9.1 --record=true Selector: app=nginx Replicas: 3 desired | 3 updated | 3 total | 3 available | 0 unavailable StrategyType: RollingUpdate MinReadySeconds: 0 RollingUpdateStrategy: 25% max unavailable, 25% max surge Pod Template: Labels: app=nginx Containers: nginx: Image: nginx:1.9.1 Port: 80/TCP Host Port: 0/TCP Environment: <none> Mounts: <none> Volumes: <none> Conditions: Type Status Reason ---- ------ ------ Available True MinimumReplicasAvailable Progressing True NewReplicaSetAvailable OldReplicaSets: <none> NewReplicaSet: nginx-deployment-c4747d96c (3/3 replicas created) Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal ScalingReplicaSet 12m deployment-controller Scaled up replica set nginx-deployment-75675f5897 to 3 Normal ScalingReplicaSet 10m deployment-controller Scaled up replica set nginx-deployment-c4747d96c to 1 Normal ScalingReplicaSet 10m deployment-controller Scaled down replica set nginx-deployment-75675f5897 to 2 Normal ScalingReplicaSet 10m deployment-controller Scaled up replica set nginx-deployment-c4747d96c to 2 Normal ScalingReplicaSet 10m deployment-controller Scaled down replica set nginx-deployment-75675f5897 to 1 Normal ScalingReplicaSet 10m deployment-controller Scaled up replica set nginx-deployment-c4747d96c to 3 Normal ScalingReplicaSet 10m deployment-controller Scaled down replica set nginx-deployment-75675f5897 to 0
從Event.Message
這一列可以看出,滾動升級的過程就是把新的deployment的ReplicaSet設置從0慢慢上升到期望,這里也就是3,同時把舊deployment的ReplicaSet設置從3慢慢降到0。
根據RollingUpdateStrategy
的設置,保證在滾動升級中可用pod的數量。
回滾Deployment
當更新失敗,或者是更新版本不穩定,k8s也可以方便的回滾到指定的歷史版本。
比如,當我們更新的時候,把1.9.1誤輸成了1.91
$ kubectl set image deployment/nginx-deployment nginx=nginx:1.91 --record deployment.apps "nginx-deployment" image updated
查看deploy,pod和rollout狀態:
$ kubectl get deploy NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE nginx-deployment 3 4 1 3 45m $ kubectl get po NAME READY STATUS RESTARTS AGE nginx-deployment-595696685f-q8zrr 0/1 ImagePullBackOff 0 29s nginx-deployment-c4747d96c-bf97v 1/1 Running 0 44m nginx-deployment-c4747d96c-ctpl2 1/1 Running 0 44m nginx-deployment-c4747d96c-rthg5 1/1 Running 0 44m $ kubectl rollout status deploy nginx-deployment Waiting for rollout to finish: 1 out of 3 new replicas have been updated...
命令kubectl rollout status deploy nginx-deployment
會阻塞等待deployment更新完成,由於我們輸入了錯誤的nginx版本,無法更新完成,這條命令會一直阻塞,所以我們先CTRL+C
終止執行。
UP-TO-DATE=1
表示創建了一個用於更新的Pod。
從kubectl get po
的結果我們得知有一個pod創建不成功。
我們再通過describe
命令看一下升級不成功的pod發生了什么:
$kubectl describe po nginx-deployment-595696685f-q8zrr Name: nginx-deployment-595696685f-q8zrr Namespace: default Node: minikube/192.168.0.62 Start Time: Mon, 31 Dec 2018 17:37:07 +0800 Labels: app=nginx pod-template-hash=1512522419 Annotations: <none> Status: Pending IP: 172.17.0.7 Controlled By: ReplicaSet/nginx-deployment-595696685f Containers: nginx: Container ID: Image: nginx:1.91 Image ID: Port: 80/TCP Host Port: 0/TCP State: Waiting Reason: ImagePullBackOff Ready: False Restart Count: 0 Environment: <none> Mounts: /var/run/secrets/kubernetes.io/serviceaccount from default-token-q244m (ro) Conditions: Type Status Initialized True Ready False PodScheduled True Volumes: default-token-q244m: Type: Secret (a volume populated by a Secret) SecretName: default-token-q244m Optional: false QoS Class: BestEffort Node-Selectors: <none> Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s node.kubernetes.io/unreachable:NoExecute for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 12m default-scheduler Successfully assigned nginx-deployment-595696685f-q8zrr to minikube Normal SuccessfulMountVolume 12m kubelet, minikube MountVolume.SetUp succeeded for volume "default-token-q244m" Normal Pulling 10m (x4 over 12m) kubelet, minikube pulling image "nginx:1.91" Warning Failed 10m (x4 over 12m) kubelet, minikube Failed to pull image "nginx:1.91": rpc error: code = Unknown desc = Error response from daemon: manifest for nginx:1.91 not found Warning Failed 10m (x4 over 12m) kubelet, minikube Error: ErrImagePull Normal BackOff 9m (x6 over 11m) kubelet, minikube Back-off pulling image "nginx:1.91" Warning Failed 1m (x40 over 11m) kubelet, minikube Error: ImagePullBackOff
可以看到,由於鏡像nginx:1.91
是不存在的,所以pod報錯:Failed to pull image "nginx:1.91":xxxx
,導致升級不成功。
通過以下命令查看升級歷史記錄,就跟svn,git差不多。
$ kubectl rollout history deploy nginx-deployment deployments "nginx-deployment" REVISION CHANGE-CAUSE 1 <none> 2 kubectl set image deployment/nginx-deployment nginx=nginx:1.9.1 --record=true 3 kubectl set image deployment/nginx-deployment nginx=nginx:1.91 --record=true
由於我們之前升級加了--record
參數,可以看到升級當時所使用命令。
使用rollout history xxx --revision=n
可以查看明細:
$ kubectl rollout history deploy nginx-deployment --revision=2 deployments "nginx-deployment" with revision #2 Pod Template: Labels: app=nginx pod-template-hash=703038527 Annotations: kubernetes.io/change-cause=kubectl set image deployment/nginx-deployment nginx=nginx:1.9.1 --record=true Containers: nginx: Image: nginx:1.9.1 Port: 80/TCP Host Port: 0/TCP Environment: <none> Mounts: <none> Volumes: <none>
用undo
操作回滾:
$ kubectl rollout undo deploy nginx-deployment deployment.apps "nginx-deployment"
或者使用--to-revision
來回滾到指定版本:
$ kubectl rollout undo deploy nginx-deployment --to-revision=2 deployment.apps "nginx-deployment"
看一下rollout,pod,rs的狀態:
$ get deployment NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE nginx-deployment 3 3 3 3 1h $ kubectl get pod NAME READY STATUS RESTARTS AGE nginx-deployment-c4747d96c-bf97v 1/1 Running 0 1h nginx-deployment-c4747d96c-ctpl2 1/1 Running 0 1h nginx-deployment-c4747d96c-rthg5 1/1 Running 0 1h $ kubectl get rs NAME DESIRED CURRENT READY AGE nginx-deployment-595696685f 0 0 0 1h nginx-deployment-75675f5897 0 0 0 1h nginx-deployment-c4747d96c 3 3 3 1h $ kubectl rollout status deploy nginx-deployment deployment "nginx-deployment" successfully rolled out
回滾成功。
describe
查看一下deployment:
$ kubectl describe deploy nginx-deployment Name: nginx-deployment Namespace: default CreationTimestamp: Mon, 31 Dec 2018 16:52:13 +0800 Labels: app=nginx Annotations: deployment.kubernetes.io/revision=4 kubernetes.io/change-cause=kubectl set image deployment/nginx-deployment nginx=nginx:1.9.1 --record=true Selector: app=nginx Replicas: 3 desired | 3 updated | 3 total | 3 available | 0 unavailable StrategyType: RollingUpdate MinReadySeconds: 0 RollingUpdateStrategy: 25% max unavailable, 25% max surge Pod Template: Labels: app=nginx Containers: nginx: Image: nginx:1.9.1 Port: 80/TCP Host Port: 0/TCP Environment: <none> Mounts: <none> Volumes: <none> Conditions: Type Status Reason ---- ------ ------ Available True MinimumReplicasAvailable Progressing True NewReplicaSetAvailable OldReplicaSets: <none> NewReplicaSet: nginx-deployment-c4747d96c (3/3 replicas created) Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal DeploymentRollback 4m deployment-controller Rolled back deployment "nginx-deployment" to revision 2 Normal ScalingReplicaSet 4m deployment-controller Scaled down replica set nginx-deployment-595696685f to 0
從Events.Message
中得知,deployment-controller將版本回滾到revision 2了。
Deployment擴容和縮容
Deployment擴容縮容的本質其實就是改變ReplicaSet的數量來控制Pod的數量,增加就是擴容,縮小就是縮容。
擴容
假設我們要將nginx從3個pod擴展到10個pod:
$ kubectl scale deploy nginx-deployment --replicas=10 deployment.extensions "nginx-deployment" scaled
查看po,rs,deployment狀態:
$ kubectl get po NAME READY STATUS RESTARTS AGE nginx-deployment-c4747d96c-2zw9w 0/1 ContainerCreating 0 2s nginx-deployment-c4747d96c-bf97v 1/1 Running 0 2h nginx-deployment-c4747d96c-ctpl2 1/1 Running 0 2h nginx-deployment-c4747d96c-fr6b6 0/1 ContainerCreating 0 2s nginx-deployment-c4747d96c-grgr6 0/1 ContainerCreating 0 2s nginx-deployment-c4747d96c-hh9tk 0/1 ContainerCreating 0 2s nginx-deployment-c4747d96c-pdz4q 0/1 ContainerCreating 0 2s nginx-deployment-c4747d96c-rthg5 1/1 Running 0 2h nginx-deployment-c4747d96c-rvfs7 0/1 ContainerCreating 0 2s nginx-deployment-c4747d96c-s5mrc 0/1 ContainerCreating 0 2s $ kubectl get rs NAME DESIRED CURRENT READY AGE nginx-deployment-595696685f 0 0 0 1h nginx-deployment-75675f5897 0 0 0 2h nginx-deployment-c4747d96c 10 10 7 2h $ kubectl get deploy NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE nginx-deployment 10 10 10 10 2h
可以看到,pod已經從3個變成10個了,這里pod的狀態還在啟動中,啟動完成之后10個pod都將會是Running
狀態。
縮容
假設我們要將nginx從10個pod減少為3個:
$ kubectl scale deploy nginx-deployment --replicas=3 deployment.extensions "nginx-deployment" scaled $ kubectl get deploy NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE nginx-deployment 3 3 3 3 2h $ kubectl get rs NAME DESIRED CURRENT READY AGE nginx-deployment-595696685f 0 0 0 1h nginx-deployment-75675f5897 0 0 0 2h nginx-deployment-c4747d96c 3 3 3 2h $ kubeget po NAME READY STATUS RESTARTS AGE nginx-deployment-c4747d96c-2zw9w 1/1 Terminating 0 3m nginx-deployment-c4747d96c-bf97v 1/1 Running 0 2h nginx-deployment-c4747d96c-ctpl2 1/1 Running 0 2h nginx-deployment-c4747d96c-hh9tk 1/1 Terminating 0 3m nginx-deployment-c4747d96c-pdz4q 0/1 Terminating 0 3m nginx-deployment-c4747d96c-rthg5 1/1 Running 0 2h
可以看到pod的數量在減少,當縮容成功之后,會只剩下3個pod。
手動擴容縮絨還不能體現k8s的精髓之處,k8s還能夠根據節點的負載情況,自動擴容縮容,這樣才能實現自動化,智能化。自動擴容的內容不在本篇中的討論范圍內。大概可能或許假以時日會寫一篇關於自動擴容的例子。
Deployment的暫停和恢復
在上述的例子中,當我們輸入更新指令之后,滾動更新就立即開始了,但有的時候,我們還需要修改其他一些配置來保證新版本的正確運行,這時就需要先暫停滾動更新,等其他配置工作完成之后,再開始滾動升級。
我們刪掉當前deployment,用初始文件重新部署一個:
$ kubectl delete deploy nginx-deployment deployment.extensions "nginx-deployment" deleted $ kubectl create -f nginx-deployment.yaml deployment.apps "nginx-deployment" created
暫停滾動更新:
$ kubectl rollout pause deploy nginx-deployment deployment.apps "nginx-deployment" paused
我們將當前的1.9.1
版本“升級”成1.7.9
:
$ kubectl set image deploy nginx-deployment nginx=nginx:1.7.9 deployment.apps "nginx-deployment" image updated
查看rollout history
$ kubectl rollout history deploy nginx-deployment deployments "nginx-deployment" REVISION CHANGE-CAUSE 1 <none>
我們發現由於我們使用rollout pause
暫停了滾動更新,該deployment並沒有更新。
繼續修改配置,完成更新操作:
$ kubectl set resources deploy nginx-deployment -c=nginx --limits=cpu=200m,memory=512Mi deployment.apps "nginx-deployment" resource requirements updated $ kubectl rollout resume deploy nginx-deployment deployment.apps/nginx-deployment resumed
配置完之后查看pod狀態:
$ kubectl get pod NAME READY STATUS RESTARTS AGE nginx-deployment-6d95f89565-7wgrb 0/1 Pending 0 2m nginx-deployment-75675f5897-9pqz5 1/1 Running 0 18m nginx-deployment-75675f5897-mdclm 1/1 Running 0 18m nginx-deployment-75675f5897-vf4lw 1/1 Running 0 18m
發現第一個Pod一直處於Pending
狀態,這劇本好像不太對…
有了前面describe
查看deployment升級失敗的例子,這里我們舉一反三一下:
$ kubectl describe pod nginx-deployment-6d95f89565-7wgrb Name: nginx-deployment-6d95f89565-7wgrb Namespace: default Node: <none> Labels: app=nginx pod-template-hash=2851945121 Annotations: <none> Status: Pending IP: Controlled By: ReplicaSet/nginx-deployment-6d95f89565 Containers: nginx: Image: nginx:1.7.9 Port: 80/TCP Host Port: 0/TCP Limits: cpu: 200m memory: 512Mi Requests: cpu: 200m memory: 512Mi Environment: <none> Mounts: /var/run/secrets/kubernetes.io/serviceaccount from default-token-q244m (ro) Conditions: Type Status PodScheduled False Volumes: default-token-q244m: Type: Secret (a volume populated by a Secret) SecretName: default-token-q244m Optional: false QoS Class: Guaranteed Node-Selectors: <none> Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s node.kubernetes.io/unreachable:NoExecute for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning FailedScheduling 4s (x12 over 2m) default-scheduler 0/1 nodes are available: 1 Insufficient cpu.
看到Events
知道了,是我的服務器太渣了,分配不起200m的cpu資源,那就分配少一些,重新分配一下:
$ kubectl set resources deploy nginx-deployment -c=nginx --limits=cpu=20m,memory=256Mi deployment.apps "nginx-deployment" resource requirements updated
等一陣子等pod啟動完畢:
$ kubectl get po NAME READY STATUS RESTARTS AGE nginx-deployment-6b58c6db49-n9jmf 1/1 Running 0 20s nginx-deployment-6b58c6db49-rbwxw 1/1 Running 0 22s nginx-deployment-6b58c6db49-rklg2 1/1 Running 0 17s
現在能正常更新完成了。
用Service暴露Deployment
到目前為止,上述deployment所生產的Pod都只能被Node內部訪問,而且而且每次pod擴容,縮容或者是升級之后,IP地址均可能改變。我們用service將deployment暴露出去,讓k8s外部的應用也可以訪問:
$ kubectl expose deploy nginx-deployment --type=NodePort --name=nginx --port=80 service "nginx" exposed
使用get svc
查看服務:
$ kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 21h nginx NodePort 10.111.219.247 <none> 80:31302/TCP 13s
這時候外部應用就可以使用31302這個端口來訪問內部的包含nginx的pod了。
這里就不着重介紹service了,service的內容就留到下一篇吧。