Kubernetes K8S之Pod生命周期與探針檢測

 

K8S中Pod的生命周期與ExecAction、TCPSocketAction和HTTPGetAction探針檢測

 

主機配置規划

服務器名稱(hostname)	系統版本	配置	內網IP	外網IP(模擬)
k8s-master	CentOS7.7	2C/4G/20G	172.16.1.110	10.0.0.110
k8s-node01	CentOS7.7	2C/4G/20G	172.16.1.111	10.0.0.111
k8s-node02	CentOS7.7	2C/4G/20G	172.16.1.112	10.0.0.112

 

Pod容器生命周期

 

Pause容器說明

每個Pod里運行着一個特殊的被稱之為Pause的容器，其他容器則為業務容器，這些業務容器共享Pause容器的網絡棧和Volume掛載卷，因此他們之間通信和數據交換更為高效。在設計時可以充分利用這一特性，將一組密切相關的服務進程放入同一個Pod中；同一個Pod里的容器之間僅需通過localhost就能互相通信。

kubernetes中的pause容器主要為每個業務容器提供以下功能：

PID命名空間：Pod中的不同應用程序可以看到其他應用程序的進程ID。

網絡命名空間：Pod中的多個容器能夠訪問同一個IP和端口范圍。

IPC命名空間：Pod中的多個容器能夠使用SystemV IPC或POSIX消息隊列進行通信。

UTS命名空間：Pod中的多個容器共享一個主機名；Volumes（共享存儲卷）。

Pod中的各個容器可以訪問在Pod級別定義的Volumes。

 

容器探針

探針是由 kubelet 對容器執行的定期診斷。要執行診斷，則需kubelet 調用由容器實現的 Handler。探針有三種類型的處理程序：

• ExecAction：在容器內執行指定命令。如果命令退出時返回碼為 0 則認為診斷成功。
• CPSocketAction：對指定端口上的容器的 IP 地址進行 TCP 檢查。如果端口打開，則診斷被認為是成功的。
• HTTPGetAction：對指定的端口和路徑上的容器的 IP 地址執行 HTTP Get 請求。如果響應的狀態碼大於等於200 且小於 400，則診斷被認為是成功的。

每次探測都將獲得以下三種結果之一：

• 成功：容器通過了診斷。
• 失敗：容器未通過診斷。
• 未知：診斷失敗，因此不會采取任何行動。

Kubelet 可以選擇是否在容器上運行三種探針執行和做出反應：

• livenessProbe：指示容器是否正在運行。如果存活探測失敗，則 kubelet 會殺死容器，並且容器將受到其重啟策略的影響。如果容器不提供存活探針，則默認狀態為 Success。
• readinessProbe：指示容器是否准備好服務請求【對外接受請求訪問】。如果就緒探測失敗，端點控制器將從與 Pod 匹配的所有 Service 的端點中刪除該 Pod 的 IP 地址。初始延遲之前的就緒狀態默認為 Failure。如果容器不提供就緒探針，則默認狀態為 Success。
• startupProbe: 指示容器中的應用是否已經啟動。如果提供了啟動探測(startup probe)，則禁用所有其他探測，直到它成功為止。如果啟動探測失敗，kubelet 將殺死容器，容器服從其重啟策略進行重啟。如果容器沒有提供啟動探測，則默認狀態為成功Success。

備注：可以以Tomcat web服務為例。

 

容器重啟策略

PodSpec 中有一個 restartPolicy 字段，可能的值為 Always、OnFailure 和 Never。默認為 Always。

Always表示一旦不管以何種方式終止運行，kubelet都將重啟；OnFailure表示只有Pod以非0退出碼退出才重啟；Nerver表示不再重啟該Pod。

restartPolicy 適用於 Pod 中的所有容器。restartPolicy 僅指通過同一節點上的 kubelet 重新啟動容器。失敗的容器由 kubelet 以五分鍾為上限的指數退避延遲（10秒，20秒，40秒…）重新啟動，並在成功執行十分鍾后重置。如 Pod 文檔中所述，一旦pod綁定到一個節點，Pod 將永遠不會重新綁定到另一個節點。

 

存活（liveness）和就緒（readiness）探針的使用場景

如果容器中的進程能夠在遇到問題或不健康的情況下自行崩潰，則不一定需要存活探針；kubelet 將根據 Pod 的restartPolicy 自動執行正確的操作。

如果你希望容器在探測失敗時被殺死並重新啟動，那么請指定一個存活探針，並指定restartPolicy 為 Always 或 OnFailure。

如果要僅在探測成功時才開始向 Pod 發送流量，請指定就緒探針。在這種情況下，就緒探針可能與存活探針相同，但是 spec 中的就緒探針的存在意味着 Pod 將在沒有接收到任何流量的情況下啟動，並且只有在探針探測成功后才開始接收流量。

 

Pod phase（階段）

Pod 的 status 定義在 PodStatus 對象中，其中有一個 phase 字段。

Pod 的運行階段（phase）是 Pod 在其生命周期中的簡單宏觀概述。該階段並不是對容器或 Pod 的綜合匯總，也不是為了做為綜合狀態機。

Pod 相位的數量和含義是嚴格指定的。除了本文檔中列舉的內容外，不應該再假定 Pod 有其他的 phase 值。

 

下面是 phase 可能的值：

• 掛起（Pending）：Pod 已被 Kubernetes 系統接受，但有一個或者多個容器鏡像尚未創建。等待時間包括調度 Pod 的時間和通過網絡下載鏡像的時間，這可能需要花點時間。
• 運行中（Running）：該 Pod 已經綁定到了一個節點上，Pod 中所有的容器都已被創建。至少有一個容器正在運行，或者正處於啟動或重啟狀態。
• 成功（Succeeded）：Pod 中的所有容器都被成功終止，並且不會再重啟。
• 失敗（Failed）：Pod 中的所有容器都已終止了，並且至少有一個容器是因為失敗終止。也就是說，容器以非0狀態退出或者被系統終止。
• 未知（Unknown）：因為某些原因無法取得 Pod 的狀態，通常是因為與 Pod 所在主機通信失敗。

 

檢測探針-就緒檢測

pod yaml腳本

[root@k8s-master lifecycle]# pwd
/root/k8s_practice/lifecycle
[root@k8s-master lifecycle]# cat readinessProbe-httpget.yaml 
apiVersion: v1
kind: Pod
metadata:
  name: readiness-httpdget-pod
  namespace: default
  labels:
    test: readiness-httpdget
spec:
  containers:
  - name: readiness-httpget
    image: registry.cn-beijing.aliyuncs.com/google_registry/nginx:1.17
    imagePullPolicy: IfNotPresent
    readinessProbe:
      httpGet:
        path: /index1.html
        port: 80
      initialDelaySeconds: 5  #容器啟動完成后，kubelet在執行第一次探測前應該等待 5 秒。默認是 0 秒，最小值是 0。
      periodSeconds: 3  #指定 kubelet 每隔 3 秒執行一次存活探測。默認是 10 秒。最小值是 1

 

創建 Pod，並查看pod狀態

[root@k8s-master lifecycle]# kubectl apply -f readinessProbe-httpget.yaml 
pod/readiness-httpdget-pod created
[root@k8s-master lifecycle]# kubectl get pod -n default -o wide
NAME                     READY   STATUS    RESTARTS   AGE   IP            NODE         NOMINATED NODE   READINESS GATES
readiness-httpdget-pod   0/1     Running   0          5s    10.244.2.25   k8s-node02   <none>           <none>

 

查看pod詳情

[root@k8s-master lifecycle]# kubectl describe pod readiness-httpdget-pod
Name:         readiness-httpdget-pod
Namespace:    default
Priority:     0
Node:         k8s-node02/172.16.1.112
Start Time:   Sat, 23 May 2020 16:10:04 +0800
Labels:       test=readiness-httpdget
Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                {"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"labels":{"test":"readiness-httpdget"},"name":"readiness-httpdget-pod","names...
Status:       Running
IP:           10.244.2.25
IPs:
  IP:  10.244.2.25
Containers:
  readiness-httpget:
    Container ID:   docker://066d66aaef191b1db08e1b3efba6a9be75378d2fe70e99400fc513b91242089c
………………
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Sat, 23 May 2020 16:10:05 +0800
    Ready:          False   ##### 狀態為False
    Restart Count:  0
    Readiness:      http-get http://:80/index1.html delay=5s timeout=1s period=3s #success=1 #failure=3
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-v48g4 (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False   ##### 為False
  ContainersReady   False   ##### 為False
  PodScheduled      True 
Volumes:
  default-token-v48g4:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-v48g4
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                From                 Message
  ----     ------     ----               ----                 -------
  Normal   Scheduled  <unknown>          default-scheduler    Successfully assigned default/readiness-httpdget-pod to k8s-node02
  Normal   Pulled     49s                kubelet, k8s-node02  Container image "registry.cn-beijing.aliyuncs.com/google_registry/nginx:1.17" already present on machine
  Normal   Created    49s                kubelet, k8s-node02  Created container readiness-httpget
  Normal   Started    49s                kubelet, k8s-node02  Started container readiness-httpget
  Warning  Unhealthy  2s (x15 over 44s)  kubelet, k8s-node02  Readiness probe failed: HTTP probe failed with statuscode: 404

由上可見，容器未就緒。

我們進入pod的第一個容器，然后創建對應的文件

[root@k8s-master lifecycle]# kubectl exec -it readiness-httpdget-pod -c readiness-httpget bash
root@readiness-httpdget-pod:/# cd /usr/share/nginx/html
root@readiness-httpdget-pod:/usr/share/nginx/html# ls
50x.html  index.html
root@readiness-httpdget-pod:/usr/share/nginx/html# echo "readiness-httpdget info" > index1.html
root@readiness-httpdget-pod:/usr/share/nginx/html# ls
50x.html  index.html  index1.html

 

之后看pod狀態與詳情

[root@k8s-master lifecycle]# kubectl get pod -n default -o wide
NAME                     READY   STATUS    RESTARTS   AGE     IP            NODE         NOMINATED NODE   READINESS GATES
readiness-httpdget-pod   1/1     Running   0          2m30s   10.244.2.25   k8s-node02   <none>           <none>
[root@k8s-master lifecycle]# kubectl describe pod readiness-httpdget-pod
Name:         readiness-httpdget-pod
Namespace:    default
Priority:     0
Node:         k8s-node02/172.16.1.112
Start Time:   Sat, 23 May 2020 16:10:04 +0800
Labels:       test=readiness-httpdget
Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                {"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"labels":{"test":"readiness-httpdget"},"name":"readiness-httpdget-pod","names...
Status:       Running
IP:           10.244.2.25
IPs:
  IP:  10.244.2.25
Containers:
  readiness-httpget:
    Container ID:   docker://066d66aaef191b1db08e1b3efba6a9be75378d2fe70e99400fc513b91242089c
………………
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Sat, 23 May 2020 16:10:05 +0800
    Ready:          True     ##### 狀態為True
    Restart Count:  0
    Readiness:      http-get http://:80/index1.html delay=5s timeout=1s period=3s #success=1 #failure=3
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-v48g4 (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True   ##### 為True
  ContainersReady   True   ##### 為True
  PodScheduled      True 
Volumes:
  default-token-v48g4:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-v48g4
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                   From                 Message
  ----     ------     ----                  ----                 -------
  Normal   Scheduled  <unknown>             default-scheduler    Successfully assigned default/readiness-httpdget-pod to k8s-node02
  Normal   Pulled     2m33s                 kubelet, k8s-node02  Container image "registry.cn-beijing.aliyuncs.com/google_registry/nginx:1.17" already present on machine
  Normal   Created    2m33s                 kubelet, k8s-node02  Created container readiness-httpget
  Normal   Started    2m33s                 kubelet, k8s-node02  Started container readiness-httpget
  Warning  Unhealthy  85s (x22 over 2m28s)  kubelet, k8s-node02  Readiness probe failed: HTTP probe failed with statuscode: 404

由上可見，容器已就緒。

 

檢測探針-存活檢測

存活檢測-執行命令

pod yaml腳本

[root@k8s-master lifecycle]# pwd
/root/k8s_practice/lifecycle
[root@k8s-master lifecycle]# cat livenessProbe-exec.yaml 
apiVersion: v1
kind: Pod
metadata:
  name: liveness-exec-pod
  labels:
    test: liveness
spec:
  containers:
  - name: liveness-exec
    image: registry.cn-beijing.aliyuncs.com/google_registry/busybox:1.24
    imagePullPolicy: IfNotPresent
    args:
    - /bin/sh
    - -c
    - touch /tmp/healthy; sleep 30; rm -rf /tmp/healthy; sleep 600
    livenessProbe:
      exec:
        command:
        - cat
        - /tmp/healthy
      initialDelaySeconds: 5   # 第一次檢測前等待5秒
      periodSeconds: 3   # 檢測周期3秒一次

這個容器生命的前 30 秒，/tmp/healthy 文件是存在的。所以在這最開始的 30 秒內，執行命令 cat /tmp/healthy 會返回成功碼。30 秒之后，執行命令 cat /tmp/healthy 就會返回失敗狀態碼。

 

創建 Pod

[root@k8s-master lifecycle]# kubectl apply -f livenessProbe-exec.yaml 
pod/liveness-exec-pod created

 

在 30 秒內，查看 Pod 的描述：

[root@k8s-master lifecycle]# kubectl get pod -o wide
NAME                READY   STATUS    RESTARTS   AGE   IP            NODE         NOMINATED NODE   READINESS GATES
liveness-exec-pod   1/1     Running   0          17s   10.244.2.21   k8s-node02   <none>           <none>
[root@k8s-master lifecycle]# kubectl describe pod liveness-exec-pod
Name:         liveness-exec-pod
Namespace:    default
Priority:     0
Node:         k8s-node02/172.16.1.112
………………
Events:
  Type    Reason     Age   From                 Message
  ----    ------     ----  ----                 -------
  Normal  Scheduled  25s   default-scheduler    Successfully assigned default/liveness-exec-pod to k8s-node02
  Normal  Pulled     24s   kubelet, k8s-node02  Container image "registry.cn-beijing.aliyuncs.com/google_registry/busybox:1.24" already present on machine
  Normal  Created    24s   kubelet, k8s-node02  Created container liveness-exec
  Normal  Started    24s   kubelet, k8s-node02  Started container liveness-exec

輸出結果顯示：存活探測器成功。

 

35 秒之后，再來看 Pod 的描述：

[root@k8s-master lifecycle]# kubectl get pod -o wide   # 顯示 RESTARTS 的值增加了 1
NAME                READY   STATUS    RESTARTS   AGE   IP            NODE         NOMINATED NODE   READINESS GATES
liveness-exec-pod   1/1     Running   1          89s   10.244.2.22   k8s-node02   <none>           <none>
[root@k8s-master lifecycle]# kubectl describe pod liveness-exec-pod
………………
Events:
  Type     Reason     Age              From                 Message
  ----     ------     ----             ----                 -------
  Normal   Scheduled  42s              default-scheduler    Successfully assigned default/liveness-exec-pod to k8s-node02
  Normal   Pulled     41s              kubelet, k8s-node02  Container image "registry.cn-beijing.aliyuncs.com/google_registry/busybox:1.24" already present on machine
  Normal   Created    41s              kubelet, k8s-node02  Created container liveness-exec
  Normal   Started    41s              kubelet, k8s-node02  Started container liveness-exec
  Warning  Unhealthy  2s (x3 over 8s)  kubelet, k8s-node02  Liveness probe failed: cat: can't open '/tmp/healthy': No such file or directory
  Normal   Killing    2s               kubelet, k8s-node02  Container liveness-exec failed liveness probe, will be restarted

由上可見，在輸出結果的最下面，有信息顯示存活探測器失敗了，因此這個容器被殺死並且被重建了。

 

存活檢測-HTTP請求

pod yaml腳本

[root@k8s-master lifecycle]# pwd
/root/k8s_practice/lifecycle
[root@k8s-master lifecycle]# cat livenessProbe-httpget.yaml 
apiVersion: v1
kind: Pod
metadata:
  name: liveness-httpget-pod
  labels:
    test: liveness
spec:
  containers:
  - name: liveness-httpget
    image: registry.cn-beijing.aliyuncs.com/google_registry/nginx:1.17
    imagePullPolicy: IfNotPresent
    ports:
    - name: http
      containerPort: 80
    livenessProbe:
      httpGet:   # 任何大於或等於 200 並且小於 400 的返回碼表示成功，其它返回碼都表示失敗。
        path: /index.html
        port: 80
        httpHeaders:  #請求中自定義的 HTTP 頭。HTTP 頭字段允許重復。
        - name: Custom-Header
          value: Awesome
      initialDelaySeconds: 5
      periodSeconds: 3

 

創建 Pod，查看pod狀態

[root@k8s-master lifecycle]# kubectl apply -f livenessProbe-httpget.yaml 
pod/liveness-httpget-pod created
[root@k8s-master lifecycle]# kubectl get pod -n default -o wide
NAME                   READY   STATUS    RESTARTS   AGE   IP            NODE         NOMINATED NODE   READINESS GATES
liveness-httpget-pod   1/1     Running   0          3s    10.244.2.27   k8s-node02   <none>           <none>

 

查看pod詳情

[root@k8s-master lifecycle]# kubectl describe pod liveness-httpget-pod
Name:         liveness-httpget-pod
Namespace:    default
Priority:     0
Node:         k8s-node02/172.16.1.112
Start Time:   Sat, 23 May 2020 16:45:25 +0800
Labels:       test=liveness
Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                {"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"labels":{"test":"liveness"},"name":"liveness-httpget-pod","namespace":"defau...
Status:       Running
IP:           10.244.2.27
IPs:
  IP:  10.244.2.27
Containers:
  liveness-httpget:
    Container ID:   docker://4b42a351414667000fe94d4f3166d75e72a3401e549fed723126d2297124ea1a
………………
    Port:           80/TCP
    Host Port:      8080/TCP
    State:          Running
      Started:      Sat, 23 May 2020 16:45:26 +0800
    Ready:          True
    Restart Count:  0
    Liveness:       http-get http://:80/index.html delay=5s timeout=1s period=3s #success=1 #failure=3
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-v48g4 (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
………………
Events:
  Type    Reason     Age        From                 Message
  ----    ------     ----       ----                 -------
  Normal  Scheduled  <unknown>  default-scheduler    Successfully assigned default/liveness-httpget-pod to k8s-node02
  Normal  Pulled     5m52s      kubelet, k8s-node02  Container image "registry.cn-beijing.aliyuncs.com/google_registry/nginx:1.17" already present on machine
  Normal  Created    5m52s      kubelet, k8s-node02  Created container liveness-httpget
  Normal  Started    5m52s      kubelet, k8s-node02  Started container liveness-httpget

由上可見，pod存活檢測正常

 

我們進入pod的第一個容器，然后刪除對應的文件

[root@k8s-master lifecycle]# kubectl exec -it liveness-httpget-pod -c liveness-httpget bash
root@liveness-httpget-pod:/# cd /usr/share/nginx/html/
root@liveness-httpget-pod:/usr/share/nginx/html# ls
50x.html  index.html
root@liveness-httpget-pod:/usr/share/nginx/html# rm -f index.html 
root@liveness-httpget-pod:/usr/share/nginx/html# ls
50x.html

 

再次看pod狀態和詳情，可見Pod的RESTARTS從0變為了1。

[root@k8s-master lifecycle]# kubectl get pod -n default -o wide   # RESTARTS 從0變為了1
NAME                   READY   STATUS    RESTARTS   AGE     IP            NODE         NOMINATED NODE   READINESS GATES
liveness-httpget-pod   1/1     Running   1          8m16s   10.244.2.27   k8s-node02   <none>           <none>
[root@k8s-master lifecycle]# kubectl describe pod liveness-httpget-pod
Name:         liveness-httpget-pod
Namespace:    default
Priority:     0
Node:         k8s-node02/172.16.1.112
Start Time:   Sat, 23 May 2020 16:45:25 +0800
Labels:       test=liveness
Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                {"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"labels":{"test":"liveness"},"name":"liveness-httpget-pod","namespace":"defau...
Status:       Running
IP:           10.244.2.27
IPs:
  IP:  10.244.2.27
Containers:
  liveness-httpget:
    Container ID:   docker://5d0962d383b1df5e59cd3d1100b259ff0415ac37c8293b17944034f530fb51c8
………………
    Port:           80/TCP
    Host Port:      8080/TCP
    State:          Running
      Started:      Sat, 23 May 2020 16:53:38 +0800
    Last State:     Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Sat, 23 May 2020 16:45:26 +0800
      Finished:     Sat, 23 May 2020 16:53:38 +0800
    Ready:          True
    Restart Count:  1
    Liveness:       http-get http://:80/index.html delay=5s timeout=1s period=3s #success=1 #failure=3
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-v48g4 (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  default-token-v48g4:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-v48g4
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                 From                 Message
  ----     ------     ----                ----                 -------
  Normal   Scheduled  <unknown>           default-scheduler    Successfully assigned default/liveness-httpget-pod to k8s-node02
  Normal   Pulled     7s (x2 over 8m19s)  kubelet, k8s-node02  Container image "registry.cn-beijing.aliyuncs.com/google_registry/nginx:1.17" already present on machine
  Normal   Created    7s (x2 over 8m19s)  kubelet, k8s-node02  Created container liveness-httpget
  Normal   Started    7s (x2 over 8m19s)  kubelet, k8s-node02  Started container liveness-httpget
  Warning  Unhealthy  7s (x3 over 13s)    kubelet, k8s-node02  Liveness probe failed: HTTP probe failed with statuscode: 404
  Normal   Killing    7s                  kubelet, k8s-node02  Container liveness-httpget failed liveness probe, will be restarted

由上可見，當liveness-httpget檢測失敗，重建了Pod容器

 

存活檢測-TCP端口

pod yaml腳本

[root@k8s-master lifecycle]# pwd
/root/k8s_practice/lifecycle
[root@k8s-master lifecycle]# cat livenessProbe-tcp.yaml 
apiVersion: v1
kind: Pod
metadata:
  name: liveness-tcp-pod
  labels:
    test: liveness
spec:
  containers:
  - name: liveness-tcp
    image: registry.cn-beijing.aliyuncs.com/google_registry/nginx:1.17
    imagePullPolicy: IfNotPresent
    ports:
    - name: http
      containerPort: 80
    livenessProbe:
      tcpSocket:
        port: 80
      initialDelaySeconds: 5
      periodSeconds: 3

 

TCP探測正常情況

創建 Pod，查看pod狀態

[root@k8s-master lifecycle]# kubectl apply -f livenessProbe-tcp.yaml
pod/liveness-tcp-pod created
[root@k8s-master lifecycle]# kubectl get pod -o wide
NAME               READY   STATUS    RESTARTS   AGE   IP            NODE         NOMINATED NODE   READINESS GATES
liveness-tcp-pod   1/1     Running   0          50s   10.244.4.23   k8s-node01   <none>           <none>

 

查看pod詳情

[root@k8s-master lifecycle]# kubectl describe pod liveness-tcp-pod
Name:         liveness-tcp-pod
Namespace:    default
Priority:     0
Node:         k8s-node01/172.16.1.111
Start Time:   Sat, 23 May 2020 18:02:46 +0800
Labels:       test=liveness
Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                {"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"labels":{"test":"liveness"},"name":"liveness-tcp-pod","namespace":"default"}...
Status:       Running
IP:           10.244.4.23
IPs:
  IP:  10.244.4.23
Containers:
  liveness-tcp:
    Container ID:   docker://4de13e7c2e36c028b2094bf9dcf8e2824bfd15b8c45a0b963e301b91ee1a926d
………………
    Port:           80/TCP
    Host Port:      8080/TCP
    State:          Running
      Started:      Sat, 23 May 2020 18:03:04 +0800
    Ready:          True
    Restart Count:  0
    Liveness:       tcp-socket :80 delay=5s timeout=1s period=3s #success=1 #failure=3
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-v48g4 (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  default-token-v48g4:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-v48g4
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason     Age        From                 Message
  ----    ------     ----       ----                 -------
  Normal  Scheduled  <unknown>  default-scheduler    Successfully assigned default/liveness-tcp-pod to k8s-node01
  Normal  Pulling    74s        kubelet, k8s-node01  Pulling image "registry.cn-beijing.aliyuncs.com/google_registry/nginx:1.17"
  Normal  Pulled     58s        kubelet, k8s-node01  Successfully pulled image "registry.cn-beijing.aliyuncs.com/google_registry/nginx:1.17"
  Normal  Created    57s        kubelet, k8s-node01  Created container liveness-tcp
  Normal  Started    57s        kubelet, k8s-node01  Started container liveness-tcp

以上是正常情況，可見存活探測成功。

 

模擬TCP探測失敗情況

將上面yaml文件中的探測TCP端口進行如下修改：

livenessProbe:
  tcpSocket:
    port: 8090  # 之前是80

 

刪除之前的pod並重新創建，並過一會兒看pod狀態

[root@k8s-master lifecycle]# kubectl apply -f livenessProbe-tcp.yaml 
pod/liveness-tcp-pod created
[root@k8s-master lifecycle]# kubectl get pod -o wide   # 可見RESTARTS變為了1，再過一會兒會變為2，之后依次疊加
NAME               READY   STATUS    RESTARTS   AGE   IP            NODE         NOMINATED NODE   READINESS GATES
liveness-tcp-pod   1/1     Running   1          25s   10.244.2.28   k8s-node02   <none>           <none>

 

pod詳情

[root@k8s-master lifecycle]# kubectl describe pod liveness-tcp-pod
Name:         liveness-tcp-pod
Namespace:    default
Priority:     0
Node:         k8s-node02/172.16.1.112
Start Time:   Sat, 23 May 2020 18:08:32 +0800
Labels:       test=liveness
………………
Events:
  Type     Reason     Age                From                 Message
  ----     ------     ----               ----                 -------
  Normal   Scheduled  <unknown>          default-scheduler    Successfully assigned default/liveness-tcp-pod to k8s-node02
  Normal   Pulled     12s (x2 over 29s)  kubelet, k8s-node02  Container image "registry.cn-beijing.aliyuncs.com/google_registry/nginx:1.17" already present on machine
  Normal   Created    12s (x2 over 29s)  kubelet, k8s-node02  Created container liveness-tcp
  Normal   Started    12s (x2 over 28s)  kubelet, k8s-node02  Started container liveness-tcp
  Normal   Killing    12s                kubelet, k8s-node02  Container liveness-tcp failed liveness probe, will be restarted
  Warning  Unhealthy  0s (x4 over 18s)   kubelet, k8s-node02  Liveness probe failed: dial tcp 10.244.2.28:8090: connect: connection refused

由上可見，liveness-tcp檢測失敗，重建了Pod容器。

 

檢測探針-啟動檢測

有時候，會有一些現有的應用程序在啟動時需要較多的初始化時間【如：Tomcat服務】。這種情況下，在不影響對觸發這種探測的死鎖的快速響應的情況下，設置存活探測參數是要有技巧的。

技巧就是使用一個命令來設置啟動探測。針對HTTP 或者 TCP 檢測，可以通過設置 failureThreshold * periodSeconds 參數來保證有足夠長的時間應對糟糕情況下的啟動時間。

示例如下：

pod yaml文件

[root@k8s-master lifecycle]# pwd
/root/k8s_practice/lifecycle
[root@k8s-master lifecycle]# cat startupProbe-httpget.yaml
apiVersion: v1
kind: Pod
metadata:
  name: startup-pod
  labels:
    test: startup
spec:
  containers:
  - name: startup
    image: registry.cn-beijing.aliyuncs.com/google_registry/tomcat:7.0.94-jdk8-openjdk
    imagePullPolicy: IfNotPresent
    ports:
    - name: web-port
      containerPort: 8080
      hostPort: 8080
    livenessProbe:
      httpGet:
        path: /index.jsp
        port: web-port
      initialDelaySeconds: 5
      periodSeconds: 10
      failureThreshold: 1
    startupProbe:
      httpGet:
        path: /index.jsp
        port: web-port
      periodSeconds: 10      #指定 kubelet 每隔 10 秒執行一次存活探測。默認是 10 秒。最小值是 1
      failureThreshold: 30   #最大的失敗次數

 

啟動pod，並查看狀態

[root@k8s-master lifecycle]# kubectl apply -f startupProbe-httpget.yaml 
pod/startup-pod created
[root@k8s-master lifecycle]# kubectl get pod -o wide
NAME          READY   STATUS    RESTARTS   AGE     IP            NODE         NOMINATED NODE   READINESS GATES
startup-pod   1/1     Running   0          8m46s   10.244.4.26   k8s-node01   <none>           <none>

 

查看pod詳情

[root@k8s-master ~]# kubectl describe pod startup-pod

有啟動探測，應用程序將會有最多 5 分鍾(30 * 10 = 300s) 的時間來完成它的啟動。一旦啟動探測成功一次，存活探測任務就會接管對容器的探測，對容器死鎖可以快速響應。 如果啟動探測一直沒有成功，容器會在 300 秒后被殺死，並且根據 restartPolicy 來設置 Pod 狀態。

 

探測器配置詳解

使用如下這些字段可以精確的控制存活和就緒檢測行為：

• initialDelaySeconds：容器啟動后要等待多少秒后存活和就緒探測器才被初始化，默認是 0 秒，最小值是 0。
• periodSeconds：執行探測的時間間隔（單位是秒）。默認是 10 秒。最小值是 1。
• timeoutSeconds：探測的超時時間。默認值是 1 秒。最小值是 1。
• successThreshold：探測器在失敗后，被視為成功的最小連續成功數。默認值是 1。存活探測的這個值必須是 1。最小值是 1。
• failureThreshold：當探測失敗時，Kubernetes 的重試次數。存活探測情況下的放棄就意味着重新啟動容器。就緒探測情況下的放棄 Pod 會被打上未就緒的標簽。默認值是 3。最小值是 1。

HTTP 探測器可以在 httpGet 上配置額外的字段：

• host：連接使用的主機名，默認是 Pod 的 IP。也可以在 HTTP 頭中設置 “Host” 來代替。
• scheme ：用於設置連接主機的方式（HTTP 還是 HTTPS）。默認是 HTTP。
• path：訪問 HTTP 服務的路徑。
• httpHeaders：請求中自定義的 HTTP 頭。HTTP 頭字段允許重復。
• port：訪問容器的端口號或者端口名。如果數字必須在 1 ～ 65535 之間。

 

相關閱讀

1、官網：配置存活、就緒和啟動探測器

2、Kubernetes K8S之Pod 生命周期與init container初始化容器

完畢！

 

---

———END———
如果覺得不錯就關注下唄 (-^O^-) ！