K8S容器探針

容器探針

探針是由 kubelet對容器執行的定期診斷。要執行診斷， kubelet 調用由容器實現的    Handler 。有三種類型的處理程序：
   ExecAction ：在容器內執行指定命令。如果命令退出時返回碼為 0 則認為診斷成功。
   TCPSocketAction ：對指定端口上的容器的 IP 地址進行 TCP 檢查。如果端口打開，則診斷被認為是成功的。
   HTTPGetAction ：對指定的端口和路徑上的容器的 IP 地址執行 HTTP Get 請求。如果響應的狀態碼大於等於 200 且小於 400 ，則診斷被認為是成功的
每次探測都將獲得以下三種結果之一：
   成功：容器通過了診斷。
   失敗：容器未通過診斷。
   未知：診斷失敗，因此不會采取任何行動

探針的方式

livenessProbe ：指示容器是否正在運行。如果存活探測失敗，則 kubelet 會殺死容器，並且容器將受到其 重啟策略 的影響。如果容器不提供存活探針，則默認狀態為 Success
readinessProbe ：指示容器是否准備好服務請求。如果就緒探測失敗，端點控制器將從與 Pod 匹配的所有 Service 的端點中刪除該 Pod 的 IP 地址。初始延遲之前的就緒狀態默認為 Failure 。如果容器不提供就緒探針，則默認狀態為 Success。

測試

檢測探針 - 就緒檢測

read.yaml

[root@k8s-master mnt]# cat read.yaml
apiVersion: v1
kind: Pod
metadata:
  name: readiness-httpget-pod
  namespace: default
spec:
  containers:
  - name: readiness-httpget-container
    image: wangyanglinux/myapp:v1
    imagePullPolicy: IfNotPresent
    readinessProbe:
      httpGet:
        port: 80
        path: /index1.html
      initialDelaySeconds: 1
      periodSeconds: 3
[root@k8s-master mnt]#

 

[root@k8s-master mnt]# vim read.yaml
[root@k8s-master mnt]# kubectl create -f read.yaml
pod/readiness-httpget-pod created
[root@k8s-master mnt]# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
myapp-pod               1/1     Running   0          70m
readiness-httpget-pod   0/1     Running   0          17s
[root@k8s-master mnt]# kubectl describe pod readiness-httpget-pod
Name:         readiness-httpget-pod
Namespace:    default
Priority:     0
Node:         k8s-node01/192.168.180.133
Start Time:   Wed, 18 Dec 2019 23:12:59 +0800
Labels:       <none>
Annotations:  <none>
Status:       Running
IP:           10.244.2.10
IPs:
  IP:  10.244.2.10
Containers:
  readiness-httpget-container:
    Container ID:   docker://566ff6cdcf44daaba316b796fb8bf6f9563ddd44000c9ae9f572fd0a6719684c
    Image:          wangyanglinux/myapp:v1
    Image ID:       docker-pullable://wangyanglinux/myapp@sha256:9c3dc30b5219788b2b8a4b065f548b922a34479577befb54b03330999d30d513
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Wed, 18 Dec 2019 23:13:01 +0800
    Ready:          False
    Restart Count:  0
    Readiness:      http-get http://:80/index1.html delay=1s timeout=1s period=3s #success=1 #failure=3
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-gx2h8 (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  default-token-gx2h8:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-gx2h8
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                From                 Message
  ----     ------     ----               ----                 -------
  Normal   Scheduled  <unknown>          default-scheduler    Successfully assigned default/readiness-httpget-pod to k8s-node01
  Normal   Pulled     66s                kubelet, k8s-node01  Container image "wangyanglinux/myapp:v1" already present on machine
  Normal   Created    66s                kubelet, k8s-node01  Created container readiness-httpget-container
  Normal   Started    66s                kubelet, k8s-node01  Started container readiness-httpget-container
  Warning  Unhealthy  0s (x22 over 63s)  kubelet, k8s-node01  Readiness probe failed: HTTP probe failed with statuscode: 404
[root@k8s-master mnt]# kubectl exec readiness-httpget-pod -it /bin/sh
/ # ls
bin    dev    etc    home   lib    media  mnt    proc   root   run    sbin   srv    sys    tmp    usr    var
/ # cd /usr/share/nginx
/usr/share/nginx # ls -l
total 0
drwxr-xr-x    1 root     root            24 Feb 25  2018 html
/usr/share/nginx # cd html/
/usr/share/nginx/html # ls -l
total 8
-rw-r--r--    1 root     root           537 Jan 10  2018 50x.html
-rw-r--r--    1 root     root            65 Mar  2  2018 index.html
/usr/share/nginx/html # cat index.html
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
/usr/share/nginx/html # echo "123" >> index1.html
/usr/share/nginx/html # exit
[root@k8s-master mnt]# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
myapp-pod               1/1     Running   1          73m
readiness-httpget-pod   1/1     Running   0          3m41s

 說明：由於index1.html不存在，會導致他重啟，手動創建后就正常了。

檢測探針 - 存活檢測

[root@k8s-master mnt]# cat live-exec.yaml
apiVersion: v1
kind: Pod
metadata:
  name: liveness-exec-pod
  namespace: default
spec:
  containers:
  - name: liveness-exec-container
    image: busybox
    imagePullPolicy: IfNotPresent
    command: ["/bin/sh","-c","touch /tmp/live ; sleep 60; rm -rf /tmp/live; sleep 3600"]
    livenessProbe:
      exec:
        command: ["test","-e","/tmp/live"]
      initialDelaySeconds: 1
      periodSeconds: 3
[root@k8s-master mnt]#

[root@k8s-master mnt]# vim live-exec.yaml
[root@k8s-master mnt]# kubectl create -f live-exec.yaml
pod/liveness-exec-pod created
[root@k8s-master mnt]# kubectl get pod -w
NAME                    READY   STATUS    RESTARTS   AGE
liveness-exec-pod       1/1     Running   0          25s
myapp-pod               1/1     Running   1          81m
readiness-httpget-pod   1/1     Running   0          11m
liveness-exec-pod       1/1     Running   1          101s











liveness-exec-pod       1/1     Running   2          3m19s
^Z
[1]+  已停止               kubectl get pod -w

說明：由於/tmp/live不存在，會一直重啟

[root@k8s-master mnt]# cat live-http.yaml
apiVersion: v1
kind: Pod
metadata:
  name: liveness-httpget-pod
  namespace: default
spec:
  containers:
  - name: liveness-httpget-container
    image: wangyanglinux/myapp:v1
    imagePullPolicy: IfNotPresent
    ports:
    - name: http
      containerPort: 80
    livenessProbe:
      httpGet:
        port: http
        path: /index.html
      initialDelaySeconds: 1
      periodSeconds: 3
      timeoutSeconds: 10
[root@k8s-master mnt]#

[root@k8s-master mnt]# kubectl create -f live-http.yaml
pod/liveness-httpget-pod created
[root@k8s-master mnt]# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
liveness-httpget-pod    1/1     Running   0          14s
myapp-pod               1/1     Running   1          90m
readiness-httpget-pod   1/1     Running   0          20m
[root@k8s-master mnt]# kubectl get pod -o wide
NAME                    READY   STATUS    RESTARTS   AGE   IP            NODE         NOMINATED NODE   READINESS GATES
liveness-httpget-pod    1/1     Running   0          26s   10.244.2.12   k8s-node01   <none>           <none>
myapp-pod               1/1     Running   1          90m   10.244.1.9    k8s-node02   <none>           <none>
readiness-httpget-pod   1/1     Running   0          20m   10.244.2.10   k8s-node01   <none>           <none>
[root@k8s-master mnt]# curl 10.244.2.12
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[root@k8s-master mnt]# curl 10.244.2.12/index.html
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[root@k8s-master mnt]# kubectl exec liveness-httpget-pod -it -- /bin/sh
/ # cd /usr/share/nginx/html/
/usr/share/nginx/html # ls -l
total 8
-rw-r--r--    1 root     root           537 Jan 10  2018 50x.html
-rw-r--r--    1 root     root            65 Mar  2  2018 index.html
/usr/share/nginx/html # rm -rf index.html
/usr/share/nginx/html # exit
[root@k8s-master mnt]# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
liveness-httpget-pod    1/1     Running   0          2m24s
myapp-pod               1/1     Running   1          92m
readiness-httpget-pod   1/1     Running   0          22m
[root@k8s-master mnt]# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
liveness-httpget-pod    1/1     Running   1          2m41s
myapp-pod               1/1     Running   1          92m
readiness-httpget-pod   1/1     Running   0          22m

說明：刪除Html,會發現Pod開始重啟了。

[root@k8s-master mnt]# cat live-tcp.yaml
apiVersion: v1
kind: Pod
metadata:
  name: probe-tcp
spec:
  containers:
  - name: nginx
    image: wangyanglinux/myapp:v1
    livenessProbe:
      initialDelaySeconds: 5
      timeoutSeconds: 1
      tcpSocket:
        port: 8080
      periodSeconds: 3
[root@k8s-master mnt]#

[root@k8s-master mnt]# vim live-tcp.yaml
[root@k8s-master mnt]# kubectl create -f live-tcp.yaml
pod/probe-tcp created
[root@k8s-master mnt]# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
liveness-httpget-pod    1/1     Running   1          9m24s
myapp-pod               1/1     Running   1          99m
probe-tcp               1/1     Running   0          5s
readiness-httpget-pod   1/1     Running   0          29m
[root@k8s-master mnt]# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
liveness-httpget-pod    1/1     Running   1          9m37s
myapp-pod               1/1     Running   1          99m
probe-tcp               1/1     Running   1          18s
readiness-httpget-pod   1/1     Running   0          29m
[root@k8s-master mnt]# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
liveness-httpget-pod    1/1     Running   1          9m41s
myapp-pod               1/1     Running   1          99m
probe-tcp               1/1     Running   1          22s
readiness-httpget-pod   1/1     Running   0          29m
[root@k8s-master mnt]# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
liveness-httpget-pod    1/1     Running   1          9m43s
myapp-pod               1/1     Running   1          99m
probe-tcp               1/1     Running   1          24s
readiness-httpget-pod   1/1     Running   0          29m
[root@k8s-master mnt]# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
liveness-httpget-pod    1/1     Running   1          9m44s
myapp-pod               1/1     Running   1          99m
probe-tcp               1/1     Running   1          25s
readiness-httpget-pod   1/1     Running   0          29m
[root@k8s-master mnt]# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
liveness-httpget-pod    1/1     Running   1          10m
myapp-pod               1/1     Running   1          100m
probe-tcp               1/1     Running   3          47s
readiness-httpget-pod   1/1     Running   0          29m
[root@k8s-master mnt]# kubectl delete -f live-tcp.yaml
pod "probe-tcp" deleted
[root@k8s-master mnt]#

說明：刪除Html,會發現Pod開始重啟了。

Pod hook

Pod hook （鈎子）是由 Kubernetes 管理的 kubelet 發起的，當容器中的進程啟動前或者容器中的進
程終止之前運行，這是包含在容器的生命周期之中。可以同時為 Pod 中的所有容器都配置 hook
Hook 的類型包括兩種：
  exec ：執行一段命令
  HTTP ：發送 HTTP 請求

 

重啟策略

PodSpec 中有一個 restartPolicy 字段，可能的值為 Always 、 OnFailure 和 Never 。默認為
Always 。 restartPolicy 適用於 Pod 中的所有容器。 restartPolicy 僅指通過同一節點上的
kubelet 重新啟動容器。失敗的容器由 kubelet 以五分鍾為上限的指數退避延遲（ 10 秒， 20 秒， 40
秒 ... ）重新啟動，並在成功執行十分鍾后重置。如 Pod 文檔 中所述，一旦綁定到一個節點， Pod 將
永遠不會重新綁定到另一個節點。

 Pod phase

Pod 的 status 字段是一個 PodStatus 對象， PodStatus 中有一個 phase 字段。
Pod 的相位（ phase ）是 Pod 在其生命周期中的簡單宏觀概述。該階段並不是對容器或 Pod 的綜合匯總，也不是為了做為綜合狀態機
Pod 相位的數量和含義是嚴格指定的。除了本文檔中列舉的狀態外，不應該再假定 Pod 有其他的phase 值
幾種常見的值

• 掛起（ Pending ）： Pod 已被 Kubernetes 系統接受，但有一個或者多個容器鏡像尚未創建。等待時間包括調度 Pod 的時間和通過網絡下載鏡像的時間，這可能需要花點時間
• 運行中（ Running ）：該 Pod 已經綁定到了一個節點上， Pod 中所有的容器都已被創建。至少有一個容器正在運行，或者正處於啟動或重啟狀態
• 成功（ Succeeded ）： Pod 中的所有容器都被成功終止，並且不會再重啟
• 失敗（ Failed ）： Pod 中的所有容器都已終止了，並且至少有一個容器是因為失敗終止。也就是說，容器以非 0 狀態退出或者被系統終止
• 未知（ Unknown ）：因為某些原因無法取得 Pod 的狀態，通常是因為與 Pod 所在主機通信失敗

[root@k8s-master mnt]# vim post.yaml
[root@k8s-master mnt]# kubectl create -f post.yaml
pod/lifecycle-demo created
[root@k8s-master mnt]# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
lifecycle-demo          1/1     Running   0          9s
liveness-httpget-pod    1/1     Running   1          40m
myapp-pod               1/1     Running   1          130m
readiness-httpget-pod   1/1     Running   0          60m
[root@k8s-master mnt]# kubectl exec lifecycle-demo -it -- /bin/bash
OCI runtime exec failed: exec failed: container_linux.go:346: starting container process caused "exec: \"/bin/bash\": stat /bin/bash: no such file or directory": unknown
command terminated with exit code 126
[root@k8s-master mnt]# kubectl exec lifecycle-demo -it -- /bin/sh
/ # cd /usr/share/message
/bin/sh: cd: can't cd to /usr/share/message
/ # cat /usr/share/message
Hello from the postStart handler
/ # exit
[root@k8s-master mnt]# cat post.yaml
apiVersion: v1
kind: Pod
metadata:
  name: lifecycle-demo
spec:
  containers:
  - name: lifecycle-demo-container
    image: wangyanglinux/myapp:v1
    lifecycle:
      postStart:
        exec:
          command: ["/bin/sh", "-c", "echo Hello from the postStart handler > /usr/share/message"]
      preStop:
        exec:
          command: ["/bin/sh", "-c", "echo Hello from the poststop handler > /usr/share/message"]
[root@k8s-master mnt]#