Pod的Terminated過程


容器中的進程對linux信號的處理方式

在一個容器啟動的時候,CMD 或者 ENTRYPOINT 里定義的命令會作為容器的主進程(main process)啟動,pid 為 1,一旦這個主進程退出了,容器也會被銷毀,容器內其他進程會被 kernel 直接 kill。

shell 程序除了不轉發 signals,還有個更可氣的特性是不響應退出信號SIGTERM和SIGKILL
但是,我們直接在linux上執行一個shell腳本,是可以通過kill -SIGTERM pid來結束掉的,就是為什么這個時候sh又響應退出信號了?
這是因為kernel會為每個進程加上默認的 signal handler,例外的是 pid=1 的進程,被 kernel 當作一個 init 角色,不會給他加上默認的 handler

在容器中,如果pid為1的進程是shell進程,那么這個進程就真的不會響應或者轉發信號了,使用docker stop來停止這樣的容器,結果就是docker發送給容器pid為1主進程的SIGTERM信號被忽略,等到stop命令默認的10s graceperiod到了
之后,docker發出的SIGKILL會最終結束這個容器。同樣的,這樣的容器部署在k8s中,當這個pod被刪除時(認為控制的或者是rc等控制的),Terminating的時長將達到30s以上(k8s默認的podGracePeriod為30s)
  

參考SIGINT SIGTERM SIGKILL區別

程序收到信號后,如果不對信號處理,就會導致程序退出,但如果程序捕獲信號進行處理,按照它的邏輯,它是不一定會退出的。

在這三個信號中,sigkill是不能被捕獲的,程序收到這個信號后,一定會退出。這就是kill -9一定能保證將程序殺死的原因。

sigterm與sigint的區別

信號 產生方式 對進程的影響
sigint 通過ctrl+c將會對當進程發送此信號 信號被當前進程樹接收到,也就是說,不僅當前進程會收到信號,它的子進程也會收到
sigterm kill命令不加參數就是發送這個信號 只有當前進程收到信號,子進程不會收到。如果當前進程被kill了,那么它的子進程的父進程將會是init,也就是pid為1的進程
兩個信號可以停止進程:SIGTERM和SIGKILL。 SIGTERM比較友好,進程能捕捉這個信號,根據您的需要來關閉程序。在關閉程序之前,您可以結束打開的記錄文件和完成正在做的任務。在某些情況下,假 如進程正在進行作業而且不能中斷,那么進程可以忽略這個SIGTERM信號。

 

k8s中Pod的終止過程

Kubernetes 在創建容器后立即發送 postStart 事件。但是,不能保證 postStart 處理程序 在容器的 entrypoint 調用之前被調用。相對於容器的代碼,postStart 處理程序以異步方式運行,但 Kubernetes 對容器的管理 會阻塞直到 postStart 處理程序完成。容器的狀態直到 postStart 處理程序完成后才會設置為 RUNNING 。

Kubernetes 在容器終止之前立即發送 preStop 事件。 Kubernetes 對容器的管理一直阻塞直到 preStop 處理程序完成, 除非 Pod 的寬限期過期。

 

Termination of Pods
Because Pods represent running processes on nodes in the cluster, it is important to allow those processes to gracefully terminate when they are no longer needed (vs being violently killed with a KILL signal and having no chance to clean up). Users should be able to request deletion and know when processes terminate, but also be able to ensure that deletes eventually complete. When a user requests deletion of a Pod, the system records the intended grace period before the Pod is allowed to be forcefully killed, and a TERM signal is sent to the main process in each container. Once the grace period has expired, the KILL signal is sent to those processes, and the Pod is then deleted from the API server. If the Kubelet or the container manager is restarted while waiting for processes to terminate, the termination will be retried with the full grace period.

An example flow:

User sends command to delete Pod, with default grace period (30s)
The Pod in the API server is updated with the time beyond which the Pod is considered “dead” along with the grace period.
Pod shows up as “Terminating” when listed in client commands
(simultaneous with 3) When the Kubelet sees that a Pod has been marked as terminating because the time in 2 has been set, it begins the Pod shutdown process.
If one of the Pod’s containers has defined a preStop hook, it is invoked inside of the container. If the preStop hook is still running after the grace period expires, step 2 is then invoked with a small (2 second) extended grace period.
The container is sent the TERM signal. Note that not all containers in the Pod will receive the TERM signal at the same time and may each require a preStop hook if the order in which they shut down matters.
(simultaneous with 3) Pod is removed from endpoints list for service, and are no longer considered part of the set of running Pods for replication controllers. Pods that shutdown slowly cannot continue to serve traffic as load balancers (like the service proxy) remove them from their rotations.
When the grace period expires, any processes still running in the Pod are killed with SIGKILL.
The Kubelet will finish deleting the Pod on the API server by setting grace period 0 (immediate deletion). The Pod disappears from the API and is no longer visible from the client.
By default, all deletes are graceful within 30 seconds. The kubectl delete command supports the --grace-period=<seconds> option which allows a user to override the default and specify their own value. The value 0 force deletes the Pod. You must specify an additional flag --force along with --grace-period=0 in order to perform force deletions.

Force deletion of pods
Force deletion of a Pod is defined as deletion of a Pod from the cluster state and etcd immediately. When a force deletion is performed, the API server does not wait for confirmation from the kubelet that the Pod has been terminated on the node it was running on. It removes the Pod in the API immediately so a new Pod can be created with the same name. On the node, Pods that are set to terminate immediately will still be given a small grace period before being force killed.

Force deletions can be potentially dangerous for some Pods and should be performed with caution. In case of StatefulSet Pods, please refer to the task documentation for deleting Pods from a StatefulSet.

 

 

參考文檔:

https://www.cnblogs.com/alexyuyu/articles/3853583.html

https://kubernetes.io/docs/concepts/workloads/pods/pod/#termination-of-pods

https://dzone.com/articles/gracefully-shutting-down-java-in-containers

https://www.jianshu.com/p/6d393cbb694ahttps://mp.weixin.qq.com/s?__biz=MzA5OTAyNzQ2OA==&mid=2649700264&idx=1&sn=f987b01eaffd5a9d6b0e899977053111&chksm=889308cbbfe481dde7228f3ce4a4088c22676b8c9db79b12948f9e5494ce829f5707cbc848e6&mpshare=1&scene=1&srcid=1015dM3AYEe6GhbKHLvAbkrV&sharer_sharetime=1571126439719&sharer_shareid=bc0f20da14de543a8c3c8d489c80ea9f&pass_ticket=VAE837i63Ws%2BsH38kXkOzNwIoYxC5%2BmfGevNFUitY%2F6DfvKJX7AkkyhyvFM0eJS%2B#rd


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM