K8S中的Job和CronJob

Job

Job負責批量處理短暫的一次性任務 (short lived one-off tasks)，即僅執行一次的任務，它保證批處理任務的一個或多個Pod成功結束。

Kubernetes支持以下幾種Job：

• 非並行Job：通常創建一個Pod直至其成功結束
• 固定結束次數的Job：設置.spec.completions，創建多個Pod，直到.spec.completions個Pod成功結束
• 帶有工作隊列的並行Job：設置.spec.Parallelism但不設置.spec.completions，當所有Pod結束並且至少一個成功時，Job就認為是成功

根據.spec.completions和.spec.Parallelism的設置，可以將Job划分為以下幾種pattern：

Job類型	使用示例	行為	completions	Parallelism
一次性Job	數據庫遷移	創建一個Pod直至其成功結束	1	1
固定結束次數的Job	處理工作隊列的Pod	依次創建一個Pod運行直至completions個成功結束	2+	1
固定結束次數的並行Job	多個Pod同時處理工作隊列	依次創建多個Pod運行直至completions個成功結束	2+	2+
並行Job	多個Pod同時處理工作隊列	創建一個或多個Pod直至有一個成功結束	1	2+

Job Controller

Job Controller負責根據Job Spec創建Pod，並持續監控Pod的狀態，直至其成功結束。如果失敗，則根據restartPolicy（只支持OnFailure和Never，不支持Always）決定是否創建新的Pod再次重試任務。

Job Spec格式

• spec.template格式同Pod
• RestartPolicy僅支持Never或OnFailure
• 單個Pod時，默認Pod成功運行后Job即結束
• .spec.completions標志Job結束需要成功運行的Pod個數，默認為1
• .spec.parallelism標志並行運行的Pod的個數，默認為1
• spec.activeDeadlineSeconds標志失敗Pod的重試最大時間，超過這個時間不會繼續重試

一個簡單的例子：

[root@k8s-master mnt]# cat job.yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: pi
spec:
  template:
    metadata:
      name: pi
    spec:
      containers:
      - name: pi
        image: perl
        command: ["perl", "-Mbignum=bpi", "-wle", "print bpi(2000)"]
      restartPolicy: Never
[root@k8s-master mnt]#

[root@k8s-master mnt]# kubectl create -f  job.yaml
job.batch/pi created
[root@k8s-master mnt]# kubectl get pod
NAME                      READY   STATUS              RESTARTS   AGE
deamonset-example-vdzjp   1/1     Running             0          19m
deamonset-example-xxt2z   1/1     Running             0          20m
pi-7s4vx                  0/1     ContainerCreating   0          5s
[root@k8s-master mnt]# kubectl get pod
NAME                      READY   STATUS              RESTARTS   AGE
deamonset-example-vdzjp   1/1     Running             0          19m
deamonset-example-xxt2z   1/1     Running             0          20m
pi-7s4vx                  0/1     ContainerCreating   0          7s
[root@k8s-master mnt]# kubectl describe pod pi-7s4vx
Name:           pi-7s4vx
Namespace:      default
Priority:       0
Node:           k8s-node02/192.168.180.136
Start Time:     Mon, 23 Dec 2019 20:21:25 +0800
Labels:         controller-uid=51af6eb5-9166-49eb-9328-8f5c6bcfe18f
                job-name=pi
Annotations:    <none>
Status:         Pending
IP:
IPs:            <none>
Controlled By:  Job/pi
Containers:
  pi:
    Container ID:
    Image:         perl
    Image ID:
    Port:          <none>
    Host Port:     <none>
    Command:
      perl
      -Mbignum=bpi
      -wle
      print bpi(2000)
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-6wcrh (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  default-token-6wcrh:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-6wcrh
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason     Age        From                 Message
  ----    ------     ----       ----                 -------
  Normal  Scheduled  <unknown>  default-scheduler    Successfully assigned default/pi-7s4vx to k8s-node02
  Normal  Pulling    19s        kubelet, k8s-node02  Pulling image "perl"
[root@k8s-master mnt]# kubectl get job
NAME   COMPLETIONS   DURATION   AGE
pi     0/1           26s        26s
[root@k8s-master mnt]# kubectl get job
NAME   COMPLETIONS   DURATION   AGE
pi     0/1           31s        31s
[root@k8s-master mnt]# kubectl describe pod pi-7s4vx
Name:           pi-7s4vx
Namespace:      default
Priority:       0
Node:           k8s-node02/192.168.180.136
Start Time:     Mon, 23 Dec 2019 20:21:25 +0800
Labels:         controller-uid=51af6eb5-9166-49eb-9328-8f5c6bcfe18f
                job-name=pi
Annotations:    <none>
Status:         Pending
IP:
IPs:            <none>
Controlled By:  Job/pi
Containers:
  pi:
    Container ID:
    Image:         perl
    Image ID:
    Port:          <none>
    Host Port:     <none>
    Command:
      perl
      -Mbignum=bpi
      -wle
      print bpi(2000)
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-6wcrh (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  default-token-6wcrh:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-6wcrh
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason     Age        From                 Message
  ----    ------     ----       ----                 -------
  Normal  Scheduled  <unknown>  default-scheduler    Successfully assigned default/pi-7s4vx to k8s-node02
  Normal  Pulling    2m11s      kubelet, k8s-node02  Pulling image "perl"

[root@k8s-master mnt]# kubectl get pod
NAME                      READY   STATUS      RESTARTS   AGE
deamonset-example-vdzjp   1/1     Running     0          28m
deamonset-example-xxt2z   1/1     Running     0          29m
pi-7s4vx                  0/1     Completed   0          9m7s
[root@k8s-master mnt]# kubectl get pod -o wide
NAME                      READY   STATUS      RESTARTS   AGE     IP            NODE         NOMINATED NODE   READINESS GATES
deamonset-example-vdzjp   1/1     Running     0          28m     10.244.2.22   k8s-node01   <none>           <none>
deamonset-example-xxt2z   1/1     Running     0          29m     10.244.1.23   k8s-node02   <none>           <none>
pi-7s4vx                  0/1     Completed   0          9m15s   10.244.1.24   k8s-node02   <none>           <none>
[root@k8s-master mnt]# kubectl logs pi-7s4vx
3.141592653589793238462643383279502884197169399375105820974944592307816678316527120190914564856692346034

固定結束次數的Job示例

apiVersion: batch/v1
kind: Job
metadata:
  name: busybox
spec:
  completions: 3
  template:
    metadata:
      name: busybox
    spec:
      containers:
      - name: busybox
        image: busybox
        command: ["echo", "hello"]
      restartPolicy: Never

CronJob

CronJob即定時任務，就類似於Linux系統的crontab，在指定的時間周期運行指定的任務。在Kubernetes 1.5，使用CronJob需要開啟batch/v2alpha1 API，即–runtime-config=batch/v2alpha1。

CronJob Spec

• .spec.schedule指定任務運行周期，格式同Cron
• .spec.jobTemplate指定需要運行的任務，格式同Job
• .spec.startingDeadlineSeconds指定任務開始的截止期限
• .spec.concurrencyPolicy指定任務的並發策略，支持Allow、Forbid和Replace三個選項

[root@k8s-master mnt]# cat cronjob.yaml
apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: hello
spec:
  schedule: "*/1 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: hello
            image: busybox
            args:
            - /bin/sh
            - -c
            - date; echo Hello from the Kubernetes cluster
          restartPolicy: OnFailure
[root@k8s-master mnt]#

 

[root@k8s-master mnt]# vim cronjob.yaml
[root@k8s-master mnt]# kubectl create -f cronjob.yaml
cronjob.batch/hello created
[root@k8s-master mnt]# kubectl get cronjob
NAME    SCHEDULE      SUSPEND   ACTIVE   LAST SCHEDULE   AGE
hello   */1 * * * *   False     0        <none>          16s
[root@k8s-master mnt]# kubectl get job
NAME               COMPLETIONS   DURATION   AGE
hello-1577105220   0/1           2s         2s
pi                 1/1           5m23s      25m
[root@k8s-master mnt]# kubectl get pod
NAME                      READY   STATUS      RESTARTS   AGE
deamonset-example-vdzjp   1/1     Running     0          45m
deamonset-example-xxt2z   1/1     Running     0          47m
hello-1577105220-9vjjn    0/1     Completed   0          50s
pi-7s4vx                  0/1     Completed   0          26m
[root@k8s-master mnt]# kubectl logs pod hello-1577105220-9vjjn
Error from server (NotFound): pods "pod" not found
[root@k8s-master mnt]# kubectl logs hello-1577105220-9vjjn
Mon Dec 23 12:47:14 UTC 2019
Hello from the Kubernetes cluster
[root@k8s-master mnt]# kubectl get job -w
NAME               COMPLETIONS   DURATION   AGE
hello-1577105220   1/1           9s         2m4s
hello-1577105280   1/1           8s         64s
hello-1577105340   0/1           4s         4s
pi                 1/1           5m23s      27m
hello-1577105340   1/1           6s         6s
hello-1577105400   0/1                      0s
hello-1577105400   0/1           0s         0s
hello-1577105400   1/1           5s         5s
^Z
[1]+  已停止               kubectl get job -w
[root@k8s-master mnt]# kubectl delete -f cronjob.yaml
cronjob.batch "hello" deleted
[root@k8s-master mnt]# kubectl delete -f job.yaml
job.batch "pi" deleted
[root@k8s-master mnt]#