K8S容器環境下GitLab-CI和GItLab Runner 部署記錄

本文轉載自查看原文 2021-03-30 10:29 1663 Docker/K8S

一、簡單介紹

GitLab-CI

GitLab CI/CD是GitLab的一部分，支持從計划到部署具有出色的用戶體驗。CI/CD是開源GitLab社區版和專有GitLab企業版的一部分。可以根據需要添加任意數量的計算節點，每個構建可以拆分為多個作業，這些作業可以在多台計算機上並行運行。
GitLab-CI輕量級，不需要復雜的安裝手段。配置簡單，與gitlab可直接適配。實時構建日志十分清晰，UI交互體驗很好。使用 YAML 進行配置，任何人都可以很方便的使用。GitLabCI 有助於DevOps人員，例如敏捷開發中，開發與運維是同一個人，最便捷的開發方式。

在大多數情況，構建項目都會占用大量的系統資源，如果讓gitlab本身來運行構建任務的話，顯然Gitlab的性能會大幅度下降。GitLab-CI最大的作用就是管理各個項目的構建狀態。因此，運行構建任務這種浪費資源的事情交給一個獨立的Gitlab Runner來做就會好很多，更重要的是Gitlab Runner 可以安裝到不同的機器上，甚至是我們本機，這樣完全就不會影響Gitlab本身了。
從GitLab8.0開始，GitLab-CI就已經集成在GitLab中，我們只需要在項目中添加一個.gitlab-ci.yaml文件，然后運行一個Runner，即可進行持續集成。

GitLab-CI：集成、開源、無縫、可擴展、更快的結果、針對交付進行了優化：

GItLab Runner

Gitlab Runner是一個開源項目，用於運行您的作業並將結果發送給gitlab。它與Gitlab CI結合使用，gitlab ci是Gitlab隨附的用於協調作用的開源持續集成服務。
Gitlab Runner是用Go編寫的，可以作為一個二進制文件運行，不需要特定於語言的要求
它皆在GNU/Linux，MacOS和Windows操作系統上運行。另外注意：如果要使用Docker，Gitlab Runner要求Docker 至少是v1.13.0版本才可以。

Kubernetes Gitlab CICD 演示圖：

二、基於Kubernetes Gitlab CICD 容器化部署記錄

Gitlab官方提供了Helm的方式在Kubernetes集群中來快速安裝，但是在使用的過程中發現Helm提供的Chart包中有很多其他額外的配置。所以這里我采用K8S自定義的方式來安裝。
Gitlab主要涉及3個應用：Redis、Postgresql、Gitlab核心程序。
本案例中使用的Gitlab-ce鏡像部署，鏡像中的Gitlab版本是13.7.4。
本案例中使用NFS作為持久化存儲方式。除此之外，還可以選擇HostPath本地持久化存儲、NAS雲端持久化存儲、Ceph分布式持久化存儲等。

注意：本示例部署所涉及到的image鏡像均導入到Harbor私有私倉（172.16.60.230）。

1）使用NFS作為持久化存儲

在NFS服務器端（172.16.60.238）創建Redis、Postgresql、Gitlab核心程序容器的持久化掛載目錄

[root@k8s-harbor01 ~]# mkdir -p /data/storage/k8s/gitlab/{postgresql,redis,gitlab}
[root@k8s-harbor01 ~]# ll /data/storage/k8s/gitlab/
total 0
drwxr-xr-x 2 root root 6 Mar 25 14:03 gitlab
drwxr-xr-x 2 root root 6 Mar 25 14:03 postgresql
drwxr-xr-x 2 root root 6 Mar 25 14:03 redis

2）部署Gitlab

可以先創建一個命名空間

[root@k8s-master01 gitlab]# kubectl create ns kube-ops

[root@k8s-master01 gitlab]# kubectl get ns|grep kube-ops
kube-ops              Active   7d18h

配置三個核心程序的容器化部署的yaml文件

[root@k8s-master01 gitlab]# pwd
/opt/k8s/k8s_project/gitlab
[root@k8s-master01 gitlab]# ll
total 12
-rw-r--r-- 1 root root 1629 Mar 25 14:05 gitlab-postgresql.yaml
-rw-r--r-- 1 root root 1207 Mar 25 14:05 gitlab-redis.yaml
-rw-r--r-- 1 root root 2691 Mar 25 14:05 gitlab.yaml

gitlab-postgresql.yaml 文件內容：

[root@k8s-master01 gitlab]# cat gitlab-postgresql.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgresql
  namespace: kube-ops
  labels:
    name: postgresql
spec:
  replicas: 1
  selector:
    matchLabels:
      name: postgresql
  template:
    metadata:
      name: postgresql
      labels:
        name: postgresql
    spec:
      containers:
      - name: postgresql
        image: 172.16.60.230/gitlab/postgresql:v1
        imagePullPolicy: IfNotPresent
        env:
        - name: DB_USER
          value: gitlab
        - name: DB_PASS
          value: passw0rd
        - name: DB_NAME
          value: gitlab_production
        - name: DB_EXTENSION
          value: pg_trgm
        ports:
        - name: postgres
          containerPort: 5432
        volumeMounts:
        - mountPath: /var/lib/postgresql
          name: data
        livenessProbe:
          exec:
            command:
            - pg_isready
            - -h
            - localhost
            - -U
            - postgres
          initialDelaySeconds: 30
          timeoutSeconds: 5
        readinessProbe:
          exec:
            command:
            - pg_isready
            - -h
            - localhost
            - -U
            - postgres
          initialDelaySeconds: 5
          timeoutSeconds: 1
      volumes:
      - name: data
        nfs:
          server: 172.16.60.238
          path: /data/storage/k8s/gitlab/postgresql
          readOnly: false

---
apiVersion: v1
kind: Service
metadata:
  name: postgresql
  namespace: kube-ops
  labels:
    name: postgresql
spec:
  ports:
    - name: postgres
      port: 5432
      targetPort: postgres
  selector:
    name: postgresql

gitlab-redis.yaml 文件內容：

[root@k8s-master01 gitlab]# cat gitlab-redis.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis
  namespace: kube-ops
  labels:
    name: redis
spec:
  replicas: 1
  selector:
    matchLabels:
      name: redis
  template:
    metadata:
      name: redis
      labels:
        name: redis
    spec:
      containers:
      - name: redis
        image: 172.16.60.230/gitlab/redis:latest
        imagePullPolicy: IfNotPresent
        ports:
        - name: redis
          containerPort: 6379
        volumeMounts:
        - mountPath: /var/lib/redis
          name: data
        livenessProbe:
          exec:
            command:
            - redis-cli
            - ping
          initialDelaySeconds: 30
          timeoutSeconds: 5
        readinessProbe:
          exec:
            command:
            - redis-cli
            - ping
          initialDelaySeconds: 5
          timeoutSeconds: 1
      volumes:
      - name: data
        nfs:
          server: 172.16.60.238
          path: /data/storage/k8s/gitlab/redis
          readOnly: false

---
apiVersion: v1
kind: Service
metadata:
  name: redis
  namespace: kube-ops
  labels:
    name: redis
spec:
  ports:
    - name: redis
      port: 6379
      targetPort: redis
  selector:
    name: redis

gitlab.yaml 文件內容：

今天的極致任務：
把所有的面試題收集完成
全力補到Python


[root@k8s-master01 gitlab]# cat gitlab.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: gitlab
  namespace: kube-ops
  labels:
    name: gitlab
spec:
  replicas: 1
  selector:
    matchLabels:
      name: gitlab
  template:
    metadata:
      name: gitlab
      labels:
        name: gitlab
    spec:
      containers:
      - name: gitlab
        image: 172.16.60.230/gitlab/gitlab-ce:latest
        imagePullPolicy: IfNotPresent
        env:
        - name: TZ
          value: Asia/Shanghai
        - name: GITLAB_TIMEZONE
          value: Beijing
        - name: GITLAB_SECRETS_DB_KEY_BASE
          value: long-and-random-alpha-numeric-string
        - name: GITLAB_SECRETS_SECRET_KEY_BASE
          value: long-and-random-alpha-numeric-string
        - name: GITLAB_SECRETS_OTP_KEY_BASE
          value: long-and-random-alpha-numeric-string
        - name: GITLAB_ROOT_PASSWORD
          value: admin321
        - name: GITLAB_ROOT_EMAIL
          value: 1025337607@qq.com
        - name: GITLAB_HOST
          value: 0.0.0.0:30004
        - name: GITLAB_PORT
          value: "80"
        - name: GITLAB_SSH_PORT
          value: "22"
        - name: GITLAB_NOTIFY_ON_BROKEN_BUILDS
          value: "true"
        - name: GITLAB_NOTIFY_PUSHER
          value: "false"
        - name: GITLAB_BACKUP_SCHEDULE
          value: daily
        - name: GITLAB_BACKUP_TIME
          value: 01:00
        - name: DB_TYPE
          value: postgres
        - name: DB_HOST
          value: postgresql
        - name: DB_PORT
          value: "5432"
        - name: DB_USER
          value: gitlab
        - name: DB_PASS
          value: passw0rd
        - name: DB_NAME
          value: gitlab_production
        - name: REDIS_HOST
          value: redis
        - name: REDIS_PORT
          value: "6379"
        ports:
        - name: http
          containerPort: 80
        - name: ssh
          containerPort: 22
        volumeMounts:
        - mountPath: /home/git/data
          name: data
        livenessProbe:
          httpGet:
            path: /
            port: 80
          initialDelaySeconds: 180
          timeoutSeconds: 5
        readinessProbe:
          httpGet:
            path: /
            port: 80
          initialDelaySeconds: 5
          timeoutSeconds: 1
      volumes:
      - name: data
        nfs:
          server: 172.16.60.238
          path: /data/storage/k8s/gitlab/gitlab
          readOnly: false

---
apiVersion: v1
kind: Service
metadata:
  name: gitlab
  namespace: kube-ops
  labels:
    name: gitlab
spec:
  type: NodePort
  ports:
    - name: http
      port: 80
      targetPort: http
      nodePort: 30004
    - name: ssh
      port: 22
      targetPort: ssh
  selector:
    name: gitlab

創建並啟動gitlab相關容器進程：

[root@k8s-master01 gitlab]# ll
total 12
-rw-r--r-- 1 root root 1629 Mar 25 14:05 gitlab-postgresql.yaml
-rw-r--r-- 1 root root 1207 Mar 25 14:05 gitlab-redis.yaml
-rw-r--r-- 1 root root 2691 Mar 25 14:05 gitlab.yaml

[root@k8s-master01 gitlab]# kubectl apply -f .
deployment.apps/postgresql created
service/postgresql created
deployment.apps/redis created
service/redis created
deployment.apps/gitlab created
service/gitlab created

稍微等一會兒（由於程序啟動順序原因，pod可能會出現重啟次數，不過最終都會啟動成功），

查看pod狀態：

[root@k8s-master01 gitlab]# kubectl get pods -n kube-ops
NAME                          READY   STATUS    RESTARTS   AGE
gitlab-5b887894d5-ntxzj       1/1     Running   1          38m
postgresql-57bf98cdf8-7mdh9   1/1     Running   1          38m
redis-56769dc6b6-c4rnq        1/1     Running   0          38m

查看svc：

[root@k8s-master01 gitlab]# kubectl get svc -n kube-ops
NAME         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                     AGE
gitlab       NodePort    10.254.48.72   <none>        80:30004/TCP,22:32280/TCP   14m
postgresql   ClusterIP   10.254.88.39   <none>        5432/TCP                    14m
redis        ClusterIP   10.254.198.0   <none>        6379/TCP

3）訪問 Gitlab

這里采用NodePort的方式，通過 http://任意node節點ip:30004 地址訪問Gitlab
Gitlab登錄用戶名：root，密碼：admin321
Gitlab登錄密碼可以在yaml文件里修改

這里容器化部署后的Gitlab版本是13.7.4

4）創建演示項目

接下來順便創建一個項目，用於演示：

接下來在服務器上git clone，進行代碼提交演示：

git clone地址是 http://gitlab-5b887894d5-ntxzj/root/kevin-test.git

地址中的gitlab-5b887894d5-ntxzj是pod名稱，在容器外部訪問不了，需要修改為對應的nodeport地址，故git clone地址可以是： http://172.16.60.234:30004/root/kevin-test.git

在其中一個node節點上進行代碼提交演示：

[root@k8s-node02 mnt]# mkdir /mnt/haha
[root@k8s-node02 mnt]# cd /mnt/haha
[root@k8s-node02 haha]# git config --global user.email "1025337607@qq.com"
[root@k8s-node02 haha]# git config --global user.name "Administrator"
[root@k8s-node02 haha]#

[root@k8s-node02 haha]# git clone http://172.16.60.234:30004/root/kevin-test.git
Cloning into 'kevin-test'...
Username for 'http://172.16.60.234:30004': root            #輸入賬號root
Password for 'http://root@172.16.60.234:30004':            #輸入賬號root的密碼
warning: You appear to have cloned an empty repository.
[root@k8s-node02 haha]# ls
kevin-test
[root@k8s-node02 haha]# cd kevin-test/
[root@k8s-node02 kevin-test]# ll
total 0
[root@k8s-node02 kevin-test]#

[root@k8s-node02 kevin-test]# touch test.md
[root@k8s-node02 kevin-test]# echo "come on" > test.md
[root@k8s-node02 kevin-test]# git add test.md
[root@k8s-node02 kevin-test]# git commit -m "add test.md"
[master (root-commit) 8ccda29] add test.md
 1 file changed, 1 insertion(+)
 create mode 100644 test.md

 [root@k8s-node02 kevin-test]# git commit -m "add test.md"
[master (root-commit) 8ccda29] add test.md
 1 file changed, 1 insertion(+)
 create mode 100644 test.md
[root@k8s-node02 kevin-test]# git push -u origin master
Username for 'http://172.16.60.234:30004': root
Password for 'http://root@172.16.60.234:30004':
Counting objects: 3, done.
Writing objects: 100% (3/3), 216 bytes | 0 bytes/s, done.
Total 3 (delta 0), reused 0 (delta 0)
To http://172.16.60.234:30004/root/kevin-test.git
 * [new branch]      master -> master
Branch master set up to track remote branch master from origin.

如何解決 "每次輸入用戶名和密碼" 的問題？

在代碼目錄.git/config文件內[remote "origin"]的url的gitlab域名前添加gitlab注冊時的"用戶名:密碼@"

[root@k8s-node02 kevin-test]# cat .git/config
[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
[remote "origin"]
        url = http://root:admin321@172.16.60.234:30004/root/kevin-test.git
        fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
        remote = origin
        merge = refs/heads/master

接着再次嘗試提交內容，就不需要手動輸入用戶名和密碼了：

[root@k8s-node02 kevin-test]# git pull
Already up-to-date.

[root@k8s-node02 kevin-test]# echo "this is gitlab test" > test.md
[root@k8s-node02 kevin-test]# git add test.md
[root@k8s-node02 kevin-test]# git commit -m "modified test.md"
[master f2fbb27] modified test.md
 1 file changed, 1 insertion(+), 1 deletion(-)
 
[root@k8s-node02 kevin-test]# git push -u origin master
Counting objects: 12, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (10/10), 845 bytes | 0 bytes/s, done.
Total 10 (delta 1), reused 0 (delta 0)
To http://root:admin321@172.16.60.234:30004/root/kevin-test.git
   fe40316..f2fbb27  master -> master
Branch master set up to track remote branch master from origin.

代碼上傳后，gitlab上展示效果如下：

進入該項目下，左側欄CICD里有三種方式：Pipelines、Jobs、Schedules

5）Gitlab Runner 安裝和注冊

gitlab runner支持多種方式安裝，我這里就采取在k8s中安裝。

官方文檔地址: https://docs.gitlab.com/runner/install/

打開gitlab，如下圖所示，左邊代表runner狀態，右邊是配置runner信息。

注意右邊欄的token信息，后面注冊runner的時候會用到：

接下來進行配置gitlab runner資源清單 (runner-configmap.yaml)

[root@k8s-master01 gitlab]# cat runner-configmap.yaml
apiVersion: v1
data:
  REGISTER_NON_INTERACTIVE: "true"
  REGISTER_LOCKED: "false"
  METRICS_SERVER: "0.0.0.0:9100"
  CI_SERVER_URL: "http://gitlab.kube-ops.svc.cluster.local/ci"
  RUNNER_REQUEST_CONCURRENCY: "4"
  RUNNER_EXECUTOR: "kubernetes"
  KUBERNETES_NAMESPACE: "kube-ops"
  KUBERNETES_PRIVILEGED: "true"
  KUBERNETES_CPU_LIMIT: "1"
  KUBERNETES_CPU_REQUEST: "500m"
  KUBERNETES_MEMORY_LIMIT: "1Gi"
  KUBERNETES_SERVICE_CPU_LIMIT: "1"
  KUBERNETES_SERVICE_MEMORY_LIMIT: "1Gi"
  KUBERNETES_HELPER_CPU_LIMIT: "500m"
  KUBERNETES_HELPER_MEMORY_LIMIT: "100Mi"
  KUBERNETES_PULL_POLICY: "if-not-present"
  KUBERNETES_TERMINATIONGRACEPERIODSECONDS: "10"
  KUBERNETES_POLL_INTERVAL: "5"
  KUBERNETES_POLL_TIMEOUT: "360"
kind: ConfigMap
metadata:
  labels:
    app: gitlab-ci-runner
  name: gitlab-ci-runner-cm
  namespace: kube-ops

需要注意：

CI_SERVER_URL 這個地址是gitlab的地址，如果gitlab在宿主機直接寫宿主機的ip即可，容器是格式為：svc名稱.命名空間.svc.cluster.local (如果都按照我的文檔來進行安裝不需要修改別的配置了)。
如果定義的gitlab域名並不是通過外網DNS解析，而是通過/etc/hosts進行映射，那么我們需要在Runner的Pod中去添加對應的hosts，需要通過--pre-clone-script參數來指定一段腳本來添加hosts信息，也就是在ConfigMap中添加環境變量RUNNER_PRE_CLONE_SCRIPT的值：

本案例，這里gitlab地址我是使用node節點的ip+port方式。如果使用gitlab域名方式，且不是外網DNS解析，比如域名地址是http://gitlab.kevin.com 則需要在上面的ConfigMap中添加環境變量RUNNER_PRE_CLONE_SCRIPT的值: RUNNER_PRE_CLONE_SCRIPT = "echo 'xx.xx.xxx.xx git.i4t.com' >> /etc/hosts" 其中xx.xx.xxx.xx 為node節點ip地址

另外記住：在ConfigMap添加新選項后，需要刪除Gitlab ci Runner Pod

因為這里我是使用envFrom來注入上面的這些環境變量而不是直接使用env(envfrom 通過將環境變量放置到ConfigMaps或Secrets來幫助減小清單文件)

如果我們想添加其他選項，那么可以在等到后面的gitlab-ci-runner的Pod容器啟動成功后，登錄gitlab-ci-runner的pod容器內部運行gitlab-ci-multi-runner register --help 命令來查看所有可使用的選項，只需要為配置的標志添加env變量即可：

gitlab-runner@gitlab-ci-runner-0:/$ gitlab-ci-multi-runner register --help
[...]
--kubernetes-cpu-limit value                          The CPU allocation given to build containers (default: "1") [$KUBERNETES_CPU_LIMIT]
--kubernetes-memory-limit value                       The amount of memory allocated to build containers (default: "4Gi") [$KUBERNETES_MEMORY_LIMIT]
--kubernetes-service-cpu-limit value                  The CPU allocation given to build service containers (default: "1") [$KUBERNETES_SERVICE_CPU_LIMIT]
--kubernetes-service-memory-limit value               The amount of memory allocated to build service containers (default: "1Gi") [$KUBERNETES_SERVICE_MEMORY_LIMIT]
--kubernetes-helper-cpu-limit value                   The CPU allocation given to build helper containers (default: "500m") [$KUBERNETES_HELPER_CPU_LIMIT]
--kubernetes-helper-memory-limit value                The amount of memory allocated to build helper containers (default: "3Gi") [$KUBERNETES_HELPER_MEMORY_LIMIT]
--kubernetes-cpu-request value                        The CPU allocation requested for build containers [$KUBERNETES_CPU_REQUEST]
...
--pre-clone-script value                              Runner-specific command script executed before code is pulled [$RUNNER_PRE_CLONE_SCRIPT]
[...]

創建資源清單的configmap

[root@k8s-master01 gitlab]# kubectl apply -f runner-configmap.yaml
configmap/gitlab-ci-runner-cm created
[root@k8s-master01 gitlab]# kubectl get configmaps  -n kube-ops
NAME                  DATA   AGE
gitlab-ci-runner-cm   19     4s

可通過下面命令來查看此configmap內容：
[root@k8s-master01 gitlab]# kubectl describe cm gitlab-ci-runner-cm -n kube-ops

此時，還需要配置一個用於注冊、運行和取消gitlab ci runner的小腳本。只有當Pod正常通過K8S (TERM信號)的終止流程時，才會觸發注銷注冊。如果強行終止Pod（SIGKILL信號），Runner將不會自己注銷自身。必須手動完成對這種Runner的清理 (注意：只有如這里在k8s集群里安裝GitLan Runner才這樣操作，二進制安裝非K8s上安裝則不受這個影響)

[root@k8s-master01 gitlab]# cat runner-scripts-cm.yaml
apiVersion: v1
data:
  run.sh: |
    #!/bin/bash
    unregister() {
        kill %1
        echo "Unregistering runner ${RUNNER_NAME} ..."
        /usr/bin/gitlab-ci-multi-runner unregister -t "$(/usr/bin/gitlab-ci-multi-runner list 2>&1 | tail -n1 | awk '{print $4}' | cut -d'=' -f2)" -n ${RUNNER_NAME}
        exit $?
    }
    trap 'unregister' EXIT HUP INT QUIT PIPE TERM
    echo "Registering runner ${RUNNER_NAME} ..."
    /usr/bin/gitlab-ci-multi-runner register -r ${GITLAB_CI_TOKEN}
    sed -i 's/^concurrent.*/concurrent = '"${RUNNER_REQUEST_CONCURRENCY}"'/' /home/gitlab-runner/.gitlab-runner/config.toml
    echo "Starting runner ${RUNNER_NAME} ..."
    /usr/bin/gitlab-ci-multi-runner run -n ${RUNNER_NAME} &
    wait
kind: ConfigMap
metadata:
  labels:
    app: gitlab-ci-runner
  name: gitlab-ci-runner-scripts
  namespace: kube-ops

創建此腳本的configmap

[root@k8s-master01 gitlab]# kubectl apply -f runner-scripts-cm.yaml
configmap/gitlab-ci-runner-scripts created

[root@k8s-master01 gitlab]# kubectl get configmaps  -n kube-ops
NAME                       DATA   AGE
gitlab-ci-runner-cm        19     4m12s
gitlab-ci-runner-scripts   1      11s

接着需要創建一個GITLAB_CI_TOKEN，然后使用gitlab ci runner token來創建一個Kubernetes secret對象。需要提前對token進行base64轉碼：

[root@k8s-master01 gitlab]# echo xWgpgrP3rSXnxvZL9oRf|base64 -w0
eFdncGdyUDNyU1hueHZaTDlvUmYK

特意注意：這里的token就是我們gitlab runner上截圖的地方,base64只有在k8s環境上需要！

登錄Gitlab，Runner右邊欄token信息如下圖：

使用上面的token創建一個Sercret對象 (gitlab-ci-token-secret.yaml)

[root@k8s-master01 gitlab]# cat gitlab-ci-token-secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: gitlab-ci-token
  namespace: kube-ops
  labels:
    app: gitlab-ci-runner
data:
  GITLAB_CI_TOKEN: eFdncGdyUDNyU1hueHZaTDlvUmYK

創建這個Secret

[root@k8s-master01 gitlab]# kubectl apply -f gitlab-ci-token-secret.yaml
secret/gitlab-ci-token created

[root@k8s-master01 gitlab]# kubectl get secret -n kube-ops|grep gitlab
gitlab-ci-token       Opaque                                1      19s

接下來創建真正運行Runner的控制器鏡像，這里使用Statefulset，在開始運行的時候，嘗試取消注冊所有的同名Runner，當節點丟失時(即NodeLost事件)，這尤其有用，然后再嘗試注冊自己並開始運行。在正常停止Pod的時候，Runner將會運行unregister命令來嘗試取消自己，所以gitlab就不能再使用這個Runner，這個則是通過kubernetes Pod生命周期中的hooks來完成的：

編譯gitlab runner的pod部署的yaml文件內容：

[root@k8s-master01 gitlab]# cat runner-statefulset.yaml
apiVersion: v1
kind: Service
metadata:
  name: gitlab-ci-runner
  namespace: kube-ops
  labels:
    app: gitlab-ci-runner
spec:
  ports:
  - port: 9100
    targetPort: 9100
    name: http-metrics
  clusterIP: None
  selector:
    app: gitlab-ci-runner
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: gitlab-ci-runner
  namespace: kube-ops
  labels:
    app: gitlab-ci-runner
spec:
  updateStrategy:
    type: RollingUpdate
  replicas: 2
  serviceName: gitlab-ci-runner
  selector:
    matchLabels:
      app: gitlab-ci-runner
  template:
    metadata:
      labels:
        app: gitlab-ci-runner
    spec:
      volumes:
      - name: gitlab-ci-runner-scripts
        projected:
          sources:
          - configMap:
              name: gitlab-ci-runner-scripts
              items:
              - key: run.sh
                path: run.sh
                mode: 0755
      serviceAccountName: gitlab-ci
      securityContext:
        runAsNonRoot: true
        runAsUser: 999
        supplementalGroups: [999]
      containers:
      - image: 172.16.60.230/gitlab/gitlab-runner:latest
        name: gitlab-ci-runner
        command:
        - /scripts/run.sh
        envFrom:
        - configMapRef:
            name: gitlab-ci-runner-cm
        - secretRef:
            name: gitlab-ci-token
        env:
        - name: RUNNER_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        ports:
        - containerPort: 9100
          name: http-metrics
          protocol: TCP
        volumeMounts:
        - name: gitlab-ci-runner-scripts
          mountPath: "/scripts"
          readOnly: true
      restartPolicy: Always

上面我們命名了一個gitlab-ci的serviceAccount，這里要新建一個rbac文件 (runner-rbac.yaml)

[root@k8s-master01 gitlab]# cat runner-rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: gitlab-ci
  namespace: kube-ops
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: gitlab-ci
  namespace: kube-ops
rules:
  - apiGroups: [""]
    resources: ["*"]
    verbs: ["*"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: gitlab-ci
  namespace: kube-ops
subjects:
  - kind: ServiceAccount
    name: gitlab-ci
    namespace: kube-ops
roleRef:
  kind: Role
  name: gitlab-ci
  apiGroup: rbac.authorization.k8s.io

創建gitlab runner

[root@k8s-master01 gitlab]# kubectl apply -f runner-rbac.yaml
serviceaccount/gitlab-ci created
role.rbac.authorization.k8s.io/gitlab-ci created
rolebinding.rbac.authorization.k8s.io/gitlab-ci created

[root@k8s-master01 gitlab]# kubectl apply -f runner-statefulset.yaml
service/gitlab-ci-runner unchanged
statefulset.apps/gitlab-ci-runner created

接下來我們檢查我們創建的資源信息：

[root@k8s-master01 gitlab]# kubectl get pod,svc,cm -n kube-ops
NAME                              READY   STATUS    RESTARTS   AGE
pod/gitlab-5b887894d5-ntxzj       1/1     Running   1          150m
pod/gitlab-ci-runner-0            1/1     Running   0          12m
pod/gitlab-ci-runner-1            1/1     Running   0          8m33s
pod/postgresql-57bf98cdf8-7mdh9   1/1     Running   1          150m
pod/redis-56769dc6b6-c4rnq        1/1     Running   0          150m

NAME                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                     AGE
service/gitlab             NodePort    10.254.252.242   <none>        80:30004/TCP,22:31455/TCP   150m
service/gitlab-ci-runner   ClusterIP   None             <none>        9100/TCP                    13m
service/postgresql         ClusterIP   10.254.173.69    <none>        5432/TCP                    150m
service/redis              ClusterIP   10.254.11.213    <none>        6379/TCP                    150m

NAME                                 DATA   AGE
configmap/gitlab-ci-runner-cm        19     36m
configmap/gitlab-ci-runner-scripts   1      32m

此時，在登錄Gitlab，查看Runner信息，發現就已經將這2個pod節點添加進來了：

這里我們也可以更改Runner的一些配置，比如添加tag標簽等：

6）Gitlab Runner 配置使用

這里上傳了一個測試項目gitlab-ci-k8s-demo-master，下面介紹：在gitlab項目里添加k8s集群（只要一個項目添加成功后，其他項目也都添加了）

另外注意：CI/CD的Auto DevOps里的"Default to Auto DevOps pipeline“ 如果有勾選，則需要去掉這個勾選，否則會對集成k8s產生影響。

接下來要配置一下kubectl，默認情況下kubectl需要使用證書才可以連接到k8s集群。在服務器上默認使用的是/root/.kube/config，gitlab 也有添加證書的位置，下面進行添加配置。

這里簡單說明下：

Kubernetes cluster name 集群名稱，這個可以隨便寫
API URL 這里實際上就是apiserver地址，通過kubectl cluster-info查看到
CA Certificate、Service Token 可以通過/root/.kube/config 或下面方式得到

Kubernetes cluster 名稱

這里順便填寫一個，比如：kevin-k8s-cluster

API URL地址

可以通過"kubectl cluster-info"命令獲取

[root@k8s-master01 gitlab]# kubectl cluster-info
Kubernetes master is running at https://127.0.0.1:8443
CoreDNS is running at https://127.0.0.1:8443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
Metrics-server is running at https://127.0.0.1:8443/api/v1/namespaces/kube-system/services/https:metrics-server:/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

由於這里的k8s集群中的kube-apiserver高可用通過local nginx+haproxy實現的，所以需要將https://127.0.0.1:8443 改為其中的一個master地址，這里填寫：https://172.16.60.231:8443

CA Certificate、Service Token 值

這里沒有使用/root/.kube/config獲取Service Token，而是采用下面的方式獲取：

這里提醒一下：cluster-admin是內置的一個角色，擁有集群所有權限的一個角色

這里先創建一個命名空間
[root@k8s-master01 gitlab]# kubectl create ns gitlab
namespace/gitlab created

因為在操作的時候會涉及rbac權限的問題，這里要創建一個rbac的文件。我們將集群的cluster的權限綁定到ServerAccount
[root@k8s-master01 gitlab]# pwd
/opt/k8s/k8s_project/gitlab

[root@k8s-master01 gitlab]# cat gitlabdemo-sa.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: gitlab
  namespace: gitlab

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: gitlab
subjects:
  - kind: ServiceAccount
    name: gitlab
    namespace: gitlab
roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: cluster-admin

進行創建
[root@k8s-master01 gitlab]# kubectl apply -f gitlabdemo-sa.yaml
serviceaccount/gitlab created
clusterrolebinding.rbac.authorization.k8s.io/gitlab created

查看
[root@k8s-master01 gitlab]# kubectl get sa -n gitlab
NAME      SECRETS   AGE
default   1         2m55s
gitlab    1         17s

上面創建的serviceaccount實際上就是一個secret，接下來我們進行獲取Service Token和CA Certificate

[root@k8s-master01 gitlab]# kubectl get secret -n gitlab
NAME                  TYPE                                  DATA   AGE
default-token-v2r6h   kubernetes.io/service-account-token   3      4m13s
gitlab-token-ddhs6    kubernetes.io/service-account-token   3      95s

[root@k8s-master01 gitlab]# kubectl get secrets gitlab-token-ddhs6 -n gitlab -o yaml
apiVersion: v1
data:
  ca.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUR5RENDQXJDZ0F3SUJBZ0lVTnI4WnAvRmxmc2xvUHJrRzdiaGNjbkYrYWVZd0RRWUpLb1pJaHZjTkFRRUwKQlFBd2FURUxNQWtHQTFVRUJoTUNRMDR4RURBT0JnTlZCQWdUQjBKbGFVcHBibWN4RURBT0JnTlZCQWNUQjBKbAphVXBwYm1jeEREQUtCZ05WQkFvVEEyczRjekVRTUE0R0ExVUVDeE1IYjNCemJuVnNiREVXTUJRR0ExVUVBeE1OCmEzVmlaWEp1WlhSbGN5MWpZVEFnRncweU1ERXhNVFV4TkRBek1EQmFHQTh5TVRJd01UQXlNakUwTURNd01Gb3cKYVRFTE1Ba0dBMVVFQmhNQ1EwNHhFREFPQmdOVkJBZ1RCMEpsYVVwcGJtY3hFREFPQmdOVkJBY1RCMEpsYVVwcApibWN4RERBS0JnTlZCQW9UQTJzNGN6RVFNQTRHQTFVRUN4TUhiM0J6Ym5Wc2JERVdNQlFHQTFVRUF4TU5hM1ZpClpYSnVaWFJsY3kxallUQ0NBU0l3RFFZSktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQU5rUmFpQ2wKM2VDNmFRSVYvUkU1OHpnajhOQTJLcTFqMmZBdW5ZNU96Q2Uza0wyTWtPRDNGUXZhZndJeXlpanU4eGtVNlVRYwprYzZ6US92a3BBTFJ5NWFCRTBQeTdpdUJTdnJwMFZMK0htbUdQSzhKcXovUlZ0RjAwemxOK1A4by8rV0ZqeExFCmFCVGtMMkkxK2tyNEFubFZhT0dJaGl6WjBWQVhmR2JWNnRVeFpCMWVkWVkzZmdlbHo0ZDRBU3MvcnB5NEpxZFYKTGErZkRvN2Z5ZEZvQ0IwS3FKN1B1VWI3c1JvVU1WR1Jnd2o0amc3ckVwWEZ6bFJBUmN5RjZneUplcXY5OWhUdQpaUUlSRVJIK3BadzJJVVNZTzF4MzZ0UlVQRHlMN0FrVzhxNU1VYVBVRkJzQ0hGM0dYeWNzdjdUSllocFAyQk84CmVwa3NFN2kxTW1LeXIvc0NBd0VBQWFObU1HUXdEZ1lEVlIwUEFRSC9CQVFEQWdFR01CSUdBMVVkRXdFQi93UUkKTUFZQkFmOENBUUl3SFFZRFZSME9CQllFRk93YStVei9LR2hDZ2RNdGlxd01xSGxyNzUyK01COEdBMVVkSXdRWQpNQmFBRk93YStVei9LR2hDZ2RNdGlxd01xSGxyNzUyK01BMEdDU3FHU0liM0RRRUJDd1VBQTRJQkFRQlh0SHJlCmNuangvNVJqYm0vbXhPNDJLWXZQTmY3NXNxNitST3JTb2pBQmRmSXlTVDlTSEpDTkFqWElvTlJHYndZWWJNSC8KMHp5WEVObGJFcDlBd2VETGNER2ZvT2ZwMTJ0UzFuQ3FWZlhhazNLWWlRRWNVQk9QMzI3bVYxU3lrOEpYWGtXawpNRUxrRDRNQWovTFlYbmh1dWdua1QxMk9VOXF1d1FURUZROTZUckNEeFpyNktzMXI4d1R1WHlqRlJudlAyZk9OCmpFQytXS3dXTm05Nk5GMHFIWWlSUHV4V2FCWWM3MXN2SWpsNW9vWWFjRnVpTlpGNEpBeFRPcVRUb0poK3crTUsKLzcrd0x4WmJHTHB3cndpYXhUZW5ZRTdpeGFyVnV0QkRTbWxlbmR3aU9FUldXV2NwWDZjdWpTSnlMTHFsRTBYNApXZHFnMmsrd3BVaEcrbGtYCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
  namespace: Z2l0bGFi
  token: ZXlKaGJHY2lPaUpTVXpJMU5pSXNJbXRwWkNJNklrWXliM0JIV0dVME9USXhjVE51VTFnd04zWm5NM2RpVEZsVFNucHlibVJ1TWtKbmMxUkhVMXB1VjFFaWZRLmV5SnBjM01pT2lKcmRXSmxjbTVsZEdWekwzTmxjblpwWTJWaFkyTnZkVzUwSWl3aWEzVmlaWEp1WlhSbGN5NXBieTl6WlhKMmFXTmxZV05qYjNWdWRDOXVZVzFsYzNCaFkyVWlPaUpuYVhSc1lXSWlMQ0pyZFdKbGNtNWxkR1Z6TG1sdkwzTmxjblpwWTJWaFkyTnZkVzUwTDNObFkzSmxkQzV1WVcxbElqb2laMmwwYkdGaUxYUnZhMlZ1TFdSa2FITTJJaXdpYTNWaVpYSnVaWFJsY3k1cGJ5OXpaWEoyYVdObFlXTmpiM1Z1ZEM5elpYSjJhV05sTFdGalkyOTFiblF1Ym1GdFpTSTZJbWRwZEd4aFlpSXNJbXQxWW1WeWJtVjBaWE11YVc4dmMyVnlkbWxqWldGalkyOTFiblF2YzJWeWRtbGpaUzFoWTJOdmRXNTBMblZwWkNJNkltSXlNV00yTW1RMExUVXpNV1F0TkdFMU55MDRNek5qTFRGbU9ETm1OakF3TkRGa1lpSXNJbk4xWWlJNkluTjVjM1JsYlRwelpYSjJhV05sWVdOamIzVnVkRHBuYVhSc1lXSTZaMmwwYkdGaUluMC5vU1ZsWGpSMk5aU3hGWWNHX2RLclJKOXljdWY4dFdkUDI1cHNSRkRoSGlsYm0zV2ZkZ09EMkdGMjAwS1JsV1hySG9nNzNybmZVOUFtakNvTWFwQ0JSVmZGdFY4TGNxOTZGeEtRcEJYbGhrU2JWaW53V2JXUXpUWEtOYmN3M3pLU2JJWkV2X09kbF96d1U1NzNYYXBWeFVORERqMEszZF9qR2xxbGg3ZGpDN1ZGb1pjRHNjR3g5UjJMZW5TVmdXY1QxX1pSZ1VVYkVHSFREQVI5Q1VwNU5sQzNJUVFBSTljQmVJcmtLeUo2blVDRi1jNGpQZlViSkVqaThMMDdtSzRNRnhNY3k4QmUwbTdHSWZGcWNIcGExc1V5VjBJb2lRSmF3Q2pwUUhvSFR6TmJkVzF3ZC1oMjIxcXJuRVRwanRrV1pMMnRZTlhfNUNTOXA3OXRZR2hJMnc=
kind: Secret
metadata:
  annotations:
    kubernetes.io/service-account.name: gitlab
    kubernetes.io/service-account.uid: b21c62d4-531d-4a57-833c-1f83f60041db
  creationTimestamp: "2021-03-25T10:38:04Z"
  managedFields:
  - apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
      f:data:
        .: {}
        f:ca.crt: {}
        f:namespace: {}
        f:token: {}
      f:metadata:
        f:annotations:
          .: {}
          f:kubernetes.io/service-account.name: {}
          f:kubernetes.io/service-account.uid: {}
      f:type: {}
    manager: kube-controller-manager
    operation: Update
    time: "2021-03-25T10:38:04Z"
  name: gitlab-token-ddhs6
  namespace: gitlab
  resourceVersion: "31458293"
  selfLink: /api/v1/namespaces/gitlab/secrets/gitlab-token-ddhs6
  uid: b460030a-616c-4713-bd36-d3dea49d2ea5
type: kubernetes.io/service-account-token

上面結果中的ca.crt內容進行base64轉碼才能得到CA Certificate

需要通過命令" echo "xxxx"|base64 -d"使用base64進行轉碼

"xxxx"是ca.crt的內容

[root@k8s-master01 gitlab]# echo "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUR5RENDQXJDZ0F3SUJBZ0lVTnI4WnAvRmxmc2xvUHJrRzdiaGNjbkYrYWVZd0RRWUpLb1pJaHZjTkFRRUwKQlFBd2FURUxNQWtHQTFVRUJoTUNRMDR4RURBT0JnTlZCQWdUQjBKbGFVcHBibWN4RURBT0JnTlZCQWNUQjBKbAphVXBwYm1jeEREQUtCZ05WQkFvVEEyczRjekVRTUE0R0ExVUVDeE1IYjNCemJuVnNiREVXTUJRR0ExVUVBeE1OCmEzVmlaWEp1WlhSbGN5MWpZVEFnRncweU1ERXhNVFV4TkRBek1EQmFHQTh5TVRJd01UQXlNakUwTURNd01Gb3cKYVRFTE1Ba0dBMVVFQmhNQ1EwNHhFREFPQmdOVkJBZ1RCMEpsYVVwcGJtY3hFREFPQmdOVkJBY1RCMEpsYVVwcApibWN4RERBS0JnTlZCQW9UQTJzNGN6RVFNQTRHQTFVRUN4TUhiM0J6Ym5Wc2JERVdNQlFHQTFVRUF4TU5hM1ZpClpYSnVaWFJsY3kxallUQ0NBU0l3RFFZSktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQU5rUmFpQ2wKM2VDNmFRSVYvUkU1OHpnajhOQTJLcTFqMmZBdW5ZNU96Q2Uza0wyTWtPRDNGUXZhZndJeXlpanU4eGtVNlVRYwprYzZ6US92a3BBTFJ5NWFCRTBQeTdpdUJTdnJwMFZMK0htbUdQSzhKcXovUlZ0RjAwemxOK1A4by8rV0ZqeExFCmFCVGtMMkkxK2tyNEFubFZhT0dJaGl6WjBWQVhmR2JWNnRVeFpCMWVkWVkzZmdlbHo0ZDRBU3MvcnB5NEpxZFYKTGErZkRvN2Z5ZEZvQ0IwS3FKN1B1VWI3c1JvVU1WR1Jnd2o0amc3ckVwWEZ6bFJBUmN5RjZneUplcXY5OWhUdQpaUUlSRVJIK3BadzJJVVNZTzF4MzZ0UlVQRHlMN0FrVzhxNU1VYVBVRkJzQ0hGM0dYeWNzdjdUSllocFAyQk84CmVwa3NFN2kxTW1LeXIvc0NBd0VBQWFObU1HUXdEZ1lEVlIwUEFRSC9CQVFEQWdFR01CSUdBMVVkRXdFQi93UUkKTUFZQkFmOENBUUl3SFFZRFZSME9CQllFRk93YStVei9LR2hDZ2RNdGlxd01xSGxyNzUyK01COEdBMVVkSXdRWQpNQmFBRk93YStVei9LR2hDZ2RNdGlxd01xSGxyNzUyK01BMEdDU3FHU0liM0RRRUJDd1VBQTRJQkFRQlh0SHJlCmNuangvNVJqYm0vbXhPNDJLWXZQTmY3NXNxNitST3JTb2pBQmRmSXlTVDlTSEpDTkFqWElvTlJHYndZWWJNSC8KMHp5WEVObGJFcDlBd2VETGNER2ZvT2ZwMTJ0UzFuQ3FWZlhhazNLWWlRRWNVQk9QMzI3bVYxU3lrOEpYWGtXawpNRUxrRDRNQWovTFlYbmh1dWdua1QxMk9VOXF1d1FURUZROTZUckNEeFpyNktzMXI4d1R1WHlqRlJudlAyZk9OCmpFQytXS3dXTm05Nk5GMHFIWWlSUHV4V2FCWWM3MXN2SWpsNW9vWWFjRnVpTlpGNEpBeFRPcVRUb0poK3crTUsKLzcrd0x4WmJHTHB3cndpYXhUZW5ZRTdpeGFyVnV0QkRTbWxlbmR3aU9FUldXV2NwWDZjdWpTSnlMTHFsRTBYNApXZHFnMmsrd3BVaEcrbGtYCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K" | base64 -d
-----BEGIN CERTIFICATE-----
MIIDyDCCArCgAwIBAgIUNr8Zp/FlfsloPrkG7bhccnF+aeYwDQYJKoZIhvcNAQEL
BQAwaTELMAkGA1UEBhMCQ04xEDAOBgNVBAgTB0JlaUppbmcxEDAOBgNVBAcTB0Jl
aUppbmcxDDAKBgNVBAoTA2s4czEQMA4GA1UECxMHb3BzbnVsbDEWMBQGA1UEAxMN
a3ViZXJuZXRlcy1jYTAgFw0yMDExMTUxNDAzMDBaGA8yMTIwMTAyMjE0MDMwMFow
aTELMAkGA1UEBhMCQ04xEDAOBgNVBAgTB0JlaUppbmcxEDAOBgNVBAcTB0JlaUpp
bmcxDDAKBgNVBAoTA2s4czEQMA4GA1UECxMHb3BzbnVsbDEWMBQGA1UEAxMNa3Vi
ZXJuZXRlcy1jYTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBANkRaiCl
3eC6aQIV/RE58zgj8NA2Kq1j2fAunY5OzCe3kL2MkOD3FQvafwIyyiju8xkU6UQc
kc6zQ/vkpALRy5aBE0Py7iuBSvrp0VL+HmmGPK8Jqz/RVtF00zlN+P8o/+WFjxLE
aBTkL2I1+kr4AnlVaOGIhizZ0VAXfGbV6tUxZB1edYY3fgelz4d4ASs/rpy4JqdV
La+fDo7fydFoCB0KqJ7PuUb7sRoUMVGRgwj4jg7rEpXFzlRARcyF6gyJeqv99hTu
ZQIRERH+pZw2IUSYO1x36tRUPDyL7AkW8q5MUaPUFBsCHF3GXycsv7TJYhpP2BO8
epksE7i1MmKyr/sCAwEAAaNmMGQwDgYDVR0PAQH/BAQDAgEGMBIGA1UdEwEB/wQI
MAYBAf8CAQIwHQYDVR0OBBYEFOwa+Uz/KGhCgdMtiqwMqHlr752+MB8GA1UdIwQY
MBaAFOwa+Uz/KGhCgdMtiqwMqHlr752+MA0GCSqGSIb3DQEBCwUAA4IBAQBXtHre
cnjx/5Rjbm/mxO42KYvPNf75sq6+ROrSojABdfIyST9SHJCNAjXIoNRGbwYYbMH/
0zyXENlbEp9AweDLcDGfoOfp12tS1nCqVfXak3KYiQEcUBOP327mV1Syk8JXXkWk
MELkD4MAj/LYXnhuugnkT12OU9quwQTEFQ96TrCDxZr6Ks1r8wTuXyjFRnvP2fON
jEC+WKwWNm96NF0qHYiRPuxWaBYc71svIjl5ooYacFuiNZF4JAxTOqTToJh+w+MK
/7+wLxZbGLpwrwiaxTenYE7ixarVutBDSmlendwiOERWWWcpX6cujSJyLLqlE0X4
Wdqg2k+wpUhG+lkX
-----END CERTIFICATE-----

上面轉碼結果中的-----BEGIN CERTIFICATE-----和 -----END CERTIFICATE-----之間的內容就是CA Certificate

運行以下命令得到Service Token輸出值：

[root@k8s-master01 gitlab]# kubectl -n gitlab describe secret gitlab-token-ddhs6
Name:         gitlab-token-ddhs6
Namespace:    gitlab
Labels:       <none>
Annotations:  kubernetes.io/service-account.name: gitlab
              kubernetes.io/service-account.uid: b21c62d4-531d-4a57-833c-1f83f60041db

Type:  kubernetes.io/service-account-token

Data
====
ca.crt:     1371 bytes
namespace:  6 bytes
token:      eyJhbGciOiJSUzI1NiIsImtpZCI6IkYyb3BHWGU0OTIxcTNuU1gwN3ZnM3diTFlTSnpybmRuMkJnc1RHU1puV1EifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJnaXRsYWIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlY3JldC5uYW1lIjoiZ2l0bGFiLXRva2VuLWRkaHM2Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImdpdGxhYiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6ImIyMWM2MmQ0LTUzMWQtNGE1Ny04MzNjLTFmODNmNjAwNDFkYiIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDpnaXRsYWI6Z2l0bGFiIn0.oSVlXjR2NZSxFYcG_dKrRJ9ycuf8tWdP25psRFDhHilbm3WfdgOD2GF200KRlWXrHog73rnfU9AmjCoMapCBRVfFtV8Lcq96FxKQpBXlhkSbVinwWbWQzTXKNbcw3zKSbIZEv_Odl_zwU573XapVxUNDDj0K3d_jGlqlh7djC7VFoZcDscGx9R2LenSVgWcT1_ZRgUUbEGHTDAR9CUp5NlC3IQQAI9cBeIrkKyJ6nUCF-c4jPfUbJEji8L07mK4MFxMcy8Be0m7GIfFqcHpa1sUyV0IoiQJawCjpQHoHTzNbdW1wd-h221qrnETpjtkWZL2tYNX_5CS9p79tYGhI2w

接着，就可以按照上面得出的幾個信息，進行添加k8s集群的操作：

點擊下面的 "Add Kubernetes cluster"按鈕，就能成功添加已有的kubernetes集成了。

如果出現下面報錯：

解決方法：

Settings => Network ，然后點擊 Outbound requests 右邊的“expand”按鈕，勾選 "Allow requests to the local network from web hooks and services"。

免責聲明！

本站轉載的文章為個人學習借鑒使用，本站對版權不負任何法律責任。如果侵犯了您的隱私權益，請聯系本站郵箱yoyou2525@163.com刪除。

猜您在找 K8s 部署 Gitlab CI Runner gitlab-ci + k8s 之gitlab-ci（一） gitlab-ci + k8s 之k8s （二）使用Gitlab-CI 實現NetCore項目Docker化並部署到阿里雲K8S gitlab之gitlab-ci和gitlab-runner<二> GitLab-CI與GitLab-Runner k8s集群部署應用----gitlab-runner gitlab之gitlab-ci自動部署 Gitlab-CI持續集成之Runner配置和CI腳本 Gitlab部署到K8S上