Jenkins和Gitlab CI/CD自動更新k8s中pod使用的鏡像說明

Jenkins

使用Jenkins的話，完成的工作主要有如下步驟：
1.從Gogs或Gitlab倉庫上拉取代碼
2.使用Maven編譯代碼，打包成jar文件
3.根據jar文件使用相對應的Dockerfile文件制作成Docker鏡像
4.把Docker鏡像推送到Nexus上的Docker倉庫(或者Harbor倉庫)
5.運行shell腳本，給k8s的master主機上執行更新pod的腳本命令 (這一步不想自動實現的話可以采取手動操作)

前四步的操作：
地址：https://www.cnblogs.com/sanduzxcvbnm/p/11803368.html
https://www.cnblogs.com/sanduzxcvbnm/p/11800008.html
https://www.cnblogs.com/sanduzxcvbnm/p/11799583.html
https://www.cnblogs.com/sanduzxcvbnm/p/11797660.html

第5步的操作說明：
k8s使用的是kuboard面板提供的有一個CI/CD集成功能，主要使用的就是這個

官方網址：https://kuboard.cn/guide/cicd/

直接在 shell 腳本中調用 Kuboard/Kubernetes API，更新對應容器鏡像的標簽。完成此接口調用后，Kubernetes 將自動完成滾動更新。
界面中提供的腳本里，容器的版本是當前 Kubernetes 中已部署的版本，通常在 Jenkins/GitlabRunner 之類的工具里，需要將該腳本的鏡像標簽參數化。

Gitlab Runner

使用Jenkins的話，完成的工作主要有如下步驟：
1.代碼提交到Gitlab倉庫后根據檢測到的.gitlab-ci.yml文件自動執行CI/CD操作
2.CI/CD操作
2.1使用Maven編譯代碼，打包成jar文件
2.2根據jar文件使用相對應的Dockerfile文件制作成Docker鏡像
2.3把Docker鏡像推送到Nexus上的Docker倉庫(或者Harbor倉庫) (這一步使用的變量可以在Gitlab中進行設置)
2.4運行shell腳本，給k8s的master主機上執行更新pod的腳本命令 (還有另一種辦法，詳看下面) (這一步不想自動實現的話可以采取手動操作)

.gitlab-ci.yml文件內容示例：

stages:
  - test
  - build
  - release
  - review
  - deploy

test:
  stage: test
  tags:
    - test
  script:
    - sleep 3
    - echo "We did it! Something else runs in parallel!"

compile:
  image: maven:3.5-jdk-8-alpine
  tags:
    - maven
  stage: build
  only:
    - dev
  script:
    - mvn clean package -Dmaven.test.skip=true
  artifacts:
    paths:
      - target/*.jar
      - target/lib

image_build:
  stage: release
  image: docker:latest
  tags:
    - docker
  services:
    - docker:dind
  variables:
    TAG: v0.9 # 鏡像版本，需要想辦法設置靈活
  script:
    - docker build -t "${CI_REGISTRY_IMAGE}:$TAG" .
    - docker rm -f test || true
    - docker login -u "${CI_REGISTRY_USER}" -p "${CI_REGISTRY_PASSWORD}" "${CI_REGISTRY}"
    - docker tag "${CI_REGISTRY_IMAGE}:$TAG" "${CI_REGISTRY}/${CI_REGISTRY_IMAGE}:$TAG"
    - docker push "${CI_REGISTRY}/${CI_REGISTRY_IMAGE}:$TAG"
  only:
    - dev

pod_update:
  stage: deploy
  tags:
    - k8s
  script:
    - echo "=============== 開始執行更新pod使用的鏡像任務  ==============="
    - pwd
    - chmod a+x ./scripts/build/update.sh
    # 這一步執行報錯：不是找不到文件就是找不到命令，原因采用的注冊runner時的docker鏡像，里面沒有curl命令，解決辦法：在這一步找一個含有curl命令的鏡像即可
    # - ./scripts/build/update.sh
    - echo "=============== 更新任務執行結束 ============================"

默認在注冊runner的時候需要填寫一個基礎的鏡像，只要使用執行器為docker類型的runner所有的操作運行都會在容器中運行。 如果全局指定了images則所有作業使用此image創建容器並在其中運行。 全局未指定image，再次查看job中是否有指定，如果有此job按照指定鏡像創建容器並運行，沒有則使用注冊runner時指定的默認鏡像。

update.sh文件內容跟Jenkins中的第五步操作一樣

Gitlab Runner上跟K8S上更新pod的另一個操作：
在.gitlab-ci.yml文件安裝kubectl工具，然后使用這個工具客戶端跟k8s進行通信，從而執行更新pod的操作

安裝kubectl工具地址：https://www.cnblogs.com/sanduzxcvbnm/p/13865238.html

示例內容如下：

image:
  name: golang:1.10.3-stretch
  entrypoint: ["/bin/sh", "-c"]

# The problem is that to be able to use go get, one needs to put
# the repository in the $GOPATH. So for example if your gitlab domain
# is mydomainperso.com, and that your repository is repos/projectname, and
# the default GOPATH being /go, then you'd need to have your
# repository in /go/src/mydomainperso.com/repos/projectname
# Thus, making a symbolic link corrects this.
before_script:
  - mkdir -p "/go/src/git.qikqiak.com/${CI_PROJECT_NAMESPACE}"
  - ln -sf "${CI_PROJECT_DIR}" "/go/src/git.qikqiak.com/${CI_PROJECT_PATH}"
  - cd "/go/src/git.qikqiak.com/${CI_PROJECT_PATH}/"

stages:
  - test
  - build
  - release
  - review
  - deploy

test:
  stage: test
  script:
    - make test

test2:
  stage: test
  script:
    - sleep 3
    - echo "We did it! Something else runs in parallel!"

compile:
  stage: build
  script:
    # Add here all the dependencies, or use glide/govendor/...
    # to get them automatically.
    - make build
  artifacts:
    paths:
      - app

image_build:
  stage: release
  image: docker:latest
  variables:
    DOCKER_DRIVER: overlay
    DOCKER_HOST: tcp://localhost:2375
  services:
    - name: docker:17.03-dind
      command: ["--insecure-registry=registry.qikqiak.com"]
  script:
    - docker info
    - docker login -u "${CI_REGISTRY_USER}" -p "${CI_REGISTRY_PASSWORD}" registry.qikqiak.com
    - docker build -t "${CI_REGISTRY_IMAGE}:latest" .
    - docker tag "${CI_REGISTRY_IMAGE}:latest" "${CI_REGISTRY_IMAGE}:${CI_COMMIT_REF_NAME}"
    - test ! -z "${CI_COMMIT_TAG}" && docker push "${CI_REGISTRY_IMAGE}:latest"
    - docker push "${CI_REGISTRY_IMAGE}:${CI_COMMIT_REF_NAME}"

deploy_review:
  image: cnych/kubectl
  stage: review
  only:
    - branches
  except:
    - tags
  environment:
    name: dev
    url: https://dev-gitlab-k8s-demo.qikqiak.com
    on_stop: stop_review
  script:
    - kubectl version
    - cd manifests/
    - sed -i "s/__CI_ENVIRONMENT_SLUG__/${CI_ENVIRONMENT_SLUG}/" deployment.yaml ingress.yaml service.yaml
    - sed -i "s/__VERSION__/${CI_COMMIT_REF_NAME}/" deployment.yaml ingress.yaml service.yaml
    - |
      if kubectl apply -f deployment.yaml | grep -q unchanged; then
          echo "=> Patching deployment to force image update."
          kubectl patch -f deployment.yaml -p "{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"ci-last-updated\":\"$(date +'%s')\"}}}}}"
      else
          echo "=> Deployment apply has changed the object, no need to force image update."
      fi
    - kubectl apply -f service.yaml || true
    - kubectl apply -f ingress.yaml
    - kubectl rollout status -f deployment.yaml
    - kubectl get all,ing -l ref=${CI_ENVIRONMENT_SLUG}

stop_review:
  image: cnych/kubectl
  stage: review
  variables:
    GIT_STRATEGY: none
  when: manual
  only:
    - branches
  except:
    - master
    - tags
  environment:
    name: dev
    action: stop
  script:
    - kubectl version
    - kubectl delete ing -l ref=${CI_ENVIRONMENT_SLUG}
    - kubectl delete all -l ref=${CI_ENVIRONMENT_SLUG}

deploy_live:
  image: cnych/kubectl
  stage: deploy
  environment:
    name: live
    url: https://live-gitlab-k8s-demo.qikqiak.com
  only:
    - tags
  when: manual
  script:
    - kubectl version
    - cd manifests/
    - sed -i "s/__CI_ENVIRONMENT_SLUG__/${CI_ENVIRONMENT_SLUG}/" deployment.yaml ingress.yaml service.yaml
    - sed -i "s/__VERSION__/${CI_COMMIT_REF_NAME}/" deployment.yaml ingress.yaml service.yaml
    - kubectl apply -f deployment.yaml
    - kubectl apply -f service.yaml
    - kubectl apply -f ingress.yaml
    - kubectl rollout status -f deployment.yaml
    - kubectl get all,ing -l ref=${CI_ENVIRONMENT_SLUG}