小白學k8s(9)-gitlab-runner實現go項目的自動化發布

• gitlab構建CI/CD
  • 准備
  • docker部署gitlab
  • 使用二進制部署gitlab-runner
  • gitlab-runner注冊
  • 配置Variables
  • 簡單先來個測試
  • 開始構建
  • 遇到的報錯

gitlab構建CI/CD

准備

docker部署gitlab

通過docker-compose啟動gitlab

version: '3'
services:
  gitlab:
    image: 'gitlab/gitlab-ce:latest'
    restart: always
    hostname: '1.1.1.1'
    environment:
      TZ: 'Asia/Shanghai'
      GITLAB_OMNIBUS_CONFIG: |
        external_url 'http://1.1.1.1:9001'
        gitlab_rails['gitlab_shell_ssh_port'] = 1022
        unicorn['port'] = 8888
        nginx['listen_port'] = 9001
    ports:
      - '9001:9001'
      - '443:443'
      - '1022:22'
    volumes:
      - ./config:/etc/gitlab
      - ./data:/var/opt/gitlab
      - ./losg:/var/log/gitlab

使用二進制部署gitlab-runner

可參考官方的安裝方式Install GitLab Runner manually on GNU/Linux

gitlab-runner注冊

安裝完成之后使用注冊命令注冊

$ gitlab-runner register

然后會提示輸入gitlab的地址以及token信息

地址信息和token我們在下面可以看到

根據提示輸入信息,需要注意的是里面的tags就是我們編寫.gitlab-ci.yml對應填寫的tag

之后在gitlab的runner中就可以看到我們注冊的gitlab-runner了

配置Variables

在gitlab中可以配置我們gitlab-runner需要的變量，比如我們的docker-hub的密碼，gitlab的賬號密碼等信息

簡單先來個測試

先來個簡單的gitlab-ci.yml測試下

stages:
  - test
  - build

variables:
  GOPROXY: https://goproxy.cn

lint:
  stage: test
  script:
    - echo "hello world lint"
  only:
    - branches
  tags:
    - golang-runner

test:
  stage: test
  script:
    - echo "hello world test"
  only:
    - branches
  cache:
    key: "bazel"
    paths:
      - .cache
  tags:
    - golang-runner

開始構建

通過helm和bazel實現在gitlab-runner中k8s應用的自動編譯，發布。

鏡像推送到docker-hub中,gitlab-runner中的helm需要配置好，這里我是用了helm默認初始化的charts結構來發布應用

gitlab-ci.yml

stages:
  - test
  - build
  - deploy

variables:
  GOPROXY: https://goproxy.cn

lint:
  stage: test
  script:
    - export GO_PROJECT_PATH="/home/gitlab-runner/goWork/src"
    - mkdir -p $GO_PROJECT_PATH
    - ln -s $(pwd) $GO_PROJECT_PATH/test
    - cd $GO_PROJECT_PATH/test
    - bash build/lint.sh
  only:
    - branches
  tags:
    - golang-runner

test:
  stage: test
  script:
    - go mod vendor
    - bash build/bazel-test.sh
  only:
    - branches
  cache:
    key: "bazel"
    paths:
      - .cache
  tags:
    - golang-runner

build:
  stage: build
  before_script:
    - url_host=`git remote get-url origin | sed -e "s/http:\/\/gitlab-ci-token:.*@//g"`
    - git remote set-url origin "http://$GIT_ACCESS_USER:$GIT_ACCESS_PASSWORD@${url_host}"
    - git config user.name $GIT_ACCESS_USER
    - git config user.email $GIT_ACCESS_EMAIL
    - git fetch --tags --force
  script:
    - docker login -u $DOCKER_ACCESS_USER -p $DOCKER_ACCESS_PASSWORD
    - go mod vendor
    - bash build/bazel-build.sh
  only:
    - master
  cache:
    key: "bazel"
    paths:
      - .cache
  tags:
    - golang-runner



include: '/.gitlab/deploy.yaml'

鏡像打包

test::util:build_docker_images() {
  local docker_registry=$1
  local docker_tag=$2
  local base_image="alpine:3.7"

  query=$(test::util::find_changes)

  if [ "$query" == "" ]; then
    test::util::log "no change and exit..."
    exit 0
  fi


  for b in ${query}; do
    b=${b//\/\/src/"/src"}

    if [[ $b == *test* ]]
        then
        continue
    fi

    local binary_file_path=$(test::util::find_binary "$b")
    local binary_name=$(test::util::get_binary_name "$b")
    local docker_build_path="dockerbuild/${binary_name}"
    local docker_file_path="${docker_build_path}/Dockerfile"
    local docker_image_tag="${docker_registry}/${binary_name}:${docker_tag}"


    test::util::log "Starting docker build for image: ${binary_name}"
    (
      rm -rf "${docker_build_path}"
      mkdir -p "${docker_build_path}"
      cp "${binary_file_path}" "${docker_build_path}/${binary_name}"
      cat <<EOF >"${docker_file_path}"
FROM ${base_image}
COPY ${binary_name} /usr/local/bin/${binary_name}
RUN mkdir /lib64 && ln -s /lib/libc.musl-x86_64.so.1 /lib64/ld-linux-x86-64.so.2
RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.ustc.edu.cn/g' /etc/apk/repositories \
  && apk update --no-cache \
ENTRYPOINT ["/usr/local/bin/${binary_name}"]
EOF
      docker build -q -t "${docker_image_tag}" "${docker_build_path}"
      docker push ${docker_image_tag}
    )


  cat <<EOF >>".gitlab/deploy.yaml"
${binary_name}:
  stage: deploy
  script:
    - bash build/deploy.sh ${docker_registry} ${binary_name} ${docker_tag}
  only:
    - tags
  when: manual
  environment:
    name: test
  tags:
    - golang-runner
EOF

done

  test::util::log "Docker builds done"
}

整體的處理思路是

1、通過bazel構建go項目。

2、構建的時候找到有改動的項目，編譯，打包鏡像，生成deploy腳本。

3、打上tag，推到gitlab中。

4、最后通過手動觸發項目的deploy，通過helm發布對應的項目到k8s中。

項目的地址gitlab-runner構建go項目

最后附上deploy的截圖

對應的項目已經部署上去了

$ kubectl get pods -n test
NAME                               READY   STATUS    RESTARTS   AGE
demo1-main-test-654fbd4df6-vlhwr   1/1     Running   0          27m
demo2-main-test-58ccfd4f8-hjcdf    1/1     Running   0          25m
demo3-main-test-6897ff9496-mhm5n   1/1     Running   0          27m

遇到的報錯

go: writing go.mod cache: mkdir /home/goWork: permission denied

給用戶gitlab-runner添加最用目錄的執行權限

sudo chown -R $(whoami):gitlab-runner /Users/zhushuyan/go/pkg && sudo chmod -R g+rwx /Users/zhushuyan/go/pkg

需要預備的知識點

這里主要使用到了helm和bazel

helm使用

bazel使用