Docker安裝GitLab與Runner（網關），常規設置，自動化用到k8s+token

本文轉載自查看原文 2020-05-22 16:19 596 git/ linux/ k8s/ gitlab/ docker

[轉]圖文詳解k8s自動化持續集成之GitLab CI/CD

Windows里面使用Debian命令行工具完成

和Docker網絡相關的命令

查看某一個容器的網絡
docker inspect 容器ID

查看docker當前網絡
docker network ls

首先創建一個網絡（下面通過docker-compose的yml文件創建，這里只做命令的了解）
docker network create gitlab-network

將現有的容器連到創建的網絡中（每個容器都要連到這個網絡里）
docker network connect gitlab-network gitlab-server   #容器名
docker network connect gitlab-network gitlab-runner

查看網絡內的容器信息
docker network inspect gitlab-network

也可以在運行時指定鏡像
docker run --network gitlab-network 鏡像名稱

移除指定的網絡
docker network rm gitlab-network

先刪掉下面要用的bridge名稱

使用Docker Compose安裝GitLab-ce和GitLab-runner

Docker Compose で GitLab + GitLab Runner の環境を整える(Window10) 重要參考

Win10下使用Docker搭建Gitlab CI自動構建平台

docker rm & docker rmi & docker prune 的差異

docker rm ：刪除一個或多個容器
docker rmi : 刪除一個或多個鏡像
docker prune：用來刪除不再使用的 docker 對象

docker+gitlab+gitlab-runner部署(CentOs7)重要參考

Docker Compose yml for Gitlab and Gitlab Runner

SMTP settings(備用參考)

#vim /etc/gitlab/gitlab.rb

gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = "smtp.exmail.qq.com" gitlab_rails['smtp_port'] = 465 gitlab_rails['smtp_user_name'] = "developers@aaa.net" gitlab_rails['smtp_password'] = "Del43@\#$@1" gitlab_rails['smtp_domain'] = "exmail.qq.com" gitlab_rails['smtp_authentication'] = "login" gitlab_rails['smtp_enable_starttls_auto'] = true gitlab_rails['smtp_tls'] = true gitlab_rails['gitlab_email_from'] = "developers@aaa.net"

測試郵件

#gitlab-ctl reconfigure
#gitlab-rails console #Notify.test_email('315360007@qq.com', 'Message Subject', 'Message Body').deliver_now

配置本地windows的host文件：

127.0.0.1 gitlab-server
127.0.0.1 nexus3

瀏覽器訪問：http://gitlab-server:9080/ 或者 http://ip:9080/
用戶名：root，密碼：自己設置

用root用戶登錄

創建測試項目：mvcdockerrunner1

添加SSH密鑰

本地windows需要在當前登錄用戶的目錄里面找到.ssh目錄下面的****.pub文件添加公鑰，同時設置config文件

Host gitlab-server
HostName gitlab-server
User git
PreferredAuthentications publickey
IdentityFile ~/.ssh/315360007
Port 23

.ssh/config

克隆代碼

手動設置激活的Runner(很重要)

1、注冊gitlab-runner（按照提示一步一步做），熟悉了直接用下面第二種

docker exec -it gitlab-runner gitlab-runner register

2. 我們會輸入 http://gitlab.local.net:9080 或 http://ip:port 也就是我們安裝在本地的GitLab

3. Please enter the gitlab-ci token for this runner 要求輸入 gitlab-ci token

在項目的管理區域->runners中可以找到(這里注冊的是share類型runner)

4. 輸入描述,如：gitlab-runner

5. 輸入tag（留空也可以，之后可以進行編輯）

6. 選擇當遇到沒有打標簽的提交時是否會執行，我們選 true

7. 是否鎖定此runner 到當前項目，我們選 false

8. 選一個執行者 executor

這一步比較重要 (ssh, docker+machine, docker-ssh+machine, kubernetes, docker, parallels, virtualbox, docker-ssh, shell)

我們選docker

9. 選擇默認使用的鏡像： docker:stable

2、注冊gitlab-runner（直接傳參數一次完成）

# 手動設置激活的Runner
docker exec -it gitlab-runner gitlab-runner register \
  --non-interactive \
  --executor "docker" \
  --docker-image docker:stable \
  --url "http://gitlab-server:9080/" \
  --registration-token "K6BiurzDBuzx23a-mV-f" \
  --description "gitlab-runner" \
  --tag-list "docker,company" \
  --run-untagged="true" \
  --locked="false" \
  --docker-privileged="true" \
  --docker-extra-hosts "gitlab-server:172.20.0.1" \
  --docker-extra-hosts "nexus3:172.20.0.1"
 
#解釋很重要
#"docker:stable"這里之所以這樣是因為基於這個鏡像它包含了docker等工具，可以在gitlab-runner執行的.gitlab-ci.yml中有docker指令時
#而無需再安裝docker 參考地址：https://docs.gitlab.com/ee/ci/docker/using_docker_build.html

#privileged=true #使用docker-in-docker時通常為true
#extra-hosts     #設置gitlab-server的網關地址

進入 gitlab-runner的容器里

 docker exec -it runner-container-id /bin/bash

查看/etc/gitlab-runner/config.toml文件

cat /etc/gitlab-runner/config.toml
concurrent = 1
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "gitlab-runner"
  url = "http://gitlab-server:9080/"
  token = "mWpRt6ry7HimmAyt86T1"
  executor = "docker"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
    [runners.cache.azure]
  [runners.docker]
    tls_verify = false
    image = "docker:stable"
    privileged = true
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/cache"]
    extra_hosts = ["gitlab-server:172.20.0.1"]
    shm_size = 0

/etc/gitlab-runner/config.toml

五、提交項目代碼完成CI/CD

Gitlab CI/CD 打包過程報 “ERROR: Uploading artifacts to coordinator... too large” 如何解決

Uploading artifacts for successful job
00:13
Uploading artifacts...
mevdockerrunner1.tar: found 1 matching files and directories 
ERROR: Uploading artifacts as "archive" to coordinator... too large archive  id=34 responseStatus=413 Request Entity Too Large status=413 token=fhcnwRyE
FATAL: too large

GitLab管理中心里設置CI/CD持續集成和部署里最大產物大小

設置和使用Gitlab-ci變量連接Kubernetes：

 1 .kube_deploy:
 2   image: boxboat/kubectl:1.17.4
 3   variables:
 4     HELM_TOKEN: ${HELM_USER_TOKEN}
 5     K8S_API_SERVER: ${KUBE_API_ADDRESS}
 6     K8S_API_SERVER_CERT: ${KUBE_API_CA}
 7   before_script:
 8     - mkdir -p /home/alpine/.kube
 9     - cp .ci/kube-config-template /home/alpine/.kube/config
10     - sed -i "s/{{ HELM_USER_TOKEN }}/${HELM_TOKEN}/g" /home/alpine/.kube/config
11     - sed -i "s/{{ KUBE_API_ADDRESS }}/${K8S_API_SERVER}/g" /home/alpine/.kube/config
12     - sed -i "s/{{ KUBE_API_CA }}/${K8S_API_SERVER_CERT}/g" /home/alpine/.kube/config

.gitlab-ci.yml

設置Kubernetes的config文件，kube-config-template文件內容：

 1 apiVersion: v1  
 2 clusters:  
 3 - cluster:
 4     certificate-authority-data: {{ KUBE_API_CA }}
 5     server: https://{{ KUBE_API_ADDRESS }}
 6   name: helm
 7 contexts:  
 8 - context:
 9     cluster: helm
10     namespace: meshop-dev
11     user: helm
12   name: helm
13 current-context: "helm"  
14 kind: Config  
15 preferences: {}  
16 users:  
17 - name: helm
18   user:
19     token: {{ HELM_USER_TOKEN }}

.ci/kube-config-template

獲取KUBE_API_CA和HELM_USER_TOKEN的值

#kubectl get sa -A   #查看所有security account
#kubectl get sa/helm -n kube-system -o yaml #查看在某一個命名空間下的某一個security account=helm的yaml，得到secrets的name
#kubectl get secrets/helm-token-wstgs  -o yaml -n kube-system #查看在某一個命名空間下的某一個secrets的name的yaml，得到token值和ca.crt值

使用變量

設置默認分支：

設置倉庫不允許推送代碼，只能自己創建分支提交，在服務器上合並分支

git流水線克隆代碼

流水線觸發器的設置，添加一個token

設置sourceTree自定義操作，設置token=上面的值

1 curl -X POST -F token=f21cbb2d6f8c10cad915a380c49b99 -F 'ref=dev' -F 'variables[MESHOP_BUILD_ONLY]=sso' https://git.runshopstore.com/api/v4/projects/2/trigger/pipeline

sso_restore.sh

設置ci里面使用觸發器

1 workflow:
2   rules:
3     - if: $MESHOP_BUILD_ONLY != null
4       when: always
5     - if: $CI_PIPELINE_SOURCE =~ /^trigger|pipeline|web|api$/
6       when: always

.gitlab-ci.yml

合並請求必須保證編譯通過：同時刪除合並請求的分支

設置關閉倉庫，測試可以使用的項目：

設置標記：

規范化git commit信息

團隊敏捷實踐 —— 使用 semantic-release 自動管理發布版本

單個項目最簡單的自動化腳本（Config項目）

stages:
  - restore
  - compile
  - version
  - build
  - deploy

.load_environment_variables: &load_environment_variables
  - echo "init environment variables."
  - for environment in $(ls -d -1 .ci/environments/*.sh);
    do source $environment;
    done

default:
  image: docker.tidebuy.net/dotnet/sdk:5.0
  tags:
    - docker
    - company
  before_script:
    - *load_environment_variables

variables:
  # docker in docker
  DOCKER_DRIVER: "overlay2"
  DOCKER_HOST: tcp://localhost:2375
  # cache
  NUGET_PACKAGES_DIRECTORY: ".nuget"
  OBJECTS_DIRECTORY: "obj"
  SOURCE_CODE_PATH: "*/*/"



.nuget_cache:
  cache:
    key: "${CI_PROJECT_NAME}-${CI_COMMIT_REF_SLUG}"
    paths:
      - "$SOURCE_CODE_PATH$OBJECTS_DIRECTORY/project.assets.json"
      - "$SOURCE_CODE_PATH$OBJECTS_DIRECTORY/*.csproj.nuget.*"
      - "$NUGET_PACKAGES_DIRECTORY"

restore:
  stage: restore
  extends: .nuget_cache
  cache:
    policy: pull-push
  script:
    - echo "dotnet restore..."
    - dotnet restore
      $CI_PROJECT_DIR/MeShop.Config/MeShop.Config.sln
      --packages $NUGET_PACKAGES_DIRECTORY
      --runtime linux-x64
      --configfile $CI_PROJECT_DIR/.ci/nuget.config
  only:
    refs:
      - branches
      - merge_requests

compile:
  stage: compile
  before_script:
    - *load_environment_variables
  extends:
    - .nuget_cache
  cache:
    policy: pull
  script:
    - echo "compile project."
    - dotnet publish MeShop.Config/MeShop.View.Config/MeShop.View.Config.csproj --no-restore
      --runtime linux-x64 -c Release -o ${CI_PROJECT_DIR}/publish
  artifacts:
    name: "${CI_JOB_NAME}-${CI_COMMIT_REF_NAME}"
    paths:
      - publish/
    expire_in: 1 days
  only:
    refs:
      - branches
      - merge_requests

version:
  stage: version
  image: meshop/semantic-release:17.1.1
  script:
    - if [ $CI_COMMIT_REF_NAME == 'alpha' -o $CI_COMMIT_REF_NAME  == 'master' ];
      then
      semantic-release;
      fi
    - if [ ! -f ./.version ];
      then
      echo "VERSION=${MESHOP_BUILD_VERSION:-$(date +%Y%m%d%H%M%S)}">.version;
      fi
    - cat .version
  artifacts:
    paths:
      - .version
    expire_in: 1 days
  needs:
    - job: compile
      artifacts: false
  rules:
    - if: "$CI_PIPELINE_SOURCE == 'merge_request_event'"
      when: never
    - if: "$CI_COMMIT_REF_NAME == 'dev'"
      when: on_success
    - if: "$CI_COMMIT_REF_NAME == 'alpha'"
      when: on_success
    - if: "$CI_COMMIT_REF_NAME  == 'master'"
      when: on_success
    - if: "$CI_COMMIT_REF_NAME  =~ /.*-bug-fix/"
      when: on_success

build:
  stage: build
  image: "docker:stable"
  services:
    - docker:stable-dind
  variables:
    DOCKER_TLS_CERTDIR: ""
  script:
    - echo "Logging to GitLab Container Registry with CI credentials..."
    - echo "$MESHOP_BUILD_REGISTRY_PASSWORD" | docker login -u "$MESHOP_BUILD_REGISTRY_USER" --password-stdin "$MESHOP_BUILD_REGISTRY"
    - source ./.version
    - image_version="${VERSION}"
    - image_name="${MESHOP_BUILD_REGISTRY}/meshop/shop/config"
    - docker build -f .ci/Dockerfile
      --tag "$image_name:$image_version"
      .
    - docker push "$image_name:$image_version"
    - if [ $MESHOP_BUILD_LATEST = 'true' ];
      then
      docker tag $image_name:$image_version $image_name:latest &&
      docker push "$image_name:latest" ;
      fi
  needs:
    - job: version
      artifacts: true
    - job: compile
      artifacts: true
  rules:
    - if: "$CI_PIPELINE_SOURCE  == 'merge_request_event'"
      when: never
    - if: "$CI_COMMIT_REF_NAME == 'dev'"
      when: on_success
    - if: "$CI_COMMIT_REF_NAME == 'alpha'"
      when: on_success
    - if: "$CI_COMMIT_REF_NAME  == 'master'"
      when: on_success
    - if: "$CI_COMMIT_REF_NAME  =~ /.*-bug-fix/"
      when: on_success

deploy:
  stage: deploy
  image: boxboat/kubectl:1.17.4
  script:
    - mkdir -p /home/alpine/.kube
    - cp .ci/kube-config-template /home/alpine/.kube/config
    - sed -i "s/{{ HELM_USER_TOKEN }}/${HELM_USER_TOKEN}/g" /home/alpine/.kube/config
    - sed -i "s/{{ KUBE_API_ADDRESS }}/${KUBE_API_ADDRESS}/g" /home/alpine/.kube/config
    - sed -i "s/{{ KUBE_API_CA }}/${KUBE_API_CA}/g" /home/alpine/.kube/config
    - kubectl rollout restart deployment/meshop-config
  rules:
    - if: "$CI_PIPELINE_SOURCE  == 'merge_request_event'"
      when: never
    - if: "$CI_COMMIT_REF_NAME == 'dev'"
      when: on_success

.gitlab-ci.yml

免責聲明！

本站轉載的文章為個人學習借鑒使用，本站對版權不負任何法律責任。如果侵犯了您的隱私權益，請聯系本站郵箱yoyou2525@163.com刪除。

猜您在找 K8s 部署 Gitlab CI Runner 使用Docker方式部署Gitlab,Gitlab-Runner並使用Gitlab提供的CI/CD功能自動化構建SpringBoot項目 Docker安裝Gitlab和Gitlab-Runner並實現項目CICD CI 自動化部署方案gitlab-runner Kubernetes（k8s）之jeckins自動化構建docker鏡像並部署到k8s集群 gitlab runner安裝與使用 gitlab-runner的安裝安裝gitlab-runner kubernetes(37)：持續集成(6)-部署gitlab-runner實現GitlabCI-k8s+gitlab+harbor gitlab + jenkins + docker + k8s