Jenkins+Ansible+Gitlab：通過curl自動推送文件&gitlab自動觸發構建

Jenkins通過curl自動推送文件&gitlab自動觸發構建

需求

通過url參數，自動將gitlab文件推送到指定環境的機器目錄下

可用變量

http://server/env-vars.html

#執行腳本，查看輸出內容
echo $BRANCH_NAME
#For a multibranch project, this will be set to the name of the branch being built, for example in case you wish to deploy to production from master but not from feature branches; if corresponding to some kind of change request, the name is generally arbitrary (refer to CHANGE_ID and CHANGE_TARGET).
echo $CHANGE_ID
#For a multibranch project corresponding to some kind of change request, this will be set to the change ID, such as a pull request number, if supported; else unset.
echo $CHANGE_URL
#For a multibranch project corresponding to some kind of change request, this will be set to the change URL, if supported; else unset.
echo $CHANGE_TITLE
#For a multibranch project corresponding to some kind of change request, this will be set to the title of the change, if supported; else unset.
echo $CHANGE_AUTHOR
#For a multibranch project corresponding to some kind of change request, this will be set to the username of the author of the proposed change, if supported; else unset.
echo $CHANGE_AUTHOR_DISPLAY_NAME
#For a multibranch project corresponding to some kind of change request, this will be set to the human name of the author, if supported; else unset.
echo $CHANGE_AUTHOR_EMAIL
#For a multibranch project corresponding to some kind of change request, this will be set to the email address of the author, if supported; else unset.
echo $CHANGE_TARGET
#For a multibranch project corresponding to some kind of change request, this will be set to the target or base branch to which the change could be merged, if supported; else unset.
echo $CHANGE_BRANCH
#For a multibranch project corresponding to some kind of change request, this will be set to the name of the actual head on the source control system which may or may not be different from BRANCH_NAME. For example in GitHub or Bitbucket this would have the name of the origin branch whereas BRANCH_NAME would be something like PR-24.
echo $CHANGE_FORK
#For a multibranch project corresponding to some kind of change request, this will be set to the name of the forked repo if the change originates from one; else unset.
echo $BUILD_NUMBER
#The current build number, such as "153"
echo $BUILD_ID
#The current build ID, identical to BUILD_NUMBER for builds created in 1.597+, but a YYYY-MM-DD_hh-mm-ss timestamp for older builds
echo $BUILD_DISPLAY_NAME
#The display name of the current build, which is something like "#153" by default.
echo $JOB_NAME
#Name of the project of this build, such as "foo" or "foo/bar".
echo $JOB_BASE_NAME
#Short Name of the project of this build stripping off folder paths, such as "foo" for "bar/foo".
echo $BUILD_TAG
#String of "jenkins-${JOB_NAME}-${BUILD_NUMBER}". All forward slashes ("/") in the JOB_NAME are replaced with dashes ("-"). Convenient to put into a resource file, a jar file, etc for easier identification.
echo $EXECUTOR_NUMBER
#The unique number that identifies the current executor (among executors of the same machine) that’s carrying out this build. This is the number you see in the "build executor status", except that the number starts from 0, not 1.
echo $NODE_NAME
#Name of the agent if the build is on an agent, or "master" if run on master
echo $NODE_LABELS
#Whitespace-separated list of labels that the node is assigned.
echo $WORKSPACE
#The absolute path of the directory assigned to the build as a workspace.
echo $JENKINS_HOME
#The absolute path of the directory assigned on the master node for Jenkins to store data.
echo $JENKINS_URL
#Full URL of Jenkins, like http://server:port/jenkins/ (note: only available if Jenkins URL set in system configuration)
echo $BUILD_URL
#Full URL of this build, like http://server:port/jenkins/job/foo/15/ (Jenkins URL must be set)
echo $JOB_URL
#Full URL of this job, like http://server:port/jenkins/job/foo/ (Jenkins URL must be set)
echo $GIT_COMMIT
#The commit hash being checked out.
echo $GIT_PREVIOUS_COMMIT
#The hash of the commit last built on this branch, if any.
echo $GIT_PREVIOUS_SUCCESSFUL_COMMIT
#The hash of the commit last successfully built on this branch, if any.
echo $GIT_BRANCH
#The remote branch name, if any.
echo $GIT_LOCAL_BRANCH
#The local branch name being checked out, if applicable.
echo $GIT_CHECKOUT_DIR
#The directory that the repository will be checked out to. This contains the value set in Checkout to a sub-directory, if used.
echo $GIT_URL
#The remote URL. If there are multiple, will be GIT_URL_1, GIT_URL_2, etc.
echo $GIT_COMMITTER_NAME
#The configured Git committer name, if any, that will be used for FUTURE commits from the current workspace. It is read from the Global Config user.name Value field of the Jenkins Configure System page.
echo $GIT_AUTHOR_NAME
#The configured Git author name, if any, that will be used for FUTURE commits from the current workspace. It is read from the Global Config user.name Value field of the Jenkins Configure System page.
echo $GIT_COMMITTER_EMAIL
#The configured Git committer email, if any, that will be used for FUTURE commits from the current workspace. It is read from the Global Config user.email Value field of the Jenkins Configure System page.
echo $GIT_AUTHOR_EMAIL
#The configured Git author email, if any, that will be used for FUTURE commits from the current workspace. It is read from the Global Config user.email Value field of the Jenkins Configure System page.

工作目錄 WORKSPACE

配置

安裝jenkins、gitlab、ansible

jenkins安裝buildWithParameter插件、gitlab插件

1.創建流水線項目

2.勾選 This project is parameterized

我添加了四個參數

1.env：指定環境的主機組（prod、test）
2.path：文件路徑（我設置默認值為${WORKSPACE} 即為工作目錄，jenkins會將倉庫克隆到此目錄）
3.file：文件名
4.branch：分支名
5.dpath：目標主機目錄（我設置默認值為/tmp，確保此目錄jenkins賬戶有權限）

3.添加gitlab源碼

添加需要添加gitlab用戶名密碼

4.添加token，用於使用url訪問

5.配置構建執行的playbook

5.1.選擇要執行的playbook文件（會在${WORKSPACE}中尋找該文件，即gitlab倉庫目錄下）

5.2.填寫hosts文件，即ansible主機組，可使用本地hosts文件

[test]
10.130.97.116

[prod]
10.130.98.187

5.3.填寫需要給playbook傳遞的變量

Extra Variables，可以使用This project is parameterized中設置的變量，用${變量名}引用

5.4.gitlab倉庫創建playbook文件

playbook中可以直接使用傳遞進來的變量

6.用curl觸發jenkins

在linux中使用curl

curl http://10.130.97.94:8888/job/1111/buildWithParameters -X POST -d env=test -d token=jbc -d file=1.txt -d branch=master -d dpath=/tmp -v

-X POST 使用post
-d 傳遞參數
-v 顯示http信息

jenkins：

gitlab自動觸發構建

1.jenkins配置gitlab webhook

1.1jenkins配置

jenkins有許多觸發項可選，但是目前只有使用下列選項測試成功

Approved Merge Requests (EE-only)
Comments
Comment (regex) for triggering a build	
Allowed branches

1.2gitlab配置

2.測試

gitlab中新建一個分支，修改其中的文件

使用此方法變量需要設置默認值，否則執行playbook會出現錯誤

3.jenkins完整配置

總結

利用jenkins+ansible+gitlab實現CICD
通過可帶參數觸發實現差異性構建
利用gitlab作為版本倉庫及權限控制中心
使用ansible自動化執行

未涉及：回滾、健康檢查

可以利用git show獲取comment信息，從而實現comment中描述構建參數。

問題

1.ERROR: Error fetching remote repo 'origin'&No such device or address

報錯內容

Started by user admin
Running as SYSTEM
Building in workspace /var/lib/jenkins/workspace/1111
using credential 6de1623b-eb77-4f23-a603-ae730013d2b1
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url http://10.130.97.94:10000/root/test1.git # timeout=10
Fetching upstream changes from http://10.130.97.94:10000/root/test1.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials 80808080
 > git fetch --tags --progress http://10.130.97.94:10000/root/test1.git +refs/heads/*:refs/remotes/origin/* # timeout=10
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from http://10.130.97.94:10000/root/test1.git
	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:899)
	at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1114)
	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1145)
	at hudson.scm.SCM.checkout(SCM.java:504)
	at hudson.model.AbstractProject.checkout(AbstractProject.java:1209)
	at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
	at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
	at hudson.model.Run.execute(Run.java:1816)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --progress http://10.130.97.94:10000/root/test1.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout: 
stderr: /var/lib/jenkins/workspace/1111@tmp/jenkins-gitclient-pass6522325890806363235.sh:行3: cat: 未找到命令
error: unable to read askpass response from '/var/lib/jenkins/workspace/1111@tmp/jenkins-gitclient-pass6522325890806363235.sh'
fatal: could not read Username for 'http://10.130.97.94:10000': No such device or address

	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2372)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1985)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:80)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:563)
	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:897)
	... 11 more
ERROR: Error fetching remote repo 'origin'
Finished: FAILURE

解決方法：

在請求串中加入身份信息即可，格式為：
https://[userName]:[password]@github.com/[username]/project.git

GitLab could not read Username for 'ip地址':No such device or address

2.ansible 執行報錯：No authentication methods available

ansible 執行報錯：No authentication methods available，本地可以執行

報錯信息

[1111] $ ansible test -m copy -a "src=/var/lib/jenkins/workspace/1111/2.txt dest=/home" -f 5
10.130.97.116 | UNREACHABLE! => {
    "changed": false, 
    "msg": "No authentication methods available", 
    "unreachable": true
}
FATAL: command execution failed

解決方法一

在項目配置中，構建里的ansible選項，Credentials中添加目的主機的用戶名密碼即可

引入新問題

缺少-p

該方法暫時放棄

解決方法二

jenkins集成pipeline,ansible流程

配置/etc/sudoers

jenkins  ALL=(ALL) NOPASSWD: ALL

修改/etc/passwd

jenkins:x:990:985:Jenkins Automation Server:/var/lib/jenkins:/bin/bash

切換用戶jenkins
su - jenkins

生成ssh key

ssh-keygen

復制ssh key

ssh-copy-id ip

對方主機需要有該賬號，useradd jenkins，設置密碼，passwd jenkins。需注意目標主機jenkins權限問題。

若yml中涉及到remote_user : root，需要去掉這句話，否則會報：fatal: [10.130.97.116]: UNREACHABLE! => {"changed": false, "msg": "Failed to authenticate: Authentication failed.", "unreachable": true}。原因無發用root身份登錄。

3.skipping: no hosts matched

原因：沒有指定hosts

解決方法

或者

使用-i hosts指定hosts文件

ansible-playbook build.yml -i /etc/ansible/hosts -f 5 -e env=test -e file=321.txt -e path=/tmp