预备环境：kubesphere-v3.0，harbor，gitlab，nexus（这些环境这里不介绍安装，我这里harbor，gitlab，nexus都是单独安装的，没有部署在k8s集群中）
且kubesphere已经创建好企业空间和DevOps工程。
环境介绍
| ip | 端口 | 角色 | 软件 |
|---|---|---|---|
| 192.168.1.36 | k8s-master | k8s-1.18相关，kubesphere-v.3.0 | |
| 192.168.1.37 | k8s-node1 | k8s-1.18相关，kubesphere-v.3.0 | |
| 192.168.1.38 | k8s-node2 | k8s-1.18相关，kubesphere-v.3.0 | |
| 192.168.1.50 | 82 | 代码仓库 | gitlab-12.9.9 |
| 192.168.1.52 | 85 | 镜像仓库 | harbor-1.9.2 |
| 192.168.1.48 | 8081 | maven仓库 | nexus3.30 |
创建凭证
创建Harbor凭证
创建Gitlab凭证
创建kubeconfig凭证
导入项目
在github中fork下来kubesphere官方提供的demo：https://github.com/kubesphere/devops-java-sample
在github上生成个人token
在自己的gitlab中导入github的token,然后填入自己的token
导入刚才fork的项目到gitlab
导入后，我们只需要master分支就行了。其他的分支全部删除。
准备工作
配置流水线
创建流水线：我这里图省事，直接流水线取名叫dev
流水线配置如下：
上传java镜像到harbor
docker pull java:openjdk-8-jre-alpine
docker login -u admin -p Harbor12345 192.168.1.52:85
docker tag java:openjdk-8-jre-alpine 192.168.1.52:85/library/java:openjdk-8-jre-alpine
docker push 192.168.1.52:85/library/java:openjdk-8-jre-alpine
修改Dockerfile
修改Dockerfile-on-prem
FROM 192.168.1.52:85/library/java:openjdk-8-jre-alpine
WORKDIR /home
COPY target/*.jar /home
ENTRYPOINT java -jar *.jar
设置命名空间
修改文件：deploy/dev/devops-sample.yaml和deploy/dev/devops-sample-svc.yaml，把所有命名空间名称改为dev
修改文件：deploy/prod/devops-sample.yaml和deploy/prod/devops-sample-svc.yaml，把所有命名空间名称改为dev-prod
kubectl create namespace dev
kubectl create namespace dev-prod
修改Jenkinsfile
pipeline {
agent {
node {
label 'maven'
}
}
parameters {
string(name:'TAG_NAME',defaultValue: '',description:'')
}
environment {
HARBOR_CREDENTIAL_ID = 'harbor-id'
GITLAB_CREDENTIAL_ID = 'gitlab-id'
KUBECONFIG_CREDENTIAL_ID = 'demo-kubeconfig'
REGISTRY = '192.168.1.52:85'
HARBOR_NAMESPACE = 'library'
GITLAB_ACCOUNT = 'root'
APP_NAME = 'devops-java-sample'
}
stages {
stage ('checkout scm') {
steps {
checkout(scm)
}
}
stage ('unit test') {
steps {
container ('maven') {
sh 'mvn clean -gs `pwd`/configuration/settings.xml test'
}
}
}
stage ('build & push') {
steps {
container ('maven') {
sh 'mvn -Dmaven.test.skip=true -gs `pwd`/configuration/settings.xml clean package'
sh 'docker build -f Dockerfile-on-prem -t $REGISTRY/$HARBOR_NAMESPACE/$APP_NAME:SNAPSHOT-$BRANCH_NAME-$BUILD_NUMBER .'
withCredentials([usernamePassword(passwordVariable : 'DOCKER_PASSWORD' ,usernameVariable : 'DOCKER_USERNAME' ,credentialsId : "$HARBOR_CREDENTIAL_ID" ,)]) {
sh 'echo "$DOCKER_PASSWORD" | docker login $REGISTRY -u "$DOCKER_USERNAME" --password-stdin'
sh 'docker push $REGISTRY/$HARBOR_NAMESPACE/$APP_NAME:SNAPSHOT-$BRANCH_NAME-$BUILD_NUMBER'
}
}
}
}
stage('push latest'){
when{
branch 'master'
}
steps{
container ('maven') {
sh 'docker tag $REGISTRY/$HARBOR_NAMESPACE/$APP_NAME:SNAPSHOT-$BRANCH_NAME-$BUILD_NUMBER $REGISTRY/$HARBOR_NAMESPACE/$APP_NAME:latest '
sh 'docker push $REGISTRY/$HARBOR_NAMESPACE/$APP_NAME:latest '
}
}
}
stage('deploy to dev') {
when{
branch 'master'
}
steps {
input(id: 'deploy-to-dev', message: 'deploy to dev?')
kubernetesDeploy(configs: 'deploy/dev/**', enableConfigSubstitution: true, kubeconfigId: "$KUBECONFIG_CREDENTIAL_ID")
}
}
stage('push with tag'){
when{
expression{
return params.TAG_NAME =~ /v.*/
}
}
steps {
container ('maven') {
input(id: 'release-image-with-tag', message: 'release image with tag?')
withCredentials([usernamePassword(credentialsId: "$GITLAB_CREDENTIAL_ID", passwordVariable: 'GIT_PASSWORD', usernameVariable: 'GIT_USERNAME')]) {
sh 'git config --global user.email "kubesphere@yunify.com" '
sh 'git config --global user.name "kubesphere" '
sh 'git tag -a $TAG_NAME -m "$TAG_NAME" '
sh 'git push http://$GIT_USERNAME:$GIT_PASSWORD@192.168.1.50:82/$GITLAB_ACCOUNT/devops-java-sample.git --tags --ipv4'
}
sh 'docker tag $REGISTRY/$HARBOR_NAMESPACE/$APP_NAME:SNAPSHOT-$BRANCH_NAME-$BUILD_NUMBER $REGISTRY/$HARBOR_NAMESPACE/$APP_NAME:$TAG_NAME '
sh 'docker push $REGISTRY/$HARBOR_NAMESPACE/$APP_NAME:$TAG_NAME '
}
}
}
stage('deploy to production') {
when{
expression{
return params.TAG_NAME =~ /v.*/
}
}
steps {
input(id: 'deploy-to-production', message: 'deploy to production?')
kubernetesDeploy(configs: 'deploy/prod/**', enableConfigSubstitution: true, kubeconfigId: "$KUBECONFIG_CREDENTIAL_ID")
}
}
}
}
修改maven配置文件
configuration/settings.xml
<?xml version="1.0" encoding="UTF-8"?>
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
<localRepository>${PWD}/artifacts/m2</localRepository>
<pluginGroups>
</pluginGroups>
<proxies>
</proxies>
<servers>
<server>
<id>nexus</id>
<username>admin</username>
<password>1234</password>
</server>
</servers>
<mirrors>
<mirror>
<id>nexus</id>
<name>nexus repository</name>
<url>http://192.168.1.48:8081/repository/group_repo/</url>
<mirrorOf>central</mirrorOf>
</mirror>
</mirrors>
<profiles>
<profile>
<id>jdk-1.8</id>
<activation>
<activeByDefault>true</activeByDefault>
<jdk>1.8</jdk>
</activation>
<properties>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
<maven.compiler.compilerVersion>1.8</maven.compiler.compilerVersion>
</properties>
</profile>
</profiles>
</settings>
修改pom文件
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<!-- Spring Boot 启动父依赖 -->
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.13.RELEASE</version>
</parent>
<packaging>jar</packaging>
<groupId>io.kubesphere.devops</groupId>
<artifactId>devops-sample</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>devops-sample :: HelloWorld Demo</name>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.13.1</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.2</version>
<configuration>
<append>true</append>
</configuration>
<executions>
<execution>
<id>agent-for-ut</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>agent-for-it</id>
<goals>
<goal>prepare-agent-integration</goal>
</goals>
</execution>
<execution>
<id>jacoco-site</id>
<phase>verify</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<fork>true</fork>
</configuration>
</plugin>
<plugin>
<groupId>org.sonarsource.scanner.maven</groupId>
<artifactId>sonar-maven-plugin</artifactId>
<version>3.6.0.1398</version>
</plugin>
</plugins>
</build>
</project>
测试
注意这里的tag_name不能和gitlab中的tag一样，否则会报错。
点击继续：
等待一段时间
kubectl get svc -n dev-prod
kubectl get svc -n dev
kubectl get pods -n dev-prod
kubectl get pods -n dev
访问30861和30961端口（部署成功）
