使用ConfigMap管理應用配置
拆分環境
| 主機名 | 角色 | ip |
|---|---|---|
| HDSS7-11.host.com | zk1.od.com(Test環境) | 10.4.7.11 |
| HDSS7-12.host.com | zk2.od.com(Prod環境) | 10.4.7.12 |
重配zookeeper
HDSS7-11.host.com上：
/opt/zookeeper/conf/zoo.cfg
1 |
tickTime=2000 |
HDSS7-12.host.com上：
/opt/zookeeper/conf/zoo.cfg
1 |
tickTime=2000 |
重啟zk(刪除數據文件)
1 |
[root@hdss7-11 ~]# /opt/zookeeper/bin/zkServer.sh restart && /opt/zookeeper/bin/zkServer.sh status |
准備資源配置清單(dubbo-monitor)
在運維主機HDSS7-200.host.com上：
vi /data/k8s-yaml/dubbo-monitor/configmap.yaml
1 |
apiVersion: v1 |
vi /data/k8s-yaml/dubbo-monitor/deployment.yaml
1 |
kind: Deployment |
應用資源配置清單
在任意一台k8s運算節點執行：
1 |
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/dubbo-monitor/configmap.yaml |
重新發版，修改dubbo項目的配置文件
修改項目源代碼
-
duboo-demo-service
dubbo-server/src/main/java/config.properties1
2dubbo.registry=zookeeper://zk1.od.com:2181
dubbo.port=28080 -
dubbo-demo-web
dubbo-client/src/main/java/config.properties1dubbo.registry=zookeeper://zk1.od.com:2181
使用Jenkins進行CI
略
修改/應用資源配置清單
k8s的dashboard上，修改deployment使用的容器版本，提交應用
驗證configmap的配置
在K8S的dashboard上，修改dubbo-monitor的configmap配置為不同的zk，重啟POD，瀏覽器打開http://dubbo-monitor.od.com 觀察效果
交付Apollo至Kubernetes集群
Apollo簡介
Apollo（阿波羅）是攜程框架部門研發的分布式配置中心，能夠集中化管理應用不同環境、不同集群的配置，配置修改后能夠實時推送到應用端，並且具備規范的權限、流程治理等特性，適用於微服務配置管理場景。
官方GitHub地址
基礎架構
簡化模型
交付apollo-configservice
准備軟件包
在運維主機HDSS7-200.host.com上：
下載官方release包
/opt/src
1 |
[root@hdss7-200 src]# ls -l|grep apollo |
執行數據庫腳本
在數據庫主機HDSS7-11.host.com上：
注意：MySQL版本應為5.6或以上！
- 更新yum源
/etc/yum.repos.d/MariaDB.repo
1 |
[mariadb] |
-
導入GPG-KEY
1[root@hdss7-11 ~]# rpm --import https://mirrors.ustc.edu.cn/mariadb/yum/RPM-GPG-KEY-MariaDB -
更新數據庫版本
1[root@hdss7-11 ~]# yum update MariaDB-server -y
1 |
[root@hdss7-11 ~]# mysql -uroot -p |
數據庫用戶授權
1 |
mysql> grant INSERT,DELETE,UPDATE,SELECT on ApolloConfigDB.* to "apolloconfig"@"10.4.7.%" identified by "123456"; |
修改初始數據
1 |
mysql> update ApolloConfigDB.ServerConfig set ServerConfig.Value="http://config.od.com/eureka" where ServerConfig.Key="eureka.service.url"; |
制作Docker鏡像
在運維主機HDSS7-200.host.com上：
-
配置數據庫連接串
/data/dockerfile/apollo-configservice1[root@hdss7-200 apollo-configservice]# cat config/application-github.properties -
更新startup.sh
/data/dockerfile/apollo-configservice/scripts/startup.sh1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63#!/bin/bash
SERVICE_NAME=apollo-configservice
## Adjust log dir if necessary
LOG_DIR=/opt/logs/apollo-config-server
## Adjust server port if necessary
SERVER_PORT=8080
APOLLO_CONFIG_SERVICE_NAME=$(hostname -i)
SERVER_URL="http://${APOLLO_CONFIG_SERVICE_NAME}:${SERVER_PORT}"
## Adjust memory settings if necessary
#export JAVA_OPTS="-Xms6144m -Xmx6144m -Xss256k -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=384m -XX:NewSize=4096m -XX:MaxNewSize=4096m -XX:SurvivorRatio=8"
## Only uncomment the following when you are using server jvm
#export JAVA_OPTS="$JAVA_OPTS -server -XX:-ReduceInitialCardMarks"
########### The following is the same for configservice, adminservice, portal ###########
export JAVA_OPTS="$JAVA_OPTS -XX:ParallelGCThreads=4 -XX:MaxTenuringThreshold=9 -XX:+DisableExplicitGC -XX:+ScavengeBeforeFullGC -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+ExplicitGCInvokesConcurrent -XX:+PrintGCDetails -XX:+HeapDumpOnOutOfMemoryError -XX:-OmitStackTraceInFastThrow -Duser.timezone=Asia/Shanghai -Dclient.encoding.override=UTF-8 -Dfile.encoding=UTF-8 -Djava.security.egd=file:/dev/./urandom"
export JAVA_OPTS="$JAVA_OPTS -Dserver.port=$SERVER_PORT -Dlogging.file=$LOG_DIR/$SERVICE_NAME.log -XX:HeapDumpPath=$LOG_DIR/HeapDumpOnOutOfMemoryError/"
# Find Java
if [[ -n "$JAVA_HOME" ]] && [[ -x "$JAVA_HOME/bin/java" ]]; then
javaexe="$JAVA_HOME/bin/java"
elif type -p java > /dev/null 2>&1; then
javaexe=$(type -p java)
elif [[ -x "/usr/bin/java" ]]; then
javaexe="/usr/bin/java"
else
echo "Unable to find Java"
exit 1
fi
if [[ "$javaexe" ]]; then
version=$("$javaexe" -version 2>&1 | awk -F '"' '/version/ {print $2}')
version=$(echo "$version" | awk -F. '{printf("%03d%03d",$1,$2);}')
# now version is of format 009003 (9.3.x)
if [ $version -ge 011000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ $version -ge 010000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ $version -ge 009000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
else
JAVA_OPTS="$JAVA_OPTS -XX:+UseParNewGC"
JAVA_OPTS="$JAVA_OPTS -Xloggc:$LOG_DIR/gc.log -XX:+PrintGCDetails"
JAVA_OPTS="$JAVA_OPTS -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=60 -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled -XX:CMSFullGCsBeforeCompaction=9 -XX:+CMSClassUnloadingEnabled -XX:+PrintGCDateStamps -XX:+PrintGCApplicationConcurrentTime -XX:+PrintHeapAtGC -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=5M"
fi
fi
printf "$(date) ==== Starting ==== \n"
cd `dirname $0`/..
chmod 755 $SERVICE_NAME".jar"
./$SERVICE_NAME".jar" start
rc=$?;
if [[ $rc != 0 ]];
then
echo "$(date) Failed to start $SERVICE_NAME.jar, return code: $rc"
exit $rc;
fi
tail -f /dev/null -
寫Dockerfile
/data/dockerfile/apollo-configservice/Dockerfile1
2
3
4
5
6
7
8
9
10
11
12FROM stanleyws/jre8:8u112
ENV VERSION 1.3.0
RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime &&\
echo "Asia/Shanghai" > /etc/timezone
ADD apollo-configservice-${VERSION}.jar /apollo-configservice/apollo-configservice.jar
ADD config/ /apollo-configservice/config
ADD scripts/ /apollo-configservice/scripts
CMD ["/apollo-configservice/scripts/startup.sh"] -
制作鏡像並推送
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41[root@hdss7-200 apollo-configservice]# docker build . -t harbor.od.com/infra/apollo-configservice:v1.3.0
Sending build context to Docker daemon 61.91 MB
Step 1 : FROM stanleyws/jre8:8u112
---> fa3a085d6ef1
Step 2 : ENV VERSION 1.3.0
---> [Warning] IPv4 forwarding is disabled. Networking will not work.
---> Running in 685d51b5adb4
---> feb4c0289f04
Removing intermediate container 685d51b5adb4
Step 3 : RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && echo "Asia/Shanghai" > /etc/timezone
---> [Warning] IPv4 forwarding is disabled. Networking will not work.
---> Running in eaa05073feeb
---> a3e3fd61ae35
Removing intermediate container eaa05073feeb
Step 4 : ADD apollo-configservice-${VERSION}.jar /apollo-configservice/apollo-configservice.jar
---> be09a59b83a2
Removing intermediate container ac6b8af3979b
Step 5 : ADD config/ /apollo-configservice/config
---> fb64fc0f3194
Removing intermediate container b73c5315ad20
Step 6 : ADD scripts/ /apollo-configservice/scripts
---> 96ff3d9b9456
Removing intermediate container 67ba203b3101
Step 7 : CMD /apollo-configservice/scripts/startup.sh
---> [Warning] IPv4 forwarding is disabled. Networking will not work.
---> Running in 80bd3f53fefc
---> 551ea2ba8de3
Removing intermediate container 80bd3f53fefc
Successfully built 551ea2ba8de3
[root@hdss7-200 apollo-configservice]# docker push harbor.od.com/infra/apollo-configservice:v1.3.0
The push refers to a repository [harbor.od.com/infra/apollo-configservice]
25efb9a44683: Pushed
b3572bb46247: Pushed
e7994b936025: Pushed
0ff1d078cbc4: Pushed
ebfb473df5c2: Pushed
aae5c057d1b6: Pushed
dee6aef5c2b6: Pushed
a464c54f93a9: Pushed
v1.3.0: digest: sha256:6a8e4fdda58de0dfba9985ebbf91c4d6f46f5274983d2efa8853b03f4e45fa06 size: 1992
解析域名
DNS主機HDSS7-11.host.com上：
/var/named/od.com.zone
1 |
mysql 60 IN A 10.4.7.11 |
准備資源配置清單
在運維主機HDSS7-200.host.com上
/data/k8s-yaml
1 |
[root@hdss7-200 k8s-yaml]# mkdir /data/k8s-yaml/apollo-configservice && cd /data/k8s-yaml/apollo-configservice |
vi deployment.yaml
1 |
kind: Deployment |
vi svc.yaml
1 |
kind: Service |
vi ingress.yaml
1 |
kind: Ingress |
vi configmap.yaml
1 |
apiVersion: v1 |
應用資源配置清單
在任意一台k8s運算節點執行：
1 |
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-configservice/configmap.yaml |
瀏覽器訪問
交付apollo-adminservice
准備軟件包
在運維主機HDSS7-200.host.com上：
下載官方release包
1 |
[root@hdss7-200 src]# ls -l|grep apollo |
制作Docker鏡像
在運維主機HDSS7-200.host.com上：
-
配置數據庫連接串
/data/dockerfile/apollo-adminservice1[root@hdss7-200 apollo-adminservice]# cat config/application-github.properties -
更新starup.sh
/data/dockerfile/apollo-adminservice/scripts/startup.sh1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64#!/bin/bash
SERVICE_NAME=apollo-adminservice
## Adjust log dir if necessary
LOG_DIR=/opt/logs/apollo-adminservice
## Adjust server port if necessary
SERVER_PORT=8080
APOLLO_ADMIN_SERVICE_NAME=$(hostname -i)
# SERVER_URL="http://localhost:${SERVER_PORT}"
SERVER_URL="http://${APOLLO_ADMIN_SERVICE_NAME}:${SERVER_PORT}"
## Adjust memory settings if necessary
#export JAVA_OPTS="-Xms2560m -Xmx2560m -Xss256k -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=384m -XX:NewSize=1536m -XX:MaxNewSize=1536m -XX:SurvivorRatio=8"
## Only uncomment the following when you are using server jvm
#export JAVA_OPTS="$JAVA_OPTS -server -XX:-ReduceInitialCardMarks"
########### The following is the same for configservice, adminservice, portal ###########
export JAVA_OPTS="$JAVA_OPTS -XX:ParallelGCThreads=4 -XX:MaxTenuringThreshold=9 -XX:+DisableExplicitGC -XX:+ScavengeBeforeFullGC -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+ExplicitGCInvokesConcurrent -XX:+PrintGCDetails -XX:+HeapDumpOnOutOfMemoryError -XX:-OmitStackTraceInFastThrow -Duser.timezone=Asia/Shanghai -Dclient.encoding.override=UTF-8 -Dfile.encoding=UTF-8 -Djava.security.egd=file:/dev/./urandom"
export JAVA_OPTS="$JAVA_OPTS -Dserver.port=$SERVER_PORT -Dlogging.file=$LOG_DIR/$SERVICE_NAME.log -XX:HeapDumpPath=$LOG_DIR/HeapDumpOnOutOfMemoryError/"
# Find Java
if [[ -n "$JAVA_HOME" ]] && [[ -x "$JAVA_HOME/bin/java" ]]; then
javaexe="$JAVA_HOME/bin/java"
elif type -p java > /dev/null 2>&1; then
javaexe=$(type -p java)
elif [[ -x "/usr/bin/java" ]]; then
javaexe="/usr/bin/java"
else
echo "Unable to find Java"
exit 1
fi
if [[ "$javaexe" ]]; then
version=$("$javaexe" -version 2>&1 | awk -F '"' '/version/ {print $2}')
version=$(echo "$version" | awk -F. '{printf("%03d%03d",$1,$2);}')
# now version is of format 009003 (9.3.x)
if [ $version -ge 011000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ $version -ge 010000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ $version -ge 009000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
else
JAVA_OPTS="$JAVA_OPTS -XX:+UseParNewGC"
JAVA_OPTS="$JAVA_OPTS -Xloggc:$LOG_DIR/gc.log -XX:+PrintGCDetails"
JAVA_OPTS="$JAVA_OPTS -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=60 -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled -XX:CMSFullGCsBeforeCompaction=9 -XX:+CMSClassUnloadingEnabled -XX:+PrintGCDateStamps -XX:+PrintGCApplicationConcurrentTime -XX:+PrintHeapAtGC -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=5M"
fi
fi
printf "$(date) ==== Starting ==== \n"
cd `dirname $0`/..
chmod 755 $SERVICE_NAME".jar"
./$SERVICE_NAME".jar" start
rc=$?;
if [[ $rc != 0 ]];
then
echo "$(date) Failed to start $SERVICE_NAME.jar, return code: $rc"
exit $rc;
fi
tail -f /dev/null -
寫Dockerfile
/data/dockerfile/apollo-adminservice/Dockerfile1
2
3
4
5
6
7
8
9
10
11
12FROM stanleyws/jre8:8u112
ENV VERSION 1.3.0
RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime &&\
echo "Asia/Shanghai" > /etc/timezone
ADD apollo-adminservice-${VERSION}.jar /apollo-adminservice/apollo-adminservice.jar
ADD config/ /apollo-adminservice/config
ADD scripts/ /apollo-adminservice/scripts
CMD ["/apollo-adminservice/scripts/startup.sh"] -
制作鏡像並推送
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37[root@hdss7-200 apollo-adminservice]# docker build . -t harbor.od.com/infra/apollo-adminservice:v1.3.0
Sending build context to Docker daemon 58.31 MB
Step 1 : FROM stanleyws/jre8:8u112
---> fa3a085d6ef1
Step 2 : ENV VERSION 1.3.0
---> Using cache
---> feb4c0289f04
Step 3 : RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && echo "Asia/Shanghai" > /etc/timezone
---> Using cache
---> a3e3fd61ae35
Step 4 : ADD apollo-adminservice-${VERSION}.jar /apollo-adminservice/apollo-adminservice.jar
---> 6a1eb9565777
Removing intermediate container 7196df9af6af
Step 5 : ADD config/ /apollo-adminservice/config
---> 9f364b732d46
Removing intermediate container 9b24669c6c78
Step 6 : ADD scripts/ /apollo-adminservice/scripts
---> b7bc5517b0fc
Removing intermediate container f3e34e759148
Step 7 : CMD /apollo-adminservice/scripts/startup.sh
---> [Warning] IPv4 forwarding is disabled. Networking will not work.
---> Running in 18c6597914b4
---> 82145db3ee88
Removing intermediate container 18c6597914b4
Successfully built 82145db3ee88
[root@hdss7-200 apollo-adminservice]# docker push harbor.od.com/infra/apollo-adminservice:v1.3.0
docker push harbor.od.com/infra/apollo-adminservice:v1.3.0
The push refers to a repository [harbor.od.com/infra/apollo-adminservice]
19b1ca6c066d: Pushed
8fa6cde49908: Pushed
0b2c9b9226cc: Pushed
ebfb473df5c2: Mounted from infra/apollo-configservice
aae5c057d1b6: Mounted from infra/apollo-configservice
dee6aef5c2b6: Mounted from infra/apollo-configservice
a464c54f93a9: Mounted from infra/apollo-configservice
v1.3.0: digest: sha256:75367caab9bad3d0d281eb3324451a0734e84b6aa3ee860e38ad758d7166a7d1 size: 1785
准備資源配置清單
在運維主機HDSS7-200.host.com上
/data/k8s-yaml
1 |
[root@hdss7-200 k8s-yaml]# mkdir /data/k8s-yaml/apollo-adminservice && cd /data/k8s-yaml/apollo-adminservice |
vi deployment.yaml
1 |
kind: Deployment |
vi configmap.yaml
1 |
apiVersion: v1 |
應用資源配置清單
在任意一台k8s運算節點執行：
1 |
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-adminservice/configmap.yaml |
瀏覽器訪問
交付apollo-portal
准備軟件包
在運維主機HDSS7-200.host.com上：
下載官方release包
1 |
[root@hdss7-200 src]# ls -l|grep apollo |
執行數據庫腳本
在數據庫主機HDSS7-11.host.com上：
數據庫腳本地址
1 |
[root@hdss7-11 ~]# mysql -uroot -p |
數據庫用戶授權
1 |
mysql> grant INSERT,DELETE,UPDATE,SELECT on ApolloPortalDB.* to "apolloportal"@"172.7.%" identified by "123456"; |
制作Docker鏡像
在運維主機HDSS7-200.host.com上：
-
配置數據庫連接串
/data/dockerfile/apollo-portal1[root@hdss7-200 apollo-portal]# cat config/application-github.properties -
配置Portal的meta service
/data/dockerfile/apollo-portal/config/apollo-env.properties1dev.meta=http://config.od.com -
更新starup.sh
/data/dockerfile/apollo-portal/scripts/startup.sh1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64#!/bin/bash
SERVICE_NAME=apollo-portal
## Adjust log dir if necessary
LOG_DIR=/opt/logs/apollo-portal-server
## Adjust server port if necessary
SERVER_PORT=8080
APOLLO_PORTAL_SERVICE_NAME=$(hostname -i)
# SERVER_URL="http://localhost:$SERVER_PORT"
SERVER_URL="http://${APOLLO_PORTAL_SERVICE_NAME}:${SERVER_PORT}"
## Adjust memory settings if necessary
#export JAVA_OPTS="-Xms2560m -Xmx2560m -Xss256k -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=384m -XX:NewSize=1536m -XX:MaxNewSize=1536m -XX:SurvivorRatio=8"
## Only uncomment the following when you are using server jvm
#export JAVA_OPTS="$JAVA_OPTS -server -XX:-ReduceInitialCardMarks"
########### The following is the same for configservice, adminservice, portal ###########
export JAVA_OPTS="$JAVA_OPTS -XX:ParallelGCThreads=4 -XX:MaxTenuringThreshold=9 -XX:+DisableExplicitGC -XX:+ScavengeBeforeFullGC -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+ExplicitGCInvokesConcurrent -XX:+PrintGCDetails -XX:+HeapDumpOnOutOfMemoryError -XX:-OmitStackTraceInFastThrow -Duser.timezone=Asia/Shanghai -Dclient.encoding.override=UTF-8 -Dfile.encoding=UTF-8 -Djava.security.egd=file:/dev/./urandom"
export JAVA_OPTS="$JAVA_OPTS -Dserver.port=$SERVER_PORT -Dlogging.file=$LOG_DIR/$SERVICE_NAME.log -XX:HeapDumpPath=$LOG_DIR/HeapDumpOnOutOfMemoryError/"
# Find Java
if [[ -n "$JAVA_HOME" ]] && [[ -x "$JAVA_HOME/bin/java" ]]; then
javaexe="$JAVA_HOME/bin/java"
elif type -p java > /dev/null 2>&1; then
javaexe=$(type -p java)
elif [[ -x "/usr/bin/java" ]]; then
javaexe="/usr/bin/java"
else
echo "Unable to find Java"
exit 1
fi
if [[ "$javaexe" ]]; then
version=$("$javaexe" -version 2>&1 | awk -F '"' '/version/ {print $2}')
version=$(echo "$version" | awk -F. '{printf("%03d%03d",$1,$2);}')
# now version is of format 009003 (9.3.x)
if [ $version -ge 011000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ $version -ge 010000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ $version -ge 009000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
else
JAVA_OPTS="$JAVA_OPTS -XX:+UseParNewGC"
JAVA_OPTS="$JAVA_OPTS -Xloggc:$LOG_DIR/gc.log -XX:+PrintGCDetails"
JAVA_OPTS="$JAVA_OPTS -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=60 -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled -XX:CMSFullGCsBeforeCompaction=9 -XX:+CMSClassUnloadingEnabled -XX:+PrintGCDateStamps -XX:+PrintGCApplicationConcurrentTime -XX:+PrintHeapAtGC -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=5M"
fi
fi
printf "$(date) ==== Starting ==== \n"
cd `dirname $0`/..
chmod 755 $SERVICE_NAME".jar"
./$SERVICE_NAME".jar" start
rc=$?;
if [[ $rc != 0 ]];
then
echo "$(date) Failed to start $SERVICE_NAME.jar, return code: $rc"
exit $rc;
fi
tail -f /dev/null -
寫Dockerfile
/data/dockerfile/apollo-portal/Dockerfile
1 |
FROM stanleyws/jre8:8u112 |
- 制作鏡像並推送
1 |
[root@hdss7-200 apollo-portal]# docker build . -t harbor.od.com/infra/apollo-portal:v1.3.0 |
解析域名
DNS主機HDSS7-11.host.com上：
/var/named/od.com.zone
1 |
portal 60 IN A 10.4.7.10 |
准備資源配置清單
在運維主機HDSS7-200.host.com上
/data/k8s-yaml
1 |
[root@hdss7-200 k8s-yaml]# mkdir /data/k8s-yaml/apollo-portal && cd /data/k8s-yaml/apollo-portal |
vi deployment.yaml
1 |
kind: Deployment |
vi svc.yaml
1 |
kind: Service |
vi ingress.yaml
1 |
kind: Ingress |
vi configmap.yaml
1 |
apiVersion: v1 |
應用資源配置清單
在任意一台k8s運算節點執行：
1 |
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-portal/configmap.yaml |
瀏覽器訪問
- 用戶名：apollo
- 密碼： admin
實戰dubbo微服務接入Apollo配置中心
改造dubbo-demo-service項目
使用IDE拉取項目（這里使用git bash作為范例）
1 |
$ git clone git@gitee.com/stanleywang/dubbo-demo-service.git |
切到apollo分支
1 |
$ git checkout -b apollo |
修改pom.xml
- 加入apollo客戶端jar包的依賴
dubbo-server/pom.xml
1 |
<dependency> |
- 修改resource段
dubbo-server/pom.xml
1 |
<resource> |
增加resources目錄
/d/workspace/dubbo-demo-service/dubbo-server/src/main
1 |
$ mkdir -pv resources/META-INF |
修改config.properties文件
/d/workspace/dubbo-demo-service/dubbo-server/src/main/resources/config.properties
1 |
dubbo.registry=${dubbo.registry} |
修改srping-config.xml文件
- beans段新增屬性
/d/workspace/dubbo-demo-service/dubbo-server/src/main/resources/spring-config.xml
1 |
xmlns:apollo="http://www.ctrip.com/schema/apollo" |
- xsi:schemaLocation段內新增屬性
/d/workspace/dubbo-demo-service/dubbo-server/src/main/resources/spring-config.xml
1 |
http://www.ctrip.com/schema/apollo http://www.ctrip.com/schema/apollo.xsd |
- 新增配置項
/d/workspace/dubbo-demo-service/dubbo-server/src/main/resources/spring-config.xml
1 |
<apollo:config/> |
- 刪除配置項（注釋）
/d/workspace/dubbo-demo-service/dubbo-server/src/main/resources/spring-config.xml
1 |
<!-- <context:property-placeholder location="classpath:config.properties"/> --> |
增加app.properties文件
/d/workspace/dubbo-demo-service/dubbo-server/src/main/resources/META-INF/app.properties
1 |
app.id=dubbo-demo-service |
提交git中心倉庫（gitee）
1 |
$ git push origin apollo |
配置apollo-portal
創建項目
-
部門
樣例部門1（TEST1）
-
應用id
dubbo-demo-service
-
應用名稱
dubbo服務提供者
-
應用負責人
apollo|apollo
-
項目管理員
apollo|apollo
提交
進入配置頁面
新增配置項1
-
Key
dubbo.registry
-
Value
zookeeper://zk1.od.com:2181
-
選擇集群
DEV
提交
新增配置項2
-
Key
dubbo.port
-
Value
20880
-
選擇集群
DEV
提交
發布配置
點擊發布，配置生效
使用jenkins進行CI
略（注意記錄鏡像的tag）
上線新構建的項目
准備資源配置清單
運維主機HDSS7-200.host.com上：
/data/k8s-yaml/dubbo-demo-service/deployment.yaml
1 |
kind: Deployment |
注意：增加了env段配置
注意：docker鏡像新版的tag
應用資源配置清單
在任意一台k8s運算節點上執行：
1 |
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/dubbo-demo-service/deployment.yaml |
觀察項目運行情況
改造dubbo-demo-web
略
配置apollo-portal
創建項目
-
部門
樣例部門1（TEST1）
-
應用id
dubbo-demo-web
-
應用名稱
dubbo服務消費者
-
應用負責人
apollo|apollo
-
項目管理員
apollo|apollo
提交
進入配置頁面
新增配置項1
-
Key
dubbo.registry
-
Value
zookeeper://zk1.od.com:2181
-
選擇集群
DEV
提交
發布配置
點擊發布，配置生效
使用jenkins進行CI
略（注意記錄鏡像的tag）
上線新構建的項目
准備資源配置清單
運維主機HDSS7-200.host.com上：
/data/k8s-yaml/dubbo-demo-consumer/deployment.yaml
1 |
kind: Deployment |
注意：增加了env段配置
注意：docker鏡像新版的tag
應用資源配置清單
在任意一台k8s運算節點上執行：
1 |
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/dubbo-demo-web/deployment.yaml |
通過Apollo配置中心動態維護項目的配置
以dubbo-demo-service項目為例，不用修改代碼
- 在http://portal.od.com 里修改dubbo.port配置項
- 重啟dubbo-demo-service項目
- 配置生效
實戰維護多套dubbo微服務環境
生產實踐
- 迭代新需求/修復BUG（編碼->提GIT）
- 測試環境發版，測試（應用通過編譯打包發布至TEST命名空間）
- 測試通過，上線（應用鏡像直接發布至PROD命名空間）
系統架構
- 物理架構
| 主機名 | 角色 | ip |
|---|---|---|
| HDSS7-11.host.com | zk-test(測試環境Test) | 10.4.7.11 |
| HDSS7-12.host.com | zk-prod(生產環境Prod) | 10.4.7.12 |
| HDSS7-21.host.com | kubernetes運算節點 | 10.4.7.21 |
| HDSS7-22.host.com | kubernetes運算節點 | 10.4.7.22 |
| HDSS7-200.host.com | 運維主機，harbor倉庫 | 10.4.7.200 |
- K8S內系統架構
| 環境 | 命名空間 | 應用 |
|---|---|---|
| 測試環境（TEST） | test | apollo-config，apollo-admin |
| 測試環境（TEST） | test | dubbo-demo-service，dubbo-demo-web |
| 生產環境（PROD） | prod | apollo-config，apollo-admin |
| 生產環境（PROD） | prod | dubbo-demo-service，dubbo-demo-web |
| ops環境（infra） | infra | jenkins，dubbo-monitor，apollo-portal |
修改/添加域名解析
DNS主機HDSS7-11.host.com上：
/var/named/od.com.zone
1 |
zk-test 60 IN A 10.4.7.11 |
Apollo的k8s應用配置
- 刪除app命名空間內應用，創建test命名空間，創建prod命名空間
- 刪除infra命名空間內apollo-configservice，apollo-adminservice應用
- 數據庫內刪除ApolloConfigDB，創建ApolloConfigTestDB，創建ApolloConfigProdDB
1 |
mysql> drop database ApolloConfigDB; |
- 准備apollo-config，apollo-admin的資源配置清單（各2套）
注：apollo-config/apollo-admin的configmap配置要點
- Test環境
1 |
application-github.properties: | |
- Prod環境
1 |
application-github.properties: | |
- 依次應用，分別發布在test和prod命名空間
- 修改apollo-portal的configmap並重啟portal
1 |
apollo-env.properties: | |
Apollo的portal配置
管理員工具
刪除應用、集群、AppNamespace，將已配置應用刪除
系統參數
-
Key
apollo.portal.envs
-
Value
TEST,PROD
查詢
- Value
TEST,PROD
保存
新建dubbo-demo-service和dubbo-demo-web項目
在TEST/PROD環境分別增加配置項並發布
發布dubbo微服務
- 准備dubbo-demo-service和dubbo-demo-web的資源配置清單（各2套）
- 依次應用，分別發布至app-test和app-prod命名空間
- 使用dubbo-monitor查驗
互聯網公司技術部的日常
- 產品經理整理需求，需求評審，出產品原型
- 開發同學夜以繼日的開發，提測
- 測試同學使用Jenkins持續集成，並發布至測試環境
- 驗證功能，通過->待上線or打回->修改代碼
- 提交發版申請，運維同學將測試后的包發往生產環境
- 無盡的BUG修復（笑cry）