k3s部署springboot+mysql集群

先決條件：

本地

• Java研發環境（idea）
• docker（用來打包鏡像）

服務器

• k3s集群
• 鏡像倉庫

步驟概要

1. 搭建 k3s 集群
2. 構建 Java 項目，打包本地鏡像
3. 將本地鏡像上傳至鏡像倉庫
4. 編寫 yaml 文件，執行 kubectl 命令部署集群。

使用國內鏡像k3s集群搭建

• server節點 curl -sfL http://rancher-mirror.cnrancher.com/k3s/k3s-install.sh | INSTALL_K3S_MIRROR=cn sh -

• 查看server節點的token cat /var/lib/rancher/k3s/server/node-token

• agent節點加入集群 curl -sfL http://rancher-mirror.cnrancher.com/k3s/k3s-install.sh | INSTALL_K3S_MIRROR=cn K3S_URL=https://myserver:6443 K3S_TOKEN=mynodetoken sh - //修改 myserver 和 mynodetoken 為相應的server的ip和server目錄下的node-token

查看集群 kubectl get nodes

$ kubectl get nodes
NAME	STATUS   ROLES                  AGE   VERSION
server	Ready    control-plane,master   3m   v1.20.6+k3s1
agent   Ready    <none>                 12s   v1.20.6+k3s1

部署流程

1. 本地打包代碼

這里從環境中讀取mysql的 hostname user password。

  
  public static void main(String[] args) {
      SpringApplication.run(DemoApplication.class, args);
  }

  @RequestMapping("/")
    public String[] Home() throws SQLException {
	//從環境變量中讀取數據庫配置
        String connectString = String.format(
                "jdbc:mysql://%s/%s?user=%s&password=%s",
                System.getenv("DB_HOST"),
                "mysql",
                System.getenv("DB_USER"),
                System.getenv("DB_PASSWORD"));
        String query = "SELECT VERSION();";
        Connection conn=getConnection(connectString);
        String result = DemoApplication.executeSql(conn,query);

        String[] res = {
                connectString,
                "mysql version is " + result
        };
        return res; //從請求中反回mysql版本，以驗證數據庫連
    }

運行 mvn spring-boot:build-image

[INFO] Successfully built image 'docker.io/library/demo:0.0.1-SNAPSHOT'
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:10 min
[INFO] Finished at: 2021-05-20T18:34:12+08:00
[INFO] ------------------------------------------------------------------------

查看本地鏡像 docker images

$ docker images
demo	0.0.1-SNAPSHOT	c5f2b13d33e9	41 years ago	224MB

2. 修改 tag 並推送至鏡像倉庫

（此處使用默認的dockerhub，自行搭建參考 https://docs.docker.com/registry/ ）

• docker tag demo:0.0.1-SNAPSHOT USERNAME/demo:0.0.1-SNAPSHOT //鏡像名在pom文件中配置

• docker push USERNAME/demo:0.0.1-SNAPSHOT

在dockerhub中查看鏡像

3. 編寫yaml文件，並執行部署命令（注釋可能會影響編譯）

配置數據庫密碼 secret.yaml

apiVersion: v1
kind: Secret
metadata:
  name: mysql-pass
type: Opaque
stringData:
  password: "123"

配置數據庫服務 mysql.yaml

# 在集群內暴露mysql服務
apiVersion: v1
kind: Service
metadata:
  name: mysql
spec:
  selector:
    app: demo
    tier: mysql
  ports:
  - port: 3306
    targetPort: 3306
---
# mysql存儲
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-pv-claim
spec:
  resources:
    requests:
      storage: 10Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
---
# mysql發布配置
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mysql
spec:
  selector:
    matchLabels:
      app: demo
      tier: mysql
  template:
    metadata:
      labels:
        app: demo
        tier: mysql
    spec:
      containers:
      - name: mysql
        image: mysql:5.7
        ports:
        - containerPort: 3306
        env:
        - name: MYSQL_ROOT_PASSWORD
          valueFrom: 
		# 從secret資源中讀取數據庫密碼
            secretKeyRef:
              name: mysql-pass
              key: password
        volumeMounts:
        - name: mysql-persistent-storage
          mountPath: /var/lib/mysql
      volumes:
      - name: mysql-persistent-storage
        persistentVolumeClaim:
          claimName: mysql-pv-claim

配置demo服務 demo.yaml

# 在集群內部暴露demo服務
apiVersion: v1
kind: Service
metadata:
  name: demo
  labels: 
    app: demo
spec:
  ports:
  - port: 8080
  selector:
    app: demo
    tier: frontend
  type: LoadBalancer # 模式為負載均衡模式，采用自帶的負載均衡
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo
spec:
  selector:
    matchLabels:
      app: demo
      tier: frontend
  replicas: 3 # 副本數量為3
  template:
    metadata:
      labels:
        app: demo
        tier: frontend
    spec:
      containers:
      - name: demo
        image: USERNAME/demo:0.0.1-SNAPSHOT
        ports:
        - containerPort: 8080
        env: # 環境變量設置，可以在代碼中讀取
        - name: DB_HOST # 數據庫hostname
          value: mysql
        - name: DB_USER # 數據庫登錄名
          value: root
        - name: DB_PASSWORD # 數據庫登錄密碼（從secret資源中讀取）
          valueFrom:
            secretKeyRef:
              name: mysql-pass
              key: password

通過ingress配置路由規則，將demo服務由集群內部8080端口，暴露到80端口 ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress
  labels:
      name: ingress
spec:
  rules:
  - http:
     paths:
     - pathType: Prefix
       path: "/"
       backend:
         service:
           name: demo
           port: 
             number: 8080

編寫 kustomization.yaml 文件管理上述配置

resources:
  - secret.yaml
  - mysql.yaml
  - demo.yaml
  - ingress.yaml

運行命令 kubectl apply -k ./ 應用上述配置

$ kubectl apply -k ./
secret/mysql-pass created
service/demo created
service/mysql created
deployment.apps/demo created
deployment.apps/mysql created
ingress.networking.k8s.io/ingress created
persistentvolumeclaim/mysql-pv-claim created

4. 服務已經部署，檢查是否部署成功

等待一段時間，運行 kubectl get pod -o wide 查看運行情況
兩個節點都運行了svc(service)以用來做負載均衡，但是因為agent節點只有一個，所以服務都運行到同一個 agent 上了，這是因為 taints 和 tolerations 策略的影響。一般的 workload 將會優先部署在 agent 上而不是 server。如果有多個 agent 節點，那就可以看到 demo 服務運行在多個節點上了。（其實是內存炸了，開不起來第三台虛擬機了）

$ kubectl get po -o wide
NAME                     READY   STATUS              RESTARTS   AGE   IP           NODE    NOMINATED NODE   READINESS GATES
svclb-demo-v4bxr         1/1     Running             0          67s   10.42.0.22   server  <none>           <none>
svclb-demo-pvhwk         1/1     Running             0          67s   10.42.1.3    agent   <none>           <none>
mysql-67cd7b46b6-t6mqs   0/1     ContainerCreating   0          67s   <none>       agent   <none>           <none>
demo-54fcdf4869-mxn46    1/1     Running             0          67s   10.42.1.5    agent   <none>           <none>
demo-54fcdf4869-xmwtf    1/1     Running             0          67s   10.42.1.6    agent   <none>           <none>
demo-54fcdf4869-7ss4z    1/1     Running             0          67s   10.42.1.4    agent   <none>           <none>

使用curl localhost訪問頁面，各個節點的80端口都可以訪問到這個服務。

$ curl localhost
["jdbc:mysql://mysql/mysql?user=root&password=123","mysql version is 5.7.34"]

成功取得數據庫版本號

到這里就完成了，謝謝觀看，歡迎提問。