k3s部署springboot+mysql集群

先决条件：

本地

• Java研发环境（idea）
• docker（用来打包镜像）

服务器

• k3s集群
• 镜像仓库

步骤概要

1. 搭建 k3s 集群
2. 构建 Java 项目，打包本地镜像
3. 将本地镜像上传至镜像仓库
4. 编写 yaml 文件，执行 kubectl 命令部署集群。

使用国内镜像k3s集群搭建

• server节点 curl -sfL http://rancher-mirror.cnrancher.com/k3s/k3s-install.sh | INSTALL_K3S_MIRROR=cn sh -

• 查看server节点的token cat /var/lib/rancher/k3s/server/node-token

• agent节点加入集群 curl -sfL http://rancher-mirror.cnrancher.com/k3s/k3s-install.sh | INSTALL_K3S_MIRROR=cn K3S_URL=https://myserver:6443 K3S_TOKEN=mynodetoken sh - //修改 myserver 和 mynodetoken 为相应的server的ip和server目录下的node-token

查看集群 kubectl get nodes

$ kubectl get nodes
NAME	STATUS   ROLES                  AGE   VERSION
server	Ready    control-plane,master   3m   v1.20.6+k3s1
agent   Ready    <none>                 12s   v1.20.6+k3s1

部署流程

1. 本地打包代码

这里从环境中读取mysql的 hostname user password。

  
  public static void main(String[] args) {
      SpringApplication.run(DemoApplication.class, args);
  }

  @RequestMapping("/")
    public String[] Home() throws SQLException {
	//从环境变量中读取数据库配置
        String connectString = String.format(
                "jdbc:mysql://%s/%s?user=%s&password=%s",
                System.getenv("DB_HOST"),
                "mysql",
                System.getenv("DB_USER"),
                System.getenv("DB_PASSWORD"));
        String query = "SELECT VERSION();";
        Connection conn=getConnection(connectString);
        String result = DemoApplication.executeSql(conn,query);

        String[] res = {
                connectString,
                "mysql version is " + result
        };
        return res; //从请求中反回mysql版本，以验证数据库连
    }

运行 mvn spring-boot:build-image

[INFO] Successfully built image 'docker.io/library/demo:0.0.1-SNAPSHOT'
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:10 min
[INFO] Finished at: 2021-05-20T18:34:12+08:00
[INFO] ------------------------------------------------------------------------

查看本地镜像 docker images

$ docker images
demo	0.0.1-SNAPSHOT	c5f2b13d33e9	41 years ago	224MB

2. 修改 tag 并推送至镜像仓库

（此处使用默认的dockerhub，自行搭建参考 https://docs.docker.com/registry/ ）

• docker tag demo:0.0.1-SNAPSHOT USERNAME/demo:0.0.1-SNAPSHOT //镜像名在pom文件中配置

• docker push USERNAME/demo:0.0.1-SNAPSHOT

在dockerhub中查看镜像

3. 编写yaml文件，并执行部署命令（注释可能会影响编译）

配置数据库密码 secret.yaml

apiVersion: v1
kind: Secret
metadata:
  name: mysql-pass
type: Opaque
stringData:
  password: "123"

配置数据库服务 mysql.yaml

# 在集群内暴露mysql服务
apiVersion: v1
kind: Service
metadata:
  name: mysql
spec:
  selector:
    app: demo
    tier: mysql
  ports:
  - port: 3306
    targetPort: 3306
---
# mysql存储
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-pv-claim
spec:
  resources:
    requests:
      storage: 10Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
---
# mysql发布配置
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mysql
spec:
  selector:
    matchLabels:
      app: demo
      tier: mysql
  template:
    metadata:
      labels:
        app: demo
        tier: mysql
    spec:
      containers:
      - name: mysql
        image: mysql:5.7
        ports:
        - containerPort: 3306
        env:
        - name: MYSQL_ROOT_PASSWORD
          valueFrom: 
		# 从secret资源中读取数据库密码
            secretKeyRef:
              name: mysql-pass
              key: password
        volumeMounts:
        - name: mysql-persistent-storage
          mountPath: /var/lib/mysql
      volumes:
      - name: mysql-persistent-storage
        persistentVolumeClaim:
          claimName: mysql-pv-claim

配置demo服务 demo.yaml

# 在集群内部暴露demo服务
apiVersion: v1
kind: Service
metadata:
  name: demo
  labels: 
    app: demo
spec:
  ports:
  - port: 8080
  selector:
    app: demo
    tier: frontend
  type: LoadBalancer # 模式为负载均衡模式，采用自带的负载均衡
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo
spec:
  selector:
    matchLabels:
      app: demo
      tier: frontend
  replicas: 3 # 副本数量为3
  template:
    metadata:
      labels:
        app: demo
        tier: frontend
    spec:
      containers:
      - name: demo
        image: USERNAME/demo:0.0.1-SNAPSHOT
        ports:
        - containerPort: 8080
        env: # 环境变量设置，可以在代码中读取
        - name: DB_HOST # 数据库hostname
          value: mysql
        - name: DB_USER # 数据库登录名
          value: root
        - name: DB_PASSWORD # 数据库登录密码（从secret资源中读取）
          valueFrom:
            secretKeyRef:
              name: mysql-pass
              key: password

通过ingress配置路由规则，将demo服务由集群内部8080端口，暴露到80端口 ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress
  labels:
      name: ingress
spec:
  rules:
  - http:
     paths:
     - pathType: Prefix
       path: "/"
       backend:
         service:
           name: demo
           port: 
             number: 8080

编写 kustomization.yaml 文件管理上述配置

resources:
  - secret.yaml
  - mysql.yaml
  - demo.yaml
  - ingress.yaml

运行命令 kubectl apply -k ./ 应用上述配置

$ kubectl apply -k ./
secret/mysql-pass created
service/demo created
service/mysql created
deployment.apps/demo created
deployment.apps/mysql created
ingress.networking.k8s.io/ingress created
persistentvolumeclaim/mysql-pv-claim created

4. 服务已经部署，检查是否部署成功

等待一段时间，运行 kubectl get pod -o wide 查看运行情况
两个节点都运行了svc(service)以用来做负载均衡，但是因为agent节点只有一个，所以服务都运行到同一个 agent 上了，这是因为 taints 和 tolerations 策略的影响。一般的 workload 将会优先部署在 agent 上而不是 server。如果有多个 agent 节点，那就可以看到 demo 服务运行在多个节点上了。（其实是内存炸了，开不起来第三台虚拟机了）

$ kubectl get po -o wide
NAME                     READY   STATUS              RESTARTS   AGE   IP           NODE    NOMINATED NODE   READINESS GATES
svclb-demo-v4bxr         1/1     Running             0          67s   10.42.0.22   server  <none>           <none>
svclb-demo-pvhwk         1/1     Running             0          67s   10.42.1.3    agent   <none>           <none>
mysql-67cd7b46b6-t6mqs   0/1     ContainerCreating   0          67s   <none>       agent   <none>           <none>
demo-54fcdf4869-mxn46    1/1     Running             0          67s   10.42.1.5    agent   <none>           <none>
demo-54fcdf4869-xmwtf    1/1     Running             0          67s   10.42.1.6    agent   <none>           <none>
demo-54fcdf4869-7ss4z    1/1     Running             0          67s   10.42.1.4    agent   <none>           <none>

使用curl localhost访问页面，各个节点的80端口都可以访问到这个服务。

$ curl localhost
["jdbc:mysql://mysql/mysql?user=root&password=123","mysql version is 5.7.34"]

成功取得数据库版本号

到这里就完成了，谢谢观看，欢迎提问。