consul實現kubernetes-1.15集群master的高可用訪問實現

1、准備consul環境，參考我之前的博客實現或參考consul的官網部署最新的consul。

2、本次測試使用的是kubernetes-1.15.0

3、初始化集群

1）准備初始化文件

controlPlaneEndpoint: "kubeadm-ha.service.hq:6443" ,kubeadm-ha.service.hq是注冊到consul的域名。kubeadm-ha是service name，service.hq是consul的domain。

# cat kubeadm-config.yaml

---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: "ipvs"
---
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controlPlaneEndpoint: "kubeadm-ha.service.hq:6443"
dns:
  type: CoreDNS
imageRepository: k8s.gcr.io
kind: ClusterConfiguration
kubernetesVersion: v1.15.0
networking:
  dnsDomain: cluster.local
  podSubnet: 192.244.0.0/16
  serviceSubnet: 192.96.0.0/12
apiServer:
  timeoutForControlPlane: 4m0s
  certSANs:
  - 10.4.6.7
  - kubeadm-ha.service.hq
  - ku13-1
controllerManager:
  extraArgs:
    address: 0.0.0.0
scheduler:
  extraArgs:
    address: 0.0.0.0
etcd:
  external:
    endpoints:
    - https://10.4.7.10:2379
    - https://10.4.6.77:2379
    - https://10.4.8.28:2379
    caFile: /etc/kubernetes/ssl/ca.pem
    certFile: /etc/etcd/ssl/etcd.pem
    keyFile: /etc/etcd/ssl/etcd-key.pem

 

2）執行初始化

# kubeadm init --config kubeadm-config.yaml  --upload-certs

等一段時間就會初始化完成，下面會輸出一系列的信息，有兩個信息非常重要，一個是加入control-plane，一個是加入worker

control-plane:
kubeadm join kubeadm-ha.service.hq:6443 --token 8snd4e.j9o0icdh1mo0ls9b     --discovery-token-ca-cert-hash sha256:4cfa22006b2be98388c14c20721005e990101d6e086ff5183644c7383149a7ed     --experimental-control-plane --certificate-key 3640e475a8cd4a57396355gf3005dd40b44ccd8cc9dda624c7159cffdfr41989 --ignore-preflight-errors=IPVSProxierCheck


worker:

kubeadm join kubeadm-ha.service.hq:6443 --token 8snd4e.j9o0icdh1mo0ls9b     --discovery-token-ca-cert-hash sha256:4cfa22006b2be98388c14c20721005e990101d6e086ff5183644c7383149a7ed     --ignore-preflight-errors=IPVSProxierCheck

copy 配置文件/etc/kubernetes/admin.conf 並創建~/.kube目錄，

# mkdir ~/.kube && cp /etc/kubernetes/admin.conf ~/.kube/config

 

4、在consul里面注冊apiserver

由於我們這里使用的是3個master節點，所以service是三個

# cat  kubeadm-ha.json

{
    "services": [
      {
        "id": "kubeadm-ha-0",
        "name": "kubeadm-ha",
        "tags": [
          "kubeconfig-addr"
        ],
        "address": "10.4.6.77",
        "port": 6443,
        "check": {
          "args": ["/data/scripts/kubeadm-ha-0.sh",""],
          "interval": "10s"
     }
   },
   {
     "id": "kubeadm-ha-1",
        "name": "kubeadm-ha",
        "tags": [
          "kubeconfig-addr"
        ],
        "address": "10.4.7.10",
        "port": 6443,
        "check": {
          "args": ["/data/scripts/kubeadm-ha-1.sh",""],
          "interval": "10s"
     }
   },
   {
     "id": "kubeadm-ha-2",
        "name": "kubeadm-ha",
        "tags": [
          "kubeconfig-addr"
        ],
        "address": "10.4.8.28",
        "port": 6443,
        "check": {
          "args": ["/data/scripts/kubeadm-ha-2.sh",""],
          "interval": "10s"
     }
   }

 ]

}

#cat kubeadm-ha-0.sh

#!/bin/bash
# check kubernetes apiserver  alive
curl -k https://10.4.8.77:6443/healthz

#cat kubeadm-ha-1.sh

#!/bin/bash
# check kubernetes apiserver  alive
curl -k https://10.4.7.10:6443/healthz

#cat kubeadm-ha-2.sh

#!/bin/bash
# check kubernetes apiserver  alive
curl -k https://10.4.8.28:6443/healthz

使生效：

#consul-reload

ps：consul-reload是自己寫的一個腳本，參考我的https://www.cnblogs.com/cuishuai/p/8194345.html

5、添加節點

1）control-plane

kubeadm join kubeadm-ha.service.hq:6443 --token 8snd4e.j9o0icdh1mo0ls9b     --discovery-token-ca-cert-hash sha256:4cfa22006b2be98388c14c20721005e990101d6e086ff5183644c7383149a7ed     --experimental-control-plane --certificate-key 3640e475a8cd4a57396355gf3005dd40b44ccd8cc9dda624c7159cffdfr41989 --ignore-preflight-errors=IPVSProxierCheck

 

 這里會報錯，找不到可執行文件ipset，所以加一個--ignore-preflight-errors=IPVSProxierCheck，保證命令順利執行。

2）worker

kubeadm join kubeadm-ha.service.hq:6443 --token 8snd4e.j9o0icdh1mo0ls9b     --discovery-token-ca-cert-hash sha256:4cfa22006b2be98388c14c20721005e990101d6e086ff5183644c7383149a7ed     --ignore-preflight-errors=IPVSProxierCheck

擴展集群變得非常方便。