kubeadm高可用安裝k8s集群1.18.5

1、 基本環境配置

1、Kubectl debug 設置一個臨時容器

2、Sidecar

3、Volume：更改目錄權限，fsGroup

4、ConfigMap和Secret

K8S官網：https://kubernetes.io/docs/setup/

最新版高可用安裝：https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/high-availability/

主機名	IP地址	說明
k8s-master01 ~ 03	192.168.0.106 ~ 20	master節點 * 3
k8s-master-lb	192.168.0.200	keepalived虛擬IP
k8s-node01 ~ 02	192.168.0.108 ~ 22	worker節點 * 2

VIP（虛擬IP）不要和公司內網IP重復，首先去ping一下，不通才可用。VIP需要和主機在同一個局域網內！

所有節點配置hosts，修改/etc/hosts如下：

[root@k8s-master01 ~]# cat /etc/hosts
192.168.0.100 k8s-master01
192.168.0.106 k8s-master02
192.168.0.107 k8s-master03
192.168.0.200 k8s-master-lb
192.168.0.108 k8s-node01
192.168.0.109 k8s-node02

所有節點關閉防火牆、selinux、dnsmasq、swap。服務器配置如下

systemctl disable --now firewalld 
systemctl disable --now dnsmasq
#systemctl disable --now NetworkManager  #CentOS8無需關閉

setenforce 0

[root@k8s-master01 ~]# cat !$
cat /etc/sysconfig/selinux
SELINUX=disabled

swapoff -a && sysctl -w vm.swappiness=0
[root@k8s-master01 ~]# vi /etc/fstab 
[root@k8s-master01 ~]# cat /etc/fstab 

#
# /etc/fstab
# Created by anaconda on Fri Nov  1 23:02:53 2019
#
# Accessible filesystems, by reference, are maintained under '/dev/disk/'.
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info.
#
# After editing this file, run 'systemctl daemon-reload' to update systemd
# units generated from this file.
#
/dev/mapper/cl-root     /                       xfs     defaults        0 0
UUID=6897cd7b-9b3a-42b0-a827-57991141b297 /boot                   ext4    defaults        1 2
#/dev/mapper/cl-swap     swap                    swap    defaults        0 0
 

安裝ntpdate（CentOS 7 無需安裝，自帶ntpdate命令）

所有節點同步時間。時間同步配置如下：

ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
echo 'Asia/Shanghai' >/etc/timezone
ntpdate time2.aliyun.com
# 加入到crontab
*/5 * * * * ntpdate time2.aliyun.com
# 加入到開機自動同步，/etc/rc.local
ntpdate time2.aliyun.com

所有節點配置limit：

ulimit -SHn 65535

Master01節點免密鑰登錄其他節點，安裝過程中生成配置文件和證書均在Master01上操作，集群管理也在Master01上操作，阿里雲或者AWS上需要單獨一台kubectl服務器。密鑰配置如下：

ssh-keygen -t rsa
for i in k8s-master01 k8s-master02 k8s-master03 k8s-node01 k8s-node02;do ssh-copy-id -i .ssh/id_rsa.pub $i;done
 

在源碼中的repo目錄配置使用的是國內倉庫源，將其復制到所有節點：

git clone https://github.com/dotbalo/k8s-ha-install.git
 

2、CentOS 7安裝yum源如下

curl -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo
yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
sed -i -e '/mirrors.cloud.aliyuncs.com/d' -e '/mirrors.aliyuncs.com/d' /etc/yum.repos.d/CentOS-Base.repo
 

所有節點升級系統並重啟，此處升級沒有升級內核，下節會單獨升級內核：

yum install wget jq psmisc vim net-tools telnet yum-utils device-mapper-persistent-data lvm2 -y
yum update -y --exclude=kernel* && reboot #CentOS7需要升級，8不需要
 

3、內核配置

（如果內核達到，可以不用跳過此步驟）

CentOS7 需要升級內核至4.18+

使用如下方式安裝最新版內核
rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-2.el7.elrepo.noarch.rpm

查看最新版內核yum --disablerepo="*" --enablerepo="elrepo-kernel" list available

[root@k8s-node01 ~]# yum --disablerepo="*" --enablerepo="elrepo-kernel" list available

安裝最新版：
yum --enablerepo=elrepo-kernel install kernel-ml kernel-ml-devel –y
安裝完成后reboot
更改內核順序：
grub2-set-default  0 && grub2-mkconfig -o /etc/grub2.cfg && grubby --args="user_namespace.enable=1" --update-kernel="$(grubby --default-kernel)" && reboot
開機后查看內核
[appadmin@k8s-node01 ~]$ uname -a
Linux k8s-node01 5.7.7-1.el7.elrepo.x86_64 #1 SMP Wed Jul 1 11:53:16 EDT 2020 x86_64 x86_64 x86_64 GNU/Linux
 

本所有節點安裝ipvsadm：

yum install ipvsadm ipset sysstat conntrack libseccomp -y

所有節點配置ipvs模塊，在內核4.19+版本nf_conntrack_ipv4已經改為nf_conntrack，本例安裝的內核為4.18，使用nf_conntrack_ipv4即可：

modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack_ipv4
cat /etc/modules-load.d/ipvs.conf 
ip_vs
ip_vs_rr
ip_vs_wrr
ip_vs_sh
nf_conntrack_ipv4
ip_tables
ip_set
xt_set
ipt_set
ipt_rpfilter
ipt_REJECT
ipip

然后執行systemctl enable --now systemd-modules-load.service即可
 

檢查是否加載：

[root@k8s-master01 ~]# lsmod | grep -e ip_vs -e nf_conntrack_ipv4
nf_conntrack_ipv4      16384  23 
nf_defrag_ipv4         16384  1 nf_conntrack_ipv4
nf_conntrack          135168  10 xt_conntrack,nf_conntrack_ipv6,nf_conntrack_ipv4,nf_nat,nf_nat_ipv6,ipt_MASQUERADE,nf_nat_ipv4,xt_nat,nf_conntrack_netlink,ip_vs
 

開啟一些k8s集群中必須的內核參數，所有節點配置k8s內核：

cat <<EOF > /etc/sysctl.d/k8s.conf
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-iptables = 1
fs.may_detach_mounts = 1
vm.overcommit_memory=1
vm.panic_on_oom=0
fs.inotify.max_user_watches=89100
fs.file-max=52706963
fs.nr_open=52706963
net.netfilter.nf_conntrack_max=2310720

net.ipv4.tcp_keepalive_time = 600
net.ipv4.tcp_keepalive_probes = 3
net.ipv4.tcp_keepalive_intvl =15
net.ipv4.tcp_max_tw_buckets = 36000
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_max_orphans = 327680
net.ipv4.tcp_orphan_retries = 3
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.ip_conntrack_max = 65536
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.tcp_timestamps = 0
net.core.somaxconn = 16384
EOF


sysctl --system
 

所有節點配置完內核后，重啟服務器，保證重啟后內核依舊加載

reboot
lsmod | grep --color=auto -e ip_vs -e nf_conntrack

4、基本組件安裝

本節主要安裝的是集群中用到的各種組件，比如Docker-ce、Kubernetes各組件等。

4.1安裝Docker組件

查看可用docker-ce版本：

yum list docker-ce.x86_64 --showduplicates | sort -r
[root@k8s-master01 k8s-ha-install]# wget https://download.docker.com/linux/centos/7/x86_64/edge/Packages/containerd.io-1.2.13-3.2.el7.x86_64.rpm 

[root@k8s-master01 k8s-ha-install]# yum install containerd.io-1.2.13-3.2.el7.x86_64.rpm -y

安裝指定版本的Docker：
yum -y install docker-ce-17.09.1.ce-1.el7.centos
安裝最新版本的Docker
yum install docker-ce –y

[root@k8s-master01 ~]# docker -v
Docker version 19.03.13, build 4484c46d9d
 

溫馨提示：

由於新版kubelet建議使用systemd，所以可以把docker的CgroupDriver改成systemd

cat > /etc/docker/daemon.json <<EOF
{
  "exec-opts": ["native.cgroupdriver=systemd"]
}
EOF
 

4.2安裝k8s組件：

yum list kubeadm.x86_64 --showduplicates | sort -r
所有節點安裝最新版本kubeadm：
yum install kubeadm -y
所有節點安裝指定版本k8s組件：

yum install -y kubeadm-1.18.5-0.x86_64 kubelet-1.18.5-0.x86_64 kubectl-1.18.5-0.x86_64
 

4.3啟動Docker和Kubernetes

所有節點設置開機自啟動Docker：

systemctl daemon-reload && systemctl enable --now docker
 

默認配置的pause鏡像使用gcr.io倉庫，國內可能無法訪問，所以這里配置Kubelet使用阿里雲的pause鏡像：

cat >/etc/sysconfig/kubelet<<EOF
KUBELET_EXTRA_ARGS="--cgroup-driver=$DOCKER_CGROUPS --pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google_containers/pause-amd64:3.1"
EOF
 

設置Kubelet開機自啟動：

systemctl daemon-reload
systemctl enable --now kubelet

5、高可用組件安裝

5.1所有Master節點通過yum安裝HAProxy和KeepAlived

yum install keepalived haproxy -y

所有Master節點配置HAProxy（詳細配置參考HAProxy文檔，所有Master節點的HAProxy配置相同）：

[root@k8s-master01 etc]# mkdir /etc/haproxy
[root@k8s-master01 etc]# vim /etc/haproxy/haproxy.cfg 
global
  maxconn  2000
  ulimit-n  16384
  log  127.0.0.1 local0 err
  stats timeout 30s

defaults
  log global
  mode  http
  option  httplog
  timeout connect 5000
  timeout client  50000
  timeout server  50000
  timeout http-request 15s
  timeout http-keep-alive 15s

frontend monitor-in
  bind *:33305
  mode http
  option httplog
  monitor-uri /monitor

frontend k8s-master
  bind 0.0.0.0:16443
  bind 127.0.0.1:16443
  mode tcp
  option tcplog
  tcp-request inspect-delay 5s
  default_backend k8s-master

backend k8s-master
  mode tcp
  option tcplog
  option tcp-check
  balance roundrobin
  default-server inter 10s downinter 5s rise 2 fall 2 slowstart 60s maxconn 250 maxqueue 256 weight 100
  server k8s-master01	192.168.0.100:6443  check
  server k8s-master02	192.168.0.106:6443  check
  server k8s-master03	192.168.0.107:6443  check
 

5.2Master01節點的配置：

[root@k8s-master01 etc]# mkdir /etc/keepalived

[root@k8s-master01 ~]# vim /etc/keepalived/keepalived.conf 
! Configuration File for keepalived
global_defs {
    router_id LVS_DEVEL
}
vrrp_script chk_apiserver {
    script "/etc/keepalived/check_apiserver.sh"
    interval 2
    weight -5
    fall 3  
    rise 2
}
vrrp_instance VI_1 {
    state MASTER
    interface ens33
    mcast_src_ip 192.168.0.100
    virtual_router_id 51
    priority 100
    advert_int 2
    authentication {
        auth_type PASS
        auth_pass K8SHA_KA_AUTH
    }
    virtual_ipaddress {
        192.168.0.200
    }
#    track_script {
#       chk_apiserver
#    }
}
 

5.3Master02節點的配置：

[root@k8s-master02 etc]# mkdir /etc/keepalived

[root@k8s-master02 ~]# vim /etc/keepalived/keepalived.conf 

! Configuration File for keepalived
global_defs {
    router_id LVS_DEVEL
}
vrrp_script chk_apiserver {
    script "/etc/keepalived/check_apiserver.sh"
    interval 2
    weight -5
    fall 3  
    rise 2
}
vrrp_instance VI_1 {
    state BACKUP
    interface ens33
    mcast_src_ip 192.168.0.106
    virtual_router_id 51
    priority 101
    advert_int 2
    authentication {
        auth_type PASS
        auth_pass K8SHA_KA_AUTH
    }
    virtual_ipaddress {
        192.168.0.200
    }
#    track_script {
#       chk_apiserver
#    }
}

 

5.4Master03節點的配置：

[root@k8s-master03 etc]# mkdir /etc/keepalived

[root@k8s-master03 ~]# vim /etc/keepalived/keepalived.conf 

! Configuration File for keepalived
global_defs {
    router_id LVS_DEVEL
}
vrrp_script chk_apiserver {
    script "/etc/keepalived/check_apiserver.sh"
    interval 2
    weight -5
    fall 3  
    rise 2
}
vrrp_instance VI_1 {
    state BACKUP
    interface ens33
    mcast_src_ip 192.168.0.1067
    virtual_router_id 51
    priority 101
    advert_int 2
    authentication {
        auth_type PASS
        auth_pass K8SHA_KA_AUTH
    }
    virtual_ipaddress {
        192.168.0.200
    }
#    track_script {
#       chk_apiserver
#    }
}
 

5.5注意上述的健康檢查是關閉的，集群建立完成后再開啟：

#    track_script {
#       chk_apiserver
#    }

5.6配置KeepAlived健康檢查文件（所有Master節點）

 

[root@k8s-master01 keepalived]# cat /etc/keepalived/check_apiserver.sh 
#!/bin/bash

err=0
for k in $(seq 1 5)
do
    check_code=$(pgrep kube-apiserver)
    if [[ $check_code == "" ]]; then
        err=$(expr $err + 1)
        sleep 5
        continue
    else
        err=0
        break
    fi
done

if [[ $err != "0" ]]; then
    echo "systemctl stop keepalived"
    /usr/bin/systemctl stop keepalived
    exit 1
else
    exit 0
fi

 

5.7啟動haproxy和keepalived（所有Master節點）

[root@k8s-master01 keepalived]# systemctl enable --now haproxy
[root@k8s-master01 keepalived]# systemctl enable --now keepalived
 

5.8配置K8S組件

https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/high-availability/

各Master節點的kubeadm-config.yaml配置文件如下：

Master01：

daocloud.io/daocloud

apiVersion: kubeadm.k8s.io/v1beta2
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: 7t2weq.bjbawausm0jaxury
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 192.168.0.100
  bindPort: 6443
nodeRegistration:
  criSocket: /var/run/dockershim.sock
  name: k8s-master01
  taints:
  - effect: NoSchedule
    key: node-role.kubernetes.io/master
---
apiServer:
  certSANs:
  - 192.168.0.200
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controlPlaneEndpoint: 192.168.0.200:16443
controllerManager: {}
dns:
  type: CoreDNS
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers
kind: ClusterConfiguration
kubernetesVersion: v1.18.5
 
networking:
  dnsDomain: cluster.local
  podSubnet: 172.168.0.0/16
  serviceSubnet: 10.96.0.0/12
scheduler: {}

==============================================
#如果這個鏡像地址無法下載，可以替換：daocloud.io/daocloud   下載
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers
kubernetesVersion: v1.18.5
 #注意安裝的Kubernetes的版本

 

更新kubeadm文件，如果你使用高版本的，可以使用這條命令生成高版本的yaml文件

kubeadm config migrate --old-config kubeadm-config.yaml --new-config new.yaml
 

所有Master節點提前下載鏡像，可以節省初始化時間：

kubeadmconfig images pull --config /root/kubeadm-config.yaml -

所有節點設置開機自啟動kubelet
systemctl enable --now kubelet

Master01節點初始化，初始化以后會在/etc/kubernetes目錄下生成對應的證書和配置文件，之后其他Master節點加入Master01即可：

kubeadminit --config /root/kubeadm-config.yaml  --upload-certs

不用配置文件初始化：
kubeadm init --control-plane-endpoint "LOAD_BALANCER_DNS:LOAD_BALANCER_PORT" --upload-certs
 

如果初始化失敗，重置后再次初始化，命令如下：

kubeadm reset

初始化成功以后，會產生Token值，用於其他節點加入時使用，因此要記錄下初始化成功生成的token值（令牌值）：

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of the control-plane node running the following command on each as root:

  kubeadm join 192.168.0.200:16443 --token 5joxsb.zo1vh747wljgzrlt \
    --discovery-token-ca-cert-hash sha256:86ee9b6a65c6d8641507e9e56e66dad47cfa15b41b52a11e175c5f9588a485b8 \
    --control-plane --certificate-key bc4726d06255be0cd54592e29068e32c5a49eb8fd30a691342412cf79b3d47c7

Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use
"kubeadm init phase upload-certs --upload-certs" to reload certs afterward.

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.0.200:16443 --token 5joxsb.zo1vh747wljgzrlt \
    --discovery-token-ca-cert-hash sha256:86ee9b6a65c6d8641507e9e56e66dad47cfa15b41b52a11e175c5f9588a485b8

 

5.9所有Master節點配置環境變量，用於訪問Kubernetes集群：

cat <<EOF >> /root/.bashrc
export KUBECONFIG=/etc/kubernetes/admin.conf
EOF
source /root/.bashrc

查看節點狀態：

[root@k8s-master01 ~]# kubectl get nodes
NAME           STATUS     ROLES     AGE       VERSION
k8s-master01   NotReady   master    14m       v1.12.3

 

采用初始化安裝方式，所有的系統組件均以容器的方式運行並且在kube-system命名空間內，此時可以查看Pod狀態：

[root@k8s-master01 ~]# kubectl get pods -n kube-system -o wide
NAME                                   READY     STATUS    RESTARTS   AGE       IP              NODE
coredns-777d78ff6f-kstsz               0/1       Pending   0          14m       <none>          <none>
coredns-777d78ff6f-rlfr5               0/1       Pending   0          14m       <none>          <none>
etcd-k8s-master01                      1/1       Running   0          14m       192.168.0.100   k8s-master01
kube-apiserver-k8s-master01            1/1       Running   0          13m       192.168.0.100   k8s-master01
kube-controller-manager-k8s-master01   1/1       Running   0          13m       192.168.0.100   k8s-master01
kube-proxy-8d4qc                       1/1       Running   0          14m       192.168.0.100   k8s-master01
kube-scheduler-k8s-master01            1/1       Running   0          13m       192.168.0.100   k8s-master01

 

6、Calico組件的安裝

注意：如果國內用戶下載Calico較慢，所有節點可以配置加速器(如果該文件有其他配置，別忘了加上去)

vim  /etc/docker/daemon.json
{
"exec-opts": ["native.cgroupdriver=systemd"], 
  "registry-mirrors": [
    "https://registry.docker-cn.com",
    "http://hub-mirror.c.163.com",
    "https://docker.mirrors.ustc.edu.cn"
  ]
}
systemctl daemon-reload
systemctl restart docker

Calico：https://www.projectcalico.org/
https://docs.projectcalico.org/getting-started/kubernetes/self-managed-onprem/onpremises


curl https://docs.projectcalico.org/manifests/calico.yaml -O
            - name: CALICO_IPV4POOL_CIDR
              value: "172.168.0.0/16"

kubectl apply -f calico.yaml

7、高可用Master

[root@k8s-master01 ~]# kubectl get secret -n kube-system

[root@k8s-master01 ~]# kubectl get secret -n kube-system bootstrap-token-7t2weq -oyaml Token過期后生成新的token：


Node節點生成
kubeadm token create --print-join-command

Master需要生成--certificate-key
kubeadm init phase upload-certs  --upload-certs
 

kubeadm join 192.168.0.200:16443 --token 9zp1xe.h5kpi1b9kd5blk76     --discovery-token-ca-cert-hash sha256:6ba6e5205ac27e39e03d3b89a639ef70f6503fb877b1cf8a332b399549471740 \
    --control-plane --certificate-key 309f945f612dd7f0d830b11868edd5135e6cf358ed503107eb645dc8d7c84405
 

8、Node節點的配置

Node節點上主要部署公司的一些業務應用，生產環境中不建議Master節點部署系統組件之外的其他Pod，測試環境可以允許Master節點部署Pod以節省系統資源。

kubeadm join 192.168.0.200:16443 --token 9zp1xe.h5kpi1b9kd5blk76     --discovery-token-ca-cert-hash sha256:6ba6e5205ac27e39e03d3b89a639ef70f6503fb877b1cf8a332b399549471740
 

9、 Metrics部署

在新版的Kubernetes中系統資源的采集均使用Metrics-server，可以通過Metrics采集節點和Pod的內存、磁盤、CPU和網絡的使用率。

Heapster更改metrics的部署文件證書，將metrics-server-3.6.1/metrics-server-deployment.yaml的front-proxy-ca.pem改為front-proxy-ca.crt

將Master01節點的front-proxy-ca.crt復制到所有Node節點
scp /etc/kubernetes/pki/front-proxy-ca.crt k8s-node01:/etc/kubernetes/pki/front-proxy-ca.crt
scp /etc/kubernetes/pki/front-proxy-ca.crt k8s-node(其他節點自行拷貝):/etc/kubernetes/pki/front-proxy-ca.crt
安裝metrics server
kubectl  create -f  metrics-server-3.6.1/

10、Dashboard部署

官方GitHub：https://github.com/kubernetes/dashboard

Dashboard用於展示集群中的各類資源，同時也可以通過Dashboard實時查看Pod的日志和在容器中執行一些命令等。

可以在官方dashboard查看到最新版dashboard

kubectl apply –f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.4/aio/deploy/recommended.yaml

[root@k8s-master01 ]# kubectl get svc -n kubernetes-dashboard
[root@k8s-master01]# kubectl edit svc kubernetes-dashboard -n !$

 

在谷歌瀏覽器（Chrome）啟動文件中加入啟動參數，用於解決無法訪問Dashboard的問題

--test-type--ignore-certificate-errors

訪問Dashboard：https://192.168.0.200:30000，選擇登錄方式為令牌（即token方式）

 

vim admin.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding 
metadata: 
  name: admin-user
  annotations:
    rbac.authorization.kubernetes.io/autoupdate: "true"
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kube-system


kubectl apply -f admin.yaml

kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')
 

將token值輸入到令牌后，單擊登錄即可訪問Dashboard

將Kube-proxy改為ipvs模式，因為在初始化集群的時候注釋了ipvs配置，所以需要自行修改一下：
kubectl edit cm kube-proxy -n kube-system
mode: “ipvs”
更新Kube-Proxy的Pod：
kubectl patch daemonset kube-proxy -p "{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"date\":\"`date +'%s'`\"}}}}}" -n kube-system
驗證Kube-Proxy模式
[root@k8s-master01 1.1.1]# curl 127.0.0.1:10249/proxyMode
ipvs

[root@k8s-master01 ~]# kubectl get node
NAME           STATUS   ROLES    AGE     VERSION
k8s-master01   Ready    master   5h50m   v1.18.5
k8s-master02   Ready    master   5h31m   v1.18.5
k8s-master03   Ready    master   5h30m   v1.18.5
k8s-node01     Ready    <none>   5h26m   v1.18.5
k8s-node02     Ready    <none>   5h26m   v1.18.5