9. 第八篇 kube-controller-manager安裝及驗證

文章轉載自：https://mp.weixin.qq.com/s?__biz=MzI1MDgwNzQ1MQ==&mid=2247483826&idx=1&sn=88f0cef6866f386fb6d779dcecded534&chksm=e9fdd446de8a5d509f69f62b328dc910d0949d1431be58e2d04eba9f5b95d74651c028d6d215&cur_album_id=1341273083637989377&scene=189#wechat_redirect

kube-controller-manager（k8s控制器管理器）是一個守護進程，它通過kube-apiserver監視集群的共享狀態（kube-apiserver收集或監視到的一些集群資源狀態，供kube-controller-manager或其它客戶端watch）, 控制器管理器並嘗試將當前的狀態向所定義的狀態遷移（移動、靠近），它本身是有狀態的，會修改集群狀態信息，如果多個控制器管理器同時生效，則會有一致性問題，所以kube-controller-manager的高可用，只能是主備模式，而kubernetes集群是采用租賃鎖實現leader選舉，需要在啟動參數中加入--leader-elect=true。

下載二進制文件https://dl.k8s.io/v1.17.0/kubernetes-server-linux-amd64.tar.gz，並把kube-controller-manager二進制文件拷貝到master節點的/data/k8s/bin/此目錄

創建kubeconfig配置文件並分發

kube-controller-manager也是作為kube-apiserver的客戶端，我們需要創建kubeconfig文件。

#!/bin/bash

cd /data/k8s/work
source /data/k8s/bin/env.sh

kubectl config set-cluster kubernetes \
  --certificate-authority=/data/k8s/work/ca.pem \
  --embed-certs=true \
  --server=${KUBE_APISERVER} \
  --kubeconfig=kube-controller-manager.kubeconfig
kubectl config set-credentials system:kube-controller-manager \
  --client-certificate=kube-controller-manager.pem \
  --client-key=kube-controller-manager-key.pem \
  --embed-certs=true \
  --kubeconfig=kube-controller-manager.kubeconfig
kubectl config set-context system:kube-controller-manager \
  --cluster=kubernetes \
  --user=system:kube-controller-manager \
  --kubeconfig=kube-controller-manager.kubeconfig
kubectl config use-context system:kube-controller-manager --kubeconfig=kube-controller-manager.kubeconfig

# 分發
for node_ip in ${MASTER_IPS[@]}
do
    echo ">>> ${node_ip}"
    scp kube-controller-manager.kubeconfig root@${node_ip}:/etc/kubernetes/
done

創建kube-controller-manager啟動文件模板

#!/bin/bash
cd /data/k8s/work
source /data/k8s/bin/env.sh
cat > kube-controller-manager.service.template <<EOF
[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
[Service]
WorkingDirectory=${K8S_DIR}/kube-controller-manager
ExecStart=/data/k8s/bin/kube-controller-manager \\
  --profiling \\
  --cluster-name=kubernetes \\
  --controllers=*,bootstrapsigner,tokencleaner \\
  --kube-api-qps=1000 \\
  --kube-api-burst=2000 \\
  --leader-elect \\
  --use-service-account-credentials\\
  --concurrent-service-syncs=2 \\
  --bind-address=0.0.0.0 \\
  --secure-port=10257 \\
  --tls-cert-file=/etc/kubernetes/cert/kube-controller-manager.pem \\
  --tls-private-key-file=/etc/kubernetes/cert/kube-controller-manager-key.pem \\
  --port=10252 \\
  --authentication-kubeconfig=/etc/kubernetes/kube-controller-manager.kubeconfig \\
  --client-ca-file=/etc/kubernetes/cert/ca.pem \\
  --requestheader-allowed-names="" \\
  --requestheader-client-ca-file=/etc/kubernetes/cert/ca.pem \\
  --requestheader-extra-headers-prefix="X-Remote-Extra-" \\
  --requestheader-group-headers=X-Remote-Group \\
  --requestheader-username-headers=X-Remote-User \\
  --authorization-kubeconfig=/etc/kubernetes/kube-controller-manager.kubeconfig \\
  --cluster-signing-cert-file=/etc/kubernetes/cert/ca.pem \\
  --cluster-signing-key-file=/etc/kubernetes/cert/ca-key.pem \\
  --experimental-cluster-signing-duration=876000h \\
  --horizontal-pod-autoscaler-sync-period=10s \\
  --concurrent-deployment-syncs=10 \\
  --concurrent-gc-syncs=30 \\
  --node-cidr-mask-size=24 \\
  --service-cluster-ip-range=${SERVICE_CIDR} \\
  --pod-eviction-timeout=6m \\
  --terminated-pod-gc-threshold=10000 \\
  --root-ca-file=/etc/kubernetes/cert/ca.pem \\
  --service-account-private-key-file=/etc/kubernetes/cert/ca-key.pem \\
  --kubeconfig=/etc/kubernetes/kube-controller-manager.kubeconfig \\
  --logtostderr=true \\
  --v=2
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
EOF

配置詳解

啟動文件模板替換節點信息並分發到對應服務器

#!/bin/bsash
cd /data/k8s/work
source /data/k8s/bin/env.sh

# 替換
for (( i=0; i < 3; i++ ))
do
    sed -e "s/##NODE_NAME##/${MASTER_NAMES[i]}/" -e "s/##NODE_IP##/${MASTER_IPS[i]}/" kube-controller-manager.service.template > kube-controller-manager-${MASTER_IPS[i]}.service
done

# 分發
for node_ip in ${MASTER_IPS[@]}
do
    echo ">>> ${node_ip}"
    scp kube-controller-manager-${node_ip}.service root@${node_ip}:/etc/systemd/system/kube-controller-manager.service
done

啟動控制器管理器

#!/bin/bash
source /data/k8s/bin/env.sh
for node_ip in ${MASTER_IPS[@]}
do
    echo ">>> ${node_ip}"
    ssh root@${node_ip} "mkdir -p ${K8S_DIR}/kube-controller-manager"
    ssh root@${node_ip} "systemctl daemon-reload && systemctl enable kube-controller-manager && systemctl restart kube-controller-manager"
done

驗證

#!/bin/bash
source /data/k8s/bin/env.sh

# 進程check
for node_ip in ${MASTER_IPS[@]}
do
    echo ">>> ${node_ip}"
    ssh root@${node_ip} "systemctl status kube-controller-manager|grep -i Active"
done

# 端口check
for node_ip in ${MASTER_IPS[@]}
do
    echo ">>> ${node_ip}"
    ssh root@${node_ip} "netstat -antp|grep -E '10257|10252'"
done

驗證結果
>>> 192.168.16.104
   Active: active (running) since Sun 2019-12-29 18:03:38 CST; 3 weeks 0 days ago
>>> 192.168.16.105
   Active: active (running) since Sat 2019-12-28 21:35:08 CST; 3 weeks 1 days ago
>>> 192.168.16.106
   Active: active (running) since Sat 2019-12-28 21:35:58 CST; 3 weeks 1 days ago
>>> 192.168.16.104
tcp 0      0 0.0.0.0:10252           0.0.0.0:*               LISTEN 754/kube-controller
tcp 0      0 0.0.0.0:10257           0.0.0.0:*               LISTEN 754/kube-controller
>>> 192.168.16.105
tcp 0      0 0.0.0.0:10252           0.0.0.0:*               LISTEN 11049/kube-controll
tcp 0      0 0.0.0.0:10257           0.0.0.0:*               LISTEN 11049/kube-controll
>>> 192.168.16.106
tcp 0      0 0.0.0.0:10252           0.0.0.0:*               LISTEN 11090/kube-controll
tcp 0      0 0.0.0.0:10257           0.0.0.0:*               LISTEN 11090/kube-controll
[root@master01 ~]#

總結

1. 控制器管理器管理控制器，各個控制器負責監視（watch）apiserver暴露的集群狀態，並不斷地嘗試把當前狀態向所期待的狀態遷移；
2. 配置使用kubeconfig訪問kube-apiserver安全端口；
3. 默認非安全端口10252，安全端口10257；
4. kube-controller-manager 3節點高可用，去競爭鎖，成為leader；