二進制方式搭建Kubernetes 1.19.3高可用集群(五)——部署dashboard
本文將介紹在二進制部署的k8s集群中部署dashboar 2.0.4，並解決部署過程中metrics-server無法啟動的問題
部署dashboard
首先，根據官方文檔來，下載配置文件(官方文檔地址：https://github.com/kubernetes/dashboard)
wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.4/aio/deploy/recommended.yaml -O dashboard-deploy.yaml由於鏡像在國外，下載可能會比較慢，所有我把鏡像放到了阿里雲上，可以按需替換下鏡像
kubernetesui/dashboard:v2.0.4 替換為 registry.cn-shanghai.aliyuncs.com/jieee/dashboard:v2.0.4
kubernetesui/metrics-scraper:v1.0.4 替換為 registry.cn-shanghai.aliyuncs.com/jieee/metrics-scraper:v1.0.4
然后直接部署
kubectl apply -f dashboard-deploy.yaml
# 檢查pod和service(默認的namespace是kubernetes-dashboard) kubectl get pod -n kubernetes-dashboard #NAME READY STATUS RESTARTS AGE #dashboard-metrics-scraper-7b59f7d4df-bj66m 1/1 Running 0 2m #kubernetes-dashboard-7df8bc567d-slbhs 1/1 Running 0 2m kubectl get svc -n kubernetes-dashboard #NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE #dashboard-metrics-scraper ClusterIP 10.120.5.204 <none> 8000/TCP 2m #kubernetes-dashboard ClusterIP 10.120.209.68 <none> 443/TCP 2m至此，dashboard就部署完成了，然后我們就可以在瀏覽器中輸入地址訪問了（我這里的地址是https://10.120.209.68）
注意：由於dashboard使用了自簽證書，所有chrome瀏覽器可能無法訪問，使用Firefox可以正常訪問
生成TOKEN
打開網頁后，需要我們登陸
支持dashboard支持2種方式登陸，一般我們選擇使用Token方式，先來創建一個Service Account
dashboard-rbac.yaml(這里我直接賦予了cluster-admin角色)
apiVersion: v1 kind: ServiceAccount metadata: labels: k8s-app: kubernetes-dashboard name: dashboard-admin namespace: kubernetes-dashboard --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: dashboard-admin-bind-cluster-role labels: k8s-app: kubernetes-dashboard roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cluster-admin subjects: - kind: ServiceAccount name: dashboard-admin namespace: kubernetes-dashboard部署並生成token
kubectl apply -f dashboard-rbac.yaml
#獲取TOKEN kubectl -n kubernetes-dashboard describe secret $(kubectl -n kubernetes-dashboard get secret | grep dashboard-admin | awk '{print $1}') #Name: dashboard-admin-token-grxgp #Namespace: kubernetes-dashboard #Labels: <none> #Annotations: kubernetes.io/service-account.name: dashboard-admin # kubernetes.io/service-account.uid: 440d60e7-f75b-429f-ad2b-1a56d33e47c8 # #Type: kubernetes.io/service-account-token # #Data #==== #ca.crt: 1363 bytes #namespace: 20 bytes #token: eyJhbGciOiJSUzI1NiIsImtpZCI6InZmWF9vS29UWE53bVhKbkdUY3ZpLXdqYlBHc3VCUzdiamMzLS1FMDZhQUEifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4tZ3J4Z3AiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiNDQwZDYwZTctZjc1Yi00MjlmLWFkMmItMWE1NmQzM2U0N2M4Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmVybmV0ZXMtZGFzaGJvYXJkOmRhc2hib2FyZC1hZG1pbiJ9.Uerz4ERXLeyKDfuNW_l-K_3xr3lh4Iyc8B5U_TnW8tlWgrYAcijTF86QESprolDmhn7s7RqVwrfUAHvmKoI_d08ApTWouu1lnoGIsn-qUovYOtAnpr-sal4TTWu9tjScodqklOw1WrICUiUFxcEN1939ERqx2oESYiKUuT2yEt2stMGUmp02QkmyiYtfk5a6sZ14LcyLL_mtC09hF4vW4dz2_QdP3qVd6l-RHS5NDFnB4bBz8m6TG6h2kY09tiGcFgjNfkQhFdy6L0F_jczufj39MrcRWofxROGKNo_vq2sSidekODjpp6TAIF43k51gW9T_qhUnrflemJAbUseqnw最后得到的一長串token就是登陸所需的token
登陸后就能看到整個集群的狀態了
可是我們發現列表中，CPU和內存使用率都是空的，這是因為我們還沒有安裝metrics-server
安裝metrics-server
官方文檔：https://github.com/kubernetes-sigs/metrics-server
按照文檔，我們先下載配置文件
wget https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.3.7/components.yaml -O metrics-server.yaml同樣的，我們替換一下鏡像地址
k8s.gcr.io/metrics-server/metrics-server:v0.3.7 替換為 registry.cn-shanghai.aliyuncs.com/jieee/metrics-server:v0.3.7
然后部署
kubectl apply -f metrics-server.yaml
#檢查pod狀態 kubectl get pod -n kube-system | grep metrics-server # metrics-server-f964c4474-t5sx9 1/1 Running 0 2m可以看到pod已經正常運行了。
然而，當我們回到dashboard中，發現CPU和內存信息還是沒有出來，我們先來看一下pod日志
kubectl logs metrics-server-f964c4474-t5sx9 -n kube-system
#... #E1107 05:15:45.224261 1 configmap_cafile_content.go:243] kube-system/extension-apiserver-authentication failed with : missing content for CA bundle "client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" #E1107 05:15:45.225200 1 configmap_cafile_content.go:243] key failed with : missing content for CA bundle "client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"我們發現了這兩條錯誤日志，原來是我們部署apiserver是沒有開啟聚合功能，那我們就來開啟一下吧
創建證書
cat > proxy-client-csr.json<<EOF { "CN": "aggregator", "hosts": [], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "ST": "Hangzhou", "L": "Hangzhou", "O": "system:masters", "OU": "System" } ] } EOF #創建證書 cfssl gencert -ca=../ca.pem -ca-key=../ca-key.pem -config=../ca-config.json -profile=kubernetes proxy-client-csr.json | cfssljson -bare proxy-client #分發證書至所有master節點 scp proxy-client*.pem root@10.0.50.101:/etc/kubernetes/pki/ scp proxy-client*.pem root@10.0.50.102:/etc/kubernetes/pki/ scp proxy-client*.pem root@10.0.50.103:/etc/kubernetes/pki/ 修改apiserver的service文件
在啟動命令中添加以下參數
vi /etc/systemd/system/kube-apiserver.service ... --proxy-client-cert-file=/etc/kubernetes/pki/proxy-client.pem \ --proxy-client-key-file=/etc/kubernetes/pki/proxy-client-key.pem \ --runtime-config=api/all=true \ --requestheader-client-ca-file=/etc/kubernetes/pki/ca.pem \ --requestheader-allowed-names=aggregator \ --requestheader-extra-headers-prefix=X-Remote-Extra- \ --requestheader-group-headers=X-Remote-Group \ --requestheader-username-headers=X-Remote-User \ ...然后分別重啟所有master節點的apiserver
systemctl daemon-reload && systemctl restart kube-apiserver重建metrics-server
kubectl replace --force -f metrics-server.yaml等待一段時間后，回到dashboard，刷新后發現 CPU和內存信息都出來了
同時，安裝完metrics-server后，我們也可以在kubelet中使用metrics-server,如：
kubectl top node #NAME CPU(cores) CPU% MEMORY(bytes) MEMORY% #kube-n-60-101.jieee.xyz 465m 11% 3503Mi 44% #kube-n-60-102.jieee.xyz 257m 6% 2600Mi 33% #kube-n-60-103.jieee.xyz 414m 10% 4092Mi 52%
證書配置
由於dashboard中使用了自簽證書，導致chrome中無法訪問，帶來了一些不便，接下來我們為dashboard配置上證書
方式一：使用已有證書
先刪除dashboard，然后修改配置文件
#刪除 kubectl delete -f dashboard-deploy.yaml #修改配置文件 vi dashboard-deploy.yaml #找到以下內容，然后刪除 --- apiVersion: v1 kind: Secret metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard-certs namespace: kubernetes-dashboard type: Opaque創建證書，可以通過阿里雲申請1年免費證書，或者通過Let’s Encrypt生成90天免費證書，建免費證書存放在$HOME/certs目錄下，取名為tls.crt和tls.key。
#創建證書 kubectl create secret generic kubernetes-dashboard-certs --from-file=$HOME/certs -n kubernetes-dashboard #重新部署dashboard kubectl apply -f dashboard-deploy.yaml如此，證書就配置完成了
方式二：使用ingress
如果集群中已存在ingress，並且ingress配置了ssl（dashboard不支持http訪問，所有必須支持ssl），那么可以用ingress卸載字簽證書並替換成新證書。
ingress的部署可以查看Kubernetes使用Ingress nginx暴露服務並配置證書
配置文件：
cat > dashboard-ingress.yaml<<EOF kind: Ingress apiVersion: networking.k8s.io/v1 metadata: name: dashboard namespace: kubernetes-dashboard annotations: nginx.ingress.kubernetes.io/ssl-redirect: "true" # 強制跳轉https nginx.ingress.kubernetes.io/rewrite-target: / nginx.ingress.kubernetes.io/secure-backends: "true" kubernetes.io/ingress.class: "nginx" nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" # 代理后端https spec: tls: - hosts: - '*.lingjie.tech' secretName: lingjie-tech rules: - host: dashboard.lingjie.tech http: paths: - path: / pathType: Prefix backend: service: name: kubernetes-dashboard port: number: 443 EOF #部署ingress kubectl apply -f dashboard-ingress.yaml然后我們綁定一下host（將dashboard.lingjie.tech綁定到ingress的service ip），就能通過https訪問了。