[經驗交流] kubeadm 安裝 kubernetes 一年過期的解決辦法

kubeadm 是 kubernetes 提供的一個初始化集群的工具，使用起來非常方便。但是它創建的apiserver、controller-manager等證書默認只有一年的有效期，同時kubelet 證書也只有一年有效期，一年之后 kubernetes 將停止服務。官方推薦一年之內至少用 kubeadm upgrade 更新一次 kubernetes 系統，更新時也會自動更新證書。不過，在產線環境或者無法連接外網的環境頻繁更新 kubernetes 不太現實。可以修改 kubeadm， 讓它初始化集群時創建10年或其它有效期的apiserver、controller-manager證書，同時配合kubelet證書自動輪換機制來解決這個問題。

轉載注明原作者地址：http://www.cnblogs.com/hahp

本文適用於 kubernetes 1.9.X

1. 修補 kubeadm

下面是我的源代碼補丁，請給所需要的 kubernetes 版本打上，然后重新編譯 kubeadm，用新編譯的 kubeadm 初始化集群：

diff -Nur kubernetes.orig/vendor/k8s.io/client-go/util/cert/cert.go kubernetes/vendor/k8s.io/client-go/util/cert/cert.go
--- kubernetes.orig/vendor/k8s.io/client-go/util/cert/cert.go    2018-02-07 17:14:40.553612448 +0800
+++ kubernetes/vendor/k8s.io/client-go/util/cert/cert.go    2018-02-10 17:20:48.301330560 +0800
@@ -104,7 +104,7 @@
         IPAddresses:  cfg.AltNames.IPs,
         SerialNumber: serial,
         NotBefore:    caCert.NotBefore,
-        NotAfter:     time.Now().Add(duration365d).UTC(),
+        NotAfter:     time.Now().Add(duration365d * 10).UTC(),
         KeyUsage:     x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature,
         ExtKeyUsage:  cfg.Usages,
     }
@@ -149,7 +149,7 @@
             CommonName: fmt.Sprintf("%s@%d", host, time.Now().Unix()),
         },
         NotBefore: time.Now(),
-        NotAfter:  time.Now().Add(time.Hour * 24 * 365),
+        NotAfter:  time.Now().Add(time.Hour * 24 * 3650),
 
         KeyUsage:              x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature | x509.KeyUsageCertSign,
         ExtKeyUsage:           []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},

2. 自動輪換 kubelet 證書

注：kubelet證書分為server和client兩種， k8s 1.9默認啟用了client證書的自動輪換，但server證書自動輪換需要用戶開啟。方法是：

2.1 增加 kubelet 參數

--feature-gates=RotateKubeletServerCertificate=true

2.2 增加 controller-manager 參數

--experimental-cluster-signing-duration=87600h0m0s
--feature-gates=RotateKubeletServerCertificate=true

2.3 創建 rbac 對象

創建rbac對象，允許節點輪換kubelet server證書：

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  annotations:
    rbac.authorization.kubernetes.io/autoupdate: "true"
  labels:
    kubernetes.io/bootstrapping: rbac-defaults
  name: system:certificates.k8s.io:certificatesigningrequests:selfnodeserver
rules:
- apiGroups:
  - certificates.k8s.io
  resources:
  - certificatesigningrequests/selfnodeserver
  verbs:
  - create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kubeadm:node-autoapprove-certificate-server
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:certificates.k8s.io:certificatesigningrequests:selfnodeserver
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:nodes