請訪問原文鏈接：https://sysin.cn/blog/kubernetes-kubeadmin-cert-100y/，查看最新版。原創作品，轉載請保留出處。
作者：gc(at)sysin.org，主頁：www.sysin.cn
隨着 Kubernetes 1.23 的發布（December 07, 2021），沒寫完的文章又要測試一下新版本才能完成。
kubeadm 默認證書為一年，一年過期后，會導致 api service 不可用，使用過程中會出現：x509: certificate has expired or is not yet valid.
Google 建議通過不停更新版本來自動更新證書，太坑 _
可以在初始化群集之前重新編譯 kubeadm，證書有效期自動為 100 年。
兼容性：
已經測試適用於以下版本
- 1.17.0
- 1.18.0
- 1.19.0
- 1.20.0
- 1.21.0
- 1.22.0
- 1.23.0
1. 獲取源碼
訪問：https://github.com/kubernetes/kubernetes/releases，下載特定版本源碼
wget https://github.com/kubernetes/kubernetes/archive/v1.23.0.tar.gz
tar -zxvf kubernetes-1.23.0.tar.gz
mv kubernetes-1.23.0 kubernetes
cd kubernetes
或者使用 git 獲取
# yum install git
git clone https://github.com/kubernetes/kubernetes.git
cd kubernetes
git checkout -b remotes/origin/release-1.23 v1.23.0
2. 修改證書有效期
查看網上的資料主要有兩個地方需要修改
2.1 修改 CA 有效期為 100 年（默認為 10 年）
vim ./staging/src/k8s.io/client-go/util/cert/cert.go
// 這個方法里面 NotAfter: now.Add(duration365d * 10).UTC()
// 默認有效期就是 10 年，改成 100 年
// 輸入 /NotAfter 查找，回車定位
func NewSelfSignedCACert(cfg Config, key crypto.Signer) (*x509.Certificate, error) {
now := time.Now()
tmpl := x509.Certificate{
SerialNumber: new(big.Int).SetInt64(0),
Subject: pkix.Name{
CommonName: cfg.CommonName,
Organization: cfg.Organization,
},
NotBefore: now.UTC(),
// NotAfter: now.Add(duration365d * 10).UTC(),
NotAfter: now.Add(duration365d * 100).UTC(),
KeyUsage: x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature | x509.KeyUsageCertSign,
BasicConstraintsValid: true,
IsCA: true,
}
certDERBytes, err := x509.CreateCertificate(cryptorand.Reader, &tmpl, &tmpl, key.Public(), key)
if err != nil {
return nil, err
}
return x509.ParseCertificate(certDERBytes)
}
2.2 修改證書有效期為 100 年（默認為 1 年）
vim ./cmd/kubeadm/app/constants/constants.go
// 就是這個常量定義 CertificateValidity，改成 * 100 年
// 輸入 /CertificateValidity 查找，回車定位
const (
// KubernetesDir is the directory Kubernetes owns for storing various configuration files
KubernetesDir = "/etc/kubernetes"
// ManifestsSubDirName defines directory name to store manifests
ManifestsSubDirName = "manifests"
// TempDirForKubeadm defines temporary directory for kubeadm
// should be joined with KubernetesDir.
TempDirForKubeadm = "tmp"
// CertificateValidity defines the validity for all the signed certificates generated by kubeadm
// CertificateValidity = time.Hour * 24 * 365
CertificateValidity = time.Hour * 24 * 365 * 100
// CACertAndKeyBaseName defines certificate authority base name
CACertAndKeyBaseName = "ca"
// CACertName defines certificate name
CACertName = "ca.crt"
// CAKeyName defines certificate name
CAKeyName = "ca.key"
git 驗證（可選），修改的內容如下：
git diff
diff --git a/cmd/kubeadm/app/constants/constants.go b/cmd/kubeadm/app/constants/constants.go
index 75adf43..54f25fa 100644
--- a/cmd/kubeadm/app/constants/constants.go
+++ b/cmd/kubeadm/app/constants/constants.go
@@ -44,7 +44,7 @@ const (
TempDirForKubeadm = "tmp"
// CertificateValidity defines the validity for all the signed certificates generated by kubeadm
- CertificateValidity = time.Hour * 24 * 365
+ CertificateValidity = time.Hour * 24 * 365 * 100
// CACertAndKeyBaseName defines certificate authority base name
CACertAndKeyBaseName = "ca"
diff --git a/staging/src/k8s.io/client-go/util/cert/cert.go b/staging/src/k8s.io/client-go/util/cert/cert.go
index 9fd097a..865d6bb 100644
--- a/staging/src/k8s.io/client-go/util/cert/cert.go
+++ b/staging/src/k8s.io/client-go/util/cert/cert.go
@@ -63,7 +63,7 @@ func NewSelfSignedCACert(cfg Config, key crypto.Signer) (*x509.Certificate, erro
Organization: cfg.Organization,
},
NotBefore: now.UTC(),
- NotAfter: now.Add(duration365d * 10).UTC(),
+ NotAfter: now.Add(duration365d * 100).UTC(),
KeyUsage: x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature | x509.KeyUsageCertSign,
BasicConstraintsValid: true,
IsCA: true,
源代碼改好了，接下來就是編譯 kubeadm 了。
3. 編譯
3.1 Docker 鏡像編譯
注意：該方法僅適用早期版本，現在已經不可用。
使用該方法可以獲得與官方一致的編譯環境。
- 查看 kube-cross 的 TAG 版本號
cat ./build/build-image/cross/VERSION
v1.23.0-go1.17.3-bullseye.0
這里我們可以使用官方容器對代碼進行編譯：k8s.gcr.io/kube-cross:v1.17.3-1
- 拉取鏡像
docker pull k8s.gcr.io/kube-cross:v1.17.3-1
無法FQ可以用下面的替代鏡像：比較遺憾都沒有更新！
docker pull gcrcontainer/kube-cross:v1.17.3-1
或者：docker pull registry.aliyuncs.com/google_containers/kube-cross:v1.17.3-1
- 編譯
# docker run --rm -v <你修改后的代碼目錄>:/go/src/k8s.io/kubernetes -it gcrcontainer/kube-cross bash
docker run --rm -v /root/kubernetes:/go/src/k8s.io/kubernetes -it k8s.gcr.io/kube-cross:v1.17.3-1 bash
cd /go/src/k8s.io/kubernetes
# 編譯 kubeadm, 這里主要編譯 kubeadm 即可
make all WHAT=cmd/kubeadm GOFLAGS=-v
# 編譯 kubelet
# make all WHAT=cmd/kubelet GOFLAGS=-v
# 編譯 kubectl
# make all WHAT=cmd/kubectl GOFLAGS=-v
# 退出容器
exit
#編譯完產物在 _output/bin/kubeadm 目錄下，
#其中 bin 是使用了軟連接
#真實路徑是_output/local/bin/linux/amd64/kubeadm
mv /usr/bin/kubeadm /usr/bin/kubeadm_backup
cp _output/local/bin/linux/amd64/kubeadm /usr/bin/kubeadm
#chmod +x /usr/bin/kubeadm
# 驗證版本
kubeadm version
3.2 本機編譯
環境需求參看 官方文檔。
本例在 AlmaLinux 8.5 x86_64 中編譯完成。
3.2.1 軟件包准備
CentOS：
yum install gcc make -y
yum install rsync jq -y
Ubuntu：
sudo apt install build-essential #(Following command will install essential commands like gcc, make etc.)
sudo apt install rsync jq -y
3.2.2 GoLang 環境
查看 kube-cross 的 TAG 版本號
# cat ./build/build-image/cross/VERSION
v1.23.0-go1.17.3-bullseye.0
#注意：之前的版本顯示這種數字格式 v1.17.3-1
- 安裝 Go 環境：
wget https://dl.google.com/go/go1.17.3.linux-amd64.tar.gz
## 或者
# wget https://golang.google.cn/dl/go1.17.3.linux-amd64.tar.gz
tar zxvf go1.17.3.linux-amd64.tar.gz -C /usr/local
# 編輯 / etc/profile 文件添加如下：
#go setting
#export GOROOT=/usr/local/go
#export GOPATH=/usr/local/gopath
#export PATH=$PATH:$GOROOT/bin
#生效
#source /etc/profile
# 這里一次性編譯，直接執行如下命令即可
export PATH=$PATH:/usr/local/go/bin
- 驗證：
go version
# 輸出如下
go version go1.17.3 linux/amd64
- 編譯：
# 編譯 kubeadm, 這里主要編譯 kubeadm 即可
make all WHAT=cmd/kubeadm GOFLAGS=-v
# 編譯 kubelet
# make all WHAT=cmd/kubelet GOFLAGS=-v
# 編譯 kubectl
# make all WHAT=cmd/kubectl GOFLAGS=-v
#編譯完產物在 _output/bin/kubeadm 目錄下，
#其中 bin 是使用了軟連接
#真實路徑是_output/local/bin/linux/amd64/kubeadm
mv /usr/bin/kubeadm /usr/bin/kubeadm_backup
cp _output/local/bin/linux/amd64/kubeadm /usr/bin/kubeadm
chmod +x /usr/bin/kubeadm
kubeadm version
# 輸出如下
kubeadm version: &version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.0", GitCommit:"ab69524f795c42094a6630298ff53f3c3ebab7f4", GitTreeState:"archive", BuildDate:"2021-12-11T06:51:50Z", GoVersion:"go1.17.3", Compiler:"gc", Platform:"linux/amd64"}
保存備用
# 壓縮保存到用戶主目錄下
tar zcvf ~/kubeadm-1.23.0.tgz -C/usr/bin/ kubeadm
# 在其他節點上替換原有版本
mv /usr/bin/kubeadm /usr/bin/kubeadm_bak
tar zxvf ./kubeadm-1.23.0.tgz -C /usr/bin/
4. 執行命令更新證書
如果是使用原版 kubeadm 安裝之后，可以手動執行命令更新證書有效期到 100 年。
可以先備份證書，證書在 /etc/kubernetes/pki
- 檢查證書到期時間
kubeadm certs check-expiration
# 早期版本 (1.19 及之前版本) 命令如下
#kubeadm alpha certs check-expiration
kubeadm alpha certs 命令 1.20 開始廢棄。
kubeadm alpha 命令 1.21 開始徹底廢棄。
參看 kubeadm alpha（替換 URL 中的版本號查看狀態）。
輸出如下：
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
CERTIFICATE EXPIRES RESIDUAL TIME CERTIFICATE AUTHORITY EXTERNALLY MANAGED
admin.conf Dec 08, 2121 05:35 UTC 99y no
apiserver Dec 08, 2121 05:35 UTC 99y ca no
apiserver-etcd-client Dec 08, 2121 05:35 UTC 99y etcd-ca no
apiserver-kubelet-client Dec 08, 2121 05:35 UTC 99y ca no
controller-manager.conf Dec 08, 2121 05:35 UTC 99y no
etcd-healthcheck-client Dec 08, 2121 05:35 UTC 99y etcd-ca no
etcd-peer Dec 08, 2121 05:35 UTC 99y etcd-ca no
etcd-server Dec 08, 2121 05:35 UTC 99y etcd-ca no
front-proxy-client Dec 08, 2121 05:35 UTC 99y front-proxy-ca no
scheduler.conf Dec 08, 2121 05:35 UTC 99y no
CERTIFICATE AUTHORITY EXPIRES RESIDUAL TIME EXTERNALLY MANAGED
ca Dec 08, 2121 05:35 UTC 99y no
etcd-ca Dec 08, 2121 05:35 UTC 99y no
front-proxy-ca Dec 08, 2121 05:35 UTC 99y no
- 續訂證書，查看可以使用的參數
kubeadm certs renew --help
This command is not meant to be run on its own. See list of available subcommands.
Usage:
kubeadm certs renew [flags]
kubeadm certs renew [command]
Available Commands:
admin.conf Renew the certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself
all Renew all available certificates
apiserver Renew the certificate for serving the Kubernetes API
apiserver-etcd-client Renew the certificate the apiserver uses to access etcd
apiserver-kubelet-client Renew the certificate for the API server to connect to kubelet
controller-manager.conf Renew the certificate embedded in the kubeconfig file for the controller manager to use
etcd-healthcheck-client Renew the certificate for liveness probes to healthcheck etcd
etcd-peer Renew the certificate for etcd nodes to communicate with each other
etcd-server Renew the certificate for serving etcd
front-proxy-client Renew the certificate for the front proxy client
scheduler.conf Renew the certificate embedded in the kubeconfig file for the scheduler manager to use
Flags:
-h, --help help for renew
Global Flags:
--add-dir-header If true, adds the file directory to the header of the log messages
--log-file string If non-empty, use this log file
--log-file-max-size uint Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited. (default 1800)
--one-output If true, only write logs to their native severity level (vs also writing to each lower severity level)
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
--skip-headers If true, avoid header prefixes in the log messages
--skip-log-headers If true, avoid headers when opening log files
-v, --v Level number for the log level verbosity
Use "kubeadm certs renew [command] --help" for more information about a command.
- 續訂全部證書
kubeadm certs renew all
- 再次查看證書有效期，全部都 100 年了
kubeadm certs check-expiration
5. 下載
已經修改好的 kubeadm 下載（1.17.0、1.18.0、1.19.0，持續更新中）：
百度網盤鏈接：https://sysin.cn/blog/kubernetes-kubeadmin-cert-100y/
參考文章：