wsl2 ubuntu20.04 上使用 kubeadm 創建一個單主集群

• PS：wsl2 ubuntu 每次重啟ip都會改變，最近一次的安裝甚至都啟動不了docker，所以本文就當是新手參考，練練如何安裝k8s，真正用於開發還是夠嗆
• 官方文檔使用 kubeadm 創建一個單主集群

環境初始化

• 建議盡可能初始化環境，命令wsl --unregister Ubuntu-20.04可重新安裝，相當於重裝系統。安裝或重置過程中，打開這個 wsl2 窗口，提示如下：

Installing, this may take a few minutes...
適用於 Linux 的 Windows 子系統實例已終止。
Please create a default UNIX user account. The username does not need to match your Windows username.
For more information visit: https://aka.ms/wslusers
Enter new UNIX username:

• 此時關掉窗口再打開，以后則默認使用 root 用戶登錄。
• 使用阿里鏡像：

sudo cat <<EOF >/etc/apt/sources.list
deb http://mirrors.aliyun.com/ubuntu/ $(lsb_release -cs) main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ $(lsb_release -cs) main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ $(lsb_release -cs)-security main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ $(lsb_release -cs)-security main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ $(lsb_release -cs)-updates main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ $(lsb_release -cs)-updates main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ $(lsb_release -cs)-proposed main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ $(lsb_release -cs)-proposed main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ $(lsb_release -cs)-backports main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ $(lsb_release -cs)-backports main restricted universe multiverse
EOF

• 更新緩存

sudo apt update

• wsl2 官方 ubuntu 系統啟動方式為 init，而 k8s 安裝過程需要系統以 systemd 方式啟動。查詢 docker 狀態的源碼，可以看到是通過systemctl is-active docker查詢，因此要想辦法在 wsl2 啟用 systemd。

• 參考Using snapd in WSL2可知： Systemd 已經預安裝，但並未激活。使用方式很簡單，就是將它放到一個“容器”，在該容器里，Systemd 的 pid 是 1。命令如下：

• 安裝daemonize

sudo apt install daemonize

sudo daemonize /usr/bin/unshare --fork --pid --mount-proc /lib/systemd/systemd --system-unit=basic.target

• 使用命令ps -aux查看進程 pid。使用以下命令進入這個“容器”，在此查看進程 pid，可以看到 pid 為 1 的進程變成了 systemd。使用命令stat /proc/1/exe亦可驗證。

exec sudo nsenter -t $(pidof systemd) -a su - $LOGNAME

安裝 Docker

• 安裝命令，參考安裝 Docker。

# 安裝 Docker CE
## 設置倉庫
### 安裝軟件包以允許 apt 通過 HTTPS 使用存儲庫
apt-get update && apt-get install \
  apt-transport-https ca-certificates curl software-properties-common

### 新增 Docker 的 官方 GPG 秘鑰
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -

### 添加 Docker apt 倉庫
add-apt-repository \
  "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) \
  stable"

## 安裝 Docker CE
apt-get update && apt-get install \
  containerd.io \
  docker-ce \
  docker-ce-cli -y

# 設置 daemon
cat > /etc/docker/daemon.json <<EOF
{
  "registry-mirrors": ["https://registry.docker-cn.com"],
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
}
EOF

mkdir -p /etc/systemd/system/docker.service.d

# 重啟 docker.
systemctl daemon-reload
systemctl restart docker

• 其它資料：
  • Cgroup 驅動

安裝 kubelet、kubeadm、kubectl

• 官方文檔安裝 kubeadm、kubelet 和 kubectl
• 官方 GPG 秘鑰無法訪問，按照阿里雲 Kubernetes 鏡像配置，設置一下安裝源然后開始安裝 kubelet、kubeadm、kubectl。因為還沒 ubuntu20.04 還沒有相應源，先用 ubuntu18.04 的。

apt-get update && apt-get install -y apt-transport-https
curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add -
cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main
EOF
apt-get update
apt-get install -y kubelet kubeadm kubectl

使用 kubeadm 創建單個控制平面集群

• 初始化文檔

• 插件列表參考文檔安裝擴展

• 污點和容忍： Taint 和 Toleration

• https://hub.docker.com/u/gotok8s

• 初始化命令：kubeadm init --pod-network-cidr=10.244.0.0/16 --control-plane-endpoint=ubuntu.wsl --image-repository=gotok8s --v=5

  • 其中 pod-network-cidr 指定網絡驅動的 CIDR，根據使用的插件而定。這里使用 flannel，文檔指示使用值10.244.0.0/16。
  • control-plane-endpoint 指定主節點地址，可以是 IP 地址或者是可以映射 ip 地址的 DNS 名稱。因此使用前要設置一下:

  cat <<EOF >>/etc/hosts
  $(ip  address | grep eth0 | grep inet | awk -F ' '  '{  print $2}' | awk -F / '{ print $1}')  ubuntu.wsl
  EOF
  

  • image-repository 指定拉取鏡像的倉庫，gotok8s是 docker hub 上的用戶，同步了安裝所需鏡像，否則國內會因訪問不了 gcr.io 而安裝失敗。
  • --v=5是調試選項，加上后可看到更詳細輸出。

root@DESKTOP-QNMROJ1:~# kubeadm init --pod-network-cidr=10.244.0.0/16 --control-plane-endpoint=ubuntu.wsl --image-repository=gotok8s
W0706 21:45:49.154779 1903997 version.go:102] could not fetch a Kubernetes version from the internet: unable to get URL "https://dl.k8s.io/release/stable-1.txt": Get https://dl.k8s.io/release/stable-1.txt: dial tcp: lookup dl.k8s.io on 127.0.0.53:53: server misbehaving
W0706 21:45:49.154839 1903997 version.go:103] falling back to the local client version: v1.18.5
W0706 21:45:49.155372 1903997 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
[init] Using Kubernetes version: v1.18.5
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [desktop-qnmroj1 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local ubuntu.wsl] and IPs [10.96.0.1 172.25.14.224]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [desktop-qnmroj1 localhost] and IPs [172.25.14.224 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [desktop-qnmroj1 localhost] and IPs [172.25.14.224 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
W0706 21:45:54.052056 1903997 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"
[control-plane] Creating static Pod manifest for "kube-scheduler"
W0706 21:45:54.053414 1903997 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 33.002925 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.18" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node desktop-qnmroj1 as control-plane by adding the label "node-role.kubernetes.io/master=''"
[mark-control-plane] Marking the node desktop-qnmroj1 as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: 8328sm.dh21due3jz2okibj
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of control-plane nodes by copying certificate authorities
and service account keys on each node and then running the following as root:

  kubeadm join ubuntu.wsl:6443 --token 8328sm.dh21due3jz2okibj \
    --discovery-token-ca-cert-hash sha256:31f44da2c275026d9d71ddcd138f8c6a5ac4fecbc167d8b6247788f9651afa53 \
    --control-plane

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join ubuntu.wsl:6443 --token 8328sm.dh21due3jz2okibj \
    --discovery-token-ca-cert-hash sha256:31f44da2c275026d9d71ddcd138f8c6a5ac4fecbc167d8b6247788f9651afa53

• 配置 cluster 信息使 kubectl 可用

# 非 root 用戶執行下面命令
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

# root用戶可執行如下命令
export KUBECONFIG=/etc/kubernetes/admin.conf

• 安裝網絡插件

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

• 控制平面節點隔離。默認情況下，出於安全原因，群集不會在控制平面節點上調度 Pod。如果希望能夠在控制平面節點上安排 Pod，例如對於用於開發的單機 Kubernetes 群集，請運行：

kubectl taint nodes --all node-role.kubernetes.io/master-

• 運行 nginx 測試。如果遇到Temporary failure in name resolution這種訪問的問題，請打開非 systemd 空間的控制台操作。

kubectl run nginx --image=nginx --port=80 --hostport=80
curl 127.0.0.1

• 監控集群狀態，大概每兩秒刷新一次。

watch kubectl get all -A

• kubeadm 常用命令
• kubectl 常用命令