使用kubeadm部署k8s集群
環境
| IP地址 | 主機名 | 節點 |
|---|---|---|
| 10.0.0.63 | k8s-master1 | master1 |
| 10.0.0.63 | k8s-master2 | master2 |
| 10.0.0.65 | k8s-node1 | node1 |
| 10.0.0.66 | k8s-node2 | node2 |
1. 簡要
kubeadm是官方社區推出的快速部署kubernetes集群工具
部署環境適用於學習和使用k8s相關軟件和功能
2. 安裝要求
3台純凈centos虛擬機,版本為7.x及以上
機器配置 2核4G以上 x3台
服務器網絡互通
禁止swap分區
3. 學習目標
學會使用kubeadm來安裝一個集群,便於學習k8s相關知識
4. 環境准備
# 1. 關閉防火牆功能
systemctl stop firewalld
systemctl disable firewalld
# 2.關閉selinux
sed -i 's/enforcing/disabled/' /etc/selinux/config
setenforce 0
# 3. 關閉swap
swapoff -a
#或將命令加入開機啟動
echo "swapoff -a" >>/etc/profile
# 4. 服務器規划
cat > /etc/hosts << EOF
10.0.0.63 k8s-master1
10.0.0.64 k8s-master2
10.0.0.65 k8s-node1
10.0.0.66 k8s-node2
EOF
#5. 臨時主機名配置方法:
hostnamectl set-hostname k8s-master1
bash
#6. 時間同步配置
yum install -y ntpdate
ntpdate time.windows.com
#開啟轉發
cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system
# 以上可以全部復制粘貼直接運行,但是主機名配置需要重新修改
5. docker安裝[所有節點都需要安裝]
#源添加
wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O /etc/yum.repos.d/docker-ce.repo
# 安裝指定版版本
yum -y install docker-ce-18.09.9-3.el7
#也可以查看版本安裝
yum list docker-ce --showduplicates | sort -r
#啟動docker
systemctl start docker
6. docker配置cgroup驅動[所有節點]
rm -f /etc/docker/*
sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{
"registry-mirrors": ["https://ajvcw8qn.mirror.aliyuncs.com"]
}
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker
7. 鏡像加速[所有節點]
curl -sSL https://get.daocloud.io/daotools/set_mirror.sh | sh -s http://f1361db2.m.daocloud.io
systemctl restart docker
#如果源太多容易出錯. 錯了就刪除一個.bak源試試看
#保留 curl -sSL https://get.daocloud.io/daotools/set_mirror.sh | sh -s http://f1361db2.m.daocloud.io
這個是阿里雲配置的加速
https://cr.console.aliyun.com/cn-hangzhou/instances/mirrors
8.kubernetes源配置[所有節點]
cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
9. 安裝kubeadm，kubelet和kubectl[所有節點]
yum install -y kubelet-1.17.0 kubeadm-1.17.0 kubectl-1.17.0
systemctl enable kubelet
10. 部署Kubernetes Master [ master 10.0.0.63]
kubeadm init \
--apiserver-advertise-address=10.0.0.63 \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version v1.17.0 \
--service-cidr=10.1.0.0/16 \
--pod-network-cidr=10.244.0.0/16
初始化后獲取到token:
kubeadm join 10.0.0.63:6443 --token 0dr1pw.ejybkufnjpalb8k6
--discovery-token-ca-cert-hash sha256:ca1aa9cb753a26d0185e3df410cad09d8ec4af4d7432d127f503f41bc2b14f2a
記住token,后面使用
10.1. kubectl命令工具配置[master]：
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
#獲取節點信息
# kubectl get nodes
[root@k8s-master1 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master1 Ready master 38m v1.17.0
10.2. 安裝網絡插件[master]
上傳kube-flannel.yaml,並執行:
kubectl apply -f kube-flannel.yaml
kubectl get pods -n kube-system
[必須全部運行起來,否則有問題.]
[root@k8s-master1 ~]# kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-9d85f5447-6jlmq 1/1 Running 1 14h
coredns-9d85f5447-qfw8w 1/1 Running 1 14h
etcd-k8s-master1 1/1 Running 3 14h
kube-apiserver-k8s-master1 1/1 Running 3 14h
kube-controller-manager-k8s-master1 1/1 Running 3 14h
kube-flannel-ds-amd64-6lq9g 1/1 Running 1 14h
kube-flannel-ds-amd64-88hwc 1/1 Running 0 13h
kube-flannel-ds-amd64-dgkwm 1/1 Running 0 13h
kube-flannel-ds-amd64-pmh75 1/1 Running 0 13h
kube-proxy-7xhnk 1/1 Running 3 14h
kube-proxy-kdp8h 1/1 Running 0 13h
kube-proxy-rg72z 1/1 Running 0 13h
kube-proxy-xnx5m 1/1 Running 0 13h
kube-scheduler-k8s-master1 1/1 Running 3 14h
11. 將node1 node2 加入master
node1加入配置
在要加入的節點種執行以下命令來加入:
kubeadm join 10.0.0.63:6443 --token fs0uwh.7yuiawec8tov5igh \
--discovery-token-ca-cert-hash sha256:471442895b5fb77174103553dc13a4b4681203fbff638e055ce244639342701d
#這個配置在安裝master的時候有過提示,請注意首先要配置 cni網絡:
# node1加入master:
[root@k8s-node1 ~]# kubeadm join 10.0.0.63:6443 --token v3p5pj.1wsrd7ybs6z14o9f \
> --discovery-token-ca-cert-hash sha256:ef5fb837f9b66fa9742b236b373203df63b1d71651dfc7211f40150d20516d85
[preflight] Running pre-flight checks
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.16" ConfigMap in the kube-system namespace
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Activating the kubelet service
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
#返回值判斷對錯,0對,其他值均為錯
[root@k8s-node1 ~]# echo $?
0
#加入成功后,master節點檢測:
[root@k8s-master ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master Ready master 22m v1.17.0
k8s-node1 Ready <none> 2m9s v1.17.0 <---- node1 成功加入到集群中
#node2加入master:
[root@k8s-node2 docker]# kubeadm join 10.0.0.63:6443 --token v3p5pj.1wsrd7ybs6z14o9f \
> --discovery-token-ca-cert-hash sha256:ef5fb837f9b66fa9742b236b373203df63b1d71651dfc7211f40150d20516d85
[preflight] Running pre-flight checks
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.16" ConfigMap in the kube-system namespace
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Activating the kubelet service
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
[root@k8s-node2 docker]# echo $?
0
# master2 加入 一樣的操作.
#加入成功后,master節點檢測:
[root@k8s-master1 docker]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master1 Ready master 76m v1.17.0
k8s-master2 Ready <none> 79s v1.17.0
k8s-node1 Ready <none> 8m23s v1.17.0
k8s-node2 Ready <none> 6m v1.17.0
12 token創建和查詢
默認token會保存24消失,過期后就不可用,如果需要重新建立token,可在master節點使用以下命令重新生成:
kubeadm token create
kubeadm token list
openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
63bca849e0e01691ae14eab449570284f0c3ddeea590f8da988c07fe2729e924
kubeadm join 10.0.0.63:6443 --discovery-token nuja6n.o3jrhsffiqs9swnu --discovery-token-ca-cert-hash 63bca849e0e01691ae14eab449570284f0c3ddeea590f8da988c07fe2729e924
13. 創建一個nginx鏡像
kubectl create deployment nginx --image=nginx
kubectl expose deployment nginx --port=80 --type=NodePort
kubectl get pod,svc
查看狀態,nginx已經啟動端口為:31907:
此時可以通過 集群的3台服務器期中一台來測試訪問:
14.部署dashboard圖形界面
上傳 kubernetes-dashboard.yaml
[root@k8s-master ~]# kubectl apply -f dashboard.yaml
[root@k8s-master ~]# kubectl get pods -n kubernetes-dashboard
[root@k8s-master ~]# kubectl get pods -n kubernetes-dashboard
NAME READY STATUS RESTARTS AGE
dashboard-metrics-scraper-566cddb686-xjgks 1/1 Running 0 39s
kubernetes-dashboard-7b5bf5d559-8ssvj 1/1 Running 0 39s
15. 獲取dashboard token, 也就是創建service account並綁定默認cluster-admin管理員集群角色
# kubectl create serviceaccount dashboard-admin -n kubernetes-dashboard
# kubectl create clusterrolebinding dashboard-admin --clusterrole=cluster-admin --serviceaccount=kubernetes-dashboard:dashboard-admin
# kubectl describe secrets -n kubernetes-dashboard $(kubectl -n kubernetes-dashboard get secret | awk '/dashboard-admin/{print $1}')
將復制的token 填寫到 上圖中的 token選項,並選擇token登錄
登錄后界面:
報錯處理:
k8s-node節點加入時報錯:
W0315 22:16:20.123204 5795 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.
[preflight] Running pre-flight checks
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables contents are not set to 1
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
處理辦法:
echo "1" >/proc/sys/net/bridge/bridge-nf-call-iptables
增加后重新加入:
kubeadm join 10.0.0.63:6443 --token 0dr1pw.ejybkufnjpalb8k6 --discovery-token-ca-cert-hash sha256:ca1aa9cb753a26d0185e3df410cad09d8ec4af4d7432d127f503f41bc2b14f2a
這里的token由kubadm服務器生成.
web頁面無法訪問處理:
重建dashboard
刪除:
kubectl delete -f dashboard.yaml
刪除后創建:
kubectl create -f dashboard.yaml
創建賬戶:
kubectl create serviceaccount dashboard-admin -n kubernetes-dashboard
查看密碼:
kubectl describe secrets -n kubernetes-dashboard $(kubectl -n kubernetes-dashboard get secret | awk '/dashboard-admin/{print $1}')
重新打開登錄即可
YAML附件[請保存為 .yaml 為后綴]
http://www.chenleilei.net/soft/kubeadm快速部署一個Kubernetes集群yaml.zip