目錄
學習文檔： https://github.com/gjmzj/kubeasz/
1、環境說明
| IP | 主機名 | 角色 | 虛擬機配置 |
|---|---|---|---|
| 192.168.56.11 | k8s-master | deploy、master1、lb1、etcd | 4c4g |
| 192.168.56.12 | k8s-master2 | master2、lb2 | 4c4g |
| 192.168.56.13 | k8s-node01 | etcd、node | 2c2g |
| 192.168.56.14 | k8s-node02 | etcd、node | 2c2g |
| 192.168.56.110 | vip | ||
| 系統內核 | 3.10 | docker版本 | 18.09 |
| k8s版本 | 1.13 | etcd版本 | 3.0 |
2、准備工作
- 四台機器，全部執行：
yum install -y epel-release
yum update -y
yum install python -y
- deploy節點安裝ansible並配置密鑰認證
yum install -y ansible
ssh-keygen
for ip in 11 12 13 14;do ssh-copy-id 192.168.56.$ip;done
- deploy節點編排K8S
[root@k8s-master ~]# git clone https://github.com/gjmzj/kubeasz.git
[root@k8s-master ~]# mv kubeasz/* /etc/ansible/
從百度雲網盤下載二進制文件 https://pan.baidu.com/s/1c4RFaA#list/path=%2F
可以根據自己所需版本，下載對應的tar包，這里我下載1.13
經過一番折騰，最終把k8s.1-13-5.tar.gz的tar包放到了depoly上
[root@k8s-master ~]# tar -zxf k8s.1-13-5.tar.gz
[root@k8s-master ~]# mv bin/* /etc/ansible/bin/
- 配置集群參數
[root@k8s-master ~]# cd /etc/ansible/
[root@k8s-master ansible]# cp example/hosts.m-masters.example hosts
cp: overwrite ‘hosts’? y
[root@k8s-master ansible]# vim hosts #根據實際情況的ip進行更改
[deploy]
192.168.56.11 NTP_ENABLED=no #設置集群是否安裝 chrony 時間同步
[etcd] #etcd集群請提供如下NODE_NAME，注意etcd集群必須是1,3,5,7...奇數個節點
192.168.56.11 NODE_NAME=etcd1
192.168.56.13 NODE_NAME=etcd2
192.168.56.14 NODE_NAME=etcd3
[kube-master]
192.168.56.11
192.168.56.12
[kube-node]
192.168.56.13
192.168.56.14
[lb] # 負載均衡(目前已支持多於2節點，一般2節點就夠了) 安裝 haproxy+keepalived
192.168.56.12 LB_ROLE=backup
192.168.56.11 LB_ROLE=master
## 集群 MASTER IP即 LB節點VIP地址，為區別與默認apiserver端口，設置VIP監聽的服務端口8443
# 公有雲上請使用雲負載均衡內網地址和監聽端口
[all:vars]
DEPLOY_MODE=multi-master
MASTER_IP="192.168.56.110" #設置vip
KUBE_APISERVER="https://{{ MASTER_IP }}:8443"
CLUSTER_NETWORK="flannel"
SERVICE_CIDR="10.68.0.0/16"
CLUSTER_CIDR="172.20.0.0/16"
NODE_PORT_RANGE="20000-40000"
CLUSTER_KUBERNETES_SVC_IP="10.68.0.1"
CLUSTER_DNS_SVC_IP="10.68.0.2"
CLUSTER_DNS_DOMAIN="cluster.local."
bin_dir="/opt/kube/bin"
ca_dir="/etc/kubernetes/ssl"
base_dir="/etc/ansible"
#修改完成后，測試hosts
[root@k8s-master ansible]# ansible all -m ping
192.168.56.12 | SUCCESS => {
"changed": false,
"ping": "pong"
}
192.168.56.13 | SUCCESS => {
"changed": false,
"ping": "pong"
}
192.168.56.14 | SUCCESS => {
"changed": false,
"ping": "pong"
}
192.168.56.11 | SUCCESS => {
"changed": false,
"ping": "pong"
}
3、分步驟安裝
3.1、創建證書和安裝准備
[root@k8s-master ansible]# ansible-playbook 01.prepare.yml
3.2、安裝etcd集群
[root@k8s-master ansible]# ansible-playbook 02.etcd.yml
[root@k8s-master ansible]# bash
#驗證etcd集群狀態
[root@k8s-master ansible]# systemctl status etcd
#在任一 etcd 集群節點上執行如下命令
[root@k8s-master ansible]# for ip in 11 13 14;do ETCDCTL_API=3 etcdctl --endpoints=https://192.168.56.$ip:2379 --cacert=/etc/kubernetes/ssl/ca.pem --cert=/etc/etcd/ssl/etcd.pem --key=/etc/etcd/ssl/etcd-key.pem endpoint health;done
https://192.168.56.11:2379 is healthy: successfully committed proposal: took = 7.967375ms
https://192.168.56.13:2379 is healthy: successfully committed proposal: took = 12.557643ms
https://192.168.56.14:2379 is healthy: successfully committed proposal: took = 9.70078ms
3.3、安裝docker
[root@k8s-master ansible]# ansible-playbook 03.docker.yml
3.4、安裝master節點
[root@k8s-master ansible]# ansible-playbook 04.kube-master.yml
#查看進程狀態
[root@k8s-master ansible]# systemctl status kube-apiserver
[root@k8s-master ansible]# systemctl status kube-controller-manager
[root@k8s-master ansible]# systemctl status kube-scheduler
[root@k8s-master ansible]# kubectl get componentstatus #查看集群狀態
NAME STATUS MESSAGE ERROR
scheduler Healthy ok
controller-manager Healthy ok
etcd-0 Healthy {"health":"true"}
etcd-1 Healthy {"health":"true"}
etcd-2 Healthy {"health":"true"}
3.5、安裝node節點
[root@k8s-master ansible]# ansible-playbook 05.kube-node.yml
[root@k8s-master ansible]# systemctl status kubelet
[root@k8s-master ansible]# systemctl status kube-proxy
[root@k8s-master ansible]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
192.168.56.11 Ready,SchedulingDisabled master 6m56s v1.13.5
192.168.56.12 Ready,SchedulingDisabled master 6m57s v1.13.5
192.168.56.13 Ready node 40s v1.13.5
192.168.56.14 Ready node 40s v1.13.5
3.6、部署集群網絡
[root@k8s-master ansible]# ansible-playbook 06.network.yml
[root@k8s-master ansible]# kubectl get pod -n kube-system #查看flannel相關pod
NAME READY STATUS RESTARTS AGE
kube-flannel-ds-amd64-856rg 1/1 Running 0 115s
kube-flannel-ds-amd64-j4542 1/1 Running 0 115s
kube-flannel-ds-amd64-q9cmh 1/1 Running 0 115s
kube-flannel-ds-amd64-rhg66 1/1 Running 0 115s
3.7、部署集群插件（dns，dashboard）
[root@k8s-master ansible]# ansible-playbook 07.cluster-addon.yml
[root@k8s-master ansible]# kubectl get svc -n kube-system #查看服務
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
heapster ClusterIP 10.68.29.48 <none> 80/TCP 64s
kube-dns ClusterIP 10.68.0.2 <none> 53/UDP,53/TCP,9153/TCP 71s
kubernetes-dashboard NodePort 10.68.117.7 <none> 443:24190/TCP 64s
metrics-server ClusterIP 10.68.107.56 <none> 443/TCP 69s
[root@k8s-master ansible]# kubectl cluster-info #查看集群信息
Kubernetes master is running at https://192.168.56.110:8443
CoreDNS is running at https://192.168.56.110:8443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
kubernetes-dashboard is running at https://192.168.56.110:8443/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
[root@k8s-master ansible]# kubectl top node #查看節點資源使用率
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
192.168.56.11 523m 13% 2345Mi 76%
192.168.56.12 582m 15% 1355Mi 44%
192.168.56.13 182m 10% 791Mi 70%
192.168.56.14 205m 11% 804Mi 71%
一步ansible安裝k8s集群命令如下：
ansible-playbook 90.setup.yml
3.8、測試DNS解析
[root@k8s-master ansible]# kubectl run nginx --image=nginx --expose --port=80
[root@k8s-master ansible]# kubectl run busybox --rm -it --image=busybox /bin/sh
/ # nslookup nginx.default.svc.cluster.local
Server: 10.68.0.2
Address: 10.68.0.2:53
Name: nginx.default.svc.cluster.local
Address: 10.68.149.79