一、環境准備
1.1 規划
| 1. 操作系統 CentOS-7.x-x86_64。 2. 關閉 iptables 和 SELinux。 3. 所有節點的主機名和 IP 地址，使用/etc/hosts 做好主機名解析。 |
|||
| 主機名 | IP地址（NAT） | CPU | 內存 |
| k8s-master | eth0 : 10.0.0.25 | 1VCPU | 2G |
| k8s-node-1 | eth0 : 10.0.0.26 | 1VCPU | 2G |
| k8s-node-1 | eth0 : 10.0.0.27 | 1VCPU | 2G |
1.2 網絡設置
1.3 配置靜態IP地址
#將 UUID 和 MAC 地址已經其它配置刪除掉，3個節點除了IP和主機名不同其他相同。
[root@k8s-master ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0 TYPE=Ethernet BOOTPROTO=static NAME=eth0 DEVICE=eth0 ONBOOT=yes IPADDR=10.0.0.25 NETMASK=255.255.255.0 GATEWAY=10.0.0.254 DNS=223.5.5.5 #重啟網絡服務 [root@k8s-master ~]# systemctl restart network #設置 DNS 解析 [root@k8s-master ~]# vi /etc/resolv.conf nameserver 223.5.5.5
1.4 關閉selinux、防火牆
setenforce 0 sed -i 's#SELINUX=enforcing#SELINUX=disabled#' /etc/selinux/config systemctl disable firewalld.service systemctl stop firewalld.service systemctl stop NetworkManager systemctl disable NetworkManager
1.5 設置主機名解析
3個節點都做
cat >>/etc/hosts<<EOF
10.0.0.25 k8s-master
10.0.0.26 k8s-node-1
10.0.0.27 k8s-node-2
EOF
1.6 配置epel源
3個節點都做
rpm -ivh http://mirrors.aliyun.com/epel/epel-release-latest-7.noarch.rpm #下載常用命令 yum install -y net-tools vim lrzsz tree screen lsof tcpdump nc mtr nmap #保證能上網 [root@k8s-master ~]# ping www.baidu.com -c3 PING www.a.shifen.com (61.135.169.121) 56(84) bytes of data. 64 bytes from 61.135.169.121: icmp_seq=1 ttl=128 time=5.41 ms 64 bytes from 61.135.169.121: icmp_seq=2 ttl=128 time=6.55 ms 64 bytes from 61.135.169.121: icmp_seq=3 ttl=128 time=8.97 ms --- www.a.shifen.com ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2023ms rtt min/avg/max/mdev = 5.418/6.981/8.974/1.486 ms
1.7 配置免秘鑰登錄
只在master節點做
[root@k8s-master ~]# ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: b1:a0:5b:02:57:0e:8f:1e:25:bf:46:1f:d1:f3:24:c4 root@k8s-master The key's randomart image is: +--[ RSA 2048]----+ | o o .+. | | X .E . | | . + * o = | | + + + + . | | + + S | | = | | . | | | | | +-----------------+ [root@k8s-master ~]# ssh-copy-id k8s-master The authenticity of host 'k8s-master (10.0.0.25)' can't be established. ECDSA key fingerprint is 75:5c:83:a1:b4:cc:bf:28:71:a5:d5:d1:94:35:3c:9a. Are you sure you want to continue connecting (yes/no)? yes /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys root@k8s-master's password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh 'k8s-master'" and check to make sure that only the key(s) you wanted were added. [root@k8s-master ~]# ssh-copy-id k8s-node-1 The authenticity of host 'k8s-node-1 (10.0.0.26)' can't be established. ECDSA key fingerprint is 75:5c:83:a1:b4:cc:bf:28:71:a5:d5:d1:94:35:3c:9a. Are you sure you want to continue connecting (yes/no)? yes /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys root@k8s-node-1's password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh 'k8s-node-1'" and check to make sure that only the key(s) you wanted were added. [root@k8s-master ~]# ssh-copy-id k8s-node-2 The authenticity of host 'k8s-node-2 (10.0.0.27)' can't be established. ECDSA key fingerprint is 75:5c:83:a1:b4:cc:bf:28:71:a5:d5:d1:94:35:3c:9a. Are you sure you want to continue connecting (yes/no)? yes /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys root@k8s-node-2's password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh 'k8s-node-2'" and check to make sure that only the key(s) you wanted were added.
二、安裝Salt-SSH並克隆本項目代碼
2.1master節點做
2.1 安裝Salt SSH [root@k8s-master ~]# yum install -y https://repo.saltstack.com/yum/redhat/salt-repo-latest-2.el7.noarch.rpm [root@k8s-master ~]# yum install -y salt-ssh git 2.2 獲取項目代碼放置在/srv目錄 [root@k8s-master ~]# git clone https://github.com/unixhot/salt-kubernetes.git [root@k8s-master ~]# cd salt-kubernetes/ [root@k8s-master salt-kubernetes]# mv * /srv/ [root@k8s-master salt-kubernetes]# cd /srv/ [root@k8s-master srv]# cp master /etc/salt/master [root@k8s-master srv]# cp roster /etc/salt/roster 2.3 下載二進制文件 鏈接: https://pan.baidu.com/s/1kJmvR9wzleHGHnSVHaNpMg 密碼: 4taa 下載完成后，將文件移動到/srv/salt/k8s/目錄下。 [root@linux-node1 ~]# cd /srv/salt/k8s/ [root@k8s-master k8s]# unzip k8s-v1.9.3-auto.zip [root@k8s-master k8s]# ls -l files/ 總用量 4 drwxr-xr-x 2 root root 91 3月 28 00:33 cfssl-1.2 drwxrwxr-x 2 root root 4096 3月 27 23:15 cni-plugins-amd64-v0.7.0 drwxr-xr-x 2 root root 31 3月 28 00:33 etcd-v3.3.1-linux-amd64 drwxr-xr-x 2 root root 45 3月 28 12:05 flannel-v0.10.0-linux-amd64 drwxr-xr-x 3 root root 16 3月 28 00:47 k8s-v1.9.3
三、Salt SSH管理的機器以及角色分配
- k8s-role: 用來設置K8S的角色
- etcd-role: 用來設置etcd的角色，如果只需要部署一個etcd，只需要在一台機器上設置即可
- etcd-name: 如果對一台機器設置了etcd-role就必須設置etcd-name
3.1master節點做
[root@k8s-master ~]# vim /etc/salt/roster k8s-master: host: 10.0.0.25 user: root priv: /root/.ssh/id_rsa minion_opts: grains: k8s-role: master etcd-role: node etcd-name: etcd-node1 k8s-node-1: host: 10.0.0.26 user: root priv: /root/.ssh/id_rsa minion_opts: grains: k8s-role: node etcd-role: node etcd-name: etcd-node2 k8s-node-2: host: 10.0.0.27 user: root priv: /root/.ssh/id_rsa minion_opts: grains: k8s-role: node etcd-role: node etcd-name: etcd-node3
四、配置參數
4.1master節點做
[root@k8s-master ~]# vim /srv/pillar/k8s.sls # -*- coding: utf-8 -*- #設置Master的IP地址(必須修改) MASTER_IP: "10.0.0.25" #設置ETCD集群訪問地址（必須修改） ETCD_ENDPOINTS: "https://10.0.0.25:2379,https://10.0.0.26:2379,https://10.0.0.27:2379" #設置ETCD集群初始化列表（必須修改） ETCD_CLUSTER: "etcd-node1=https://10.0.0.25:2380,etcd-node2=https://10.0.0.26:2380,etcd-node3=https://10.0.0.27:2380" #通過Grains FQDN自動獲取本機IP地址，請注意保證主機名解析到本機IP地址 NODE_IP: {{ grains['fqdn_ip4'][0] }} #設置BOOTSTARP的TOKEN，可以自己生成 BOOTSTRAP_TOKEN: "ad6d5bb607a186796d8861557df0d17f" #配置Service IP地址段 SERVICE_CIDR: "10.1.0.0/16" #Kubernetes服務 IP (從 SERVICE_CIDR 中預分配) CLUSTER_KUBERNETES_SVC_IP: "10.1.0.1" #Kubernetes DNS 服務 IP (從 SERVICE_CIDR 中預分配) CLUSTER_DNS_SVC_IP: "10.1.0.2" #設置Node Port的端口范圍 NODE_PORT_RANGE: "20000-40000" #設置POD的IP地址段 POD_CIDR: "10.2.0.0/16" #設置集群的DNS域名 CLUSTER_DNS_DOMAIN: "cluster.local."
五、執行SaltStack狀態
5.1master節點做
5.1 測試Salt SSH聯通性 [root@k8s-master ~]# salt-ssh '*' test.ping
5.2 部署Etcd，由於Etcd是基礎組建，需要先部署，目標為部署etcd的節點。 [root@k8s-master ~]# salt-ssh -L 'k8s-master,k8s-node-1,k8s-node-2' state.sls k8s.etcd5.3 部署K8S集群 由於包比較大，這里執行時間較長，5分鍾+，如果執行有失敗可以再次執行即可！ [root@k8s-master ~]# salt-ssh '*' state.highstate
六、測試Kubernetes安裝
6.1master節點做
[root@k8s-master ~]# source /etc/profile [root@k8s-master ~]# kubectl get cs [root@k8s-master ~]# kubectl get node
七、測試Kubernetes集群和Flannel網絡
7.1master節點做
[root@k8s-master ~]# kubectl run net-test --image=alpine --replicas=2 sleep 360000 需要等待拉取鏡像，可能稍有的慢，請等待。 [root@linux-node1 ~]# kubectl get pod -o wide 測試聯通性，如果都能ping通，說明Kubernetes集群部署完畢，。 [root@k8s-master ~]# ping -c 1 10.2.12.2 PING 10.2.12.2 (10.2.12.2) 56(84) bytes of data. 64 bytes from 10.2.12.2: icmp_seq=1 ttl=61 time=8.72 ms --- 10.2.12.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 8.729/8.729/8.729/0.000 ms [root@k8s-master ~]# ping -c 1 10.2.24.2 PING 10.2.24.2 (10.2.24.2) 56(84) bytes of data. 64 bytes from 10.2.24.2: icmp_seq=1 ttl=61 time=22.9 ms --- 10.2.24.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 22.960/22.960/22.960/0.000 ms
八、如何新增Kubernetes節點
1. 設置主機名解析 2. 設置SSH無密碼登錄 3. 在/etc/salt/roster里面，增加對應的機器IP 4. 執行SaltStack狀態：salt-ssh '*' state.highstate 5. [root@k8s-node-2 ~]# vim /etc/salt/roster k8s-node-3: host: 10.0.0.28 user: root priv: /root/.ssh/id_rsa minion_opts: grains: k8s-role: node 6. [root@linux-node1 ~]# salt-ssh '*' state.highstate