Flannel容器集群網絡部署
Overlay Network：覆蓋網絡，在基礎網絡上疊加的一種虛擬網絡技術模式，該網絡中的主機通過虛擬鏈路連接起來。
VXLAN：將源數據包封裝到UDP中，並使用基礎網絡的IP/MAC作為外層報文頭進行封裝，然后在以太網上傳輸，到達目的地后由隧道端點解封裝並將數據發送給目標地址。
Flannel：是Overlay網絡的一種，也是將源數據包封裝在另一種網絡包里面進行路由轉發和通信，目前已經支持UDP、VXLAN、AWS VPC和GCE路由等數據轉發方式。
flannel工作原理：
Falnnel要用etcd存儲自身一個子網信息，所以要保證能成功連接Etcd，寫入預定義子網段：
etcdctl --endpoints=https://192.168.0.123:2379,https://192.168.0.125:2379,https://192.168.0.126:2379 \ --ca-file=/opt/kubernetes/ssl/ca.pem \ --cert-file=/opt/kubernetes/ssl/etcd.pem \ --key-file=/opt/kubernetes/ssl/etcd-key.pem \ set /coreos.com/network/config '{ "Network": "172.17.0.0/16", "Backend": {"Type": "vxlan", "DirectRouting": true}}'
准備二進制包
wget https://github.com/coreos/flannel/releases/download/v0.10.0/flannel-v0.10.0-linux-amd64.tar.gz tar xf flannel-v0.10.0-linux-amd64.tar.gz scp flanneld mk-docker-opts.sh 192.168.0.125:/opt/kubernetes/bin/ scp flanneld mk-docker-opts.sh 192.168.0.126:/opt/kubernetes/bin/
flannel配置文件
[root@k8s-node02 bin]# vim /opt/kubernetes/cfg/flanneld FLANNEL_OPTIONS="--etcd-endpoints=https://192.168.0.123:2379,https://192.168.0.125:2379,https://192.168.0.126:2379 \ -etcd-cafile=/opt/kubernetes/ssl/ca.pem \ -etcd-certfile=/opt/kubernetes/ssl/etcd.pem \ -etcd-keyfile=/opt/kubernetes/ssl/etcd-key.pem"
配置flannel系統服務
[root@k8s-node01 ~]# vim /usr/lib/systemd/system/flanneld.service [Unit] Description=Flanneld overlay address etcd agent After=network-online.target network.target Before=docker.service [Service] Type=notify EnvironmentFile=/opt/kubernetes/cfg/flanneld ExecStart=/opt/kubernetes/bin/flanneld --ip-masq $FLANNEL_OPTIONS ExecStartPost=/opt/kubernetes/bin/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/subnet.env Restart=on-failure [Install] WantedBy=multi-user.target
將配置復制到其他node節點
[root@k8s-node01 ~]# scp /opt/kubernetes/cfg/flanneld 192.168.0.126:/opt/kubernetes/cfg [root@k8s-node01 ~]# scp /usr/lib/systemd/system/flanneld.service 192.168.0.126:/usr/lib/systemd/system/
啟動flannel
systemctl daemon-reload
systemctl enable flanneld
systemctl restart flanneld
systemctl status flanneld
安裝docker
yum install -y yum-utils device-mapper-persistent-data lvm2 yum-config-manager \ --add-repo \ https://download.docker.com/linux/centos/docker-ce.repo yum install docker-ce -y curl -sSL https://get.daocloud.io/daotools/set_mirror.sh | sh -s http://bc437cce.m.daocloud.io systemctl start docker systemctl enable docker
配置docker使用flannel網絡
[root@k8s-node01 ~]# vim /usr/lib/systemd/system/docker.service [Unit] Description=Docker Application Container Engine Documentation=https://docs.docker.com After=network-online.target firewalld.service Wants=network-online.target [Service] Type=notify EnvironmentFile=/run/flannel/subnet.env ExecStart=/usr/bin/dockerd $DOCKER_NETWORK_OPTIONS ExecReload=/bin/kill -s HUP $MAINPID LimitNOFILE=infinity LimitNPROC=infinity LimitCORE=infinity TimeoutStartSec=0 Delegate=yes KillMode=process Restart=on-failure StartLimitBurst=3 StartLimitInterval=60s [Install] WantedBy=multi-user.target
重新加載
systemctl daemon-reload
systemctl restart docker
查看網絡信息，確保docker0 和flannel同網段
[root@k8s-node01 ~]# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:80:79:49 brd ff:ff:ff:ff:ff:ff inet 192.168.0.125/24 brd 192.168.0.255 scope global ens32 valid_lft forever preferred_lft forever inet6 fe80::20c:29ff:fe80:7949/64 scope link valid_lft forever preferred_lft forever 3: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default link/ether fe:65:1b:16:27:46 brd ff:ff:ff:ff:ff:ff inet 172.17.84.0/32 scope global flannel.1 valid_lft forever preferred_lft forever inet6 fe80::fc65:1bff:fe16:2746/64 scope link valid_lft forever preferred_lft forever 4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default link/ether 02:42:5e:ab:96:76 brd ff:ff:ff:ff:ff:ff inet 172.17.84.1/24 brd 172.17.84.255 scope global docker0 valid_lft forever preferred_lft forever [root@k8s-node02 ~]# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:7a:e6:7b brd ff:ff:ff:ff:ff:ff inet 192.168.0.126/24 brd 192.168.0.255 scope global ens32 valid_lft forever preferred_lft forever inet6 fe80::20c:29ff:fe7a:e67b/64 scope link valid_lft forever preferred_lft forever 3: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default link/ether c6:53:99:79:c0:cc brd ff:ff:ff:ff:ff:ff inet 172.17.34.0/32 scope global flannel.1 valid_lft forever preferred_lft forever inet6 fe80::c453:99ff:fe79:c0cc/64 scope link valid_lft forever preferred_lft forever 4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default link/ether 02:42:37:33:61:29 brd ff:ff:ff:ff:ff:ff inet 172.17.34.1/24 brd 172.17.34.255 scope global docker0 valid_lft forever preferred_lft forever
測試不同節點互通，在當前節點訪問另一個Node節點docker0 IP：
[root@k8s-node01 ~]# ping 172.17.34.1 PING 172.17.34.1 (172.17.34.1) 56(84) bytes of data. 64 bytes from 172.17.34.1: icmp_seq=1 ttl=64 time=0.435 ms 64 bytes from 172.17.34.1: icmp_seq=2 ttl=64 time=0.263 ms