關於flannel插件的原理園內有許多其他的優秀文章，可以參考。此處不做描述。
二進制部署步驟：
1.到github上下載flannel二進制包：https://github.com/flannel-io/flannel/releases
2.解壓后得到兩個文件：
flanneld ——————> 作用：flannel的服務主程序
mk-docker-opts.sh ——>作用：講flannel主程序創建的subnet.env文件改寫成可被符合dockerd服務規范的system參數文件
3.編輯/usr/lib/systemd/system/flannel.service
[Unit] Description=Kubernetes API Server After=network.target [Service] #EnvironmentFile=-/cloud/k8s/conf/kube-flannel.conf #ExecStart=/cloud/k8s/bin/flanneld $FLANNEL_OPTS #flanneld啟動參數 ExecStart=/cloud/k8s/bin/flanneld \ #和etcd通信的物理網卡IP --public-ip=10.9.0.29 \ --etcd-endpoints=https://10.9.0.29:2379,https://10.9.0.33:2379,https://10.9.0.37:2379 \ --etcd-keyfile=/cloud/k8s/cert/client-key.pem \ --etcd-certfile=/cloud/k8s/cert/client.pem \ --etcd-cafile=/cloud/k8s/cert/ca.pem \ #和etcd通信的物理網卡名 --iface=tun0 \ #設置自動生成的subnet.env文件的保存位置 --subnet-file=/cloud/k8s/conf/subnet.env \ #flanneld程序監聽端口 --healthz-port=2401 #重要：flanneld程序啟動后，會生成subnet.env文件，可以設置systemd在啟動flanneld后調用上面的安裝包內的mk-docker-opts.sh腳本根據flanneld程序生成的subnet.env文件內的參數重新生成成可以被dockerd程序識別新env文件。 ExecStartPost=/k8s/flannel/bin/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /cloud/k8s/conf/docker-subnet.env Restart=no RestartSec=30 Type=notify LimitNOFILE=65536 [Install] WantedBy=multi-user.target
4.在etcd內設置subnet的信息：
提示：由於flanneld僅支持etcdv2版本的api命令，而且v2和新的v3不兼容，所以設置的時候也需要使用etcdctl⒉版本的命令，否則，在flanneld啟動后會出現"Couldn't fetch network config: SubnetMax is not in the range of the Network"的錯誤。
ETCDCTL_API=2 /cloud/etcd/bin/etcdctl --endpoints=https://10.9.0.33:2379 --ca-file=/cloud/etcd/cert/ca.pem --cert-file=/cloud/etcd/cert/etcd.pem --key-file=/cloud/etcd/cert/etcd-key.pem put /coreos.com/network/config '{"Network": "10.18.0.0/16", "SubnetLen": 24, "SubnetMin":"10.18.1.0","SubnetMax": "10.18.20.0", "Backend": {"Type": "vxlan"}}'
查看已插入的數據：
[root@k8s-220 ~]# ETCDCTL_API=2 /cloud/etcd/bin/etcdctl --endpoints=https://10.9.0.33:2379 --ca-file=/cloud/etcd/cert/ca.pem --cert-file=/cloud/etcd/cert/etcd.pem --key-file=/cloud/etcd/cert/etcd-key.pem get /coreos.com/network/config/ {"Network": "10.18.0.0/16", "SubnetLen": 24, "SubnetMin":"10.18.1.0","SubnetMax": "10.18.20.0", "Backend": {"Type": "vxlan"}}
5.啟動flannel：
systemctl start flannel
查看flanneld生成的/cloud/k8s/conf/subnet.env
#flannel網絡的主網段
FLANNEL_NETWORK=10.18.0.0/16
#此flanneld服務所取用的主網段中的子網段 FLANNEL_SUBNET=10.18.18.1/24 FLANNEL_MTU=1450 FLANNEL_IPMASQ=false
查看flanneld生成的/cloud/k8s/conf/docker-subnet.env
DOCKER_OPT_BIP="--bip=10.18.18.1/24" DOCKER_OPT_IPMASQ="--ip-masq=true" DOCKER_OPT_MTU="--mtu=1450" DOCKER_OPTS=" --bip=10.18.18.1/24 --ip-masq=true --mtu=1450"
可以發現，docker-subnet.env只是經過mk-docker-opts.sh的處理可被systemd調用的EnvironmentFile參數文件。
6.編輯/usr/lib/systemd/system/docker.service
[Unit] Description=Docker Application Container Engine Documentation=https://docs.docker.com After=network-online.target firewalld.service containerd.service Wants=network-online.target Requires=docker.socket containerd.service [Service] Type=notify #此處設置的就是上面flanneld生成的docker-subnet.env文件的路徑 EnvironmentFile=/cloud/k8s/conf/docker-subnet.env #在原生的啟動命令后添加docker-subnet.env文件中的$DOCKER_OPTS $DOCKER_CGROUPS參數 ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock $DOCKER_OPTS $DOCKER_CGROUPS ExecReload=/bin/kill -s HUP $MAINPID TimeoutSec=0 RestartSec=2 Restart=always StartLimitBurst=3 StartLimitInterval=60s LimitNOFILE=infinity LimitNPROC=infinity LimitCORE=infinity TasksMax=infinity containers Delegate=yes KillMode=process OOMScoreAdjust=-500 [Install] WantedBy=multi-user.target
7.啟動dockerd
systemctl start docker
8.查看docker0網卡
[root@k8s-220 conf]# ifconfig docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450 #可以看到，docker0調用的就是flannel.1網卡所給的網段。 inet 10.18.18.1 netmask 255.255.255.0 broadcast 10.18.18.255 inet6 fe80::42:9fff:fed4:f022 prefixlen 64 scopeid 0x20<link> ether 02:42:9f:d4:f0:22 txqueuelen 0 (Ethernet) RX packets 4 bytes 224 (224.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 9 bytes 726 (726.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.1.220 netmask 255.255.255.0 broadcast 192.168.1.255 inet6 fe80::20c:29ff:fe79:8bef prefixlen 64 scopeid 0x20<link> ether 00:0c:29:79:8b:ef txqueuelen 1000 (Ethernet) RX packets 37871410 bytes 14333747643 (13.3 GiB) RX errors 0 dropped 148 overruns 0 frame 0 TX packets 35136117 bytes 16113012411 (15.0 GiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 flannel.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450 inet 10.18.18.0 netmask 255.255.255.255 broadcast 10.18.18.0 inet6 fe80::7ca9:5fff:fe76:be4c prefixlen 64 scopeid 0x20<link> ether 7e:a9:5f:76:be:4c txqueuelen 0 (Ethernet) RX packets 5 bytes 420 (420.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 8 bytes 672 (672.0 B) TX errors 0 dropped 5 overruns 0 carrier 0 collisions 0
9.查看etcd的網段分配記錄:
[root@k8s-220 ~]# ETCDCTL_API=2 /cloud/etcd/bin/etcdctl --endpoints=https://10.9.0.33:2379 --ca-file=/cloud/etcd/cert/ca.pem --cert-file=/cloud/etcd/cert/etcd.pem --key-file=/cloud/etcd/cert/etcd-key.pem ls /coreos.com/network/ /coreos.com/network/config /coreos.com/network/subnets [root@k8s-220 ~]# ETCDCTL_API=2 /cloud/etcd/bin/etcdctl --endpoints=https://10.9.0.33:2379 --ca-file=/cloud/etcd/cert/ca.pem --cert-file=/cloud/etcd/cert/etcd.pem --key-file=/cloud/etcd/cert/etcd-key.pem ls /coreos.com/network/subnets /coreos.com/network/subnets/10.18.8.0-24 /coreos.com/network/subnets/10.18.13.0-24 /coreos.com/network/subnets/10.18.14.0-24 /coreos.com/network/subnets/10.18.18.0-24
避坑小貼士：通常情況下是已經啟動了dockerd服務，已經分配了172.17.0.0網段的IP地址給了docker0，此時再想更換docker0的IP地址，會發現即使按照上面的方法設置重啟了docker服務后，docker0網卡的IP依然后默認的172.17.0.0網段的。其中一個重要的原因是，docker服務中存在未刪除的docker容器，包括沒有啟動的容器，必須全部刪除已經存在的容器后再systemctl restart docker，才能修改docker0的默認地址。換句話說，docker --bip 只有在沒有容器存在的情況下，才能起作用。當然也可以使用“重啟系統”的終極大招。