关于flannel插件的原理园内有许多其他的优秀文章，可以参考。此处不做描述。
二进制部署步骤：
1.到github上下载flannel二进制包：https://github.com/flannel-io/flannel/releases
2.解压后得到两个文件：
flanneld ——————> 作用：flannel的服务主程序
mk-docker-opts.sh ——>作用：讲flannel主程序创建的subnet.env文件改写成可被符合dockerd服务规范的system参数文件
3.编辑/usr/lib/systemd/system/flannel.service
[Unit] Description=Kubernetes API Server After=network.target [Service] #EnvironmentFile=-/cloud/k8s/conf/kube-flannel.conf #ExecStart=/cloud/k8s/bin/flanneld $FLANNEL_OPTS #flanneld启动参数 ExecStart=/cloud/k8s/bin/flanneld \ #和etcd通信的物理网卡IP --public-ip=10.9.0.29 \ --etcd-endpoints=https://10.9.0.29:2379,https://10.9.0.33:2379,https://10.9.0.37:2379 \ --etcd-keyfile=/cloud/k8s/cert/client-key.pem \ --etcd-certfile=/cloud/k8s/cert/client.pem \ --etcd-cafile=/cloud/k8s/cert/ca.pem \ #和etcd通信的物理网卡名 --iface=tun0 \ #设置自动生成的subnet.env文件的保存位置 --subnet-file=/cloud/k8s/conf/subnet.env \ #flanneld程序监听端口 --healthz-port=2401 #重要：flanneld程序启动后，会生成subnet.env文件，可以设置systemd在启动flanneld后调用上面的安装包内的mk-docker-opts.sh脚本根据flanneld程序生成的subnet.env文件内的参数重新生成成可以被dockerd程序识别新env文件。 ExecStartPost=/k8s/flannel/bin/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /cloud/k8s/conf/docker-subnet.env Restart=no RestartSec=30 Type=notify LimitNOFILE=65536 [Install] WantedBy=multi-user.target
4.在etcd内设置subnet的信息：
提示：由于flanneld仅支持etcdv2版本的api命令，而且v2和新的v3不兼容，所以设置的时候也需要使用etcdctl⒉版本的命令，否则，在flanneld启动后会出现"Couldn't fetch network config: SubnetMax is not in the range of the Network"的错误。
ETCDCTL_API=2 /cloud/etcd/bin/etcdctl --endpoints=https://10.9.0.33:2379 --ca-file=/cloud/etcd/cert/ca.pem --cert-file=/cloud/etcd/cert/etcd.pem --key-file=/cloud/etcd/cert/etcd-key.pem put /coreos.com/network/config '{"Network": "10.18.0.0/16", "SubnetLen": 24, "SubnetMin":"10.18.1.0","SubnetMax": "10.18.20.0", "Backend": {"Type": "vxlan"}}'
查看已插入的数据：
[root@k8s-220 ~]# ETCDCTL_API=2 /cloud/etcd/bin/etcdctl --endpoints=https://10.9.0.33:2379 --ca-file=/cloud/etcd/cert/ca.pem --cert-file=/cloud/etcd/cert/etcd.pem --key-file=/cloud/etcd/cert/etcd-key.pem get /coreos.com/network/config/ {"Network": "10.18.0.0/16", "SubnetLen": 24, "SubnetMin":"10.18.1.0","SubnetMax": "10.18.20.0", "Backend": {"Type": "vxlan"}}
5.启动flannel：
systemctl start flannel
查看flanneld生成的/cloud/k8s/conf/subnet.env
#flannel网络的主网段
FLANNEL_NETWORK=10.18.0.0/16
#此flanneld服务所取用的主网段中的子网段 FLANNEL_SUBNET=10.18.18.1/24 FLANNEL_MTU=1450 FLANNEL_IPMASQ=false
查看flanneld生成的/cloud/k8s/conf/docker-subnet.env
DOCKER_OPT_BIP="--bip=10.18.18.1/24" DOCKER_OPT_IPMASQ="--ip-masq=true" DOCKER_OPT_MTU="--mtu=1450" DOCKER_OPTS=" --bip=10.18.18.1/24 --ip-masq=true --mtu=1450"
可以发现，docker-subnet.env只是经过mk-docker-opts.sh的处理可被systemd调用的EnvironmentFile参数文件。
6.编辑/usr/lib/systemd/system/docker.service
[Unit] Description=Docker Application Container Engine Documentation=https://docs.docker.com After=network-online.target firewalld.service containerd.service Wants=network-online.target Requires=docker.socket containerd.service [Service] Type=notify #此处设置的就是上面flanneld生成的docker-subnet.env文件的路径 EnvironmentFile=/cloud/k8s/conf/docker-subnet.env #在原生的启动命令后添加docker-subnet.env文件中的$DOCKER_OPTS $DOCKER_CGROUPS参数 ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock $DOCKER_OPTS $DOCKER_CGROUPS ExecReload=/bin/kill -s HUP $MAINPID TimeoutSec=0 RestartSec=2 Restart=always StartLimitBurst=3 StartLimitInterval=60s LimitNOFILE=infinity LimitNPROC=infinity LimitCORE=infinity TasksMax=infinity containers Delegate=yes KillMode=process OOMScoreAdjust=-500 [Install] WantedBy=multi-user.target
7.启动dockerd
systemctl start docker
8.查看docker0网卡
[root@k8s-220 conf]# ifconfig docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450 #可以看到，docker0调用的就是flannel.1网卡所给的网段。 inet 10.18.18.1 netmask 255.255.255.0 broadcast 10.18.18.255 inet6 fe80::42:9fff:fed4:f022 prefixlen 64 scopeid 0x20<link> ether 02:42:9f:d4:f0:22 txqueuelen 0 (Ethernet) RX packets 4 bytes 224 (224.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 9 bytes 726 (726.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.1.220 netmask 255.255.255.0 broadcast 192.168.1.255 inet6 fe80::20c:29ff:fe79:8bef prefixlen 64 scopeid 0x20<link> ether 00:0c:29:79:8b:ef txqueuelen 1000 (Ethernet) RX packets 37871410 bytes 14333747643 (13.3 GiB) RX errors 0 dropped 148 overruns 0 frame 0 TX packets 35136117 bytes 16113012411 (15.0 GiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 flannel.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450 inet 10.18.18.0 netmask 255.255.255.255 broadcast 10.18.18.0 inet6 fe80::7ca9:5fff:fe76:be4c prefixlen 64 scopeid 0x20<link> ether 7e:a9:5f:76:be:4c txqueuelen 0 (Ethernet) RX packets 5 bytes 420 (420.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 8 bytes 672 (672.0 B) TX errors 0 dropped 5 overruns 0 carrier 0 collisions 0
9.查看etcd的网段分配记录:
[root@k8s-220 ~]# ETCDCTL_API=2 /cloud/etcd/bin/etcdctl --endpoints=https://10.9.0.33:2379 --ca-file=/cloud/etcd/cert/ca.pem --cert-file=/cloud/etcd/cert/etcd.pem --key-file=/cloud/etcd/cert/etcd-key.pem ls /coreos.com/network/ /coreos.com/network/config /coreos.com/network/subnets [root@k8s-220 ~]# ETCDCTL_API=2 /cloud/etcd/bin/etcdctl --endpoints=https://10.9.0.33:2379 --ca-file=/cloud/etcd/cert/ca.pem --cert-file=/cloud/etcd/cert/etcd.pem --key-file=/cloud/etcd/cert/etcd-key.pem ls /coreos.com/network/subnets /coreos.com/network/subnets/10.18.8.0-24 /coreos.com/network/subnets/10.18.13.0-24 /coreos.com/network/subnets/10.18.14.0-24 /coreos.com/network/subnets/10.18.18.0-24
避坑小贴士：通常情况下是已经启动了dockerd服务，已经分配了172.17.0.0网段的IP地址给了docker0，此时再想更换docker0的IP地址，会发现即使按照上面的方法设置重启了docker服务后，docker0网卡的IP依然后默认的172.17.0.0网段的。其中一个重要的原因是，docker服务中存在未删除的docker容器，包括没有启动的容器，必须全部删除已经存在的容器后再systemctl restart docker，才能修改docker0的默认地址。换句话说，docker --bip 只有在没有容器存在的情况下，才能起作用。当然也可以使用“重启系统”的终极大招。