k8s爬坑筆記-Flannel篇

一.kubernetes網絡模型

kubernetes網絡模型（CNI）
Container Network Interface：容器網絡接口
k8s網絡模型設計要求：
1.一個Pod一個IP
2.每個Pod獨立IP，Pod內所有容器共享網絡（同一個IP）
3.所有容器都可以與其他容器通信
4.所有節點都可以與所有容器通信

flannel 小規模集群（100台以下）
calic 大規模集群
Contiv 集合flannel和calic的優點

二.安裝配置flannel

2.1.在所有的node機器上安裝docker

    yum install -y yum-utils   device-mapper-persistent-data   lvm2
    yum-config-manager     --add-repo     https://download.docker.com/linux/centos/docker-ce.repo
    yum install docker-ce -y
#如果出現拉取鏡像較慢的情況，可以配置docker鏡像站來進行加速
    curl -sSL https://get.daocloud.io/daotools/set_mirror.sh | sh -s http://f1361db2.m.daocloud.io

    systemctl enable docker
    systemctl start docker
    
    vim /etc/docker/daemon.json
    {
      "log-driver": "json-file",
      "log-opts": {
        "max-size": "5g",
        "max-file": "3"
      }
    }
    systemctl restart docker

2.2.配置安裝flannel

使用flannel作為k8s網絡模型，使用VXLAN作為數據轉發方式

2.2.1.寫入分配的子網到etcd中，供flannel使用

/opt/etcd/bin/etcdctl --ca-file=/opt/etcd/ssl/ca.pem --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem --endpoints="https://192.168.244.226:2379,https://192.168.244.227:2379,https://192.168.244.228:2379" set /coreos.com/network/config '{ "Network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}'

查看配置

/opt/etcd/bin/etcdctl --ca-file=/opt/etcd/ssl/ca.pem --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem --endpoints="https://192.168.244.226:2379,https://192.168.244.227:2379,https://192.168.244.228:2379" get /coreos.com/network/config

2.2.3.下載安裝flannel

wget https://github.com/coreos/flannel/releases/download/v0.11.0/flannel-v0.11.0-linux-amd64.tar.gz

tar -zxvf flannel-v0.11.0-linux-amd64.tar.gz

mkdir /opt/kubernetes/{bin,cfg,ssl,logs} -p
mv flanneld mk-docker-opts.sh /opt/kubernetes/bin/

執行安裝腳本

            #!/bin/bash

            ETCD_ENDPOINTS=${1:-"http://127.0.0.1:2379"}

            cat <<EOF >/opt/kubernetes/cfg/flanneld

            FLANNEL_OPTIONS="--etcd-endpoints=${ETCD_ENDPOINTS} \
            -etcd-cafile=/opt/etcd/ssl/ca.pem \
            -etcd-certfile=/opt/etcd/ssl/server.pem \
            -etcd-keyfile=/opt/etcd/ssl/server-key.pem"

            EOF

            cat <<EOF >/usr/lib/systemd/system/flanneld.service
            [Unit]
            Description=Flanneld overlay address etcd agent
            After=network-online.target network.target
            Before=docker.service

            [Service]
            Type=notify
            EnvironmentFile=/opt/kubernetes/cfg/flanneld
            ExecStart=/opt/kubernetes/bin/flanneld --ip-masq \$FLANNEL_OPTIONS
            ExecStartPost=/opt/kubernetes/bin/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/subnet.env
            Restart=on-failure

            [Install]
            WantedBy=multi-user.target

            EOF

            cat <<EOF >/usr/lib/systemd/system/docker.service

            [Unit]
            Description=Docker Application Container Engine
            Documentation=https://docs.docker.com
            After=network-online.target firewalld.service
            Wants=network-online.target

            [Service]
            Type=notify
            # 讀取flannel分配的子網網段
            EnvironmentFile=/run/flannel/subnet.env
            # 修改docker啟動配置，設置docker啟動的子網
            ExecStart=/usr/bin/docker \$DOCKER_NETWORK_OPTIONS
            ExecReload=/bin/kill -s HUP \$MAINPID
            LimitNOFILE=infinity
            LimitNPROC=infinity
            LimitCORE=infinity
            TimeoutStartSec=0
            Delegate=yes
            KillMode=process
            Restart=on-failure
            StartLimitBurst=3
            StartLimitInterval=60s

            [Install]
            WantedBy=multi-user.target

            EOF

            systemctl daemon-reload
            systemctl enable flanneld
            systemctl restart flanneld
            systemctl restart docker

sh -x flannel.sh https://192.168.244.226:2379,https://192.168.244.227:2379,https://192.168.244.228:2379

2.2.4.查看ETCD中flannel注冊的網段信息

/opt/etcd/bin/etcdctl -ca-file=/opt/etcd/ssl/ca.pem --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem --endpoints="https://192.168.244.226:2379,https://192.168.244.227:2379,https://192.168.244.228:2379" ls /coreos.com/network/subnets/

查看docker和flannel的網段信息，docker會使用flannel中的一個子網網段IP

2.2.5.測試兩台node之間容器的通信

docker run --rm -it busybox