calico
默认提供三种网络模式（这里只讲解基础部分，关于ebpf，custom cni 等暂时不考虑）
overlay 网络两种
vxlan
crossubnet 子网内不封装
always
ipip tunnel （可以支持BGP）
crossubnet 子网内不封装
always
underlay 网络
BGP：
1. full mesh
2. RR + IBGP 需要硬件支持
安装
curl https://projectcalico.docs.tigera.io/manifests/calico.yaml -O 更改podcidr kubectl apply -f calico.yaml curl -L https://github.com/projectcalico/calico/releases/download/v3.22.0/calicoctl-linux-amd64 -o calicoctl chmod +x ./calicoctl calicoctl node status
root@us-test00:~# ip r
default via 10.160.105.1 dev eth0 proto dhcp src 10.160.105.6 metric 100
10.160.105.0/24 dev eth0 proto kernel scope link src 10.160.105.6
168.63.129.16 via 10.160.105.1 dev eth0 proto dhcp src 10.160.105.6 metric 100
169.254.169.254 via 10.160.105.1 dev eth0 proto dhcp src 10.160.105.6 metric 100
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
blackhole 172.18.64.0/26 proto bird
172.18.96.128/26 via 10.160.105.8 dev tunl0 proto bird onlink
172.18.101.64/26 via 10.160.105.7 dev tunl0 proto bird onlink
root@us-test00:~# calicoctl node status
Calico process is running.
IPv4 BGP status
+--------------+-------------------+-------+----------+-------------+
| PEER ADDRESS | PEER TYPE | STATE | SINCE | INFO |
+--------------+-------------------+-------+----------+-------------+
| 10.160.105.7 | node-to-node mesh | up | 06:21:43 | Established |
| 10.160.105.8 | node-to-node mesh | up | 06:21:44 | Established |
+--------------+-------------------+-------+----------+-------------+
IPv6 BGP status
No IPv6 peers found.
root@us-test00:~# kubectl get ippool
NAME AGE
default-ipv4-ippool 53s
root@us-test00:~# kubectl get ippool default-ipv4-ippool -o yaml
apiVersion: crd.projectcalico.org/v1
kind: IPPool
metadata:
annotations:
projectcalico.org/metadata: '{"uid":"dac32e37-fcd6-4163-9b33-af6423aea269","creationTimestamp":"2022-03-07T06:21:38Z"}'
creationTimestamp: "2022-03-07T06:21:38Z"
generation: 1
name: default-ipv4-ippool
resourceVersion: "255454"
uid: e4bddd4c-a40a-4edf-9713-78e85ed27086
spec:
allowedUses:
- Workload
- Tunnel
blockSize: 26
cidr: 172.18.64.0/18
ipipMode: Always
natOutgoing: true
nodeSelector: all()
vxlanMode: Never
默认运行在 node-node-mesh 模式下的 full mesh bgp 模型
调整为overlay 网络模式（目前使用azure 仅支持vxlan网络访问）
选择最合适的网络模型（https://projectcalico.docs.tigera.io/networking/determine-best-networking）
例如在我的测试环境，是基于azure 来做 only can use vxlan
默认overlay 网络下pod 也就是private 网络想要访问public 网络就要通过snat 或者 masqurade
Chain cali-nat-outgoing (1 references)
pkts bytes target prot opt in out source destination
121 7548 MASQUERADE all -- * * 0.0.0.0/0 0.0.0.0/0 / * cali:Dw4T8UWPnCLxRJiI */ match-set cali40masq-ipam-pools src ! match-set cali40all-ipam-pools dst
cilium
self-manager 的k8s 集群整体安装cilium 的方式类似：
通过 cilium CLI 安装（本次采用）
通过 yml 文件快速创建
通过 helm 进行安装（values 参数列表https://docs.cilium.io/en/v1.10/helm-reference/）
cilium install --config ipam=kubernetes # 只需要注意更改ipam 类型为kubernetes 即可，因为我们之前kubeadm 初始化时制定了 pod-netcidr 。默认为的cluster-pool 模式
查看目前的cilium 运行在vxlan 模式下
root@us-test00:~# ip r
default via 10.160.105.1 dev eth0 proto dhcp src 10.160.105.6 metric 100
10.160.105.0/24 dev eth0 proto kernel scope link src 10.160.105.6
168.63.129.16 via 10.160.105.1 dev eth0 proto dhcp src 10.160.105.6 metric 100
169.254.169.254 via 10.160.105.1 dev eth0 proto dhcp src 10.160.105.6 metric 100
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
172.18.64.0/24 via 172.18.64.165 dev cilium_host src 172.18.64.165
172.18.64.165 dev cilium_host scope link
172.18.65.0/24 via 172.18.64.165 dev cilium_host src 172.18.64.165 mtu 1450
172.18.66.0/24 via 172.18.64.165 dev cilium_host src 172.18.64.165 mtu 1450
#安装hubble 和 ui
cilium hubble enable
cilium hubble enable --ui
配置对应的ingress 访问ui