calico
默認提供三種網絡模式（這里只講解基礎部分，關於ebpf，custom cni 等暫時不考慮）
overlay 網絡兩種
vxlan
crossubnet 子網內不封裝
always
ipip tunnel （可以支持BGP）
crossubnet 子網內不封裝
always
underlay 網絡
BGP：
1. full mesh
2. RR + IBGP 需要硬件支持
安裝
curl https://projectcalico.docs.tigera.io/manifests/calico.yaml -O 更改podcidr kubectl apply -f calico.yaml curl -L https://github.com/projectcalico/calico/releases/download/v3.22.0/calicoctl-linux-amd64 -o calicoctl chmod +x ./calicoctl calicoctl node status
root@us-test00:~# ip r
default via 10.160.105.1 dev eth0 proto dhcp src 10.160.105.6 metric 100
10.160.105.0/24 dev eth0 proto kernel scope link src 10.160.105.6
168.63.129.16 via 10.160.105.1 dev eth0 proto dhcp src 10.160.105.6 metric 100
169.254.169.254 via 10.160.105.1 dev eth0 proto dhcp src 10.160.105.6 metric 100
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
blackhole 172.18.64.0/26 proto bird
172.18.96.128/26 via 10.160.105.8 dev tunl0 proto bird onlink
172.18.101.64/26 via 10.160.105.7 dev tunl0 proto bird onlink
root@us-test00:~# calicoctl node status
Calico process is running.
IPv4 BGP status
+--------------+-------------------+-------+----------+-------------+
| PEER ADDRESS | PEER TYPE | STATE | SINCE | INFO |
+--------------+-------------------+-------+----------+-------------+
| 10.160.105.7 | node-to-node mesh | up | 06:21:43 | Established |
| 10.160.105.8 | node-to-node mesh | up | 06:21:44 | Established |
+--------------+-------------------+-------+----------+-------------+
IPv6 BGP status
No IPv6 peers found.
root@us-test00:~# kubectl get ippool
NAME AGE
default-ipv4-ippool 53s
root@us-test00:~# kubectl get ippool default-ipv4-ippool -o yaml
apiVersion: crd.projectcalico.org/v1
kind: IPPool
metadata:
annotations:
projectcalico.org/metadata: '{"uid":"dac32e37-fcd6-4163-9b33-af6423aea269","creationTimestamp":"2022-03-07T06:21:38Z"}'
creationTimestamp: "2022-03-07T06:21:38Z"
generation: 1
name: default-ipv4-ippool
resourceVersion: "255454"
uid: e4bddd4c-a40a-4edf-9713-78e85ed27086
spec:
allowedUses:
- Workload
- Tunnel
blockSize: 26
cidr: 172.18.64.0/18
ipipMode: Always
natOutgoing: true
nodeSelector: all()
vxlanMode: Never
默認運行在 node-node-mesh 模式下的 full mesh bgp 模型
調整為overlay 網絡模式（目前使用azure 僅支持vxlan網絡訪問）
選擇最合適的網絡模型（https://projectcalico.docs.tigera.io/networking/determine-best-networking）
例如在我的測試環境，是基於azure 來做 only can use vxlan
默認overlay 網絡下pod 也就是private 網絡想要訪問public 網絡就要通過snat 或者 masqurade
Chain cali-nat-outgoing (1 references)
pkts bytes target prot opt in out source destination
121 7548 MASQUERADE all -- * * 0.0.0.0/0 0.0.0.0/0 / * cali:Dw4T8UWPnCLxRJiI */ match-set cali40masq-ipam-pools src ! match-set cali40all-ipam-pools dst
cilium
self-manager 的k8s 集群整體安裝cilium 的方式類似：
通過 cilium CLI 安裝（本次采用）
通過 yml 文件快速創建
通過 helm 進行安裝（values 參數列表https://docs.cilium.io/en/v1.10/helm-reference/）
cilium install --config ipam=kubernetes # 只需要注意更改ipam 類型為kubernetes 即可，因為我們之前kubeadm 初始化時制定了 pod-netcidr 。默認為的cluster-pool 模式
查看目前的cilium 運行在vxlan 模式下
root@us-test00:~# ip r
default via 10.160.105.1 dev eth0 proto dhcp src 10.160.105.6 metric 100
10.160.105.0/24 dev eth0 proto kernel scope link src 10.160.105.6
168.63.129.16 via 10.160.105.1 dev eth0 proto dhcp src 10.160.105.6 metric 100
169.254.169.254 via 10.160.105.1 dev eth0 proto dhcp src 10.160.105.6 metric 100
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
172.18.64.0/24 via 172.18.64.165 dev cilium_host src 172.18.64.165
172.18.64.165 dev cilium_host scope link
172.18.65.0/24 via 172.18.64.165 dev cilium_host src 172.18.64.165 mtu 1450
172.18.66.0/24 via 172.18.64.165 dev cilium_host src 172.18.64.165 mtu 1450
#安裝hubble 和 ui
cilium hubble enable
cilium hubble enable --ui
配置對應的ingress 訪問ui