原文 https://www.myf5.net/post/2330.htm
k8s Nodeport方式下service訪問，iptables處理邏輯
2017年07月11日 0條評論 976次閱讀 0人點贊
借用網上一張圖，畫的挺好~ 來自http://blog.csdn.net/xinghun_4/article/details/50492041
對應解釋如下
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
|
[root@docker3 ~]# iptables -nL -t nat -v
Chain PREROUTING (policy ACCEPT 2 packets, 1152 bytes)
pkts bytes target prot opt in out source destination
10837 3999K KUBE-SERVICES all -- * * 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
====》1. 進入的請求，首先匹配PREROUTING 鏈里的該規則，並進入KUBE-SERVICES鏈
238 27286 DOCKER all -- * * 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
Chain INPUT (policy ACCEPT 2 packets, 1152 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 8 packets, 480 bytes)
pkts bytes target prot opt in out source destination
61563 3722K KUBE-SERVICES all -- * * 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
0 0 DOCKER all -- * * 0.0.0.0/0 !127.0.0.0/8 ADDRTYPE match dst-type LOCAL
Chain POSTROUTING (policy ACCEPT 8 packets, 480 bytes)
pkts bytes target prot opt in out source destination
61840 3742K KUBE-POSTROUTING all -- * * 0.0.0.0/0 0.0.0.0/0 /* kubernetes postrouting rules */
====》6. 需要路由到另一個主機的連接經由POSTROUTING該規則，進入KUBE-POSTROUTING 鏈
62 4323 MASQUERADE all -- * !docker0 10.2.39.0/24 0.0.0.0/0
0 0 MASQUERADE all -- * !docker0 10.1.33.0/24 0.0.0.0/0
Chain DOCKER (2 references)
pkts bytes target prot opt in out source destination
0 0 RETURN all -- docker0 * 0.0.0.0/0 0.0.0.0/0
Chain KUBE-MARK-DROP (0 references)
pkts bytes target prot opt in out source destination
0 0 MARK all -- * * 0.0.0.0/0 0.0.0.0/0 MARK or 0x8000
Chain KUBE-MARK-MASQ (6 references)
pkts bytes target prot opt in out source destination
2 128 MARK all -- * * 0.0.0.0/0 0.0.0.0/0 MARK or 0x4000
Chain KUBE-NODEPORTS (1 references)
pkts bytes target prot opt in out source destination
2 128 KUBE-MARK-MASQ tcp -- * * 0.0.0.0/0 0.0.0.0/0 /* default/k8s-nginx: */ tcp dpt:30780
2 128 KUBE-SVC-2RMP45C4XWDG5BGC tcp -- * * 0.0.0.0/0 0.0.0.0/0 /* default/k8s-nginx: */ tcp dpt:30780
====》3.請求命中上述兩個規則，一個進行進入KUBE-MARK-MASQ 鏈進行標記，一個進入KUBE-SVC-2RMP45C4XWDG5BGC鏈
Chain KUBE-POSTROUTING (1 references)
pkts bytes target prot opt in out source destination
1 64 MASQUERADE all -- * * 0.0.0.0/0 0.0.0.0/0 /* kubernetes service traffic requiring SNAT */ mark match 0x4000/0x4000
====》7. 執行源地址轉換(在flannel網絡這里轉換的地址是flannel.1即flannel在ifconfig里輸出接口的地址），發往另一個node
Chain KUBE-SEP-D5T62RWZFFOCR77Q (1 references)
pkts bytes target prot opt in out source destination
0 0 KUBE-MARK-MASQ all -- * * 10.2.39.3 0.0.0.0/0 /* default/k8s-nginx: */
1 64 DNAT tcp -- * * 0.0.0.0/0 0.0.0.0/0 /* default/k8s-nginx: */ tcp to:10.2.39.3:80
====》5-2. DNAT到本機，交給INPUT
Chain KUBE-SEP-IK3IYR4STYKRJP77 (1 references)
pkts bytes target prot opt in out source destination
0 0 KUBE-MARK-MASQ all -- * * 10.2.39.2 0.0.0.0/0 /* kube-system/kube-dns:dns-tcp */
0 0 DNAT tcp -- * * 0.0.0.0/0 0.0.0.0/0 /* kube-system/kube-dns:dns-tcp */ tcp to:10.2.39.2:53
Chain KUBE-SEP-WV6S37CDULKCYEVE (1 references)
pkts bytes target prot opt in out source destination
0 0 KUBE-MARK-MASQ all -- * * 10.2.39.2 0.0.0.0/0 /* kube-system/kube-dns:dns */
0 0 DNAT udp -- * * 0.0.0.0/0 0.0.0.0/0 /* kube-system/kube-dns:dns */ udp to:10.2.39.2:53
Chain KUBE-SEP-X7YOSBI66WAQ7F6X (2 references)
pkts bytes target prot opt in out source destination
0 0 KUBE-MARK-MASQ all -- * * 172.16.199.17 0.0.0.0/0 /* default/kubernetes:https */
0 0 DNAT tcp -- * * 0.0.0.0/0 0.0.0.0/0 /* default/kubernetes:https */ recent: SET name: KUBE-SEP-X7YOSBI66WAQ7F6X side: source mask: 255.255.255.255 tcp to:172.16.199.17:6443
Chain KUBE-SEP-YXWG4KEJCDIRMCO5 (1 references)
pkts bytes target prot opt in out source destination
0 0 KUBE-MARK-MASQ all -- * * 10.2.4.2 0.0.0.0/0 /* default/k8s-nginx: */
1 64 DNAT tcp -- * * 0.0.0.0/0 0.0.0.0/0 /* default/k8s-nginx: */ tcp to:10.2.4.2:80
====》5-1. DNAT到另一個node，隨后執行postrouting
Chain KUBE-SERVICES (2 references)
pkts bytes target prot opt in out source destination
0 0 KUBE-SVC-2RMP45C4XWDG5BGC tcp -- * * 0.0.0.0/0 169.169.148.143 /* default/k8s-nginx: cluster IP */ tcp dpt:80
0 0 KUBE-SVC-NPX46M4PTMTKRN6Y tcp -- * * 0.0.0.0/0 169.169.0.1 /* default/kubernetes:https cluster IP */ tcp dpt:443
0 0 KUBE-SVC-TCOU7JCQXEZGVUNU udp -- * * 0.0.0.0/0 169.169.0.53 /* kube-system/kube-dns:dns cluster IP */ udp dpt:53
0 0 KUBE-SVC-ERIFXISQEP7F7OF4 tcp -- * * 0.0.0.0/0 169.169.0.53 /* kube-system/kube-dns:dns-tcp cluster IP */ tcp dpt:53
2 128 KUBE-NODEPORTS all -- * * 0.0.0.0/0 0.0.0.0/0 /* kubernetes service nodeports; NOTE: this must be the last rule in this chain */ ADDRTYPE match dst-type LOCAL
====》2.連接隨后並該規則匹配，進入KUBE-NODEPORTS 鏈
Chain KUBE-SVC-2RMP45C4XWDG5BGC (2 references)
pkts bytes target prot opt in out source destination
1 64 KUBE-SEP-D5T62RWZFFOCR77Q all -- * * 0.0.0.0/0 0.0.0.0/0 /* default/k8s-nginx: */ statistic mode random probability 0.50000000000
1 64 KUBE-SEP-YXWG4KEJCDIRMCO5 all -- * * 0.0.0.0/0 0.0.0.0/0 /* default/k8s-nginx: */
====》4.請求被該鏈中兩個規則匹配，這里只有兩個規則，按照50% RR規則進行負載均衡分發，兩個目標鏈都是進行DNAT，一個轉到本node的pod IP上，一個轉到另一台宿主機的pod上，因為該service下只有兩個pod
==========》亂入的廣告,www.myf5.net《==================
Chain KUBE-SVC-ERIFXISQEP7F7OF4 (1 references)
pkts bytes target prot opt in out source destination
0 0 KUBE-SEP-IK3IYR4STYKRJP77 all -- * * 0.0.0.0/0 0.0.0.0/0 /* kube-system/kube-dns:dns-tcp */
Chain KUBE-SVC-NPX46M4PTMTKRN6Y (1 references)
pkts bytes target prot opt in out source destination
0 0 KUBE-SEP-X7YOSBI66WAQ7F6X all -- * * 0.0.0.0/0 0.0.0.0/0 /* default/kubernetes:https */ recent: CHECK seconds: 10800 reap name: KUBE-SEP-X7YOSBI66WAQ7F6X side: source mask: 255.255.255.255
0 0 KUBE-SEP-X7YOSBI66WAQ7F6X all -- * * 0.0.0.0/0 0.0.0.0/0 /* default/kubernetes:https */
Chain KUBE-SVC-TCOU7JCQXEZGVUNU (1 references)
pkts bytes target prot opt in out source destination
0 0 KUBE-SEP-WV6S37CDULKCYEVE all -- * * 0.0.0.0/0 0.0.0.0/0 /* kube-system/kube-dns:dns */
|
本地下載 在被轉發目標node的物理網口抓包（需decode as vxlan)