OpenvSwitch端口鏡像

OVS上實現端口鏡像的基本流程如下:

• 創建 mirror ，在 mirror 中指定鏡像數據源及鏡像目的地
• 將創建的 mirror 應用到 bridge 中

鏡像數據源可以通過下面幾個選項來指定:

• select_all : 布爾值，設置為 true 時，進出該 mirror 所生效的 bridge 上的每個數據包都將被鏡像
• select_dst_port : 從該 port 離開虛擬交換機的數據包將會被鏡像，從Guest角度看是Guest網絡接口的流入方向
• select_src_port : 從該 port 進入虛擬交換機的數據包將會被鏡像，從Guest角度看是Guest網絡接口的流出方向
• select_vlan : 指定特定VLAN做為數據源，整個VLAN的數據包都會鏡像到目的地

鏡像目的地可以用下面選項來指定:

• output_port : 將數據包鏡像到特定的 port
• output_vlan : 將數據包鏡像到指定VLAN, 原始數據的VLAN tag會被剝掉。若鏡像多個VLAN到同一個VLAN，沒有辦法區分鏡像后的數據包來源於哪個VLAN。

下面我們通過實例來說明OVS上的鏡像機制。我們的第一個實驗拓朴結構如下圖，我們將流入 tap1 網絡接口的數據包鏡像到 tap3 中:

首先構造環境:

ovs-vsctl add-br br0
ovs-vsctl add-port br0 tap1 -- set interface tap1 type=internal
ovs-vsctl add-port br0 tap2 -- set interface tap2 type=internal
ovs-vsctl add-port br0 tap3 -- set interface tap3 type=internal
ip netns add ns1
ip netns add ns2
ip netns add ns3
ip link set dev tap1 netns ns1
ip link set dev tap2 netns ns2
ip link set dev tap3 netns ns3
ip netns exec ns1 ip addr add 10.10.10.11/24 dev tap1
ip netns exec ns1 ip link set up tap1
ip netns exec ns2 ip addr add 10.10.10.12/24 dev tap2
ip netns exec ns2 ip link set up tap2
ip netns exec ns3 ip link set up tap3

　　

我們從 ns1 中PING ns2 的IP:

[root@centos3 vagrant]# ip netns exec ns1 ping 10.10.10.12 -c 2
PING 10.10.10.12 (10.10.10.12) 56(84) bytes of data.
64 bytes from 10.10.10.12: icmp_seq=1 ttl=64 time=0.060 ms
64 bytes from 10.10.10.12: icmp_seq=2 ttl=64 time=0.106 ms

--- 10.10.10.12 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 0.060/0.083/0.106/0.023 ms

　　

在 ns3 中運行 tcpdump 觀察是否能收到數據包, 可以看到此時 ns3 的 tap3 並不會收到 tap1 訪問 tap2 的數據包:

[root@centos3 vagrant]# ip netns exec ns3 tcpdump -i tap3 -e -nn icmp or arp
tcpdump: WARNING: tap3: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tap3, link-type EN10MB (Ethernet), capture size 65535 bytes
^C
0 packets captured
0 packets received by filter
0 packets dropped by kernel

　　

接下來我們創建了相應的 mirror , 並將其應用到 br0 上:

ovs-vsctl -- --id=@tap1 get port tap1  \
          -- --id=@tap3 get port tap3  \
          -- --id=@m create mirror name=m0 select_dst_port=@tap1 output_port=@tap3 \
          -- set bridge br0 mirrors=@m

　　

此時查看 OVS 上的 mirror :

[root@centos3 vagrant]# ovs-vsctl list mirror
_uuid               : 98b89127-cf94-4926-8d5f-76145154b03c
external_ids        : {}
name                : "m0"
output_port         : 1dcde312-e33d-439f-b646-9db92416a586
output_vlan         : []
select_all          : false
select_dst_port     : [16db4055-5b5b-411d-8e98-87813ba14eff]
select_src_port     : []
select_vlan         : []
statistics          : {tx_bytes=0, tx_packets=0}

　　

再次進行PING實驗:

[root@centos3 vagrant]# ip netns exec ns1 ping 10.10.10.12 -c 2
PING 10.10.10.12 (10.10.10.12) 56(84) bytes of data.
64 bytes from 10.10.10.12: icmp_seq=1 ttl=64 time=0.274 ms
64 bytes from 10.10.10.12: icmp_seq=2 ttl=64 time=0.341 ms

--- 10.10.10.12 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 0.274/0.307/0.341/0.037 ms

　　

在 ns3 上抓包可以看到成功獲得 tap2 回應 tap1 的ICMP響應數據包:

[root@centos3 vagrant]# ip netns exec ns3 tcpdump -i tap3 -e -nn icmp or arp
tcpdump: WARNING: tap3: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tap3, link-type EN10MB (Ethernet), capture size 65535 bytes
23:22:11.919448 26:1e:74:67:6c:cc > 16:fe:12:ad:f0:4f, ethertype IPv4 (0x0800), length 98: 10.10.10.12 > 10.10.10.11: ICMP echo reply, id 4411, seq 1, length 64
23:22:12.919823 26:1e:74:67:6c:cc > 16:fe:12:ad:f0:4f, ethertype IPv4 (0x0800), length 98: 10.10.10.12 > 10.10.10.11: ICMP echo reply, id 4411, seq 2, length 64
23:22:16.929503 26:1e:74:67:6c:cc > 16:fe:12:ad:f0:4f, ethertype ARP (0x0806), length 42: Request who-has 10.10.10.11 tell 10.10.10.12, length 28
^C
3 packets captured
3 packets received by filter
0 packets dropped by kernel

　　

這種方式對應了 Cisco 的 SPAN 方式, 下面我們來實驗 RSPAN 方式。我們將拓朴修改為如下圖所示，我們首先在 br0 上將 tap1 的數據包鏡像到物定VLAN: 111 , 在 br1 上再從VLAN: 111 中將數據包鏡像到 tap4 :

准備拓朴:

ip link add p0 type veth peer name p1
ovs-vsctl add-port br0 p0
ovs-vsctl add-port br1 p1
ovs-vsctl add-port br1 tap4 -- set interface tap4 type=internal
ip netns add ns4
ip link set tap4 netns ns4
ip netns exec ns4 ip link set up tap4

　　

關閉VLAN: 111 的MAC學習功能，避免影響正常網絡轉發:

ovs-vsctl set bridge br0 flood_vlans=111
ovs-vsctl set bridge br1 flood_vlans=111

　　

首先創建一個 mirror 將 tap1 的數據包鏡像到VLAN: 111 :

ovs-vsctl -- --id=@tap1 get port tap1  \
          -- --id=@m create mirror name=m1 select_src_port=@tap1 output_vlan=111 \
          -- set bridge br0 mirrors=@m

　　

查看 OVS 上的 mirror :

[root@centos3 vagrant]# ovs-vsctl list mirror
_uuid               : fd39fdb2-ab69-47c1-bde9-01b7b40dd4d3
external_ids        : {}
name                : "m1"
output_port         : []
output_vlan         : 111
select_all          : false
select_dst_port     : []
select_src_port     : [16db4055-5b5b-411d-8e98-87813ba14eff]
select_vlan         : []
statistics          : {tx_bytes=13426, tx_packets=157}

　　

再次從 tap1 發送PING包到 tap2 , 我們在 tap4 上抓包, 可以看到 tap4 上可以收到鏡像的數據包:

[root@centos3 vagrant]# ip netns exec ns4 tcpdump -i tap4 -e -nn icmp or arp
tcpdump: WARNING: tap4: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tap4, link-type EN10MB (Ethernet), capture size 65535 bytes
14:45:58.762260 16:fe:12:ad:f0:4f > 26:1e:74:67:6c:cc, ethertype 802.1Q (0x8100), length 102: vlan 111, p 0, ethertype IPv4, 10.10.10.11 > 10.10.10.12: ICMP echo request, id 9722, seq 1, length 64
14:45:59.806203 16:fe:12:ad:f0:4f > 26:1e:74:67:6c:cc, ethertype 802.1Q (0x8100), length 102: vlan 111, p 0, ethertype IPv4, 10.10.10.11 > 10.10.10.12: ICMP echo request, id 9722, seq 2, length 64
^C
2 packets captured
2 packets received by filter
0 packets dropped by kernel

　　

此時， tap4 為 trunk 模式，所有VLAN的數據包都可以收到。我們將 tap4 的 tag 設置為 111 :

ovs-vsctl set port tap4 tap=111

此時，再次重復PING訪問， tap4 上依然可以收到鏡像的數據包, 只不過VLAN TAG已經被剝除:

[root@centos3 vagrant]# ip netns exec ns4 tcpdump -i tap4 -e -nn icmp or arp
tcpdump: WARNING: tap4: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tap4, link-type EN10MB (Ethernet), capture size 65535 bytes
14:47:24.060144 16:fe:12:ad:f0:4f > 26:1e:74:67:6c:cc, ethertype IPv4 (0x0800), length 98: 10.10.10.11 > 10.10.10.12: ICMP echo request, id 9816, seq 1, length 64
14:47:25.117508 16:fe:12:ad:f0:4f > 26:1e:74:67:6c:cc, ethertype IPv4 (0x0800), length 98: 10.10.10.11 > 10.10.10.12: ICMP echo request, id 9816, seq 2, length 64
^C
2 packets captured
2 packets received by filter
0 packets dropped by kernel

　　

我們再次將 tag 設置為 110 ，再次從 tap4 抓包則收不到數據包。

我們可以添加另一個鏡像規則，將鏡像VLAN的數據包鏡像到 tap4 :

ovs-vsctl -- --id=@tap4 get port tap4  \
          -- --id=@m create mirror name=m2 select_vlan=111 select_all=true output_port=@tap4 \
          -- add bridge br1 mirrors @m

　　

查看OVS上的 mirror :

[root@centos3 vagrant]# ovs-vsctl list mirror
_uuid               : e758d8ae-408d-4e4f-a1c4-eaa8dd99bce5
external_ids        : {}
name                : "m2"
output_port         : e6a7d9b6-1032-4f9a-84f1-54b69f6db315
output_vlan         : []
select_all          : true
select_dst_port     : []
select_src_port     : []
select_vlan         : [111]
statistics          : {tx_bytes=0, tx_packets=0}

_uuid               : fd39fdb2-ab69-47c1-bde9-01b7b40dd4d3
external_ids        : {}
name                : "m1"
output_port         : []
output_vlan         : 111
select_all          : false
select_dst_port     : []
select_src_port     : [16db4055-5b5b-411d-8e98-87813ba14eff]
select_vlan         : []
statistics          : {tx_bytes=14378, tx_packets=169}

　　

在 tap4 上抓包, 得到的數據包已經被剝掉VLAN TAG:

[root@centos3 vagrant]# ip netns exec ns4 tcpdump -i tap4 -e -nn icmp or arp
tcpdump: WARNING: tap4: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tap4, link-type EN10MB (Ethernet), capture size 65535 bytes
14:50:37.842958 16:fe:12:ad:f0:4f > 26:1e:74:67:6c:cc, ethertype IPv4 (0x0800), length 98: 10.10.10.11 > 10.10.10.12: ICMP echo request, id 10044, seq 1, length 64
14:50:38.904391 16:fe:12:ad:f0:4f > 26:1e:74:67:6c:cc, ethertype IPv4 (0x0800), length 98: 10.10.10.11 > 10.10.10.12: ICMP echo request, id 10044, seq 2, length 64
^C
2 packets captured
2 packets received by filter
0 packets dropped by kernel

　　

使用鏡像到物定VLAN的方式會導致原始的VLAN TAG信息丟失，如果要鏡像多個VLAN的數據到同一目的地則會造成混亂。這種場景下，最好將數據包鏡像到一個GRE port，基於這種方式可以實現 ERSPAN 模式。

清除 mirror 設置:

ovs-vsctl clear bridge br0 mirrors
ovs-vsctl clear bridge br1 mirrors

　　

添加一個GRE端口:

ovs-vsctl add-port br0 gre0 -- set interface gre0 type=gre options:key=0x1000 options:remote_ip=10.95.30.43

　　

創建 mirror 將 tap1 的數據包鏡像至GRE端口:

ovs-vsctl -- --id=@tap1 get port tap1  \
          -- --id=@gre0 get port gre0  \
          -- --id=@m create mirror name=m3 select_src_port=@tap1 output_port=@gre0 \
          -- set bridge br0 mirrors=@m

　　

查看 mirror :

[root@centos3 vagrant]# ovs-vsctl list mirror
_uuid               : 3c63e590-d42c-454a-a7f9-baa3e0ebb77f
external_ids        : {}
name                : "m3"
output_port         : e71af801-289f-466b-9f66-d1e5cf396233
output_vlan         : []
select_all          : false
select_dst_port     : []
select_src_port     : [16db4055-5b5b-411d-8e98-87813ba14eff]
select_vlan         : []
statistics          : {tx_bytes=0, tx_packets=0}

　　

我們在外網出口 eth0 上抓包，可以看到，GRE數據包已經發送:

[root@centos3 vagrant]# tcpdump -ieth0 -nn -e proto gre
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
15:46:50.236791 08:00:27:6b:57:88 > 52:54:00:12:35:02, ethertype IPv4 (0x0800), length 140: 10.0.2.15 > 10.95.30.43: GREv0, key=0x1111, proto TEB (0x6558), length 106: 86:5e:7a:ba:4a:5f > e2:ee:06:10:cb:70, ethertype IPv4 (0x0800), length 98: 10.10.10.11 > 10.10.10.12: ICMP echo request, id 1941, seq 1, length 64
15:46:51.237935 08:00:27:6b:57:88 > 52:54:00:12:35:02, ethertype IPv4 (0x0800), length 140: 10.0.2.15 > 10.95.30.43: GREv0, key=0x1111, proto TEB (0x6558), length 106: 86:5e:7a:ba:4a:5f > e2:ee:06:10:cb:70, ethertype IPv4 (0x0800), length 98: 10.10.10.11 > 10.10.10.12: ICMP echo request, id 1941, seq 2, length 64
^C
2 packets captured
2 packets received by filter
0 packets dropped by kernel