两端的br40不能设置ip，否则影响vrf中的路由
学习了用frrouting 实现l3 gateway可以实现不同子网的通信。现在动手动配置neigh表、vrf策略路由表、fdb表来实现实现不同子网间通信
10.10.18.214节点
1、在在vetp 10.10.18.214 给对端vetp 10.10.18.213设置neigh，因为4.4.4.254网关会发arp请求，请求
10.10.18.213的mac,而且添加的mac 00:00:01:02:03:08 是vetp 10.10.18.213上br40的mac
ip n add 10.10.18.213 dev br40 lladdr 00:00:01:02:03:08 nud perm
[root@evpn2 ~]# ip n show vrf vrf-green
4.4.4.3 dev br30 lladdr 96:43:7c:c4:79:8e STALE
10.10.18.213 dev br40 lladdr 00:00:01:02:03:08 PERMANENT
fe80::9443:7cff:fec4:798e dev br30 lladdr 96:43:7c:c4:79:8e STALE
2、ip netns exec host3 ping 5.5.5.213 时发送arp请求，同样在netns host3添加neigh项
[root@evpn2 ~]# ip netns exec host3 ip n
4.4.4.254 dev eth0 lladdr 26:01:66:a5:9c:cc STALE
4.4.4.1 dev eth0 FAILED
5.5.5.213 dev eth0 lladdr 32:e3:6a:fe:8b:55 PERMANENT
这里5.5.5.213的mac可以不是5.5.5.213的真实mac，用vetp 10.10.18.214上br40的mac也行
3、设置fdb bridge fdb append to 00:00:01:02:03:08 dev vxlan40 dst 10.10.18.213
[root@evpn2 ~]# bridge fdb show | grep 00:00:01:02:03:08 00:00:01:02:03:08 dev dev vxlan40 dst 10.10.18.213 self permanent 4、添加路由 5.5.5.213 via 10.10.18.213 dev br40 proto bgp metric 20 onlink
[root@evpn2 ~]# ip route show vrf vrf-green
4.4.4.0/24 dev br30 proto kernel scope link src 4.4.4.254
5.5.5.213 via 10.10.18.213 dev br40 proto bgp metric 20 onlink
10.10.18.213节点上
设置4.4.4.3 的neigh项 [root@evpn213 ~]# ip netns exec host3 ip n 4.4.4.3 dev eth0 lladdr 00:00:01:02:03:08 PERMANENT 5.5.5.1 dev eth0 lladdr a2:fd:cb:4f:71:41 STALE 设置10.10.18.214 的neigh项 [root@evpn213 ~]# ip n show vrf vrf-green 5.5.5.213 dev br30 lladdr 32:e3:6a:fe:8b:55 PERMANENT 10.10.18.214 dev br40 lladdr 00:00:01:02:03:09 PERMANENT 添加路由 [root@evpn213 ~]# ip r show vrf vrf-green 4.4.4.3 via 10.10.18.214 dev br40 onlink 5.5.5.0/24 dev br30 proto kernel scope link src 5.5.5.1 添加fdb项 [root@evpn213 ~]# bridge fdb show | grep vxlan40 02:63:4b:80:0d:25 dev dev vxlan40 vlan 1 master br40 permanent 02:63:4b:80:0d:25 dev dev vxlan40 master br40 permanent 00:00:01:02:03:09 dev dev vxlan40 dst 10.10.18.214 self permanent
一开始把fdb项配到了master br40上导致，报文总是发不出去
[root@evpn213 ~]# bridge fdb show | grep 00:00:01:02:03:09 00:00:01:02:03:09 dev dev br40 vlan 1 master br40 permanent 00:00:01:02:03:09 dev dev br40 master br40 permanent 00:00:01:02:03:09 dev dev vxlan40 dst 10.10.18.214 self permanent [root@evpn213 ~]# bridge fdb del 00:00:01:02:03:09 dev br40
10.10.18.214节点上执行ping
[root@evpn2 ~]# ip netns exec host3 ping 5.5.5.213 PING 5.5.5.213 (5.5.5.213) 56(84) bytes of data. 64 bytes from 5.5.5.213: icmp_seq=1 ttl=62 time=0.575 ms 64 bytes from 5.5.5.213: icmp_seq=2 ttl=62 time=0.514 ms 64 bytes from 5.5.5.213: icmp_seq=3 ttl=62 time=0.530 ms 64 bytes from 5.5.5.213: icmp_seq=4 ttl=62 time=0.566 ms ^C
10.10.18.213节点上执行tcpdump
[root@evpn213 ~]# tcpdump -i vxlan40 -eennvv dst host 4.4.4.3 tcpdump: listening on vxlan40, link-type EN10MB (Ethernet), capture size 262144 bytes 14:21:52.986497 00:00:01:02:03:08 > 00:00:01:02:03:09, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 61601, offset 0, flags [none], proto ICMP (1), length 84) 5.5.5.213 > 4.4.4.3: ICMP echo reply, id 10643, seq 1, length 64 14:21:54.044733 00:00:01:02:03:08 > 00:00:01:02:03:09, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 61673, offset 0, flags [none], proto ICMP (1), length 84) 5.5.5.213 > 4.4.4.3: ICMP echo reply, id 10643, seq 2, length 64 14:21:55.084773 00:00:01:02:03:08 > 00:00:01:02:03:09, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 61696, offset 0, flags [none], proto ICMP (1), length 84) 5.5.5.213 > 4.4.4.3: ICMP echo reply, id 10643, seq 3, length 64 14:21:56.124749 00:00:01:02:03:08 > 00:00:01:02:03:09, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 61743, offset 0, flags [none], proto ICMP (1), length 84) 5.5.5.213 > 4.4.4.3: ICMP echo reply, id 10643, seq 4, length 64 14:21:57.164776 00:00:01:02:03:08 > 00:00:01:02:03:09, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 61784, offset 0, flags [none], proto ICMP (1), length 84) 5.5.5.213 > 4.4.4.3: ICMP echo reply, id 10643, seq 5, length 64 ^C 5 packets captured 5 packets received by filter 0 packets dropped by kernel
[root@evpn213 ~]# tcpdump -i enp1s0 -eennvv dst port 4789
tcpdump: listening on enp1s0, link-type EN10MB (Ethernet), capture size 262144 bytes
14:23:42.204802 fa:16:3e:c1:8f:98 > fa:16:3e:a3:27:66, ethertype IPv4 (0x0800), length 148: (tos 0x0, ttl 64, id 3744, offset 0, flags [none], proto UDP (17), length 134)
10.10.18.214.53381 > 10.10.18.213.4789: [udp sum ok] VXLAN, flags [I] (0x08), vni 40
00:00:01:02:03:09 > 00:00:01:02:03:08, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 11410, offset 0, flags [DF], proto ICMP (1), length 84)
4.4.4.3 > 5.5.5.213: ICMP echo request, id 10643, seq 106, length 64
14:23:42.204885 fa:16:3e:a3:27:66 > fa:16:3e:c1:8f:98, ethertype IPv4 (0x0800), length 148: (tos 0x0, ttl 64, id 40091, offset 0, flags [none], proto UDP (17), length 134)
10.10.18.213.40576 > 10.10.18.214.4789: [bad udp cksum 0x3a42 -> 0xd400!] VXLAN, flags [I] (0x08), vni 40
00:00:01:02:03:08 > 00:00:01:02:03:09, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 1546, offset 0, flags [none], proto ICMP (1), length 84)
5.5.5.213 > 4.4.4.3: ICMP echo reply, id 10643, seq 106, length 64
14:23:43.244787 fa:16:3e:c1:8f:98 > fa:16:3e:a3:27:66, ethertype IPv4 (0x0800), length 148: (tos 0x0, ttl 64, id 3767, offset 0, flags [none], proto UDP (17), length 134)
10.10.18.214.53381 > 10.10.18.213.4789: [udp sum ok] VXLAN, flags [I] (0x08), vni 40
00:00:01:02:03:09 > 00:00:01:02:03:08, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 11508, offset 0, flags [DF], proto ICMP (1), length 84)
4.4.4.3 > 5.5.5.213: ICMP echo request, id 10643, seq 107, length 64
14:23:43.244860 fa:16:3e:a3:27:66 > fa:16:3e:c1:8f:98, ethertype IPv4 (0x0800), length 148: (tos 0x0, ttl 64, id 40123, offset 0, flags [none], proto UDP (17), length 134)
10.10.18.213.40576 > 10.10.18.214.4789: [bad udp cksum 0x3a42 -> 0xd400!] VXLAN, flags [I] (0x08), vni 40
00:00:01:02:03:08 > 00:00:01:02:03:09, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 1603, offset 0, flags [none], proto ICMP (1), length 84)
5.5.5.213 > 4.4.4.3: ICMP echo reply, id 10643, seq 107, length 64
14:23:44.284854 fa:16:3e:c1:8f:98 > fa:16:3e:a3:27:66, ethertype IPv4 (0x0800), length 148: (tos 0x0, ttl 64, id 3790, offset 0, flags [none], proto UDP (17), length 134)
10.10.18.214.53381 > 10.10.18.213.4789: [udp sum ok] VXLAN, flags [I] (0x08), vni 40
00:00:01:02:03:09 > 00:00:01:02:03:08, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 11583, offset 0, flags [DF], proto ICMP (1), length 84)
4.4.4.3 > 5.5.5.213: ICMP echo request, id 10643, seq 108, length 64
14:23:44.284937 fa:16:3e:a3:27:66 > fa:16:3e:c1:8f:98, ethertype IPv4 (0x0800), length 148: (tos 0x0, ttl 64, id 40146, offset 0, flags [none], proto UDP (17), length 134)
10.10.18.213.40576 > 10.10.18.214.4789: [bad udp cksum 0x3a42 -> 0xd400!] VXLAN, flags [I] (0x08), vni 40
00:00:01:02:03:08 > 00:00:01:02:03:09, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 1670, offset 0, flags [none], proto ICMP (1), length 84)
5.5.5.213 > 4.4.4.3: ICMP echo reply, id 10643, seq 108, length 64
^C
6 packets captured
6 packets received by filter
0 packets dropped by kernel