Vxlan BGP-EVPN分布式網關ENSP實驗

實驗拓撲圖如下：

忽略Spine之間的互聯及peer

underlay網絡使用的協議是OSPF

 VTEP地址規划如下：

由於本場景中所有設備都是單節點，不涉及隧道的負載，所以VTEP地址和建立BGP的地址可以是同一個地址

對於是有m-lag的節點或其他需要實現隧道負載的場景，VTEP地址和建立BGP的地址不能使用相同的地址。

 詳細業務規划如下：

   假設有兩個租戶(業務)，分別為業務A和業務B，詳細規划如下：

配置信息如下：

<Spine1>dis current-configuration
sysname Spine1
#
evpn-overlay enable
#
interface GE1/0/0
undo portswitch
undo shutdown
ip address 10.1.13.1 255.255.255.0
ospf network-type p2p
ospf enable 1 area 0.0.0.0
#
interface GE1/0/1
undo portswitch
undo shutdown
ip address 10.1.14.1 255.255.255.0
ospf network-type p2p
ospf enable 1 area 0.0.0.0
#
interface GE1/0/2
undo portswitch
undo shutdown
ip address 10.1.15.1 255.255.255.0
ospf network-type p2p
ospf enable 1 area 0.0.0.0
#
interface GE1/0/3
undo portswitch
undo shutdown
ip address 10.1.16.1 255.255.255.0
ospf network-type p2p
ospf enable 1 area 0.0.0.0
#
interface GE1/0/4
undo portswitch
undo shutdown
ip address 10.1.12.1 255.255.255.0
ospf network-type p2p
ospf enable 1 area 0.0.0.0
#
interface LoopBack0
ip address 1.1.1.1 255.255.255.255
ospf enable 1 area 0.0.0.0
#
bgp 100
peer 2.2.2.2 as-number 100
peer 2.2.2.2 connect-interface LoopBack0
peer 3.3.3.3 as-number 100
peer 3.3.3.3 connect-interface LoopBack0
peer 4.4.4.4 as-number 100
peer 4.4.4.4 connect-interface LoopBack0
peer 5.5.5.5 as-number 100
peer 5.5.5.5 connect-interface LoopBack0
peer 6.6.6.6 as-number 100
peer 6.6.6.6 connect-interface LoopBack0
#
ipv4-family unicast
 undo peer 3.3.3.3 enable
 undo peer 4.4.4.4 enable
 undo peer 5.5.5.5 enable
 undo peer 6.6.6.6 enable
 undo peer 2.2.2.2 enable
#
l2vpn-family evpn
 undo policy vpn-target
 peer 2.2.2.2 enable
 peer 2.2.2.2 advertise irb
 peer 2.2.2.2 reflect-client
 peer 3.3.3.3 enable
 peer 3.3.3.3 advertise irb
 peer 3.3.3.3 reflect-client
 peer 4.4.4.4 enable
 peer 4.4.4.4 advertise irb
 peer 4.4.4.4 reflect-client
 peer 5.5.5.5 enable
 peer 5.5.5.5 advertise irb
 peer 5.5.5.5 reflect-client
 peer 6.6.6.6 enable
 peer 6.6.6.6 advertise irb
 peer 6.6.6.6 reflect-client
#
ospf 1
area 0.0.0.0
#

Spine1

<Spine2>dis current-configuration
sysname Spine2
#
evpn-overlay enable
#
interface GE1/0/0
undo portswitch
undo shutdown
ip address 10.1.23.2 255.255.255.0
ospf network-type p2p
ospf enable 1 area 0.0.0.0
#
interface GE1/0/1
undo portswitch
undo shutdown
ip address 10.1.24.2 255.255.255.0
ospf network-type p2p
ospf enable 1 area 0.0.0.0
#
interface GE1/0/2
undo portswitch
undo shutdown
ip address 10.1.25.2 255.255.255.0
ospf network-type p2p
ospf enable 1 area 0.0.0.0
#
interface GE1/0/3
undo portswitch
undo shutdown
ip address 10.1.26.2 255.255.255.0
ospf network-type p2p
ospf enable 1 area 0.0.0.0
#
interface GE1/0/4
undo portswitch
undo shutdown
ip address 10.1.12.2 255.255.255.0
ospf network-type p2p
ospf enable 1 area 0.0.0.0
#
interface LoopBack0
ip address 2.2.2.2 255.255.255.255
ospf enable 1 area 0.0.0.0
#
bgp 100
peer 1.1.1.1 as-number 100
peer 1.1.1.1 connect-interface LoopBack0
peer 3.3.3.3 as-number 100
peer 3.3.3.3 connect-interface LoopBack0
peer 4.4.4.4 as-number 100
peer 4.4.4.4 connect-interface LoopBack0
peer 5.5.5.5 as-number 100
peer 5.5.5.5 connect-interface LoopBack0
peer 6.6.6.6 as-number 100
peer 6.6.6.6 connect-interface LoopBack0
#
ipv4-family unicast
 undo peer 1.1.1.1 enable
 undo peer 3.3.3.3 enable
 undo peer 4.4.4.4 enable
 undo peer 5.5.5.5 enable
 undo peer 6.6.6.6 enable
#
l2vpn-family evpn
 undo policy vpn-target
 peer 1.1.1.1 enable
 peer 1.1.1.1 advertise irb
 peer 1.1.1.1 reflect-client
 peer 3.3.3.3 enable
 peer 3.3.3.3 advertise irb
 peer 3.3.3.3 reflect-client
 peer 4.4.4.4 enable
 peer 4.4.4.4 advertise irb
 peer 4.4.4.4 reflect-client
 peer 5.5.5.5 enable
 peer 5.5.5.5 advertise irb
 peer 5.5.5.5 reflect-client
 peer 6.6.6.6 enable
 peer 6.6.6.6 advertise irb
 peer 6.6.6.6 reflect-client
#
ospf 1
area 0.0.0.0
#

       
[~Leaf1]dis current-configuration
#
sysname Leaf1
#
evpn-overlay enable
#
ip vpn-instance vpnA
ipv4-family
 route-distinguisher 1111:1111
 vpn-target 1111:1111 export-extcommunity evpn
 vpn-target 1111:1111 import-extcommunity evpn
vxlan vni 5010
#
bridge-domain 10
vxlan vni 10
evpn
 route-distinguisher 10:10
 vpn-target 10:10 export-extcommunity
 vpn-target 1111:1111 export-extcommunity
 vpn-target 10:10 import-extcommunity
arp broadcast-suppress mismatch-discard enable
#
interface Vbdif10
ip binding vpn-instance vpnA
ip address 10.1.1.1 255.255.255.0
mac-address 00e0-1010-0001
vxlan anycast-gateway enable
arp collect host enable
#
interface GE1/0/0
undo portswitch
undo shutdown
ip address 10.1.13.3 255.255.255.0
ospf network-type p2p
ospf enable 1 area 0.0.0.0
#
interface GE1/0/1
undo portswitch
undo shutdown
ip address 10.1.23.3 255.255.255.0
ospf network-type p2p
ospf enable 1 area 0.0.0.0
#
interface GE1/0/2
undo shutdown
#
interface GE1/0/2.10 mode l2
encapsulation dot1q vid 10
bridge-domain 10
#
interface LoopBack0
ip address 3.3.3.3 255.255.255.255
ospf enable 1 area 0.0.0.0
#
interface Nve1
source 3.3.3.3
vni 10 head-end peer-list protocol bgp
#
bgp 100
peer 1.1.1.1 as-number 100
peer 1.1.1.1 connect-interface LoopBack0
peer 2.2.2.2 as-number 100
peer 2.2.2.2 connect-interface LoopBack0
#
ipv4-family unicast
 undo peer 1.1.1.1 enable
 undo peer 2.2.2.2 enable
#
l2vpn-family evpn
 policy vpn-target
 peer 1.1.1.1 enable
 peer 1.1.1.1 advertise irb
 peer 2.2.2.2 enable
 peer 2.2.2.2 advertise irb
#
ospf 1
area 0.0.0.0
#

<Leaf2>dis current-configuration
#
sysname Leaf2
#
evpn-overlay enable
#
ip vpn-instance vpnA
ipv4-family
 route-distinguisher 1111:1111
 vpn-target 1111:1111 export-extcommunity evpn
 vpn-target 1111:1111 import-extcommunity evpn
vxlan vni 5010
#
bridge-domain 20
vxlan vni 20
evpn
 route-distinguisher 20:20
 vpn-target 20:20 export-extcommunity
 vpn-target 1111:1111 export-extcommunity
 vpn-target 20:20 import-extcommunity
arp broadcast-suppress mismatch-discard enable
#
bridge-domain 30
vxlan vni 30
evpn
 route-distinguisher 30:30
 vpn-target 30:30 export-extcommunity
 vpn-target 1111:1111 export-extcommunity
 vpn-target 30:30 import-extcommunity
arp broadcast-suppress mismatch-discard enable
#
interface Vbdif20
ip binding vpn-instance vpnA
ip address 20.1.1.1 255.255.255.0
mac-address 00e0-2020-0001
vxlan anycast-gateway enable
arp collect host enable
#
interface Vbdif30
ip binding vpn-instance vpnA
ip address 30.1.1.1 255.255.255.0
mac-address 00e0-3030-0001
vxlan anycast-gateway enable
arp collect host enable
#
interface GE1/0/0
undo portswitch
undo shutdown
ip address 10.1.14.4 255.255.255.0
ospf network-type p2p
ospf enable 1 area 0.0.0.0
#
interface GE1/0/1
undo portswitch
undo shutdown
ip address 10.1.24.4 255.255.255.0
ospf network-type p2p
ospf enable 1 area 0.0.0.0
#
interface GE1/0/2
undo shutdown
#
interface GE1/0/2.20 mode l2
encapsulation dot1q vid 20
bridge-domain 20
#
interface GE1/0/2.30 mode l2
encapsulation dot1q vid 30
bridge-domain 30
#
interface LoopBack0
ip address 4.4.4.4 255.255.255.255
ospf enable 1 area 0.0.0.0
#
interface Nve1
source 4.4.4.4
vni 20 head-end peer-list protocol bgp
vni 30 head-end peer-list protocol bgp
#
interface NULL0
#
bgp 100
peer 1.1.1.1 as-number 100
peer 1.1.1.1 connect-interface LoopBack0
peer 2.2.2.2 as-number 100
peer 2.2.2.2 connect-interface LoopBack0
#
ipv4-family unicast
 undo peer 1.1.1.1 enable
 undo peer 2.2.2.2 enable
#
l2vpn-family evpn
 policy vpn-target
 peer 1.1.1.1 enable
 peer 1.1.1.1 advertise irb
 peer 2.2.2.2 enable
 peer 2.2.2.2 advertise irb
#
ospf 1
area 0.0.0.0
#

<Leaf3>dis current-configuration
#
sysname Leaf3
#
evpn-overlay enable
#
ip vpn-instance vpnA
ipv4-family
  route-distinguisher 1111:1111
  vpn-target 1111:1111 export-extcommunity evpn
  vpn-target 1111:1111 import-extcommunity evpn
vxlan vni 5010
#
ip vpn-instance vpnB
ipv4-family
  route-distinguisher 2222:2222
  vpn-target 2222:2222 export-extcommunity evpn
  vpn-target 2222:2222 import-extcommunity evpn
vxlan vni 5020
#
bridge-domain 20
vxlan vni 20
evpn
  route-distinguisher 20:20
  vpn-target 20:20 export-extcommunity
  vpn-target 1111:1111 export-extcommunity
  vpn-target 20:20 import-extcommunity
arp broadcast-suppress mismatch-discard enable
#
bridge-domain 40
vxlan vni 40
evpn
  route-distinguisher 40:40
  vpn-target 40:40 export-extcommunity
  vpn-target 2222:2222 export-extcommunity
  vpn-target 40:40 import-extcommunity
arp broadcast-suppress mismatch-discard enable
#
interface Vbdif20
ip binding vpn-instance vpnA
ip address 20.1.1.1 255.255.255.0
mac-address 00e0-2020-0001
vxlan anycast-gateway enable
arp collect host enable
#
interface Vbdif40
ip binding vpn-instance vpnB
ip address 40.1.1.1 255.255.255.0
mac-address 00e0-4040-0001
vxlan anycast-gateway enable
arp collect host enable
#
interface GE1/0/0
undo portswitch
undo shutdown
ip address 10.1.15.5 255.255.255.0
ospf network-type p2p
ospf enable 1 area 0.0.0.0
#
interface GE1/0/1
undo portswitch
undo shutdown
ip address 10.1.25.5 255.255.255.0
ospf network-type p2p
ospf enable 1 area 0.0.0.0
#
interface GE1/0/2
undo shutdown
#
interface GE1/0/2.20 mode l2
encapsulation dot1q vid 20
bridge-domain 20
#
interface GE1/0/2.40 mode l2
encapsulation dot1q vid 40
bridge-domain 40
#
interface LoopBack0
ip address 5.5.5.5 255.255.255.255
ospf enable 1 area 0.0.0.0
#
interface Nve1
source 5.5.5.5
vni 20 head-end peer-list protocol bgp
vni 40 head-end peer-list protocol bgp
#
bgp 100
peer 1.1.1.1 as-number 100
peer 1.1.1.1 connect-interface LoopBack0
peer 2.2.2.2 as-number 100
peer 2.2.2.2 connect-interface LoopBack0
#
ipv4-family unicast
  undo peer 1.1.1.1 enable
  undo peer 2.2.2.2 enable
#
l2vpn-family evpn
  policy vpn-target
  peer 1.1.1.1 enable
  peer 1.1.1.1 advertise irb
  peer 2.2.2.2 enable
  peer 2.2.2.2 advertise irb
#
ospf 1
area 0.0.0.0
#

<Leaf4>dis current-configuration
#
sysname Leaf4
#
evpn-overlay enable
#
ip vpn-instance vpnA
ipv4-family
  route-distinguisher 1111:1111
  vpn-target 1111:1111 export-extcommunity evpn
  vpn-target 1111:1111 import-extcommunity evpn
vxlan vni 5010
#
ip vpn-instance vpnB
ipv4-family
  route-distinguisher 2222:2222
  vpn-target 2222:2222 export-extcommunity evpn
  vpn-target 2222:2222 import-extcommunity evpn
vxlan vni 5020
#
bridge-domain 30
vxlan vni 30
evpn
  route-distinguisher 30:30
  vpn-target 30:30 export-extcommunity
  vpn-target 1111:1111 export-extcommunity
  vpn-target 30:30 import-extcommunity
arp broadcast-suppress mismatch-discard enable
#
bridge-domain 40
vxlan vni 40
evpn
  route-distinguisher 40:40
  vpn-target 40:40 export-extcommunity
  vpn-target 2222:2222 export-extcommunity
  vpn-target 40:40 import-extcommunity
arp broadcast-suppress mismatch-discard enable
#
bridge-domain 50
vxlan vni 50
evpn
  route-distinguisher 50:50
  vpn-target 50:50 export-extcommunity
  vpn-target 2222:2222 export-extcommunity
  vpn-target 50:50 import-extcommunity
arp broadcast-suppress mismatch-discard enable
#
interface Vbdif30
ip binding vpn-instance vpnA
ip address 30.1.1.1 255.255.255.0
mac-address 00e0-3030-0001
vxlan anycast-gateway enable
arp collect host enable
#
interface Vbdif40
ip binding vpn-instance vpnB
ip address 40.1.1.1 255.255.255.0
mac-address 00e0-4040-0001
vxlan anycast-gateway enable
arp collect host enable
#
interface Vbdif50
ip binding vpn-instance vpnB
ip address 50.1.1.1 255.255.255.0
mac-address 00e0-5050-0001
vxlan anycast-gateway enable
arp collect host enable
#
interface GE1/0/0
undo portswitch
undo shutdown
ip address 10.1.16.6 255.255.255.0
ospf network-type p2p
ospf enable 1 area 0.0.0.0
#
interface GE1/0/1
undo portswitch
undo shutdown
ip address 10.1.26.6 255.255.255.0
ospf network-type p2p
ospf enable 1 area 0.0.0.0
#
interface GE1/0/2
undo shutdown
#
interface GE1/0/2.30 mode l2
encapsulation dot1q vid 30
bridge-domain 30
#
interface GE1/0/2.40 mode l2
encapsulation dot1q vid 40
bridge-domain 40
#
interface GE1/0/2.50 mode l2
encapsulation dot1q vid 50
bridge-domain 50
#
interface LoopBack0
ip address 6.6.6.6 255.255.255.255
ospf enable 1 area 0.0.0.0
#
interface Nve1
source 6.6.6.6
vni 30 head-end peer-list protocol bgp
vni 40 head-end peer-list protocol bgp
vni 50 head-end peer-list protocol bgp
#
bgp 100
peer 1.1.1.1 as-number 100
peer 1.1.1.1 connect-interface LoopBack0
peer 2.2.2.2 as-number 100
peer 2.2.2.2 connect-interface LoopBack0
#
ipv4-family unicast
  undo peer 1.1.1.1 enable
  undo peer 2.2.2.2 enable
#
l2vpn-family evpn
  policy vpn-target
  peer 1.1.1.1 enable
  peer 1.1.1.1 advertise irb
  peer 2.2.2.2 enable
  peer 2.2.2.2 advertise irb
#
ospf 1
area 0.0.0.0
#

sysname Vswitch1
#
vlan batch 10
#
interface GigabitEthernet0/0/1
port link-type trunk
port trunk allow-pass vlan 2 to 4094
#
interface GigabitEthernet0/0/2
port link-type access
port default vlan 10
#
interface GigabitEthernet0/0/3
port link-type access
port default vlan 10
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
sysname Vswitch2
#
vlan batch 20 30
#
interface GigabitEthernet0/0/1
port link-type trunk
port trunk allow-pass vlan 2 to 4094
#
interface GigabitEthernet0/0/2
port link-type access
port default vlan 20
#
interface GigabitEthernet0/0/3
port link-type access
port default vlan 30
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
sysname Vswitch3
#
vlan batch 20 40
#
interface GigabitEthernet0/0/1
port link-type trunk
port trunk allow-pass vlan 2 to 4094
#
interface GigabitEthernet0/0/2
port link-type access
port default vlan 20
#
interface GigabitEthernet0/0/3
port link-type access
port default vlan 40
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
sysname Vswitch4
#
vlan batch 30 40 50
#
interface GigabitEthernet0/0/1
port link-type trunk
port trunk allow-pass vlan 2 to 4094
#
interface GigabitEthernet0/0/2
port link-type access
port default vlan 30
#
interface GigabitEthernet0/0/3
port link-type access
port default vlan 40
#
interface GigabitEthernet0/0/4
port link-type access
port default vlan 50

 在Spine設備查看EVPN鄰居關系：

 查看每個leaf上的vxlan隧道：

 在所有PC上Ping網關地址，以PC1為例，其他設備同，如下：

目的是為了讓所有leaf上學習到ARP信息：

 在所有的leaf上查看ARP信息，如下：

 leaf1在BD10下游2個主機

 leaf2在BD20和BD30下各有一台主機

 leaf3在BD20和BD40下各有一台主機

 leaf4在BD30、BD40和BD50下各有一台主機

 不同leaf相同BD下的主機互ping一下，為了觀察MAC地址表

以PC5 ping PC3為例，如下：

先看下PC5的MAC地址，如下：

然后在leaf2上查看BD20的MAC地址表，是否有PC5的MAC地址：

leaf2上已經記錄了PC5的MAC地址，如下：

查看ARP的廣播抑制表，以leaf2為例，如下：

如果同網段互訪的所有BUM幀都在leaf上進行頭端復制，向所有相同VNI泛洪，必然比占用大量的資源開銷，開啟ARP的廣播抑制后，當leaf收到BUM幀后，先查自身的ARP抑制表，匹配則廣播變單播，表象如下：

查看每個leaf上的實例路由表，查看是否學習到了路由：

4個leaf都學習到了同租戶的所有ARP路由，如下：

 

 

 有路由后驗證跨leaf同網段和不同網段的互訪是否正常：

PC1訪問PC7跨網段互訪，可以ping通，如下圖：

PC1訪問PC7的ICMP抓包如下圖：

跨網段互訪VXLAN封裝的是三層VNI

 

 PC8訪問PC6為同網段互訪，可以ping通，如下圖：

 

  PC8訪問PC6的ICMP抓包如下圖：

同網段互訪VXLAN封裝的是二層VNI

 

 

 接下來看一下EVPN路由是什么樣子的，以leaf1為例，如下圖：

 

 看一下路由條目的詳細信息：

路由更新過程的抓包，以leaf1發送的BGP Updata報文為例，如下：（Type2路由）

TYPE2路由的字段解讀：

 接下來再看一下Type5類路由是怎么傳遞的

Type5類路由是外部路由產生的，在leaf1上配置一個loop100口，然后引入直連

在leaf1上需要增加以下配置：

interface LoopBack100
 ip binding vpn-instance vpnA
 ip address 100.1.1.1 255.255.255.255
bgp 100
 ipv4-family vpn-instance vpnA
  import-route direct
  advertise l2vpn evpn

在leaf1上看下EVPN的路由表，如下：

看下leaf1發送的Updata報文，如下：（Type5路由）

TYPE5路由的字段解讀：

 接下來在看下Type3的UPdata路由，以leaf1為例。如下圖：（Type3路由）

 

 Type3的路由抓包如下：

TYPE3路由的字段解讀：

 關於RT值：

同網段互訪使用BD下的RT進行導入導出（二層互訪）

不同網段的互訪使用VPN實例下的RT進行導入導出（三層互訪）

如上圖：BD下的eiRT為10:10  ,VPN實例下的eiRT值為1111:1111

默認情況下UPDATA路由會攜帶RT值為10:10

如果想讓RT值1111:1111也添加到BGP路由的團體屬性中，需要在BD的EVPN視圖下 再次配置export RT： （1111:1111）

這樣BGP路由會攜帶RT（10:10）RT（1111:1111）

一旦路由添加上RT屬性，對於BD下的RT和VPN實例下的RT值是不做區分的，看報文如下：

在報文中的表示方式除了RT值不同以外，無其他區分標識

那么這樣當路由到達對端設備時，對端就可以用任何一個RT值進行導入即可。

如：leaf2的要接受leaf1的路由（不同網段的）

那就需要在leaf2的實例下導入BGP路由對於的RT值，上面提到該路由攜帶了兩個RT(10:10)RT(1111:1111)

兩個RT值有沒有任何區分的標識

所以leaf2在實例下導入iRT（1111:1111）可以接收改路由，導入iRT（10:10）也可以接收該路由

如下：

雖然在實例下導入RT值時可以導入任一個RT值，但是在實際使用中最好還是BD的RT互相導入導出，實例下的RT互相導入導出。

如下圖：

關於ENSP做Vxlan實驗遇到的問題：

1、重啟可能丟失配置，做好配置后把導出一份，防止配置丟失（勤導出，不然你可能會哭。。。。）

2、重啟后可能同網段互通出現問題、在leaf的子接口下undo bridge-domain 然后在重新配置，即可解決。

3、關於Vbox下將CE虛擬機的內存調的盡可能大，不然可能有時可能會看不到現象或者直接卡死，我的物理主機內存有380G，所以每個CE128我給40G內存

4、wireshark一定要用最新版，老版本抓包看不到EVPN的路由信息。

綜上所述：四個結論

1、本端VPN實例下的導出值不配置也沒關系，只要BD的EVPN下配置了對應的導出即可

2、我要導入對端發送的路由時，可以導入對端實例下的eRT值，也可以導入對端EVPN下的eRT值

3、為了維護方便，查看方便，最好還是VPN與VPN對應，EVPN與EVPN對應

4、由於是EVPN鄰居，傳遞的是EVPN路由，所以只有在本地的EVPN下配置的eRT才會被添加到EVPN路由上。

 

ENSP、CE鏡像、抓包軟件可在百度雲下載：

鏈接：https://pan.baidu.com/s/1F8NwUZnzqzvTanM69jd5Tg
提取碼：v2t6
復制這段內容后打開百度網盤手機App，操作更方便哦