(場景:H3C交換機在多數做鏡像之后,同級聯芯片下端口由於業務流量及鏡像流量同時傳輸,有機會因流量突發導致buff不夠造成端口擁塞,擁塞現象會導致交換機采取“丟棄”操作。)
這里以鏡像目的口Te1/3/0/47正常情況:
涉及命令:
#
debug port mapping chassis 1 slot 2
bcm chassis 1 slot 2 chip 0 show/c
#
[NZX-1Z3(I06I07)-7510E]probe
[NZX-1Z3(I06I07)-7510E-probe]debug port mapping chassis 1 slot 3
[Interface] [Unit] [Port] [Name] [Combo?] [Active?] [IfIndex] [MID] [Link]
===============================================================================
XGE1/3/0/47 0 53 xe3 no no 0x152 7 up
第一次BCM命令:bcm chassis 1 slot 2 chip 0 show/c
PERQ_PKT(2).xe3 : 2,551,492,040,745 +2,301,709,200,310 162,633/s
PERQ_PKT(3).xe3 : 69,985 +69,985
PERQ_PKT(6).xe3 : 11 +11
PERQ_PKT(7).xe3 : 98,466,386 +98,466,386 11/s
PERQ_BYTE(2).xe3 : 2,274,408,370,946,342+2,088,422,758,496,242 136,802,146/s
PERQ_BYTE(3).xe3 : 5,878,890 +5,878,890
PERQ_BYTE(6).xe3 : 968 +968
PERQ_BYTE(7).xe3 : 9,131,069,746 +9,131,069,746 1,210/s
UC_PERQ_PKT(7).xe3: 272,951 +272,951
UC_PERQ_BYTE(7).xe3: 22,927,884 +22,927,884
PERQ_DROP_PKT(2).xe3: 36,303,435,197 +36,303,428,262
PERQ_DROP_BYTE(2).xe3: 50,089,943,447,973 +50,089,937,462,658
第二次BCM命令:bcm chassis 1 slot 2 chip 0 show/c
TBYT.xe3 : 2,223,395,095,444,475 +9,383,827,558 132,498,827/s
PERQ_PKT(2).xe3 : 2,551,504,069,885 +12,029,140 168,068/s
PERQ_PKT(7).xe3 : 98,466,530 +144 1/s
PERQ_BYTE(2).xe3 : 2,274,417,994,755,252 +9,623,808,910 135,999,999/s
PERQ_BYTE(7).xe3 : 9,131,083,945 +14,199 169/s
UC_PERQ_PKT(7).xe3: 272,953 +2
UC_PERQ_BYTE(7).xe3: 22,928,052 +168
(第二次BCM命令,就真的沒看到紅色字體部分了,證明端口正常)
------------------------------------------------------------------------------------------------------------------------------------
這里以端口已擁塞為例:
本地鏡像組:
Mirroring group 1:
Type: Local
Status: Active
Mirroring port:
Ten-GigabitEthernet1/0/0/10 Both
Ten-GigabitEthernet1/0/0/11 Both
Monitor port: Ten-GigabitEthernet1/0/0/1
2個10G端口為鏡像源端口,1個10G端口為鏡像目的端口。
應用側發現鏡像存在丟包情況后,查看鏡像目的端口接口下帶寬利用率低於80%,且接口統計信息無異常。
協調二線排查:
現網存在多打一的情況,鏡像組1跟2均擁有兩個源端口口,但只有一個目的口,且都為10G口。
mirroring-group 1
monitor-port Ten-GigabitEthernet1/0/0/1
mirroring-port interface Ten-GigabitEthernet1/0/0/10 interface Ten-GigabitEthernet1/0/0/11
mirroring-group 2
monitor-port Ten-GigabitEthernet2/0/0/1
mirroring-port interface Ten-GigabitEthernet2/0/0/10 interface Ten-GigabitEthernet2/0/0/11
3、 多次查看show/c值,目的口存在drop字段,且有計數增長的情況,端口確實存在擁塞現象:
====debug port mapping chassis 1 slot 0====
[Interface] [Unit] [Port] [Name] [Combo?] [Active?] [IfIndex] [MID] [Link]
===============================================================================
XGE1/0/0/1 0 8 xe3 no no 0x1 0 up
====debug port mapping chassis 2 slot 0====
[Interface] [Unit] [Port] [Name] [Combo?] [Active?] [IfIndex] [MID] [Link]
===============================================================================
XGE2/0/0/1 0 8 xe3 no no 0x619 64 up
第一次查看為drop的計數信息,第二次查看為相對於第一次的增量信息,發現存在增長,判斷接口存在擁塞。
[IDC-Intranet-HS125X-probe]bcm chassis 1 slot 0 chip 0 show/c
PERQ_DROP_PKT(2).xe3: 459,789,413,589 +459,789,413,589
PERQ_DROP_BYTE(2).xe3: 562,487,066,848,637+562,487,066,848,637
[IDC-Intranet-HS125X-probe]bcm chassis 1 slot 0 chip 0 show/c
PERQ_DROP_PKT(2).xe3: 459,789,769,121 +355,532 1,392/s
PERQ_DROP_BYTE(2).xe3: 562,487,452,855,732 +386,007,095 1,451,382/s
[IDC-Intranet-HS125X-probe]bcm chassis 2 slot 0 chip 0 show/c
PERQ_DROP_PKT(2).xe3: 508,463,912,685 +459,985,643,023 1,852/s
PERQ_DROP_BYTE(2).xe3: 622,188,285,240,693+559,284,395,936,733 2,209,637/s
[IDC-Intranet-HS125X-probe]bcm chassis 2 slot 0 chip 0 show/c
PERQ_DROP_PKT(2).xe3: 508,463,927,782 +15,097 4,336/s
PERQ_DROP_BYTE(2).xe3: 622,188,302,024,390 +16,783,697 5,021,082/s
現網擁塞情況有以下兩種辦法改善:
1、 將目的口更改為更高速率的接口,如40G口。
2、 將目的口更改為二層聚合口,增加成員端口。