在實際的情況下，有時候會遇到設備無故重啟的問題，這個時候，我們需要判斷一下重啟的根本原因是什么，是否有規避的方法等。
這里記錄了幾個N3K異常重啟的問題。
案例1：
設備型號：N3K-C3048TP-1GE
OS版本：6.0(2)U4(4)
階段1：采集信息。我們需要采集的信息如下：
1、收集core文件
N3K# show core
2、收集tech文件
#tac-pac bootflash:
如此會在bootflash自動創建一個名為“show_tech_out.gz”的文件，然后可以通過ftp或者tftp方式導出
階段2：分析（這里是一個故障的實例）
========
問題描述
========
N3K unexpectedly reboot in 2016 Dec 10 08:23
2016 Dec 10 08:23:20.318 N3K-01 %$ VDC-1 %$ %SYSMGR-2-SERVICE_CRASHED: Service "bcm_usd" (PID 3257) hasn't caught signal 6 (core will be saved).
2016 Dec 10 08:23:20.336 N3K-01 %$ VDC-1 %$ %SYSMGR-2-HAP_FAILURE_SUP_RESET: System reset due to service "bcm_usd" in vdc 1 has had a hap failure
========
分析過程
========
1. 重啟原因
`show system reset-reason`
----- reset reason for Supervisor-module 1 (from Supervisor in slot 1) ---
1) At 948672 usecs after Thu Dec 10 08:23 2016
Reason: Reset triggered due to HA policy of Reset
Service: bcm_usd hap reset
Version: 6.0(2)U1(2)
2. 啟動前后
Software
BIOS: version 1.2.0
loader: version N/A
kickstart: version 6.0(2)U4(4)
system: version 6.0(2)U4(4)
Power Sequencer Firmware:
Module 1: version v4.4
BIOS compile time: 08/22/2014
kickstart image file is: bootflash:///n3000-uk9-kickstart.6.0.2.U4.4.bin
kickstart compile time: 3/12/2015 0:00:00 [03/12/2015 17:34:09]
system image file is: bootflash:///n3000-uk9.6.0.2.U4.4.bin
system compile time: 3/12/2015 0:00:00 [03/12/2015 20:28:55]
Hardware
cisco Nexus 3048 Chassis ("48x1GE + 4x10G Supervisor")
Intel(R) Celeron(R) CPU P450 with 3665256 kB of memory.
Processor Board ID FOCXXXXXXXX
Device name: N3K-01
bootflash: 2007040 kB
Kernel uptime is 0 day(s), 08 hour(s), 37 minute(s), 21 second(s)
Last reset at 948672 usecs after Thu Dec 10 08:23 2016
Reason: Reset triggered due to HA policy of Reset
System version: 6.0(2)U1(2)
Service: bcm_usd hap reset
我們可以看到系統重置是由bcm_usd hap重置引起的。 如果可以解碼core文件，我們可以發現6.0（2）U1（2）這個版本也命中bug，bug ID：CSCuo88390（Nexus 3k bcm_usd crash after parity error）的錯誤。 從show version，我們看到當前版本是6.0.2.U4.4，這是修復了bug的版本。
bug的詳細信息如下：
Nexus 3000 switch may reboot unexpectedly. Last reset reason is recorded as 'bcm_usd hap reset'.
`show system reset-reason`
----- reset reason for Supervisor-module 1 (from Supervisor in slot 1) ---
1) At 369448 usecs after Tue May 6 16:49:37 2014
Reason: Reset triggered due to HA policy of Reset
Service: bcm_usd hap reset
Version: 6.0(2)U1(1a)
Conditions:
The crash occurs while recovering from a parity error. This bug will not be seen if a parity error does not occur first.
Workaround:
Not known at this time.
Further Problem Description:
=======
建議
=======
如果該錯誤在短時間內持續發生，建議直接更換設備，該錯誤可能是由於硬件導致，如觀察段時間內並未發生，則可以忽略這個問題。
案例2：
1、采集信息
采集的信息還是和案例1中一直，查看core文件，采集tech文件
2、分析
異常log如下：
2016 Dec 11 01:23:18.782 N3K-01 %$ VDC-1 %$ %BCM_USD-2-BCM_USD_NOTIFICATION_IMP: FAST REBOOT DISABLED
2016 Dec 11 01:23:18.782 N3K-01 %$ VDC-1 %$ %USER-2-SYSTEM_MSG: CLIS: loading cmd files begin - clis
2016 Dec 11 01:23:18.782 N3K-01 %$ VDC-1 %$ %USER-2-SYSTEM_MSG: CLIS: loading cmd files end - clis
2016 Dec 11 01:23:18.782 N3K-01 %$ VDC-1 %$ %USER-2-SYSTEM_MSG: CLIS: init begin - clis
2016 Dec 11 01:23:18.782 N3K-01 %$ VDC-1 %$ %USER-0-SYSTEM_MSG: Starting bcm_attach - bcm_usd
2016 Dec 11 01:23:18.782 N3K-01 %$ VDC-1 %$ %USER-0-SYSTEM_MSG: Finished bcm_attach... - bcm_usd
2016 Dec 11 01:23:18.782 N3K-01 %$ VDC-1 %$ %VDC_MGR-2-VDC_ONLINE: vdc 1 has come online
2016 Dec 11 01:23:18.782 N3K-01 %$ VDC-1 %$ %ETHPC-2-PORTS_UP:
其次，從show core中沒有看到有core文件
再看重啟原因：
`show system reset-reason`
----- reset reason for Supervisor-module 1 (from Supervisor in slot 1) ---
1) No time
Reason: Unknown
Service:
Version: 6.0(2)U4(4)
2) No time
Reason: Unknown
Service:
Version: 6.0(2)U4(4)
3) No time
Reason: Unknown
Service:
Version: 6.0(2)U4(4)
Mon Dec 11 01:23:18 2016: Card Uptime Record
----------------------------------------------
Uptime: 28938, 0 days 6 hour(s) 3 minute(s) 12 second(s)
Reset Reason: Unknown (0)
Reset Reason SW: Unknown (0)
Reset Reason (HW): Unknown
Card Mode..........................: Runtime
這個問題很可能是和PSU（Power Supply Unit）的的關系，超出規定范圍的電源電壓值，可能是電源相關程序觸發設備重啟，以防止進一步的損壞。“Unknown”的重置原因表面設備由於電源序列發生器組件重新加載：由於電源不足或PSU出現問題。
可以看到，目前設備的PS2沒有up.
Power Supply:
Voltage: 12 Volts
-----------------------------------------------------------
PS Model Input Power Current Status Type (Watts) (Amps)
-----------------------------------------------------------
1 N2200-PAC-400W-B AC 396.00 33.00 ok
2 -- -- -- -- fail/not-powered-up
由此，這個重啟問題可能是由於電源硬件問題引起的。建議檢查外部電源是否有問題，是否有其他的設備和該N3K連接相同的PDU，還可以檢查次設備日志，來判斷當時是否有電源問題。如果沒有外源的問題，那建議更換電源。
案例3：
1、采集信息：
采集的信息和案例1,2中提到的一樣。
2、分析：
2.1 先檢查logging，觀察FAN，PS是否log報錯。
2.2 設備重啟原因是 I2C bus 擁塞，導致設備認為 FAN 被拔出，這種應該不是硬件問題。 I2C 通道負責在主板、電源、風扇、SFP 等之間的通信；如果 I2C 通道擁堵，可能導致設備不能正確讀取 FAN 狀態。
%KERN-2-SYSTEM_MSG: [23949956.295550] clk_flush: Couldn't Clear Bus – kernel << I2C bus 擁塞
CST: %PFMA-0-SYS_SHUTDOWN_FAN_REMOVAL: System shutdown in 120 seconds due to fan missing or failed
CST: %NOHMS-2-NOHMS_ENV_ERR_FAN_READ: System minor alarm in fan tray 1: failed to read TACH << 風扇讀取失敗
Wed Oct 10 15:10:38 2018: Card Uptime Record
----------------------------------------------
Uptime: 3674, 0 days 1 hour(s) 1 minute(s) 14 second(s)
Reset Reason: Unknown (0)
Reset Reason SW: Powered-down due to fan policy trigger (104)
Reset Reason (HW): Unknown
Card Mode..........................: Runtime
N3K 重啟，啟動以后，風扇和電源狀態都正常：
Fan:
------------------------------------------------------
Fan Model Hw Status
------------------------------------------------------
Fan-1 NXA-FAN-30CFM-B -- ok
Fan-2 NXA-FAN-30CFM-B -- ok
Fan-3 NXA-FAN-30CFM-B -- ok
Fan-4 NXA-FAN-30CFM-B -- ok
PS-1 N2200-PAC-400W-B -- ok
PS-2 N2200-PAC-400W-B -- ok
Power Supply:
Voltage: 12 Volts
-----------------------------------------------------------
PS Model Input Power Current Status Type (Watts) (Amps)
-----------------------------------------------------------
1 N2200-PAC-400W-B AC 396.00 33.00 ok
2 N2200-PAC-400W-B AC 396.00 33.00 ok
建議：無已知的 bug匹配，建議升級推薦版本。如果再次遇到同樣問題，可以更換機箱。
https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus3000/sw/recommended_release/b_Minimum_and_Recommended_Cisco_NX-OS_Releases_for_Cisco_Nexus_3000_Series_Switches.html
其他log分析：N3K 重啟之前，有一些其他 kernel 和電源相關的 log，是顯示性問題，實際上沒有影響
%KERN-3-SYSTEM_MSG: [23949949.272393] PFM: Error while trying to service 0x2aa, err=-14 - kernel
https://bst.cloudapps.cisco.com/bugsearch/bug/CSCur58188/?reffering_site=dumpcr
CST: %PFMA-2-PS_FAIL: Power supply 1 failed or shutdown (Serial number DCA00000000)
CST: %NOHMS-2-NOHMS_DIAG_ERR_PS_FAIL: System minor alarm on power supply 2: failed or not powered up
https://bst.cloudapps.cisco.com/bugsearch/bug/CSCva03834/?reffering_site=dumpcr
https://bst.cloudapps.cisco.com/bugsearch/bug/CSCux99539/?reffering_site=dumpcr
設備一直報錯：
%$ VDC-1 %$ Oct 10 19:36:06 %KERN-2-SYSTEM_MSG: [ 5375.470604] clk_flush: Couldn't Clear Bus - kernel
最終該問題摘要：
1、有兩台N3K,當重啟Device2時，Device1無法轉發信息，當Device2恢復后，一切正常了。
2、Device2重啟時異常：流量從一個接口進來，但是沒有從對應的接口出去。在沖Device1之后，問題得到解決。
3、建議：目前使用的6.0(2)A6(1a)版本太低，建議升級
https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus3000/sw/recommended_release/b_Minimum_and_Recommended_Cisco_NX-OS_Releases_for_Cisco_Nexus_3000_Series_Switches.html?dtid=osscdc000283