轉自：https://blog.csdn.net/lsy673908720/article/details/90215501
工欲善其事，必先利其器。操作系統崩潰是高級Linux運維工程師很頭疼的事情，但也不要害怕，前輩們已經給我們磨了好多把刀，其中有一把叫做Kdump。目前在主流的Linux發行版如Ubuntu、Centos等已經默認安裝或者倉庫中含有工具包，但是經過裁剪的嵌入式Linux里面沒有包含Kdump。本文章使用busybox建立最小文件系統，講述移植Kdump至嵌入式ARM64的環境中。
一、使用busybox建立最小文件系統
這個我就不講了，度娘上一大堆文章。建立rootfs文件夾，將busybox下的 _install內容全部拷貝至rootfs中。
二、編譯Kdump工具
（1）從下面的網站下載最新的kexec-tools源碼包。
http://kernel.org/pub/linux/utils/kernel/kexec/kexec-tools.tar.gz
（2）解壓源碼包。
tar xvpzf kexec-tools.tar.gz
（3）進入到kexec-tools中，並進行配置。
LDFLAGS=-static ./configure ARCH=arm64 --build=x86_64-linux-gnu --host=aarch64-linux-gnu --target=aarch64-linux-gnu --without-xen
這里使用靜態編譯。
（4）然后使用make進行編譯。
Make
（5）將build目錄下生成文件拷貝至rootfs文件夾中。
三、內核配置
（1）在"Processor type and features.“下使能"kexec system call”。
CONFIG_KEXEC=y
（2）在"Filesystem" -> “Pseudo filesystems.“下使能"sysfs file system support”。
CONFIG_SYSFS=y
（3）在"Kernel hacking.“下使能"Compile the kernel with debug info” 。
CONFIG_DEBUG_INFO=Y
（4）在"Processor type and features"下使能"kernel crash dumps”。
CONFIG_CRASH_DUMP=y
（5）在"Filesystems" -> “Pseudo filesystems"下使能”/proc/vmcore support"。
CONFIG_PROC_VMCORE=y
然后重新編譯內核，生成Image。
四、建立文件系統鏡像
（1）建立IMG文件。
dd if=/dev/zero of=rootfs.img bs=1M count=4k oflag=direct
（2）將IMG格式化為ext4格式。
mkfs.ext4 rootfs.img
（3）建立ubuntu-mount文件夾。
mkdir ubuntu-mount
（4）將IMG與文件夾進行掛載。
mount rootfs.img ubuntu-mount
（5）將rootfs里面的內容復制到IMG中。
cp -rfp rootfs/* ubuntu-mount/
（6）取消IMG和文件夾的掛載。
umount ubuntu-mount
（7）檢查文件系統的正確性。
e2fsck -p -f rootfs.img
五、實驗操作
（1）當我們做完以上的工作后，就可以通過QEMU啟動ARM64的最小系統了。命令如下：
qemu-system-aarch64 -machine virt -M virt,gic_version=3 -cpu cortex-a57 -machine type=virt -nographic -smp 2 -m 1024 -kernel ./Image -hda ./rootfs.img -append “console=ttyAMA0 root=/dev/vda rw crashkernel=128M nr_cpus=2”
以下是我實驗的啟動流程：
root@ubuntu:/home/alvin/share/busybox_arm64# qemu-system-aarch64 -machine virt -M virt,gic_version=3 -cpu cortex-a57 -machine type=virt -nographic -smp 2 -m 1024 -kernel ./Image -hda ./rootfs.img -append “console=ttyAMA0 root=/dev/vda rw crashkernel=128M nr_cpus=2”
WARNING: Image format was not specified for ‘./rootfs.img’ and probing guessed raw.
Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
Specify the ‘raw’ format explicitly to remove the restrictions.
[ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x411fd070]
[ 0.000000] Linux version 5.0.7 (root@ubuntu) (gcc version 7.3.0 (Ubuntu/Linaro 7.3.0-27ubuntu1~18.04)) #12 SMP PREEMPT Sat May 11 15:14:29 PDT 2019
[ 0.000000] Machine model: linux,dummy-virt
[ 0.000000] efi: Getting EFI parameters from FDT:
[ 0.000000] efi: UEFI not found.
[ 0.000000] crashkernel reserved: 0x0000000077e00000 - 0x000000007fe00000 (128 MB)
[ 0.000000] cma: Reserved 32 MiB at 0x0000000075c00000
[ 0.000000] NUMA: No NUMA configuration found
[ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000007fffffff]
[ 0.000000] NUMA: NODE_DATA [mem 0x77df4840-0x77df5fff]
[ 0.000000] Zone ranges:
[ 0.000000] DMA32 [mem 0x0000000040000000-0x000000007fffffff]
[ 0.000000] Normal empty
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000040000000-0x000000007fffffff]
[ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000007fffffff]
[ 0.000000] psci: probing for conduit method from DT.
[ 0.000000] psci: PSCIv0.2 detected in firmware.
[ 0.000000] psci: Using standard PSCI v0.2 function IDs
[ 0.000000] psci: Trusted OS migration not required
[ 0.000000] random: get_random_bytes called from start_kernel+0xa8/0x40c with crng_init=0
[ 0.000000] percpu: Embedded 23 pages/cpu @(ptrval) s56024 r8192 d29992 u94208
[ 0.000000] Detected PIPT I-cache on CPU0
[ 0.000000] CPU features: detected: ARM erratum 832075
[ 0.000000] CPU features: detected: ARM erratum 834220
[ 0.000000] CPU features: detected: EL2 vector hardening
[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 258048
[ 0.000000] Policy zone: DMA32
[ 0.000000] Kernel command line: console=ttyAMA0 root=/dev/vda rw crashkernel=128M nr_cpus=2
[ 0.000000] Memory: 845240K/1048576K available (11004K kernel code, 1604K rwdata, 5332K rodata, 1408K init, 384K bss, 170568K reserved, 32768K cma-reserved)
[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[ 0.000000] rcu: Preemptible hierarchical RCU implementation.
[ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2.
[ 0.000000] Tasks RCU enabled.
[ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
[ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[ 0.000000] GICv3: Distributor has no Range Selector support
[ 0.000000] GICv3: no VLPI support, no direct LPI support
[ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000
[ 0.000000] arch_timer: cp15 timer(s) running at 62.50MHz (virt).
[ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x1cd42e208c, max_idle_ns: 881590405314 ns
[ 0.000174] sched_clock: 56 bits at 62MHz, resolution 16ns, wraps every 4398046511096ns
[ 0.008201] Console: colour dummy device 80x25
[ 0.010926] Calibrating delay loop (skipped), value calculated using timer frequency… 125.00 BogoMIPS (lpj=250000)
[ 0.011187] pid_max: default: 32768 minimum: 301
[ 0.012558] LSM: Security Framework initializing
[ 0.014434] Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes)
[ 0.015704] Inode-cache hash table entries: 65536 (order: 7, 524288 bytes)
[ 0.015999] Mount-cache hash table entries: 2048 (order: 2, 16384 bytes)
[ 0.016112] Mountpoint-cache hash table entries: 2048 (order: 2, 16384 bytes)
[ 0.082658] ASID allocator initialised with 32768 entries
[ 0.091943] rcu: Hierarchical SRCU implementation.
[ 0.107577] EFI services will not be available.
[ 0.116690] smp: Bringing up secondary CPUs …
[ 0.158098] Detected PIPT I-cache on CPU1
[ 0.158751] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000
[ 0.159082] CPU1: Booted secondary processor 0x0000000001 [0x411fd070]
[ 0.163232] smp: Brought up 1 node, 2 CPUs
[ 0.163311] SMP: Total of 2 processors activated.
[ 0.163389] CPU features: detected: GIC system register CPU interface
[ 0.163424] CPU features: detected: 32-bit EL0 Support
[ 0.163520] CPU features: detected: CRC32 instructions
[ 0.171461] CPU: All CPU(s) started at EL1
[ 0.171777] alternatives: patching kernel code
[ 0.200862] devtmpfs: initialized
[ 0.220424] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 0.220680] futex hash table entries: 512 (order: 3, 32768 bytes)
[ 0.228215] pinctrl core: initialized pinctrl subsystem
[ 0.251460] DMI not present or invalid.
[ 0.262311] NET: Registered protocol family 16
[ 0.266512] audit: initializing netlink subsys (disabled)
[ 0.269487] audit: type=2000 audit(0.208:1): state=initialized audit_enabled=0 res=1
[ 0.285140] cpuidle: using governor menu
[ 0.287626] vdso: 2 pages (1 code @ (ptrval), 1 data @ (ptrval))
[ 0.287785] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[ 0.297348] DMA: preallocated 256 KiB pool for atomic allocations
[ 0.313825] Serial: AMBA PL011 UART driver
[ 0.381830] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 39, base_baud = 0) is a PL011 rev1
[ 0.490940] printk: console [ttyAMA0] enabled
[ 0.651007] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[ 0.651932] HugeTLB registered 32.0 MiB page size, pre-allocated 0 pages
[ 0.652686] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[ 0.653521] HugeTLB registered 64.0 KiB page size, pre-allocated 0 pages
[ 0.660698] cryptd: max_cpu_qlen set to 1000
[ 0.676782] ACPI: Interpreter disabled.
[ 0.699521] vgaarb: loaded
[ 0.703209] SCSI subsystem initialized
[ 0.718287] usbcore: registered new interface driver usbfs
[ 0.720782] usbcore: registered new interface driver hub
[ 0.723444] usbcore: registered new device driver usb
[ 0.733534] pps_core: LinuxPPS API ver. 1 registered
[ 0.734813] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti giometti@linux.it
[ 0.736854] PTP clock support registered
[ 0.746659] EDAC MC: Ver: 3.0.0
[ 0.757551] Advanced Linux Sound Architecture Driver Initialized.
[ 0.776621] clocksource: Switched to clocksource arch_sys_counter
[ 0.782977] VFS: Disk quotas dquot_6.6.0
[ 0.784515] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 0.794319] pnp: PnP ACPI: disabled
[ 0.867760] NET: Registered protocol family 2
[ 0.878383] tcp_listen_portaddr_hash hash table entries: 512 (order: 1, 8192 bytes)
[ 0.879454] TCP established hash table entries: 8192 (order: 4, 65536 bytes)
[ 0.880970] TCP bind hash table entries: 8192 (order: 5, 131072 bytes)
[ 0.881576] TCP: Hash tables configured (established 8192 bind 8192)
[ 0.884265] UDP hash table entries: 512 (order: 2, 16384 bytes)
[ 0.885425] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
[ 0.887818] NET: Registered protocol family 1
[ 0.896387] RPC: Registered named UNIX socket transport module.
[ 0.910707] RPC: Registered udp transport module.
[ 0.911119] RPC: Registered tcp transport module.
[ 0.912991] RPC: Registered tcp NFSv4.1 backchannel transport module.
[ 0.923200] hw perfevents: enabled with armv8_pmuv3 PMU driver, 5 counters available
[ 0.925509] kvm [1]: HYP mode not available
[ 0.992384] Initialise system trusted keyrings
[ 0.997978] workingset: timestamp_bits=44 max_order=18 bucket_order=0
[ 1.060250] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[ 1.069144] NFS: Registering the id_resolver key type
[ 1.069994] Key type id_resolver registered
[ 1.070221] Key type id_legacy registered
[ 1.070831] nfs4filelayout_init: NFSv4 File Layout Driver Registering…
[ 1.073270] 9p: Installing v9fs 9p2000 file system support
[ 1.129431] Key type asymmetric registered
[ 1.129886] Asymmetric key parser ‘x509’ registered
[ 1.130961] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 245)
[ 1.131756] io scheduler mq-deadline registered
[ 1.134068] io scheduler kyber registered
[ 1.193670] pl061_gpio 9030000.pl061: PL061 GPIO chip @0x0000000009030000 registered
[ 1.206992] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:
[ 1.208269] pci-host-generic 4010000000.pcie: IO 0x3eff0000…0x3effffff -> 0x00000000
[ 1.210415] pci-host-generic 4010000000.pcie: MEM 0x10000000…0x3efeffff -> 0x10000000
[ 1.211020] pci-host-generic 4010000000.pcie: MEM 0x8000000000…0xffffffffff -> 0x8000000000
[ 1.213354] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]
[ 1.216257] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00
[ 1.218452] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 1.219166] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]
[ 1.219642] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]
[ 1.220051] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]
[ 1.236339] pci 0000:00:01.0: BAR 6: assigned [mem 0x10000000-0x1003ffff pref]
[ 1.238306] pci 0000:00:01.0: BAR 4: assigned [mem 0x8000000000-0x8000003fff 64bit pref]
[ 1.239271] pci 0000:00:02.0: BAR 4: assigned [mem 0x8000004000-0x8000007fff 64bit pref]
[ 1.240016] pci 0000:00:02.0: BAR 0: assigned [io 0x1000-0x107f]
[ 1.241310] pci 0000:00:01.0: BAR 0: assigned [io 0x1080-0x109f]
[ 1.263130] EINJ: ACPI disabled.
[ 1.320069] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)
[ 1.325531] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)
[ 1.372240] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[ 1.391891] SuperH (H)SCI(F) driver initialized
[ 1.396244] msm_serial: driver initialized
[ 1.407937] cacheinfo: Unable to detect cache hierarchy for CPU 0
[ 1.466535] loop: module loaded
[ 1.478452] virtio_blk virtio1: [vda] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB)
[ 1.543860] libphy: Fixed MDIO Bus: probed
[ 1.552503] tun: Universal TUN/TAP device driver, 1.6
[ 1.570756] thunder_xcv, ver 1.0
[ 1.571914] thunder_bgx, ver 1.0
[ 1.573689] nicpf, ver 1.0
[ 1.579220] hclge is initializing
[ 1.581401] hns3: Hisilicon Ethernet Network Driver for Hip08 Family - version
[ 1.582293] hns3: Copyright © 2017 Huawei Corporation.
[ 1.583795] e1000e: Intel® PRO/1000 Network Driver - 3.2.6-k
[ 1.585393] e1000e: Copyright© 1999 - 2015 Intel Corporation.
[ 1.587092] igb: Intel® Gigabit Ethernet Network Driver - version 5.4.0-k
[ 1.589120] igb: Copyright © 2007-2014 Intel Corporation.
[ 1.590132] igbvf: Intel® Gigabit Virtual Function Network Driver - version 2.4.0-k
[ 1.591243] igbvf: Copyright © 2009 - 2012 Intel Corporation.
[ 1.596258] sky2: driver version 1.30
[ 1.601774] VFIO - User Level meta-driver version: 0.3
[ 1.617845] ehci_hcd: USB 2.0 ‘Enhanced’ Host Controller (EHCI) Driver
[ 1.619470] ehci-pci: EHCI PCI platform driver
[ 1.621804] ehci-platform: EHCI generic platform driver
[ 1.623722] ehci-orion: EHCI orion driver
[ 1.625425] ehci-exynos: EHCI EXYNOS driver
[ 1.627358] ohci_hcd: USB 1.1 ‘Open’ Host Controller (OHCI) Driver
[ 1.629208] ohci-pci: OHCI PCI platform driver
[ 1.630863] ohci-platform: OHCI generic platform driver
[ 1.634266] ohci-exynos: OHCI EXYNOS driver
[ 1.640986] usbcore: registered new interface driver usb-storage
[ 1.670598] rtc-pl031 9010000.pl031: registered as rtc0
[ 1.677241] i2c /dev entries driver
[ 1.715681] sdhci: Secure Digital Host Controller Interface driver
[ 1.718305] sdhci: Copyright© Pierre Ossman
[ 1.723334] Synopsys Designware Multimedia Card Interface Driver
[ 1.731635] sdhci-pltfm: SDHCI platform and OF driver helper
[ 1.742333] ledtrig-cpu: registered to indicate activity on CPUs
[ 1.767205] usbcore: registered new interface driver usbhid
[ 1.768038] usbhid: USB HID core driver
[ 1.801497] NET: Registered protocol family 17
[ 1.805294] 9pnet: Installing 9P2000 support
[ 1.806524] Key type dns_resolver registered
[ 1.812974] registered taskstats version 1
[ 1.813598] Loading compiled-in X.509 certificates
[ 1.827961] input: gpio-keys as /devices/platform/gpio-keys/input/input0
[ 1.834872] rtc-pl031 9010000.pl031: setting system clock to 2019-05-14T12:07:42 UTC (1557835662)
[ 1.846911] ALSA device list:
[ 1.847890] No soundcards found.
[ 1.857713] uart-pl011 9000000.pl011: no DMA platform data
[ 1.916602] EXT4-fs (vda): mounted filesystem with ordered data mode. Opts: (null)
[ 1.918270] VFS: Mounted root (ext4 filesystem) on device 254:0.
[ 1.923838] devtmpfs: mounted
[ 2.225180] Freeing unused kernel memory: 1408K
[ 2.227492] Run /sbin/init as init process
Please press Enter to activate this console.
Processing /etc/profile… Done
/ #
在QEMU的啟動參數里面配置crashkernel=128M，通過cat /proc/iomem 可以查看預留內存是否成功。
/ # cat /proc/iomem
09000000-09000fff : pl011@9000000
09000000-09000fff : pl011@9000000
09010000-09010fff : pl031@9010000
09010000-09010fff : rtc-pl031
09030000-09030fff : pl061@9030000
09030000-09030fff : pl061@9030000
10000000-3efeffff : pcie@10000000
10000000-1003ffff : 0000:00:01.0
40000000-7fffffff : System RAM
40080000-4107ffff : Kernel code
41080000-411dffff : reserved
411e0000-413d4fff : Kernel data
48000000-480fffff : reserved
74c00000-77bfffff : reserved
77dbe000-77dbefff : reserved
77dbf000-77decfff : reserved
77def000-77df0fff : reserved
77df1000-77df1fff : reserved
77df2000-77df6fff : reserved
77df7000-77dfefff : reserved
77dff000-77dfffff : reserved
77e00000-7fdfffff : Crash kernel
7fe00000-7fffffff : reserved
4010000000-401fffffff : PCI ECAM
8000000000-ffffffffff : pcie@10000000
8000000000-8000003fff : 0000:00:01.0
8000000000-8000003fff : virtio-pci-modern
8000004000-8000007fff : 0000:00:02.0
8000004000-8000007fff : virtio-pci-modern
Crashkernel的內存選擇可以看以下內核Documentation\kdump\ kdump.txt文件。
Extended crashkernel syntax
While the “crashkernel=size[@offset]” syntax is sufficient for most
configurations, sometimes it’s handy to have the reserved memory dependent
on the value of System RAM – that’s mostly for distributors that pre-setup
the kernel command line to avoid a unbootable system after some memory has
been removed from the machine.
The syntax is:
crashkernel=:[,:,…][@offset]
range=start-[end]
For example:
crashkernel=512M-2G:64M,2G-:128M
- 1
This would mean:
1) if the RAM is smaller than 512M, then don’t reserve anything
(this is the “rescue” case)
2) if the RAM size is between 512M and 2G (exclusive), then reserve 64M
3) if the RAM size is larger than 2G, then reserve 128M
（2）然后輸入以下命令
/ # kexec -p --command-line="console=ttyAMA0 root=/dev/vda rw nr_cpus=2 nr_cpus=1 " /Image
（3）進行觸發崩潰的操作
echo c > /proc/sysrq-trigger
當觸發完成后，會打印崩潰信息，並快速啟動第二個內核。打印流程如下：
/ # echo c > /proc/sysrq-trigger
[ 127.702257] sysrq: SysRq : Trigger a crash
[ 127.703189] Kernel panic - not syncing: sysrq triggered crash
[ 127.704394] CPU: 0 PID: 1477 Comm: sh Kdump: loaded Not tainted 5.0.7 #12
[ 127.705172] Hardware name: linux,dummy-virt (DT)
[ 127.706310] Call trace:
[ 127.706555] dump_backtrace+0x0/0x178
[ 127.707455] show_stack+0x14/0x20
[ 127.708767] dump_stack+0x90/0xb4
[ 127.710017] panic+0x13c/0x2ec
[ 127.711068] sysrq_handle_reboot+0x0/0x18
[ 127.711595] __handle_sysrq+0xa8/0x1a0
[ 127.712287] write_sysrq_trigger+0x64/0x80
[ 127.712856] proc_reg_write+0x60/0xa0
[ 127.713542] __vfs_write+0x30/0x180
[ 127.713927] vfs_write+0xa4/0x1b0
[ 127.714907] ksys_write+0x60/0xd8
[ 127.715649] __arm64_sys_write+0x18/0x20
[ 127.716322] el0_svc_common+0x60/0x100
[ 127.716675] el0_svc_handler+0x2c/0x80
[ 127.717526] el0_svc+0x8/0xc
[ 127.719019] SMP: stopping secondary CPUs
[ 127.721121] Starting crashdump kernel…
[ 127.721999] Bye!
[ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x411fd070]
[ 0.000000] Linux version 5.0.7 (root@ubuntu) (gcc version 7.3.0 (Ubuntu/Linaro 7.3.0-27ubuntu1~18.04)) #12 SMP PREEMPT Sat May 11 15:14:29 PDT 2019
[ 0.000000] Machine model: linux,dummy-virt
[ 0.000000] efi: Getting EFI parameters from FDT:
[ 0.000000] efi: UEFI not found.
[ 0.000000] Reserving 1KB of memory at 0x7fdff000 for elfcorehdr
[ 0.000000] cma: Reserved 32 MiB at 0x000000007dc00000
[ 0.000000] NUMA: No NUMA configuration found
[ 0.000000] NUMA: Faking a node at [mem 0x0000000077e00000-0x000000007fdfffff]
[ 0.000000] NUMA: NODE_DATA [mem 0x7fdb3840-0x7fdb4fff]
[ 0.000000] Zone ranges:
[ 0.000000] DMA32 [mem 0x0000000077e00000-0x000000007fdfffff]
[ 0.000000] Normal empty
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000077e00000-0x000000007fdfffff]
[ 0.000000] Initmem setup node 0 [mem 0x0000000077e00000-0x000000007fdfffff]
[ 0.000000] psci: probing for conduit method from DT.
[ 0.000000] psci: PSCIv0.2 detected in firmware.
[ 0.000000] psci: Using standard PSCI v0.2 function IDs
[ 0.000000] psci: Trusted OS migration not required
[ 0.000000] Number of cores (2) exceeds configured maximum of 1 - clipping
[ 0.000000] random: get_random_bytes called from start_kernel+0xa8/0x40c with crng_init=0
[ 0.000000] percpu: Embedded 23 pages/cpu @(ptrval) s56024 r8192 d29992 u94208
[ 0.000000] Detected PIPT I-cache on CPU0
[ 0.000000] CPU features: detected: ARM erratum 832075
[ 0.000000] CPU features: detected: ARM erratum 834220
[ 0.000000] CPU features: detected: EL2 vector hardening
[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 32256
[ 0.000000] Policy zone: DMA32
[ 0.000000] Kernel command line: console=ttyAMA0 root=/dev/vda rw nr_cpus=2 nr_cpus=1
[ 0.000000] Memory: 61708K/131072K available (11004K kernel code, 1604K rwdata, 5332K rodata, 1408K init, 384K bss, 36596K reserved, 32768K cma-reserved)
[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[ 0.000000] rcu: Preemptible hierarchical RCU implementation.
[ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=1.
[ 0.000000] Tasks RCU enabled.
[ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
[ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[ 0.000000] GICv3: Distributor has no Range Selector support
[ 0.000000] GICv3: no VLPI support, no direct LPI support
[ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000
[ 0.000000] arch_timer: cp15 timer(s) running at 62.50MHz (virt).
[ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x1cd42e208c, max_idle_ns: 881590405314 ns
[ 0.000105] sched_clock: 56 bits at 62MHz, resolution 16ns, wraps every 4398046511096ns
[ 0.008090] Console: colour dummy device 80x25
[ 0.010680] Calibrating delay loop (skipped), value calculated using timer frequency… 125.00 BogoMIPS (lpj=250000)
[ 0.010848] pid_max: default: 32768 minimum: 301
[ 0.012233] LSM: Security Framework initializing
[ 0.012974] Dentry cache hash table entries: 16384 (order: 5, 131072 bytes)
[ 0.013178] Inode-cache hash table entries: 8192 (order: 4, 65536 bytes)
[ 0.013406] Mount-cache hash table entries: 512 (order: 0, 4096 bytes)
[ 0.013453] Mountpoint-cache hash table entries: 512 (order: 0, 4096 bytes)
[ 0.077223] ASID allocator initialised with 32768 entries
[ 0.086200] rcu: Hierarchical SRCU implementation.
[ 0.102112] EFI services will not be available.
[ 0.111226] smp: Bringing up secondary CPUs …
[ 0.111363] smp: Brought up 1 node, 1 CPU
[ 0.111411] SMP: Total of 1 processors activated.
[ 0.111561] CPU features: detected: GIC system register CPU interface
[ 0.111608] CPU features: detected: 32-bit EL0 Support
[ 0.111703] CPU features: detected: CRC32 instructions
[ 0.117106] CPU: All CPU(s) started at EL1
[ 0.117762] alternatives: patching kernel code
[ 0.140863] devtmpfs: initialized
[ 0.162106] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 0.162323] futex hash table entries: 256 (order: 2, 16384 bytes)
[ 0.175144] pinctrl core: initialized pinctrl subsystem
[ 0.219055] DMI not present or invalid.
[ 0.229449] NET: Registered protocol family 16
[ 0.233400] audit: initializing netlink subsys (disabled)
[ 0.237532] audit: type=2000 audit(0.180:1): state=initialized audit_enabled=0 res=1
[ 0.259222] cpuidle: using governor menu
[ 0.266291] vdso: 2 pages (1 code @ (ptrval), 1 data @ (ptrval))
[ 0.267049] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[ 0.277356] DMA: preallocated 256 KiB pool for atomic allocations
[ 0.298623] Serial: AMBA PL011 UART driver
[ 0.395531] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 39, base_baud = 0) is a PL011 rev1
[ 0.491048] printk: console [ttyAMA0] enabled
[ 0.740496] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[ 0.741311] HugeTLB registered 32.0 MiB page size, pre-allocated 0 pages
[ 0.742767] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[ 0.743413] HugeTLB registered 64.0 KiB page size, pre-allocated 0 pages
[ 0.751955] cryptd: max_cpu_qlen set to 1000
[ 0.764345] ACPI: Interpreter disabled.
[ 0.788136] vgaarb: loaded
[ 0.795412] SCSI subsystem initialized
[ 0.811495] usbcore: registered new interface driver usbfs
[ 0.812958] usbcore: registered new interface driver hub
[ 0.814128] usbcore: registered new device driver usb
[ 0.832093] pps_core: LinuxPPS API ver. 1 registered
[ 0.832387] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti giometti@linux.it
[ 0.835593] PTP clock support registered
[ 0.837876] EDAC MC: Ver: 3.0.0
[ 0.863939] Advanced Linux Sound Architecture Driver Initialized.
[ 0.886023] clocksource: Switched to clocksource arch_sys_counter
[ 0.888000] VFS: Disk quotas dquot_6.6.0
[ 0.888922] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 0.897723] pnp: PnP ACPI: disabled
[ 1.009366] NET: Registered protocol family 2
[ 1.019164] tcp_listen_portaddr_hash hash table entries: 256 (order: 0, 4096 bytes)
[ 1.020359] TCP established hash table entries: 1024 (order: 1, 8192 bytes)
[ 1.021504] TCP bind hash table entries: 1024 (order: 2, 16384 bytes)
[ 1.022935] TCP: Hash tables configured (established 1024 bind 1024)
[ 1.025222] UDP hash table entries: 256 (order: 1, 8192 bytes)
[ 1.026914] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
[ 1.029588] NET: Registered protocol family 1
[ 1.050538] RPC: Registered named UNIX socket transport module.
[ 1.052753] RPC: Registered udp transport module.
[ 1.053121] RPC: Registered tcp transport module.
[ 1.053526] RPC: Registered tcp NFSv4.1 backchannel transport module.
[ 1.069153] hw perfevents: enabled with armv8_pmuv3 PMU driver, 5 counters available
[ 1.071736] kvm [1]: HYP mode not available
[ 1.131574] Initialise system trusted keyrings
[ 1.134596] workingset: timestamp_bits=44 max_order=15 bucket_order=0
[ 1.235184] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[ 1.249084] NFS: Registering the id_resolver key type
[ 1.249858] Key type id_resolver registered
[ 1.252719] Key type id_legacy registered
[ 1.253450] nfs4filelayout_init: NFSv4 File Layout Driver Registering…
[ 1.256423] 9p: Installing v9fs 9p2000 file system support
[ 1.303254] Key type asymmetric registered
[ 1.303654] Asymmetric key parser ‘x509’ registered
[ 1.305720] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 245)
[ 1.308168] io scheduler mq-deadline registered
[ 1.309265] io scheduler kyber registered
[ 1.407589] pl061_gpio 9030000.pl061: PL061 GPIO chip @0x0000000009030000 registered
[ 1.427945] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:
[ 1.429773] pci-host-generic 4010000000.pcie: IO 0x3eff0000…0x3effffff -> 0x00000000
[ 1.433178] pci-host-generic 4010000000.pcie: MEM 0x10000000…0x3efeffff -> 0x10000000
[ 1.435046] pci-host-generic 4010000000.pcie: MEM 0x8000000000…0xffffffffff -> 0x8000000000
[ 1.437441] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]
[ 1.441159] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00
[ 1.443696] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 1.444684] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]
[ 1.445111] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]
[ 1.445505] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]
[ 1.477322] pci 0000:00:01.0: BAR 6: assigned [mem 0x10000000-0x1003ffff pref]
[ 1.479077] pci 0000:00:01.0: BAR 4: assigned [mem 0x8000000000-0x8000003fff 64bit pref]
[ 1.480625] pci 0000:00:02.0: BAR 4: assigned [mem 0x8000004000-0x8000007fff 64bit pref]
[ 1.482593] pci 0000:00:02.0: BAR 0: assigned [io 0x1000-0x107f]
[ 1.483426] pci 0000:00:01.0: BAR 0: assigned [io 0x1080-0x109f]
[ 1.513825] EINJ: ACPI disabled.
[ 1.668632] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[ 1.703495] SuperH (H)SCI(F) driver initialized
[ 1.709393] msm_serial: driver initialized
[ 1.721495] cacheinfo: Unable to detect cache hierarchy for CPU 0
[ 1.780444] loop: module loaded
[ 1.791479] virtio_blk virtio1: [vda] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB)
[ 1.886915] libphy: Fixed MDIO Bus: probed
[ 1.893437] tun: Universal TUN/TAP device driver, 1.6
[ 1.923676] thunder_xcv, ver 1.0
[ 1.928028] thunder_bgx, ver 1.0
[ 1.929089] nicpf, ver 1.0
[ 1.935420] hclge is initializing
[ 1.936083] hns3: Hisilicon Ethernet Network Driver for Hip08 Family - version
[ 1.937151] hns3: Copyright © 2017 Huawei Corporation.
[ 1.941014] e1000e: Intel® PRO/1000 Network Driver - 3.2.6-k
[ 1.943745] e1000e: Copyright© 1999 - 2015 Intel Corporation.
[ 1.944904] igb: Intel® Gigabit Ethernet Network Driver - version 5.4.0-k
[ 1.945775] igb: Copyright © 2007-2014 Intel Corporation.
[ 1.948065] igbvf: Intel® Gigabit Virtual Function Network Driver - version 2.4.0-k
[ 1.948854] igbvf: Copyright © 2009 - 2012 Intel Corporation.
[ 1.954323] sky2: driver version 1.30
[ 1.965789] VFIO - User Level meta-driver version: 0.3
[ 1.995451] ehci_hcd: USB 2.0 ‘Enhanced’ Host Controller (EHCI) Driver
[ 1.996575] ehci-pci: EHCI PCI platform driver
[ 2.000118] ehci-platform: EHCI generic platform driver
[ 2.002008] ehci-orion: EHCI orion driver
[ 2.005424] ehci-exynos: EHCI EXYNOS driver
[ 2.022126] ohci_hcd: USB 1.1 ‘Open’ Host Controller (OHCI) Driver
[ 2.023849] ohci-pci: OHCI PCI platform driver
[ 2.026236] ohci-platform: OHCI generic platform driver
[ 2.028953] ohci-exynos: OHCI EXYNOS driver
[ 2.035819] usbcore: registered new interface driver usb-storage
[ 2.077659] rtc-pl031 9010000.pl031: registered as rtc0
[ 2.086105] i2c /dev entries driver
[ 2.147938] sdhci: Secure Digital Host Controller Interface driver
[ 2.148616] sdhci: Copyright© Pierre Ossman
[ 2.155715] Synopsys Designware Multimedia Card Interface Driver
[ 2.168046] sdhci-pltfm: SDHCI platform and OF driver helper
[ 2.186261] ledtrig-cpu: registered to indicate activity on CPUs
[ 2.212538] usbcore: registered new interface driver usbhid
[ 2.226491] usbhid: USB HID core driver
[ 2.264339] NET: Registered protocol family 17
[ 2.269099] 9pnet: Installing 9P2000 support
[ 2.271692] Key type dns_resolver registered
[ 2.278302] registered taskstats version 1
[ 2.280976] Loading compiled-in X.509 certificates
[ 2.299717] input: gpio-keys as /devices/platform/gpio-keys/input/input0
[ 2.306642] rtc-pl031 9010000.pl031: setting system clock to 2019-05-14T12:09:54 UTC (1557835794)
[ 2.312154] ALSA device list:
[ 2.312482] No soundcards found.
[ 2.317908] uart-pl011 9000000.pl011: no DMA platform data
[ 2.397763] EXT4-fs (vda): recovery complete
[ 2.402662] EXT4-fs (vda): mounted filesystem with ordered data mode. Opts: (null)
[ 2.404383] VFS: Mounted root (ext4 filesystem) on device 254:0.
[ 2.409448] devtmpfs: mounted
[ 2.712434] Freeing unused kernel memory: 1408K
[ 2.715067] Run /sbin/init as init process
Please press Enter to activate this console.
Processing /etc/profile… Done
（4）然后將proc目錄下的vmcore文件拷貝至你的存儲設備中即可，vmcore就是第一個內核崩潰時的內存鏡像。
這里提供一個腳本
#!/bin/sh
KDUMP_PATH="/home"
function save_dump_file()
{
coredir="${KDUMP_PATH}/127.0.0.1-date +"%Y-%m-%d-%H:%M""
mkdir -p $coredir
if [ ! -f /sbin/vmcore-dmesg ];then
echo “Skipping saving vmcore-dmesg.txt. File /sbin/vmcore-dmesg is not present”
return;
fi
echo “kdump: saving vmcore-dmesg.txt to $coredir”
/sbin/vmcore-dmesg /proc/vmcore > $coredir/vmcore-dmesg.txt
cp /proc/vmcore $coredir
}
if [ -s /proc/vmcore ];then
echo “save core dump file…”
save_dump_file
reboot
else
# kexec setting
kexec -p --command-line="console=ttyAMA0 root=/dev/vda rw nr_cpus=2 nr_cpus=1 " /Image
fi
該腳本的首先判斷如果存在/proc/vmcore，則轉儲core dump文件到dump目錄，否則就使用kexec命令來加載捕捉內核。可以將以上腳本設置為系統啟動自動運行即可，dump出來的文件會保存在KDUMP_PATH目錄下。
接下來就是通過crash或者gdb工具進行分析。
ubuntu和centos等操作系統比較還提供一個工具叫makedumpfile的工具對vmcore進行壓縮，以減少體積。這個工具我只試了X86_64下的，在arm64下會有點難搞，具體原由可以看其README。