報錯信息:
Maximum number of threads (200) created for connector with address null and port 9443 # There is insufficient memory for the Java Runtime Environment to continue. # Cannot create GC thread. Out of system resources.
一、查看當前用戶線程和文件句柄數是否超出限制
(1)顯示當前用戶進程限制:ulimit -a顯示結果:
core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 256612 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 102400 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 10240 cpu time (seconds, -t) unlimited max user processes (-u) 1024 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited(2)改動全部 linux 用戶的環境變量文件:
vi /etc/profile ulimit -u 10000 ulimit -n 4096保存后執行#source /etc/profile 使其生效
二、查看當前port號進程信息和GC使用情況
(1)顯示port的PID:lsof -i:port演示樣例:lsof -i:7074
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME java 3195 ligang 34u IPv4 37416693 0t0 TCP *:7074 (LISTEN)(2)gc信息統計:jstat -gcutil PID
演示樣例:jstat -gcutil 3195
S0 S1 E O P YGC YGCT FGC FGCT GCT 12.63 0.00 52.03 78.63 99.13 4148 24.274 200 40.246 64.520(3)輸出線程數:ps -mp PID -o THREAD,tid,time | wc -l
演示樣例:ps -mp 3195 -o THREAD,tid,time | wc -l
43
三、查看進程內存使用情況及定位到相應程序
(1)內存使用情況:top -p PID演示樣例:top 3195
top - 15:29:27 up 25 days, 20:05, 2 users, load average: 0.01, 0.05, 0.01 Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie Cpu(s): 0.0%us, 0.1%sy, 0.0%ni, 99.8%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 8058868k total, 6821684k used, 1237184k free, 181936k buffers Swap: 2097144k total, 492300k used, 1604844k free, 1897320k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 3195 ligang 20 0 4862m 196m 10m S 0.0 2.5 7:57.48 java(2)找到該進程后。怎樣定位詳細線程或代碼呢,首先顯示線程列表,並依照CPU占用高的線程排序:
ps -mp PID -o THREAD,tid,time | sort -rn | head -10
演示樣例:ps -mp PID -o THREAD,tid,time | sort -rn | head -10
USER %CPU PRI SCNT WCHAN USER SYSTEM TID TIME ligang 0.6 - - - - - - 00:07:58 ligang 0.2 19 - futex_ - - 3270 00:02:49 ligang 0.0 19 - inet_c - - 3277 00:00:00 ligang 0.0 19 - inet_c - - 3273 00:00:00 ligang 0.0 19 - inet_c - - 3271 00:00:00 ligang 0.0 19 - inet_c - - 3203 00:00:05 ligang 0.0 19 - futex_ - - 7644 00:00:00 ligang 0.0 19 - futex_ - - 3420 00:00:00 ligang 0.0 19 - futex_ - - 3288 00:00:06(3)將須要的線程ID轉換為16進制格式:printf "%x\n" TID
演示樣例:printf "%x\n" 3270
cc6
(4)最后打印線程的堆棧信息:jstack PID |grep cc6 -A 30
演示樣例:
jstack 2633 |grep e18 -A 30
結果就能夠看到哪段代碼導致的問題...