在高並發的大數據 場景下Linux服務器報錯fork: retry:資源暫時不可用
jvm會產生一個hs_err_pid74299.log類似這樣的日志文件
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Cannot create GC thread. Out of system resources.
# Possible reasons:
# The system is out of physical RAM or swap space
# In 32 bit mode, the process size limit was hit
# Possible solutions:
# Reduce memory load on the system
# Increase physical memory or swap space
# Check if swap backing store is full
# Use 64 bit Java on a 64 bit OS
# Decrease Java heap size (-Xmx/-Xms)
# Decrease number of Java threads
# Decrease Java thread stack sizes (-Xss)
# Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
# Out of Memory Error (gcTaskThread.cpp:46), pid=74299, tid=140016377341760
#
# JRE version: (7.0_71-b14) (build )
# Java VM: Java HotSpot(TM) 64-Bit Server VM (24.71-b01 mixed mode linux-amd64 )
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
默認情況下Linux服務起的core core file size設置為0，需要調整該參數，但是這個參數並不能 解決問題；
問題的根本原因在於服務器的運行應用程序的打開文件的最大數及最大進程數設置的相對較小默認為4096
需要修改如下配置：
/etc/security/limits.conf
* soft nofile 327680
* hard nofile 327680
hdfs soft nproc 131072
hdfs hard nproc 131072
mapred soft nproc 131072
mapred hard nproc 131072
hbase soft nproc 131072
hbase hard nproc 131072
zookeeper soft nproc 131072
zookeeper hard nproc 131072
hive soft nproc 131072
hive hard nproc 131072
root soft nproc 131072
root hard nproc 131072
cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.4 (Maipo)
ll /etc/security/limits.d/
總用量 4
-rw-r--r-- 1 root root 228 7月 20 16:08 20-nproc.conf_bak_20190720
問題得到解決。