Docker無法正常啟動的原因及解決辦法

一、Docker啟動異常表現：

1.狀態反復restaring，用命令查看

$docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 21c09be88c11 docker.xxxx.cn:5000/xxx-tes/xxx_tes:1.0.6 "/usr/local/tomcat..." 9 days ago Restarting (1) Less than a second ago xxx10

2.Docker日志有明顯問題：

       $docker logs [容器名/容器ID]

二、Docker啟動異常的可能原因：

2.1.內存不夠 
Docker 啟動至少需要2G內存，首先執行free -mh命令查看剩余內存是否足夠

直接查看內存

$free -mh total used free shared buff/cache available Mem: 15G 14G 627M 195M 636M 726M Swap: 0B 0B 0B

分析日志

  有時候一瞬間內存過載溢出，導致部分進程被殺死，看起來內存也是夠用的，事實上docker還是會反復重啟，就需要通過docker日志和系統日志信的息來進一步分析：

分析docker日志

查看docker日志看到內存溢出的信息，要仔細翻閱才能找到信息，並不是在最下面

$docker logs [容器名/容器ID]|less 
Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000769990000, 1449590784, 0) failed; error='Cannot allocate memory' (errno=12) # # There is insufficient memory for the Java Runtime Environment to continue. # Native memory allocation (malloc) failed to allocate 1449590784 bytes for committing reserved memory. # An error report file with more information is saved as: # //hs_err_pid1.log Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000769990000, 1449590784, 0) failed; error='Cannot allocate memory' (errno=12) # # There is insufficient memory for the Java Runtime Environment to continue. # Native memory allocation (malloc) failed to allocate 1449590784 bytes for committing reserved memory. # An error report file with more information is saved as: # /tmp/hs_err_pid1.log Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000769990000, 1449590784, 0) failed; error='Cannot allocate memory' (errno=12) # # There is insufficient memory for the Java Runtime Environment to continue. # Native memory allocation (malloc) failed to allocate 1449590784 bytes for committing reserved memory. # Can not save log file, dump to screen.. # # There is insufficient memory for the Java Runtime Environment to continue. # Native memory allocation (malloc) failed to allocate 1449590784 bytes for committing reserved memory. # Possible reasons: # The system is out of physical RAM or swap space # In 32 bit mode, the process size limit was hit # Possible solutions: # Reduce memory load on the system # Increase physical memory or swap space # Check if swap backing store is full # Use 64 bit Java on a 64 bit OS # Decrease Java heap size (-Xmx/-Xms) # Decrease number of Java threads # Decrease Java thread stack sizes (-Xss) # Set larger code cache with -XX:ReservedCodeCacheSize= # This output file may be truncated or incomplete. # # Out of Memory Error (os_linux.cpp:2756), pid=1, tid=140325689620224 # # JRE version: (7.0_79-b15) (build ) # Java VM: Java HotSpot(TM) 64-Bit Server VM (24.79-b02 mixed mode linux-amd64 compressed oops) # Core dump written. Default location: //core or core.1

分析系統日志

查看系統日志，發現有大量由於內存溢出，進程被殺死的記錄

$grep -i 'Out of Memory' /var/log/messages Apr 7 10:04:02 centos106 kernel: Out of memory: Kill process 1192 (java) score 54 or sacrifice child Apr 7 10:08:00 centos106 kernel: Out of memory: Kill process 2301 (java) score 54 or sacrifice child Apr 7 10:09:59 centos106 kernel: Out of memory: Kill process 28145 (java) score 52 or sacrifice child Apr 7 10:20:40 centos106 kernel: Out of memory: Kill process 2976 (java) score 54 or sacrifice child Apr 7 10:21:08 centos106 kernel: Out of memory: Kill process 3577 (java) score 47 or sacrifice child Apr 7 10:21:08 centos106 kernel: Out of memory: Kill process 3631 (java) score 47 or sacrifice child Apr 7 10:21:08 centos106 kernel: Out of memory: Kill process 3634 (java) score 47 or sacrifice child Apr 7 10:21:08 centos106 kernel: Out of memory: Kill process 3640 (java) score 47 or sacrifice child Apr 7 10:21:08 centos106 kernel: Out of memory: Kill process 3654 (java) score 47 or sacrifice child Apr 7 10:27:27 centos106 kernel: Out of memory: Kill process 6998 (java) score 51 or sacrifice child Apr 7 10:27:28 centos106 kernel: Out of memory: Kill process 7027 (java) score 52 or sacrifice child Apr 7 10:28:10 centos106 kernel: Out of memory: Kill process 7571 (java) score 42 or sacrifice child Apr 7 10:28:10 centos106 kernel: Out of memory: Kill process 7586 (java) score 42 or sacrifice chil

2.2.端口沖突

該docker監聽端口已經被其他進程占用，一般此種問題容易出現在新部署的服務，或在原有機器上部署新的后台服務，所以在部署之前應該執行命令檢查端口是否已經被占用，如果上線后發現占有則應改為可用端口再重啟之。

檢查命令：    $netstat -nltp|grep [規划的端口號]

三、對策

3.1.內存不夠的對策：

對策1:

    3.1.1 saltstack的minion在運行過久之后，可能占用大量內存，需要將其重啟。重啟命令可能有時並不起作用。主要檢查運行狀態，如果未成功停止，則重新重啟；

對策2:

    3.2.2 ELK日志收集程序或者其他java進程占用過高，用top和ps命令排查，謹慎確定進程的作用，在確保不影響業務的情況下，停止相關進程；

對策3:

   釋放被占用的內存（buff/cache）：
   $sync  #將內存數據寫入磁盤
   $echo 3 > /proc/sys/vm/drop_caches  #釋放被占用的內存

對策4:

    有時候並不是buff/cache過高導致內存不夠用，確實是被很多必要的進程消耗掉了內存，那就需要從機器資源分配使用的層面去考慮和解決了。

3.2 端口沖突的對策

對策1:

 一般此種問題容易出現在新部署的服務，或在原有機器上部署新的后台服務，所以在部署之前應該執行命令檢查端口是否已經被占用，如果上線后發現占有則應改為可用端口再重啟之。
檢查命令：    $netstat -nltp|grep [規划的端口號]

來源：https://blog.csdn.net/u010716706/article/details/69524863