垃圾回收G1日志解析

G1垃圾回收器介紹

首先要搞清楚G1垃圾回收的過程

主要分為下面4個步驟：

初始標記（Initial Marking）           由於常規標記對象是否直接關聯到GC roots，過程簡單，停頓時間短

並發標記（Concurrent Marking） 由於可達性性分析，存在較長停頓

最終標記（Final Marking）           由於把並發標記時漏掉的那部分垃圾內存標記起來，比較少，停頓時間短

篩選回收（Live Data Couting and Evacuation）  對標記后的垃圾內存進行整理，按指定計划進行回收。

 

在下面的案例中，紅色部分明確標明了這4個階段。

 

 

 G1垃圾收集器參數說明

以我的spark參數為例：

spark.executor.extraJavaOptions -Dtag=mobius.optimize -XX:+UseG1GC -XX:+PrintFlagsFinal -XX:+PrintReferenceGC -XX:+PrintGCDetails

-XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps-XX:+PrintAdaptiveSizePolicy -XX:+PrintGC -Xloggc:/tmp/spark.gc.log

-XX:+UnlockDiagnosticVMOptions -XX:+G1SummarizeConcMark -XX:InitiatingHeapOccupancyPercent=35 -XX:ConcGCThreads=15

-XX:+PrintGCApplicationConcurrentTime -XX:-UseGCOverheadLimit

 

下面是其拆分后的注釋：

spark.executor.extraJavaOptions \    傳入java參數給spark各個excutor

-Dtag=mobius.optimize \　　　　　　標記名

-XX:+UseG1GC \　　　　　　　　　指定垃圾收集器

-XX:+PrintFlagsFinal \　　　　　　　java對象引用分為四種：強引用FinalReference，軟引用SoftReference，弱引用WeakReference，虛引用PhantomReference。一般都是強引用，比如new一個對象並賦值

-XX:+PrintReferenceGC \　　　　　打印垃圾對象的引用數量

-XX:+PrintGCDetails \　　　　　　　 打印詳細GC日志

-XX:+PrintGCDateStamps \　　　　 打印可讀的日期時間戳 比如2018-11-12

-XX:+PrintGCTimeStamps \　　　　打印可讀的時間戳 比如 09:10:11

-XX:+PrintAdaptiveSizePolicy \         啟用自適應大小（為了避免fullGC，允許改變eden空間和survivor空間大小）

-XX:+PrintGC \　　　　　　　　　　打印GC日志

-Xloggc:/tmp/spark.gc.log \　　　　   指定GC日志路徑

-XX:+UnlockDiagnosticVMOptions \

-XX:+G1SummarizeConcMark \         

-XX:InitiatingHeapOccupancyPercent=35 \

-XX:ParallelGCThreads=33 \　　　　　　 Parallel和G1收集器的標記階段的線程數l量，默認33，對應GC日志 [ GC Workers: 33 ]（並行：多個垃圾收集線程並行工作，此時用戶線程處於等待狀態，stop the world）

-XX:ConcGCThreads=15 \　　　　　　     CMS和G1回收器的前三個標記階段都是多線程的，指定GC並發線程數量，默認0（並發：用戶線程與垃圾收集線程同時執行，不一定是並行，可能會交替執行）

-XX:+PrintGCApplicationConcurrentTime \         打印應用程序執行時間

-XX:-UseGCOverheadLimit \　　　　　       　  取消垃圾回收時的內存限制（減號）

-XX:MaxGCPauseMillis=200　　　　  　　　　垃圾回收停頓最長時間，默認是200ms

 

 

日志分析案例

下面一段日志是我在spark計算任務中產生的垃圾回收日志：

2018-11-13T05:32:44.473+0800: 1.857: Application time: 0.2130324 seconds
2018-11-13T05:32:44.571+0800: 1.954: Application time: 0.0970449 seconds
2018-11-13T05:32:44.696+0800: 2.079: Application time: 0.1249397 seconds
2018-11-13T05:32:44.874+0800: 2.258: Application time: 0.1776871 seconds
 2.258: [G1Ergonomics (Concurrent Cycles) request concurrent cycle initiation, reason: requested by GC cause, GC cause: Metadata GC Threshold]
 2.258: [G1Ergonomics (Concurrent Cycles) initiate concurrent cycle, reason: concurrent cycle initiation requested]
2018-11-13T05:32:44.875+0800: 2.259: [GC pause (Metadata GC Threshold) (young) (initial-mark) 2.260: 
[G1Ergonomics (CSet Construction) start choosing CSet, _pending_cards: 0, predicted base time: 10.00 ms, 
remaining time: 190.00 ms, target pause time: 200.00 ms]
 2.260: [G1Ergonomics (CSet Construction) add young regions to CSet, eden: 60 regions, survivors: 0 regions, predicted young region time: 3605.71 ms]
 2.260: [G1Ergonomics (CSet Construction) finish choosing CSet, eden: 60 regions, survivors: 0 regions, old: 0 regions, 
predicted pause time: 3615.71 ms, target pause time: 200.00 ms]
2018-11-13T05:32:44.948+0800: 2.331: [SoftReference, 0 refs, 0.0002103 secs]2018-11-13T05:32:44.948+0800: 2.332: 
[WeakReference, 970 refs, 0.0005817 secs]2018-11-13T05:32:44.949+0800: 2.332: [FinalReference, 4265 refs, 0.0266394 secs]
2018-11-13T05:32:44.975+0800: 2.359: [PhantomReference, 0 refs, 3 refs, 0.0000585 secs]2018-11-13T05:32:44.976+0800: 2.359: 
[JNI Weak Reference, 0.0001374 secs], 0.1040572 secs]
   [Parallel Time: 29.1 ms, GC Workers: 33]
      [GC Worker Start (ms): Min: 2260.1, Avg: 2265.0, Max: 2265.9, Diff: 5.8]
      [Ext Root Scanning (ms): Min: 0.2, Avg: 18.2, Max: 22.7, Diff: 22.5, Sum: 691.2]
      [Update RS (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
         [Processed Buffers: Min: 0, Avg: 0.0, Max: 0, Diff: 0, Sum: 0]
      [Scan RS (ms): Min: 0.0, Avg: 0.0, Max: 0.1, Diff: 0.1, Sum: 0.3]
      [Code Root Scanning (ms): Min: 0.0, Avg: 0.3, Max: 8.6, Diff: 8.6, Sum: 10.0]
      [Object Copy (ms): Min: 0.0, Avg: 3.9, Max: 22.9, Diff: 22.9, Sum: 147.1]
      [Termination (ms): Min: 0.0, Avg: 1.3, Max: 2.3, Diff: 2.3, Sum: 50.3]
         [Termination Attempts: Min: 1, Avg: 3.0, Max: 17, Diff: 16, Sum: 115]
      [GC Worker Other (ms): Min: 0.0, Avg: 0.0, Max: 0.1, Diff: 0.1, Sum: 1.6]
      [GC Worker Total (ms): Min: 22.7, Avg: 23.7, Max: 28.6, Diff: 5.9, Sum: 900.6]
      [GC Worker End (ms): Min: 2288.6, Avg: 2288.7, Max: 2288.8, Diff: 0.2]
   [Code Root Fixup: 1.0 ms]
   [Code Root Purge: 0.1 ms]
   [Clear CT: 1.0 ms]
   [Other: 72.8 ms]
      [Choose CSet: 0.0 ms]
      [Ref Proc: 68.9 ms]
      [Ref Enq: 0.2 ms]
      [Redirty Cards: 0.4 ms]
      [Humongous Register: 0.1 ms]
      [Humongous Reclaim: 0.1 ms]
      [Free CSet: 0.5 ms]
   [Eden: 240.0M(512.0M)->0.0B(496.0M) Survivors: 0.0B->16.0M Heap: 254.0M(10.0G)->31.7M(10.0G)]
 [Times: user=0.44 sys=0.09, real=0.11 secs]
2018-11-13T05:32:44.980+0800: 2.363: [GC concurrent-root-region-scan-start]
2018-11-13T05:32:44.996+0800: 2.380: [GC concurrent-root-region-scan-end, 0.0167701 secs]
2018-11-13T05:32:44.996+0800: 2.380: [GC concurrent-mark-start]
2018-11-13T05:32:45.013+0800: 2.396: [GC concurrent-mark-end, 0.0160995 secs]
2018-11-13T05:32:45.013+0800: 2.396: Application time: 0.0328728 seconds
2018-11-13T05:32:45.013+0800: 2.397: [GC remark 2018-11-13T05:32:45.013+0800: 2.397: 
[Finalize Marking, 0.0331140 secs] 2018-11-13T05:32:45.047+0800: 2.430: 
[GC ref-proc2018-11-13T05:32:45.047+0800: 2.430: 
[SoftReference, 0 refs, 0.0001959 secs]2018-11-13T05:32:45.047+0800: 2.430: 
[WeakReference, 0 refs, 0.0001148 secs]2018-11-13T05:32:45.047+0800: 2.430: 
[FinalReference, 0 refs, 0.0001158 secs]2018-11-13T05:32:45.047+0800: 2.430: 
[PhantomReference, 0 refs, 0 refs, 0.0002233 secs]2018-11-13T05:32:45.047+0800: 2.431: 
[JNI Weak Reference, 0.0000572 secs], 0.0008127 secs] 2018-11-13T05:32:45.047+0800: 2.431: [Unloading, 0.0078450 secs], 0.0442183 secs]
 [Times: user=0.36 sys=0.01, real=0.05 secs]
2018-11-13T05:32:45.058+0800: 2.441: Application time: 0.0001231 seconds
2018-11-13T05:32:45.058+0800: 2.442: [GC cleanup 35M->35M(10G), 0.0134109 secs]
 [Times: user=0.04 sys=0.00, real=0.01 secs]
2018-11-13T05:32:45.144+0800: 2.528: Application time: 0.0721037 seconds
2018-11-13T05:32:45.392+0800: 2.775: Application time: 0.2464978 seconds
2018-11-13T05:32:45.805+0800: 3.188: Application time: 0.4129881 seconds
2018-11-13T05:32:46.806+0800: 4.190: Application time: 1.0001058 seconds

下面是每條日志含義解析：
Application time： 應用程序時間
。。。