最近在項目中用到了google的heap profiler工具來分析內存占用，效果非常顯著，因此在這里寫一篇博客記錄一下使用過程中遇到的一些問題。
heap profiler依賴於tcmalloc，所以先要在本機安裝tcmalloc，安裝過程非常的簡單。然后開始使用tcmalloc進行編譯自己寫的程序。
- 生成堆棧快照
先寫一段申請大量內存的代碼：
heap_profiler.cpp
1 #include <iostream> 2 #include <unistd.h> 3 4 int* create(unsigned int size) 5 { 6 return new int[size]; 7 } 8 9 int main() 10 { 11 int count = 10; 12 int* array[count]; 13 14 unsigned int size = 1024 * 1024; 15 for (int i = 0; i < count; ++i) { 16 sleep(1); 17 array[i] = create(10 * size); 18 19 int* b = new int[2 * size]; 20 } 21 22 for (int i = 0; i < count; ++i) { 23 delete[] array[i]; 24 } 25 }
接着進行編譯：
$ g++ heap_profiler.cpp -ltcmalloc -g -o main
然后執行如下命令：
$ HEAPPROFILE=test ./main Starting tracking the heap Dumping heap profile to test.0001.heap (136 MB currently in use) Dumping heap profile to test.0002.heap (240 MB currently in use) Dumping heap profile to test.0003.heap (376 MB currently in use) Dumping heap profile to test.0004.heap (480 MB currently in use) Dumping heap profile to test.0005.heap (Exiting, 80 MB in use) $ ls heap_profiler.cpp main test.0001.heap test.0002.heap test.0003.heap test.0004.heap test.0005.heap
可以看到，這里生成了幾個.heap文件，並且知道程序退出時，還有80M的內存在未釋放。使用pprof命令即可對這些文件進行分析。
這里需要特別注意一點，筆者之前因為項目本身用的tcmalloc是采用靜態鏈接方式，即如下所示，編譯時，靜態鏈接了static_lib下的libtcmalloc.a：
$ ls -lh total 8.0K -rw-rw-r-- 1 minglee minglee 279 Dec 15 16:42 heap_profiler.cpp drwxrwxr-x 2 minglee minglee 4.0K Dec 15 17:06 static_lib $ ls -lh static_lib/ total 5.7M -rw-rw-r-- 1 minglee minglee 5.6M Dec 15 17:03 libtcmalloc.a $ g++ heap_profiler.cpp -ltcmalloc -g -o main -Lstatic_lib/ -lpthread static_lib//libtcmalloc.a(stacktrace.o): In function `GetStackTraceWithContext_libunwind(void**, int, int, void const*)': /home/minglee/install_packet/gperftools-2.7/src/stacktrace_libunwind-inl.h:112: undefined reference to `_Ux86_64_getcontext' /home/minglee/install_packet/gperftools-2.7/src/stacktrace_libunwind-inl.h:116: undefined reference to `_ULx86_64_init_local' /home/minglee/install_packet/gperftools-2.7/src/stacktrace_libunwind-inl.h:120: undefined reference to `_ULx86_64_step' /home/minglee/install_packet/gperftools-2.7/src/stacktrace_libunwind-inl.h:138: undefined reference to `_ULx86_64_step' /home/minglee/install_packet/gperftools-2.7/src/stacktrace_libunwind-inl.h:131: undefined reference to `_ULx86_64_get_reg' static_lib//libtcmalloc.a(stacktrace.o): In function `GetStackTrace_libunwind(void**, int, int)': /home/minglee/install_packet/gperftools-2.7/src/stacktrace_libunwind-inl.h:112: undefined reference to `_Ux86_64_getcontext' /home/minglee/install_packet/gperftools-2.7/src/stacktrace_libunwind-inl.h:116: undefined reference to `_ULx86_64_init_local' /home/minglee/install_packet/gperftools-2.7/src/stacktrace_libunwind-inl.h:120: undefined reference to `_ULx86_64_step' /home/minglee/install_packet/gperftools-2.7/src/stacktrace_libunwind-inl.h:138: undefined reference to `_ULx86_64_step' /home/minglee/install_packet/gperftools-2.7/src/stacktrace_libunwind-inl.h:131: undefined reference to `_ULx86_64_get_reg' static_lib//libtcmalloc.a(stacktrace.o): In function `GetStackFramesWithContext_libunwind(void**, int*, int, int, void const*)': /home/minglee/install_packet/gperftools-2.7/src/stacktrace_libunwind-inl.h:112: undefined reference to `_Ux86_64_getcontext' /home/minglee/install_packet/gperftools-2.7/src/stacktrace_libunwind-inl.h:116: undefined reference to `_ULx86_64_init_local' /home/minglee/install_packet/gperftools-2.7/src/stacktrace_libunwind-inl.h:120: undefined reference to `_ULx86_64_step' /home/minglee/install_packet/gperftools-2.7/src/stacktrace_libunwind-inl.h:124: undefined reference to `_ULx86_64_get_reg' /home/minglee/install_packet/gperftools-2.7/src/stacktrace_libunwind-inl.h:138: undefined reference to `_ULx86_64_step' /home/minglee/install_packet/gperftools-2.7/src/stacktrace_libunwind-inl.h:143: undefined reference to `_ULx86_64_get_reg' /home/minglee/install_packet/gperftools-2.7/src/stacktrace_libunwind-inl.h:131: undefined reference to `_ULx86_64_get_reg' static_lib//libtcmalloc.a(stacktrace.o): In function `GetStackFrames_libunwind(void**, int*, int, int)': /home/minglee/install_packet/gperftools-2.7/src/stacktrace_libunwind-inl.h:112: undefined reference to `_Ux86_64_getcontext' /home/minglee/install_packet/gperftools-2.7/src/stacktrace_libunwind-inl.h:116: undefined reference to `_ULx86_64_init_local' /home/minglee/install_packet/gperftools-2.7/src/stacktrace_libunwind-inl.h:120: undefined reference to `_ULx86_64_step' /home/minglee/install_packet/gperftools-2.7/src/stacktrace_libunwind-inl.h:124: undefined reference to `_ULx86_64_get_reg' /home/minglee/install_packet/gperftools-2.7/src/stacktrace_libunwind-inl.h:138: undefined reference to `_ULx86_64_step' /home/minglee/install_packet/gperftools-2.7/src/stacktrace_libunwind-inl.h:143: undefined reference to `_ULx86_64_get_reg' /home/minglee/install_packet/gperftools-2.7/src/stacktrace_libunwind-inl.h:131: undefined reference to `_ULx86_64_get_reg' collect2: error: ld returned 1 exit status $ g++ heap_profiler.cpp -ltcmalloc -g -o main -Lstatic_lib/ -lpthread -lunwind $ HEAPPROFILE=test ./main $
導致執行的時候沒有任何的反應，也不會出現 “Starting tracking the heap” 提示，更不會生成 .heap 文件。所以切記使用heap_profiler的時候需要使用動態鏈接，如果不想使用動態鏈接，也可以通過加代碼的方式去生成.heap文件：
1 #include <iostream> 2 #include <unistd.h> 3 #include <gperftools/heap-profiler.h> 4 5 int* create(unsigned int size) 6 { 7 return new int[size]; 8 } 9 10 int main() 11 { 12 HeapProfilerStart("test"); 13 int count = 10; 14 int* array[count]; 15 16 unsigned int size = 1024 * 1024 * 10; 17 for (int i = 0; i < count; ++i) { 18 sleep(1); 19 array[i] = create(size); 20 21 int* b = new int[2 * size]; 22 } 23 24 for (int i = 0; i < count; ++i) { 25 delete[] array[i]; 26 } 27 HeapProfilerStop(); 28 }
注意 第12行 和 第27行 增加的兩個函數，HeapProfilerStart() 和 HeapProfilerStop()（頭文件在<gperftools/heap-profiler.h>中），分別用來開啟和關閉堆棧分析器，HeapProfilerStart() 需要一個參數，這個參數就是.heap文件（也就是堆棧快照）的前綴。這個前綴也可以通過環境變量 HEAPPROFILE 來設置。這也編譯出來的代碼，直接執行，也可以生產.heap文件：
$ g++ heap_profiler.cpp -ltcmalloc -g -o main -Lstatic_lib/ -lpthread -lunwind $ ./main Starting tracking the heap Dumping heap profile to test.0001.heap (136 MB currently in use) Dumping heap profile to test.0002.heap (240 MB currently in use) Dumping heap profile to test.0003.heap (376 MB currently in use) Dumping heap profile to test.0004.heap (480 MB currently in use)
可以看到，使用動態編譯的方式libtcmalloc的方式來使用heap profiler能顯示出更多的信息，比如程序退出時是否有未釋放的內存。以下的分析階段都是采用動態編譯的方式進行。
- 使用pprof命令進行分析：
$ HEAPPROFILE=test ./main Starting tracking the heap Dumping heap profile to test.0001.heap (136 MB currently in use) Dumping heap profile to test.0002.heap (240 MB currently in use) Dumping heap profile to test.0003.heap (376 MB currently in use) Dumping heap profile to test.0004.heap (480 MB currently in use) Dumping heap profile to test.0005.heap (Exiting, 80 MB in use)
$ pprof --text main test.0004.heap Using local file main. Using local file test.0004.heap. Total: 480.0 MB 400.0 83.3% 83.3% 400.0 83.3% create 80.0 16.7% 100.0% 480.0 100.0% main 0.0 0.0% 100.0% 480.0 100.0% __libc_start_main
可以很清晰的看到內存分配的函數以及分配的內存總量。各列含義的解讀：
- 第一列包含直接占用的內存
- 第四列包含自身和所有被調用的函數占用的內存
- 第二列和第五列僅僅是第一列和第四列數字的百分比表示
- 第三列是第二列從第一行到當前行的累加值。（比如：二行三列= 一行二列 + 二行二列； 三行三列 = 一行二列 + 二行二列 + 三行二列）
另外還可以加上--stack選項（與--text同時使用）：
$ pprof --text --stack main test.0004.heap Using local file main. Using local file test.0004.heap. Total: 480.0 MB Stacks: 83886080 (00000000004009aa) /home/minglee/workspace/test_code/heap_profiler/heap_profiler.cpp:19:main (00007f9644cb3c04) ??:0:__libc_start_main 419430400 (00000000004008b8) /home/minglee/workspace/test_code/heap_profiler/heap_profiler.cpp:7:create (000000000040096d) /home/minglee/workspace/test_code/heap_profiler/heap_profiler.cpp:17:main (00007f9644cb3c04) ??:0:__libc_start_main Leak of 419430400 bytes in 10 objects allocated from: @ 004008b8 unknown @ 000000000040096d main /home/minglee/workspace/test_code/heap_profiler/heap_profiler.cpp:17 @ 00007f9644cb3c04 __libc_start_main ??:0 Leak of 83886080 bytes in 10 objects allocated from: @ 004009aa unknown @ 00007f9644cb3c04 __libc_start_main ??:0 400.0 83.3% 83.3% 400.0 83.3% create 80.0 16.7% 100.0% 480.0 100.0% main 0.0 0.0% 100.0% 480.0 100.0% __libc_start_main
--text的選項，在查看簡單的程序時還是不錯的，但是面對復雜的程序時，就顯得心有余力不足了。這個時候可以使用--gv選項：
$ pprof --gv main test.0004.heap Using local file main. Using local file test.0004.heap. Dropping nodes with <= 2.4 MB; edges with <= 0.5 abs(MB) sh: dot: command not found
這里報錯是因為--gv選項需要安裝 graphviz 和 gv：
$ sudo yum install graphviz gv
安裝完之后如果報出如下錯誤：
$ pprof --gv main test.0004.heap Using local file main. Using local file test.0004.heap. Dropping nodes with <= 2.4 MB; edges with <= 0.5 abs(MB) gv: Unable to open the display.
說明無法打開顯示器，也就是說，--gv選項，需要在帶圖形界面的系統上使用。轉到圖形界面系統上做分析，可以得到下圖：
可以看到main函數占用了80M內存，占所有占用內存的16.7%，main直接或間接占用了內存480M，占所有未釋放內存的100%，下面的create函數占用內存400M，占所有未釋放內存的83.3%。顯示的結果非常清晰明了，能夠清晰的定位到問題。
此外，為了生成明確的堆棧，編譯優化建議不要開，O0就好，最好再加上編譯選項 -fno-omit-frame-pointer 這樣能更好的顯示出完整堆棧，定位起問題來會更加的輕松。