環境
kibana6.4.3
logstash6.4.3
es6.4.3
日志收集服務器內存8g
前景
elk 服務器搭建好的前幾天服務都正常
大概一周以后出現kibana 連接不上，登錄服務器發現只有 kibana 掛掉，es 和 logstash 都正常運行，排查日志發現：
<--- Last few GCs ---> [7232:0x28c9b10] 1349425042 ms: Mark-sweep 1360.5 (1431.3) -> 1360.5 (1431.3) MB, 1475.4 / 0.0 ms allocation failure GC in old space requested [7232:0x28c9b10] 1349426671 ms: Mark-sweep 1360.5 (1431.3) -> 1360.5 (1431.3) MB, 1628.7 / 0.1 ms last resort GC in old space requested [7232:0x28c9b10] 1349428162 ms: Mark-sweep 1360.5 (1431.3) -> 1360.5 (1431.3) MB, 1490.7 / 0.0 ms last resort GC in old space requested <--- JS stacktrace ---> ==== JS stack trace ========================================= Security context: 0x2e54d3625879 <JSObject> 2: /* anonymous */(aka /* anonymous */) [/opt/kibana-6.4.3/src/server/status/lib/cgroup.js:70] [bytecode=0x1d79566d9321 offset=20](this=0x754e6a822d1 <undefined>,controller=0x31e957539041 <String[4]: pids>) 3: arguments adaptor frame: 3->1 4: forEach(this=0x31e957539061 <JSArray[1]>) 5: /* anonymous */(aka /* anonymous */) [/opt/kibana-6.4.3/src/server/status/lib/cgroup.js:69] [b... FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory 1: node::Abort() [./../node/bin/node] 2: 0x8c21ec [./../node/bin/node] 3: v8::Utils::ReportOOMFailure(char const*, bool) [./../node/bin/node] 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [./../node/bin/node] 5: v8::internal::Factory::NewFixedArray(int, v8::internal::PretenureFlag) [./../node/bin/node] 6: v8::internal::HashTable<v8::internal::StringTable, v8::internal::StringTableShape>::NewInternal(v8::internal::Isolate*, int, v8::internal::PretenureFlag) [./../node/bin/node] 7: v8::internal::HashTable<v8::internal::StringTable, v8::internal::StringTableShape>::New(v8::internal::Isolate*, int, v8::internal::PretenureFlag, v8::internal::MinimumCapacity) [./../node/bin/node] 8: v8::internal::HashTable<v8::internal::StringTable, v8::internal::StringTableShape>::EnsureCapacity(v8::internal::Handle<v8::internal::StringTable>, int, v8::internal::PretenureFlag) [./../node/bin/node] 9: v8::internal::StringTable::LookupString(v8::internal::Isolate*, v8::internal::Handle<v8::internal::String>) [./../node/bin/node] 10: v8::internal::LookupIterator::PropertyOrElement(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, bool*, v8::internal::LookupIterator::Configuration) [./../node/bin/node] 11: v8::internal::Runtime_SetProperty(int, v8::internal::Object**, v8::internal::Isolate*) [./../node/bin/node] 12: 0x22a4d09842fd
發現是前端 js 報的內存 oom 異常，通過網上資料發現node.js 的默認內存大小為1.4g
Node 中通過 JavaScript 使用內存時只能使用部分內存（64位系統下約為1.4 GB，32位系統下約為0.7 GB），這就是我們編譯項目時為什么會出現內存泄露了，因為前端項目如果非常的龐大，webpack 編譯時就會占用很多的系統資源，如果超出了V8對 Node 默認的內存限制大小就會出現node內存溢出的報錯:CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
進入 kibana/bin 目錄（也可以設置在 node 的環境變量里，我是加在 kibana 文件中）
vi kibana
添加內存空間的環境變量（以下 size 為1g，根據實際情況修改）
NODE_OPTIONS="--max-old-space-size=12288"
https://segmentfault.com/a/1190000017972349
https://banyudu.com/posts/dynamic-change-nodejs-memory-limit.3c34c2
https://forum.search-guard.com/t/kibana-7-6-memory-issue-with-sg/1814