報錯有兩條
[2019-11-07T11:02:06,892][WARN ] The shutdown process appears to be stalled due to busy or blocked plugins. Check the logs for more information.
[2019-11-07T11:02:06,892][WARN ][logstash.shutdownwatcher ] { "inflight_count"=>0, "stalling_thread_info"=>{ "other"=>[ { "thread_id"=>91, "name"=>"[main]<file", "current_call"=>"[...]/vendor/bundle/jruby/2.3.0/gems/filewatch-0.9.0/lib/filewatch/watch.rb:288:in `sleep'" }, { "thread_id"=>92, "name"=>"[main]<file", "current_call"=>"[...]/vendor/bundle/jruby/2.3.0/gems/filewatch-0.9.0/lib/filewatch/watch.rb:288:in `sleep'" }, { "thread_id"=>93, "name"=>"[main]<file", "current_call"=>"[...]/vendor/bundle/jruby/2.3.0/gems/filewatch-0.9.0/lib/filewatch/watch.rb:288:in `sleep'" } ], ["LogStash::Filters::Urldecode", {"all_fields"=>"true", "id"=>"49443b0959cc6af6c75b19b63d9ecc590cb6e44e7ce63f56bfdccf380bd65c76"}]=>[ { "thread_id"=>81, "name"=>nil, "current_call"=>"[...]/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:90:in `read_batch'" } ], ["LogStash::Filters::Mutate", {"remove_field"=>["body", "ua", "tags", "host", "path", "@version", "@timestamp", "message"], "id"=>"0e751702fed79225eaa91f3c7178cbda5576d3869ac64eb16e2fe3f933928ce4"}]=>[ { "thread_id"=>82, "name"=>nil, "current_call"=>"[...]/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:90:in `read_batch'" }, { "thread_id"=>83, "name"=>nil, "current_call"=>"[...]/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:90:in `read_batch'" } ], ["LogStash::Filters::Urldecode", {"all_fields"=>"true", "id"=>"c5fff38520e48c5625e1ba5e8147ac528ba49134d776c64e2446b07829e9f448"}]=>[ { "thread_id"=>84, "name"=>nil, "current_call"=>"[...]/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:90:in `read_batch'" } ] }}
logstash 啟動是, 配置了修改配置自動重啟,
--config.reload.automatic
本來都好好的, 但是一次修改配置后, 某台機器上的logstash 開始不再上傳日志了. 發現配置沒有生效.
同時 logs/logstassh-plain.log 開始變大, 每隔幾秒就打印一次上面的兩條日志.
搜索了好久, 沒有結果(看來大家logstash 都用的非常6了, 居然沒有和我類似的錯誤).
(以下都是我結合官方文檔自己想的, 不代表對)
大意是說
- 有過濾器太慢了, 阻塞了停止;
- 如果強行停止,可能會丟失事件;
- 如果覺得丟幾條沒有關系, 可以在啟動的時候使用 --pipeline.unsafe_shutdown
后來
我非常暴力的kill 掉進程, 重啟了logstash .
一切正常了.
問題的原因並不知道, 但是主要耽誤兩天了, 我先把服務搞起來.
以后有緣再說.