报错有两条
[2019-11-07T11:02:06,892][WARN ] The shutdown process appears to be stalled due to busy or blocked plugins. Check the logs for more information.
[2019-11-07T11:02:06,892][WARN ][logstash.shutdownwatcher ] { "inflight_count"=>0, "stalling_thread_info"=>{ "other"=>[ { "thread_id"=>91, "name"=>"[main]<file", "current_call"=>"[...]/vendor/bundle/jruby/2.3.0/gems/filewatch-0.9.0/lib/filewatch/watch.rb:288:in `sleep'" }, { "thread_id"=>92, "name"=>"[main]<file", "current_call"=>"[...]/vendor/bundle/jruby/2.3.0/gems/filewatch-0.9.0/lib/filewatch/watch.rb:288:in `sleep'" }, { "thread_id"=>93, "name"=>"[main]<file", "current_call"=>"[...]/vendor/bundle/jruby/2.3.0/gems/filewatch-0.9.0/lib/filewatch/watch.rb:288:in `sleep'" } ], ["LogStash::Filters::Urldecode", {"all_fields"=>"true", "id"=>"49443b0959cc6af6c75b19b63d9ecc590cb6e44e7ce63f56bfdccf380bd65c76"}]=>[ { "thread_id"=>81, "name"=>nil, "current_call"=>"[...]/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:90:in `read_batch'" } ], ["LogStash::Filters::Mutate", {"remove_field"=>["body", "ua", "tags", "host", "path", "@version", "@timestamp", "message"], "id"=>"0e751702fed79225eaa91f3c7178cbda5576d3869ac64eb16e2fe3f933928ce4"}]=>[ { "thread_id"=>82, "name"=>nil, "current_call"=>"[...]/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:90:in `read_batch'" }, { "thread_id"=>83, "name"=>nil, "current_call"=>"[...]/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:90:in `read_batch'" } ], ["LogStash::Filters::Urldecode", {"all_fields"=>"true", "id"=>"c5fff38520e48c5625e1ba5e8147ac528ba49134d776c64e2446b07829e9f448"}]=>[ { "thread_id"=>84, "name"=>nil, "current_call"=>"[...]/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:90:in `read_batch'" } ] }}
logstash 启动是, 配置了修改配置自动重启,
--config.reload.automatic
本来都好好的, 但是一次修改配置后, 某台机器上的logstash 开始不再上传日志了. 发现配置没有生效.
同时 logs/logstassh-plain.log 开始变大, 每隔几秒就打印一次上面的两条日志.
搜索了好久, 没有结果(看来大家logstash 都用的非常6了, 居然没有和我类似的错误).
(以下都是我结合官方文档自己想的, 不代表对)
大意是说
- 有过滤器太慢了, 阻塞了停止;
- 如果强行停止,可能会丢失事件;
- 如果觉得丢几条没有关系, 可以在启动的时候使用 --pipeline.unsafe_shutdown
后来
我非常暴力的kill 掉进程, 重启了logstash .
一切正常了.
问题的原因并不知道, 但是主要耽误两天了, 我先把服务搞起来.
以后有缘再说.