logstash配置文件信息如下:
1 input { 2 redis { 3 host => "192.168.116.133" 4 port => "6379" 5 db => "6" 6 data_type => "list" 7 key => "apache-accesslog" 8 } 9 } 10 11 filter { 12 grok { 13 match => { "message" => "%{COMBINEDAPACHELOG}" } 14 } 15 } 16 17 output { 18 elasticsearch { 19 hosts => ["192.168.116.131:9200"] 20 index => "apache-accesslog-%{+YYYY.MM.dd}" 21 } 22 }
启动logstash提示如下错误:
Redis connection problem {:exception=>#<Redis::CommandError: ERR unknown command 'script'>, :level=>:warn}
出现错误原因在于:
logstash有一个叫batch_count的参数,该参数是指从队列中一次读取多少条数据,默认是读取125条,可以将batch_count设置为1,关闭这个功能,即可解决上述启动报错!
修改logstash配置文件
1 input { 2 redis { 3 host => "192.168.116.133" 4 port => "6379" 5 db => "6" 6 data_type => "list" 7 batch_count => "1" 8 key => "apache-accesslog" 9 } 10 } 11 12 filter { 13 grok { 14 match => { "message" => "%{COMBINEDAPACHELOG}" } 15 } 16 } 17 18 output { 19 elasticsearch { 20 hosts => ["192.168.116.131:9200"] 21 index => "apache-accesslog-%{+YYYY.MM.dd}" 22 } 23 }
启动成功,问题解决
1 [root@www conf.d]# /opt/logstash/bin/logstash -f indexer.conf 2 Settings: Default pipeline workers: 1 3 Pipeline main started