【原創】大數據基礎之Logstash（3）應用之http（in和out）

一個logstash很容易通過http打斷成兩個logstash實現跨服務器或者跨平台間數據同步，比如原來的流程是

logstash: nginx log -> kafka

打斷成兩個是

logstash1: nginx log -> http out

logstash2: http in ->kafka

具體如下

http out

  filter {
    ruby {
      code => "event.cancel if not event.get('message').include?'something'"
    }
  }

output {
    http {
      url => "http://test.server:10000"
      codec => "plain"
      format => "json"
      content_type => "application/json"
      http_method => "post"
    }
}

可以通過filter跳過不需要的記錄

http in

input {
    http {
        host => "0.0.0.0"
        port => 10000
        additional_codecs => {"application/json"=>"json"}
        codec => "plain"
        threads => 4
        ssl => false
    }
}

 

http in

參考：https://www.elastic.co/guide/en/logstash/current/plugins-inputs-http.html

http out

參考：https://www.elastic.co/guide/en/logstash/current/plugins-outputs-http.html