測試目的是,用ELK處理在業務中用戶定義的json log日志,則試PHP腳本如下:
<?php for( $i=0;$i<100;$i++) { $reg = array( 'method' => 'login', 'user_id' => rand(1000,3000), 'user_name' => "name_".rand(1,3000 ), 'level' => 1, 'register_time' => time(), ); $str = json_encode( $reg ); file_put_contents( "testlog" , $str."\n" , FILE_APPEND ); $reg = array( 'method' => 'register', 'user_id' => rand(1000,3000), 'user_name' => "name_".rand(1,3000 ), 'level' => rand(1,30), 'login_time' => time(), ); $str = json_encode( $reg ); file_put_contents( "testlog" , $str."\n" , FILE_APPEND ); }
循環生成注冊log和登錄log保存到testlog文件中,結果如下:
{"method":"register","user_id":2933,"user_name":"name_91","level":27,"login_time":1470179550}
{"method":"login","user_id":1247,"user_name":"name_979","level":1,"register_time":1470179550}
{"method":"register","user_id":2896,"user_name":"name_1972","level":17,"login_time":1470179550}
{"method":"login","user_id":2411,"user_name":"name_2719","level":1,"register_time":1470179550}
{"method":"register","user_id":1588,"user_name":"name_1484","level":4,"login_time":1470179550}
{"method":"login","user_id":2507,"user_name":"name_1190","level":1,"register_time":1470179550}
{"method":"register","user_id":2382,"user_name":"name_234","level":21,"login_time":1470179550}
{"method":"login","user_id":1208,"user_name":"name_443","level":1,"register_time":1470179550}
{"method":"register","user_id":1331,"user_name":"name_1297","level":3,"login_time":1470179550}
{"method":"login","user_id":2809,"user_name":"name_743","level":1,"register_time":1470179550}
logstash目錄下建立配置文件
vim config/json.conf
input { file { path => "/home/bona/logstash-2.3.4/testlog" start_position => "beginning" codec => "json" } } output { elasticsearch { hosts => ["192.168.68.135:9200"] index => "data_%{method}" } }
重點是index中,%{method} 來匹配log中的method字段.
以上log就會分別建立data_login data_register兩個索引, 要注意的是索引名稱必須全部小寫
ES中已經成功以method建立了索引
參考資料:
http://udn.yyuap.com/doc/logstash-best-practice-cn/output/elasticsearch.html