1、nginx access配置：
2、nginx原始日志：
192.168.1.148 - - [04/May/2017:16:34:18 +0800] "GET /api/Home/GetConfig?configMD5=&storesysno=0001 HTTP/1.1" 200 816 "-" "Apache-HttpClient/4.5.2 (Java/1.7.0_45)" - 0.005 0.005
3、注意一一匹配，注意空格
192.168.1.148 => $remote_addr
- => -
- => $remote_user
[04/May/2017:16:34:18 +0800] =>[$time_local]
"GET /api/Home/GetConfig?configMD5=&storesysno=0001 HTTP/1.1" => "$request"
200=> $status
816 =>$body_bytes_sent
"-"=> "$http_referer"
"Apache-HttpClient/4.5.2 (Java/1.7.0_45)" => "$http_user_agent"
- =>$http_x_forwarded_for
0.005=> $request_time
0.005 =>$upstream_response_time';
4、 grok
%{IP:clientip} - (?:%{WORD:user_name}|-) \[%{HTTPDATE:timestamp}\] "%{WORD:request_action} %{DATA:request} HTTP/%{NUMBER:http_version}" %{NUMBER:status:int} %{NUMBER:body_sent:int} "(?:%{URI:referrer}|-)" "%{DATA:agent}" %{URIPARAM1:http_x_forwarded_fo} %{NUMBER:request_time:float} %{NUMBER:response_time:float}
某些字段為“-”，可能導致grokfailure，此時我們可以通過(?:%{XX:XX}|-)的方式進行匹配，即為空時顯示“-”
實際配置環境：
match => ["message","%{IP:clientip} - (?:%{WORD:user_name}|-) \[%{HTTPDATE:timestamp}\] \"%{WORD:request_action} %{DATA:request} HTTP/%{NUMBER:http_version}\" %{NUMBER:status:int} %{N UMBER:body_sent:int} \"(?:%{URI:referrer}|-)\" \"%{DATA:agent}\" %{GREEDYDATA:http_x_forwarded_fo} %{NUMBER:request_time:float} %{NUMBER:response_time:float}" ] }
OR
match => ["message","%{IP:client_ip} - (?:%{WORD:user_name}|-) \[%{HTTPDATE:timestamp}\] \"(%{WORD:request_action}|-) (%{DATA:request}|-) HTTP/(%{NUMBER:http_version}|-)\" (%{NUMBER:status:int}|-) (%{NUMBER:body_sent:int}|-) \"(?:%{URI:referrer}|-)
\" \"(%{DATA:agent}|-)\" (%{GREEDYDATA:http_x_forwarded_for}|-) (%{NUMBER:request_time:float}|-) (%{NUMBER:response_time:float}|-) (%{IPORHOST:domain}|-) (?:%{HOSTPORT:upstream_addr}|-)"]
########################
NGINX_ERROR (?<timestamp>%{YEAR}[./]%{MONTHNUM}[./]%{MONTHDAY} %{TIME}) \[%{LOGLEVEL:severity}\] %{POSINT:pid}#%{NUMBER}: %{GREEDYDATA:errormessage}
ps：（不停報錯，倒騰了半個小時）
1、match后整個用中括號[] 括起來。
2、雙引號""必須要轉移
logstash2.2 官方解釋：
logstash 5.4:
https://github.com/elastic/logstash/blob/v1.4.2/patterns/grok-patterns
http://grokconstructor.appspot.com/do/match#result
http://blog.csdn.net/yanggd1987/article/details/50486779