Goaccess解析nginx日志備忘

參考

http://nginx.org/en/docs/http/ngx_http_log_module.html?&_ga=1.92028562.949762386.1481787781#log_format
https://www.goaccess.io/man#custom-log

ssh root@server 'zcat logs/access*.gz' | goaccess --log-format='%h [%d:%t %^] "%r" %s %b "%R" "%u" %T' --date-format=%d/%b/%Y --time-format=%T -o dist/report.html

展開

goaccess默認的日志格式與nginx的默認格式一致

%h %^[%d:%t %^] "%r" %s %b "%R" "%u"

log_format combined '$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent"';

自定義日志格式

現在增加一個響應時間, 刪除$remote_user, 並把$body_bytes_sent修改為$bytes_sent,

修改nginx.conf

log_format myfmt '$remote_addr [$time_local] '
                    '"$request" $status $bytes_sent '
                    '"$http_referer" "$http_user_agent" $request_time';
access_log logs/access.log myfmt;

修改goaccess 配置文件

# vim /usr/local/etc/goaccess.conf 
time-format %H:%M:%S
date-format %d/%b/%Y
log-format %h [%d:%t %^] "%r" %s %b "%R" "%u" %T

執行示例

cat logs/access.log | goaccess -o ~/static/report.html

goaccess -f logs/access.log -o ~/static/report.html --real-time-html

變量說明

1.$remote_addr 與$http_x_forwarded_for 用以記錄客戶端的ip地址；
 2.$remote_user ：用來記錄客戶端用戶名稱；
 3.$time_local ： 用來記錄訪問時間與時區；
 4.$request ： 用來記錄請求的url與http協議；
 5.$status ： 用來記錄請求狀態；成功是200，
 6.$body_bytes_sent ：記錄發送給客戶端文件主體內容大小；
 7.$http_referer ：用來記錄從那個頁面鏈接訪問過來的；
 8.$http_user_agent ：記錄客戶端瀏覽器的相關信息；

nginx日志相關

Syntax: log_format name [escape=default|json] string ...;
Default:
log_format combined "...";
Context: http
Specifies log format.

The escape parameter (1.11.8) allows setting json or default characters escaping in variables, by default, default escaping is used.

The log format can contain common variables, and variables that exist only at the time of a log write:

$bytes_sent
the number of bytes sent to a client
$connection
connection serial number
$connection_requests
the current number of requests made through a connection (1.1.18)
$msec
time in seconds with a milliseconds resolution at the time of the log write
$pipe
“p” if request was pipelined, “.” otherwise
$request_length
request length (including request line, header, and request body)
$request_time
request processing time in seconds with a milliseconds resolution; time elapsed between the first bytes were read from the client and the log write after the last bytes were sent to the client
$status
response status
$time_iso8601
local time in the ISO 8601 standard format
$time_local
local time in the Common Log Format
In the modern nginx versions variables $status (1.3.2, 1.2.2), $bytes_sent (1.3.8, 1.2.5), $connection (1.3.8, 1.2.5), $connection_requests (1.3.8, 1.2.5), $msec (1.3.9, 1.2.6), $request_time (1.3.9, 1.2.6), $pipe (1.3.12, 1.2.7), $request_length (1.3.12, 1.2.7), $time_iso8601 (1.3.12, 1.2.7), and $time_local (1.3.12, 1.2.7) are also available as common variables.
Header lines sent to a client have the prefix “sent_http_”, for example, $sent_http_content_range.

The configuration always includes the predefined “combined” format:

log_format combined '$remote_addr - $remote_user [$time_local] '
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent"';