NXlog文檔：
http://nxlog.org/docs/nxlog-ce/nxlog-reference-manual.html
https://nxlog.org/documentation/nxlog-community-edition-reference-manual-v20928
Windows服務器上安裝NXlog，然后配置為將Windows服務器上的事件日志發送到Linux syslog服務器（192.168.200.29:514）
<Extension _syslog>
Module xm_syslog #收集事件日志，所有的事件日志默認都被收集
</Extension>
<Input in> #配置input
Module im_msvistalog
# For windows 2003 and earlier use the following: # Module im_mseventlog </Input> <Output out> #配置output Module om_tcp #配置為使用tcp模式 Host 192.168.200.29 #遠端syslog server IP Port 514 #遠端syslog server端口 Exec to_syslog_snare(); </Output> <Route 1> Path in => out #源輸入對應輸出 </Route>
配置完成后，如果nxlog出現如下錯誤：配置完成后，ERROR Couldn't read next event, corrupted eventlog?; The data is invalid.
則是因為windows8.1下windows事件日志種類過多導致，可以自定義需要保存的事件日志種類，設置如下：
<Extension _syslog> Module xm_syslog </Extension> <Input in> Module im_msvistalog # For windows 2003 and earlier use the following: # Module im_mseventlog ReadFromLast FALSE SavePos FALSE Query <QueryList>\ <Query Id="0">\ <Select Path="Security">*</Select>\ #配置為只發送Security Log </Query>\ </QueryList> </Input> <Output out> Module om_tcp Host 192.168.200.29 Port 514 Exec to_syslog_snare(); </Output> <Route 1> Path in => out </Route>
配置為同時接收多個事件日志：
<Input in> Module im_msvistalog # For windows 2003 and earlier use the following: # Module im_mseventlog ReadFromLast FALSE SavePos FALSE Query <QueryList>\ <Query Id="0">\ <Select Path="Application">*</Select>\ <Select Path="System">*</Select>\ <Select Path="Security">*</Select>\ </Query>\ </QueryList> </Input>
Linux收集到的日志如下：
每條事件日志對應一個log
如果Linux上某個目錄下文件過多，刪除方式如下：ls | xargs -n 10 rm -fr ls
## This is a sample configuration file. See the nxlog reference manual about the ## configuration options. It should be installed locally and is also available ## online at http://nxlog.org/docs/ ## Please set the ROOT to the folder your nxlog was installed into, ## otherwise it will not start. #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension _syslog> Module xm_json #日志輸出格式為json </Extension> <Input in> Module im_msvistalog # For windows 2003 and earlier use the following: # Module im_mseventlog ReadFromLast FALSE SavePos FALSE Query <QueryList>\ <Query Id="0">\ <Select Path="Security">*</Select>\ #配置輸入源為收集Security Log </Query>\ </QueryList> </Input> <Input systemin> #配置第二個輸入源名為 systemin Module im_msvistalog # For windows 2003 and earlier use the following: # Module im_mseventlog ReadFromLast FALSE SavePos FALSE Query <QueryList>\ <Query Id="0">\ <Select Path="System">*</Select>\ #配置收集System Log </Query>\ </QueryList> </Input> <Output out> #配置第一個輸出方式，對應接收input in Module om_file File 'E:\logtest\seclog.json' #輸出到seclog.json文件 Exec to_json(); </Output> <Output systemout> #配置第二個輸出方式，對應接收input systemin Module om_file File 'E:\logtest\syslog.json' Exec to_json(); </Output> <Route 1> Path in => out #將第一個input in 對應到ouput out </Route> <Route 2> Path systemin => systemout #將第二個input systemin 對應到 output systemout </Route>
結果：
將兩個input輸出到同一個output：
<Route 1> Path in, systemin => out </Route>
按時間對output file進行rotate（未測試）：
define DIR C:\\logdir <Output out> Module om_file File "%DIR%\\test.log" <Schedule> Every 1 sec Exec out->rotate_to("%DIR%\\test."+ strftime(now(), "%Y%m%d%H%M%S")); </Schedule> </Output>
按大小對output file進行rotate：
<Extension _syslog> Module xm_json </Extension> <Input secin> Module im_msvistalog # For windows 2003 and earlier use the following: # Module im_mseventlog ReadFromLast FALSE SavePos FALSE Query <QueryList>\ <Query Id="0">\ <Select Path="Security">*</Select>\ </Query>\ </QueryList> </Input> <Input systemin> Module im_msvistalog # For windows 2003 and earlier use the following: # Module im_mseventlog ReadFromLast FALSE SavePos FALSE Query <QueryList>\ <Query Id="0">\ <Select Path="System">*</Select>\ </Query>\ </QueryList> </Input> <Output secout> Module om_file CreateDir TRUE sync FALSE File "e:\logtest\sec_" + $Hostname + "_" + month(now()) + ".json" Exec if secout->file_size() > 20M \ { \ $newfile = "e:\logtest\sec_" + $Hostname + "_" + strftime(now(), "%Y%m%d%H%M%S") + ".json"; \ secout->rotate_to($newfile); \ }; Exec to_json(); </Output> <Output systemout> Module om_file File 'E:\logtest\syslog.json' Exec to_json(); </Output> <Route 1> Path secin => secout </Route> <Route 2> Path systemin => systemout </Route>
output還可以編輯為如下，效果相同：
<Output secout> Module om_file CreateDir TRUE sync FALSE File "e:\logtest\sec_" + $Hostname + "_" + month(now()) + ".json" #此為當前正在寫入的文件 <Exec> if secout->file_size() > 20M { $newfile = "e:\logtest\sec_" + $Hostname + "_" + strftime(now(), "%Y%m%d%H%M%S") + ".json"; secout->rotate_to($newfile); } </Exec> Exec to_json(); </Output>
輸出如下：
收集IIS Log：
IIS仍為源格式
<Input IIS01> #按照不同站點分為多個Input Module im_file File "d:\\iislog\W3SVC2\\\\u_ex*.log" SavePos TRUE </Input> <Output IIS01>同其他</Output>
IIS log輸出為json格式
<Extension w3c> Module xm_csv Fields $date, $time, $s-ip, $cs-method, $cs-uri-stem, $cs-uri-query, $s-port, $cs-username, $c-ip, $csUser-Agent, $cs-Referer, $sc-status, $sc-substatus, $sc-win32-status, $time-taken FieldTypes string, string, string, string, string, string, integer, string, string, string, string, integer, integer, integer, integer Delimiter ' ' QuoteChar '"' EscapeControl FALSE UndefValue - </Extension> <Input iis01in> Module im_file File 'E:\IISLog\W3SVC1\u_ex*' SavePos TRUE Exec if $raw_event =~ /^#/ drop(); \ else \ { \ w3c->parse_csv(); \ $EventTime = parsedate($date + " " + $time); \ $SourceName = "IIS"; \ $raw_event = to_json(); \ } </Input> <Output iis01out> Module om_file CreateDir TRUE sync FALSE
Exec $Hostname = 'server01'; File "e:\logtest\iis_" + $Hostname + "_" + month(now()) + ".json" <Exec> if iis01out->file_size() > 200M { $newfile = "e:\logtest\iis_" + $Hostname + "_" + strftime(now(), "%Y%m%d%H%M%S") + ".json"; iis01out->rotate_to($newfile); } </Exec> # Exec to_json(); </Output> <Route iis01> Path iis01in => iis01out </Route>
Using the Query directive
The im_msvistalog has a Query directive which can be used to specify an XML Query that gets passed to the Windows EventLog API in order to read only the selected events. The Windows Event Viewer can help construct such XML queries. The following example will only collect only process creation event records from the Sysmon source.
Query <QueryList> \
<Query Id="0">\
<Select Path="Microsoft-Windows-Sysmon/Operational">*[System[(EventID='1')]]</Select>\
</Query>\
</QueryList>
The event records filtered with the Query directive do not reach NXLog so this might be slightly more efficient than the next native NXLog filtering method.
Filtering with NXLog's log processing language
The NXLog log processing language is available for use by all modules and may be easier to write than the XML query syntax provided by the Windows EventLog API that the im_msvistalog exposes. The following NXLog style filter statement achieves the same as the XML Query above.
Exec if not ($Channel == 'Microsoft-Windows-Sysmon' and $EventID == 1) drop();
The following filtering rule will remove event records that are HTTP network connections to a specific server:
Exec if $SourceName == 'Microsoft-Windows-Sysmon' and $DestinationPort == 80 and $DestinationIp == 10.0.0.1 drop();
nxlog-ce-2.9.1504出現如下報錯，可以替換成版本“nxlog-ce-2.9.1347”解決
報錯：ERROR if-else failed at line 61, character 312 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; procedure 'parse_csv' failed at line 61, character 98 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; Not enough fields in CSV input, expected 15, got 11 in input 'Exchange.asmx &CorrelationID=<empty>;&cafeReqId=bdbb53c3-5227-4601-8dbe-a5a7fb72c7b9; 80 xx\tuliqi 106.3.4.150 AppleExchangeWebServices/806.1+ExchangeSync/121 - 200 0 0 9'