logstash使用grok过滤数据


有一段线上日志:

2020-07-14 11:37:04.556 INFO  [com.lyf.action.PlayAction:124] - [ 播放日志 add ] userid: 0 vid: 8079245, vtime: -1
┏━━━━━ Debug [native.update d_stcs_month_page set vcount = ifnul...] ━━━
┣ SQL:  update d_stcs_month_page set vcount = ifnull(vcount,0) + 1 where deptid = ? and vyear = ? and vmonth = ?
┣ 参数:         [0, 2020, 7]
┣ 位置:         com.chaoxing.video.mobile.front.service.stcs.PageStcsThreadTask.run(PageStcsThreadTask.java:76)
┣ 时间:         1ms
┣ 更新:         [1]
┗━━━━━ Debug [native.update d_stcs_month_page set vcount = ifnul...] ━━━

想把log打印的内容部分通过grok读取出来(前提:使用了多行合并),也就是[ 播放日志 add ] userid: 0 vid: 8079245, vtime: -1这一段,下面是过滤规则:

filter {
  if [type] == "debug" {
    grok {
      match => ["message", "%{TIMESTAMP_ISO8601:logdate} %{LOGLEVEL:level} (?<class>.*) - (?<log>[^┏]*)(?<sql>.*)"]
    }
    date {
      match => ["logdate", "yyyy-MM-dd HH:mm:ss.SSS"]
    }
  }
}

grokdebug分析结果如下:

{
  "logdate": [
    [
      "2020-07-14 11:37:04.556"
    ]
  ],
  "YEAR": [
    [
      "2020"
    ]
  ],
  "MONTHNUM": [
    [
      "07"
    ]
  ],
  "MONTHDAY": [
    [
      "14"
    ]
  ],
  "HOUR": [
    [
      "11",
      null
    ]
  ],
  "MINUTE": [
    [
      "37",
      null
    ]
  ],
  "SECOND": [
    [
      "04.556"
    ]
  ],
  "ISO8601_TIMEZONE": [
    [
      null
    ]
  ],
  "level": [
    [
      "INFO"
    ]
  ],
  "class": [
    [
      " [com.lyf.action.PlayAction:124]"
    ]
  ],
  "log": [
    [
      "[ 播放日志 add ] userid: 0 vid: 8079245, vtime: -1\n"
    ]
  ],
  "sql": [
    [
      "┏━━━━━ Debug [native.update d_stcs_month_page set vcount = ifnul...] ━━━"
    ]
  ]
}


免责声明!

本站转载的文章为个人学习借鉴使用,本站对版权不负任何法律责任。如果侵犯了您的隐私权益,请联系本站邮箱yoyou2525@163.com删除。



 
粤ICP备18138465号  © 2018-2025 CODEPRJ.COM