kafka的錯誤日志log監控

本文轉載自查看原文 2019-08-12 11:05 4518 zabbix

例如:需要監控下面的日志中的error

日志文件是這個 /data1/confluent-5.2.2/logs/connect/kafka-connect.log

1/ 參考上一篇安裝zabbix_agent:https://www.cnblogs.com/hongfeng2019/p/11338513.html

2/ 運行python腳本

mkdir -p /home/script
vim /home/script/check_log.py

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import sys

import re


def prePos(seekfile):
    global curpos
    try:
        cf = open(seekfile)
    except IOError:
        curpos = 0
        return curpos
    except FileNotFoundError:
        curpos = 0
        return curpos
    else:
        try:
            curpos = int(cf.readline().strip())
        except ValueError:
            curpos = 0
            cf.close()
            return curpos
        cf.close()
    return curpos


def lastPos(filename):
    with open(filename) as lfile:
        if lfile.readline():
            lfile.seek(0, 2)
        else:
            return 0
        lastPos = lfile.tell()
    return lastPos


def getSeekFile():
    try:
        seekfile = sys.argv[2]
    except IndexError:
        seekfile = '/tmp/logseek'
    return seekfile


def getKey():
    try:
        tagKey = str(sys.argv[3])
    except IndexError:
        tagKey = 'Error'
    return tagKey


def getResult(filename, seekfile, tagkey):
    # 獲取上一次檢查文件時文件末尾的位置
    destPos = prePos(seekfile)
    # 獲取當前文件末尾位置
    curPos = lastPos(filename)

    if curPos < destPos:
        curpos = 0

    try:
        f = open(filename)
    except IOError:
        print('Could not open file: %s' % filename)
    except FileNotFoundError:
        print('Could not open file: %s' % filename)
    else:
        # 指針偏移到上次檢查文件是的末尾位置
        f.seek(destPos)
        global result
        result = 0
        while curPos != 0 and f.tell() < curPos:
            rresult = f.readline().strip()

            if re.search(tagkey, rresult,re.I):
                with open(sys.argv[4], 'a+') as wsf:
                #with open('/tmp/55.log', 'a+') as wsf:
                    wsf.write(rresult + "\n")
                result+=1

        with open(seekfile, 'w+') as sf:
            sf.write(str(curPos))
    finally:
        f.close()
    return result


if __name__ == "__main__":
    result = 0
    curpos = 0
    tagkey = getKey()
    seekfile = getSeekFile()
    result = getResult(sys.argv[1], seekfile, tagkey)
    #result = getResult('cookies.dat', '/tmp/1.log', 'error',)
    print(result)

    # UserParameter = check_log, /usr/bin/python/scripts/log.py/opt/a.log/opt/logseek Error
    # 第一個參數代表要監控的日志文件
    # 第二個參數代表每次查看完日志文件緩存看到哪里了的日志文件
    # 第三個參數代表監控的關鍵字
    # 第四個參數代表篩選出的日志存放位置

View Code

# 第一個參數代表要監控的日志文件

# 第二個參數代表每次查看完日志文件緩存看到哪里了的日志文件

# 第三個參數代表監控的關鍵字,如error 注: 有括號的要轉義'\[ERROR\]'

# 第四個參數代表篩選出的error日志存放位置

mkdir /tmp/question_log

chmod -R 777 /tmp/question_log

/bin/python /home/script/check_log.py /data1/confluent-5.2.2/logs/connect/kafka-connect.log /tmp/question_log/check_kafka-connect_tag.log error /tmp/question_log/kafka-connect_question.log

運行兩次測試: 最新的error數為0

3/ 建立一個user的鍵值腳本

vim /etc/zabbix/zabbix_agentd.d/userparameter_script.conf

UserParameter=check_kafka-connect_error_log,/bin/python /home/script/check_log.py /data1/confluent-5.2.2/logs/connect/kafka-connect.log /tmp/question_log/check_dispatcher_tag.log error /tmp/question_log/dispatcher_question.log

4/ 讓zabbix用戶有訪問log文件的權限

chown zabbix.root /data1/confluent-5.2.2/logs/connect/kafka-connect.log

5/ 從服務器上看能否拿到error的數值

[root@backup ~]# zabbix_get -s 10.52.110.48 -k check_kafka-connect_error_log
0

6/ 登陸zabbix的管理界面,建監控項

為監控的機器添加一個監控項,監控項的鍵值從上面的第三步的鍵值腳本中來

check_kafka-connect_error_log