Prometheus自定義metrics監控進程存活狀態

一. 監控進程存活

有時候我們需要監控進程的狀態,由於我們常用的node_exporter並不能覆蓋所有監控項，這里我們使用自定義的方式對進程進行監控。

 

二. 自定義Python腳本定義metrics值

2.1 安裝pip

yum install -y python-pip

2.2 編寫py腳本

# coding: utf-8
import sys
import psutil

from prometheus_client import CollectorRegistry, Gauge, write_to_textfile

monitor_list = [{'name': 'gitlab','desc':'gitlab-process'},{'name': 'nginx','desc':'nginx-process'},]

def checkProcessCount(process_name):
    count = 0
    for proc in psutil.process_iter():
        try:
            if process_name.lower() in proc.name().lower():
                count +=1
        except (psutil.NoSuchProcess, psutil.AccessDenied, psutil.ZombieProcess):
            pass
    print count
    return count


def save_metrics():
    registry = CollectorRegistry()
    gauge = Gauge('process_number', 'Number of Process',['name'], registry=registry)
    for p in monitor_list:
        count = checkProcessCount(p['name'])
        gauge.labels(name=p['name']).set(count)
    write_to_textfile('/var/lib/node_exporter/textfile_collector/metadata.prom', registry)


if __name__ == '__main__':
    save_metrics()

把腳本加入到定時任務中,根據自定義定的metrics值去匹配做rules報警