最近研究了prometheus+grafana的系統監控,使用grafana的報警功能,grafana支持很多種通知渠道,下文記錄使用到的幾種notification channels,分別是email,kafka和webhook
官網文檔 http://docs.grafana.org/alerting/notifications/
- 發送郵件
修改smtp,找到%GRAFANA_HOME%/conf/defaults.ini(注windows環境)
[smtp] enabled = true #開戶email發送配置 host = smtp.xxx.com:25 #此處需要加上端口號 user =username@xxx.com #郵箱賬號 # If the password contains # or ; you have to wrap it with triple quotes. Ex """#password;""" password =1234567890 #郵箱密碼 cert_file = key_file = skip_verify = true #跳過校驗 from_address = admin@grafana.localhost from_name = Grafana ehlo_identity =
- kafka
向kafka發送消息,需要kafka REST proxy,我使用的中間件是confluent,安裝過程沒什么特別之外,配置(confluent_home/etc/kafka-rest/kafka-rest.properties)如下
#id=kafka-rest-test-server #schema.registry.url=http://localhost:8081 #zookeeper.connect=localhost:2181 #bootstrap.servers=PLAINTEXT://localhost:9092 zookeeper.connect=xx.xx.xx.xx:2181 bootstrap.servers=PLAINTEXT://xx.xx.xx.xx:9092
啟用kafka-rest服務
bin/kafka-rest-start ../etc/kafka-rest/kafka-rest.properties
添加notification channels
點擊 “send test”按鈕,瀏覽kafka-rest服務,alert-gateway-test-1已經在kafka topic里了,開發者可消費kafka的消息,進行后續(自定義)處理
- webhook(推薦)
回調參數
{ "title": "My alert", "ruleId": 1, "ruleName": "Load peaking!", "ruleUrl": "http://url.to.grafana/db/dashboard/my_dashboard?panelId=2", "state": "alerting", "imageUrl": "http://s3.image.url", "message": "Load is peaking. Make sure the traffic is real and spin up more webfronts", "evalMatches": [ { "metric": "requests", "tags": {}, "value": 122 } ] }
更詳細說明,可以參考官網http://docs.grafana.org/alerting/notifications/