准備工作
通過 釘釘 進行告警，首先需要有一個釘釘群。和釘釘機器人。
進入釘釘群設置中的智能群助手
添加自定義機器人
安全設置選擇加簽
此處需要保存加簽秘鑰。
完成添加器人
此處需要保存 webhook 地址。
部署 prometheus-webhook-dingtalk
Github上已經有人寫好了項目可以直接使用，用於釘釘機器人推送告警信息
項目地址https://github.com/timonwong/prometheus-webhook-dingtalk
部署dingtalk
wget https://github.com/timonwong/prometheus-webhook-dingtalk/releases/download/v2.0.0/prometheus-webhook-dingtalk-2.0.0.linux-amd64.tar.gz tar xf prometheus-webhook-dingtalk-2.0.0.linux-amd64.tar.gz ln -s prometheus-webhook-dingtalk-2.0.0.linux-amd64 prometheus-webhook
准備配置文件
mv config.example.yml config.yml [root@ops prometheus-webhook]# cat config.yml ## Request timeout timeout: 5s # 超時時間 # 模板文件 templates: - templates/alertmanager-dingtalk.tmpl targets: webhook1: url: https://oapi.dingtalk.com/robot/send?access_token=xxxxx # webhook 地址 # secret for signature secret: SECdcc64f10xxxxxx # 加簽秘鑰 message: text: '{{ template "email.to.message" . }}'
准備模板文件
[root@ops prometheus-webhook]# cat templates/alertmanager-dingtalk.tmpl {{ define "email.to.message" }} {{- if gt (len .Alerts.Firing) 0 -}} {{- range $index, $alert := .Alerts -}} ========= **監控告警** ========= **告警程序:** Alertmanager **告警類型:** {{ $alert.Labels.alertname }} **告警級別:** {{ $alert.Labels.severity }} 級 **告警狀態:** {{ .Status }} **故障主機:** {{ $alert.Labels.instance }} {{ $alert.Labels.device }} **告警主題:** {{ .Annotations.summary }} **告警詳情:** {{ $alert.Annotations.message }}{{ $alert.Annotations.description}} **主機標簽:** {{ range .Labels.SortedPairs }} </br> [{{ .Name }}: {{ .Value | markdown | html }} ] {{- end }} </br> **故障時間:** {{ ($alert.StartsAt.Add 28800e9).Format "2006-01-02 15:04:05" }} ========= = end = ========= {{- end }} {{- end }} {{- if gt (len .Alerts.Resolved) 0 -}} {{- range $index, $alert := .Alerts -}} ========= 告警恢復 ========= **告警程序:** Alertmanager **告警主題:** {{ $alert.Annotations.summary }} **告警主機:** {{ .Labels.instance }} **告警類型:** {{ .Labels.alertname }} **告警級別:** {{ $alert.Labels.severity }} 級 **告警狀態:** {{ .Status }} **告警詳情:** {{ $alert.Annotations.message }}{{ $alert.Annotations.description}} **故障時間:** {{ ($alert.StartsAt.Add 28800e9).Format "2006-01-02 15:04:05" }} **恢復時間:** {{ ($alert.EndsAt.Add 28800e9).Format "2006-01-02 15:04:05" }} ========= = **end** = ========= {{- end }} {{- end }} {{- end }}
准備配置文件
[root@ops prometheus-webhook]# cat /usr/lib/systemd/system/prometheus-webhook-dingtalk.service [Unit] Description=Node Exporter Wants=network-online.target After=network-online.target [Service] ExecStart=/usr/local/prometheus-webhook-dingtalk-2.0.0.linux-amd64/prometheus-webhook-dingtalk --config.file=/usr/local/prometheus-webhook-dingtalk-2.0.0.linux-amd64/config.yml [Install] WantedBy=default.target
啟動程序
systemctl start prometheus-webhook-dingtalk.service systemctl status prometheus-webhook-dingtalk.service systemctl enable prometheus-webhook-dingtalk.service
alertmanager 配置文件
route: # 路由組 group_by: ['alertname', 'app'] # group_wait: 30s group_interval: 40s repeat_interval: 1m receiver: webhook # 默認組 receivers: # 收件人組 - name: webhook webhook_configs: - url: http://localhost:8060/dingtalk/webhook1/send # dingtalk 地址 send_resolved: true inhibit_rules: - source_match: severity: 'critical' target_match_re: severity: '.*' equal: ['instance']
測試
告警事件
告警恢復
