Alertmanager對接釘釘
alertmanager 釘釘告警配置
1.安裝prometheus-webhook-dingtalk
1.1 二進制包安裝
# 二進制包下載 https://github.com/timonwong/prometheus-webhook-dingtalk/releases wget https://github.com/timonwong/prometheus-webhook-dingtalk/releases/download/v0.3.0/prometheus-webhook-dingtalk-0.3.0.linux-amd64.tar.gz # 啟動服務 ./prometheus-webhook-dingtalk --ding.profile="webhook1=https://oapi.dingtalk.com/robot/send?access_token={替換成自己的dingding token}"
1.2 docker鏡像安裝
前提是已經完成docker安裝
docker pull timonwong/prometheus-webhook-dingtalk # 啟動容器 docker run -d -p 8060:8060 --name webhook timonwong/prometheus-webhook --ding.profile="webhook1=https://oapi.dingtalk.com/robot/send?access_token={替換成自己的dingding token}
2.alertmanager中配置
cat alertmanager.yml
global: resolve_timeout: 5m route: receiver: webhook group_wait: 30s group_interval: 5m repeat_interval: 5m group_by: [alertname] routes: - receiver: webhook group_wait: 10s receivers: - name: webhook webhook_configs: - url: http://localhost:8060/dingtalk/webhook1/send send_resolved: true
完整配置:
cat alertmanager.yml
global: # 在沒有報警的情況下聲明為已解決的時間 resolve_timeout: 2m # 配置郵件發送信息 smtp_smarthost: 'smtp.qiye.aliyun.com:465' smtp_from: 'your_email' smtp_auth_username: 'your_email' smtp_auth_password: 'email_passwd' smtp_hello: 'your_email' smtp_require_tls: false # 所有報警信息進入后的根路由,用來設置報警的分發策略 route: # 這里的標簽列表是接收到報警信息后的重新分組標簽,例如,接收到的報警信息里面有許多具有 cluster=A 和 alertname=LatncyHigh 這樣的標簽的報警信息將會批量被聚合到一個分組里面 group_by: ['alertname', 'cluster'] # 當一個新的報警分組被創建后,需要等待至少group_wait時間來初始化通知,這種方式可以確保您能有足夠的時間為同一分組來獲取多個警報,然后一起觸發這個報警信息。 group_wait: 30s # 當第一個報警發送后,等待'group_interval'時間來發送新的一組報警信息。 group_interval: 5m # 如果一個報警信息已經發送成功了,等待'repeat_interval'時間來重新發送他們 repeat_interval: 5m # 默認的receiver:如果一個報警沒有被一個route匹配,則發送給默認的接收器 receiver: default # 優先使用default發送 # 上面所有的屬性都由所有子路由繼承,並且可以在每個子路由上進行覆蓋。 routes: #子路由,使用email發送 - receiver: email match_re: serverity : email # label 匹配email group_wait: 10s receivers: - name: 'default' webhook_configs: - url: http://localhost:8060/dingtalk/webhook1/send send_resolved: true # 發送已解決通知 - name: 'email' email_configs: - to: 'email@qq.com' send_resolved: true
Alertmanager對接微信
1.下載安裝包
#下載安裝包
wget https://github.com/prometheus/alertmanager/releases/download/v0.15.0-rc.1/alertmanager-0.15.0-rc.1.linux-amd64.tar.gz
#解壓
tar -zxvf alertmanager-0.15.0-rc.1.linux-amd64.tar.gz
#修改名稱
sudo mv alertmanager-0.15.0-rc.1.linux-amd64 /usr/local/alertmanager
cd /usr/local/alertmanager
1.1 配置alertmager
vim alertmanager.yml
global: resolve_timeout: 2m # 郵箱 smtp_smarthost: 'smtp.qiye.aliyun.com:465' smtp_from: 'your_email' smtp_auth_username: 'your_email' smtp_auth_password: 'passwd' smtp_hello: 'your_email' smtp_require_tls: false # 微信模版 templates: - '/var/lib/alertmanager/wechat.tmpl' route: group_by: ['alertname'] group_wait: 30s group_interval: 5m repeat_interval: 5m receiver: wechat routes: - receiver: email match_re: serverity: email group_wait: 10s receivers: # 微信 - name: 'wechat' wechat_configs: - corp_id: '企業ID' to_party: '部門ID' agent_id: '1000002' api_secret: 's2yqcDSLo6y81j6ONsztxUj_qeZQ3OoefCZzxDBqcrc' send_resolved: true # 釘釘 - name: 'dingding' webhook_configs: - url: http://localhost:8060/dingtalk/webhook1/send send_resolved: true # email - name: 'email' email_configs: - to: 'email@test.com' send_resolved: true
參數說明:
corp_id: 企業微信賬號唯一 ID, 可以在我的企業中查看。
to_party: 需要發送的組。
agent_id: 第三方企業應用的 ID,可以在自己創建的第三方企業應用詳情頁面查看。
api_secret: 第三方企業應用的密鑰,可以在自己創建的第三方企業應用詳情頁面查看。
1.2 創建告警模板
vim wechat.tmpl
{{ define "wechat.default.message" }} {{ range $i, $alert :=.Alerts }} ========監控報警========== 告警狀態:{{ .Status }} 告警級別:{{ $alert.Labels.severity }} 告警類型:{{ $alert.Labels.alertname }} 告警應用:{{ $alert.Annotations.summary }} 告警主機:{{ $alert.Labels.instance }} 告警詳情:{{ $alert.Annotations.description }} 觸發閥值:{{ $alert.Annotations.value }} 告警時間:{{ $alert.StartsAt.Format "2006-01-02 15:04:05" }} ========end============= {{ end }} {{ end }}
1.3 配置自啟動服務
vim /lib/systemd/system/alertmanager.service
[Unit] Description=Alertmanager for Prometheus After=network-online.target [Service] Type=simple ExecStart=/usr/local/alertmanager/alertmanager --config.file=/usr/local/alertmanager/alertmanager.yml --storage.path=/usr/local/alertmanager/data/ Restart=on-failur ExecStop=/bin/kill -9 $MAINPID [Install] WantedBy=multi-user.target
2.啟動服務
sudo systemctl daemon-reload sudo systemctl start alertmanager.service sudo systemctl status alertmanager.service