Prometheus 監控之 zookeeper

Git 項目地址：https://github.com/jiankunking/zookeeper_exporter
exporter 下載地址：https://github.com/carlpett/zookeeper_exporter/releases/download/v1.0.2/zookeeper_exporter
注意：export 適合 zookeeper3.4+
①下載 zookeeper_export
wget https://github.com/carlpett/zookeeper_exporter/releases/download/v1.0.2/zookeeper_exporter
②啟動 zookeeper_export
nohup /usr/local/bin/zookeeper_exporter >>/dev/null 2>&1 &
③查看是否正常

④將 export 加入到 prometheus 服務端。

⑤登陸 grafana，導入模板；搜索 Zookeeper Exporer Overview 或者 拷貝 pid 9236

zookeeper alter 監控參考如下：

groups:
- name: zookeeperStatsAlert
  rules:
  - alert: 堆積請求數過大
    expr: avg(zk_outstanding_requests) by (instance) > 10    for: 1m
    labels:      severity: critical
    annotations:
      summary: "Instance {{ $labels.instance }} "
      description: "積請求數過大"
  - alert: 阻塞中的 sync 過多
    expr: avg(zk_pending_syncs) by (instance) > 10
    for: 1m
    labels:
      severity: critical
    annotations:
      summary: "Instance {{ $labels.instance }} "
      description: "塞中的 sync 過多"
  - alert: 平均響應延遲過高
    expr: avg(zk_avg_latency) by (instance) > 10
    for: 1m
    labels:
      severity: critical
    annotations:
      summary: "Instance {{ $labels.instance }} "
      description: '平均響應延遲過高'
  - alert: 打開文件描述符數大於系統設定的大小
    expr: zk_open_file_descriptor_count > zk_max_file_descriptor_count * 0.85
    for: 1m
    labels:
      severity: critical
    annotations:
      summary: "Instance {{ $labels.instance }} "
      description: '打開文件描述符數大於系統設定的大小'
  - alert: zookeeper服務器宕機
    expr: zk_up == 0
    for: 1m
    labels:
      severity: critical
    annotations:
      summary: "Instance {{ $labels.instance }} "
      description: 'zookeeper服務器宕機'
  - alert: zk主節點丟失
    expr: absent(zk_server_state{state="leader"})  != 1
    for: 1m
    labels:
      severity: critical
    annotations:
      summary: "Instance {{ $labels.instance }} "
      description: 'zk主節點丟失'

 

 

需要指定閾值的指標

zk_outstanding_requests 堆積請求數
zk_pending_syncs 阻塞中的 sync 操作
zk_avg_latency 平均 響應延遲
zk_open_file_descriptor_count 打開 文件描述符 數
zk_max_file_descriptor_count 最大 文件描述符 數
zk_up 1
zk_server_state 主從狀態
zk_num_alive_connections 活躍連接數

source:https://hacpai.com/article/1575868724409