碼上快樂

相關內容    簡體    繁體

基於Prometheus+Grafana搭建可視化監控服務 (一) Prometheus監控

本文轉載自   查看原文   2021-08-15 11:40   106    Prometheus+Grafana/ OS.Linux

基於Prometheus+Grafana搭建可視化監控服務 (一) Prometheus及Exporter安裝

目錄

一、概述

Prometheus官網文檔: https://prometheus.io/docs/introduction/overview/

各組件默認端口:
prometheus ：9090
node 默認端口：9091(原默認端口為9100,與elasticsearch端口沖突，故這里改為9091)
mysql_exporter默認端口：9104
redis_exporter 默認端口：9121
rabbitmq_prometheus 默認端口： 15692
elasticsearch_exporter 默認端口： 9114
kafka_exporter 默認端口： 9308
blackbox_exporter默認端口: 9115
mongodb_exporter 端口: 9216
alertmanager默認端口：9093

注意：
1.本文使用的各程序版本是當下最新版本，實際需要到官網去看是否有更新的版本，推薦使用最新版本。
2.本文Prometheus等程序安裝及監控目標都為Linux環境(CentOS7)
3.Promethues支持pull和push兩種方式，本文描述的是pull方式
4.exporter一般都部署在監控目標服務器上，blackbox_exporter則和prometheus部署在一起

二、安裝Prometheus

2.1.安裝Prometheus

[root@server ~]# cd /usr/local/src
[root@server src]# wget https://github.com/prometheus/prometheus/releases/download/v2.28.1/prometheus-2.28.1.linux-amd64.tar.gz
[root@server src]# mkdir -p /usr/local/prometheus/
[root@server src]# tar xvf prometheus-2.28.1.linux-amd64.tar.gz -C /usr/local/prometheus/
[root@server src]# mv /usr/local/prometheus/prometheus-2.28.1.linux-amd64/ /usr/local/prometheus/prometheus

2.2.將Promethues配置成系統服務

[root@server src]# vi /etc/systemd/system/prometheus.service

[Unit]
Description=Prometheus Server
Documentation=https://prometheus.io/docs/introduction/overview/
After=network.target
  
[Service]
Restart=on-failure
WorkingDirectory=/usr/local/prometheus/prometheus
ExecStart=/usr/local/prometheus/prometheus/prometheus \
  --config.file=/usr/local/prometheus/prometheus/prometheus.yml \
  --storage.tsdb.path=/usr/local/prometheus/prometheus/data \
  --storage.tsdb.retention.time=30d
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -SIGINT $MAINPID
[Install]
WantedBy=multi-user.target

config.file指定配置文件地址
storage.tsdb.path指定數據文件目錄
storage.tsdb.retention.time指定數據文件保留時間

Promethues提供了很多的命令行配置選項，可通過./prometheus -h進行查看

2.3.通過systemctl啟動prometheus

[root@server src]# systemctl daemon-reload
[root@server src]# systemctl start prometheus
[root@server src]# systemctl status prometheus
[root@server src]# systemctl enable prometheus
[root@server src]# systemctl reload prometheus

2.4.prometheus界面

通過瀏覽器訪問http://服務器IP:9090就可以訪問到prometheus的主界面
http://172.16.111.48:9090/targets

三、監控遠程Linux主機

https://github.com/prometheus/node_exporter/

3.1.安裝node_exporter

[root@dbserver ~]# cd /usr/local/src
[root@dbserver src]# wget https://github.com/prometheus/node_exporter/releases/download/v1.2.0/node_exporter-1.2.0.linux-amd64.tar.gz
[root@dbserver src]# mkdir -p /usr/local/prometheus/
[root@dbserver src]# tar xvf node_exporter-1.2.0.linux-amd64.tar.gz -C /usr/local/prometheus/
[root@dbserver src]# mv /usr/local/prometheus/node_exporter-1.2.0.linux-amd64/ /usr/local/prometheus/node_exporter

3.2.將node_exporter配置成系統服務

[root@dbserver src]# vi /etc/systemd/system/node_exporter.service

[Unit]
Description=node_exporter Server
Documentation=https://prometheus.io/docs/introduction/overview/
After=network.target
  
[Service]
Restart=on-failure
WorkingDirectory=/usr/local/prometheus/node_exporter
ExecStart=/usr/local/prometheus/node_exporter/node_exporter \
  --web.listen-address=0.0.0.0:9101
ExecStop=/bin/kill -SIGINT $MAINPID

[Install]
WantedBy=multi-user.target

3.3.通過systemctl啟動node_exporter

[root@dbserver src]# systemctl daemon-reload
[root@dbserver src]# systemctl start node_exporter
[root@dbserver src]# systemctl status node_exporter
[root@dbserver src]# systemctl enable node_exporter

3.4.驗證node_exporter

[root@dbserver src]# curl "http://127.0.0.1:9101/metrics"

3.5.防火牆配置

[root@dbserver src]# firewall-cmd --permanent --add-port=9101/tcp
[root@dbserver src]# firewall-cmd --reload

#也可以限定僅能指定的IP訪問
[root@dbserver src]# firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="192.168.1.100" port protocol="tcp" port="9101" accept"

#或可直接關閉防火牆
[root@dbserver src]# systemctl stop firewalld
[root@dbserver src]# systemctl disable firewalld

3.6.Prometheus側配置

3.6.1.prometheus.yml配置調整

[root@dbserver src]# vi /usr/local/prometheus/prometheus/prometheus.yml

scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
    - targets: ['127.0.0.1:9090']

  - job_name: 'node_exporter'
    static_configs:
    - targets: ['192.168.1.100:9101','192.168.1.101:9101']
      labels:
        service: 服務A
    - targets: ['192.168.1.102:9101','192.168.1.103:9101']
      labels:
        service: 服務B

3.6.2.校驗配置正確性

[root@dbserver src]# /usr/local/prometheus/prometheus/promtool check config /usr/local/prometheus/prometheus/prometheus.yml

3.6.3.重新加載配置

[root@dbserver src]# systemctl reload prometheus

3.6.4.驗證結果

訪問http://ip:9090/targets查看

四、監控MySQL

https://github.com/prometheus/mysqld_exporter/

4.1.創建監控用賬號

CREATE USER 'exporter'@'localhost' IDENTIFIED BY 'Exporter#123';
GRANT PROCESS,RELOAD,REPLICATION CLIENT, SELECT ON *.* TO 'exporter'@'localhost';
FLUSH PRIVILEGES;

4.2.安裝mysqld_exporter

[root@dbserver ~]# cd /usr/local/src
[root@dbserver src]# wget https://github.com/prometheus/mysqld_exporter/releases/download/v0.13.0/mysqld_exporter-0.13.0.linux-amd64.tar.gz
[root@dbserver src]# mkdir -p /usr/local/prometheus/
[root@dbserver src]# tar xvf mysqld_exporter-0.13.0.linux-amd64.tar.gz -C /usr/local/prometheus/
[root@dbserver src]# mv /usr/local/prometheus/mysqld_exporter-0.13.0.linux-amd64/ /usr/local/prometheus/mysqld_exporter
```

### 4.3.增加配置文件
```shell
[root@dbserver src]# vi /usr/local/prometheus/mysqld_exporter/.my.cnf
[client]
user=exporter
password=Exporter#123
socket=/var/lib/mysql/mysql.sock

4.4.將mysqld_exporter配置成系統服務

[root@dbserver src]# vi /etc/systemd/system/mysqld_exporter.service

[Unit]
Description=mysqld_exporter Server
Documentation=https://github.com/prometheus/mysqld_exporter/
After=network.target

[Service]
Environment=DATA_SOURCE_NAME=exporter:Exporter#123@(localhost:3306)/
ExecStart=/usr/local/prometheus/mysqld_exporter/mysqld_exporter \
  --web.listen-address=0.0.0.0:9104 \
  --config.my-cnf=/usr/local/prometheus/mysqld_exporter/.my.cnf \
  --collect.binlog_size \
  --collect.slave_status \
  --collect.slave_hosts \
  --collect.engine_innodb_status \
  --collect.info_schema.processlist \
  --collect.info_schema.innodb_metrics \
  --collect.info_schema.innodb_tablespaces \
  --collect.info_schema.clientstats \
  --collect.perf_schema.tableiowaits \
  --collect.perf_schema.indexiowaits \
  --collect.perf_schema.tablelocks \
  --collect.perf_schema.file_events \
  --collect.perf_schema.eventswaits
ExecStop=/bin/kill -SIGINT $MAINPID
Restart=on-failure
[Install]
WantedBy=multi-user.target

4.5.通過systemctl啟動mysqld_exporter

[root@dbserver src]# systemctl daemon-reload
[root@dbserver src]# systemctl start mysqld_exporter
[root@dbserver src]# systemctl status mysqld_exporter
[root@dbserver src]# systemctl enable mysqld_exporter

4.6.驗證mysqld_exporter

[root@dbserver src]# curl "http://127.0.0.1:9104/metrics"

4.7.防火牆配置

[root@dbserver src]# firewall-cmd --permanent --add-port=9104/tcp
[root@dbserver src]# firewall-cmd --reload

#也可以限定僅能指定的IP訪問
[root@dbserver src]# firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="192.168.1.100" port protocol="tcp" port="9104" accept"

#或可直接關閉防火牆
[root@dbserver src]# systemctl stop firewalld
[root@dbserver src]# systemctl disable firewalld

4.8.Prometheus側配置

4.8.1.prometheus.yml配置調整

[root@dbserver src]# vi /usr/local/prometheus/prometheus/prometheus.yml

scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
    - targets: ['127.0.0.1:9090']

  - job_name: 'mysql_exporter'
    static_configs:
    - targets: ['192.168.1.100:9104']
      labels:
        service: 服務A-MySQL
    - targets: ['192.168.1.101:9104']
      labels:
        service: 服務B-MySQL

4.8.2.校驗配置正確性

[root@dbserver src]# /usr/local/prometheus/prometheus/promtool check config /usr/local/prometheus/prometheus/prometheus.yml

4.8.3.重新加載配置

[root@dbserver src]# systemctl reload prometheus

4.8.4.驗證結果

訪問http://ip:9090/targets查看

五、監控Redis

https://github.com/oliver006/redis_exporter/

5.1.安裝redis_exporter

[root@dbserver ~]# cd /usr/local/src/
[root@dbserver src]# wget https://github.com/oliver006/redis_exporter/releases/download/v1.24.0/redis_exporter-v1.24.0.linux-amd64.tar.gz
[root@dbserver src]# mkdir -p /usr/local/prometheus/
[root@dbserver src]# tar xvf redis_exporter-v1.24.0.linux-amd64.tar.gz -C /usr/local/prometheus/
[root@dbserver src]# mv /usr/local/prometheus/redis_exporter-v1.24.0.linux-amd64/ /usr/local/prometheus/redis_exporter

5.2.將redis_exporter配置成系統服務

[root@dbserver src]# vi /etc/systemd/system/redis_exporter.service

[Unit]
Description=redis_exporter Server
Documentation=https://github.com/oliver006/redis_exporter
After=network.target

[Service]
ExecStart=/usr/local/prometheus/redis_exporter/redis_exporter \
  --web.listen-address=0.0.0.0:9121 \
  --redis.addr=127.0.0.1:6379 \
  --redis.password=Test#123
ExecStop=/bin/kill -SIGINT $MAINPID
Restart=on-failure
[Install]
WantedBy=multi-user.target

5.3.通過systemctl啟動redis_exporter

[root@dbserver src]# systemctl daemon-reload
[root@dbserver src]# systemctl start redis_exporter
[root@dbserver src]# systemctl status redis_exporter
[root@dbserver src]# systemctl enable redis_exporter

5.4.驗證redis_exporter

[root@dbserver src]# curl "http://127.0.0.1:9121/metrics"

5.5.防火牆配置

[root@dbserver src]# firewall-cmd --permanent --add-port=9121/tcp
[root@dbserver src]# firewall-cmd --reload

#也可以限定僅能指定的IP訪問
[root@dbserver src]# firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="192.168.1.100" port protocol="tcp" port="9121" accept"

#或可直接關閉防火牆
[root@dbserver src]# systemctl stop firewalld
[root@dbserver src]# systemctl disable firewalld

5.6.Prometheus側配置

5.6.1.prometheus.yml配置調整

[root@dbserver src]# vi /usr/local/prometheus/prometheus/prometheus.yml

scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
    - targets: ['127.0.0.1:9090']

  - job_name: 'redis_exporter'
    static_configs:
    - targets: ['192.168.1.100:9121']
      labels:
        service: 服務A-Redis
    - targets: ['192.168.1.101:9121']
      labels:
        service: 服務B-Redis

5.6.2.校驗配置正確性

[root@dbserver src]# /usr/local/prometheus/prometheus/promtool check config /usr/local/prometheus/prometheus/prometheus.yml

5.6.3.重新加載配置

[root@dbserver src]# systemctl reload prometheus

5.6.4.驗證結果

訪問http://ip:9090/targets查看

5.6.5 Redis集群監控配置示例

(本人未驗證)

  - job_name: 'redis_exporter_targets'
    static_configs:
      - targets:
        - redis://192.168.1.101:7000
        - redis://192.168.1.102:7001
        - redis://192.168.1.103:7002
        - redis://192.168.1.104:7003
        - redis://192.168.1.105:7004
        - redis://192.168.1.106:7005
    metrics_path: /scrape
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: 192.168.1.100:9121
  ## config for scraping the exporter itself
  - job_name: 'redis_exporter'
    static_configs:
      - targets:
        - 192.168.1.100:9121

六、監控RabbitMQ

6.1.安裝rabbitmq_prometheus

RabbitMQ 3.8版本以后已內置支持Prometheus插件
https://www.rabbitmq.com/prometheus.html

啟用方式：

[root@dbserver src]# rabbitmq-plugins enable rabbitmq_prometheus

默認端口號為15692
啟動后可訪問 http://ip:15692/metrics

如要修改默認配置，對應配置文件為 /etc/rabbitmq/rabbitmq.conf
prometheus.path = /metrics
prometheus.tcp.port = 15692

也可以使用單獨的rabbitmq_exporter來監控，可適應rabbitmq3.8以前版本
（注意 3.6.x與更早版本監控有差別，需要注意)
https://github.com/kbudde/rabbitmq_exporter/

6.2.驗證rabbitmq_prometheus

[root@dbserver src]# curl "http://127.0.0.1:15692/metrics"

6.3.防火牆配置

[root@dbserver src]# firewall-cmd --permanent --add-port=15692/tcp
[root@dbserver src]# firewall-cmd --reload

#也可以限定僅能指定的IP訪問
[root@dbserver src]# firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="192.168.1.100" port protocol="tcp" port="15692" accept"

#或可直接關閉防火牆
[root@dbserver src]# systemctl stop firewalld
[root@dbserver src]# systemctl disable firewalld

6.4.Prometheus側配置

6.4.1.prometheus.yml配置調整

[root@dbserver src]# vi /usr/local/prometheus/prometheus/prometheus.yml

scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
    - targets: ['127.0.0.1:9090']

  - job_name: 'rabbitmq_exporter'
    static_configs:
    - targets: ['192.168.1.100:15692']
      labels:
        service: 服務A-RabbitMQ
    - targets: ['192.168.1.101:15692']
      labels:
        service: 服務B-RabbitMQ

6.4.2.校驗配置正確性

[root@dbserver src]# /usr/local/prometheus/prometheus/promtool check config /usr/local/prometheus/prometheus/prometheus.yml

6.4.3.重新加載配置

[root@dbserver src]# systemctl reload prometheus

6.4.4.驗證結果

訪問http://ip:9090/targets查看

七、監控Elasticsearch

Elasticsearch Exporter
https://grafana.com/oss/prometheus/exporters/elasticsearch-exporter/
https://github.com/prometheus-community/elasticsearch_exporter

7.1.安裝elasticsearch_exporter

[root@dbserver ~]# cd /usr/local/src
[root@dbserver src]# wget https://github.com/justwatchcom/elasticsearch_exporter/releases/download/v1.2.1/elasticsearch_exporter-1.2.1.linux-amd64.tar.gz
[root@dbserver src]# mkdir -p /usr/local/prometheus/
[root@dbserver src]# tar xvf elasticsearch_exporter-1.2.1.linux-amd64.tar.gz -C /usr/local/prometheus/
[root@dbserver src]# mv /usr/local/prometheus/elasticsearch_exporter-1.2.1.linux-amd64/ /usr/local/prometheus/elasticsearch_exporter

7.2.將elasticsearch_exporter配置成系統服務

[root@dbserver src]# vi /etc/systemd/system/elasticsearch_exporter.service

[Unit]
Description=elasticsearch_exporter Server
Documentation=https://github.com/prometheus-community/elasticsearch_exporter
After=network.target

[Service]
ExecStart=/usr/local/prometheus/elasticsearch_exporter/elasticsearch_exporter \
  --web.listen-address=0.0.0.0:9114 \
  --es.uri=http://127.0.0.1:9200
ExecStop=/bin/kill -SIGINT $MAINPID
Restart=on-failure
[Install]
WantedBy=multi-user.target

如果有啟用賬號密碼，則需要攜帶上 http://admin:pass@localhost:9200

7.3.通過systemctl啟動elasticsearch_exporter

[root@dbserver src]# systemctl daemon-reload
[root@dbserver src]# systemctl start elasticsearch_exporter
[root@dbserver src]# systemctl status elasticsearch_exporter
[root@dbserver src]# systemctl enable elasticsearch_exporter

7.4.驗證elasticsearch_exporter

[root@dbserver src]# curl "http://127.0.0.1:9114/metrics"

7.5.防火牆配置

[root@dbserver src]# firewall-cmd --permanent --add-port=9114/tcp
[root@dbserver src]# firewall-cmd --reload

#也可以限定僅能指定的IP訪問
[root@dbserver src]# firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="192.168.1.100" port protocol="tcp" port="9114" accept"

#或可直接關閉防火牆
[root@dbserver src]# systemctl stop firewalld
[root@dbserver src]# systemctl disable firewalld

7.6.Prometheus側配置

7.6.1.prometheus.yml配置調整

[root@dbserver src]# vi /usr/local/prometheus/prometheus/prometheus.yml

scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
    - targets: ['127.0.0.1:9090']

  - job_name: 'elasticsearch_exporter'
    static_configs:
    - targets: ['192.168.1.100:9114']
      labels:
        service: 服務A-Elasticsearch
    - targets: ['192.168.1.101:9114']
      labels:
        service: 服務B-Elasticsearch

7.6.2.校驗配置正確性

[root@dbserver src]# /usr/local/prometheus/prometheus/promtool check config /usr/local/prometheus/prometheus/prometheus.yml

7.6.3.重新加載配置

[root@dbserver src]# systemctl reload prometheus

7.6.4.驗證結果

訪問http://ip:9090/targets查看

八、監控Kafka

https://github.com/danielqsj/kafka_exporter/

8.1.安裝kafka_exporter

[root@dbserver ~]# cd /usr/local/src/
[root@dbserver src]# wget https://github.com/danielqsj/kafka_exporter/releases/download/v1.3.1/kafka_exporter-1.3.1.linux-amd64.tar.gz
[root@dbserver src]# mkdir -p /usr/local/prometheus/
[root@dbserver src]# tar xvf kafka_exporter-1.3.1.linux-amd64.tar.gz -C /usr/local/prometheus/
[root@dbserver src]# mv /usr/local/prometheus/kafka_exporter-1.3.1.linux-amd64/ /usr/local/prometheus/kafka_exporter

8.2.將kafka_exporter配置成系統服務

[root@dbserver src]# vi /etc/systemd/system/kafka_exporter.service

[Unit]
Description=kafka_exporter Server
Documentation=https://github.com/danielqsj/kafka_exporter
After=network.target

[Service]
ExecStart=/usr/local/prometheus/kafka_exporter/kafka_exporter \
  --web.listen-address=0.0.0.0:9308 \
  --kafka.server=127.0.0.1:9092
ExecStop=/bin/kill -SIGINT $MAINPID
Restart=on-failure
[Install]
WantedBy=multi-user.target

8.3.通過systemctl啟動kafka_exporter

[root@dbserver src]# systemctl daemon-reload
[root@dbserver src]# systemctl start kafka_exporter
[root@dbserver src]# systemctl status kafka_exporter
[root@dbserver src]# systemctl enable kafka_exporter

8.4.驗證kafka_exporter

[root@dbserver src]# curl "http://127.0.0.1:9308/metrics"

8.5.防火牆配置

[root@dbserver src]# firewall-cmd --permanent --add-port=9308/tcp
[root@dbserver src]# firewall-cmd --reload

#也可以限定僅能指定的IP訪問
[root@dbserver src]# firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="192.168.1.100" port protocol="tcp" port="9308" accept"

#或可直接關閉防火牆
[root@dbserver src]# systemctl stop firewalld
[root@dbserver src]# systemctl disable firewalld

8.6.Prometheus側配置

8.6.1.prometheus.yml配置調整

[root@dbserver src]# vi /usr/local/prometheus/prometheus/prometheus.yml

scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
    - targets: ['127.0.0.1:9090']

  - job_name: 'elasticsearch_exporter'
    static_configs:
    - targets: ['192.168.1.100:9308']
      labels:
        service: 服務A-Kafka
    - targets: ['192.168.1.101:9308']
      labels:
        service: 服務B-Kafka

8.6.2.校驗配置正確性

[root@dbserver src]# /usr/local/prometheus/prometheus/promtool check config /usr/local/prometheus/prometheus/prometheus.yml

8.6.3.重新加載配置

[root@dbserver src]# systemctl reload prometheus

8.6.4.驗證結果

訪問http://ip:9090/targets查看

九、監控Doris

Doris已內置支持對FE,BE的Promotheus監控支持
無需額外配置，防護牆允許Prometheus訪問 8030,8040端口即可

9.1.驗證doris_prometheus

#FE
[root@dbserver src]# curl "http://127.0.0.1:8030/metrics"

#BE
[root@dbserver src]# curl "http://127.0.0.1:8040/metrics"

9.2.Prometheus側配置

9.2.1.prometheus.yml配置調整

[root@dbserver src]# vi /usr/local/prometheus/prometheus/prometheus.yml

scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
    - targets: ['127.0.0.1:9090']

  - job_name: 'doris_cluster'
    static_configs:
    - targets: ['192.168.1.100:8030','192.168.1.101:8030']
      labels:
	    service: 服務A-Doris-FE
        group: fe

    - targets: ['192.168.1.102:8040','192.168.1.103:8040','192.168.1.104:8040']
      labels:
	    service: 服務A-Doris-BE
        group: be

9.2.2.校驗配置正確性

[root@dbserver src]# /usr/local/prometheus/prometheus/promtool check config /usr/local/prometheus/prometheus/prometheus.yml

9.2.3.重新加載配置

[root@dbserver src]# systemctl reload prometheus

9.2.4.驗證結果

訪問http://ip:9090/targets查看

十、監控端口(HTTP/TCP等)

10.1.安裝blackbox_exporter

這里使用 blackbox_exporter來監控HTTP/TCP

[root@appserver ~]# cd /usr/local/src/
[root@appserver src]# wget https://github.com/prometheus/blackbox_exporter/releases/download/v0.19.0/blackbox_exporter-0.19.0.linux-amd64.tar.gz
[root@appserver src]# tar xvf blackbox_exporter-0.19.0.linux-amd64.tar.gz -C /usr/local/prometheus/
[root@appserver src]# mv /usr/local/prometheus/blackbox_exporter-0.19.0.linux-amd64/ /usr/local/prometheus/blackbox_exporter

10.2.將blackbox_exporter配置成系統服務

編輯啟動文件

[root@appserver src]# vi /etc/systemd/system/blackbox_exporter.service

[Unit]
Description=blackbox_exporter Server
Documentation=https://prometheus.io/docs/introduction/overview/
After=network.target
  
[Service]
Restart=on-failure
WorkingDirectory=/usr/local/prometheus/blackbox_exporter
ExecStart=/usr/local/prometheus/blackbox_exporter/blackbox_exporter \
  --web.listen-address ":9115" \
  --config.file=/usr/local/prometheus/blackbox_exporter/blackbox.yml
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -SIGINT $MAINPID

[Install]
WantedBy=multi-user.target

10.3.通過systemctl啟動blackbox_exporter

[root@appserver src]# systemctl daemon-reload
[root@appserver src]# systemctl start blackbox_exporter
[root@appserver src]# systemctl status blackbox_exporter
[root@appserver src]# systemctl enable blackbox_exporter

10.4.Prometheus側配置

10.4.1.prometheus.yml配置調整

[root@dbserver src]# vi /usr/local/prometheus/prometheus/prometheus.yml

scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
    - targets: ['127.0.0.1:9090']

# 網站監控
  - job_name: 'http_status'
    metrics_path: /probe
    params:
      module: [http_2xx]
    static_configs:
      - targets: ['http://a1.domain.com:81']
        labels:
          env: PROD
          domain: "a1.domain.com"
          port: "81"
          service: 服務A
          instance: "http://a1.domain.com:81"
          group: web
      - targets: ['http://a1.domain.com:82/abc']
        labels:
          env: PROD
          domain: "a1.domain.com"
          port: "82"
          service: XX API接口
          instance: "http://a1.domain.com:82/abc"
          group: api

    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - target_label: __address__
        replacement: 192.168.1.100:9115 
		
# ping 檢測
  - job_name: 'ping_status'
    metrics_path: /probe
    params:
      module: [icmp]
    static_configs:
      - targets: ['192.168.1.100']
        labels:
          env: PROD
          service: 服務B IP
          instance: "192.168.1.100"
          group: ping

    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - target_label: __address__
        replacement: 192.168.1.100:9115 
		
# 端口監控
  - job_name: 'port_status'
    metrics_path: /probe
    params:
      module: [tcp_connect]
    static_configs:
      - targets: ['192.168.1.100:84']
        labels:
          env: PROD
          port: "84"
          service: 服務B XX端口
          instance: "192.168.1.100:84"
          group: port

    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - target_label: __address__
        replacement: 192.168.1.100:9115

blackbox_exporter涉及的module配置文件 /usr/local/prometheus/blackbox_exporter/blackbox.yml

網上示例參考：https://github.com/tools-env/blackbox_exporter/blob/master/example.yml

10.4.2.校驗配置正確性

[root@dbserver src]# /usr/local/prometheus/prometheus/promtool check config /usr/local/prometheus/prometheus/prometheus.yml

10.4.3.重新加載配置

[root@dbserver src]# systemctl reload prometheus

10.4.4.驗證結果

訪問http://ip:9090/targets查看

十一、監控MongoDB

https://github.com/percona/mongodb_exporter

11.1.安裝mongodb_exporter

[root@dbserver ~]# cd /usr/local/src/
[root@dbserver src]# wget https://github.com/percona/mongodb_exporter/releases/download/v0.20.7/mongodb_exporter-0.20.7.linux-amd64.tar.gz
[root@dbserver src]# mkdir -p /usr/local/prometheus/
[root@dbserver src]# tar xvf mongodb_exporter-0.20.7.linux-amd64.tar.gz -C /usr/local/prometheus/
[root@dbserver src]# mv /usr/local/prometheus/mongodb_exporter-0.20.7.linux-amd64 /usr/local/prometheus/mongodb_exporter

11.2.將mongodb_exporter配置成系統服務

[root@dbserver src]# vi /etc/systemd/system/mongodb_exporter.service

[Unit]
Description=mongodb_exporter Server
Documentation=https://github.com/percona/mongodb_exporter
After=network.target

[Service]
ExecStart=/usr/local/prometheus/mongodb_exporter/mongodb_exporter \
  --web.listen-address=0.0.0.0:9216 \
  --mongodb.uri=mongodb://username:password@127.0.0.1:27017/admin?replicaSet=rs0&&authSource=admin
ExecStop=/bin/kill -SIGINT $MAINPID
Restart=on-failure
[Install]
WantedBy=multi-user.target

11.3.通過systemctl啟動mongodb_exporter

[root@dbserver src]# systemctl daemon-reload
[root@dbserver src]# systemctl start mongodb_exporter
[root@dbserver src]# systemctl status mongodb_exporter
[root@dbserver src]# systemctl enable mongodb_exporter

11.4.驗證mongodb_exporter

[root@dbserver src]# curl "http://127.0.0.1:9216/metrics"

11.5.防火牆配置

[root@dbserver src]# firewall-cmd --permanent --add-port=9216/tcp
[root@dbserver src]# firewall-cmd --reload

#也可以限定僅能指定的IP訪問
[root@dbserver src]# firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="192.168.1.100" port protocol="tcp" port="9216" accept"

#或可直接關閉防火牆
[root@dbserver src]# systemctl stop firewalld
[root@dbserver src]# systemctl disable firewalld

11.6.Prometheus側配置

11.6.1.prometheus.yml配置調整

[root@dbserver src]# vi /usr/local/prometheus/prometheus/prometheus.yml

scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
    - targets: ['127.0.0.1:9090']

  - job_name: 'mongodb_exporter'
    static_configs:
    - targets: ['192.168.1.100:9216']
      labels:
        service: 服務A-MongoDB

附:其他參考資料


免責聲明！

本站轉載的文章為個人學習借鑒使用，本站對版權不負任何法律責任。如果侵犯了您的隱私權益，請聯系本站郵箱yoyou2525@163.com刪除。



猜您在找 Prometheus + Grafana搭建可視化監控系統(1) Prometheus 監控Mysql服務器及Grafana可視化 Docker搭建Prometheus+grafana監控系統 部署Prometheus+Grafana監控 prometheus+grafana監控redis Prometheus+Grafana 監控 Docker Jmeter + Grafana搭建實時監控可視化 基於docker 搭建Prometheus+Grafana Prometheus+Grafana監控宿主機 ubuntu 18 docker 搭建Prometheus+Grafana
 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM