Prometheus數據持久化存儲（二）

Prometheus（普羅米修斯）

Prometheus數據持久化存儲（二）

Prometheus+Influx+Grafana+Mysql數據持久化存儲

• Prometheus的存儲數據庫默認只保留15天的數據，Grafana存儲的配置以及圖表都還在容器之中它自己生成的sqlit數據庫中長期存儲並不友好。
• 為了更適應老板的需求，長期存儲數據，我們可以添加時序數據庫InfluxDB作為prometheus后端存儲，可以修改grafana的存儲為mysql以便自己做一些自定義操作的時候方便一點。

Prometheus+Influx存儲數據

cat docker-compose-prometheus-influxdb.yml

version: "2.3"
services:
  prometheus:
    image: prom/prometheus:latest
    container_name: prometheus
    hostname: prometheus
    restart: always
    volumes:
      - /prometheus/prometheus/config:/etc/prometheus
      - /prometheus/prometheus/data:/prometheus
      - /etc/localtime:/etc/localtime
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--log.level=info'
      - '--web.listen-address=0.0.0.0:9090'
      - '--storage.tsdb.path=/prometheus'
      - '--storage.tsdb.retention=15d'
      - '--query.max-concurrency=50'
      - '--web.enable-lifecycle'
    ports:
      - "9090:9090"
    depends_on:
      - influxdb
    logging:
      driver: "json-file"
      options:
        max-size: "1g"
    networks:
      - prom_monitor
  influxdb:
    image: influxdb:latest
    container_name: influxdb
    hostname: influxdb
    restart: always
    volumes:
      - /prometheus/prometheus/influxdb/config:/etc/influxdb
      - /prometheus/prometheus/influxdb/data:/var/lib/influxdb/data
      - /etc/localtime:/etc/localtime
    ports:
      - "8086:8086"
      - "8083:8083"
    environment:
      - INFLUXDB_DB=prometheus
      - INFLUXDB_ADMIN_ENABLED=true
      - INFLUXDB_ADMIN_USER=admin
      - INFLUXDB_ADMIN_PASSWORD=adminpwd
      - INFLUXDB_USER=prometheus
      - INFLUXDB_USER_PASSWORD=prometheuspwd
      - INFLUXDB_CONFIG_PATH=/etc/influxdb/influxdb.conf
    logging:
      driver: "json-file"
      options:
        max-size: "1g"
    networks:
      - prom_monitor

networks:
  prom_monitor:
    driver: bridge

mkdir -p influxdb/data influxdb/config

cat influxdb/config/influxdb.conf

[meta]
  dir = "/var/lib/influxdb/meta"

[data]
  dir = "/var/lib/influxdb/data"
  engine = "tsm1"
  wal-dir = "/var/lib/influxdb/wal"

通過以上文件看出，重新指定了Prometheus的存儲數據庫為InfluxDB，當然，還需要配置Prometheus.yml文件。

cat config/prometheus.yml
在Prometheus的配置文件中，添加遠程讀寫。

# 指定了Prometheus的存儲數據庫為InfluxDB
remote_write:
  - url: "http://influxdb:8086/api/v1/prom/write?db=prometheus&u=prometheus&p=prometheuspwd"
remote_read:
  - url: "http://influxdb:8086/api/v1/prom/read?db=prometheus&u=prometheus&p=prometheuspwd"

其它配置：

cat config/prometheus.yml
添加如下配置

scrape_configs:
  - job_name: 'linux-server'	# 定義任務名
    scrape_interval: 5s
    static_configs:
    file_sd_configs:
      - files:
        - '/etc/prometheus/fileconfig/*-nodes.json'

• file_sd_configs啟動prometheus的文件發現功能，可以將配置主機等信息寫入文件中，如果用戶修改了配置，比如增加或刪除了某些主機，那么Prometheus會自動發現這些更改。

cat config/fileconfig/office-nodes.json

[
    {
        "targets": ["192.168.1.20:9100"],
        "labels": {
            "instance": "192.168.1.20",
            "alias": "office0",
            "job": "node"
        }
    },
    {
        "targets": ["192.168.1.17:9111"],
        "labels": {
            "instance": "192.168.1.17",
            "alias": "office3",
            "job": "node"
        }
    }
]

登錄influxdb數據庫查看相應的數據信息

docker exec -it influxdb bash

influx

use prometheus
show MEASUREMENTS
select * from up
可以查看到相關數據。

完整的prometheus.yml文件

global:
  scrape_interval: 15s # 默認抓取間隔, 15秒向目標抓取一次數據。
  external_labels:
    monitor: 'codelab-monitor'

# 這里表示抓取對象的配置
scrape_configs:

  - job_name: 'linux-server'	# 定義任務名
    scrape_interval: 5s

    static_configs:
    file_sd_configs:
      - files:
        - '/etc/prometheus/fileconfig/*-nodes.json'

# 指定了Prometheus的存儲數據庫為InfluxDB
remote_write:
  - url: "http://influxdb:8086/api/v1/prom/write?db=prometheus&u=prometheus&p=prometheuspwd"
remote_read:
  - url: "http://influxdb:8086/api/v1/prom/read?db=prometheus&u=prometheus&p=prometheuspwd"

# 配置告警規則位置
rule_files:
  - "rules/*rules.yml"

# 配置alertmanager告警發送消息配置
alerting:
  alertmanagers:
    - static_configs:
      - targets: ['192.168.1.20:9093']

附加：

• influxdb時序數據庫的基本命令

# influxdb相關操作
#登錄數據庫
influx
或
influx -host 127.0.0.1 -port 8086 -username prometheus -password prometheuspwd
#查看數據庫
show databases
#切換數據庫
use prometheus
#查看數據表
show MEASUREMENTS
#查看數據
select * from up
#查看用戶
show users
#創建用戶
create user "username" with password 'password' with all privileges
#刪除用戶
drop user username

Grafana+Mysql存儲數據

cat docker-compose-grafana-mysql.yml

version: "2.3"
services:
  grafana:
    image: grafana/grafana:latest
    container_name: grafana
    hostname: grafana
    restart: always
    volumes:
      - /prometheus/grafana/config:/etc/grafana
      - /prometheus/grafana/logs:/var/log/grafana
      - /prometheus/grafana/data:/var/lib/grafana
      - /prometheus/grafana/dashboards:/etc/grafana/provisioning/dashboards
      - /etc/localtime:/etc/localtime
    ports:
      - "3000:3000"
    user: "104"
    depends_on:
      - db
    networks:
      - prom_monitor

  db:
    image: mysql:5.7
    command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
    container_name: mysql
    hostname: mysql
    restart: always
    environment:
      - MYSQL_ROOT_PASSWORD=Admin_pwd
      - MYSQL_DATABASE=grafana
      - MYSQL_USER=grafana
      - MYSQL_PASSWORD=grafana
    volumes:
      - /prometheus/grafana/mysql:/var/lib/mysql
      - /etc/localtime:/etc/localtime
    expose:
      - "3306"
    networks:
      - prom_monitor

networks:
  prom_monitor:
    driver: bridge

grafana配置文件增加數據庫信息

cat config/grafana.ini

[database]

type = mysql
host = mysql:3306
name = grafana
user = grafana
password = grafana
url = mysql://grafana:grafana@mysql:3306/grafana

[session]
provider = mysql
provider_config = `grafana:grafana@tcp(mysql:3306)/grafana`
cookie_name = grafana_session
cookie_secure = false
session_life_time = 86400

啟動服務

docker-compose -f docker-compose-grafana-mysql.yml up -d

登錄數據庫查看信息

docker exec -it mysql bash

mysql -ugrafana -pgrafana

mysql> use grafana;
mysql> show tables;
可查看到相關的數據表

參考資料-Prometheus+Grafana監控簡介