Prometheus的搭建與使用

1.Prometheus監控框架工具介紹

Prometheus是由谷歌研發的一款開源的監控軟件，它通過安裝在遠程機器上的exporter，通過HTTP協議從遠程的機器收集數據並存儲在本地的時序數據庫上。目前已經被雲計算本地基金會托管，是繼k8s托管的第二個項目，號稱是下一代監控。

2.Prometheus架構圖

3.Prometheus優缺點

Prometheus使用在多維度上靈活的查詢語言(PromQl)。PromQL (Prometheus Query Language) 是 Prometheus 自己開發的數據查詢 DSL 語言。
Prometheus具有易管理、易集成、可擴展、支持自動發信等優勢。
同時Prometheus后端用 golang語言開發，前端是 Grafana，二次開發需要掌握相關語言。
Prometheus的流行和Kubernetes密不可分，支持對Kubernetes、容器、OpenStack的監控。

4.Prometheus官網

https://prometheus.io/

5.Prometheus的部署

1.下載Prometheus：wget https://github.com/prometheus/prometheus/releases/download/v2.30.3/prometheus-2.30.3.linux-amd64.tar.gz
2.解壓：tar xf prometheus-2.30.3.linux-amd64.tar.gz -C /usr/local/ && mv /usr/local/prometheus-2.30.3.linux-amd64 /usr/local/prometheus
3.運行：/usr/local/prometheus/prometheus --config.file="/usr/local/prometheus/prometheus.yml"
[root@k8s-node2:10.200.202.196 ~]# lsof -i:9090
COMMAND   PID USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
prometheu 2296 root   7u IPv6 27875607     0t0 TCP *:websm (LISTEN)
prometheu 2296 root   10u IPv6 27877535     0t0 TCP localhost:51698->localhost:websm (ESTABLISHED)
prometheu 2296 root   11u IPv6 27873732     0t0 TCP localhost:websm->localhost:51698 (ESTABLISHED)

6.訪問Prometheus web

http://10.200.202.196:9090/

7.查看已監控機器Targets

默認只監控了本機 點擊Status -->Targets

8.通過http獲取主機監控數據

http://10.200.202.196:9090/metrics

9.在Web界面可以通過關鍵字查詢監控項

10.監控遠程linux主機

1.下載node_exporter：wget https://github.com/prometheus/node_exporter/releases/download/v1.2.2/node_exporter-1.2.2.linux-amd64.tar.gz
2.解壓node_exporter：tar xf node_exporter-1.2.2.linux-amd64.tar.gz && mv node_exporter-1.2.2.linux-amd64 /usr/local/node_exporter
3.啟動node_exporter：/usr/local/node_exporter
4.通過瀏覽器訪問http://10.200.202.49:9100/metrics可以查看到node_exporter在被監控端手機的監控信息

11.回到Prometheus服務器配置文件添加客戶端的配置段

vim /usr/local/prometheus/prometheus.yml
 - job_name: 'harbor'
   static_configs:
   - targets: ['10.200.202.196:9100']
​
 - job_name: 'k8s-master1'
   static_configs:
   - targets: ['10.200.202.75:9100']
​
 - job_name: 'k8s-node1'
   static_configs:
   - targets: ['10.200.202.49:9100']

12.重啟Prometheus

/usr/local/prometheus/prometheus --config.file="/usr/local/prometheus/prometheus.yml"

13.查看已監控主機

14.查看數據圖表

15.監控遠程MySQL

# 在Client上安裝mysqld_exporter組件和MySQL 
# 下載地址：https://github.com/prometheus/mysqld_exporter/releases/download/v0.13.0/mysqld_exporter-0.13.0.linux-amd64.tar.gz
[root@client ~]# tar -xf mysqld_exporter-0.13.0.linux-amd64.tar.gz -C /usr/local/
[root@client ~]# mv /usr/local/mysqld_exporter-0.13.0.linux-amd64/ /usr/local/mysqld_exporter
[root@client ~]# yum -y install mariadb\*
​
# 啟動MySQL並授權
[root@client ~]# systemctl restart mariadb
[root@client ~]# mysql -uroot
MariaDB [(none)]> grant select,replication client,process on *.* to 'mysql_monitor'@'localhost' identified by '123.com';
#授權IP為localhost，因為不是prometheus服務器直接找mariadb獲取數據，而是prometheus服務器找mysql_exporter，mysql_exporter再找mariadb。所以這個localhost是指的mysql_exporter的IP
MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.00 sec)
​
# 創建一個mariadb配置文件，寫上剛剛授權的用戶名和密碼
[root@client ~]# vim /usr/local/mysqld_exporter/.my.cnf
[client]
user=mysql_monitor
password=123.com
​
# 啟動mysqld_exporter
[root@client ~]# nohup /usr/local/mysqld_exporter/mysqld_exporter --config.my-cnf=/usr/local/mysqld_exporter/.my.cnf &
[root@client ~]# lsof -i:9104
COMMAND     PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
mysqld_ex 60421 root   3u IPv6 111828     0t0 TCP *:peerwire (LISTEN)
​
# 在Prometheus服務器的配置文件里添加被監控的mariadb的配置段
[root@prometheus ~]# vim /usr/local/prometheus/prometheus.yml 
  - job_name: 'mariadb'
   static_configs:
    - targets: ['10.10.11.113:9104']
[root@prometheus ~]# pkill prometheus
[root@prometheus ~]# /usr/local/prometheus/prometheus --config.file="/usr/local/prometheus/prometheus.yml"
​
# 回到web管理界面 -->點Status -->點Targets -->可以看到監控mariadb了

16.安裝Grafana

https://grafana.com/docs/grafana/latest/installation/rpm/

# 配置yum倉庫
vim /etc/yum.repos.d/grafana.repo
[grafana]
name=grafana
baseurl=https://packages.grafana.com/enterprise/rpm
repo_gpgcheck=1
enabled=1
gpgcheck=1
gpgkey=https://packages.grafana.com/gpg.key
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt

# 安裝grafana
yum install -y grafana-enterprise

# 啟動grafana
systemctl daemon-reload
systemctl start grafana-server
systemctl status grafana-server
systemctl enable grafana-server

[root@k8s-node2:10.200.202.196 ~]# lsof -i:3000
COMMAND     PID    USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
grafana-s 52262 grafana    8u  IPv6 28255553      0t0  TCP *:hbci (LISTEN)

17.訪問Grafana

http://10.200.202.196:3000/
默認賬號密碼：admin admin

18.Grafana連接Prometheus

將Prometheus服務器收集到的數據作為一個數據源添加到grafana，讓grafana可以得到Prometheus的數據

添加數據源

配置數據源

添加圖表

配置圖表

配置好圖表后記得保存儀表板