由於最近公司服務頻繁出問題,老板很生氣,下面的人都很不好過,於是老大讓加一下業務監控,來觀察線上數據狀態。
但是由於qa環境數據量太少,所以自己搭建了一套環境做相關監控,並且寫了個腳本模仿生產上的數據,並且做了很多指標,寫了一些Grafana的PromQL語法,網上的資料也很少,我就來補充一下,這個下一章再講。
注意:
docker和docker-compose請自行下載安裝
創建相關目錄並給予權限,持久化目錄需要給777權限,否則容器啟動失敗
cd ~ && mkdir prometheus && chmod 777 prometheus
cd prometheus && mkdir grafana_data prometheus_data && chmod 777 grafana_data prometheus_data
vim docker-compose.yml
version: "3.7"
services:
node-exporter:
image: prom/node-exporter:latest
container_name: "node-exporter0"
ports:
- "9100:9100"
restart: always
prometheus:
image: prom/prometheus:latest
container_name: "prometheus0"
restart: always
ports:
- "9090:9090"
volumes:
- "./prometheus.yml:/etc/prometheus/prometheus.yml"
- "./prometheus_data:/prometheus"
grafana:
image: grafana/grafana
container_name: "grafana0"
ports:
- "3000:3000"
restart: always
volumes:
- "./grafana_data:/var/lib/grafana"
注:prometheus主機需要部署以上容器,其他被監控主機只需要部署node-exporter
3.prometheus配置文件(根據自己的服務器ip進行修改targets內容)
vim prometheus.yml
global:
scrape_interval: 15s # 默認抓取周期
external_labels:
monitor: 'codelab-monitor'
scrape_configs:
- job_name: 'node-exporter' #服務的名稱
scrape_interval: 5s
metrics_path: /metrics #獲取指標的url
static_configs:
- targets: ['139.224.224.229:9100'] # 這個為監聽指定服務服務的ip和port,需要修改為自己的ip,貌似雲服務必須用公網ip
4.docker-compose啟動
docker-compose up -d # 掛載目錄給權限,不然會啟動失敗
如果啟動報如下的"沒有權限"錯誤:
grafana0 | You may have issues with file permissions, more information here: http://docs.grafana.org/installation/docker/#migrate-to-v51-or-later
grafana0 | mkdir: can't create directory '/var/lib/grafana/plugins': Permission denied
grafana0 | GF_PATHS_DATA='/var/lib/grafana' is not writable.
grafana0 | You may have issues with file permissions, more information here: http://docs.grafana.org/installation/docker/#migrate-to-v51-or-later
grafana0 | mkdir: can't create directory '/var/lib/grafana/plugins': Permission denied
grafana0 | GF_PATHS_DATA='/var/lib/grafana' is not writable.
grafana0 | You may have issues with file permissions, more information here: http://docs.grafana.org/installation/docker/#migrate-to-v51-or-later
grafana0 | mkdir: can't create directory '/var/lib/grafana/plugins': Permission denied
grafana0 | GF_PATHS_DATA='/var/lib/grafana' is not writable.
grafana0 | You may have issues with file permissions, more information here: http://docs.grafana.org/installation/docker/#migrate-to-v51-or-later
grafana0 | mkdir: can't create directory '/var/lib/grafana/plugins': Permission denied
grafana0 | GF_PATHS_DATA='/var/lib/grafana' is not writable.
grafana0 | You may have issues with file permissions, more information here: http://docs.grafana.org/installation/docker/#migrate-to-v51-or-later
grafana0 | mkdir: can't create directory '/var/lib/grafana/plugins': Permission denied
則需要將掛載目錄賦予權限,即:
chmod 777 grafana_data/ prometheus_data/ #個人根據配置文件中具體的掛在目錄進行權限修改
5.訪問主機的9090端口可以查到Prometheus監控到的數據,訪問3000端口是grafana的界面,賬密admin/admin
6.配置監控,點擊齒輪,選擇Data Sources
7.點擊Add data source
8.填寫prometheus地址 ip+端口或者主機名+端口(需要配置hosts)
9.點擊save&test
10.點擊+號,選擇import
11.輸入模板號9276,點擊load。 8919
11.導入模板完成