istio+prometheus+grafana 流量監控

 從零到一配置 OpenAPI：

 

概要

       Istio采集指標prometheus+grafana方案，搭建（promethues+prometheus-postgresql-adapter+pg_prometheus） promethues·監控存儲平台，

通過Istio+grafana 進行istio流量注入grafana圖表展示，實現對API流量的統計。

配置 Prometheus 並且把數據存儲至  Postgres (Prometheus + Postgres[TimescaleDB(pg_prometheus)])

 

 

安裝配置 Prometheus

1. 安裝prometheus 
2. 配置promethues.yaml. 配置參數如下：

global:
scrape_interval: 5s
evaluation_interval: 5s
external_labels:
monitor: 'codelab-monitor'
scrape_configs:
- job_name: prometheus
static_configs:
- targets:
- node-exporter-default:9100

- job_name: dx-servicemesh
static_configs:
- targets:
- istio-telemetry.istio-system:42422

remote_write:
- url: "http://prometheus-postgresql-adapter-default:9201/write"  # 遠程寫入pg,需要用適配器轉接
remote_read:
- url: "http://prometheus-postgresql-adapter-default:9201/read"

 

配置 Prometheus 數據存儲至 Postgres

 

要將TimescaleDB和PostgreSQL連接到Prometheus，有兩個組件：a. 的 Prometheus_PostgreSQL_Adapter   b.具有pg_prometheus和timescaledb擴展名的PostgreSQL數據庫

1.安裝pg_prometheus:

• 在postgresql的postgresql.conf中添加：shared_preload_libraries = 'pg_prometheus'，重啟psql
• 使用psql創建擴展：CREATE EXTENSION pg_prometheus;
• 創建role: CREATE ROLE admin WITH LOGIN PASSWORD 'admin';
• 授權role admin：GRANT ALL ON SCHEMA prometheus TO admin;
• 創建table：SELECT create_prometheus_table('metrics');

2.安裝prometheus-postgresql-adapter：

         啟動命令如下，需要注意的是，由於在容器內部運行，-pg.host必須指定postgresql主機的地址，而非loopback接口:

        -pg-host\=pg-prometheus-default -pg-password\=123456 -pg-prometheus-log-samples  # 連接pg數據庫

 

配置集群外部服務接入，並監控訪問流量 （Istio + Prometheus + Grafana）

安裝 Istio

 

Istio.yam:l


# Gateway描述了在網格邊緣運行的負載均衡器，用於接收傳入或傳出的HTTP / TCP連接
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: gateway-external-svcentry
spec:
selector:
istio: ingressgateway # use Istio default gateway implementation
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- external.with.svcentry

---
# VirtualService定義了一組尋址主機時要應用的流量路由規則
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: vs-external-svcentry
spec:
hosts:
- external.with.svcentry
gateways:
- gateway-external-svcentry
http:
- match:
- uri:
prefix: /
route:
- destination:
port:
number: 80
host: google-external-svcentry.service-pro.svc.cluster.local


---

apiVersion: v1
kind: Service
metadata:
name: google-external-svcentry
spec:
ports:
- name: http-8205
port: 80
protocol: TCP
targetPort: 8205
sessionAffinity: None
type: ClusterIP
status:
loadBalancer: {}

---

apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
generation: 1
name: se-external-svcentry
spec:
endpoints:
- address: 10.8.1.157  #接入外部服務IP
labels: {}
hosts:
- google-external-svcentry.service-pro.svc.cluster.local
ports:
- name: http-8205
number: 8205  #外部服務端口
protocol: HTTP
resolution: STATIC

 

安裝 Grafana

Grafana接入promethues圖表顯示，

Grafana接入pgsql進行圖表展示。

 配置 Prometheus 抓取 Istio 訪問請求

mixer組件中遙測相關的對外提供的Kubernetes的Service的服務名是istio-telemetry，mixer對外開放的exporter的數據查詢接口是/metrics；

- job_name: dx-servicemesh
static_configs:
- targets:
- istio-telemetry.istio-system:42422

 

Istio 接入外部服務

開放服務的IP和端口，接入istio服務。（這邊開放了一個Google的IP和端口：10.7.122.10:30025）

apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
generation: 1
name: se-external-svcentry
spec:
endpoints:
- address: 10.7.122.10  #接入外部服務IP
labels: {}
hosts:
- google-external-svcentry.service-pro.svc.cluster.local
ports:
- name: http-8205
number: 30025  #外部服務端口
protocol: HTTP
resolution: STATIC

 

 

  

自己搭建的一個小demo：

1.	pg查詢Google翻譯istio_requests_total：

	連接pg數據庫： 
	命令： psql -U postgres -h 10.7.122.10 -p 30018 -d postgres 

	password：123456

	查詢語句：

	SELECT time, name, value, labels FROM metrics WHERE name = 'istio_requests_total' AND time >= '2020-01-03T09:54:53Z' AND time <= '2020-02-03T10:59:53Z'  AND labels @> '{"destination_service":"google-external-svcentry.service-pro.svc.cluster.local","monitor":"codelab-monitor"}' ORDER BY time \g 



2.  promethues：http://10.7.122.10:30008/graph

	PromQL:istio_requests_total{destination_service="google-external-svcentry.service-pro.svc.cluster.local"}


3. Grafana: http://10.7.122.10:31234/dashboard/new?tab=queries&panelId=2&edit&fullscreen&orgId=1

	user: admin 
	password:admin
	Metrics:istio_requests_total{destination_service="google-external-svcentry.service-pro.svc.cluster.local"}