組件說明：

1.MetricServer：是kubernetes集群資源使用情況的聚合器，收集數據給kubernetes集群內使用，如kubectl，hpa，scheduler等

2.prometheusOperator:是一個系統監測和警報工具箱，用來存儲監控數據。

3.NodeExporter：用於各Node的關鍵度量指標狀態數據

4.KubeStateMetrics:收集kubernetes集群內資源對象數據，制定告警規則

5.Prometheus:采集pull方式收集apiserver,scheduler,controller-manager,kubelet組件數據，通過Http協議傳輸

6.Grafana:是可視化數據統計和監控平台。

[root@k8s-master01 ~]# mkdir prometheus
[root@k8s-master01 ~]# cd prometheus/
[root@k8s-master01 prometheus]#

Git下項目

[root@k8s-master01 prometheus]# git clone https://github.com/coreos/kube-prometheus.git
正克隆到 'kube-prometheus'...
remote: Enumerating objects: 12120, done.
remote: Counting objects: 100% (871/871), done.
remote: Compressing objects: 100% (319/319), done.
remote: Total 12120 (delta 515), reused 731 (delta 461), pack-reused 11249
接收對象中: 100% (12120/12120), 6.13 MiB | 3.02 MiB/s, done.
處理 delta 中: 100% (7480/7480), done.

到yaml文件的模板目錄

[root@k8s-master01 prometheus]# cd kube-prometheus/manifests/
[root@k8s-master01 manifests]#

修改 grafana-service.yaml 文件，使用 nodepode 方式訪問 grafana：

apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: grafana
    app.kubernetes.io/name: grafana
    app.kubernetes.io/part-of: kube-prometheus
    app.kubernetes.io/version: 7.5.4
  name: grafana
  namespace: monitoring
spec:
  #增加NodePort訪問方式
  type: NodePort
  ports:
  - name: http
    port: 3000
    targetPort: http
    #增加端口
    nodePort: 30100
  selector:
    app.kubernetes.io/component: grafana
    app.kubernetes.io/name: grafana
    app.kubernetes.io/part-of: kube-prometheus

修改prometheus-service.yaml

apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: prometheus
    app.kubernetes.io/name: prometheus
    app.kubernetes.io/part-of: kube-prometheus
    app.kubernetes.io/version: 2.26.0
    prometheus: k8s
  name: prometheus-k8s
  namespace: monitoring
spec:
  type: NodePort
  ports:
  - name: web
    port: 9090
    targetPort: web
    nodePort: 30200
  selector:
    app: prometheus
    app.kubernetes.io/component: prometheus
    app.kubernetes.io/name: prometheus
    app.kubernetes.io/part-of: kube-prometheus
    prometheus: k8s
  sessionAffinity: ClientIP

修改alertmanager-service.yaml

apiVersion: v1
kind: Service
metadata:
  labels:
    alertmanager: main
    app.kubernetes.io/component: alert-router
    app.kubernetes.io/name: alertmanager
    app.kubernetes.io/part-of: kube-prometheus
    app.kubernetes.io/version: 0.21.0
  name: alertmanager-main
  namespace: monitoring
spec:
  type: NodePort
  ports:
  - name: web
    port: 9093
    targetPort: web
    nodePort: 30300
  selector:
    alertmanager: main
    app: alertmanager
    app.kubernetes.io/component: alert-router
    app.kubernetes.io/name: alertmanager
    app.kubernetes.io/part-of: kube-prometheus
  sessionAffinity: ClientIP

[root@k8s-master01 manifests]# kubectl apply -f ../manifests/

多執行2遍可能有依賴

[root@k8s-master01 manifests]# kubectl get pod -n monitoring
NAME                                   READY   STATUS    RESTARTS   AGE
alertmanager-main-0                    2/2     Running   0          5m59s
alertmanager-main-1                    2/2     Running   0          5m43s
alertmanager-main-2                    2/2     Running   0          5m20s
grafana-7dc5f8f9f6-prwwk               1/1     Running   0          6m2s
kube-state-metrics-5cbd67455c-ppqbx    4/4     Running   0          5m39s
node-exporter-6s9dz                    2/2     Running   0          5m57s
node-exporter-jgmsz                    2/2     Running   0          5m57s
node-exporter-swhz5                    2/2     Running   0          5m57s
prometheus-adapter-668748ddbd-qmtdc    1/1     Running   0          5m53s
prometheus-k8s-0                       3/3     Running   1          5m48s
prometheus-k8s-1                       3/3     Running   1          5m48s
prometheus-operator-7447bf4dcb-xsb4b   1/1     Running   0          6m17s
[root@k8s-master01 manifests]#

top查看下狀態

[root@k8s-master01 manifests]# kubectl top node
NAME           CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%   
k8s-master01   818m         20%    1741Mi          22%       
k8s-node01     180m         4%     1174Mi          15%       
k8s-node02     175m         4%     1196Mi          15%