Prometheus監控k8s(2)-手動部署Prometheus

https://www.cnblogs.com/panwenbin-logs/p/10123833.html

https://github.com/yunlzheng/prometheus-book/blob/master/kubernetes/deploy-prometheus-in-kubernetes.md

https://www.cnblogs.com/xzkzzz/p/10208115.html

1  部署環境

根據之前的環境部署

l   K8S中已有內部DNS服務coredns（參考前面的文檔）

l   已有可使用的動態PV（參考前面的文檔）

 

2  准備文件和環境

2.1  下載文件

項目地址https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/prometheus

git clone https://github.com/kubernetes/kubernetes.git 

#我嫌麻煩就直接克隆kubernetes整個項目了,當然下載zip包其實只有幾十M，更快

或者

# 從github官網下載promethues的yaml部署文件

curl -O https://raw.githubusercontent.com/kubernetes/kubernetes/master/cluster/addons/prometheus/prometheus-rbac.yaml
curl -O https://raw.githubusercontent.com/kubernetes/kubernetes/master/cluster/addons/prometheus/prometheus-configmap.yaml
curl -O https://raw.githubusercontent.com/kubernetes/kubernetes/master/cluster/addons/prometheus/prometheus-service.yaml
curl -O https://raw.githubusercontent.com/kubernetes/kubernetes/master/cluster/addons/prometheus/prometheus-statefulset.yaml

 

[root@k8s-master prometheus]# ls
kubernetes-master
[root@k8s-master prometheus]# cd kubernetes-master/cluster/addons/

[root@k8s-master addons]# mkdir ~/prometheus/
[root@k8s-master addons]# mv prometheus/* /root/prometheus/
[root@k8s-master addons]# cd /root/prometheus/
[root@k8s-master prometheus]# ls
alertmanager-configmap.yaml   kube-state-metrics-deployment.yaml  node-exporter-service.yaml  prometheus-service.yaml
alertmanager-deployment.yaml  kube-state-metrics-rbac.yaml        OWNERS                      prometheus-statefulset.yaml
alertmanager-pvc.yaml         kube-state-metrics-service.yaml     prometheus-configmap.yaml   README.md
alertmanager-service.yaml     node-exporter-ds.yml                prometheus-rbac.yaml
[root@k8s-master prometheus]#

 

2.2  命名空間的問題

默認prometheus使用的是kube-system名稱空間， 網上有很多文章說自定義一下方便管理。

但我自定義之后出現

  Warning  FailedCreate  2m40s (x16 over 5m24s)  statefulset-controller  create Pod prometheus-0 in StatefulSet prometheus failed error: pods "prometheus-0" is forbidden: pods with system-cluster-critical priorityClass is not permitted in k8s-monitor namespace

prometheus-statefulset.yaml 起不來，還是用系統的kube-system，這一塊還是別動了吧，可能和版本有關，這個坑還是別踩了吧

2.3  建立分類目錄方便管理

mkdir node-exporter kube-state-metrics alertmanager prometheus 
#將每個組件單獨放入一個目錄中，方便部署及管理
mv node-exporter-* node-exporter
mv alertmanager-* alertmanager
mv kube-state-metrics-* kube-state-metrics
mv prometheus-* prometheus

 

 

 

3  安裝prometheus-server

3.1  PV和PVC

如果沒有動態PVC，手動創建

apiVersion: v1
kind: PersistentVolume
metadata:
  name: prometheus-data
spec:
  capacity:
    storage: 25Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Recycle
  storageClassName: prometheus-data   #storageClassName與prometheus-statefulset.yaml中volumeClaimTemplates下定義的需要保持一致
  nfs:
    path: /data/volumes/v2
    server: 10.6.76.25

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: prometheus-data
  namespace: kube-system
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  storageClassName: prometheus-data

[root@k8s-master prometheus]# kubectl apply -f pv-pvc.yaml
[root@k8s-master prometheus]# kubectl get pv,pvc|grep prometheus-data

persistentvolume/prometheus-data                                               25Gi       RWO            Recycle          Bound    kube-system/prometheus-data                prometheus-data                    18h

修改statefulset.yaml
# 刪掉最下面的10行
  volumeClaimTemplates:
  - metadata:
      name: prometheus-data
    spec:
      storageClassName: standard
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: "16Gi"

# 新增下面3行，注意縮進
        - name: prometheus-data
          persistentVolumeClaim:  
            claimName: prometheus-data

 

有nfs那種自動創建的

  volumeClaimTemplates:
  - metadata:
      name: prometheus-data
    spec:
 storageClassName: managed-nfs-storage #storageClassName: standard
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: "16Gi"

 

3.2  設置外網訪問

默認prometheus的service端口類型為ClusterIP，為了可以集群外訪問，修改為NodePort

#[root@k8s-master prometheus]# cat prometheus-service.yaml
kind: Service
apiVersion: v1
metadata:
  name: prometheus
  namespace: kube-system
  labels:
    kubernetes.io/name: "Prometheus"
    kubernetes.io/cluster-service: "true"
    addonmanager.kubernetes.io/mode: Reconcile
spec:
 type: NodePort
  ports:
    - name: http
      port: 9090
      protocol: TCP
      targetPort: 9090
  selector:
k8s-app: Prometheus

 

3.3 部署

[root@k8s-master prometheus]# kubectl apply -f  Prometheus
configmap/prometheus-config created
serviceaccount/prometheus created
clusterrole.rbac.authorization.k8s.io/prometheus created
clusterrolebinding.rbac.authorization.k8s.io/prometheus created
service/prometheus created
statefulset.apps/prometheus created
[root@k8s-master prometheus]#
[root@k8s-master prometheus]# kubectl get  all -n kube-system | grep prometheus

pod/prometheus-0                            2/2     Running   0          63m


service/prometheus             ClusterIP   10.111.156.156   <none>        9090:31500/TCP                 63m




statefulset.apps/prometheus   1/1     63m


[root@k8s-master prometheus]#

 

4  訪問prometheus(node節點IP：端口)