前提：已搭建好kubernetes集群、安裝完dashboard
默認安裝的dashboard無法展示集群的度量指標信息，此時就需要安裝heapster插件
Heapster 插件使用包含三部分內容：
- Heapster：顯示各 Nodes、Pods 的 CPU、內存、負載等利用率曲線圖。
- InfluxDB：存儲 Pod 信息相關的數據庫, Heapster 獲取數據之后, 可以指定存儲在 InfluxDB。
- Grafana：這個主要是用於顯示 InfluxDB 里面的數據情況, 可以讓我們很直觀看到數據變化。
1：InfluxDB安裝
1.1：下載
##離線安裝
wget https://repos.influxdata.com/rhel/6Server/x86_64/stable/influxdb-1.2.0.x86_64.rpm rpm -ivh influxdb-1.2.0.x86_64.rpm
1.2：修改配置
InfluxDB 1.1開始WEB管理默認是禁用的，所以裝完並沒有啟用8083端口了，需要到配置文件里啟用。
進入到influxDB的配置文件目錄
cd /etc/influxdb/
修改influxdb.conf（紅色字體為改動部分）
[root@MyCentos7 influxdb]# vim influxdb.conf
##主要開放8086和8083端口
[admin] # Determines whether the admin service is enabled. enabled = true # The default bind address used by the admin service. bind-address = ":8083" [http] # Determines whether HTTP endpoint is enabled. enabled = true # The bind address used by the HTTP service. bind-address = ":8086"
##啟動后TCP端口:8083 為InfluxDB 管理控制台
##TCP端口:8086 為客戶端和InfluxDB通信時的HTTP API
1.3：啟動influxDB
systemctl start influxdb
1.4：驗證
1.4.1：輸入influx，檢測influxDB是否正常運行
[root@MyCentos7 influxdb]# influx Connected to http://localhost:8086 version 1.2.0 InfluxDB shell version: 1.2.0 >
1.4.2：查看web界面是否可用
在瀏覽器輸入網址之后如果web頁面能夠正常顯示就可以
http://192.168.126.130:8083
成功
2：Heapster安裝
2.1：下載
到 heapster release 頁面（https://github.com/kubernetes/heapster/releases）下載最新版本的 heapster【使用yaml文件安裝heapster】
wget -P /service/docker/k8s https://github.com/kubernetes/heapster/archive/v1.5.3.zip unzip v1.5.3.zip cd heapster-1.5.3/deploy/kube-config/influxdb
2.2：修改配置
修改heapster.yaml（紅色字體為改動部分）
apiVersion: v1 kind: ServiceAccount metadata: name: heapster namespace: kube-system --- apiVersion: extensions/v1beta1 kind: Deployment metadata: name: heapster namespace: kube-system spec: replicas: 1 template: metadata: labels: task: monitoring k8s-app: heapster spec: serviceAccountName: heapster containers: - name: heapster image: daocloud.io/liukuan73/heapster-amd64:v1.5.2 ##國外鏡像訪問不了，改成daocloud的鏡像 imagePullPolicy: IfNotPresent command: - /heapster - --source=kubernetes:http://192.168.126.130:8080?inClusterConfig=false ##注意是http，下文會解釋改動原因 - --sink=influxdb:http://192.168.126.130:8086 ##此處為influxdb所在服務器的地址，默認8086端口 --- apiVersion: v1 kind: Service metadata: labels: task: monitoring # For use as a Cluster add-on (https://github.com/kubernetes/kubernetes/tree/master/cluster/addons) # If you are NOT using this as an addon, you should comment out this line. kubernetes.io/cluster-service: 'true' kubernetes.io/name: Heapster name: heapster namespace: kube-system spec: ports: - port: 80 targetPort: 8082 selector: k8s-app: heapster
2.3：安裝
[root@MyCentos7 heapster]# kubectl create -f heapster.yaml
serviceaccount "heapster" created
deployment "heapster" created
service "heapster" created
2.4：驗證
檢測pods狀態
[root@MyCentos7 heapster]# kubectl get pods -n kube-system NAME READY STATUS RESTARTS AGE heapster-3287734661-q2vdh 1/1 Running 0 3h kubernetes-dashboard-2094756401-k09kb 1/1 Running 4 2d
檢查 kubernets dashboard 界面，看是顯示各 Nodes、Pods 的 CPU、內存、負載等利用率曲線圖；
2.5：說明
heapster與influxdb相互配置之后，Heapster容器單獨啟動時，會連接influxdb，並創建名為k8s的數據庫！！！！！
3：Grafana安裝
3.1：下載安裝
wget https://s3-us-west-2.amazonaws.com/grafana-releases/release/grafana-5.1.3-1.x86_64.rpm sudo yum localinstall grafana-5.1.3-1.x86_64.rpm
grafana目錄結構：
/usr/sbin/grafana-server
/etc/init.d/grafana-server ##上述命令的拷貝，啟動腳本 /etc/sysconfig/grafana-server ##環境變量 /etc/grafana/grafana.ini ##配置文件 /var/log/grafana/grafana.log ##日志文件 /var/lib/grafana/grafana.db ##sqlite3數據庫
3.2：啟動
[root@MyCentos7 grafana]# systemctl start grafana-server.service ##啟動 [root@MyCentos7 grafana]# systemctl enable grafana-server.service ##開機自啟動
[root@MyCentos7 grafana]# systemctl status grafana-server.service ##查看狀態 ● grafana-server.service - Grafana instance Loaded: loaded (/usr/lib/systemd/system/grafana-server.service; enabled; vendor preset: disabled) Active: active (running) since 二 2018-06-19 09:29:10 CST; 1h 25min ago Docs: http://docs.grafana.org Main PID: 1133 (grafana-server) Memory: 41.1M CGroup: /system.slice/grafana-server.service └─1133 /usr/sbin/grafana-server --config=/etc/grafana/grafana.ini --pidfile=/var/run/grafana/grafana-serv... 6月 19 09:29:09 MyCentos7 grafana-server[1133]: t=2018-06-19T09:29:09+0800 lvl=info msg="Executing migration" l...ser" 6月 19 09:29:09 MyCentos7 grafana-server[1133]: t=2018-06-19T09:29:09+0800 lvl=info msg="Skipping migration con...ser" 6月 19 09:29:09 MyCentos7 grafana-server[1133]: t=2018-06-19T09:29:09+0800 lvl=info msg="Starting plugin search...gins 6月 19 09:29:10 MyCentos7 grafana-server[1133]: t=2018-06-19T09:29:10+0800 lvl=info msg="Initializing Alerting"...gine 6月 19 09:29:10 MyCentos7 grafana-server[1133]: t=2018-06-19T09:29:10+0800 lvl=info msg="Initializing CleanUpSe...anup 6月 19 09:29:10 MyCentos7 systemd[1]: Started Grafana instance. 6月 19 09:29:11 MyCentos7 grafana-server[1133]: t=2018-06-19T09:29:11+0800 lvl=info msg="Initializing Stream Manager" 6月 19 09:29:11 MyCentos7 grafana-server[1133]: t=2018-06-19T09:29:11+0800 lvl=info msg="Initializing HTTP Serv...ket= 6月 19 09:31:11 MyCentos7 grafana-server[1133]: t=2018-06-19T09:31:11+0800 lvl=info msg="Request Completed" log...rer= 6月 19 09:31:11 MyCentos7 grafana-server[1133]: t=2018-06-19T09:31:11+0800 lvl=info msg="Request Completed" log...rer= Hint: Some lines were ellipsized, use -l to show in full.
3.3：查看端口（默認3000）
[root@MyCentos7 grafana]# netstat -tnlp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 1/systemd tcp 0 0 192.168.122.1:53 0.0.0.0:* LISTEN 1588/dnsmasq tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1125/sshd tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN 1126/cupsd tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1366/master tcp 0 0 127.0.0.1:6010 0.0.0.0:* LISTEN 2124/sshd: root@pts tcp6 0 0 :::6443 :::* LISTEN 1577/kube-apiserver tcp6 0 0 :::2379 :::* LISTEN 1138/etcd tcp6 0 0 :::10251 :::* LISTEN 718/kube-scheduler tcp6 0 0 :::2380 :::* LISTEN 1138/etcd tcp6 0 0 :::10252 :::* LISTEN 804/kube-controller tcp6 0 0 :::111 :::* LISTEN 1/systemd tcp6 0 0 :::8080 :::* LISTEN 1577/kube-apiserver tcp6 0 0 :::8083 :::* LISTEN 1132/influxd tcp6 0 0 :::8086 :::* LISTEN 1132/influxd tcp6 0 0 :::22 :::* LISTEN 1125/sshd tcp6 0 0 ::1:631 :::* LISTEN 1126/cupsd tcp6 0 0 :::3000 :::* LISTEN 1133/grafana-server tcp6 0 0 :::8088 :::* LISTEN 1132/influxd tcp6 0 0 ::1:25 :::* LISTEN 1366/master tcp6 0 0 ::1:6010 :::* LISTEN 2124/sshd: root@pts tcp6 0 0 :::4001 :::* LISTEN 1138/etcd tcp6 0 0 :::9994 :::* LISTEN 1816/docker-proxy-c [root@MyCentos7 grafana]#
3.4：訪問
瀏覽器打開，http://192.168.126.130:3000
默認用戶名密碼，admin/admin
4：grafana配置influxdb+heapster
4.1：配置influxdb數據源（heapster在influxdb中已生成名為k8s的數據庫）
保存。
有可能出現的報錯：
1、保存並測試時報錯：Network Error: Bad Gateway(502)是數據庫http的ip配置問題幾個都設置成127.0.0.1
2、提示不是私密鏈接不讓保存
4.2導入配置模板
4.2.1：導入步驟
參見：http://docs.grafana.org/reference/export_import/
4.2.2 官方集成heapster和influxdb的模板
模板1：Kubernetes Node Statistics ( via Heapster and Influxdb )
下載地址：https://grafana.com/dashboards/3646
模板2：Kubernetes Pod Statistics ( via Heapster and Influxdb )
下載地址：https://grafana.com/dashboards/3649
4.3：自定義展示樣式
根據自己需要，配置想要展示的圖形。
參見grafana官方樣式demo：
遇到的問題
安裝
執行heapster.yaml之后發現此pod報錯
[root@MyCentos7 heapster]# kubectl get pod -n kube-system 【查看】 NAME READY STATUS RESTARTS AGE heapster-536928098-lq6rw 0/1 CrashLoopBackOff 1 13s kubernetes-dashboard-2094756401-k09kb 1/1 Running 3 1d monitoring-grafana-994836776-7v5vs 1/1 Running 0 2h monitoring-influxdb-528312800-c5mc5 1/1 Running 0 2h
查看詳細信息
[root@MyCentos7 heapster]# kubectl describe pod heapster-536928098-lq6rw -n kube-system 【查看詳情】 . . . Events: FirstSeen LastSeen Count From SubObjectPath Type Reason Message --------- -------- ----- ---- ------------- -------- ------ ------- 43s 43s 1 {default-scheduler } Normal Scheduled Successfully assigned heapster-536928098-lq6rw to mycentos7-1 42s 42s 1 {kubelet mycentos7-1} spec.containers{heapster} Normal Created Created container with docker id 5ebff0e56512; Security:[seccomp=unconfined] 42s 42s 1 {kubelet mycentos7-1} spec.containers{heapster} Normal Started Started container with docker id 5ebff0e56512 41s 41s 1 {kubelet mycentos7-1} spec.containers{heapster} Normal Started Started container with docker id 942b1e9a5475 41s 41s 1 {kubelet mycentos7-1} spec.containers{heapster} Normal Created Created container with docker id 942b1e9a5475; Security:[seccomp=unconfined] 40s 39s 2 {kubelet mycentos7-1} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "heapster" with CrashLoopBackOff: "Back-off 10s restarting failed container=heapster pod=heapster-536928098-lq6rw_kube-system(ae2c26f4-6a44-11e8-bca6-000c29db8621)" 43s 24s 4 {kubelet mycentos7-1} Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Falling back to DNSDefault policy. 42s 24s 3 {kubelet mycentos7-1} spec.containers{heapster} Normal Pulled Container image "daocloud.io/megvii/heapster-amd64:v1.5.1" already present on machine 24s 24s 1 {kubelet mycentos7-1} spec.containers{heapster} Normal Created Created container with docker id 8722d44da2f3; Security:[seccomp=unconfined] 24s 24s 1 {kubelet mycentos7-1} spec.containers{heapster} Normal Started Started container with docker id 8722d44da2f3 40s 12s 4 {kubelet mycentos7-1} spec.containers{heapster} Warning BackOff Back-off restarting failed docker container 24s 12s 2 {kubelet mycentos7-1} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "heapster" with CrashLoopBackOff: "Back-off 20s restarting failed container=heapster pod=heapster-536928098-lq6rw_kube-system(ae2c26f4-6a44-11e8-bca6-000c29db8621)"
以為是鏡像的問題，就換了一個鏡像，再重新安裝一遍但是又報錯
[root@MyCentos7 heapster]# kubectl delete -f heapster.yaml 【刪除原來已安裝的pod】 serviceaccount "heapster" deleted deployment "heapster" deleted service "heapster" deleted [root@MyCentos7 heapster]# vim heapster.yaml 【修改鏡像地址】 [root@MyCentos7 heapster]# kubectl create -f heapster.yaml 【再次安裝】 serviceaccount "heapster" created deployment "heapster" created service "heapster" created [root@MyCentos7 heapster]# kubectl get pod -n kube-system 【查看】 NAME READY STATUS RESTARTS AGE heapster-3191791685-36chj 0/1 ContainerCreating 0 8s kubernetes-dashboard-2094756401-k09kb 1/1 Running 3 1d monitoring-grafana-994836776-7v5vs 1/1 Running 0 2h monitoring-influxdb-528312800-c5mc5 1/1 Running 0 2h
再次查看安裝詳情
[root@MyCentos7 heapster]# kubectl describe pod heapster-3191791685-36chj -n kube-system 【查看詳情】 . . . Events: FirstSeen LastSeen Count From SubObjectPath Type Reason Message --------- -------- ----- ---- ------------- -------- ------ ------- 29s 29s 1 {kubelet mycentos7-1} spec.containers{heapster} Normal Pulling pulling image "daocloud.io/liukuan73/heapster-amd64:v1.5.2" 29s 29s 1 {default-scheduler } Normal Scheduled Successfully assigned heapster-3191791685-36chj to mycentos7-1 19s 19s 1 {kubelet mycentos7-1} spec.containers{heapster} Normal Started Started container with docker id c58314f7fb9d 19s 19s 1 {kubelet mycentos7-1} spec.containers{heapster} Normal Pulled Successfully pulled image "daocloud.io/liukuan73/heapster-amd64:v1.5.2" 19s 19s 1 {kubelet mycentos7-1} spec.containers{heapster} Normal Created Created container with docker id 0f0222e9cda7; Security:[seccomp=unconfined] 19s 19s 1 {kubelet mycentos7-1} spec.containers{heapster} Normal Started Started container with docker id 0f0222e9cda7 19s 19s 1 {kubelet mycentos7-1} spec.containers{heapster} Normal Created Created container with docker id c58314f7fb9d; Security:[seccomp=unconfined] 18s 17s 2 {kubelet mycentos7-1} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "heapster" with CrashLoopBackOff: "Back-off 10s restarting failed container=heapster pod=heapster-3191791685-36chj_kube-system(ef1a20f7-6a48-11e8-bca6-000c29db8621)" 29s 2s 4 {kubelet mycentos7-1} Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Falling back to DNSDefault policy. 18s 2s 3 {kubelet mycentos7-1} spec.containers{heapster} Warning BackOff Back-off restarting failed docker container 19s 2s 2 {kubelet mycentos7-1} spec.containers{heapster} Normal Pulled Container image "daocloud.io/liukuan73/heapster-amd64:v1.5.2" already present on machine 2s 2s 1 {kubelet mycentos7-1} spec.containers{heapster} Normal Created Created container with docker id 559f12cda043; Security:[seccomp=unconfined] 2s 2s 1 {kubelet mycentos7-1} spec.containers{heapster} Normal Started Started container with docker id 559f12cda043 2s 2s 1 {kubelet mycentos7-1} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "heapster" with CrashLoopBackOff: "Back-off 20s restarting failed container=heapster pod=heapster-3191791685-36chj_kube-system(ef1a20f7-6a48-11e8-bca6-000c29db8621)"
里面報一樣的錯，所以驗證與鏡像無關
沒有找到解決方法，只有查看此容器的日志
[root@MyCentos7 heapster]# kubectl logs heapster-3191791685-36chj -n kube-system I0607 12:06:16.258534 1 heapster.go:78] /heapster --source=kubernetes:https://kubernetes.default --sink=influxdb:http://monitoring-influxdb.kube-system.svc:8086 I0607 12:06:16.258587 1 heapster.go:79] Heapster version v1.5.2 F0607 12:06:16.258645 1 heapster.go:183] Failed to create source provide: open /var/run/secrets/kubernetes.io/serviceaccount/token: no such file or directory
發現問題所在，趕緊查找解決方法
解決方法：
修改配置文件heapster.yaml
把 - --source=kubernetes:https://kubernetes.default 改成 - --source=kubernetes:http://《你的apiserver地址:相應的端口》?inClusterConfig=false
inClusterConfig=false : 不使用service accounts中的kube config信息；
於是再次刪除已安裝的pod，之后修改配置文件重新安裝
[root@MyCentos7 heapster]# kubectl delete -f heapster.yaml serviceaccount "heapster" deleted deployment "heapster" deleted service "heapster" deleted [root@MyCentos7 heapster]# vim heapster.yaml [root@MyCentos7 heapster]# kubectl create -f heapster.yaml serviceaccount "heapster" created deployment "heapster" created service "heapster" created [root@MyCentos7 heapster]# kubectl get pods -n kube-system NAME READY STATUS RESTARTS AGE heapster-4233553026-0t8nb 1/1 Running 0 14s kubernetes-dashboard-2094756401-k09kb 1/1 Running 3 1d monitoring-grafana-994836776-7v5vs 1/1 Running 0 3h monitoring-influxdb-528312800-c5mc5 1/1 Running 0 3h
成功
參考：
https://tonybai.com/2017/01/20/integrate-heapster-for-kubernetes-dashboard/