k8s-1.20集群部署

第一章  背景說明

1.   k8s調用機制

 

2.組件分布情況

master節點組件：apiserver、controller-manager、scheduler

node節點組件：kubelet、kube-proxy

其他組件：coredns、CNI網絡（flannel或者Calico）

3.組件介紹

apiserver：負責整個k8s集群內部接口消息的接收及轉發

controller-manager：負責維護集群的狀態，比如故障檢測、自動擴展、滾動新等

scheduler：負責資源的調度，按照預定的調度策略將 Pod 調度到相應的機器上

kubelet：負責維護容器的生命周期，同時也負責 Volume（CSI）和網絡（CNI）的管理

kube-proxy：負責為 Service 提供 cluster 內部的服務發現和負載均衡

coredns：負責為整個集群提供 DNS 服務

CNI網絡：負責整個集群的網絡 

第二章  部署

服務	要求
內核	內核版本在3.10+
centos/redhat系統	使用7.5以上的系統
多master節點	需要准備一個vip

 

 

 

 

 

1.環境准備 

服務器要求：

• 建議最小硬件配置：2核CPU、2G內存、40G硬盤
• 服務器最好可以訪問外網，會有從網上拉取鏡像需求，如果服務器不能上網，需要提前下載對應鏡像並導入節點

服務器整體規划：

角色	IP	其他單裝組件
k8s-master1	192.168.31.61	docker，etcd，nginx，keepalived
k8s-master2	192.168.31.62	docker，etcd，nginx，keepalived
k8s-node1	192.168.31.63	docker，etcd
負載均衡器對外IP	192.168.31.88 (VIP)

 

2.部署架構圖

3.系統初始化

安裝ansible工具

所有節點都安裝

 

 

配置ssh免密登錄

在master-1上操作

 

編輯ansible配置文件，在master-1主機上操作

 

測試是否互通

  

基本初始化操作

直接使用ansible劇本，對防火牆，主機名，swap等進行更改

 

[root@k8s-master-1 ~]# cat basic.yaml

- hosts: k8s-all

  tasks:

  - name: 關閉selinux

    shell: sed -ri 's/enforcing/disabled/' /etc/selinux/config  && setenforce 0

    ignore_errors: yes

  - name: 分發hosts文件

    copy: src=/etc/hosts dest=/etc/hosts

  - name: 分發k8s.conf

    copy: src=k8s.conf dest=/etc/sysctl.d/k8s.conf  # 注意此處src的k8s.conf需要准備好

  - name: 參數生效

    shell: sysctl --system

  - name: 關閉swap

    lineinfile:

        dest: /etc/selinux/config

        regexp: '^SELINUX'

        line: 'SELINUX=disabled'

  - name: 關閉防火牆

    service: name=firewalld state=stopped enabled=no

 

[root@k8s-master-1 ~]# cat k8s.conf

net.bridge.bridge-nf-call-ip6tables = 1

net.bridge.bridge-nf-call-iptables = 1

net.ipv4.ip_forward = 1

 

ansible-playbook basic.yaml

4.部署nginx+keepalived

只在兩台master節點上安裝,在官網上下載nginx的rpm包進行安裝

Kubernetes作為容器集群系統，通過健康檢查+重啟策略實現了Pod故障自我修復能力，通過調度算法實現將Pod分布式部署，並保持預期副本數，根據Node失效狀態自動在其他Node拉起Pod，實現了應用層的高可用性。

針對Kubernetes集群，高可用性還應包含以下兩個層面的考慮：Etcd數據庫的高可用性和Kubernetes Master組件的高可用性。 而kubeadm搭建的K8s集群，Etcd只起了一個，存在單點，所以我們這里會獨立搭建一個Etcd集群。

Master節點扮演着總控中心的角色，通過不斷與工作節點上的Kubelet和kube-proxy進行通信來維護整個集群的健康工作狀態。如果Master節點故障，將無法使用kubectl工具或者API做任何集群管理。

Master節點主要有三個服務kube-apiserver、kube-controller-manager和kube-scheduler，其中kube-controller-manager和kube-scheduler組件自身通過選擇機制已經實現了高可用，所以Master高可用主要針對kube-apiserver組件，而該組件是以HTTP API提供服務，因此對他高可用與Web服務器類似，增加負載均衡器對其負載均衡即可，並且可水平擴容。

 

 

kube-apiserver高可用架構圖：

• Nginx是一個主流Web服務和反向代理服務器，這里用四層實現對apiserver實現負載均衡。
• Keepalived是一個主流高可用軟件，基於VIP綁定實現服務器雙機熱備，在上述拓撲中，Keepalived主要根據Nginx運行狀態判斷是否需要故障轉移（偏移VIP），例如當Nginx主節點掛掉，VIP會自動綁定在Nginx備節點，從而保證VIP一直可用，實現Nginx高可用。

注：為了節省機器，這里與K8s master節點機器復用。也可以獨立於k8s集群之外部署，只要nginx與apiserver能通信就行。

 

 

安裝軟件包

在兩台master上執行

yum install epel-release -y

##nginx使用RPM包安裝

Rpm包網址：http://nginx.org/packages/rhel/7/x86_64/RPMS/

wget http://nginx.org/packages/rhel/7/x86_64/RPMS/nginx-1.20.0-1.el7.ngx.x86_64.rpm

 

修改nginx配置文件

兩台master節點上一樣

[root@k8s-master-1 ~]# cat /etc/nginx/nginx.conf

worker_processes auto;

#user root;

events {

    worker_connections  1024;

    use epoll;

}

error_log  /var/log/nginx/error.log notice;

pid        /var/run/nginx.pid;

 

stream {

    upstream kube-apiservers {

        hash  $remote_addr consistent; # 輪詢方式

        server 192.168.2.34:6443  weight=6 max_fails=1 fail_timeout=10s; #代理得url

        server 192.168.2.174:6443 weight=6 max_fails=1 fail_timeout=10s; #代理得url

    }

 

    server {

        listen 8443; # 端口

        proxy_connect_timeout 30s;

        proxy_timeout 60s;

        proxy_pass kube-apiservers;

    }

}

 

修改Keepalived配置

在master-1節點上執行，為主

yum install keepalived -y

cat > /etc/keepalived/keepalived.conf << EOF

global_defs {

   notification_email {

     [e-mail]

     [e-mail]

     [e-mail]

   }

   notification_email_from [e-mail] 

   smtp_server 127.0.0.1

   smtp_connect_timeout 30

   router_id NGINX_MASTER

}

 

vrrp_script check_nginx {

    script "/etc/keepalived/check_nginx.sh"

}

 

vrrp_instance VI_1 {

    state MASTER

    interface eth0  # 修改為實際網卡名

    virtual_router_id 51 # VRRP 路由 ID實例，每個實例是唯一的

    priority 100    # 優先級，備服務器設置 90

    advert_int 1    # 指定VRRP 心跳包通告間隔時間，默認1秒

    authentication {

        auth_type PASS     

        auth_pass 1111

    } 

    # 虛擬IP

    virtual_ipaddress {

        192.168.5.8

    }

    track_script {

        check_nginx

    }

}

EOF

 

• vrrp_script：指定檢查nginx工作狀態腳本（根據nginx狀態判斷是否故障轉移）
• virtual_ipaddress：虛擬IP（VIP）

准備上述配置文件中檢查nginx運行狀態的腳本：

cat > /etc/keepalived/check_nginx.sh  << "EOF"
#!/bin/bash

nginx_num=`ps -ef|grep ^[n]ginx|wc -l`

if [[ $nginx_num -eq 0  ]]

  then

  systemctl stop keepalived && exit 1

else

  exit 0

fi

EOF

chmod +x /etc/keepalived/check_nginx.sh

 

 

cat > /etc/keepalived/keepalived.conf << EOF

global_defs {

   notification_email {

     [e-mail]

     [e-mail]

     [e-mail]

   }

   notification_email_from [e-mail] 

   smtp_server 127.0.0.1

   smtp_connect_timeout 30

   router_id NGINX_BACKUP

}

 

vrrp_script check_nginx {

    script "/etc/keepalived/check_nginx.sh"

}

 

vrrp_instance VI_1 {

    state BACKUP

    interface eth0

    virtual_router_id 51 # VRRP 路由 ID實例，每個實例是唯一的

    priority 90

    advert_int 1

    authentication {

        auth_type PASS     

        auth_pass 1111

    } 

    virtual_ipaddress {

        192.168.5.8

    }

    track_script {

        check_nginx

    }

}

EOF

注：keepalived根據腳本返回狀態碼（0為工作正常，非0不正常）判斷是否故障轉移。

啟動nginx，keepalived

在兩台master主機上都執行如下命令

systemctl daemon-reload

systemctl start nginx

systemctl start keepalived

systemctl enable nginx

systemctl enable keepalived

 

 查看keepalived工作狀態

 

Nginx+keepalived高可用測試

關閉主節點Nginx，測試VIP是否漂移到備節點服務器。

在Nginx Master執行 pkill nginx
在Nginx Backup，ip addr命令查看已成功綁定VIP。

第三章 部署ETCD集群

Etcd 是一個分布式鍵值存儲系統，Kubernetes使用Etcd進行數據存儲，kubeadm搭建默認情況下只啟動一個Etcd Pod，存在單點故障，生產環境強烈不建議，所以我們這里使用3台服務器組建集群，可容忍1台機器故障，當然，你也可以使用5台組建集群，可容忍2台機器故障。

節點名稱	IP
K8s-master-1	192.168.5.66
K8s-master-2	192.168.5.117
K8s-node-1	192.168.5.82

 

注：為了節省機器，這里與K8s節點機器復用。也可以獨立於k8s集群之外部署，只要apiserver能連接到就行。

准備cfssl證書生成工具

cfssl是一個開源的證書管理工具，使用json文件生成證書，相比openssl更方便使用。

找任意一台服務器操作，這里用k8s-master-1節點。

wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64
chmod +x cfssl_linux-amd64 cfssljson_linux-amd64 cfssl-certinfo_linux-amd64
mv cfssl_linux-amd64 /usr/local/bin/cfssl
mv cfssljson_linux-amd64 /usr/local/bin/cfssljson
mv cfssl-certinfo_linux-amd64 /usr/bin/cfssl-certinfo

生成ETCD證書

自簽證書頒發機構（CA）

創建工作目錄：

mkdir -p ~/etcd_tls

cd ~/etcd_tls

 自簽CA：

cat > ca-config.json << EOF
{
  "signing": {
    "default": {
      "expiry": "87600h"
    },
    "profiles": {
      "www": {
         "expiry": "87600h",
         "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"
        ]
      }
    }
  }
}
EOF

cat > ca-csr.json << EOF
{
    "CN": "etcd CA",
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "Beijing",
            "ST": "Beijing"
        }
    ]
}
EOF

 

生成證書：

cfssl gencert -initca ca-csr.json | cfssljson -bare ca -

會生成ca.pem和ca-key.pem文件

 

使用自簽CA簽發Etcd HTTPS證書

創建證書申請文件：

cat > server-csr.json << EOF
{
    "CN": "etcd",
    "hosts": [
    "192.168.5.66",
    "192.168.5.117",
    "192.168.5.82"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "BeiJing",
            "ST": "BeiJing"
        }
    ]
}
EOF

 

注：上述文件hosts字段中IP為所有etcd節點的集群內部通信IP，一個都不能少！為了方便后期擴容可以多寫幾個預留的IP。

生成證書：

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server

會生成server.pem和server-key.pem文件。

從GitHub下載etcd二進制文件

下載地址：

https://github.com/etcd-io/etcd/releases/download/v3.4.9/etcd-v3.4.9-linux-amd64.tar.gz

 

部署ETCD集群

以下在節點1上操作(k8s-master-1)，為簡化操作，待會將節點1生成的所有文件拷貝到節點2(k8s-master-2)和節點3(k8s-node-1)。

創建工作目錄並解壓二進制包

mkdir –p /opt/etcd/{bin,cfg,ssl}
tar zxvf etcd-v3.4.9-linux-amd64.tar.gz
mv etcd-v3.4.9-linux-amd64/{etcd,etcdctl} /opt/etcd/bin/

 創建ETCD配置文件

cat > /opt/etcd/cfg/etcd.conf << EOF
#[Member]
ETCD_NAME="etcd-1"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.5.66:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.5.66:2379"

#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.5.66:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.5.66:2379"
ETCD_INITIAL_CLUSTER="etcd-1=https://192.168.5.66:2380,etcd-2=https://192.168.5.117:2380,etcd-3=https://192.168.5.82:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
EOF

 

說明：

• ETCD_NAME：節點名稱，集群中唯一
• ETCDDATADIR：數據目錄
• ETCDLISTENPEER_URLS：集群通信監聽地址
• ETCDLISTENCLIENT_URLS：客戶端訪問監聽地址
• ETCDINITIALADVERTISEPEERURLS：集群通告地址
• ETCDADVERTISECLIENT_URLS：客戶端通告地址
• ETCDINITIALCLUSTER：集群節點地址
• ETCDINITIALCLUSTER_TOKEN：集群Token
• ETCDINITIALCLUSTER_STATE：加入集群的當前狀態，new是新集群，existing表示加入已有集群

systemd管理etcd

cat > /usr/lib/systemd/system/etcd.service << EOF
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
EnvironmentFile=/opt/etcd/cfg/etcd.conf
ExecStart=/opt/etcd/bin/etcd \
--cert-file=/opt/etcd/ssl/server.pem \
--key-file=/opt/etcd/ssl/server-key.pem \
--peer-cert-file=/opt/etcd/ssl/server.pem \
--peer-key-file=/opt/etcd/ssl/server-key.pem \
--trusted-ca-file=/opt/etcd/ssl/ca.pem \
--peer-trusted-ca-file=/opt/etcd/ssl/ca.pem \
--logger=zap
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
EOF 

拷貝證書

把前面的證書拷貝到配置文件中的路徑：

cp ~/etcd_tls/ca*pem ~/etcd_tls/server*pem /opt/etcd/ssl/

將節點1所有生成的文件拷貝到節點2和節點3

scp -r /opt/etcd/ root@192.168.5.117:/opt/

scp /usr/lib/systemd/system/etcd.service root@192.168.5.117:/usr/lib/systemd/system/

 

scp -r /opt/etcd/ root@192.168.5.82:/opt/

scp /usr/lib/systemd/system/etcd.service root@192.168.5.82:/usr/lib/systemd/system/

 

然后在節點2和節點3分別修改etcd.conf配置文件中的節點名稱和當前服務器IP：

vi /opt/etcd/cfg/etcd.conf
#[Member]
ETCD_NAME="etcd-1"   # 修改此處，節點2改為etcd-2，節點3改為etcd-3
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.31.71:2380"   # 修改此處為當前服務器IP
ETCD_LISTEN_CLIENT_URLS="https://192.168.31.71:2379" # 修改此處為當前服務器IP

#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.31.71:2380" # 修改此處為當前服務器IP
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.31.71:2379" # 修改此處為當前服務器IP
ETCD_INITIAL_CLUSTER="etcd-1=https://192.168.31.71:2380,etcd-2=https://192.168.31.72:2380,etcd-3=https://192.168.31.73:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"

 

 

 

最后啟動etcd並設置開機啟動

注：etcd節點要一起啟動

ansible操作

ansible etcd -m service -a 'name=etcd state=started enabled=yes daemon-reload=yes'

注：etcd是在ansible里做的分組，這個組下面只有etcd這三台的主機ip

 

 

 

 查看集群狀態

ETCDCTL_API=3 /opt/etcd/bin/etcdctl --cacert=/opt/etcd/ssl/ca.pem --cert=/opt/etcd/ssl/server.pem --key=/opt/etcd/ssl/server-key.pem --endpoints="https://192.168.5.66:2379,https://192.168.5.117:2379,https://192.168.5.82:2379" endpoint health --write-out=table

 

如果輸出上面信息，就說明集群部署成功。

如果有問題第一步先看日志：/var/log/message 或 journalctl -u etcd

 

查看etcd主備

ETCDCTL_API=3 /opt/etcd/bin/etcdctl --cacert=/opt/etcd/ssl/ca.pem --cert=/opt/etcd/ssl/server.pem --key=/opt/etcd/ssl/server-key.pem --endpoints="https://192.168.5.66:2379,https://192.168.5.117:2379,https://192.168.5.82:2379" endpoint status --write-out=table

 

第四章Docker安裝

安裝

所有節點全部安裝

##更換yum源

wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O /etc/yum.repos.d/docker-ce.repo

 

##yum安裝docker，所有節點都安裝

yum -y install docker-ce-19.03.15

 

##啟動並加入開機自啟動

systemctl enable docker && systemctl start docker

 

注：安裝的docker版本需要與k8s的版本做匹配，並注意k8s在1.20版本開始將要不支持docker，將於1.23版本后徹底不支持docker（即：1.23版本之前可以使用docker）

替代docker目錄比較成熟的產品：

1、podman

2、containerd   # 推薦，資料較多

 配置

[root@mater01 roles]# cat /etc/docker/daemon.json

{

  "registry-mirrors": ["https://b9pmyelo.mirror.aliyuncs.com"], # 公網鏡像加速器，內網不用配置

  "insecure-registries":["123.60.214.87:18080"], # harbor倉庫地址，無harbor不需要配置

  "data-root": "/home/docker_data"  # 修改docker存儲目錄，要查看最大存儲的磁盤在哪個目錄下

}

 

 

 

{

        "registry-mirrors":["https://b9pmyelo.mirror.aliyuncs.com"],

        "insecure-registries":["119.3.224.205:18080"],

        "data-root":"/home/docker_data"

}

##注意json文件中不能有空格

 

##創建鏡像目錄

mkdir -p /home/docker_data

##重啟

root@GQ-202-DWGL-K8SJQ02 roles]# systemctl restart docker  # 重啟

 

注：此文件需要單獨創建，10.217.54.220:18080是預選規划的地址，如harbor倉庫是https協議的就不要配置，只需要配置http協議的，修改這里任何內容，都需要重啟docker

第五章 Harbor安裝

鏡像倉庫盡量找一台單獨機器部署

下載鏡像

wget https://storage.googleapis.com/harbor-releases/release-1.8.0/harbor-offline-installer-v2.1.2.tgz

docker-compose

下載地址：curl -L https://github.com/docker/compose/releases/download/1.18.0/docker-compose-`uname -s`-`uname -m` -o /usr/bin/docker-compose

注：執行這條命令會直接給放到/usr/bin下，只需再給加執行權限即可

 

瀏覽器下載：https://github.com/docker/compose/releases/download/1.18.0/docker-compose-Linux-x86_64

 

配置

[root@master01]# cp docker-compose-Linux-x86_64 /usr/bin/docker-compose

[root@master01]# chmod +x /usr/bin/docker-compose

[root@master01]# docker-compose –version

docker-compose version 1.18.0, build 8dd22a9

[root@master01]# tar xf harbor-offline-installer-v2.1.2.tgz

[root@master01]# cd harbor

[root@master01 harbor]# cp harbor.yml.tmpl harbor.yml

 

[root@master01 harbor]# vim harbor.yml

  5 hostname: 10.217.54.220

10   port: 18080

 11

 12 # https related config

 13 #https:

 14   # https port for harbor, default is 443

 15   #port: 443

 16   # The path of cert and key files for nginx

 17   #certificate: /your/certificate/path

 18   #private_key: /your/private/key/path

27 harbor_admin_password: Harbor135

40 data_volume: /data/harbor

 

注：標紅部分需要根據實際情況修改

hostname à 訪問ip地址（可域名/IP）

port à 訪問端口（根據實際情況修改）

13-18是配置https訪問的端口及證書所在位置，如沒有，則注釋

harbor_admin_password à 登錄界面默認密碼

data_volume à 相關容器存儲目錄

 

安裝

[root@master01 harbor]# ./prepare  # 初始化准備工作

[root@master01 harbor]# ./install.sh # 輸出內容較多，省略

 

 

 

注：此步大致分為，

1、生成docker-compose.yml文件 

2、導入鏡像 

3、啟動

 

驗證

 

注：此處會默認找docker-compose.yml文件，如果當前目錄下沒有，則需要指定docker-compose.yml文件（參數-f）

 

界面驗證：

http://123.60.214.87:18080/

 

Docker登錄驗證

 

 

第六章 k8s部署

更換yum源

所有節點全部操作

cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

注：上述yum源是7系統的，如果安裝系統為8系統，則把el7替換為el8

 

安裝kubeadm、kubectl、kubelet

由於版本更新頻繁，這里指定版本號部署：所有節點都需要安裝

ansible all -m shell -a 'yum install -y kubelet-1.20.0 kubeadm-1.20.0 kubectl-1.20.0'

 

ansible all -m service -a 'name=kubelet enabled=yes'

 

ansible all -m shell -a "mkdir -p /home/kubelet_data && echo 'KUBELET_EXTRA_ARGS=--root-dir=/home/kubelet_data' > /etc/sysconfig/kubelet"

 

初始化k8s-master-1節點

cat > kubeadm-config.yaml << EOF

apiVersion: kubeadm.k8s.io/v1beta2

bootstrapTokens:

- groups:

  - system:bootstrappers:kubeadm:default-node-token

  token: 9037x2.tcaqnpaqkra9vsbw

  ttl: 24h0m0s

  usages:

  - signing

  - authentication

kind: InitConfiguration

localAPIEndpoint:

  advertiseAddress: 192.168.5.66

  bindPort: 6443

nodeRegistration:

  criSocket: /var/run/dockershim.sock

  name: k8s-master-1  #注意修改主機名

  taints:

  - effect: NoSchedule

    key: node-role.kubernetes.io/master

---

apiServer:

  certSANs:  #包含所有Master/LB/VIP IP，一個都不能少！為了方便后期擴容可以多寫幾個預留的IP。

  - k8s-master-1

  - k8s-master-2

  - 192.168.5.66

  - 192.168.5.117

  - 192.168.5.82

  - 192.168.5.8

  - 127.0.0.1

  extraArgs:

    authorization-mode: Node,RBAC

  timeoutForControlPlane: 4m0s

apiVersion: kubeadm.k8s.io/v1beta2

certificatesDir: /etc/kubernetes/pki

clusterName: kubernetes

controlPlaneEndpoint: 192.168.5.8:80 #負載均衡虛擬IP（VIP）和端口

controllerManager: {}

dns:

  type: CoreDNS

etcd:

  external:  #使用外部etcd

    endpoints:

    - https://192.168.5.66:2379 #etcd集群3個節點

    - https://192.168.5.117:2379

    - https://192.168.5.82:2379

    caFile: /opt/etcd/ssl/ca.pem #連接etcd所需證書

    certFile: /opt/etcd/ssl/server.pem

    keyFile: /opt/etcd/ssl/server-key.pem

imageRepository: registry.aliyuncs.com/google_containers #由於默認拉取鏡像地址k8s.gcr.io國內無法訪問，這里指定阿里雲鏡像倉庫地址

kind: ClusterConfiguration

kubernetesVersion: v1.19.0 #K8s版本，與上面安裝的一致

networking:

  dnsDomain: cluster.local

  podSubnet: 10.244.0.0/16  #Pod網絡，與下面部署的CNI網絡組件yaml中保持一致

  serviceSubnet: 10.96.0.0/12  #集群內部虛擬網絡，Pod統一訪問入口

scheduler: {}

EOF

 

執行初始化命令

kubeadm init --config=kubeadm-config.yaml

[root@k8s-master-1 ~]# kubeadm  init --config=kubeadm-config.yaml

[init] Using Kubernetes version: v1.20.0

[preflight] Running pre-flight checks

        [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/

[preflight] Pulling images required for setting up a Kubernetes cluster

[preflight] This might take a minute or two, depending on the speed of your internet connection

[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'

[certs] Using certificateDir folder "/etc/kubernetes/pki"

[certs] Generating "ca" certificate and key

[certs] Generating "apiserver" certificate and key

[certs] apiserver serving cert is signed for DNS names [k8s-master-1 k8s-master-2 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [172.96.0.1 192.168.5.66 192.168.5.8 192.168.5.117 192.168.5.82 127.0.0.1]

[certs] Generating "apiserver-kubelet-client" certificate and key

[certs] Generating "front-proxy-ca" certificate and key

[certs] Generating "front-proxy-client" certificate and key

[certs] External etcd mode: Skipping etcd/ca certificate authority generation

[certs] External etcd mode: Skipping etcd/server certificate generation

[certs] External etcd mode: Skipping etcd/peer certificate generation

[certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation

[certs] External etcd mode: Skipping apiserver-etcd-client certificate generation

[certs] Generating "sa" key and public key

[kubeconfig] Using kubeconfig folder "/etc/kubernetes"

[endpoint] WARNING: port specified in controlPlaneEndpoint overrides bindPort in the controlplane address

[kubeconfig] Writing "admin.conf" kubeconfig file

[endpoint] WARNING: port specified in controlPlaneEndpoint overrides bindPort in the controlplane address

[kubeconfig] Writing "kubelet.conf" kubeconfig file

[endpoint] WARNING: port specified in controlPlaneEndpoint overrides bindPort in the controlplane address

[kubeconfig] Writing "controller-manager.conf" kubeconfig file

[endpoint] WARNING: port specified in controlPlaneEndpoint overrides bindPort in the controlplane address

[kubeconfig] Writing "scheduler.conf" kubeconfig file

[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"

[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"

[kubelet-start] Starting the kubelet

[control-plane] Using manifest folder "/etc/kubernetes/manifests"

[control-plane] Creating static Pod manifest for "kube-apiserver"

[control-plane] Creating static Pod manifest for "kube-controller-manager"

[control-plane] Creating static Pod manifest for "kube-scheduler"

[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s

[apiclient] All control plane components are healthy after 11.559587 seconds

[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace

[kubelet] Creating a ConfigMap "kubelet-config-1.20" in namespace kube-system with the configuration for the kubelets in the cluster

[upload-certs] Skipping phase. Please see --upload-certs

[mark-control-plane] Marking the node k8s-master-1 as control-plane by adding the labels "node-role.kubernetes.io/master=''" and "node-role.kubernetes.io/control-plane='' (deprecated)"

[mark-control-plane] Marking the node k8s-master-1 as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]

[bootstrap-token] Using token: 9037x2.tcaqnpaqkra9vsbw

[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles

[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes

[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials

[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token

[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster

[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace

[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key

[addons] Applied essential addon: CoreDNS

[endpoint] WARNING: port specified in controlPlaneEndpoint overrides bindPort in the controlplane address

[addons] Applied essential addon: kube-proxy

 

Your Kubernetes control-plane has initialized successfully!

 

To start using your cluster, you need to run the following as a regular user:

 

  mkdir -p $HOME/.kube

  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

  sudo chown $(id -u):$(id -g) $HOME/.kube/config

 

Alternatively, if you are the root user, you can run:

 

  export KUBECONFIG=/etc/kubernetes/admin.conf

 

You should now deploy a pod network to the cluster.

Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:

  https://kubernetes.io/docs/concepts/cluster-administration/addons/

 

You can now join any number of control-plane nodes by copying certificate authorities

and service account keys on each node and then running the following as root:

 

  kubeadm join 192.168.5.8:8443 --token 9037x2.tcaqnpaqkra9vsbw \

    --discovery-token-ca-cert-hash sha256:39c7b865f08c6a0f410e5ae5adbd45a3d1e49822739acf4ab6156d05ae27d0c8 \

    --control-plane

 

Then you can join any number of worker nodes by running the following on each as root:

 

kubeadm join 192.168.5.8:8443 --token 9037x2.tcaqnpaqkra9vsbw \

    --discovery-token-ca-cert-hash sha256:39c7b865f08c6a0f410e5ae5adbd45a3d1e49822739acf4ab6156d05ae27d0c8

[root@k8s-master-1 ~]#

kubeadm init執行后，做了哪些事

[init]：指定版本進行初始化操作

[preflight] ：初始化前的檢查和下載所需要的Docker鏡像文件

[kubelet-start] ：生成kubelet的配置文件，沒有這個文件kubelet無法啟動，所以初始化之前的kubelet實際上啟動失敗。

[certificates]：生成Kubernetes使用的證書，存放在/etc/kubernetes/pki目錄中。

[kubeconfig] ：生成 KubeConfig 文件，存放在/etc/kubernetes目錄中，組件之間通信需要使用對應文件。

[control-plane]：使用/etc/kubernetes/manifest目錄下的YAML文件，安裝Master 組件。

[etcd]：使用/etc/kubernetes/manifest/etcd.yaml安裝Etcd服務。

[wait-control-plane]：等待control-plan部署的Master組件啟動。

[apiclient]：檢查Master組件服務狀態。

[uploadconfig]：更新配置

[kubelet]：使用configMap配置kubelet。

[patchnode]：更新CNI信息到Node上，通過注釋的方式記錄。

[mark-control-plane]：為當前節點打標簽，打了角色Master，和不可調度標簽，這樣默認就不會使用Master節點來運行Pod。

[bootstrap-token]：生成token記錄下來，后邊使用kubeadm join往集群中添加節點時會用到

[addons]：安裝附加組件CoreDNS和kube-proxy

 

初始化后操作

圖中圈出部分說明：

        第一部分：在本地創建操作權限認證目錄，將admin的認證復制到默認目錄

        第二部分：master節點加入集群命令

        第三部分：node幾點加入集群命令

 

拷貝kubectl使用的連接k8s認證文件到默認路徑：

 

假設配錯參數可以重新初始化操作：

kubeadm reset -f  kubeadm-config.yaml

添加其它master節點

將Master1節點生成的證書拷貝到Master2：

 scp -r /etc/kubernetes/pki/ 192.168.5.117:/etc/kubernetes/

 

復制加入master join命令在master2執行

kubeadm join 192.168.5.8:8443 --token 9037x2.tcaqnpaqkra9vsbw \

    --discovery-token-ca-cert-hash sha256:39c7b865f08c6a0f410e5ae5adbd45a3d1e49822739acf4ab6156d05ae27d0c8 \

    --control-plane

添加node節點

復制前面init初始化中加入node的命令  在其它node節點上執行

kubeadm join 192.168.5.8:8443 --token 9037x2.tcaqnpaqkra9vsbw \

    --discovery-token-ca-cert-hash sha256:39c7b865f08c6a0f410e5ae5adbd45a3d1e49822739acf4ab6156d05ae27d0c8

 

 

注：加入集群命令的token有效期為24小時，過期后重新獲取命令如下

kubeadm token create --print-join-command

注：這個token可以理解為驗證碼，在一段時間內有效，過期后，需要重新生成，但不影響以加入集群的節點

 

驗證

注：如STATUS出現NotReady狀態的是因為還沒有部署CNI網絡

 

訪問負載均衡測試

找K8s集群中任意一個節點，使用curl查看K8s版本測試，使用VIP訪問

[root@k8s-master-2 ~]# curl -k https://192.168.5.8:8443/version

{

  "major": "1",

  "minor": "20",

  "gitVersion": "v1.20.0",

  "gitCommit": "af46c47ce925f4c4ad5cc8d1fca46c7b77d13b38",

  "gitTreeState": "clean",

  "buildDate": "2020-12-08T17:51:19Z",

  "goVersion": "go1.15.5",

  "compiler": "gc",

  "platform": "linux/amd64"

 

可以正確獲取到K8s版本信息，說明負載均衡器搭建正常。

該請求數據流程：curl -> vip(nginx) -> apiserver

通過查看Nginx日志也可以看到轉發apiserver IP

 

查看整個pod狀態，看到coredns還在創建中

 

解決方法：

需要等flannel網絡打通后，coredns會自動創建好

 

部署CNI網絡

Calico是一個純三層的數據中心網絡方案，是目前Kubernetes主流的網絡方案，這里采用flannel部署

kubectl apply -f calico.yaml
kubectl get pods -n kube-system

 

驗證：

如遇到如上問題，修改kube-controller-manager.yaml和kube-scheduler.yaml文件，在兩台master都操作

[root@k8s-master-2 ~]# sed -ri 's/- --port=0/#&/' /etc/kubernetes/manifests/kube-controller-manager.yaml

[root@k8s-master-2 ~]# sed -ri 's/- --port=0/#&/' /etc/kubernetes/manifests/kube-scheduler.yaml

[root@k8s-master-2 ~]# systemctl restart kubelet.service

 

注：全部master節點都需要修改

 

再次驗證coredns：

 

k8s從鏡像倉庫中拉取鏡像

[root@k8s-master-1 ~]# kubectl create secret docker-registry docker-harbor-registry --docker-server=123.60.214.87:18080 --docker-username=admin --docker-p

assword='Harbor135'

編輯yaml文件

[root@k8s-master-1 ~]# cat nginx_test.yaml

apiVersion: apps/v1

kind: Deployment

metadata:

  name: nginx2.1

spec:

  replicas: 1

  selector:

    matchLabels:

       app: nginx2.1

  template:

    metadata:

      labels:

        app: nginx2.1

    spec:

      imagePullSecrets:

      - name: docker-harbor-registry

      containers:

      - name: nginx2.1

        image: 123.60.214.87:18080/test/nginx2.1

        command: [ "/bin/bash", "-ce", "tail -f /dev/null" ]

 

第七章 部署Dashboard

Dashboard是官方提供的一個UI，可用於基本管理K8s資源。

[root@k8s-master-1 ~]# cat dashboard.yaml

# Copyright 2017 The Kubernetes Authors.

#

# Licensed under the Apache License, Version 2.0 (the "License");

# you may not use this file except in compliance with the License.

# You may obtain a copy of the License at

#

#     http://www.apache.org/licenses/LICENSE-2.0

#

# Unless required by applicable law or agreed to in writing, software

# distributed under the License is distributed on an "AS IS" BASIS,

# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

# See the License for the specific language governing permissions and

# limitations under the License.

 

apiVersion: v1

kind: Namespace

metadata:

  name: kubernetes-dashboard

 

---

 

apiVersion: v1

kind: ServiceAccount

metadata:

  labels:

    k8s-app: kubernetes-dashboard

  name: kubernetes-dashboard

  namespace: kubernetes-dashboard

 

---

 

kind: Service

apiVersion: v1

metadata:

  labels:

    k8s-app: kubernetes-dashboard

  name: kubernetes-dashboard

  namespace: kubernetes-dashboard

spec:

  ports:

    - port: 443

      targetPort: 8443

  selector:

    k8s-app: kubernetes-dashboard

 

---

 

apiVersion: v1

kind: Secret

metadata:

  labels:

    k8s-app: kubernetes-dashboard

  name: kubernetes-dashboard-certs

  namespace: kubernetes-dashboard

type: Opaque

 

---

 

apiVersion: v1

kind: Secret

metadata:

  labels:

    k8s-app: kubernetes-dashboard

  name: kubernetes-dashboard-csrf

  namespace: kubernetes-dashboard

type: Opaque

data:

  csrf: ""

 

---

 

apiVersion: v1

kind: Secret

metadata:

  labels:

    k8s-app: kubernetes-dashboard

  name: kubernetes-dashboard-key-holder

  namespace: kubernetes-dashboard

type: Opaque

 

---

 

kind: Service

apiVersion: v1

metadata:

  labels:

    k8s-app: kubernetes-dashboard

  name: kubernetes-dashboard

  namespace: kubernetes-dashboard

spec:

  type: NodePort

  ports:

    - port: 443

      targetPort: 8443

      nodePort: 31443

  selector:

    k8s-app: kubernetes-dashboard

 

---

 

kind: ConfigMap

apiVersion: v1

metadata:

  labels:

    k8s-app: kubernetes-dashboard

  name: kubernetes-dashboard-settings

  namespace: kubernetes-dashboard

 

---

 

kind: Role

apiVersion: rbac.authorization.k8s.io/v1

metadata:

  labels:

    k8s-app: kubernetes-dashboard

  name: kubernetes-dashboard

  namespace: kubernetes-dashboard

rules:

  # Allow Dashboard to get, update and delete Dashboard exclusive secrets.

  - apiGroups: [""]

    resources: ["secrets"]

    resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs", "kubernetes-dashboard-csrf"]

    verbs: ["get", "update", "delete"]

    # Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map.

  - apiGroups: [""]

    resources: ["configmaps"]

    resourceNames: ["kubernetes-dashboard-settings"]

    verbs: ["get", "update"]

    # Allow Dashboard to get metrics.

  - apiGroups: [""]

    resources: ["services"]

    resourceNames: ["heapster", "dashboard-metrics-scraper"]

    verbs: ["proxy"]

  - apiGroups: [""]

    resources: ["services/proxy"]

    resourceNames: ["heapster", "http:heapster:", "https:heapster:", "dashboard-metrics-scraper", "http:dashboard-metrics-scraper"]

    verbs: ["get"]

 

---

 

kind: ClusterRole

apiVersion: rbac.authorization.k8s.io/v1

metadata:

  labels:

    k8s-app: kubernetes-dashboard

  name: kubernetes-dashboard

rules:

  # Allow Metrics Scraper to get metrics from the Metrics server

  - apiGroups: ["metrics.k8s.io"]

    resources: ["pods", "nodes"]

    verbs: ["get", "list", "watch"]

 

---

 

apiVersion: rbac.authorization.k8s.io/v1

kind: RoleBinding

metadata:

  labels:

    k8s-app: kubernetes-dashboard

  name: kubernetes-dashboard

  namespace: kubernetes-dashboard

roleRef:

  apiGroup: rbac.authorization.k8s.io

  kind: Role

  name: kubernetes-dashboard

subjects:

  - kind: ServiceAccount

    name: kubernetes-dashboard

    namespace: kubernetes-dashboard

 

---

 

apiVersion: rbac.authorization.k8s.io/v1

kind: ClusterRoleBinding

metadata:

  name: kubernetes-dashboard

roleRef:

  apiGroup: rbac.authorization.k8s.io

  kind: ClusterRole

  name: kubernetes-dashboard

subjects:

  - kind: ServiceAccount

    name: kubernetes-dashboard

    namespace: kubernetes-dashboard

 

---

 

kind: Deployment

apiVersion: apps/v1

metadata:

  labels:

    k8s-app: kubernetes-dashboard

  name: kubernetes-dashboard

  namespace: kubernetes-dashboard

spec:

  replicas: 1

  revisionHistoryLimit: 10

  selector:

    matchLabels:

      k8s-app: kubernetes-dashboard

  template:

    metadata:

      labels:

        k8s-app: kubernetes-dashboard

    spec:

      containers:

        - name: kubernetes-dashboard

          image: kubernetesui/dashboard:v2.0.4

          imagePullPolicy: Always

          ports:

            - containerPort: 8443

              protocol: TCP

          env:

            - name: ACCEPT_LANGUAGE

              value: zh

          args:

            - --auto-generate-certificates

            - --namespace=kubernetes-dashboard

            # Uncomment the following line to manually specify Kubernetes API server Host

            # If not specified, Dashboard will attempt to auto discover the API server and connect

            # to it. Uncomment only if the default does not work.

            # - --apiserver-host=http://my-address:port

          volumeMounts:

            - name: kubernetes-dashboard-certs

              mountPath: /certs

              # Create on-disk volume to store exec logs

            - mountPath: /tmp

              name: tmp-volume

          livenessProbe:

            httpGet:

              scheme: HTTPS

              path: /

              port: 8443

            initialDelaySeconds: 30

            timeoutSeconds: 30

          securityContext:

            allowPrivilegeEscalation: false

            readOnlyRootFilesystem: true

            runAsUser: 1001

            runAsGroup: 2001

      volumes:

        - name: kubernetes-dashboard-certs

          secret:

            secretName: kubernetes-dashboard-certs

        - name: tmp-volume

          emptyDir: {}

      serviceAccountName: kubernetes-dashboard

      nodeSelector:

        "kubernetes.io/os": linux

      # Comment the following tolerations if Dashboard must not be deployed on master

      tolerations:

        - key: node-role.kubernetes.io/master

          effect: NoSchedule

 

---

 

kind: Service

apiVersion: v1

metadata:

  labels:

    k8s-app: dashboard-metrics-scraper

  name: dashboard-metrics-scraper

  namespace: kubernetes-dashboard

spec:

  ports:

    - port: 8000

      targetPort: 8000

  selector:

    k8s-app: dashboard-metrics-scraper

 

---

 

kind: Deployment

apiVersion: apps/v1

metadata:

  labels:

    k8s-app: dashboard-metrics-scraper

  name: dashboard-metrics-scraper

  namespace: kubernetes-dashboard

spec:

  replicas: 1

  revisionHistoryLimit: 10

  selector:

    matchLabels:

      k8s-app: dashboard-metrics-scraper

  template:

    metadata:

      labels:

        k8s-app: dashboard-metrics-scraper

      annotations:

        seccomp.security.alpha.kubernetes.io/pod: 'runtime/default'

    spec:

      containers:

        - name: dashboard-metrics-scraper

          image: kubernetesui/metrics-scraper:v1.0.4

          ports:

            - containerPort: 8000

              protocol: TCP

          livenessProbe:

            httpGet:

              scheme: HTTP

              path: /

              port: 8000

            initialDelaySeconds: 30

            timeoutSeconds: 30

          volumeMounts:

          - mountPath: /tmp

            name: tmp-volume

          securityContext:

            allowPrivilegeEscalation: false

            readOnlyRootFilesystem: true

            runAsUser: 1001

            runAsGroup: 2001

      serviceAccountName: kubernetes-dashboard

      nodeSelector:

        "kubernetes.io/os": linux

      # Comment the following tolerations if Dashboard must not be deployed on master

      tolerations:

        - key: node-role.kubernetes.io/master

          effect: NoSchedule

      volumes:

        - name: tmp-volume

          emptyDir: {}

kubectl apply -f kubernetes-dashboard.yaml

# 查看部署
kubectl get pods -n kubernetes-dashboard

創建service account並綁定，默認cluster-admin為管理員集群角色

 

[root@k8s-master-1 ~]# kubectl create serviceaccount dashboard-admin -n kube-system

serviceaccount/dashboard-admin created

[root@k8s-master-1 ~]# kubectl create clusterrolebinding dashboard-admin --clusterrole=cluster-admin --serviceaccount=kube-system:dashboard-admin

clusterrolebinding.rbac.authorization.k8s.io/dashboard-admin created

[root@k8s-master-1 ~]# kubectl describe secrets -n kube-system $(kubectl -n kube-system get secret | awk '/dashboard-admin/{print $1}')

Name:         dashboard-admin-token-hcvbq

Namespace:    kube-system

Labels:       <none>

Annotations:  kubernetes.io/service-account.name: dashboard-admin

              kubernetes.io/service-account.uid: 1f70af4b-588f-4fb8-b784-10bef10db7cf

 

Type:  kubernetes.io/service-account-token

 

Data

====

ca.crt:     1066 bytes

namespace:  11 bytes

token:      eyJhbGciOiJSUzI1NiIsImtpZCI6Im5MMDlZeWZ2ZVdDM0x0RHk5YmdWOWJjeFpPUFVIUTZaOVJfM1VaUXpYcG8ifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9

uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4taGN2YnEiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3Vud

C5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiMWY3MGFmNGItNTg4Zi00ZmI4LWI3ODQtMTBiZWYxMGRiN2NmIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW5

0Omt1YmUtc3lzdGVtOmRhc2hib2FyZC1hZG1pbiJ9.i3xgiLVefiC02cn9UQTN4ZJNdoXJ0lgBuPmXm2YK7DwzokX4HVNSPMllh-drJ5NhazeQ1sVKEjQry9aACLGkM3Le26i6hPIiykeVfGjmiDO2pKbJ6uEe9u3SI5q5JuLIGKrsAF9aDAk0x0Eq637P

TRU9kJk_sTiU6gNWw9-ax5zLvX-dlzr3nSAPbudkhuou3eb0K8q9hZR9kY9QM8YZhK7ribCW6BGrT_4XeH4efdKLtrbFhQ7iyl2xgL8gjF_f5EoyzXc5k2YyfImMpuzeuJE0a_KgOxy9-Z66eGzrchRnqJ9vZbTo42i4ywW4OIrCbI-ezYVobBfxkkJ1Uh

0Q4Q

驗證登錄：使用生成的token登錄

 

 

登錄后提示  匿名用戶沒有權限，測試環境的話用下面命令解決

kubectl create clusterrolebinding test:anonymous --clusterrole=cluster-admin  --user=system:anonymous

 

 

 

 

第八章 監控組件 - Heapster 部署

1、下載官方提供的 Heapster 組件部署的 yaml 文件

# 新建文件夾，用於存放 Heapster 部署所需的 yaml 文件

mkdir heapster

cd heapster

# 獲取相關 yaml 文件

wget https://raw.githubusercontent.com/kubernetes/heapster/master/deploy/kube-config/influxdb/grafana.yaml

wget https://raw.githubusercontent.com/kubernetes/heapster/master/deploy/kube-config/influxdb/heapster.yaml

wget https://raw.githubusercontent.com/kubernetes/heapster/master/deploy/kube-config/influxdb/influxdb.yaml

wget https://raw.githubusercontent.com/kubernetes/heapster/master/deploy/kube-config/rbac/heapster-rbac.yaml

 

2、修改 yaml 中 image 的值

k8s.gcr.io 全部修改為 registry.cn-hangzhou.aliyuncs.com/google_containers

將所有yaml文件中出現k8s.gcr.io的替換

 

 

3、部署 Heapster

kubectl create -f heapster/

 

4、幾分鍾后，刷新 Dashboard 面板