環境：centos7.6、kubernet1.17.2
離線包下載：鏈接: https://pan.baidu.com/s/1SDdwYQg0pefMI-IFA_j0QQ 提取碼: hfhi
kubernet1.17.2安裝可見：https://www.cnblogs.com/wukc/p/13280685.html
流程：安裝helm->storge安裝使用nfs模式-> 安裝kubesphere
1、安裝helm
將文件/helm/helm 放到/usr/local/bin 目錄下 將/helm/tiller.tar 進行docker load -i tiller.tar ，並推送到harbor倉庫(如果沒有harbor倉庫可以參考https://www.cnblogs.com/wukc/p/13273488.html進行安裝) 命令創建harbor項目 curl -u "admin:harborpasswd" -X POST -H "Content-Type: application/json" "172.21.210.20/api/projects" -d "{ \"project_name\": \"tiller\", \"public\": 1}" #harbor密碼 將index.yaml 放到一個http://172.21.210.19上 172.21.210.20/tiller/tiller:v2.16.3 生成認證文件kubectl apply -f helm-rbac.yml 安裝helm-server helm init --service-account tiller --tiller-image=172.21.210.20/tiller/tiller:v2.16.3 --stable-repo-url http://172.21.210.19 驗證 helm version 、 helm ls # http://172.21.210.19/這個是我用apache http做的，這兒下面放了一個文件index.yaml文件，可以從這兒下載到 https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts/index.yaml yum -y install socat
[root@master1 ~]# helm version
Client: &version.Version{SemVer:"v2.16.3", GitCommit:"1ee0254c86d4ed6887327dabed7aa7da29d7eb0d", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.16.3", GitCommit:"1ee0254c86d4ed6887327dabed7aa7da29d7eb0d", GitTreeState:"clean"}
2、storge安裝使用nfs模式
配置nfs服務器
yum install -y nfs-utils rpcbind #自己選擇一個服務器作為nfs服務器，不一定用master，下面操作都在nfs服務器上操作 systemctl start nfs && systemctl enable nfs systemctl start rpcbind 創建磁盤並掛載到目錄data1 #有磁盤空間就不需要進行掛載盤操作，可以跳過 pvcreate /dev/vdb vgcreate vg_data /dev/vdb lvcreate -l 100%VG -n lv_nfs vg_data mkdir /data1 mkfs.xfs /dev/mapper/vg_data-lv_nfs echo "UUID=`lsblk -f|grep vg_data-lv_nfs|awk '{print $3}'` /data xfs defaults 0 2" >> /etc/fstab more /etc/fstab mount -a
配置k8s使用的nfs存儲目錄
mkdir -p /data/k8s
chmod 755 /data/k8s
vim /etc/exports
/data/k8s *(rw,sync,no_root_squash)
systemctl restart nfs
3、k8s配置storge
加載鏡像 docker load -i nfs-client.tar 並上傳harbor 172.21.210.20/nfs/nfs-client-provisioner:latest curl -u "admin:harborpasswd" -X POST -H "Content-Type: application/json" "172.21.210.20/api/projects" -d "{ \"project_name\": \"nfs\", \"public\": 1}" 加載class.yaml rbac.yaml kubectl apply -f . 修改deployment.yaml鏡像 和nfs ip 並啟動 #這兒注意要修改的 標記一個默認的 StorageClass #kubectl patch storageclass （stor名字） -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}' kubectl patch storageclass es-data-db -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}' 驗證 kubectl get storageclass [root@master1 nfs]# kubectl get storageclass NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE es-data-db (default) fuseim.pri/ifs Delete Immediate false 5m26s
4、下載kubesphere的離線包並還原images
執行以下命令，下載7.2G映像包。 curl -L https://kubesphere.io/download/images/latest > kubesphere-all-images-v2.1.1.tar.gz \ && tar -zxf kubesphere-all-images-v2.1.1.tar.gz && cd kubesphere-images-v2.1.1 將映像包加載到docker中。如果您只想設置默認的最小安裝，只需加載ks_minimal_images。具有所有可選組件和示例的安裝都需要按以下方式加載所有映像。 docker load < ks_minimal_images.tar docker load < openpitrix_images.tar docker load < ks_logging_images.tar docker load < ks_devops_images.tar docker load < istio_images.tar docker load < ks_notification_images.tar docker load < example_images.tar
5、下載項目，創建harbor項目，推送images到harbor
將項目克隆ks-installer到本地，然后進入scripts文件夾。 git clone https://github.com/kubesphere/ks-installer.git cd ks-installer/scripts 由於我們必須將一批images推送到Harbor的不同項目，因此我們可以使用以下腳本來創建相應的項目。 vi create_project_harbor.sh .... url="http://172.21.210.20" #替換自己的harbor倉庫 user="admin" passwd="Harbor12345" .... chmod u+x create_project_harbor.sh && ./create_project_harbor.sh 執行以下腳本，將我們上面已加載的圖像批量推送到Harbor注冊表。 chmod u+x push-image-list.sh && ./push-image-list.sh 172.21.210.20
6、修改配置文件進行安裝
編輯kubesphere-minimal.yaml(最小化安裝)或kubesphere-complete-setup.yaml(全安裝) ··· etcd: monitoring: False endpointIps: 172.21.210.4,172.21.210.5,172.21.210.6 #改成自己的etcd地址 port: 2379 ............ tlsEnable: True alerting: enabled: true local_registry: 172.21.210.20:80 #這兒添加一行 Add a new field of Harbor address to this line. kind: ConfigMap ···.............. containers: - name: installer image: 172.21.210.20/kubesphere/ks-installer:v2.1.1 #修改成倉庫鏡像 imagePullPolicy: "Always"
7、安裝、查看日志
下面進行執行安裝 kubectl apply -f kubesphere-complete-setup.yaml 驗證安裝 kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f
8、問題處理
問題一：安裝完成后發現pod 沒正常 kubectl describe pods uc-jenkins-update-center-59f5b6c65d-dkd2w -n=kubesphere-devops-system #查看pod日志 發現報錯是沒有找到鏡像，進行推送鏡像 docker tag kubesphere/jenkins-uc:v2.1.1 172.21.210.20/kubesphere/jenkins-uc:v2.1.1 docker push 172.21.210.20/kubesphere/jenkins-uc:v2.1.1 pods節點等一會兒自動ok ---------------------------------------------------------------------------------------------------------------- 問題二：第一次失敗進行kubectl delete -f kubesphere-complete-setup.yaml ，再次執行創建失敗報錯： rror from server (Forbidden): error when creating "kubesphere-complete-setup.yaml": configmaps "ks-installer" is forbidden: unable to create new content in namespace kubesphere-system because it is being terminated Error from server (Forbidden): error when creating "kubesphere-complete-setup.yaml": serviceaccounts "ks-installer" is forbidden: unable to create new content in namespace kubesphere-system because it is being terminated Error from server (Forbidden): error when creating "kubesphere-complete-setup.yaml": deployments.apps "ks-installer" is forbidden: unable to create new content in namespace kubesphere-system because it is being terminated 原因是：namespaces沒有刪除到，進行強制刪除命名空間 查看命名空間 # kubectl get ns NAME STATUS AGE default Active 15h kube-node-lease Active 15h kube-public Active 15h kube-system Active 15h kubesphere-system Terminating 28m 發現kubesphere-system一直處於Terminating 狀態。無法刪除命名空間！！ 解決方法 kubectl get ns kubesphere-system -o json > kubesphere-system.json 將 "spec": { "finalizers": [ "kubernetes" ] }, 更改為： "spec": { }, 新開一個窗口運行kubectl proxy跑一個API代理在本地的8081端口 # kubectl proxy --port=8081 最后運行curl命令進行刪除 curl -k -H "Content-Type:application/json" -X PUT --data-binary @kubesphere-system.json http://127.0.0.1:8081/api/v1/namespaces/kubesphere-system/finalize # kubectl get ns ------------------------------------------------------------------------------------------------------------------------------------------------------------ 問題三：
這個錯誤是集群里的metrics-server不正常造成的，可以把集群里的metrics-server刪掉，如果metrics-server是helm安裝的用helm del --purge xxx刪除
helm list #進行查看
helm del --purge metrics-server
9、kubesphere安裝完成后提示：
************************************************** ##################################################### ### Welcome to KubeSphere! ### ##################################################### Console: http://172.21.210.4:30880 Account: admin Password: P@88w0rd NOTES： 1. After logging into the console, please check the monitoring status of service components in the "Cluster Status". If the service is not ready, please wait patiently. You can start to use when all components are ready. 2. Please modify the default password after login. #####################################################