【四】K8s集群—HarBor 私有倉庫部署

一、概述

前面我們把 K8s 集群部署好了，但是每次拉取鏡像時都通過公有鏡像倉庫拉取非常慢，效率不高，於是我們需要在本地搭建一個私有鏡像倉庫來提供 K8s 集群使用，這樣我們提交鏡像和拉取鏡像時就非常方便，速度也快。

Kubernetes 集群部署-kubeadm方式 這是前一篇 K8s 集群部署筆記。

二、准備工作

設置主機名

[root@localhost ~]# hostnamectl  set-hostname harbor

添加 Host 解析

[root@harbor ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.115.11 k8s-master01
192.168.115.12 k8s-node01
192.168.115.13 k8s-node02
192.168.115.14 hub.test.com

k8s 集群每個節點添加解析（注意：K8s 每個節點，不是 Harbor）

[root@k8s-master01 ~]# echo "192.168.115.14 hub.test.com" >> /etc/hosts
[root@k8s-node01 ~]# echo "192.168.115.14 hub.test.com" >> /etc/hosts
[root@k8s-node02 ~]# echo "192.168.115.14 hub.test.com" >> /etc/hosts

安裝 Docker

[root@harbor ~]# yum install -y yum-utils device-mapper-persistent-data lvm2
[root@harbor ~]# yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
[root@harbor ~]# yum update -y && yum install -y docker-ce

創建 /etc/docker 目錄

[root@harbor ~]# mkdir /etc/docker
# 配置 daemon.json
cat > /etc/docker/daemon.json <<EOF
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
      "max-size": "100m"  
  },
  "insecure-registries": ["https://hub.test.com"]
}
EOF

[root@harbor ~]# mkdir -p /etc/systemd/system/docker.service.d

重啟 docker 服務

[root@harbor ~]# systemctl daemon-reload && systemctl restart docker && systemctl enable docker

K8s 集群每個節點都需要添加（注意：K8s 每個節點，不是 Harbor）

[root@k8s-master01 ~]# cat /etc/docker/daemon.json 
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
      "max-size": "100m"  
  },	# 注意這里有個英文逗號！！！
  "insecure-registries": ["https://hub.test.com"]
}

三、創建 https 證書

安裝 openssl

[root@harbor]# yum install openssl -y

創建證書目錄，並賦予權限

[root@harbor ~]# mkdir -p /cert/harbor
[root@harbor ~]# chmod -R 777 /cert/harbor
[root@harbor ~]# cd /cert/harbor

創建服務器證書密鑰文件 harbor.key

[root@harbor harbor]# openssl genrsa -des3 -out harbor.key 2048

輸入密碼，確認密碼，自己隨便定義，但是要記住，后面會用到。

創建服務器證書的申請文件 harbor.csr

[root@harbor harbor]# openssl req -new -key harbor.key -out harbor.csr

輸入密鑰文件的密碼, 然后一路回車。

備份一份服務器密鑰文件

[root@harbor harbor]# cp harbor.key harbor.key.org

去除文件口令

[root@harbor harbor]# openssl rsa -in harbor.key.org -out harbor.key

輸入密鑰文件的密碼

創建一個自當前日期起為期十年的證書 harbor.crt

[root@harbor harbor]# openssl x509 -req -days 3650 -in harbor.csr -signkey harbor.key -out harbor.crt

四、安裝 Harbor

下載 Harbor 包

下載鏈接：https://github.com/goharbor/harbor/releases

解壓並安裝 harbor

[root@harbor ~]# tar -zxvf harbor-offline-installer-v2.2.2.tgz
[root@harbor ~]# mv harbor /usr/local/
[root@harbor ~]# cd /usr/local/harbor/

修改配置文件

[root@harbor ~]# cd /usr/local/harbor/
[root@harbor harbor]# ls
common.sh  harbor.v2.2.2.tar.gz  harbor.yml.tmpl  install.sh  LICENSE  prepare
[root@harbor harbor]# cp harbor.yml.tmpl harbor.yml
[root@harbor harbor]# vim harbor.yml

需要修改的地方如下，其他的默認即可

將http端口改成10080，因為默認用的80端口已經被占用，http可以指定任意端口；

開始安裝 harbor

[root@harbor harbor]# ./install.sh 
[Step 0]: checking if docker is installed ...

Note: docker version: 20.10.6

[Step 1]: checking docker-compose is installed ...

Note: docker-compose version: 1.23.1

[Step 2]: loading Harbor images ...
Loaded image: goharbor/harbor-jobservice:v2.2.2
Loaded image: goharbor/harbor-exporter:v2.2.2
Loaded image: goharbor/nginx-photon:v2.2.2
Loaded image: goharbor/trivy-adapter-photon:v2.2.2
Loaded image: goharbor/prepare:v2.2.2
Loaded image: goharbor/harbor-db:v2.2.2
Loaded image: goharbor/harbor-registryctl:v2.2.2
Loaded image: goharbor/notary-server-photon:v2.2.2
Loaded image: goharbor/notary-signer-photon:v2.2.2
Loaded image: goharbor/redis-photon:v2.2.2
Loaded image: goharbor/registry-photon:v2.2.2
Loaded image: goharbor/chartmuseum-photon:v2.2.2
Loaded image: goharbor/harbor-portal:v2.2.2
Loaded image: goharbor/harbor-core:v2.2.2
Loaded image: goharbor/harbor-log:v2.2.2
...省略部分輸出信息
[Step 5]: starting Harbor ...
Creating network "harbor_harbor" with the default driver
Creating harbor-log ... done
Creating registryctl   ... done
Creating harbor-db     ... done
Creating harbor-portal ... done
Creating redis         ... done
Creating registry      ... done
Creating harbor-core   ... done
Creating nginx             ... done
Creating harbor-jobservice ... done
✔ ----Harbor has been installed and started successfully.----

測試 Harbor，在 k8s 集群節點中下載鏡像推送到 Harbor（注意：K8s 某個節點測試）

登錄 Harbor

[root@k8s-master01 ~]# docker login https://hub.test.com
Username: admin
Password: Harbor12345 # 默認密碼，可通過 harbor.yml 配置文件修改

下載鏡像推送到 Harbor

[root@k8s-node01 ~]# docker pull nginx
[root@k8s-node01 ~]# docker tag nginx:latest hub.test.com/library/mynginx:v1
[root@k8s-node01 ~]# docker push hub.test.com/library/mynginx:v1

創建 Pod 測試

[root@k8s-master01 ~]# kubectl run nginx-deployment --image=hub.test.com/library/mynginx:v1 --port=80
[root@k8s-master01 ~]# kubectl get svc
NAME               TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
kubernetes         ClusterIP   10.96.0.1      <none>        443/TCP   131m
nginx-deployment   ClusterIP   10.102.181.9   <none>        80/TCP    9m27s
[root@k8s-master01 ~]# kubectl get pod -o wide
NAME               READY   STATUS    RESTARTS   AGE     IP           NODE         NOMINATED NODE   READINESS GATES
nginx-deployment   1/1     Running   0          9m46s   10.244.2.3   k8s-node02   <none>           <none>

訪問 Pod 測試

[root@k8s-master01 ~]# curl -i 10.244.2.3
HTTP/1.1 200 OK
Server: nginx/1.19.10
Date: Mon, 24 May 2021 08:55:12 GMT
...

五、Windows 訪問 Harbor Web界面

Windows 添加 hosts 解析路徑

C:\Windows\System32\drivers\etc\hosts

添加信息

192.168.115.14 hub.test.com

瀏覽器訪問測試

https://hub.test.com

用戶密碼：admin / Harbor12345

可以看到這是剛剛我們在 K8s 集群推送的 nginx 鏡像，下載次數為1，剛剛構建 Pod 的時候拉取了這個鏡像。