參考文檔:
Kubernetes 部署metrics-server - 簡書 (jianshu.com)
kubernetes(k8s)部署wordpress示例① - 簡書 (jianshu.com)
k8s-v1.22安裝metrics-server_zhangdd的博客-CSDN博客
僅提供參考。
如有問題請點擊頁腳(聯系博主)發聵
一、需求分析
目的:本次實驗是使用kubernets平台部署WordPress博客系統,最終可對外提供訪問。
需求:1、可提供持久化存儲的MySQL;2、配置探針檢測應用是否存活;3、 配置 HPA,讓應用能夠自動應對流量高峰期。4、增加滾動更新策略,這樣可以保證在更新應用的時候,服務不會被中斷。5、把部署的應用整合到一個 YAML 文件 wordpress-all.yaml 中;6、端口映射,外部可訪問。
注意:使用pv存儲要事先搭建好NFS服務,本文不涉及NFS搭建。
二、平台規划
主機 | IP |
Master | 192.168.30.30 |
Node1 | 192.168.30.31 |
三、部署流程
1、創建PV和PVC
1.1、進入master節點的終端,先創建一個用於WordPress服務的namespace
kubectl create namespace blog //創建一個命名空間,名稱為blog
1.2、創建PV和PVC用於為MySQL提供持續存儲
新建一個WordPress文件夾,用於存放相關yaml文件
mkdir /root/wordpress
編寫PVyaml文件
#####master#####
mkdir -p /nfs/pv1
vim /root/wordpress/pv1.yaml
#####pv的yaml文件(注意修改)
apiVersion: v1
kind: PersistentVolume
metadata:
name: mypv
namespace: blog
spec:
capacity:
storage: 1Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Recycle
nfs:
path: /nfs/pv1
server: 192.168.30.30
//namespace使用創建的blog
//ReadWriteOnce 可讀寫但只能被一個節點掛載
//path: /nfs/pv 掛載到本機/nfs/pv目錄,需要事先創建好此目錄
//server: 192.168.30.30 NFS服務器的地址,根據自己實際情況填寫
kubectl apply -f /root/wordpress/pv1.yaml
1.3、創建PVC
vim /root/wordpress/pvc1.yaml
#pvc的yaml文件
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mypvc
namespace: blog
spec:
accessModes:
- ReadWriteOnce
volumeName: mypv
resources:
requests:
storage: 1Gi
kubectl apply -f /root/wordpress/pvc1.yaml
2、創建MySQL
2.1、創建MySQL的Deployment對象
#####master#####
vim /root/wordpress/mysql-db.yaml
#mysql的deploymnet的文件
//注:
//namespace使用上一步自己創建的命名空間
//創建wordpress時,mysql相關參數需與此處創建mysql時保存一致
apiVersion: apps/v1
kind: Deployment
metadata:
name: mysql-deploy
namespace: blog
labels:
apps: mysql
spec:
selector:
matchLabels:
app: mysql
template:
metadata:
labels:
app: mysql
spec:
containers:
- name: mysql
image: mysql:5.6
imagePullPolicy: IfNotPresent
ports:
- containerPort: 3306
name: dbport
env:
- name: MYSQL_ROOT_PASSWORD
value: rootPassWord
- name: MYSQL_DATABASE
value: wordpress
- name: MYSQL_USER
value: wordpress
- name: MYSQL_PASSWORD
value: wordpress
volumeMounts:
- name: db-pv
mountPath: /var/lib/mysql
volumes:
- name: db-pv
persistentVolumeClaim:
claimName: mypvc
kubectl apply -f mysql-db.yaml
2.2、創建MySQL的service
#創建mysql的service.yaml文件
vim /root/wordpress/mysql-service.yaml
apiVersion: v1
kind: Service
metadata:
name: mysql
namespace: blog
spec:
selector:
app: mysql
ports:
- name: mysqlport
protocol: TCP
port: 3306
targetPort: 3306
kubectl apply -f /root/wordpress/mysql-service.yaml
3、創建WordPress
3.1、創建 Wordpress 的 Deployment 對象,添加滾動更新策略,配置資源限額,添加探針。
對 pod 增加 liveness probe 和 rediness probe 兩個探針,每 10s 檢測一次應用是否可讀,每 3s 檢測一次應用是否存活。
存活探針liveness采用tcpSocket端口探測方式,每3s一次檢測80端口是否存活;就緒探針readiness采用httpGet方式每10s一次探測wardpress主頁的http返回值是否正常。
編輯wordpress的yaml文件
#####master#####
vim /root/wordpress/wordpressdeploy.yaml
#wordpress的deploymnet的文件
//注:
//namespace使用上一步自己創建的命名空間
//密碼都要與MySQL密碼相匹配
//host填MySQL的虛擬IP
//resources里定義資源的上限和下限
apiVersion: apps/v1
kind: Deployment
metadata:
name: wordpress-deploy
namespace: blog
labels:
app: wordpress
spec:
selector:
matchLabels:
app: wordpress
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 50%
maxUnavailable: 0
template:
metadata:
labels:
app: wordpress
spec:
containers:
- name: wordpress
image: wordpress
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
name: wdport
env:
- name: WORDPRESS_DB_HOST
value: mysql:3306
- name: WORDPRESS_DB_USER
value: wordpress
- name: WORDPRESS_DB_PASSWORD
value: wordpress
resources:
requests:
cpu: 100m
memory: 100Mi
limits:
cpu: 100m
memory: 500Mi
livenessProbe:
tcpSocket:
port: 80
initialDelaySeconds: 3
periodSeconds: 3
readinessProbe:
tcpSocket:
port: 80
initialDelaySeconds: 5
periodSeconds: 10
kubectl apply -f wordpressdeploy.yaml
注:如果部署后訪問wordpress的web界面顯示連接不上數據庫,則該k8s環境的DNS有問題,只需將以上指向Mysql的值(mysql:3306)改為(mysql服務的IP地址:3306)跳過DNS即可,如果可以連上則不用改。
3.2、創建 Wordpress 的 Service 對象
#創建wordpress的service.yaml文件
vim /root/wordpress/wordpress-service.yaml
//NodePort屬性映射端口,提供外部訪問
apiVersion: v1
kind: Service
metadata:
name: wordpress
namespace: blog
spec:
type: NodePort
selector:
app: wordpress
ports:
- name: wordpressport
protocol: TCP
port: 80
targetPort: wdport
kubectl apply -f /root/wordpress/wordpress-service.yaml
3.3、配置 HPA,讓應用能夠自動應對流量高峰期。
HPA需要metrics-server進行資源監控,如沒部署,可參考第四點進行部署。
kubectl autoscale deployment wordpress-deploy --cpu-percent=10 --min=1 --max=10 -n blog
HPA負載測試
HPA創建完畢,現在進行HPA的負載測試,查看是否能夠進行自動橫向擴展
#啟動一個用於測試的容器
kubectl run v1 -it --image=busybox /bin/sh
#登錄到容器執行如下命令
while true; do wget -q -O- http://wordpress服務的IP:端口; done
此容器會不停的去訪問去獲取wordpress的界面
重開一個終端用於查看HPA的狀態。
#重開一個終端執行以下命令
kubectl get hpa -n blog -w
kubectl get pod -n blog
幾分鍾后可以看到,自動橫向拓展了8個pod,當我停止了curl命令,不一會兒pod數量又恢復了
所以HPA成功。
3.4、主機瀏覽器嘗試訪問,http://IP:端口
此IP是節點的IP,端口通過kubectl get svc -n blog,對應查詢
http://192.168.30.30:端口/
已經可以訪問,進行安裝操作,填入對應數據庫的信息,wordpress就搭建完成。
3.5、測試滾動更新
#將wordpressdeploy.yaml中wordpress的image版本改為5.8
image: wordpress:5.8
#再次部署,
kubectl apply -f wordpressdeploy.yaml
#查看滾動更新狀態,同時不同的訪問web界面是否存在影響。
可以看到滾動更新成功,web界面依然一直可以訪問,並不受影響。
四、(補充)master節點metrics-server部署
在使用HPA的時候,k8s系統需要進行資源監控,才能使HPA生效,所有需要部署metrics-server用於監測node,pod等的CPU,內存使用情況。
1、修改apiserver文件,新增如下配置(已有項不用加)
vim /etc/kubernetes/manifests/kube-apiserver.yaml
········
- --tls-cert-file=/etc/kubernetes/pki/apiserver.crt
- --tls-private-key-file=/etc/kubernetes/pki/apiserver.key
- --requestheader-allowed-names=aggregator
- --requestheader-extra-headers-prefix=X-Remote-Extra-
- --requestheader-group-headers=X-Remote-Group
- --requestheader-username-headers=X-Remote-User
- --runtime-config=api/all=true
- --enable-aggregator-routing=true
······
#重啟
systemctl restart kubelet.service
2、下載部署文件
因為部署文件中image地址指向的是國外的鏡像,如果可以訪問外網則可以直接拉取,否則手動拉取鏡像。
######在node1節點手動拉取鏡像
docker pull registry.cn-shanghai.aliyuncs.com/szr0728/metrics-server:v0.5.0
docker tag registry.cn-shanghai.aliyuncs.com/szr0728/metrics-server:v0.5.0 k8s.gcr.io/metrics-server/metrics-server:v0.5.0
######在master節點
yum install wget -y
cd
wget https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.5.0/components.yaml
#在components.yaml文件中添加如下參數
- --kubelet-insecure-tls
3、部署
kubectl apply -f components.yaml
#查看狀態是否為running
kubectl -n kube-system get pods -l k8s-app=metrics-server
#查看是否有報錯
kubectl -n kube-system logs -l k8s-app=metrics-server -f
#查看資源監控能否正常顯示
kubectl top node
五、把部署的應用整合到一個 YAML 文件 wordpress-all.yaml 中
#刪除前面步驟創建的資源
kubectl delete deploy wordpress-deploy mysql-deploy -n blog
kubectl delete hpa wordpress-deploy -n blog
kubectl delete svc wordpress mysql -n blog
kubectl delete pvc mypvc -n blog
kubectl delete pv mypv
rm -rf /nfs/pv1/*
#wordpress-all.yaml文件
#vim wordpress-all.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: mypv
namespace: blog
spec:
capacity:
storage: 1Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Recycle
nfs:
path: /nfs/pv1
server: 192.168.30.30
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mypvc
namespace: blog
spec:
accessModes:
- ReadWriteOnce
volumeName: mypv
resources:
requests:
storage: 1Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: mysql-deploy
namespace: blog
labels:
apps: mysql
spec:
selector:
matchLabels:
app: mysql
template:
metadata:
labels:
app: mysql
spec:
containers:
- name: mysql
image: mysql:5.6
imagePullPolicy: IfNotPresent
ports:
- containerPort: 3306
name: dbport
env:
- name: MYSQL_ROOT_PASSWORD
value: rootPassWord
- name: MYSQL_DATABASE
value: wordpress
- name: MYSQL_USER
value: wordpress
- name: MYSQL_PASSWORD
value: wordpress
volumeMounts:
- name: db-pv
mountPath: /var/lib/mysql
volumes:
- name: db-pv
persistentVolumeClaim:
claimName: mypvc
---
apiVersion: v1
kind: Service
metadata:
name: mysql
namespace: blog
spec:
selector:
app: mysql
ports:
- name: mysqlport
protocol: TCP
port: 3306
targetPort: 3306
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: wordpress-deploy
namespace: blog
labels:
app: wordpress
spec:
selector:
matchLabels:
app: wordpress
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 50%
maxUnavailable: 0
template:
metadata:
labels:
app: wordpress
spec:
containers:
- name: wordpress
image: wordpress
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
name: wdport
env:
- name: WORDPRESS_DB_HOST
value: mysql:3306
- name: WORDPRESS_DB_USER
value: wordpress
- name: WORDPRESS_DB_PASSWORD
value: wordpress
resources:
requests:
cpu: 100m
memory: 100Mi
limits:
cpu: 100m
memory: 500Mi
livenessProbe:
tcpSocket:
port: 80
initialDelaySeconds: 3
periodSeconds: 3
readinessProbe:
tcpSocket:
port: 80
initialDelaySeconds: 5
periodSeconds: 10
---
apiVersion: v1
kind: Service
metadata:
name: wordpress
namespace: blog
spec:
type: NodePort
selector:
app: wordpress
ports:
- name: wordpressport
protocol: TCP
port: 80
targetPort: wdport
---
apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
name: wordpress-hpa
namespace: blog
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: wordpress-deploy
minReplicas: 1
maxReplicas: 10
metrics:
- type: Resource
resource:
name: memory
targetAverageUtilization: 50
kubectl apply -f wordpress-all.yaml
部署完成后,查看pod和HPA狀態,查看svc的NodePort端口,訪問web。
六、問題記錄:
1、yaml文件中密碼的值如果是數字要加引號
2、部署完metrics-server后依然unknow
問題:ScalingActive False FailedGetResourceMetric the HPA was unable to compute the replica count: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: an error on the server ("Internal Server Error: \"/apis/metrics.k8s.io/v1beta1/namespaces/blog/pods?labelSelector=app%!D(MISSING)wordpress\": the server could not find the requested resource") has prevented the request from succeeding (get pods.metrics.k8s.io)
通過排查發現metrics-server服務沒有running,查看日志:Failed to make webhook authorizer request: the server could not find the requested resource E1121 12:53:58.3613851 errors.go:77] the server could not find the requested resource
解決方法:查看自己環境的k8s版本,如果是1.21+,部署metrics-server:v0.5.0以下的則會報錯,只需部署0.5版本以上即可解決,部署方法詳見上文。
3、掛載遇到的問題
如果掛載遇到如下文件,請檢查NFS服務器相關設置