一、 helm Charts文件組織結構
Charts是HELM使用Kubernetes程序包打包格式、一個Chart就是一個描述一組Kubernetes資源的文件的集合
事實上、一個單獨的Charts既能用於部署簡單應用，例如一個memcached pod 也能部署復雜的應用如http服務器db服務器cache服務器
例如一個wordpress Charts的目錄結構應該如下所示
[root@master stable]# tree wordpress/ wordpress/ ├── Chart.yaml ├── OWNERS ├── README.md ├── requirements.lock ├── requirements.yaml ├── templates │ ├── deployment.yaml │ ├── externaldb-secrets.yaml │ ├── _helpers.tpl │ ├── ingress.yaml │ ├── NOTES.txt │ ├── pvc.yaml │ ├── secrets.yaml │ ├── servicemonitor.yaml │ ├── svc.yaml │ ├── tests │ │ └── test-mariadb-connection.yaml │ └── tls-secrets.yaml ├── values-production.yaml ├── values.schema.json └── values.yaml
1、Chart.yaml：當前Charts的描述信息、yaml格式的文件
2、LICENSE：當前Charts的許可信息，純文本文件；此為可選文件
3、README.md：易讀格式的README文件；可選
4、當前Charts依賴關系描述文件；可選
5、values.yaml：當前Charts用到的默認配置值。
6、ci：目錄、存放當前Charts依賴到的所有Charts文件
7、templates：目錄、存放當前Charts用到的模板文件，可應用於Charts生成有效的Kubernetes清單文件
二、Charts文件組織格式
1、例如一個redis Charts的Chart.yaml應該如下所示
[root@master redis]# cat Chart.yaml apiVersion: v1 name: redis version: 10.5.7 appVersion: 5.0.7 # The redis chart is deprecated and no longer maintained. For details deprecation, see the PROCESSES.md file. deprecated: true #當前Chart是否已廢棄、可選字段，布爾值 description: DEPRECATED Open source, advanced key-value store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets and sorted sets. keywords: - redis - keyvalue - database home: http://redis.io/ icon: https://bitnami.com/assets/stacks/redis/img/redis-stack-220x234.png sources: - https://github.com/bitnami/bitnami-docker-redis maintainers: [] engine: gotpl
maintainers項目維護者信息
[root@master jenkins]# cat Chart.yaml apiVersion: v1 name: jenkins home: https://jenkins.io/ version: 2.3.0 appVersion: lts description: Open source continuous integration server. It supports multiple SCM tools including CVS, Subversion and Git. It can execute Apache Ant and Apache Maven-based projects as well as arbitrary scripts. sources: - https://github.com/jenkinsci/jenkins - https://github.com/jenkinsci/docker-jnlp-slave - https://github.com/maorfr/kube-tasks - https://github.com/jenkinsci/configuration-as-code-plugin maintainers: #項目維護者信息、主要嵌套name、email和URL幾個屬性組成；可選字段 - name: lachie83 email: lachlan.evenson@microsoft.com - name: viglesiasce email: viglesias@google.com - name: maorfr email: maor.friedman@redhat.com - name: torstenwalter email: mail@torstenwalter.de - name: mogaal email: garridomota@gmail.com - name: wmcdona89 email: wmcdona89@gmail.com icon: https://wiki.jenkins-ci.org/download/attachments/2916393/logo.png
三、模板和值
模板遵循Go模板語言格式，並支持50種以上的來自Spring庫的模板函數附件，以及為數補多少的其他專用函數。所有的模板文件都存儲與templates目中
在當前被引用時，此目錄中的所有模板文件都會傳遞給模板引擎進行處理
模板引擎中用的值(value)有如下兩種提供方式
1、通過的文件提供，通過用於提供默認值
2、在運行"helm install" 命令時傳遞包含所需要的的自定義值YAML文件；此處傳遞的值會覆蓋默認值
下面的示例是wordpress中的模板文件deployment的部分內容
[root@master stable]# cat wordpress/templates/deployment.yaml
apiVersion: {{ template "wordpress.deployment.apiVersion" . }}
kind: Deployment
metadata:
name: {{ template "wordpress.fullname" . }}
labels: {{- include "wordpress.labels" . | nindent 4 }}
spec:
selector:
matchLabels: {{- include "wordpress.matchLabels" . | nindent 6 }}
{{- if .Values.updateStrategy }}
strategy: {{ toYaml .Values.updateStrategy | nindent 4 }}
{{- end }}
replicas: {{ .Values.replicaCount }}
template:
metadata:
labels: {{- include "wordpress.labels" . | nindent 8 }}
{{- if or .Values.podAnnotations .Values.metrics.enabled }}
annotations:
{{- if .Values.podAnnotations }}
{{- include "wordpress.tplValue" (dict "value" .Values.podAnnotations "context" $) | nindent 8 }}
{{- end }}
{{- if .Values.metrics.podAnnotations }}
{{- include "wordpress.tplValue" (dict "value" .Values.metrics.podAnnotations "context" $) | nindent 8 }}
{{- end }}
{{- end }}
spec:
{{- include "wordpress.imagePullSecrets" . | indent 6 }}
{{- if .Values.schedulerName }}
schedulerName: {{ .Values.schedulerName | quote }}
{{- end }}
hostAliases:
- ip: "127.0.0.1"
hostnames:
- "status.localhost"
而在values.yaml一類的文件中定義時，既可以將它定義為全局作用於、也可以定義為僅供Charts目錄下的某個Charts所使用
一般來說上級Charts可以訪問下級的Charts中的值、而下級Charts不能訪問其上級Charts的值
service: type: LoadBalancer ## HTTP Port ## port: 80 ## HTTPS Port ## httpsPort: 443 ## HTTPS Target Port ## defaults to https unless overridden to the specified port. ## if you want the target port to be "http" or "80" you can specify that here. ## httpsTargetPort: https ## Metrics Port ## metricsPort: 9117 ## Node Ports to expose ## nodePorts: ## http: <to set explicitly, choose port between 30000-32767> ## https: <to set explicitly, choose port between 30000-32767> ## metrics: <to set explicitly, choose port between 30000-32767> nodePorts: http: "" https: "" metrics: ""
Go 模板語法請參考godoc站點中內容、地址為：https://godoc.org/text/template
四、自定義Charts
1、生成一個空Charts
[root@master ~]# helm create luoahongchart Creating luoahongchart [root@master ~]# tree luoahongchart/ luoahongchart/ ├── charts ├── Chart.yaml ├── templates │ ├── deployment.yaml │ ├── _helpers.tpl │ ├── ingress.yaml │ ├── NOTES.txt │ ├── serviceaccount.yaml │ ├── service.yaml │ └── tests │ └── test-connection.yaml └── values.yaml 3 directories, 9 files
由命令生成的各文件還有着各自應該具有的通用組織結構框架、例如Chart.yaml文件的默認內容如下
[root@master luoahongchart]# cat Chart.yaml apiVersion: v1 appVersion: "1.0" description: A Helm chart for Kubernetes name: luoahongchart version: 0.1.0
事實上，它甚至直接在values.yaml 將要使用的鏡像文件定義中為nginx生成了一個可直接安裝容器化nginx應用的Charts，
期中的部分內容如下所示：
[root@master luoahongchart]# grep -vE "#|^$" values.yaml
replicaCount: 1
image:
repository: nginx
tag: stable
pullPolicy: IfNotPresent
imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""
serviceAccount:
create: true
name:
podSecurityContext: {}
securityContext: {}
service:
type: ClusterIP
port: 80
因此，用戶僅需要在各文件現有框架的基礎上按需進行修改即可定義出所需的Chart來
2.修改Charts以部署自定義服務
這里以此前使用的容器應用""為例來說明如何定義一個Charts
[root@master luoahongchart]# grep -vE "#|^$" values.yaml
replicaCount: 1
image:
repository: kubernetes/myapp #更改nginx為kubernetes/myapp
tag: v1 #更改stable為v1
pullPolicy: IfNotPresent
imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""
serviceAccount:
create: true
name:
podSecurityContext: {}
securityContext: {}
service:
type: ClusterIP
port: 80
而后通過"helm lint" 命令確認修改后的是否遵循最佳實踐且模板格式良好
[root@master ~]# ll total 4 -rw-------. 1 root root 1404 Apr 21 09:52 anaconda-ks.cfg drwxr-xr-x 2 root root 108 Jul 9 17:21 k8s drwxr-xr-x 4 root root 93 Jul 13 16:19 luoahongchart [root@master ~]# helm lint luoahongchart ==> Linting luoahongchart [INFO] Chart.yaml: icon is recommended 1 chart(s) linted, no failures
多數情況下，"helm lint"命令報告的錯誤信息、根據其錯誤提示中的行號信息即能定位出錯誤所在、確保一切問題都得以解決之后、即可通過"helm install"命令調試運行以查看由Charts定義的容器化應用是否能夠正確部署
[root@master ~]# helm install --name myapp --dry-run --debug ./luoahongchart --set service.type=NodePort [debug] Created tunnel using local port: '38624' [debug] SERVER: "127.0.0.1:38624" [debug] Original chart version: "" [debug] CHART PATH: /root/luoahongchart NAME: myapp REVISION: 1 RELEASED: Mon Jul 13 16:34:30 2020 CHART: luoahongchart-0.1.0 USER-SUPPLIED VALUES: service: type: NodePort
確認上述命令輸出信息無誤后、移除命令中的"--dry-run" 選項后再次運行命令即可部署完成應用的部署
[root@master ~]# helm install --name myapp ./luoahongchart --set service.type=NodePort
NAME: myapp
LAST DEPLOYED: Mon Jul 13 16:36:37 2020
NAMESPACE: default
STATUS: DEPLOYED
RESOURCES:
==> v1/Deployment
NAME READY UP-TO-DATE AVAILABLE AGE
myapp-luoahongchart 0/1 0 0 0s
==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
myapp-luoahongchart-6777bd6b65-fslx2 0/1 ContainerCreating 0 0s
==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
myapp-luoahongchart NodePort 10.99.123.248 <none> 80:30435/TCP 0s
==> v1/ServiceAccount
NAME SECRETS AGE
myapp-luoahongchart 1 0s
NOTES:
1. Get the application URL by running these commands:
export NODE_PORT=$(kubectl get --namespace default -o jsonpath="{.spec.ports[0].nodePort}" services myapp-luoahongchart)
export NODE_IP=$(kubectl get nodes --namespace default -o jsonpath="{.items[0].status.addresses[0].address}")
echo http://$NODE_IP:$NODE_PORT
而后、通過刪除NOTES中的命令提示運行相關的命令獲取訪問端點后即可通過瀏覽器訪問相應的服務
[root@master ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
myapp-luoahongchart-6777bd6b65-fslx2 1/1 Running 0 6m5s 10.244.2.19 nodes2 <none> <none>
[root@master ~]#export NODE_PORT=$(kubectl get --namespace default -o jsonpath="{.spec.ports[0].nodePort}" services myapp-luoahongchart)
[root@master ~]#export NODE_IP=$(kubectl get nodes --namespace default -o jsonpath="{.items[0].status.addresses[0].address}")
[root@master ~]#echo http://$NODE_IP:$NODE_PORT
http://192.168.118.18:30435
而后通過瀏覽器訪問測試所部屬的myapp應用
3.Charts倉庫
至此、一個自定義的基於本地設定完成、不過、它僅能用於本地訪問、當然用戶可以通過"helm package"命令將其打包為tar格式后分享給團隊或者社區：
[root@master ~]# helm package ./luoahongchart Successfully packaged chart and saved it to: /root/luoahongchart-0.1.0.tgz [root@master ~]# helm serve Regenerating index. This may take a moment. Now serving you on 127.0.0.1:8879
此命令會占據當前終端，於是、另起一個終端即可測試訪問倉庫服務中Charts：
[root@master ~]# helm search local NAME CHART VERSION APP VERSION DESCRIPTION local/luoahongchart 0.1.0 1.0 A Helm chart for Kubernetes incubator/puppet-forge 0.1.8 1.10.0 Distribute locally developed Puppet modules and proxy to ...
向外分享創建好的，只能自己基於web服務器程序來滿足、如果要外部訪問建議使用https的服務器提供倉庫服務
添加
[root@master luoahongchart]# helm repo add incubator https://kubernetes-charts.storage.googleapis.com "incubator" has been added to your repositories helm repo add incubator https://kubernetes-charts.storage.googleapis.com
查看
[root@master luoahongchart]# helm repo list NAME URL local http://127.0.0.1:8879/charts stable https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts incubator https://kubernetes-charts.storage.googleapis.com
更新
[root@master ~]# helm repo update Hang tight while we grab the latest from your chart repositories... ...Skip local chart repository ...Successfully got an update from the "stable" chart repository ...Unable to get an update from the "incubator" chart repository (https://kubernetes-charts-incubator.storage.googleapis.com): Get https://kubernetes-charts-incubator.storage.googleapis.com/index.yaml: dial tcp 34.64.4.80:443: connect: connection timed out Update Complete.
而刪除制定的倉庫配置"helm repo remove <REPO_NAME>"
4、配置依賴關系
構建存在依賴關系的charts時、還需要為其定義依賴項、例如、前面創建的myapp依賴於數據庫管庫系統MYSQL時、在luoahongchart
的目錄中創建如下文件：
[root@master ~]# cat ./luoahongchart/requirements.yaml dependencies: - name: mysql version: 0.6.0 repository: https://kubernetes-charts.storage.googleapis.com
而后、需要運行"helm dependency update" 命令為Charts更新依賴關系
運行下面的命令來引入定義的MySQL依賴項時、會自動下載MySQL相關的charts程序包至./luoahongchart/charts/子目錄中
[root@master ~]# helm dependency update ./luoahongchart Hang tight while we grab the latest from your chart repositories... ...Unable to get an update from the "local" chart repository (http://127.0.0.1:8879/charts): Get http://127.0.0.1:8879/charts/index.yaml: dial tcp 127.0.0.1:8879: connect: connection refused ...Successfully got an update from the "stable" chart repository ...Successfully got an update from the "incubator" chart repository Update Complete. Saving 1 charts Downloading mysql from repo https://kubernetes-charts.storage.googleapis.com Deleting outdated charts
更新過程中helm會自動生成一個鎖定文件requirements.lock、以便后續再次獲取依賴關系時使用已知的工作版本
[root@master ~]# ll ./luoahongchart total 16 drwxr-xr-x 2 root root 29 Jul 13 17:31 charts -rw-r--r-- 1 root root 109 Jul 13 16:05 Chart.yaml -rw-r--r-- 1 root root 237 Jul 13 17:31 requirements.lock -rw-r--r-- 1 root root 108 Jul 13 17:24 requirements.yaml drwxr-xr-x 3 root root 146 Jul 13 16:05 templates -rw-r--r-- 1 root root 1519 Jul 13 16:19 values.yaml
程序包至./luoahongchart/charts/子目錄中
[root@master ~]# ll ./luoahongchart/charts/ total 8 -rw-r--r-- 1 root root 7589 Jul 13 17:31 mysql-0.6.0.tgz
此時、再次部署myapp Charts、就會同事部署依賴到mysql Charts、另外、用戶也可以手動將鎖依賴到的程序包直接放置於luoahongchart/charts/目錄中
來定義依賴關系、此時不要在使用requirements.yaml文件