Helm-4: 讓編排兼容不同版本K8S

在使用Helm過程中, 經常會遇到編排需要兼容不同K8S版本的問題. 考慮如下場景:

1. 以前編寫的Deployment資源, 其apiVersion為 apps/v1beta1, 但后來新的版本中已經改為 apps/v1,希望能兼容

2. 在K8S 1.11以前, 默認CRD既不支持subResources, 也不能夠通過 UpdateStatus 更新狀態, 必須使用 Update. 而在這之后, 必須使用 UpdateStatus 才能更新狀態.

Helm 內置了一系列內部對象,可以針對這些情況進行編排.

Deployment兼容多版本K8S

針對以上第一個問題, 我們可以直接在 _helpers.tpl 中加入以下內容:

{{/*
Define apiVersion for Deployment
*/}}
{{- define "deployApiVersion" -}} 
{{- if .Capabilities.APIVersions.Has "apps/v1beta1/Deployment" -}}
apps/v1beta1
{{- else -}}
apps/v1
{{- end -}}
{{- end -}}

這里判斷這套K8S是否具備 apps/v1beta/Deployment , 如果有, 使用 apps/v1 ,否則就是舊版本的 apps/v1beta1

然后, Deployment引用這一段即可:

kind: Deployment
apiVersion: {{ include "deployApiVersion" . }}

檢測配置 disableSubresources 是否開啟

經過查詢文檔, disableSubresources 默認開啟是在 1.11 版本, 因此我們可以簡單一點, 只判斷小版本是否大於等於 "11", 當然這里沒有判斷大版本.

disableSubresources: {{ if ge .Capabilities.KubeVersion.Minor "11" }}false{{ else }}true{{ end }}

Capabilities 實現

我們可能在某些場景下, 希望自己的程序能夠直接去判斷我們對接的K8S集群有哪些能力. 這部分實現我們可以參考 helm 源碼. 在 pkg/action/action.go中揭示了其實現方式:

// capabilities builds a Capabilities from discovery information.
func (c *Configuration) getCapabilities() (*chartutil.Capabilities, error) {
        if c.Capabilities != nil {
                return c.Capabilities, nil
        }
        dc, err := c.RESTClientGetter.ToDiscoveryClient()
        if err != nil {
                return nil, errors.Wrap(err, "could not get Kubernetes discovery client")
        }
        // force a discovery cache invalidation to always fetch the latest server version/capabilities.
        dc.Invalidate()
        kubeVersion, err := dc.ServerVersion()
        if err != nil {
                return nil, errors.Wrap(err, "could not get server version from Kubernetes")
        }
        // Issue #6361:
        // Client-Go emits an error when an API service is registered but unimplemented.
        // We trap that error here and print a warning. But since the discovery client continues
        // building the API object, it is correctly populated with all valid APIs.
        // See https://github.com/kubernetes/kubernetes/issues/72051#issuecomment-521157642
        apiVersions, err := GetVersionSet(dc)
        if err != nil {
                if discovery.IsGroupDiscoveryFailedError(err) {
                        c.Log("WARNING: The Kubernetes server has an orphaned API service. Server reports: %s", err)
                        c.Log("WARNING: To fix this, kubectl delete apiservice <service-name>")
                } else {
                        return nil, errors.Wrap(err, "could not get apiVersions from Kubernetes")
                }
        }

        c.Capabilities = &chartutil.Capabilities{
                APIVersions: apiVersions,
                KubeVersion: chartutil.KubeVersion{
                        Version: kubeVersion.GitVersion,
                        Major:   kubeVersion.Major,
                        Minor:   kubeVersion.Minor,
                },
        }
        return c.Capabilities, nil
}

dry-run 沒問題的chart一定能夠在集群上部署嗎?

在解決第一個問題時, 我事先通過 -dry-run 測試我寫出的chart, 結果安裝到集群卻失敗了. 其實, 在Helm Chart進行dryrun時, 是不會與集群進行交互的.

在 helm 源碼中, 它的解釋是編排chart的作者的期望可能是連接到集群,但用戶卻不一定如此期望:

        // A `helm template` or `helm install --dry-run` should not talk to the remote cluster.
        // It will break in interesting and exotic ways because other data (e.g. discovery)
        // is mocked. It is not up to the template author to decide when the user wants to
        // connect to the cluster. So when the user says to dry run, respect the user's
        // wishes and do not connect to the cluster.
        if !dryRun && c.RESTClientGetter != nil {
                rest, err := c.RESTClientGetter.ToRESTConfig()
                if err != nil {
                        return hs, b, "", err
                }
                files, err2 = engine.RenderWithClient(ch, values, rest)
        } else {
                files, err2 = engine.Render(ch, values)
        }