1、概述
應用為用戶提供完整的業務功能,由一個或多個特定功能的組件組成。一般來說,根據一個應用的功能以及與外部環境通信的方式,它可以由一個或多個 Kubernetes 工作負載(例如部署、有狀態副本集和守護進程集)、服務和CRD等資源類型組成。
Application資源類型是Kubernetes特別興趣組(kubernetes-sigs)里面的開源項目application中自定義的一個CRD資源。目前application工程穩定版本為v0.8.3(2020年6月10日發布,此工程較為簡單,只有applocation這一個CRD及對應控制類,此工程功能已經趨於完整,master分支已經15個月沒更新了)。在Kubernetes集群中想使用Application資源類型的話需要提前在k8s集群中發布此CRD資源,並運行此工程里面的application_controller.go。
application工程主要提供以下功能:
- 應用會維護與其組件的級聯關系,刪除應用時會級聯刪除應用所有組件
- 應用級健康檢查
2、applications.app.k8s.io數據結構
application資源類型數據結構文件為application_types.go:
// Copyright 2020 The Kubernetes Authors. // SPDX-License-Identifier: Apache-2.0 package v1beta1 import ( "regexp" "strings" corev1 "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) // Constants for condition const ( // Ready => controller considers this resource Ready Ready = "Ready" // Qualified => functionally tested Qualified = "Qualified" // Settled => observed generation == generation + settled means controller is done acting functionally tested Settled = "Settled" // Cleanup => it is set to track finalizer failures Cleanup = "Cleanup" // Error => last recorded error Error = "Error" ReasonInit = "Init" ) // Descriptor defines the Metadata and informations about the Application. type Descriptor struct { // Type is the type of the application (e.g. WordPress, MySQL, Cassandra). Type string `json:"type,omitempty"` // Version is an optional version indicator for the Application. Version string `json:"version,omitempty"` // Description is a brief string description of the Application. Description string `json:"description,omitempty"` // Icons is an optional list of icons for an application. Icon information includes the source, size, // and mime type. Icons []ImageSpec `json:"icons,omitempty"` // Maintainers is an optional list of maintainers of the application. The maintainers in this list maintain the // the source code, images, and package for the application. Maintainers []ContactData `json:"maintainers,omitempty"` // Owners is an optional list of the owners of the installed application. The owners of the application should be // contacted in the event of a planned or unplanned disruption affecting the application. Owners []ContactData `json:"owners,omitempty"` // Keywords is an optional list of key words associated with the application (e.g. MySQL, RDBMS, database). Keywords []string `json:"keywords,omitempty"` // Links are a list of descriptive URLs intended to be used to surface additional documentation, dashboards, etc. Links []Link `json:"links,omitempty"` // Notes contain a human readable snippets intended as a quick start for the users of the Application. // CommonMark markdown syntax may be used for rich text representation. Notes string `json:"notes,omitempty"` } // ApplicationSpec defines the specification for an Application. type ApplicationSpec struct { // ComponentGroupKinds is a list of Kinds for Application's components (e.g. Deployments, Pods, Services, CRDs). It // can be used in conjunction with the Application's Selector to list or watch the Applications components. ComponentGroupKinds []metav1.GroupKind `json:"componentKinds,omitempty"` // Descriptor regroups information and metadata about an application. Descriptor Descriptor `json:"descriptor,omitempty"` // Selector is a label query over kinds that created by the application. It must match the component objects' labels. // More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#label-selectors Selector *metav1.LabelSelector `json:"selector,omitempty"` // AddOwnerRef objects - flag to indicate if we need to add OwnerRefs to matching objects // Matching is done by using Selector to query all ComponentGroupKinds AddOwnerRef bool `json:"addOwnerRef,omitempty"` // Info contains human readable key,value pairs for the Application. // +patchStrategy=merge // +patchMergeKey=name Info []InfoItem `json:"info,omitempty" patchStrategy:"merge" patchMergeKey:"name"` // AssemblyPhase represents the current phase of the application's assembly. // An empty value is equivalent to "Succeeded". AssemblyPhase ApplicationAssemblyPhase `json:"assemblyPhase,omitempty"` } // ComponentList is a generic status holder for the top level resource type ComponentList struct { // Object status array for all matching objects Objects []ObjectStatus `json:"components,omitempty"` } // ObjectStatus is a generic status holder for objects type ObjectStatus struct { // Link to object Link string `json:"link,omitempty"` // Name of object Name string `json:"name,omitempty"` // Kind of object Kind string `json:"kind,omitempty"` // Object group Group string `json:"group,omitempty"` // Status. Values: InProgress, Ready, Unknown Status string `json:"status,omitempty"` } // ConditionType encodes information on the condition type ConditionType string // Condition describes the state of an object at a certain point. type Condition struct { // Type of condition. Type ConditionType `json:"type" protobuf:"bytes,1,opt,name=type,casttype=StatefulSetConditionType"` // Status of the condition, one of True, False, Unknown. Status corev1.ConditionStatus `json:"status" protobuf:"bytes,2,opt,name=status,casttype=k8s.io/api/core/v1.ConditionStatus"` // The reason for the condition's last transition. // +optional Reason string `json:"reason,omitempty" protobuf:"bytes,4,opt,name=reason"` // A human readable message indicating details about the transition. // +optional Message string `json:"message,omitempty" protobuf:"bytes,5,opt,name=message"` // Last time the condition was probed // +optional LastUpdateTime metav1.Time `json:"lastUpdateTime,omitempty" protobuf:"bytes,3,opt,name=lastProbeTime"` // Last time the condition transitioned from one status to another. // +optional LastTransitionTime metav1.Time `json:"lastTransitionTime,omitempty" protobuf:"bytes,3,opt,name=lastTransitionTime"` } // ApplicationStatus defines controller's the observed state of Application type ApplicationStatus struct { // ObservedGeneration is the most recent generation observed. It corresponds to the // Object's generation, which is updated on mutation by the API Server. // +optional ObservedGeneration int64 `json:"observedGeneration,omitempty" protobuf:"varint,1,opt,name=observedGeneration"` // Conditions represents the latest state of the object // +optional // +patchMergeKey=type // +patchStrategy=merge Conditions []Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,10,rep,name=conditions"` // Resources embeds a list of object statuses // +optional ComponentList `json:",inline,omitempty"` // ComponentsReady: status of the components in the format ready/total // +optional ComponentsReady string `json:"componentsReady,omitempty"` } // ImageSpec contains information about an image used as an icon. type ImageSpec struct { // The source for image represented as either an absolute URL to the image or a Data URL containing // the image. Data URLs are defined in RFC 2397. Source string `json:"src"` // (optional) The size of the image in pixels (e.g., 25x25). Size string `json:"size,omitempty"` // (optional) The mine type of the image (e.g., "image/png"). Type string `json:"type,omitempty"` } // ContactData contains information about an individual or organization. type ContactData struct { // Name is the descriptive name. Name string `json:"name,omitempty"` // Url could typically be a website address. URL string `json:"url,omitempty"` // Email is the email address. Email string `json:"email,omitempty"` } // Link contains information about an URL to surface documentation, dashboards, etc. type Link struct { // Description is human readable content explaining the purpose of the link. Description string `json:"description,omitempty"` // Url typically points at a website address. URL string `json:"url,omitempty"` } // InfoItem is a human readable key,value pair containing important information about how to access the Application. type InfoItem struct { // Name is a human readable title for this piece of information. Name string `json:"name,omitempty"` // Type of the value for this InfoItem. Type InfoItemType `json:"type,omitempty"` // Value is human readable content. Value string `json:"value,omitempty"` // ValueFrom defines a reference to derive the value from another source. ValueFrom *InfoItemSource `json:"valueFrom,omitempty"` } // InfoItemType is a string that describes the value of InfoItem type InfoItemType string const ( // ValueInfoItemType const string for value type ValueInfoItemType InfoItemType = "Value" // ReferenceInfoItemType const string for ref type ReferenceInfoItemType InfoItemType = "Reference" ) // InfoItemSource represents a source for the value of an InfoItem. type InfoItemSource struct { // Type of source. Type InfoItemSourceType `json:"type,omitempty"` // Selects a key of a Secret. SecretKeyRef *SecretKeySelector `json:"secretKeyRef,omitempty"` // Selects a key of a ConfigMap. ConfigMapKeyRef *ConfigMapKeySelector `json:"configMapKeyRef,omitempty"` // Select a Service. ServiceRef *ServiceSelector `json:"serviceRef,omitempty"` // Select an Ingress. IngressRef *IngressSelector `json:"ingressRef,omitempty"` } // InfoItemSourceType is a string type InfoItemSourceType string // Constants for info type const ( SecretKeyRefInfoItemSourceType InfoItemSourceType = "SecretKeyRef" ConfigMapKeyRefInfoItemSourceType InfoItemSourceType = "ConfigMapKeyRef" ServiceRefInfoItemSourceType InfoItemSourceType = "ServiceRef" IngressRefInfoItemSourceType InfoItemSourceType = "IngressRef" ) // ConfigMapKeySelector selects a key from a ConfigMap. type ConfigMapKeySelector struct { // The ConfigMap to select from. corev1.ObjectReference `json:",inline"` // The key to select. Key string `json:"key,omitempty"` } // SecretKeySelector selects a key from a Secret. type SecretKeySelector struct { // The Secret to select from. corev1.ObjectReference `json:",inline"` // The key to select. Key string `json:"key,omitempty"` } // ServiceSelector selects a Service. type ServiceSelector struct { // The Service to select from. corev1.ObjectReference `json:",inline"` // The optional port to select. Port *int32 `json:"port,omitempty"` // The optional HTTP path. Path string `json:"path,omitempty"` // Protocol for the service Protocol string `json:"protocol,omitempty"` } // IngressSelector selects an Ingress. type IngressSelector struct { // The Ingress to select from. corev1.ObjectReference `json:",inline"` // The optional host to select. Host string `json:"host,omitempty"` // The optional HTTP path. Path string `json:"path,omitempty"` // Protocol for the ingress Protocol string `json:"protocol,omitempty"` } // ApplicationAssemblyPhase tracks the Application CRD phases: pending, succeeded, failed type ApplicationAssemblyPhase string // Constants const ( // Used to indicate that not all of application's components // have been deployed yet. Pending ApplicationAssemblyPhase = "Pending" // Used to indicate that all of application's components // have already been deployed. Succeeded = "Succeeded" // Used to indicate that deployment of application's components // failed. Some components might be present, but deployment of // the remaining ones will not be re-attempted. Failed = "Failed" ) // +kubebuilder:object:root=true // +kubebuilder:resource:categories=all,shortName=app // +kubebuilder:subresource:status // +kubebuilder:printcolumn:name="Type",type=string,description="The type of the application",JSONPath=`.spec.descriptor.type`,priority=0 // +kubebuilder:printcolumn:name="Version",type=string,description="The creation date",JSONPath=`.spec.descriptor.version`,priority=0 // +kubebuilder:printcolumn:name="Owner",type=boolean,description="The application object owns the matched resources",JSONPath=`.spec.addOwnerRef`,priority=0 // +kubebuilder:printcolumn:name="Ready",type=string,description="Numbers of components ready",JSONPath=`.status.componentsReady`,priority=0 // +kubebuilder:printcolumn:name="Age",type=date,description="The creation date",JSONPath=`.metadata.creationTimestamp`,priority=0 // Application is the Schema for the applications API type Application struct { metav1.TypeMeta `json:",inline"` metav1.ObjectMeta `json:"metadata,omitempty"` Spec ApplicationSpec `json:"spec,omitempty"` Status ApplicationStatus `json:"status,omitempty"` } // +kubebuilder:object:root=true // ApplicationList contains a list of Application type ApplicationList struct { metav1.TypeMeta `json:",inline"` metav1.ListMeta `json:"metadata,omitempty"` Items []Application `json:"items"` } func init() { SchemeBuilder.Register(&Application{}, &ApplicationList{}) } // StripVersion the version part of gv func StripVersion(gv string) string { if gv == "" { return gv } re := regexp.MustCompile(`^[vV][0-9].*`) // If it begins with only version, (group is nil), return empty string which maps to core group if re.MatchString(gv) { return "" } return strings.Split(gv, "/")[0] } // Copyright 2020 The Kubernetes Authors. // SPDX-License-Identifier: Apache-2.0 package v1beta1 import ( "regexp" "strings" corev1 "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) // Constants for condition const ( // Ready => controller considers this resource Ready Ready = "Ready" // Qualified => functionally tested Qualified = "Qualified" // Settled => observed generation == generation + settled means controller is done acting functionally tested Settled = "Settled" // Cleanup => it is set to track finalizer failures Cleanup = "Cleanup" // Error => last recorded error Error = "Error" ReasonInit = "Init" ) // Descriptor defines the Metadata and informations about the Application. type Descriptor struct { // Type is the type of the application (e.g. WordPress, MySQL, Cassandra). Type string `json:"type,omitempty"` // Version is an optional version indicator for the Application. Version string `json:"version,omitempty"` // Description is a brief string description of the Application. Description string `json:"description,omitempty"` // Icons is an optional list of icons for an application. Icon information includes the source, size, // and mime type. Icons []ImageSpec `json:"icons,omitempty"` // Maintainers is an optional list of maintainers of the application. The maintainers in this list maintain the // the source code, images, and package for the application. Maintainers []ContactData `json:"maintainers,omitempty"` // Owners is an optional list of the owners of the installed application. The owners of the application should be // contacted in the event of a planned or unplanned disruption affecting the application. Owners []ContactData `json:"owners,omitempty"` // Keywords is an optional list of key words associated with the application (e.g. MySQL, RDBMS, database). Keywords []string `json:"keywords,omitempty"` // Links are a list of descriptive URLs intended to be used to surface additional documentation, dashboards, etc. Links []Link `json:"links,omitempty"` // Notes contain a human readable snippets intended as a quick start for the users of the Application. // CommonMark markdown syntax may be used for rich text representation. Notes string `json:"notes,omitempty"` } // ApplicationSpec defines the specification for an Application. type ApplicationSpec struct { // ComponentGroupKinds is a list of Kinds for Application's components (e.g. Deployments, Pods, Services, CRDs). It // can be used in conjunction with the Application's Selector to list or watch the Applications components. ComponentGroupKinds []metav1.GroupKind `json:"componentKinds,omitempty"` // Descriptor regroups information and metadata about an application. Descriptor Descriptor `json:"descriptor,omitempty"` // Selector is a label query over kinds that created by the application. It must match the component objects' labels. // More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#label-selectors Selector *metav1.LabelSelector `json:"selector,omitempty"` // AddOwnerRef objects - flag to indicate if we need to add OwnerRefs to matching objects // Matching is done by using Selector to query all ComponentGroupKinds AddOwnerRef bool `json:"addOwnerRef,omitempty"` // Info contains human readable key,value pairs for the Application. // +patchStrategy=merge // +patchMergeKey=name Info []InfoItem `json:"info,omitempty" patchStrategy:"merge" patchMergeKey:"name"` // AssemblyPhase represents the current phase of the application's assembly. // An empty value is equivalent to "Succeeded". AssemblyPhase ApplicationAssemblyPhase `json:"assemblyPhase,omitempty"` } // ComponentList is a generic status holder for the top level resource type ComponentList struct { // Object status array for all matching objects Objects []ObjectStatus `json:"components,omitempty"` } // ObjectStatus is a generic status holder for objects type ObjectStatus struct { // Link to object Link string `json:"link,omitempty"` // Name of object Name string `json:"name,omitempty"` // Kind of object Kind string `json:"kind,omitempty"` // Object group Group string `json:"group,omitempty"` // Status. Values: InProgress, Ready, Unknown Status string `json:"status,omitempty"` } // ConditionType encodes information on the condition type ConditionType string // Condition describes the state of an object at a certain point. type Condition struct { // Type of condition. Type ConditionType `json:"type" protobuf:"bytes,1,opt,name=type,casttype=StatefulSetConditionType"` // Status of the condition, one of True, False, Unknown. Status corev1.ConditionStatus `json:"status" protobuf:"bytes,2,opt,name=status,casttype=k8s.io/api/core/v1.ConditionStatus"` // The reason for the condition's last transition. // +optional Reason string `json:"reason,omitempty" protobuf:"bytes,4,opt,name=reason"` // A human readable message indicating details about the transition. // +optional Message string `json:"message,omitempty" protobuf:"bytes,5,opt,name=message"` // Last time the condition was probed // +optional LastUpdateTime metav1.Time `json:"lastUpdateTime,omitempty" protobuf:"bytes,3,opt,name=lastProbeTime"` // Last time the condition transitioned from one status to another. // +optional LastTransitionTime metav1.Time `json:"lastTransitionTime,omitempty" protobuf:"bytes,3,opt,name=lastTransitionTime"` } // ApplicationStatus defines controller's the observed state of Application type ApplicationStatus struct { // ObservedGeneration is the most recent generation observed. It corresponds to the // Object's generation, which is updated on mutation by the API Server. // +optional ObservedGeneration int64 `json:"observedGeneration,omitempty" protobuf:"varint,1,opt,name=observedGeneration"` // Conditions represents the latest state of the object // +optional // +patchMergeKey=type // +patchStrategy=merge Conditions []Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,10,rep,name=conditions"` // Resources embeds a list of object statuses // +optional ComponentList `json:",inline,omitempty"` // ComponentsReady: status of the components in the format ready/total // +optional ComponentsReady string `json:"componentsReady,omitempty"` } // ImageSpec contains information about an image used as an icon. type ImageSpec struct { // The source for image represented as either an absolute URL to the image or a Data URL containing // the image. Data URLs are defined in RFC 2397. Source string `json:"src"` // (optional) The size of the image in pixels (e.g., 25x25). Size string `json:"size,omitempty"` // (optional) The mine type of the image (e.g., "image/png"). Type string `json:"type,omitempty"` } // ContactData contains information about an individual or organization. type ContactData struct { // Name is the descriptive name. Name string `json:"name,omitempty"` // Url could typically be a website address. URL string `json:"url,omitempty"` // Email is the email address. Email string `json:"email,omitempty"` } // Link contains information about an URL to surface documentation, dashboards, etc. type Link struct { // Description is human readable content explaining the purpose of the link. Description string `json:"description,omitempty"` // Url typically points at a website address. URL string `json:"url,omitempty"` } // InfoItem is a human readable key,value pair containing important information about how to access the Application. type InfoItem struct { // Name is a human readable title for this piece of information. Name string `json:"name,omitempty"` // Type of the value for this InfoItem. Type InfoItemType `json:"type,omitempty"` // Value is human readable content. Value string `json:"value,omitempty"` // ValueFrom defines a reference to derive the value from another source. ValueFrom *InfoItemSource `json:"valueFrom,omitempty"` } // InfoItemType is a string that describes the value of InfoItem type InfoItemType string const ( // ValueInfoItemType const string for value type ValueInfoItemType InfoItemType = "Value" // ReferenceInfoItemType const string for ref type ReferenceInfoItemType InfoItemType = "Reference" ) // InfoItemSource represents a source for the value of an InfoItem. type InfoItemSource struct { // Type of source. Type InfoItemSourceType `json:"type,omitempty"` // Selects a key of a Secret. SecretKeyRef *SecretKeySelector `json:"secretKeyRef,omitempty"` // Selects a key of a ConfigMap. ConfigMapKeyRef *ConfigMapKeySelector `json:"configMapKeyRef,omitempty"` // Select a Service. ServiceRef *ServiceSelector `json:"serviceRef,omitempty"` // Select an Ingress. IngressRef *IngressSelector `json:"ingressRef,omitempty"` } // InfoItemSourceType is a string type InfoItemSourceType string // Constants for info type const ( SecretKeyRefInfoItemSourceType InfoItemSourceType = "SecretKeyRef" ConfigMapKeyRefInfoItemSourceType InfoItemSourceType = "ConfigMapKeyRef" ServiceRefInfoItemSourceType InfoItemSourceType = "ServiceRef" IngressRefInfoItemSourceType InfoItemSourceType = "IngressRef" ) // ConfigMapKeySelector selects a key from a ConfigMap. type ConfigMapKeySelector struct { // The ConfigMap to select from. corev1.ObjectReference `json:",inline"` // The key to select. Key string `json:"key,omitempty"` } // SecretKeySelector selects a key from a Secret. type SecretKeySelector struct { // The Secret to select from. corev1.ObjectReference `json:",inline"` // The key to select. Key string `json:"key,omitempty"` } // ServiceSelector selects a Service. type ServiceSelector struct { // The Service to select from. corev1.ObjectReference `json:",inline"` // The optional port to select. Port *int32 `json:"port,omitempty"` // The optional HTTP path. Path string `json:"path,omitempty"` // Protocol for the service Protocol string `json:"protocol,omitempty"` } // IngressSelector selects an Ingress. type IngressSelector struct { // The Ingress to select from. corev1.ObjectReference `json:",inline"` // The optional host to select. Host string `json:"host,omitempty"` // The optional HTTP path. Path string `json:"path,omitempty"` // Protocol for the ingress Protocol string `json:"protocol,omitempty"` } // ApplicationAssemblyPhase tracks the Application CRD phases: pending, succeeded, failed type ApplicationAssemblyPhase string // Constants const ( // Used to indicate that not all of application's components // have been deployed yet. Pending ApplicationAssemblyPhase = "Pending" // Used to indicate that all of application's components // have already been deployed. Succeeded = "Succeeded" // Used to indicate that deployment of application's components // failed. Some components might be present, but deployment of // the remaining ones will not be re-attempted. Failed = "Failed" ) // +kubebuilder:object:root=true // +kubebuilder:resource:categories=all,shortName=app // +kubebuilder:subresource:status // +kubebuilder:printcolumn:name="Type",type=string,description="The type of the application",JSONPath=`.spec.descriptor.type`,priority=0 // +kubebuilder:printcolumn:name="Version",type=string,description="The creation date",JSONPath=`.spec.descriptor.version`,priority=0 // +kubebuilder:printcolumn:name="Owner",type=boolean,description="The application object owns the matched resources",JSONPath=`.spec.addOwnerRef`,priority=0 // +kubebuilder:printcolumn:name="Ready",type=string,description="Numbers of components ready",JSONPath=`.status.componentsReady`,priority=0 // +kubebuilder:printcolumn:name="Age",type=date,description="The creation date",JSONPath=`.metadata.creationTimestamp`,priority=0 // Application is the Schema for the applications API type Application struct { metav1.TypeMeta `json:",inline"` metav1.ObjectMeta `json:"metadata,omitempty"` Spec ApplicationSpec `json:"spec,omitempty"` Status ApplicationStatus `json:"status,omitempty"` } // +kubebuilder:object:root=true // ApplicationList contains a list of Application type ApplicationList struct { metav1.TypeMeta `json:",inline"` metav1.ListMeta `json:"metadata,omitempty"` Items []Application `json:"items"` } func init() { SchemeBuilder.Register(&Application{}, &ApplicationList{}) } // StripVersion the version part of gv func StripVersion(gv string) string { if gv == "" { return gv } re := regexp.MustCompile(`^[vV][0-9].*`) // If it begins with only version, (group is nil), return empty string which maps to core group if re.MatchString(gv) { return "" } return strings.Split(gv, "/")[0] }
對應CRD為app.k8s.io_applications.yaml,通過配置文件可以查看application資源的apiGroups、apiVersions 和 resources 以及資源的 scope信息,可以看到資源scope是Namespaced。
# Copyright 2020 The Kubernetes Authors. # SPDX-License-Identifier: Apache-2.0 apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: api-approved.kubernetes.io: https://github.com/kubernetes-sigs/application/pull/2 controller-gen.kubebuilder.io/version: v0.4.0 creationTimestamp: null name: applications.app.k8s.io spec: group: app.k8s.io names: categories: - all kind: Application listKind: ApplicationList plural: applications shortNames: - app singular: application scope: Namespaced versions: - additionalPrinterColumns: - description: The type of the application jsonPath: .spec.descriptor.type name: Type type: string - description: The creation date jsonPath: .spec.descriptor.version name: Version type: string - description: The application object owns the matched resources jsonPath: .spec.addOwnerRef name: Owner type: boolean - description: Numbers of components ready jsonPath: .status.componentsReady name: Ready type: string - description: The creation date jsonPath: .metadata.creationTimestamp name: Age type: date name: v1beta1 schema: ........
想在集群使用applications資源類型的話,需要執行下面命令。
kubectl apply -f app.k8s.io_applications.yaml
3、applications.app.k8s.io應用示例詳解
3.1 創建應用
在zmc-test namespcace下創建應用test-app,這里注意應用資源元數據里面的app.kubernetes.io/version和app.kubernetes.io/name這兩個標簽,它們在應用資源類型中特別重要,應用和其組件的關聯關系都是通過這兩個標簽的維護的。addOwnerRef字段為true的話應用會維護和其所有組件的級聯刪除關系。componentKinds字段表示當前應用的組件只能包含componentKinds中定義的資源類型。
{ "apiVersion": "app.k8s.io/v1beta1", "kind": "Application", "metadata": { "name": "test-app", "namespace": "zmc-test", "labels": { //通過判斷當前namespace下所有屬於componentKinds的資源實例是否包含以下兩個標簽與應用維護關聯關系 "app.kubernetes.io/version": "v1", "app.kubernetes.io/name": "test-app" } }, "spec": { "selector": { "matchLabels": { "app.kubernetes.io/version": "v1", "app.kubernetes.io/name": "test-app" } }, "addOwnerRef": true, //維護和其所有組件的級聯刪除關系 "componentKinds": [ //當前應用只能包含如下資源類型 { "group": "", "kind": "Service" }, { "group": "apps", "kind": "Deployment" }, { "group": "apps", "kind": "StatefulSet" }, { "group": "extensions", "kind": "Ingress" }, { "group": "servicemesh.zmc.io", "kind": "Strategy" }, { "group": "servicemesh.zmc.io", "kind": "ServicePolicy" } ] } }
3.2 創建應用組件
在zmc-test下創建service和deployment實例,它們元數據中都包含如下標簽,創建過程比較簡單,本文忽略。
"app.kubernetes.io/version": "v1", "app.kubernetes.io/name": "test-app"
3.3 application_controller.go維護應用和其組件關系
首先需要在kubernetes集群中運行application工程或者將application_controller.go加入到一個contoller manager的項目里,運行過程本文忽略,下面來剖析下application_controller.go源碼,application_contoller.go依賴conditio.go和status.go
主要邏輯:
1.維護當前應用和其組件的級聯關系
2.組織當前應用的status值(所有組件的狀態,就緒組件數量,應用conditions)
注意:下文中的組件指的都是當前應用的所有組件,例如當前示例應用test app,其組件有2個(一個deployment資源實例和一個service資源實例)
// Copyright 2020 The Kubernetes Authors. // SPDX-License-Identifier: Apache-2.0 package controllers import ( "context" "fmt" "github.com/go-logr/logr" "k8s.io/apimachinery/pkg/api/equality" apierrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/api/meta" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/runtime/schema" "k8s.io/apimachinery/pkg/types" utilerrors "k8s.io/apimachinery/pkg/util/errors" "k8s.io/client-go/util/retry" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" appv1beta1 "sigs.k8s.io/application/api/v1beta1" ) const ( loggerCtxKey = "logger" ) // ApplicationReconciler reconciles a Application object type ApplicationReconciler struct { client.Client Mapper meta.RESTMapper Log logr.Logger Scheme *runtime.Scheme } // +kubebuilder:rbac:groups=app.k8s.io,resources=applications,verbs=get;list;watch;create;update;patch;delete // +kubebuilder:rbac:groups=app.k8s.io,resources=applications/status,verbs=get;update;patch // +kubebuilder:rbac:groups=*,resources=*,verbs=list;get;update;patch;watch func (r *ApplicationReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) { rootCtx := context.Background() logger := r.Log.WithValues("application", req.NamespacedName) ctx := context.WithValue(rootCtx, loggerCtxKey, logger) var app appv1beta1.Application err := r.Get(ctx, req.NamespacedName, &app) if err != nil { if apierrors.IsNotFound(err) { return ctrl.Result{}, nil } return ctrl.Result{}, err } // Application is in the process of being deleted, so no need to do anything. if app.DeletionTimestamp != nil { return ctrl.Result{}, nil } //更新應用組件(給當前應用組件資源實例維護級聯關系) resources, errs := r.updateComponents(ctx, &app) //組織當前應用的status值(就緒組件數量,所有組件的狀態,應用conditions) newApplicationStatus := r.getNewApplicationStatus(ctx, &app, resources, &errs) newApplicationStatus.ObservedGeneration = app.Generation if equality.Semantic.DeepEqual(newApplicationStatus, &app.Status) { return ctrl.Result{}, nil } //更新應用狀態 err = r.updateApplicationStatus(ctx, req.NamespacedName, newApplicationStatus) return ctrl.Result{}, err } //更新應用組件(給當前應用的組件資源實例維護級聯關系) func (r *ApplicationReconciler) updateComponents(ctx context.Context, app *appv1beta1.Application) ([]*unstructured.Unstructured, []error) { var errs []error //根據selector獲取組件資源實例 resources := r.fetchComponentListResources(ctx, app.Spec.ComponentGroupKinds, app.Spec.Selector, app.Namespace, &errs) if app.Spec.AddOwnerRef { ownerRef := metav1.NewControllerRef(app, appv1beta1.GroupVersion.WithKind("Application")) *ownerRef.Controller = false if err := r.setOwnerRefForResources(ctx, *ownerRef, resources); err != nil { errs = append(errs, err) } } return resources, errs } //組織當前應用的status值(所有組件的狀態,就緒組件數量,應用conditions) func (r *ApplicationReconciler) getNewApplicationStatus(ctx context.Context, app *appv1beta1.Application, resources []*unstructured.Unstructured, errList *[]error) *appv1beta1.ApplicationStatus { //獲取當前應用的組件資源實例的狀態 objectStatuses := r.objectStatuses(ctx, resources, errList) errs := utilerrors.NewAggregate(*errList) //應用是否就緒 就緒組件個數 aggReady, countReady := aggregateReady(objectStatuses) newApplicationStatus := app.Status.DeepCopy() newApplicationStatus.ComponentList = appv1beta1.ComponentList{ Objects: objectStatuses, } newApplicationStatus.ComponentsReady = fmt.Sprintf("%d/%d", countReady, len(objectStatuses)) if errs != nil { setReadyUnknownCondition(newApplicationStatus, "ComponentsReadyUnknown", "failed to aggregate all components' statuses, check the Error condition for details") } else if aggReady { setReadyCondition(newApplicationStatus, "ComponentsReady", "all components ready") } else { setNotReadyCondition(newApplicationStatus, "ComponentsNotReady", fmt.Sprintf("%d components not ready", len(objectStatuses)-countReady)) } if errs != nil { setErrorCondition(newApplicationStatus, "ErrorSeen", errs.Error()) } else { clearErrorCondition(newApplicationStatus) } return newApplicationStatus } //根據selector獲取組件資源實例 func (r *ApplicationReconciler) fetchComponentListResources(ctx context.Context, groupKinds []metav1.GroupKind, selector *metav1.LabelSelector, namespace string, errs *[]error) []*unstructured.Unstructured { logger := getLoggerOrDie(ctx) var resources []*unstructured.Unstructured if selector == nil { logger.Info("No selector is specified") return resources } for _, gk := range groupKinds { mapping, err := r.Mapper.RESTMapping(schema.GroupKind{ Group: appv1beta1.StripVersion(gk.Group), Kind: gk.Kind, }) if err != nil { logger.Info("NoMappingForGK", "gk", gk.String()) continue } list := &unstructured.UnstructuredList{} list.SetGroupVersionKind(mapping.GroupVersionKind) if err = r.Client.List(ctx, list, client.InNamespace(namespace), client.MatchingLabels(selector.MatchLabels)); err != nil { logger.Error(err, "unable to list resources for GVK", "gvk", mapping.GroupVersionKind) *errs = append(*errs, err) continue } for _, u := range list.Items { resource := u resources = append(resources, &resource) } } return resources } //給當前應用的組件資源實例維護級聯關系 func (r *ApplicationReconciler) setOwnerRefForResources(ctx context.Context, ownerRef metav1.OwnerReference, resources []*unstructured.Unstructured) error { logger := getLoggerOrDie(ctx) for _, resource := range resources { ownerRefs := resource.GetOwnerReferences() ownerRefFound := false for i, refs := range ownerRefs { if ownerRef.Kind == refs.Kind && ownerRef.APIVersion == refs.APIVersion && ownerRef.Name == refs.Name { ownerRefFound = true if ownerRef.UID != refs.UID { ownerRefs[i] = ownerRef } } } if !ownerRefFound { ownerRefs = append(ownerRefs, ownerRef) } resource.SetOwnerReferences(ownerRefs) err := r.Client.Update(ctx, resource) if err != nil { // We log this error, but we continue and try to set the ownerRefs on the other resources. logger.Error(err, "ErrorSettingOwnerRef", "gvk", resource.GroupVersionKind().String(), "namespace", resource.GetNamespace(), "name", resource.GetName()) } } return nil } //獲取當前應用的組件資源實例的狀態 func (r *ApplicationReconciler) objectStatuses(ctx context.Context, resources []*unstructured.Unstructured, errs *[]error) []appv1beta1.ObjectStatus { logger := getLoggerOrDie(ctx) var objectStatuses []appv1beta1.ObjectStatus for _, resource := range resources { os := appv1beta1.ObjectStatus{ Group: resource.GroupVersionKind().Group, Kind: resource.GetKind(), Name: resource.GetName(), Link: resource.GetSelfLink(), } s, err := status(resource) if err != nil { logger.Error(err, "unable to compute status for resource", "gvk", resource.GroupVersionKind().String(), "namespace", resource.GetNamespace(), "name", resource.GetName()) *errs = append(*errs, err) } os.Status = s objectStatuses = append(objectStatuses, os) } return objectStatuses } //計算當前應用就緒組件的數量是不是和組件總數一致 func aggregateReady(objectStatuses []appv1beta1.ObjectStatus) (bool, int) { countReady := 0 for _, os := range objectStatuses { if os.Status == StatusReady { countReady++ } } if countReady == len(objectStatuses) { return true, countReady } return false, countReady } func (r *ApplicationReconciler) updateApplicationStatus(ctx context.Context, nn types.NamespacedName, status *appv1beta1.ApplicationStatus) error { if err := retry.RetryOnConflict(retry.DefaultRetry, func() error { original := &appv1beta1.Application{} if err := r.Get(ctx, nn, original); err != nil { return err } original.Status = *status if err := r.Client.Status().Update(ctx, original); err != nil { return err } return nil }); err != nil { return fmt.Errorf("failed to update status of Application %s/%s: %v", nn.Namespace, nn.Name, err) } return nil } func (r *ApplicationReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). For(&appv1beta1.Application{}). Complete(r) } func getLoggerOrDie(ctx context.Context) logr.Logger { logger, ok := ctx.Value(loggerCtxKey).(logr.Logger) if !ok { panic("context didn't contain logger") } return logger }
// Copyright 2020 The Kubernetes Authors. // SPDX-License-Identifier: Apache-2.0 package controllers import ( "strings" appsv1 "k8s.io/api/apps/v1" batchv1 "k8s.io/api/batch/v1" corev1 "k8s.io/api/core/v1" policyv1beta1 "k8s.io/api/policy/v1beta1" "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" "k8s.io/apimachinery/pkg/runtime" ) // Constants defining labels const ( StatusReady = "Ready" StatusInProgress = "InProgress" StatusUnknown = "Unknown" StatusDisabled = "Disabled" ) //獲取對象狀態 func status(u *unstructured.Unstructured) (string, error) { gk := u.GroupVersionKind().GroupKind() switch gk.String() { case "StatefulSet.apps": return stsStatus(u) case "Deployment.apps": return deploymentStatus(u) case "ReplicaSet.apps": return replicasetStatus(u) case "DaemonSet.apps": return daemonsetStatus(u) case "PersistentVolumeClaim": return pvcStatus(u) case "Service": return serviceStatus(u) case "Pod": return podStatus(u) case "PodDisruptionBudget.policy": return pdbStatus(u) case "ReplicationController": return replicationControllerStatus(u) case "Job.batch": return jobStatus(u) default: return statusFromStandardConditions(u) } } // Status from standard conditions func statusFromStandardConditions(u *unstructured.Unstructured) (string, error) { condition := StatusReady // Check Ready condition _, cs, found, err := getConditionOfType(u, StatusReady) if err != nil { return StatusUnknown, err } if found && cs == corev1.ConditionFalse { condition = StatusInProgress } // Check InProgress condition _, cs, found, err = getConditionOfType(u, StatusInProgress) if err != nil { return StatusUnknown, err } if found && cs == corev1.ConditionTrue { condition = StatusInProgress } return condition, nil } // Statefulset func stsStatus(u *unstructured.Unstructured) (string, error) { sts := &appsv1.StatefulSet{} if err := runtime.DefaultUnstructuredConverter.FromUnstructured(u.Object, sts); err != nil { return StatusUnknown, err } if sts.Status.ObservedGeneration == sts.Generation && sts.Status.Replicas == *sts.Spec.Replicas && sts.Status.ReadyReplicas == *sts.Spec.Replicas && sts.Status.CurrentReplicas == *sts.Spec.Replicas { return StatusReady, nil } return StatusInProgress, nil } // Deployment func deploymentStatus(u *unstructured.Unstructured) (string, error) { deployment := &appsv1.Deployment{} if err := runtime.DefaultUnstructuredConverter.FromUnstructured(u.Object, deployment); err != nil { return StatusUnknown, err } replicaFailure := false progressing := false available := false for _, condition := range deployment.Status.Conditions { switch condition.Type { case appsv1.DeploymentProgressing: if condition.Status == corev1.ConditionTrue && condition.Reason == "NewReplicaSetAvailable" { progressing = true } case appsv1.DeploymentAvailable: if condition.Status == corev1.ConditionTrue { available = true } case appsv1.DeploymentReplicaFailure: if condition.Status == corev1.ConditionTrue { replicaFailure = true break } } } if deployment.Status.ObservedGeneration == deployment.Generation && deployment.Status.Replicas == *deployment.Spec.Replicas && deployment.Status.ReadyReplicas == *deployment.Spec.Replicas && deployment.Status.AvailableReplicas == *deployment.Spec.Replicas && deployment.Status.Conditions != nil && len(deployment.Status.Conditions) > 0 && (progressing || available) && !replicaFailure { return StatusReady, nil } return StatusInProgress, nil } // Replicaset func replicasetStatus(u *unstructured.Unstructured) (string, error) { rs := &appsv1.ReplicaSet{} if err := runtime.DefaultUnstructuredConverter.FromUnstructured(u.Object, rs); err != nil { return StatusUnknown, err } replicaFailure := false for _, condition := range rs.Status.Conditions { switch condition.Type { case appsv1.ReplicaSetReplicaFailure: if condition.Status == corev1.ConditionTrue { replicaFailure = true break } } } if rs.Status.ObservedGeneration == rs.Generation && rs.Status.Replicas == *rs.Spec.Replicas && rs.Status.ReadyReplicas == *rs.Spec.Replicas && rs.Status.AvailableReplicas == *rs.Spec.Replicas && !replicaFailure { return StatusReady, nil } return StatusInProgress, nil } // Daemonset func daemonsetStatus(u *unstructured.Unstructured) (string, error) { ds := &appsv1.DaemonSet{} if err := runtime.DefaultUnstructuredConverter.FromUnstructured(u.Object, ds); err != nil { return StatusUnknown, err } if ds.Status.ObservedGeneration == ds.Generation && ds.Status.DesiredNumberScheduled == ds.Status.NumberAvailable && ds.Status.DesiredNumberScheduled == ds.Status.NumberReady { return StatusReady, nil } return StatusInProgress, nil } // PVC func pvcStatus(u *unstructured.Unstructured) (string, error) { pvc := &corev1.PersistentVolumeClaim{} if err := runtime.DefaultUnstructuredConverter.FromUnstructured(u.Object, pvc); err != nil { return StatusUnknown, err } if pvc.Status.Phase == corev1.ClaimBound { return StatusReady, nil } return StatusInProgress, nil } // Service func serviceStatus(u *unstructured.Unstructured) (string, error) { service := &corev1.Service{} if err := runtime.DefaultUnstructuredConverter.FromUnstructured(u.Object, service); err != nil { return StatusUnknown, err } stype := service.Spec.Type if stype == corev1.ServiceTypeClusterIP || stype == corev1.ServiceTypeNodePort || stype == corev1.ServiceTypeExternalName || stype == corev1.ServiceTypeLoadBalancer && isEmpty(service.Spec.ClusterIP) && len(service.Status.LoadBalancer.Ingress) > 0 && !hasEmptyIngressIP(service.Status.LoadBalancer.Ingress) { return StatusReady, nil } return StatusInProgress, nil } // Pod func podStatus(u *unstructured.Unstructured) (string, error) { pod := &corev1.Pod{} if err := runtime.DefaultUnstructuredConverter.FromUnstructured(u.Object, pod); err != nil { return StatusUnknown, err } for _, condition := range pod.Status.Conditions { if condition.Type == corev1.PodReady && (condition.Reason == "PodCompleted" || condition.Status == corev1.ConditionTrue) { return StatusReady, nil } } return StatusInProgress, nil } // PodDisruptionBudget func pdbStatus(u *unstructured.Unstructured) (string, error) { pdb := &policyv1beta1.PodDisruptionBudget{} if err := runtime.DefaultUnstructuredConverter.FromUnstructured(u.Object, pdb); err != nil { return StatusUnknown, err } if pdb.Status.ObservedGeneration == pdb.Generation && pdb.Status.CurrentHealthy >= pdb.Status.DesiredHealthy { return StatusReady, nil } return StatusInProgress, nil } func replicationControllerStatus(u *unstructured.Unstructured) (string, error) { rc := &corev1.ReplicationController{} if err := runtime.DefaultUnstructuredConverter.FromUnstructured(u.Object, rc); err != nil { return StatusUnknown, err } if rc.Status.ObservedGeneration == rc.Generation && rc.Status.Replicas == *rc.Spec.Replicas && rc.Status.ReadyReplicas == *rc.Spec.Replicas && rc.Status.AvailableReplicas == *rc.Spec.Replicas { return StatusReady, nil } return StatusInProgress, nil } func jobStatus(u *unstructured.Unstructured) (string, error) { job := &batchv1.Job{} if err := runtime.DefaultUnstructuredConverter.FromUnstructured(u.Object, job); err != nil { return StatusUnknown, err } if job.Status.StartTime == nil { return StatusInProgress, nil } return StatusReady, nil } func hasEmptyIngressIP(ingress []corev1.LoadBalancerIngress) bool { for _, i := range ingress { if isEmpty(i.IP) { return true } } return false } func isEmpty(s string) bool { return len(strings.TrimSpace(s)) == 0 } func getConditionOfType(u *unstructured.Unstructured, conditionType string) (string, corev1.ConditionStatus, bool, error) { conditions, found, err := unstructured.NestedSlice(u.Object, "status", "conditions") if err != nil || !found { return "", corev1.ConditionFalse, false, err } for _, c := range conditions { condition, ok := c.(map[string]interface{}) if !ok { continue } t, found := condition["type"] if !found { continue } condType, ok := t.(string) if !ok { continue } if condType == conditionType { reason := condition["reason"].(string) conditionStatus := condition["status"].(string) return reason, corev1.ConditionStatus(conditionStatus), true, nil } } return "", corev1.ConditionFalse, false, nil }
// Copyright 2020 The Kubernetes Authors. // SPDX-License-Identifier: Apache-2.0 package controllers import ( corev1 "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" appv1beta1 "sigs.k8s.io/application/api/v1beta1" ) func setReadyCondition(appStatus *appv1beta1.ApplicationStatus, reason, message string) { setCondition(appStatus, appv1beta1.Ready, corev1.ConditionTrue, reason, message) } // NotReady - shortcut to set ready condition to false func setNotReadyCondition(appStatus *appv1beta1.ApplicationStatus, reason, message string) { setCondition(appStatus, appv1beta1.Ready, corev1.ConditionFalse, reason, message) } // Unknown - shortcut to set ready condition to unknown func setReadyUnknownCondition(appStatus *appv1beta1.ApplicationStatus, reason, message string) { setCondition(appStatus, appv1beta1.Ready, corev1.ConditionUnknown, reason, message) } // setErrorCondition - shortcut to set error condition func setErrorCondition(appStatus *appv1beta1.ApplicationStatus, reason, message string) { setCondition(appStatus, appv1beta1.Error, corev1.ConditionTrue, reason, message) } // clearErrorCondition - shortcut to set error condition func clearErrorCondition(appStatus *appv1beta1.ApplicationStatus) { setCondition(appStatus, appv1beta1.Error, corev1.ConditionFalse, "NoError", "No error seen") } func setCondition(appStatus *appv1beta1.ApplicationStatus, ctype appv1beta1.ConditionType, status corev1.ConditionStatus, reason, message string) { var c *appv1beta1.Condition for i := range appStatus.Conditions { if appStatus.Conditions[i].Type == ctype { c = &appStatus.Conditions[i] } } if c == nil { addCondition(appStatus, ctype, status, reason, message) } else { // check message ? if c.Status == status && c.Reason == reason && c.Message == message { return } now := metav1.Now() c.LastUpdateTime = now if c.Status != status { c.LastTransitionTime = now } c.Status = status c.Reason = reason c.Message = message } } func addCondition(appStatus *appv1beta1.ApplicationStatus, ctype appv1beta1.ConditionType, status corev1.ConditionStatus, reason, message string) { now := metav1.Now() c := appv1beta1.Condition{ Type: ctype, LastUpdateTime: now, LastTransitionTime: now, Status: status, Reason: reason, Message: message, } appStatus.Conditions = append(appStatus.Conditions, c) }
3.4 查看應用詳情及其組件詳情
應用詳情:
會發現應用維護了其status值,包括所有組件的狀態,就緒組件數量,應用conditions。
apiVersion: app.k8s.io/v1beta1 kind: Application metadata: creationTimestamp: "2021-12-27T01:50:54Z" generation: 1 labels: app.kubernetes.io/name: test-app app.kubernetes.io/version: v1 managedFields: ..... manager: cb-controller-manager operation: Update time: "2021-12-27T01:50:54Z" name: test-app namespace: zmc-test resourceVersion: "1873685" selfLink: /apis/app.k8s.io/v1beta1/namespaces/zmc-test/applications/test-app uid: f9c4d23a-b5a8-40a9-b046-eea3a40e11dc spec: addOwnerRef: true componentKinds: - group: "" kind: Service - group: apps kind: Deployment - group: apps kind: StatefulSet - group: extensions kind: Ingress - group: servicemesh.zmc.io kind: Strategy - group: servicemesh.zmc.io kind: ServicePolicy selector: matchLabels: app.kubernetes.io/name: test-app app.kubernetes.io/version: v1 status: components: - kind: Service link: /api/v1/namespaces/zmc-test/services/nginx name: nginx status: Ready - group: apps kind: Deployment link: /apis/apps/v1/namespaces/zmc-test/deployments/nginx-v1 name: nginx-v1 status: Ready componentsReady: 2/2 conditions: - lastTransitionTime: "2021-12-27T01:50:58Z" lastUpdateTime: "2021-12-27T01:50:58Z" message: all components ready reason: ComponentsReady status: "True" type: Ready - lastTransitionTime: "2021-12-27T01:50:54Z" lastUpdateTime: "2021-12-27T01:50:54Z" message: No error seen reason: NoError status: "False" type: Error observedGeneration: 1
組件詳情:
查看當前應用的所有組件(deployment和service)都會發現它們元數據部分都維護了級聯關系,刪除應用的話對應會刪除應用的所有組件。
........ ownerReferences: - apiVersion: app.k8s.io/v1beta1 blockOwnerDeletion: true controller: false kind: Application name: app-1227-v1 uid: f9c4d23a-b5a8-40a9-b046-eea3a40e11dc .......