Kubebuilder認證配置文件的加載

在kubernetes二次開發-Kubebuilder最佳實踐中，我們簡單使用了Kubebuilder來資源創建、驗證等操作，那么你一定很好奇，程序是如何連接到api server做認證和鑒權的，下面我們來簡單看下。

來到main.go

	mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{
		Scheme:                 scheme,
		MetricsBindAddress:     metricsAddr,
		Port:                   9443,
		HealthProbeBindAddress: probeAddr,
		LeaderElection:         enableLeaderElection,
		LeaderElectionID:       "1de8eaa9.demo.kubebuilder.io",
	})

重點就在“ctrl.GetConfigOrDie()”中，追蹤后，最終會來到這個地方：

sigs.k8s.io/controller-runtime/pkg/client/config/config.go

// loadConfig loads a REST Config as per the rules specified in GetConfig.
func loadConfig(context string) (*rest.Config, error) {
	// If a flag is specified with the config location, use that
	if len(kubeconfig) > 0 {
		return loadConfigWithContext("", &clientcmd.ClientConfigLoadingRules{ExplicitPath: kubeconfig}, context)
	}

	// If the recommended kubeconfig env variable is not specified,
	// try the in-cluster config.
	kubeconfigPath := os.Getenv(clientcmd.RecommendedConfigPathEnvVar)
	if len(kubeconfigPath) == 0 {
		if c, err := loadInClusterConfig(); err == nil {
			return c, nil
		}
	}

	// If the recommended kubeconfig env variable is set, or there
	// is no in-cluster config, try the default recommended locations.
	//
	// NOTE: For default config file locations, upstream only checks
	// $HOME for the user's home directory, but we can also try
	// os/user.HomeDir when $HOME is unset.
	//
	// TODO(jlanford): could this be done upstream?
	loadingRules := clientcmd.NewDefaultClientConfigLoadingRules()
	if _, ok := os.LookupEnv("HOME"); !ok {
		u, err := user.Current()
		if err != nil {
			return nil, fmt.Errorf("could not get current user: %v", err)
		}
		loadingRules.Precedence = append(loadingRules.Precedence, filepath.Join(u.HomeDir, clientcmd.RecommendedHomeDir, clientcmd.RecommendedFileName))
	}

	return loadConfigWithContext("", loadingRules, context)
}

該方法完成的功能如下：

• （1）如果初始化了kubeconfig，則從kubeconfig中讀取集群配置

• （2）否則從環境變量KUBECONFIG讀取，若沒有則從集群內部讀取，這種場景適用於部署到kubernetes中的場景，它是這樣讀取的：

  
  // InClusterConfig returns a config object which uses the service account
  // kubernetes gives to pods. It's intended for clients that expect to be
  // running inside a pod running on kubernetes. It will return ErrNotInCluster
  // if called from a process not running in a kubernetes environment.
  func InClusterConfig() (*Config, error) {
      const (
          tokenFile  = "/var/run/secrets/kubernetes.io/serviceaccount/token"
          rootCAFile = "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
      )
      host, port := os.Getenv("KUBERNETES_SERVICE_HOST"), os.Getenv("KUBERNETES_SERVICE_PORT")
      if len(host) == 0 || len(port) == 0 {
          return nil, ErrNotInCluster
      }
  
      token, err := ioutil.ReadFile(tokenFile)
      if err != nil {
          return nil, err
      }
  
      tlsClientConfig := TLSClientConfig{}
  
      if _, err := certutil.NewPool(rootCAFile); err != nil {
          klog.Errorf("Expected to load root CA config from %s, but got err: %v", rootCAFile, err)
      } else {
          tlsClientConfig.CAFile = rootCAFile
      }
  
      return &Config{
          // TODO: switch to using cluster DNS.
          Host:            "https://" + net.JoinHostPort(host, port),
          TLSClientConfig: tlsClientConfig,
          BearerToken:     string(token),
          BearerTokenFile: tokenFile,
      }, nil
  }
  

  讀取POD上內“/var/run/secrets/kubernetes.io/serviceaccount/”下的“token”和“ca.crt”文件，如：

  [root@master kubebuilder-demo]# kubectl get pods
  NAME             READY   STATUS    RESTARTS   AGE
  redis-sample-0   1/1     Running   0          25m
  redis-sample-1   1/1     Running   0          25m
  #
  [root@master kubebuilder-demo]# kubectl exec -it redis-sample-0 -- sh
  /data # ls -l /var/run/secrets/kubernetes.io/serviceaccount
  total 0
  lrwxrwxrwx    1 root     root            13 Feb 17 10:16 ca.crt -> ..data/ca.crt
  lrwxrwxrwx    1 root     root            16 Feb 17 10:16 namespace -> ..data/namespace
  lrwxrwxrwx    1 root     root            12 Feb 17 10:16 token -> ..data/token
  /data # 
  /data # cat /var/run/secrets/kubernetes.io/serviceaccount/token
  eyJhbGciOiJSUzI1NiIsImtpZCI6IkxONVBTQm90R3JaT21ET3pkdmZhaWN1ak9lcGZ0WjRNemRudUhndjNmRGcifQ.eyJhdWQiOlsiaHR0cHM6Ly9rdWJlcm5ldGVzLmRlZmF1bHQuc3ZjLmNsdXN0ZXIubG9jYWwiXSwiZXhwIjoxNjc2NjI4OTc0LCJpYXQiOjE2NDUwOTI5NzQsImlzcyI6Imh0dHBzOi8va3ViZXJuZXRlcy5kZWZhdWx0LnN2Yy5jbHVzdGVyLmxvY2FsIiwia3ViZXJuZXRlcy5pbyI6eyJuYW1lc3BhY2UiOiJkZWZhdWx0IiwicG9kIjp7Im5hbWUiOiJyZWRpcy1zYW1wbGUtMCIsInVpZCI6ImZmMTA3YWNhLTU1NzYtNDc4NS04Y2YwLWE4YWE4ZTE2NmU4ZCJ9LCJzZXJ2aWNlYWNjb3VudCI6eyJuYW1lIjoiZGVmYXVsdCIsInVpZCI6ImZkNWMzNTQ3LWY5ZjctNDZlOC1iNjFhLTdjNWVmMjczMWU4ZSJ9LCJ3YXJuYWZ0ZXIiOjE2NDUwOTY1ODF9LCJuYmYiOjE2NDUwOTI5NzQsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDpkZWZhdWx0OmRlZmF1bHQifQ.xUf0OZxfTPitPgFF9AUKX439G_BJv5MDY25uTvpa8zj5QkZzaaV-238cZEpMU_cG3i2gtA_xfsw2nXKvfedmv1ZPbtcnovEVP-rCunO5DD8tSm478lsx0RxgzhJpvaVLxwyxwPeQyM8wcVPsXUvYt1ZvlemWWYqX739bRApHFsXIKtUhMAcvhz7byCfATBYLO0TFbrEUWNkT8y8ZDgqoogzRYs6cKi1thGuEaqF406Kt0GYUl06KjEAOdbzHyVpu-bsTz_OOZXWQVxSCquMrTZdffOK11DJrtADORPdavEQOde1Kf-LXaRdxh_-NbUVo9alFyfwiv9gegcUjXNHc6w/data # 
  /data # cat /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
  -----BEGIN CERTIFICATE-----
  MIIC5zCCAc+gAwIBAgIBADANBgkqhkiG9w0BAQsFADAVMRMwEQYDVQQDEwprdWJl
  cm5ldGVzMB4XDTIyMDIwOTA1MzIyOVoXDTMyMDIwNzA1MzIyOVowFTETMBEGA1UE
  AxMKa3ViZXJuZXRlczCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMuF
  j6sLBDFDukzsS6WmwyhqHqkFpz2IZpVT8SaPdxtuKJcLRQGZWs5slWIlJY2tYhb0
  BUO1YUP4b83lT61ZlBASZJYfKQ3UUyMiCkAUqy/Bxih8ItibYBJxXcK8nVMqgZVF
  aFSKF0psfm3MZsNWuStYn2qLrdLAE1P4JeDkd+E+iX0t+DfEQdjvgfuJwzfUC7Ip
  bN3XvXCBkV3oTo+61Ijv0aygUhQ3nD5H9Q+Fyh8pWwFBQUVec++2t/MVRtriSXoS
  510YbtsYr08RXzjv7w0kUV7TFnYaDdSbiIYgYGwbRwhNsQua8AH4jtbYGbzvFx0u
  vmV+kx3mtZ71NiwpNK0CAwEAAaNCMEAwDgYDVR0PAQH/BAQDAgKkMA8GA1UdEwEB
  /wQFMAMBAf8wHQYDVR0OBBYEFMC5ezheys2mejyq1qMrijX6jgCrMA0GCSqGSIb3
  DQEBCwUAA4IBAQAUgRrfXor/MegRzLtUZITVZTM7nXM13BYVeqjxMIdT16A5pwZM
  N44SG9Q2xezC3WhPoT69qg8tR+EoqJBY+o/00mH4uMOkED+Dbu4J6QCwBg0g/v2T
  4sifXh4tmYybCxCvdh/ZS7lmROFXYJXpoPbIQ/n1cIABpwbGPLaQKk+apIlmE61Q
  5iLSeT7RAKULm2gpJc122wVDHvk1vzhn0u+6SDGHKmjQIYFceGWLecfzNAjTqOx8
  13MMYR2rck90ATArxyXKm6gtCbCs74jspU5dvUnJQxHIcrcVL4RWlo/tmU3+wci6
  S17d5NVpPFOnmjYeg2Cq0VsTDXjUHF/dNOuF
  -----END CERTIFICATE-----
  /data #
  

更多詳細信息見：https://kubernetes.io/docs/tasks/run-application/access-api-from-pod/#directly-accessing-the-rest-api

• （3）如果上面兩個地方都沒有找到，則會讀取默認配置。即：$HOME下

  //拼湊路徑：
  filepath.Join(u.HomeDir, clientcmd.RecommendedHomeDir, clientcmd.RecommendedFileName)//
  

  const (
  	RecommendedConfigPathFlag   = "kubeconfig"
  	RecommendedConfigPathEnvVar = "KUBECONFIG"
  	RecommendedHomeDir          = ".kube"
  	RecommendedFileName         = "config"
  	RecommendedSchemaName       = "schema"
  )
  

下面是摘錄自kubernetes文檔中，關於在POD內訪問api-server的一段描述，包含了如何訪問API-Server，如何通過認證

While running in a Pod, the Kubernetes apiserver is accessible via a Service named kubernetes in the default namespace. Therefore, Pods can use the kubernetes.default.svc hostname to query the API server. Official client libraries do this automatically.
當應用運行在POD內部時，它會使用default命名空間下的名為“kubernetes”的service來訪問API-Server，即使用kubernetes.default.svc來查詢api-server

The recommended way to authenticate to the API server is with a service account credential. By default, a Pod is associated with a service account, and a credential (token) for that service account is placed into the filesystem tree of each container in that Pod, at /var/run/secrets/kubernetes.io/serviceaccount/token.

If available, a certificate bundle is placed into the filesystem tree of each container at /var/run/secrets/kubernetes.io/serviceaccount/ca.crt, and should be used to verify the serving certificate of the API server.

Finally, the default namespace to be used for namespaced API operations is placed in a file at /var/run/secrets/kubernetes.io/serviceaccount/namespace in each container

token、namespace和ca.crt，這三個文件是怎么被放入到POD中的？
這些token、namespace和ca.crt是通過投射的方式，放入到POD中的，有關於該過程的詳細解釋見：https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/#bound-service-account-token-volume

token、namespace和ca.crt，這三個文件的作用？
這三個文件由於參與到Pod進程與API Server認證的過程中，起到了類似secret（私密憑據）的作用，所以它們被稱為Kubernetes Secret對象。Secret從屬於Service Account資源對象，屬於Service Account的一部分，在一個Service Account對象里面可以包括多個不同的Secret對象，分別用於不同目的的認證活動。
更多細節，可以查看《kubernetes權威指南 第5版- 6.4節　Service Account》

有關token，如何在認證過程中發揮作用？
Pod中的客戶端調用Kubernetes API時，在HTTP Header中傳遞了一個Token字符串，這類似於之前提到的HTTP Token認證方式，但有以下幾個不同之處。
◎　這個Token的內容來自Pod里指定路徑下的一個文件（/run/secrets/kubernetes.io/serviceaccount/token），這種Token是動態生成的，確切地說，是由Kubernetes Controller進程用API Server的私鑰（--service-account-private-key-file指定的私鑰）簽名生成的一個JWT Secret。
◎　在官方提供的客戶端REST框架代碼里，通過HTTPS方式與APIServer建立連接后，會用Pod里指定路徑下的一個CA證書（/run/secrets/kubernetes.io/serviceaccount/ca.crt）驗證API Server發來的證書，驗證是否為CA證書簽名的合法證書。
◎　API Server在收到這個Token以后，采用自己的私鑰（實際上是使用service-accountkey-file參數指定的私鑰，如果沒有設置此參數，則默認采用tls-private-key-file指定的參數，即自己的私鑰）對Token進行合法性驗證。

更多詳情見：https://kubernetes.io/zh-cn/docs/reference/access-authn-authz/