目錄
認證安全
任何用途操作集群的資源對象是,都要經歷三種安全相關的操作:
- 任何用戶來訪問時, 都需要完成kubernetes系統認證操作
- 認證通過后, 進行授權檢查
- 准入控制, 檢查是否有權限操作其它的一些資源操作
認證方式:
- 令牌認證
- SSL 秘鑰認證, 也是最常用的方式.
- RBAC 全程:
Role Base AccessControl授權檢查機制
客戶端 --> API Server 傳遞的參數.
user: username, uid
group:
extra:
API
Request path:
/apis/apps/v1/namespaces/default/deployments/myapp-deploy/
請求動作 HTTP request verb:
get post pu delete
映射到kubernetes請求的動作:
get list create update patch watch proxy redirect delete deletecollection
請求訪問的資源 Resource
請求的子資源 Subresource
API group
serviceAccountName 和 userAccount
kubernetes 有兩類認證值的用戶賬號，分別是
serviceAccountName: 是用於Pod客戶端認證使用的賬戶
userAccount: 用於真實用戶，或者說操作集群資源的人使用的賬戶
serviceaccount 創建
可以使用kubectl create serviceaccount USER_NAME 來創建
[root@master manifests]# kubectl create serviceaccount admin
serviceaccount/admin created
[root@master manifests]# kubectl get sa
NAME SECRETS AGE
admin 1 2s
default 1 37d
[root@master manifests]# kubectl describe sa admin
Name: admin
Namespace: default
Labels: <none>
Annotations: <none>
Image pull secrets: <none>
Mountable secrets: admin-token-48wdj
Tokens: admin-token-48wdj # 自動生產的token
Events: <none>
當創建一個sa后，會自動創建一個secret
[root@master manifests]# kubectl get secret
NAME TYPE DATA AGE
admin-token-48wdj kubernetes.io/service-account-token 3 3m3s # 這里自動創建的是認證信息，但不代表有權限。
default-token-bc86p kubernetes.io/service-account-token 3 37d
[root@master manifests]# kubectl describe secret admin-token-48wdj
Name: admin-token-48wdj
Namespace: default
Labels: <none>
Annotations: kubernetes.io/service-account.name: admin
kubernetes.io/service-account.uid: dc343339-8eaf-4022-a03a-4cc8df8c9ebf
Type: kubernetes.io/service-account-token
Data
====
token: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImFkbWluLXRva2VuLTQ4d2RqIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiZGMzNDMzMzktOGVhZi00MDIyLWEwM2EtNGNjOGRmOGM5ZWJmIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50OmRlZmF1bHQ6YWRtaW4ifQ.xgehKowaM6sHykd6vbfi4hu4M0DxFAYrfuY9fkN_8vkcpi_c4HC8yl0MZD4GQivRVyEPb_7X6A81BkaHLVEHvbt7bl20Gk5F2tnFxWbSGMGNh2pRa0GxtnWEO03EyGHt1Yl_YGdqT13UnwfUFC5302P7dvuC15OKjd3az-vaMu3YgUkAKyceZZRltBasmoWVHdAY1u2kVFpjT60TYPtAyV6eAeDaucB94Ye60EmyhAHBNP8DkdFbWcHT25rdRQa72B_IVZ_ZaeCfJtZkpsg4XplniC1OuXV_25nqXCNRbIqp_yaH7dT0NKhT9_EYEC4b8MYZcL2e_RZyNgBWSUf-9Q
ca.crt: 1025 bytes
namespace: 7 bytes
使用admin 的SA
定義一個清單文件，使用admin的SA
[root@master manifests]# cat pod-sa-demo.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-sa-demo
namespace: default
labels:
app: myapp
tier: frontend
annotations:
jubaozhu.com/created-by: "cluster admin"
spec:
containers:
- name: myapp
image: ikubernetes/myapp:v1
serviceAccountName: admin
[root@master manifests]# kubectl apply -f pod-sa-demo.yaml
pod/pod-sa-demo created
[root@master manifests]# kubectl get pods
NAME READY STATUS RESTARTS AGE
pod-sa-demo 1/1 Running 0 3s
查看該Pods的詳細信息中servername
[root@master manifests]# kubectl describe pods pod-sa-demo | grep SecretName
SecretName: admin-token-48wdj # 這里和自動升昌的admin對應自動生成的secret名稱是一樣的。
也可以通過--dry-run 和 -o yaml 來獲取yaml配置文件
[root@master manifests]# kubectl create serviceaccount admin --dry-run -o yaml
apiVersion: v1
kind: ServiceAccount
metadata:
creationTimestamp: null
name: admin
這樣就能把導出的yaml格式的保存到文件中，在使用kubectl apply 來執行。
測試 URL訪問kubernetes資源
首先，本地要啟動一個proxy代理，之后通過這個代理來訪問
[root@master ~]# kubectl proxy --port=8080
Starting to serve on 127.0.0.1:8080
再起一個新的終端進行測試
[root@master pki]# curl http://localhost:8080/apis/apps/v1/namespaces/kube-system/deployments/coredns
{
"kind": "Deployment",
"apiVersion": "apps/v1",
"metadata": {
"name": "coredns",
"namespace": "kube-system",
"selfLink": "/apis/apps/v1/namespaces/kube-system/deployments/coredns",
"uid": "c730eb98-8fe7-47a8-9856-1fc3ff2044aa",
"resourceVersion": "1039",
"generation": 1,
"creationTimestamp": "2019-07-09T08:36:50Z",
"labels": {
"k8s-app": "kube-dns"
},
"annotations": {
"deployment.kubernetes.io/revision": "1"
}
},
"spec": {
"replicas": 2,
"selector": {
"matchLabels": {
"k8s-app": "kube-dns"
... ...
... ...
/apis/apps/v1/namespaces/kube-system/deployments/coredns 這里的訪問路徑都是固定套路。
這里的路徑區別
以 api 其實的資源，均為核心資源
剩下的其他資源，都需要以 apis 為起始路徑。
有哪些資源的客戶端需要和APIserver有交互
[root@master pki]# kubectl config set-cluster mycluster --kubeconfig=/tmp/test.cnf --server="https://10.0.20.20:6443" --certificate-authority=/etc/kubernetes/pki/ca.crt --embed-certs=true
Cluster "mycluster" set.
[root@master pki]# kubectl config view --kubeconfig=/tmp/test.cnf
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: DATA+OMITTED
server: https://10.0.20.20:6443
name: mycluster
contexts: []
current-context: ""
kind: Config
preferences: {}
users: []
授權插件:
- Node
- ABAC
- RBAC
- Webhook http的回調機制
角色機制: 給特定的角色賦權, 在把對應的角色賦予給用戶, 那么該用戶就擁有該角色的權限.
APIserver客戶端定義的配置文件
查看config命令幫助：
[root@master manifests]# kubectl config --help
Modify kubeconfig files using subcommands like "kubectl config set current-context my-context"
The loading order follows these rules:
1. If the --kubeconfig flag is set, then only that file is loaded. The flag may only be set once and no merging takes
place.
2. If $KUBECONFIG environment variable is set, then it is used as a list of paths (normal path delimiting rules for
your system). These paths are merged. When a value is modified, it is modified in the file that defines the stanza. When
a value is created, it is created in the first file that exists. If no files in the chain exist, then it creates the
last file in the list.
3. Otherwise, ${HOME}/.kube/config is used and no merging takes place.
Available Commands:
current-context Displays the current-context
delete-cluster Delete the specified cluster from the kubeconfig
delete-context Delete the specified context from the kubeconfig
get-clusters Display clusters defined in the kubeconfig
get-contexts Describe one or many contexts
rename-context Renames a context from the kubeconfig file.
set Sets an individual value in a kubeconfig file
set-cluster Sets a cluster entry in kubeconfig
set-context Sets a context entry in kubeconfig
set-credentials Sets a user entry in kubeconfig
unset Unsets an individual value in a kubeconfig file
use-context Sets the current-context in a kubeconfig file
view Display merged kubeconfig settings or a specified kubeconfig file
Usage:
kubectl config SUBCOMMAND [options]
Use "kubectl <command> --help" for more information about a given command.
Use "kubectl options" for a list of global command-line options (applies to all commands).
| 名稱 | 注解 |
|---|---|
| set-cluster | 設定集群 |
| set-credentials | 設定用戶賬號 |
| set-context | 設定上下文 |
| use-context | 設定當前上下文 |
可以直接通過命令的方式進行創建，也可以使用配置清單來定義后，使用kubectl apply -f 來執行設定；
查看當前配置文件的示例說明
[root@master manifests]# kubectl config view
apiVersion: v1 # 這里可以看出，config也是一個標准的kubernetes資源對象
clusters: # 集群列表
- cluster: # 定義一個集群，在下面可以定義多個
certificate-authority-data: DATA+OMITTED # 服務器發過來的認證方式
server: https://10.0.20.20:6443 # 訪問驗證地址
name: kubernetes # 集群名稱
contexts: # 上下文列表
- context:
cluster: kubernetes # 對應用戶能訪問的集群名稱
user: kubernetes-admin # 該上下文對應的用戶
name: kubernetes-admin@kubernetes # 此上下文的名稱
current-context: kubernetes-admin@kubernetes # 當前上下文
kind: Config
preferences: {}
users: # 用戶列表
- name: kubernetes-admin # 用戶已名稱
user:
client-certificate-data: REDACTED # 客戶端的認證方式
client-key-data: REDACTED # key信息
kubernetes 集群相關的私有CA證書
[root@master ~]# ll /etc/kubernetes/pki/
-rw-r--r-- 1 root root 1233 Jul 9 16:36 apiserver.crt
-rw-r--r-- 1 root root 1090 Jul 9 16:36 apiserver-etcd-client.crt
-rw------- 1 root root 1675 Jul 9 16:36 apiserver-etcd-client.key
-rw------- 1 root root 1675 Jul 9 16:36 apiserver.key
-rw-r--r-- 1 root root 1099 Jul 9 16:36 apiserver-kubelet-client.crt
-rw------- 1 root root 1679 Jul 9 16:36 apiserver-kubelet-client.key
-rw-r--r-- 1 root root 1025 Jul 9 16:36 ca.crt
-rw------- 1 root root 1679 Jul 9 16:36 ca.key
drwxr-xr-x 2 root root 162 Jul 9 16:36 etcd
-rw-r--r-- 1 root root 1038 Jul 9 16:36 front-proxy-ca.crt
-rw------- 1 root root 1675 Jul 9 16:36 front-proxy-ca.key
-rw-r--r-- 1 root root 1058 Jul 9 16:36 front-proxy-client.crt
-rw------- 1 root root 1679 Jul 9 16:36 front-proxy-client.key
-rw------- 1 root root 1679 Jul 9 16:36 sa.key
-rw------- 1 root root 451 Jul 9 16:36 sa.pub
創建新的apiserver的賬號及證書
新建的密鑰對就直接放到kubernetes集群對應的pki目錄下
[root@master ~]# cd /etc/kubernetes/pki/
[root@master pki]# ll
total 60
-rw-r--r-- 1 root root 1233 Jul 9 16:36 apiserver.crt
-rw-r--r-- 1 root root 1090 Jul 9 16:36 apiserver-etcd-client.crt
-rw------- 1 root root 1675 Jul 9 16:36 apiserver-etcd-client.key
-rw------- 1 root root 1675 Jul 9 16:36 apiserver.key
-rw-r--r-- 1 root root 1099 Jul 9 16:36 apiserver-kubelet-client.crt
-rw------- 1 root root 1679 Jul 9 16:36 apiserver-kubelet-client.key
-rw-r--r-- 1 root root 1025 Jul 9 16:36 ca.crt
-rw------- 1 root root 1679 Jul 9 16:36 ca.key
drwxr-xr-x 2 root root 162 Jul 9 16:36 etcd
-rw-r--r-- 1 root root 1038 Jul 9 16:36 front-proxy-ca.crt
-rw------- 1 root root 1675 Jul 9 16:36 front-proxy-ca.key
-rw-r--r-- 1 root root 1058 Jul 9 16:36 front-proxy-client.crt
-rw------- 1 root root 1679 Jul 9 16:36 front-proxy-client.key
-rw------- 1 root root 1679 Jul 9 16:36 sa.key
-rw------- 1 root root 451 Jul 9 16:36 sa.pub
創建私鑰
[root@master pki]# (umask 077; openssl genrsa -out tracy.key 2048)
Generating RSA private key, 2048 bit long modulus
.........................+++
.....................................................+++
e is 65537 (0x10001)
[root@master pki]# ll tracy.key
-rw------- 1 root root 1679 Aug 19 16:07 tracy.key
然后基於tracy.key私鑰來生成證書使用kubernetes集群的ca.crt來簽署
生成證書簽署請求
[root@master pki]# openssl req -new -key tracy.key -out tracy.csr -subj "/CN=tracy"
# 注意，最后的 "/CN=tracy" 中的tracy，就是用戶名
用ca.crt 來簽署
[root@master pki]# openssl x509 -req -in tracy.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out tracy.csr -days 365
Signature ok
subject=/CN=tracy
Getting CA Private Key
[root@master pki]# ll tracy.*
-rw-r--r-- 1 root root 973 Aug 19 16:14 tracy.csr
-rw------- 1 root root 1679 Aug 19 16:07 tracy.key
驗證查看生成的證書
[root@master pki]# openssl x509 -in tracy.csr -text -noout
Certificate:
Data:
Version: 1 (0x0)
Serial Number:
ea:48:9e:47:1b:2e:19:ac
Signature Algorithm: sha256WithRSAEncryption
Issuer: CN=kubernetes # 這里可以看到是kubernetes自己的ca.crt證書簽署的
Validity
Not Before: Aug 19 08:14:53 2019 GMT # 證書有效期限起始
Not After : Aug 18 08:14:53 2020 GMT # 證書有效期限結束
Subject: CN=tracy # 作為用戶賬號連入kubernetes用戶
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:cd:e3:a1:7a:db:17:70:57:16:b3:f6:6a:6c:87:
50:5f:2f:38:86:67:93:fc:b2:ba:5f:c4:60:16:66:
2f:67:29:01:c7:f0:b5:d8:ca:c6:db:ca:42:8a:a8:
ce:17:ac:d1:43:64:eb:a8:1b:34:6c:97:d9:49:dc:
24:b9:ab:8c:cd:be:6a:ad:ff:70:64:40:51:40:0f:
16:d5:61:73:dc:3f:aa:57:4f:7a:c8:60:a7:37:ee:
53:d0:ec:25:78:fc:4a:af:81:67:2c:d8:08:0f:a0:
4d:38:0f:28:36:4b:f7:c6:bb:a2:0f:4b:33:e4:77:
44:83:4d:14:2e:ef:ce:d4:6a:47:37:09:0b:9d:41:
ad:07:62:8e:06:94:71:04:f7:1b:a2:e7:6a:d6:86:
8e:7c:45:69:5e:a2:74:22:6a:8f:47:c8:ef:ba:f7:
d4:08:05:6e:c0:ae:36:bc:68:4b:7f:a5:c8:1c:87:
c4:34:a7:28:94:96:49:b1:cc:53:c0:75:7b:c9:44:
f4:78:c9:f4:fa:e2:5a:55:64:0d:dd:0a:40:31:19:
4e:80:f2:cf:02:f4:48:ae:65:93:a9:e7:41:d7:0b:
e1:7a:27:38:dc:4f:9d:12:be:8e:62:2b:73:98:0c:
45:07:c5:ad:43:91:06:0e:30:4c:5c:63:9d:25:de:
bb:29
Exponent: 65537 (0x10001)
Signature Algorithm: sha256WithRSAEncryption
74:7b:c9:fd:9e:2d:4e:84:c1:f8:83:8d:9a:37:c1:50:27:e3:
f6:1c:d4:ea:ae:4f:83:f3:dc:23:81:01:fd:3d:02:e1:26:7c:
8a:94:a7:dd:0a:3c:86:52:45:2a:25:b3:63:3f:51:15:87:0f:
f9:25:8b:eb:b0:7e:0f:7c:4a:f1:23:b9:72:7e:c6:87:55:16:
7a:bf:29:2e:da:5f:3c:4c:f2:b2:77:09:fb:fe:5a:38:67:1b:
db:c5:db:4b:64:87:1c:c7:e8:71:84:92:fd:39:17:9d:7b:b3:
dc:4b:1a:56:a8:75:63:1e:be:80:ac:98:a6:d7:41:96:8d:44:
b4:13:79:c6:08:07:b8:4a:8c:5b:04:09:a6:8d:73:47:b5:a8:
0b:97:40:66:cf:04:bc:01:93:4a:6a:ed:b2:91:94:51:12:0e:
dd:c9:70:c9:fd:65:ed:49:c2:1a:89:0f:6e:ed:69:1c:d5:2c:
cb:42:f4:ce:69:bd:99:a2:ff:ea:a7:69:5f:f7:43:b5:c3:03:
86:67:95:6e:1e:80:c7:8c:35:ac:7f:e6:e2:4e:11:08:dc:cd:
16:02:e1:cd:59:4d:70:e8:fe:5a:85:4a:18:97:bb:97:f4:6a:
82:ee:e3:c1:73:b4:2a:24:73:bd:f1:fc:08:02:6c:5d:e1:53:
設定用戶賬號
[root@master pki]# kubectl config set-credentials tracy --client-certificate=tracy.csr --client-key=tracy.key --embed-certs=true
User "tracy" set.
[root@master pki]# kubectl config view
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: DATA+OMITTED
server: https://10.0.20.20:6443
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: kubernetes-admin
name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
- name: tracy # 這里可以看到剛剛設定的用戶賬號
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
tracy加入上下文
[root@master pki]# kubectl config set-context tracy@kubernetes --cluster=kubernetes --user=tracy
Context "tracy@kubernetes" created.
[root@master pki]# kubectl config view
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: DATA+OMITTED
server: https://10.0.20.20:6443
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: kubernetes-admin
name: kubernetes-admin@kubernetes
- context: # 這里開始向下就是tracy的上下文相關的配置了
cluster: kubernetes
user: tracy
name: tracy@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
- name: tracy
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
應用剛剛增加的tracy的對應的上下文
[root@master pki]# kubectl config use-context tracy@kubernetes
Switched to context "tracy@kubernetes".
[root@master pki]# kubectl config view
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: DATA+OMITTED
server: https://10.0.20.20:6443
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: kubernetes-admin
name: kubernetes-admin@kubernetes
- context:
cluster: kubernetes
user: tracy
name: tracy@kubernetes
current-context: tracy@kubernetes # 看到當前上下文已經切換到 tracy@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
- name: tracy
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
測試
由於tracy@kubernetes 沒有賦予任何角色，所以對集群是沒有任何操作權限
[root@master pki]# kubectl get pods
Error from server (Forbidden): pods is forbidden: User "tracy" cannot list resource "pods" in API group "" in the namespace "default"
這里報錯意思是 default 名稱空間中，不能獲取數據
重新切換至kubernetes-admin@kubernetes測試
[root@master pki]# kubectl config use-context kubernetes-admin@kubernetes
Switched to context "kubernetes-admin@kubernetes".
[root@master pki]# kubectl get pods
NAME READY STATUS RESTARTS AGE
pod-sa-demo 1/1 Running 0 3d3h
保存配置文件
默認 kubernetes-admin 的配置文件保存目錄是~/.kube/config文件中。
也可以保存到別的目錄下：
[root@master ~]# kubectl config set-cluster test --kubeconfig=/tmp/test.conf --server="https://10.0.20.20:6443" --certificate-authority=/etc/kubernetes/pki/ca.crt --embed-certs=true
Cluster "test" set.
[root@master ~]# kubectl config view --kubeconfig=/tmp/test.conf
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: DATA+OMITTED
server: https://10.0.20.20:6443
name: test
contexts: []