kubernetes Dashboard 使用RBAC 權限認證控制

kubernetes RBAC實戰

環境准備

先用kubeadm安裝好kubernetes集群，[包地址在此](https://market.aliyun.com/products/56014009/cmxz022571.html#sku=yuncode1657100000) 好用又方便，服務周到，童叟無欺

本文目的，讓名為devuser的用戶只能有權限訪問特定namespace下的pod

先用kubeadm安裝好kubernetes集群，[包地址在此](https://market.aliyun.com/products/56014009/cmxz022571.html#sku=yuncode1657100000) 好用又方便，服務周到，童叟無欺

本文目的，讓名為devuser的用戶只能有權限訪問特定namespace下的pod

命令行kubectl訪問

安裝cfssl

 此工具生成證書非常方便, pem證書與crt證書,編碼一致可直接使用 wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 chmod +x cfssl_linux-amd64 mv cfssl_linux-amd64 /bin/cfssl wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 chmod +x cfssljson_linux-amd64 mv cfssljson_linux-amd64 /bin/cfssljson wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 chmod +x cfssl-certinfo_linux-amd64 mv cfssl-certinfo_linux-amd64 /bin/cfssl-certinfo 

簽發客戶端證書

根據ca證書與么鑰簽發用戶證書

根證書已經在/etc/kubernetes/pki目錄下了

[root@master1 ~]# ls /etc/kubernetes/pki/ apiserver.crt ca-config.json devuser-csr.json front-proxy-ca.key sa.pub apiserver.key ca.crt devuser-key.pem front-proxy-client.crt apiserver-kubelet-client.crt ca.key devuser.pem front-proxy-client.key apiserver-kubelet-client.key devuser.csr front-proxy-ca.crt sa.key

注意以下幾個文件： `ca.crt ca.key ca-config.json devuser-csr.json`

創建ca-config.json文件

cat > ca-config.json < devuser-csr.json < 校驗證書 cfssl-certinfo -cert kubernetes.pem

生成config文件

kubeadm已經生成了admin.conf，我們可以直接利用這個文件，省的自己再去配置集群參數

$ cp /etc/kubernetes/admin.conf devuser.kubeconfig

設置客戶端認證參數:

kubectl config set-credentials devuser \ --client-certificate=/etc/kubernetes/ssl/devuser.pem \ --client-key=/etc/kubernetes/ssl/devuser-key.pem \ --embed-certs=true \ --kubeconfig=devuser.kubeconfig

設置上下文參數：

kubectl config set-context kubernetes \ --cluster=kubernetes \ --user=devuser \ --namespace=kube-system \ --kubeconfig=devuser.kubeconfig

設置莫認上下文：

kubectl config use-context kubernetes --kubeconfig=devuser.kubeconfig

以上執行一個步驟就可以看一下 devuser.kubeconfig的變化。里面最主要的三個東西

• cluster: 集群信息，包含集群地址與公鑰
• user: 用戶信息，客戶端證書與私鑰，正真的信息是從證書里讀取出來的，人能看到的只是給人看的。
• context: 維護一個三元組，namespace cluster 與 user

創建角色

創建一個叫pod-reader的角色

[root@master1 ~]# cat pod-reader.yaml kind: Role apiVersion: rbac.authorization.k8s.io/v1 metadata: namespace: kube-system name: pod-reader rules: - apiGroups: [""] # "" indicates the core API group resources: ["pods"] verbs: ["get", "watch", "list"]

kubectl create -f pod-reader.yaml

綁定用戶

創建一個角色綁定，把pod-reader角色綁定到 devuser上

[root@master1 ~]# cat devuser-role-bind.yaml kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: read-pods namespace: kube-system subjects: - kind: User name: devuser # 目標用戶 apiGroup: rbac.authorization.k8s.io roleRef: kind: Role name: pod-reader # 角色信息 apiGroup: rbac.authorization.k8s.io

kubectl create -f devuser-role-bind.yaml

使用新的config文件

$ rm .kube/config && cp devuser.kubeconfig .kube/config

效果, 已經沒有別的namespace的權限了，也不能訪問node信息了：

[root@master1 ~]# kubectl get node Error from server (Forbidden): nodes is forbidden: User "devuser" cannot list nodes at the cluster scope [root@master1 ~]# kubectl get pod -n kube-system NAME READY STATUS RESTARTS AGE calico-kube-controllers-55449f8d88-74x8f 1/1 Running 0 8d calico-node-clpqr 2/2 Running 0 8d kube-apiserver-master1 1/1 Running 2 8d kube-controller-manager-master1 1/1 Running 1 8d kube-dns-545bc4bfd4-p6trj 3/3 Running 0 8d kube-proxy-tln54 1/1 Running 0 8d kube-scheduler-master1 1/1 Running 1 8d [root@master1 ~]# kubectl get pod -n default Error from server (Forbidden): pods is forbidden: User "devuser" cannot list pods in the namespace "default": role.rbac.authorization.k8s.io "pod-reader" not found

dashboard訪問

service account原理

k8s里面有兩種用戶，一種是User，一種就是service account，User給人用的，service account給進程用的，讓進程有相關的權限。

如dasboard就是一個進程，我們就可以創建一個service account給它，讓它去訪問k8s。

我們看一下是如何把admin權限賦給dashboard的：

╰─➤ cat dashboard-admin.yaml apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding metadata: name: kubernetes-dashboard labels: k8s-app: kubernetes-dashboard roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cluster-admin subjects: - kind: ServiceAccount name: kubernetes-dashboard namespace: kube-system

把 kubernetes-dashboard 這個ServiceAccount綁定到cluster-admin這個ClusterRole上，這個cluster role非常牛逼，啥權限都有

[root@master1 ~]# kubectl describe clusterrole cluster-admin -n kube-system Name: cluster-admin Labels: kubernetes.io/bootstrapping=rbac-defaults Annotations: rbac.authorization.kubernetes.io/autoupdate=true PolicyRule: Resources Non-Resource URLs Resource Names Verbs --------- ----------------- -------------- ----- [*] [] [*] *.* [] [] [*]

而創建dashboard時創建了這個service account:

apiVersion: v1 kind: ServiceAccount metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kube-system

然后deployment里指定service account

volumes: - name: kubernetes-dashboard-certs secret: secretName: kubernetes-dashboard-certs - name: tmp-volume emptyDir: {} serviceAccountName: kubernetes-dashboard

更安全的做法

[root@master1 ~]# cat admin-token.yaml kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: admin annotations: rbac.authorization.kubernetes.io/autoupdate: "true" roleRef: kind: ClusterRole name: cluster-admin apiGroup: rbac.authorization.k8s.io subjects: - kind: ServiceAccount name: admin namespace: kube-system --- apiVersion: v1 kind: ServiceAccount metadata: name: admin namespace: kube-system labels: kubernetes.io/cluster-service: "true" addonmanager.kubernetes.io/mode: Reconcile

[root@master1 ~]# kubectl get secret -n kube-system|grep admin admin-token-7rdhf kubernetes.io/service-account-token 3 14m

[root@master1 ~]# kubectl describe secret admin-token-7rdhf -n kube-system Name: admin-token-7rdhf Namespace: kube-system Labels: Annotations: kubernetes.io/service-account.name=admin kubernetes.io/service-account.uid=affe82d4-d10b-11e7-ad03-00163e01d684 Type: kubernetes.io/service-account-token Data ==== ca.crt: 1025 bytes namespace: 11 bytes token: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi10b2tlbi03cmRoZiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJhZG1pbiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6ImFmZmU4MmQ0LWQxMGItMTFlNy1hZDAzLTAwMTYzZTAxZDY4NCIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlLXN5c3RlbTphZG1pbiJ9.jSfQhFsY7V0ZmfqxM8lM_UUOoUhI86axDSeyVVtldSUY-BeP2Nw4q-ooKGJTBBsrOWvMiQePcQxJTKR1K4EIfnA2FOnVm4IjMa40pr7-oRVY37YnR_1LMalG9vrWmqFiqIsKe9hjkoFDuCaP7UIuv16RsV7hRlL4IToqmJMyJ1xj2qb1oW4P1pdaRr4Pw02XBz9yBpD1fs-lbwheu1UKcEnbHS_0S3zlmAgCrpwDFl2UYOmgUKQVpJhX4wBRRQbwo1Sn4rEFVI1NIa9l_lM7Mf6YEquLHRu3BCZTdu9YfY9pevQz4OfHE0NOvDIqmGRL8Z9kPADAXbljWzcD1m1xCQ

用此token在界面上登錄即可

 

 

 

 

 

 

----------------------------------------------------------------------------------------------------------------------------------------

在參考大神的教程安裝kubernetes集群的同時安裝了dashboard；部署時使用到的dashboard-rbac.yaml文件如下：

apiVersion: v1
kind: ServiceAccount
metadata:
name: dashboard
namespace: kube-system
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1alpha1
metadata:
name: dashboard
subjects:
- kind: ServiceAccount
name: dashboard
namespace: kube-system
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.io
可以看到，我們用的是cluster-admin這個角色，這個角色什么權限都有，如果我們的dashboard是暴露外網可訪問的，就很危險了；所以決定自己定義集群的角色，使dashboard默認角色擁有較少的權限。

我們這里使用RBAC來做認證。

注意: 因為我們前面安裝dashboard的時候沒有采用https的請求，而k8s的認證需要使用HTTPS的訪問，這一點從官網和dashboard的官方文檔都能看出來：

Login view has been introduced in release 1.7. In order to make it appear in Dashboard you need to enable and access Dashboard over HTTPS.Dashboard 升級

刪除之前安裝的dashboard 的deployment,svc等
$ sudo kubectl delete deploy kubernetes-dashboard -n kube-system
$ sudo kubectl delete svc kubernetes-dashboard -n kube-system
## 如果dashboard需要一個超級管理員的用戶，那么ClusterRoleBinding，sa不用刪
$ sudo kubectl delete ClusterRoleBinding dashboard -n kube-system
$ sudo kubectl delete sa dashboard -n kube-system


部署Dashboard
還是采用dashboard官方提供的文件:kubernetes-dashboard.ayml

下載文件
#wget https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard.yaml

對部署文件做一些修改：
將Service的type改為NodePort
鏡像修改；因為gcr.io的鏡像需要科學上網才能下載，所以我們可以用別的鏡像倉庫的替代
k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.3

修改為：

## 這是我在阿里雲鏡像庫中找到的鏡像，你也可以在其他鏡像庫中找到替代
## 如果是在dockerhub的鏡像，也建議加上阿里雲鏡像加速，這樣速度回快很多
registry.cn-hangzhou.aliyuncs.com/kube_containers/kubernetes-dashboard-amd64:v1.8.3
1
2
3
阿里雲鏡像加速配置

修改證書
這一步是因為我的機器在使用--auto-generate-certificates自動生成證書后，訪問dashboard報錯：NET::ERR_CERT_INVALID,查看dashboard的日志提示證書未找到，為解決這個問題，將生成好的dashboard.crt和dashboard.key掛載到容器的/certs下，然后重新發布deployment即可

CA證書的生成可以參考這個和這個

掛載：
volumes:
- name: kubernetes-dashboard-certs
hostPath:
path: /home/share/certs
type: Directory

這里我采取的是hostPath方式掛載,這個需要保證dashboard調度到的node上都要有這個文件；其他掛載的方式可以參考官網

修改后的文件 dashboard-deploy.yaml

# ------------------- Dashboard Service Account ------------------- #

apiVersion: v1
kind: ServiceAccount
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kube-system

---
# ------------------- Dashboard Role & Role Binding ------------------- #

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: kubernetes-dashboard-minimal
namespace: kube-system
rules:
# Allow Dashboard to create 'kubernetes-dashboard-key-holder' secret.
- apiGroups: [""]
resources: ["secrets"]
verbs: ["create"]
# Allow Dashboard to create 'kubernetes-dashboard-settings' config map.
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["create"]
# Allow Dashboard to get, update and delete Dashboard exclusive secrets.
- apiGroups: [""]
resources: ["secrets"]
resourceNames: ["kubernetes-dashboard-key-holder"]
verbs: ["get", "update", "delete"]
# Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map.
- apiGroups: [""]
resources: ["configmaps"]
resourceNames: ["kubernetes-dashboard-settings"]
verbs: ["get", "update"]
# Allow Dashboard to get metrics from heapster.
- apiGroups: [""]
resources: ["services"]
resourceNames: ["heapster"]
verbs: ["proxy"]
- apiGroups: [""]
resources: ["services/proxy"]
resourceNames: ["heapster", "http:heapster:", "https:heapster:"]
verbs: ["get"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: kubernetes-dashboard-minimal
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: kubernetes-dashboard-minimal
subjects:
- kind: ServiceAccount
name: kubernetes-dashboard
namespace: kube-system

---
# ------------------- Dashboard Deployment ------------------- #

kind: Deployment
apiVersion: apps/v1beta2
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kube-system
spec:
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
k8s-app: kubernetes-dashboard
template:
metadata:
labels:
k8s-app: kubernetes-dashboard
spec:
containers:
- name: kubernetes-dashboard
image: registry.cn-hangzhou.aliyuncs.com/kube_containers/kubernetes-dashboard-amd64:v1.8.3
ports:
- containerPort: 8443
protocol: TCP
args:
- --auto-generate-certificates
- --token-ttl=5400 # 設置token過期時間
# Uncomment the following line to manually specify Kubernetes API server Host
# If not specified, Dashboard will attempt to auto discover the API server and connect
# to it. Uncomment only if the default does not work.
# - --apiserver-host=http://my-address:port
volumeMounts:
- name: kubernetes-dashboard-certs
mountPath: /certs
# Create on-disk volume to store exec logs
- mountPath: /tmp
name: tmp-volume
livenessProbe:
httpGet:
scheme: HTTPS
path: /
port: 8443
initialDelaySeconds: 30
timeoutSeconds: 30
volumes:
- name: kubernetes-dashboard-certs
hostPath:
path: /home/share/certs
type: Directory
- name: tmp-volume
emptyDir: {}
serviceAccountName: kubernetes-dashboard
# Comment the following tolerations if Dashboard must not be deployed on master
tolerations:
- key: node-role.kubernetes.io/master
effect: NoSchedule

---
# ------------------- Dashboard Service ------------------- #

kind: Service
apiVersion: v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kube-system
spec:
ports:
- port: 443
targetPort: 8443
nodePort: 30505
selector:
k8s-app: kubernetes-dashboard
type: NodePort

原文：https://blog.csdn.net/gunner2014/article/details/80966671

-----------------------------------------------------

部署 kubectl 命令行工具

kubectl 是 kubernetes 集群的命令行管理工具，本文檔介紹安裝和配置它的步驟。

kubectl 默認從 ~/.kube/config 文件讀取 kube-apiserver 地址、證書、用戶名等信息，如果沒有配置，執行 kubectl 命令時可能會出錯：

$ kubectl get pods
The connection to the server localhost:8080 was refused - did you specify the right host or port?

本文檔只需要部署一次，生成的 kubeconfig 文件與機器無關。

下載和分發 kubectl 二進制文件

下載和解壓：

wget https://dl.k8s.io/v1.10.4/kubernetes-client-linux-amd64.tar.gz
tar -xzvf kubernetes-client-linux-amd64.tar.gz

分發到所有使用 kubectl 的節點：

source /opt/k8s/bin/environment.sh
for node_ip in ${NODE_IPS[@]} do echo ">>> ${node_ip}" scp kubernetes/client/bin/kubectl k8s@${node_ip}:/opt/k8s/bin/ ssh k8s@${node_ip} "chmod +x /opt/k8s/bin/*" done

創建 admin 證書和私鑰

kubectl 與 apiserver https 安全端口通信，apiserver 對提供的證書進行認證和授權。

kubectl 作為集群的管理工具，需要被授予最高權限。這里創建具有最高權限的 admin 證書。

創建證書簽名請求：

cat > admin-csr.json <<EOF {  "CN": "admin",  "hosts": [],  "key": {  "algo": "rsa",  "size": 2048  },  "names": [  {  "C": "CN",  "ST": "BeiJing",  "L": "BeiJing",  "O": "system:masters",  "OU": "4Paradigm"  }  ] } EOF

• O 為 system:masters，kube-apiserver 收到該證書后將請求的 Group 設置為 system:masters；
• 預定義的 ClusterRoleBinding cluster-admin 將 Group system:masters 與 Role cluster-admin 綁定，該 Role 授予所有 API的權限；
• 該證書只會被 kubectl 當做 client 證書使用，所以 hosts 字段為空；

生成證書和私鑰：

cfssl gencert -ca=/etc/kubernetes/cert/ca.pem \
  -ca-key=/etc/kubernetes/cert/ca-key.pem \
  -config=/etc/kubernetes/cert/ca-config.json \
  -profile=kubernetes admin-csr.json | cfssljson -bare admin
ls admin*

創建 kubeconfig 文件

kubeconfig 為 kubectl 的配置文件，包含訪問 apiserver 的所有信息，如 apiserver 地址、CA 證書和自身使用的證書；

source /opt/k8s/bin/environment.sh
# 設置集群參數 kubectl config set-cluster kubernetes \ --certificate-authority=/etc/kubernetes/cert/ca.pem \ --embed-certs=true \ --server=${KUBE_APISERVER} \ --kubeconfig=kubectl.kubeconfig # 設置客戶端認證參數 kubectl config set-credentials admin \ --client-certificate=admin.pem \ --client-key=admin-key.pem \ --embed-certs=true \ --kubeconfig=kubectl.kubeconfig # 設置上下文參數 kubectl config set-context kubernetes \ --cluster=kubernetes \ --user=admin \ --kubeconfig=kubectl.kubeconfig # 設置默認上下文 kubectl config use-context kubernetes --kubeconfig=kubectl.kubeconfig

• --certificate-authority：驗證 kube-apiserver 證書的根證書；
• --client-certificate、--client-key：剛生成的 admin 證書和私鑰，連接 kube-apiserver 時使用；
• --embed-certs=true：將 ca.pem 和 admin.pem 證書內容嵌入到生成的 kubectl.kubeconfig 文件中(不加時，寫入的是證書文件路徑)；

分發 kubeconfig 文件

分發到所有使用 kubectl 命令的節點：

source /opt/k8s/bin/environment.sh
for node_ip in ${NODE_IPS[@]} do echo ">>> ${node_ip}" ssh k8s@${node_ip} "mkdir -p ~/.kube" scp kubectl.kubeconfig k8s@${node_ip}:~/.kube/config ssh root@${node_ip} "mkdir -p ~/.kube" scp kubectl.kubeconfig root@${node_ip}:~/.kube/config done

• 保存到用戶的 ~/.kube/config 文件；