kubeadm安裝部署kubernetes 1.11.3(單主節點)


由於此處docker代理無法使用,因此,請各位設置有效代理進行部署,勿使用文檔中的docker代理。整體部署步驟不用改動。謝謝各位支持。

1、部署背景

操作系統版本:CentOS Linux release 7.5.1804 (Core)
docker-ce版本:18.06.1-ce
kubernetes版本:1.11.3
kubeadm版本:v1.11.3

2、節點划分

master節點:
    主機名:k8s-master-52
    ip地址:192.168.40.52
node1節點:
    主機名:k8s-node-53
    ip地址:192.168.40.53
node2節點:
    主機名:k8s-node-54
    ip地址:192.168.40.54

3、部署前提

1、關閉selinux、firewalld。
2、開啟內核轉發。
3、關閉swap交換分區
4、master免密鑰登錄所有node節點
5、所有節點配置ntp時間同步服務,保證節點時間一致。
6、加載ipvs相關模塊

4、集群所有節點初始化

1、加載ipvs相關模塊以及安裝依賴關系
    安裝依賴。
    yum install ipset ipvsadm conntrack-tools.x86_64 -y
    
    加載模塊。
    modprobe ip_vs_rr
    modprobe ip_vs_wrr
    modprobe ip_vs_sh
    modprobe ip_vs

    查看模塊加載信息。
    lsmod| grep ip_vs
    
2、開啟內核轉發,並使之生效
cat <<EOF | tee /etc/sysctl.d/k8s.conf
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

sysctl -p /etc/sysctl.d/k8s.conf
3、關閉selinux,關閉swap分區,關閉firewalld。

#關閉防火牆,並且禁止自動啟動。
systemctl stop firewalld
systemctl disable firewalld

#關閉selinux
sed -i 's#enforcing#disabled#ig' /etc/sysconfig/selinux

#關閉swap分區
swapoff -a && sysctl -w vm.swappiness=0

#修改文件最大打開數量
echo -e '*\tsoft\tnproc\t4096\nroot\tsoft\tnproc\tunlimited' > /etc/security/limits.d/20-nproc.conf
echo -e '* soft nofile 65536\n* hard nofile 65536' > /etc/security/limits.conf
4、配置時間同步以及hosts解析,以及實現master節點通過免密鑰登錄node節點

#安裝ntp命令,同時配置任務計划
yum install ntp -y
任務計划命令如下:
*/5 * * * *  /usr/sbin/ntpdate  0.centos.pool.ntp.org > /dev/null 2> /dev/null

#配置服務器通過hostname可以解析,保證master和node節點上一致,內容如下:
192.168.40.52 k8s-master-52 master
192.168.40.53 k8s-node-53
192.168.40.54 k8s-node-54

#配置master節點通過免秘鑰登錄node節點
ssh-keygen -t rsa
一路回車,生成公鑰和私鑰。

ssh-copy-id -i ~/.ssh/id_rsa.pub k8s-node-53
ssh-copy-id -i ~/.ssh/id_rsa.pub k8s-node-54

初始化完成之后,最好能重啟服務器。

5、在master節點進行操作

1、配置kubernetes yum源。
vim
/etc/yum.repos.d/kubernetes.repo,內容如下: [kubernetes] name=kubernetes baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/ enabled=1 gpgcheck=0

2、配置docker-ce yum源。
yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

3、安裝docker-ce、kubernetes。
yum install docker-ce kubelet kubeadm kubectl
軟件及依賴的版本如下:
  

4、配置docker容器代理、啟動docker-ce,同時配置docker、kubelet開機自動啟動。
配置代理如下:
  編輯文件:/usr/lib/systemd/system/docker.service
  Environment="HTTPS_PROXY=http://www.ik8s.io:10080"
  Environment="NO_PROXY=127.0.0.0/8,192.168.0.0/16"

重新加載相關服務配置。
systemctl daemon-reload

#啟動docker
systemctl start docker

#配置docker、kubelet開機自動啟動
systemctl enable docker
systemctl enable kubelet
在此處,kubelet不用啟動,在kubeadm初始化服務器的時候,初始化完成,會自動啟動kubelet服務。
5、初始化master節點
[root@k8s-master-52 ]# kubeadm init --kubernetes-version=v1.11.3 --pod-network-cidr=10.244.0.0/16 --service-cidr=10.96.0.0/12 
命令解析:
  --kubernetes-version=v1.11.3:指定kubernetes版本
  --pod-network-cidr=10.244.0.0/16:指定pod網絡地址池
  --service-cidr=10.96.0.0/12:指定service網絡地址池

命令執行輸出如下:
[init] using Kubernetes version: v1.11.3
[preflight] running pre-flight checks
I0913 20:48:31.926894 2304 kernel_validator.go:81] Validating kernel version
I0913 20:48:31.926940 2304 kernel_validator.go:96] Validating kernel config
  [WARNING SystemVerification]: docker version is greater than the most recently validated version. Docker version: 18.06.1-ce. Max validated version: 17.03
[preflight/images] Pulling images required for setting up a Kubernetes cluster
[preflight/images] This might take a minute or two, depending on the speed of your internet connection
[preflight/images] You can also perform this action in beforehand using 'kubeadm config images pull'
....
中間信息,此處不再給出。根據網絡情況,下載鏡像,因此初始化的時間不固定。
[addons] Applied essential addon: CoreDNS #1.11版本開始支持coredns,1.10.X版本使用的為kube dns。
[addons] Applied essential addon: kube-proxy
Your Kubernetes master has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
You can now join any number of machines by running the following on each node
as root:
kubeadm join 192.168.40.52:6443 --token k5mudw.bri3lujvlsxffbqo --discovery-token-ca-cert-hash sha256:f6cf089d5aff3230996f75ca71e74273095c901c1aa45f1325ade0359aeb336e
 
注意:要記住最后一行,kubeadm join這一行的信息,這行信息為node加入集群要執行的命令,請務必復制記錄。
 
其中master節點在初始化的時候,會pull docker hub中的鏡像。如下:
  

 

查看端口占用情況,如下:

  

 

其中6443為apiserver的https端口。

從1.11版本開始,默認為ipvs。1.10.X及其之前版本為iptables。
從1.11版本開始支持coredns,1.10.X版本使用的為kube dns。

 

創建配置文件,使kubectl客戶端能正常進行命令進行kubernetes集群的相關操作。

[root@k8s-master-52 ]# mkdir -p $HOME/.kube
[root@k8s-master-52 ]# cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
 
查看集群狀態信息。
[root@k8s-master-52 manifests]# kubectl get cs
NAME STATUS MESSAGE ERROR
scheduler Healthy ok
controller-manager Healthy ok
etcd-0 Healthy {"health": "true"}

查看集群節點信息。
[root@k8s-master-52 manifests]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master-52 NotReady master 12m v1.11.3
未安裝網絡組件造成節點status狀態為notready。
 
6、安裝flannel網絡插件。
#直接引用官方的部署模板文件進行安裝。
[root@k8s-master-52 ]# kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
    clusterrole.rbac.authorization.k8s.io/flannel created
    clusterrolebinding.rbac.authorization.k8s.io/flannel created
    serviceaccount/flannel created
    configmap/kube-flannel-cfg created
    daemonset.extensions/kube-flannel-ds-amd64 created
    daemonset.extensions/kube-flannel-ds-arm64 created
    daemonset.extensions/kube-flannel-ds-arm created
    daemonset.extensions/kube-flannel-ds-ppc64le created
    daemonset.extensions/kube-flannel-ds-s390x created
 
[root@k8s-master-52 ]# kubectl get pods -n kube-system
  
只有等到flannel鏡像下載完成,pods才能正常啟動。 

 6、在node節點操作

1、配置kubernetes yum源。
vim /etc/yum.repos.d/kubernetes.repo,內容如下:

[kubernetes]
name=kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0

2、配置docker-ce yum源。
yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

3、安裝docker-ce、kubernetes。
yum install docker-ce kubelet kubeadm kubectl
軟件及依賴的版本如下:
  

4、配置docker容器代理、啟動docker-ce,同時配置docker、kubelet開機自動啟動。
配置代理如下:
  編輯文件:/usr/lib/systemd/system/docker.service
  Environment="HTTPS_PROXY=http://www.ik8s.io:10080"
  Environment="NO_PROXY=127.0.0.0/8,192.168.0.0/16"

重新加載相關服務配置。
systemctl daemon-reload

#啟動docker
systemctl start docker

#配置docker、kubelet開機自動啟動
systemctl enable docker
systemctl enable kubelet
在此處,kubelet不用啟動,在kubeadm初始化服務器的時候,初始化完成,會自動啟動kubelet服務。

5、
安裝flannel網絡插件。
#直接引用官方的部署模板文件進行安裝。
 
在k8s-node-53節點上進行安裝。
[root@k8s-node-53 ]# kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.extensions/kube-flannel-ds-amd64 created
daemonset.extensions/kube-flannel-ds-arm64 created
daemonset.extensions/kube-flannel-ds-arm created
daemonset.extensions/kube-flannel-ds-ppc64le created
daemonset.extensions/kube-flannel-ds-s390x created
   
在k8s-node-54節點上進行安裝。
[root@k8s-node-54 ]# kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.extensions/kube-flannel-ds-amd64 created
daemonset.extensions/kube-flannel-ds-arm64 created
daemonset.extensions/kube-flannel-ds-arm created
daemonset.extensions/kube-flannel-ds-ppc64le created
daemonset.extensions/kube-flannel-ds-s390x created
 
將k8s-node-53節點加入k8s集群。

[root@k8s-node-53 ~]# kubeadm join 192.168.40.52:6443 --token k5mudw.bri3lujvlsxffbqo --discovery-token-ca-cert-hash sha256:f6cf089d5aff3230996f75ca71e74273095c901c1aa45f1325ade0359aeb336e
[preflight] running pre-flight checks
[WARNING RequiredIPVSKernelModulesAvailable]: the IPVS proxier will not be used, because the following required kernel modules are not loaded: [ip_vs_sh ip_vs ip_vs_rr ip_vs_wrr] or no builtin kernel ipvs support: map[ip_vs_wrr:{} ip_vs_sh:{} nf_conntrack_ipv4:{} ip_vs:{} ip_vs_rr:{}]
you can solve this problem with following methods:
1. Run 'modprobe -- ' to load missing kernel modules;
2. Provide the missing builtin kernel ipvs support

I0913 21:13:20.983878 1794 kernel_validator.go:81] Validating kernel version
I0913 21:13:20.983943 1794 kernel_validator.go:96] Validating kernel config
[WARNING SystemVerification]: docker version is greater than the most recently validated version. Docker version: 18.06.1-ce. Max validated version: 17.03
[discovery] Trying to connect to API Server "192.168.40.52:6443"
[discovery] Created cluster-info discovery client, requesting info from "https://192.168.40.52:6443"
[discovery] Requesting info from "https://192.168.40.52:6443" again to validate TLS against the pinned public key
[discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "192.168.40.52:6443"
[discovery] Successfully established connection with API Server "192.168.40.52:6443"
[kubelet] Downloading configuration for the kubelet from the "kubelet-config-1.11" ConfigMap in the kube-system namespace
[kubelet] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[preflight] Activating the kubelet service
[tlsbootstrap] Waiting for the kubelet to perform the TLS Bootstrap...
[patchnode] Uploading the CRI Socket information "/var/run/dockershim.sock" to the Node API object "k8s-node-53" as an annotation

This node has joined the cluster:
* Certificate signing request was sent to master and a response
was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the master to see this node join the cluster.

 

將k8s-node-54節點加入k8s集群。

[root@k8s-node-54 ~]# kubeadm join 192.168.40.52:6443 --token k5mudw.bri3lujvlsxffbqo --discovery-token-ca-cert-hash sha256:f6cf089d5aff3230996f75ca71e74273095c901c1aa45f1325ade0359aeb336e
[preflight] running pre-flight checks
[WARNING RequiredIPVSKernelModulesAvailable]: the IPVS proxier will not be used, because the following required kernel modules are not loaded: [ip_vs_sh ip_vs ip_vs_rr ip_vs_wrr] or no builtin kernel ipvs support: map[ip_vs:{} ip_vs_rr:{} ip_vs_wrr:{} ip_vs_sh:{} nf_conntrack_ipv4:{}]
you can solve this problem with following methods:
1. Run 'modprobe -- ' to load missing kernel modules;
2. Provide the missing builtin kernel ipvs support

I0913 21:21:03.915755 11043 kernel_validator.go:81] Validating kernel version
I0913 21:21:03.915806 11043 kernel_validator.go:96] Validating kernel config
[WARNING SystemVerification]: docker version is greater than the most recently validated version. Docker version: 18.06.1-ce. Max validated version: 17.03
[discovery] Trying to connect to API Server "192.168.40.52:6443"
[discovery] Created cluster-info discovery client, requesting info from "https://192.168.40.52:6443"
[discovery] Requesting info from "https://192.168.40.52:6443" again to validate TLS against the pinned public key
[discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "192.168.40.52:6443"
[discovery] Successfully established connection with API Server "192.168.40.52:6443"
[kubelet] Downloading configuration for the kubelet from the "kubelet-config-1.11" ConfigMap in the kube-system namespace
[kubelet] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[preflight] Activating the kubelet service
[tlsbootstrap] Waiting for the kubelet to perform the TLS Bootstrap...
[patchnode] Uploading the CRI Socket information "/var/run/dockershim.sock" to the Node API object "k8s-node-54" as an annotation

This node has joined the cluster:
* Certificate signing request was sent to master and a response
was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the master to see this node join the cluster.

7、創建角色,使用k8s dashboard查看集群狀態。

vim dashboard-admin.yaml
內容如下:
apiVersion: rbac.authorization.k8s.io
/v1beta1 kind: ClusterRoleBinding metadata: name: kubernetes-dashboard labels: k8s-app: kubernetes-dashboard roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cluster-admin subjects: - kind: ServiceAccount name: kubernetes-dashboard namespace: kube-system

執行以下命令創建角色:

kubectl create -f dashboard-admin.yaml

8、安裝k8s dashboard

 

vim kubernetes-dashboard.yaml
內容如下:
apiVersion: v1 kind: Secret metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard-certs namespace: kube-system type: Opaque --- # ------------------- Dashboard Service Account ------------------- # apiVersion: v1 kind: ServiceAccount metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kube-system --- # ------------------- Dashboard Role & Role Binding ------------------- # kind: Role apiVersion: rbac.authorization.k8s.io/v1 metadata: name: kubernetes-dashboard-minimal namespace: kube-system rules: # Allow Dashboard to create 'kubernetes-dashboard-key-holder' secret. - apiGroups: [""] resources: ["secrets"] verbs: ["create"] # Allow Dashboard to create 'kubernetes-dashboard-settings' config map. - apiGroups: [""] resources: ["configmaps"] verbs: ["create"] # Allow Dashboard to get, update and delete Dashboard exclusive secrets. - apiGroups: [""] resources: ["secrets"] resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs"] verbs: ["get", "update", "delete"] # Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map. - apiGroups: [""] resources: ["configmaps"] resourceNames: ["kubernetes-dashboard-settings"] verbs: ["get", "update"] # Allow Dashboard to get metrics from heapster. - apiGroups: [""] resources: ["services"] resourceNames: ["heapster"] verbs: ["proxy"] - apiGroups: [""] resources: ["services/proxy"] resourceNames: ["heapster", "http:heapster:", "https:heapster:"] verbs: ["get"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: kubernetes-dashboard-minimal namespace: kube-system roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: kubernetes-dashboard-minimal subjects: - kind: ServiceAccount name: kubernetes-dashboard namespace: kube-system --- # ------------------- Dashboard Deployment ------------------- # kind: Deployment apiVersion: apps/v1beta2 metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kube-system spec: replicas: 1 revisionHistoryLimit: 10 selector: matchLabels: k8s-app: kubernetes-dashboard template: metadata: labels: k8s-app: kubernetes-dashboard spec: containers: - name: kubernetes-dashboard image: k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.0 ports: - containerPort: 8443 protocol: TCP args: - --auto-generate-certificates # Uncomment the following line to manually specify Kubernetes API server Host # If not specified, Dashboard will attempt to auto discover the API server and connect # to it. Uncomment only if the default does not work. # - --apiserver-host=http://my-address:port volumeMounts: - name: kubernetes-dashboard-certs mountPath: /certs # Create on-disk volume to store exec logs - mountPath: /tmp name: tmp-volume livenessProbe: httpGet: scheme: HTTPS path: / port: 8443 initialDelaySeconds: 30 timeoutSeconds: 30 volumes: - name: kubernetes-dashboard-certs secret: secretName: kubernetes-dashboard-certs - name: tmp-volume emptyDir: {} serviceAccountName: kubernetes-dashboard # Comment the following tolerations if Dashboard must not be deployed on master tolerations: - key: node-role.kubernetes.io/master effect: NoSchedule --- # ------------------- Dashboard Service ------------------- # kind: Service apiVersion: v1 metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kube-system spec: type: NodePort ports: - port: 443 targetPort: 8443 nodePort: 30001 selector: k8s-app: kubernetes-dashboard

 

執行以下命令安裝dashboard:

kubectl apply -f  kubernetes-dashboard.yaml 

訪問dashboard url如下:

https://192.168.40.54:30001
此處使用集群中任一節點ip,即可訪問dashboard頁面。

9、生成token認證文件

在主節點上進行執行。

[root@k8s-master-52 opt]# kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')
Name: admin-user-token-hddfq
Namespace: kube-system
Labels: <none>
Annotations: kubernetes.io/service-account.name=admin-user
kubernetes.io/service-account.uid=2d23955c-b75d-11e8-a770-5254007ec152

Type: kubernetes.io/service-account-token

Data
====
ca.crt: 1025 bytes
namespace: 11 bytes
token: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi11c2VyLXRva2VuLWhkZGZxIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluLXVzZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiIyZDIzOTU1Yy1iNzVkLTExZTgtYTc3MC01MjU0MDA3ZWMxNTIiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06YWRtaW4tdXNlciJ9.5GakSIdKw7H62P5Bk3c8879Jc68cAN9gcQRMYvaWLo-Cq6cwnpOoz6fwYm1AoFRfJ_ddMoctqB_rp72j_AqSO0ihp3_H_1dX31bo_ddp1xtj5Yg3IswhcxU2RCBmoIn0JmgCeWxoIt_KAYpNJBJqJKR5oIS2hr_Xfew5GNXRC6_OE9fm7ljRy4XqkBTaj6_1K0wUrmoC4WFHQGZzTUq6mmVsJlD_o3J35sMzi993WtP0APeBc6v66RokHW5EAECN9__ipA9cQlqmtLkgFydORMvUmd4bOWNFoNticx_M6poDlzTLRqmKY5I3mxJmhCCHr2gp7X0auo1enLW765t-7g

  使用最后生成的token認證內容登錄dashboard。

 


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM