Centos7上安裝Kubernetes集群部署docker

一、安裝前准備
1.操作系統詳情
需要三台主機，都最小化安裝 centos7.3,並update到最新

[root@master ~]# cat /etc/redhat-release 

CentOS Linux release 7.3.1611 (Core)

角色       主機名         IP
Master  master  192.168.1.14
node1  slave-1  192.168.1.15
node2  slave-2  192.168.1.16

2.在每台主機上關閉firewalld改用iptables
輸入以下命令，關閉firewalld

[root@master ~]# systemctl stop firewalld.service #停止firewall 

[root@master ~]# systemctl disable firewalld.service #禁止firewall開機啟動

 3.安裝ntp服務

[root@master ~]# yuminstall -y ntp wget net-tools 

[root@master ~]# systemctl start ntpd systemctl enable ntpd

 二、安裝配置

注：kubernetes，etcd等已經進去centos epel源，可以直接yum安裝（需要安裝epel-release）

1.安裝Kubernetes Master
使用以下命令安裝kubernetes 和 etcd

# yum install -y kubernetes etcd
編輯/etc/etcd/etcd.conf 使etcd監聽所有的ip地址，確保下列行沒有注釋，並修改為下面的值

[root@master ~]# cat /etc/etcd/etcd.conf 
# [member] ETCD_NAME=default ETCD_DATA_DIR="/var/lib/etcd/default.etcd" 
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379" 
ETCD_INITIAL_CLUSTER="default=http://192.168.1.14:2380" 
#[cluster] 
ETCD_ADVERTISE_CLIENT_URLS="http://192.168.1.14:2379"

編輯Kubernetes API server的配置文件 /etc/kubernetes/apiserver，確保下列行沒有被注釋，並為下列的值

[root@master ~]# cat /etc/kubernetes/apiserver
###
# kubernetes system config
#
# The following values are used to configure the kube-apiserver # # The address on the local server to listen to. KUBE_API_ADDRESS="--address=0.0.0.0" # The port on the local server to listen on. KUBE_API_PORT="--port=8080" # Port minions listen on KUBELET_PORT="--kubelet_port=10250" # Comma separated list of nodes in the etcd cluster KUBE_ETCD_SERVERS="--etcd_servers=http://192.168.1.14:2379" # Address range to use for services KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16" # default admission control policies KUBE_ADMISSION_CONTROL="--admission_control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota" # Add your own! KUBE_API_ARGS=""

啟動etcd, kube-apiserver, kube-controller-manager and kube-scheduler服務，並設置開機自啟

[root@master ~]# cat /script/kubenetes_service.sh

for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler; do 
    systemctl restart $SERVICES
    systemctl enable $SERVICES
    systemctl status $SERVICES 
done

[root@master ~]# sh /script/kubenetes_service.sh

在etcd中定義flannel network的配置，這些配置會被flannel service下發到nodes:

[root@master ~]# etcdctl mk /centos.com/network/config '{"Network":"172.17.0.0/16"}'

添加iptables規則，允許相應的端口

[root@master ~]# iptables -I INPUT -p tcp --dport 2379 -j ACCEPT

[root@master ~]# iptables -I INPUT -p tcp --dport 10250 -j ACCEPT

[root@master ~]# iptables -I INPUT -p tcp --dport 8080 -j ACCEPT 

[root@master ~]# iptables-save 

或者寫入iptables配置文件 /etc/sysconfig/iptables

查看節點信息（我們還沒有配置節點信息，所以這里應該為空）

[root@master ~]# kubectl get nodes

NAME LABELS STATUS

2. 安裝Kubernetes Nodes

注：下面這些步驟應該在node1和node2上執行（也可以添加更多的node）

使用yum安裝kubernetes 和 flannel

[root@slave1 ~]# yum install -y flannel kubernetes

為flannel service配置etcd服務器，編輯/etc/sysconfig/flanneld文件中的下列行以連接到master

[root@slave1 ~]# cat /etc/sysconfig/flanneld

FLANNEL_ETCD="http://192.168.1.14:2379" #改為etcd服務器的ip
FLANNEL_ETCD_PREFIX="/centos.com/network"

編輯/etc/kubernetes/config 中kubernetes的默認配置，確保KUBE_MASTER的值是連接到Kubernetes master API server：

[root@slave1 ~]# cat /etc/kubernetes/config

KUBE_MASTER="--master=http://192.168.1.14:8080"

編輯/etc/kubernetes/kubelet 如下行：

node1:

[root@slave1 ~]# cat /etc/kubernetes/kubelet

KUBELET_ADDRESS="--address=0.0.0.0"
KUBELET_PORT="--port=10250"
KUBELET_HOSTNAME="--hostname_override=192.168.1.15"
KUBELET_API_SERVER="--api_servers=http://192.168.1.14:8080"
KUBELET_ARGS=""

node2:

[root@slave2 ~]# cat /etc/kubernetes/kubelet

KUBELET_ADDRESS="--address=0.0.0.0"
KUBELET_PORT="--port=10250"
KUBELET_HOSTNAME="--hostname_override=192.168.1.16"
KUBELET_API_SERVER="--api_servers=http://192.168.1.14:8080"
KUBELET_ARGS=""

啟動kube-proxy, kubelet, docker 和 flanneld services服務，並設置開機自啟

[root@slave1 ~]# cat /script/kubernetes_node_service.sh

for SERVICES in kube-proxy kubelet docker flanneld; do 
systemctl restart $SERVICES
systemctl enable $SERVICES
systemctl status $SERVICES 
done

在每個node節點，你應當注意到你有兩塊新的網卡docker0 和 flannel0。你應該得到不同的ip地址范圍在flannel0上，就像下面這樣：

node1:

[root@slave1 ~]# ip a | grep flannel | grep inet
inet 172.17.11.0/16 scope global flannel0

node2:

[root@slave2 ~]# ip a | grep flannel | grep inet
inet 172.17.60.0/16 scope global flannel0

添加iptables規則：

[root@slave1 ~]# iptables -I INPUT -p tcp --dport 2379 -j ACCEPT
[root@slave1 ~]# iptables -I INPUT -p tcp --dport 10250 -j ACCEPT
[root@slave1 ~]# iptables -I INPUT -p tcp --dport 8080 -j ACCEPT

現在登陸kubernetes master節點驗證minions的節點狀態：

[root@master ~]# kubectl get nodes
NAME           STATUS    AGE
192.168.1.15   Ready     2h
192.168.1.16   Ready     2h

至此，kubernetes集群已經配置並運行了，我們可以繼續下面的步驟。

三、創建 Pods (Containers)
為了創建一個pod，我們需要在kubernetes master上面定義一個yaml 或者 json配置文件。然后使用kubectl命令創建pod

[root@slave1 ~]# mkdir -p /k8s/pods
[root@slave1 ~]# cd /k8s/pods/
[root@slave1 ~]# cat nginx.yaml

在nginx.yaml內容如下：

apiVersion: v1
kind: Pod
metadata:
name: nginx
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80

創建pod:

[root@slave1 ~]# kubectl create -f nginx.yaml

此時有如下報錯：

Error from server: error when creating "nginx.yaml": Pod "nginx" is forbidden: no API token found for service account default/default, retry after the token is automatically created and added to the service account
解決辦法是編輯/etc/kubernetes/apiserver 去除 KUBE_ADMISSION_CONTROL中的SecurityContextDeny,ServiceAccount，並重啟kube-apiserver.service服務：

#cat /etc/kubernetes/apiserver
KUBE_ADMISSION_CONTROL="--admission_control=NamespaceLifecycle,NamespaceExists,LimitRanger,ResourceQuota"

#systemctl restart kube-apiserver.service
之后重新創建pod:

# kubectl create -f nginx.yaml
pods/nginx
查看pod:

# kubectl get pod nginx
NAME READY STATUS RESTARTS AGE
nginx 0/1 Image: nginx is not ready on the node 0 34s
這里STATUS一直是這個，創建不成功，下面排錯。通過查看pod的描述發現如下錯誤：

# kubectl describe pod nginx
Wed, 28 Oct 2015 10:25:30 +0800 Wed, 28 Oct 2015 10:25:30 +0800 1 {kubelet 192.168.1.16} implicitly required container POD pulled Successfully pulled Pod container image "gcr.io/google_containers/pause:0.8.0"
Wed, 28 Oct 2015 10:25:30 +0800 Wed, 28 Oct 2015 10:25:30 +0800 1 {kubelet 192.168.1.16} implicitly required container POD failed Failed to create docker container with error: no such image
Wed, 28 Oct 2015 10:25:30 +0800 Wed, 28 Oct 2015 10:25:30 +0800 1 {kubelet 192.168.1.16} failedSync Error syncing pod, skipping: no such image
Wed, 28 Oct 2015 10:27:30 +0800 Wed, 28 Oct 2015 10:29:30 +0800 2 {kubelet 192.168.1.16} implicitly required container POD failed Failed to pull image "gcr.io/google_containers/pause:0.8.0": image pull failed for gcr.io/google_containers/pause:0.8.0, this may be because there are no credentials on this request. details: (API error (500): invalid registry endpoint "http://gcr.io/v0/". HTTPS attempt: unable to ping registry endpoint https://gcr.io/v0/
v2 ping attempt failed with error: Get https://gcr.io/v2/: dial tcp 173.194.72.82:443: i/o timeout

這里可能會遇到pod狀態一直處於Penning的問題，此時可以通過kubectl describe pods/pod-name來查看pod信息，如果沒有出錯信息，那么Minion一直處於下載鏡像中，下載好之后pod即會成功啟動。

從網上找到 pause:0.8.0 的鏡像，然后再每個node上導入鏡像：

請在境外docker服務器執行 docker pull 命令下載鏡像

gcr.io/google_containers/pause:latest
gcr.io/google_containers/pause:1.0
gcr.io/google_containers/pause:0.8.0

再用導出鏡像

docker save -o pause.tar gcr.io/google_containers/pause
gzip pause.tar

最后把這個包放到 kubernetes 環境所有的 docker 服務器上

 docker load -i pause.tar.gz

在執行以下命令即可成功創建pod

[root@master ~]#kubectl create -f nginx.yaml
pods/nginx

查看pod

[root@master ~]# kubectl get pod nginx
NAME READY STATUS RESTARTS AGE
nginx 1/1 Running 0 2min

前往nodes節點上查看docker images

[root@slave1 ~]# docker images
REPOSITORY                                            TAG                 IMAGE ID            CREATED             SIZE
registry.access.redhat.com/rhel7/pod-infrastructure   latest              34d3450d733b        10 weeks ago        205 MB
gcr.io/google_containers/pause                        0.8.0               bf595365a558        2 years ago         241.7 kB