1.1 服務器基本信息
本次安裝采用一個master、5個node、3個etcd，node節點兩塊硬盤，60G磁盤用於docker storage，xxx改為自己的域名或主機名
| 節點/主機名 |
功能 |
IP |
內存 |
磁盤 |
CPU |
| master1.xxx.net |
Master節點 |
192.168.10.110 |
16G |
40G |
8C |
| node1.xxx.net |
Node節點 |
192.168.10.112 |
8G |
40G/60G |
4C |
| node2.xxx.net |
Node節點 |
192.168.10.113 |
8G |
40G/60G |
4C |
| node3.xxx.net |
Node節點 |
192.168.10.116 |
8G |
40G/60G |
4C |
| node4.xxx.net |
Node節點 |
192.168.10.114 |
8G |
40G/60G |
4C |
| node5.xxx.net |
Node節點 |
192.168.10.117 |
8G |
40G/60G |
4C |
| etcd1.xxx.net |
etcd |
192.168.10.109 |
4G |
40G |
2C |
| etcd2.xxx.net |
etcd |
192.168.10.111 |
4G |
40G |
2C |
| etcd3.xxx.net |
etcd |
192.168.10.115 |
4G |
40G |
2C |
1.2 基本配置
所有節點hosts文件配置
[root@openshift-master1 ~]# cat /etc/hosts 192.168.10.110 master1.xxx.net 192.168.10.112 node1.xxx.net 192.168.10.113 node2.xxx.net 192.168.10.116 node3.xxx.net 192.168.10.114 node4.xxx.net 192.168.10.117 node5.xxx.net 192.168.10.109 etcd1.xxx.net 192.168.10.111 etcd2.xxx.net 192.168.10.115 etcd3.xxx.net
Master1節點SSH互信
[root@master1 ~]# ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Created directory '/root/.ssh'. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: SHA256:yFOKV/QRdQoxQ12uW8v0UWmpLcrxDQo8VNyVEuRWHlE root@master1.xxx.net The key's randomart image is: +---[RSA 2048]----+ | ..X*++==E| | . ..*o*o.+| | o.. .oo=.| | o =o ..+ .| | . * S+ ..+oo | | . . + ==+o.| | +..o..| | | | | +----[SHA256]-----+ for i in `cat /etc/hosts | grep -v openshift | grep xxx.net | awk '{print $2}'`;do ssh-copy-id -i .ssh/id_rsa.pub $i;done
所有節點安裝基本環境
yum install wget git net-tools bind-utils iptables-services bridge-utils bash-completion kexec-tools sos psacct vim ntpdate httpd-tools -y
所有節點更改時區並同步時間
ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime ntpdate cn.ntp.org.cn yum update reboot yum install docker-1.13.1 -y 備注：docker可安裝docker-ce
所有節點激活網絡
nmcli con show
nmcli con up ens160
nmcli con mod ens160 connection.autoconnect yes
systemctl restart NetworkManager
所有Node節點更改/etc/sysconfig/docker-storage-setup如下：
DEVS=/dev/sdb
VG=docker-vg
所有Node節點執行docker-storage-setup
[root@openshift-node1 ~]# docker-storage-setup WARNING: Device for PV 28oz2p-ZKrx-gSc2-k6Tg-E49Y-MK4A-YcQq7h not found or rejected by a filter. WARNING: Device for PV 28oz2p-ZKrx-gSc2-k6Tg-E49Y-MK4A-YcQq7h not found or rejected by a filter. INFO: Device node /dev/sdb1 exists. WARNING: Device for PV 28oz2p-ZKrx-gSc2-k6Tg-E49Y-MK4A-YcQq7h not found or rejected by a filter. Physical volume "/dev/sdb1" successfully created. WARNING: Device for PV 28oz2p-ZKrx-gSc2-k6Tg-E49Y-MK4A-YcQq7h not found or rejected by a filter. Volume group "docker-vg" successfully created WARNING: Device for PV 28oz2p-ZKrx-gSc2-k6Tg-E49Y-MK4A-YcQq7h not found or rejected by a filter. Using default stripesize 64.00 KiB. Rounding up size to full physical extent 84.00 MiB Thin pool volume with chunk size 512.00 KiB can address at most 126.50 TiB of data. Logical volume "docker-pool" created. Logical volume docker-vg/docker-pool changed.
所有節點更改docker倉庫地址
cat /etc/sysconfig/docker # /etc/sysconfig/docker # Modify these options if you want to change the way the docker daemon runs # OPTIONS='--selinux-enabled --log-driver=journald --signature-verification=false' OPTIONS='--selinux-enabled --log-driver=journald --registry-mirror=https://docker.mirrors.ustc.edu.cn'
Master節點更改epel源,並安裝ansible
yum -y install https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/e/epel-release-7-11.noarch.rpm sed -i -e "s/^enabled=1/enabled=0/" /etc/yum.repos.d/epel.repo yum -y --enablerepo=epel install ansible pyOpenSSL
1.3 etcd集群安裝
安裝etcd集群（按需安裝，不與openshift在同一集群）
所有節點關閉firewalld
[root@etcd1 ~]# systemctl stop firewalld [root@etcd1 ~]# systemctl disable firewalld Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service. Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
所有etcd節點開啟iptables
[root@etcd1 ~]# systemctl start iptables [root@etcd1 ~]# systemctl enable iptables Created symlink from /etc/systemd/system/basic.target.wants/iptables.service to /usr/lib/systemd/system/iptables.service
所有etcd節點安裝etcd，OpenShift高級安裝模式無需自行配置etcd
yum install etcd -y
1.4 OpenShift高級安裝
參考文檔：https://docs.openshift.org/latest/install_config/install/advanced_install.html
在master1節點上
[root@master1 ~]# cat /etc/ansible/hosts # Create an OSEv3 group that contains the masters, nodes, and etcd groups [OSEv3:children] masters nodes etcd #lb # Set variables common for all OSEv3 hosts [OSEv3:vars] # SSH user, this user should allow ssh based auth without requiring a password ansible_ssh_user=root ansible_become=yes debug_level=2 openshift_deployment_type=origin # If ansible_ssh_user is not root, ansible_become must be set to true #ansible_become=true openshift_repos_enable_testing=true openshift_enable_service_catalog=false template_service_broker_install=false # uncomment the following to enable htpasswd authentication; defaults to DenyAllPasswordIdentityProvider openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', 'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider', 'filename': '/etc/origin/master/htpasswd'}] openshift_disable_check=disk_availability,docker_storage,memory_availability,docker_image_availability # config for metrics openshift_release=3.6.1 openshift_clock_enabled=true #openshift_master_cluster_method=native #openshift_master_cluster_hostname=openshift.xxx.net #openshift_master_cluster_public_hostname=openshift.xxx.net #openshift_node_kubelet_args={'pods-per-core': ['10'], 'max-pods': ['250'], 'image-gc-high-threshold': ['90'], 'image-gc-low-threshold': ['80']} # host group for masters [masters] master1.xxx.net #master2.xxx.net # host group for lb #[lb] #lb.xxx.net # host group for etcd [etcd] etcd3.xxx.net etcd5.xxx.net etcd4.xxx.net # host group for nodes, includes region info [nodes] master1.xxx.net #master2.xxx.net node1.xxx.net node2.xxx.net openshift_node_labels="{'region': 'infra', 'zone': 'default'}" node3.xxx.net openshift_node_labels="{'region': 'infra', 'zone': 'default'}" node4.xxx.net node5.xxx.net openshift_node_labels="{'region': 'infra', 'zone': 'default'}"
下載openshift-ansible，一般安裝什么版本的openshift，就下載對應的tar包，但是要修改hosts文件對應的版本openshift_release=3.6.1：
wget https://github.com/openshift/openshift-ansible/archive/openshift-ansible-3.6.173.0.104-1.tar.gz
解壓並執行安裝(安裝之前可以把鏡像提前下載下來，避免因為網絡問題導致安裝失敗)：
ansible-playbook -i /etc/ansible/hosts openshift-ansible-openshift-ansible-3.6.173.0.104-1/playbooks/byo/config.yml
安裝成功如下：
1.5 驗證安裝
Master1節點上驗證node
[root@master1 ~]# oc get nodes NAME STATUS AGE VERSION master1.xxx.net Ready,SchedulingDisabled 38m v1.6.1+5115d708d7 node1.xxx.net Ready 38m v1.6.1+5115d708d7 node2.xxx.net Ready 38m v1.6.1+5115d708d7 node3.xxx.net Ready 38m v1.6.1+5115d708d7 node4.xxx.net Ready 38m v1.6.1+5115d708d7 node5.xxx.net Ready 38m v1.6.1+5115d708d7
Master1節點上驗證etcd
[root@master1 ~]# yum install etcd -y [root@master1 ~]# etcdctl -C https://etcd1.xxx.net:2379,https://etcd3.xxx.net:2379,https://etcd2.xxx.net:2379 --ca-file=/etc/origin/master/master.etcd-ca.crt --cert-file=/etc/origin/master/master.etcd-client.crt --key-file=/etc/origin/master/master.etcd-client.key cluster-health member 17c82e7e21b639e7 is healthy: got healthy result from https://192.168.10.109:2379 member 3bd39337b17b1a4e is healthy: got healthy result from https://192.168.10.111:2379 member 62cacf31d21cfcd4 is healthy: got healthy result from https://192.168.10.115:2379 cluster is healthy [root@master1 ~]# etcdctl -C https://etcd1.xxx.net:2379,https://etcd3.xxx.net:2379,https://etcd2.xxx.net:2379 --ca-file=/etc/origin/master/master.etcd-ca.crt --cert-file=/etc/origin/master/master.etcd-client.crt --key- file=/etc/origin/master/master.etcd-client.key member list 17c82e7e21b639e7: name=etcd1.xxx.net peerURLs=https://192.168.10.109:2380 clientURLs=https://192.168.10.109:2379 isLeader=false 3bd39337b17b1a4e: name=etcd2.xxx.net peerURLs=https://192.168.10.111:2380 clientURLs=https://192.168.10.111:2379 isLeader=false 62cacf31d21cfcd4: name=etcd3.xxx.net peerURLs=https://192.168.10.115:2380 clientURLs=https://192.168.10.115:2379 isLeader=true
1.6 訪問控制台
高級安裝模式下會安裝router(安裝在infra節點上)、registry、console，均可直接使用。
創建控制台賬號Master節點
htpasswd -b /etc/origin/master/htpasswd dev dev2018
登錄控制台：https://master1.xxx.net:8443
此地址需要解析到master主機上
搭建完成