一、環境配置
1.安裝環境
虛擬機:CentOS 7.6 KVM 操作系統:rhel-server-7.3-x86_64 Grid集群安裝包:linuxx64_12201_grid_home DB數據庫安裝包:linuxx64_12201_database
2.IP划分
Public-IP 10.0.7.110 ehs-rac-01 10.0.7.120 ehs-rac-02 Private-IP 10.0.0.110 ehs-rac-01-priv 10.0.0.120 ehs-rac-02-priv Virtual-IP 10.0.7.113 ehs-rac-01-vip 10.0.7.123 ehs-rac-02-vip Scan-IP 10.0.7.101
3.ASM磁盤划分
OCR 3塊:asm-diskc 2G 、asm-diskd 2G、asm-diske 2G GIMR 1塊:asm-diskf 40G 數據 1塊:asm-diskg 10G
4.Linux 系統安裝時選項包
Server with GUI #帶GUI的服務器 Hardware Monitoring Utilities #硬件監控實用程序 Large Systems Performance #大型系統性能 Network file system client #網絡文件系統客戶端 Performance Tools #性能工具 Compatibility Libraries #兼容性庫 Development Tools #開發工具
5.虛擬機配置
先配置一台虛擬機(第一個節點 ehs-rac-01) 內存設置8G 添加安裝ORACLE軟件盤 目錄為:/u01 添加ASM存儲盤(3塊OCR,1塊GIMR,1塊存放數據)
二、安裝前准備
1.修改 /etc/selinux/config 配置
[root@ehs-rac-01 ~]# sed -i 's/^SELINUX=.*/SELINUX=disabled/' /etc/selinux/config; setenforce 0 ; cat /etc/selinux/config | grep -i SELINUX= | grep -v "^#" SELINUX=disabled
2.關閉防火牆
[root@ehs-rac-01 ~]# systemctl stop firewalld; systemctl disable firewalld Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service.
3.配置 redhat yum 源
由於 Redhat 系統沒有注冊,使用 yum 會有錯誤:This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
參考文檔:https://www.cnblogs.com/opma/p/11606765.html
配置本地 yum 源
[root@ehs-rac-01 ~]# mkdir /mnt/rhel73iso [root@ehs-rac-01 ~]# mount -o loop -t iso9660 /root/rhel-server-7.3-x86_64-dvd.iso /mnt/rhel73iso mount: /dev/loop0 寫保護,將以只讀方式掛載 [root@ehs-rac-01 ~]# cd /etc/yum.repos.d [root@ehs-rac-01 yum.repos.d]# ll 總用量 4 -rw-r--r--. 1 root root 358 9月 10 15:24 redhat.repo [root@ehs-rac-01 yum.repos.d]# vim /etc/yum.repos.d/rhel-iso.repo [ISO-DVD] name=Red Hat Enterprise Linux $releasever - $basearch - Source baseurl=file:///mnt/rhel73iso enabled=1 gpgcheck=0 [root@ehs-rac-01 yum.repos.d]# df -h 文件系統 容量 已用 可用 已用% 掛載點 /dev/mapper/rhel-root 35G 6.9G 29G 20% / devtmpfs 3.9G 0 3.9G 0% /dev tmpfs 3.9G 84K 3.9G 1% /dev/shm tmpfs 3.9G 9.4M 3.9G 1% /run tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup /dev/vda1 1014M 155M 860M 16% /boot tmpfs 799M 28K 799M 1% /run/user/0 tmpfs 799M 4.0K 799M 1% /run/user/42 /dev/loop0 3.6G 3.6G 0 100% /mnt/rhel73iso
4.增加新網卡
參考文檔:https://www.cnblogs.com/opma/p/11606823.html
5.安裝依賴包
[root@ehs-rac-01 ~]# yum -y install binutils compat-libcap1 \
compat-libstdc++ gcc gcc-c++ glibc glibc-devel \
libgcc libstdc++ libstdc++-devel libaio sysstat \
libaio-devel elfutils-libelf-devel unixODBC \
unixODBC-devel libXext libXtst libX11 libXau \
libxcb ksh make libXi
6.配置內核參數
[root@ehs-rac-01 ~]# cat >> /etc/sysctl.conf << EOF fs.file-max = 6815744 kernel.sem = 250 32000 100 128 kernel.shmmni = 4096 kernel.shmall = 1073741824 kernel.shmmax = 4398046511104 kernel.panic_on_oops = 1 net.core.rmem_default = 262144 net.core.rmem_max = 4194304 net.core.wmem_default = 262144 net.core.wmem_max = 1048576 net.ipv4.conf.all.rp_filter = 2 net.ipv4.conf.default.rp_filter = 2 fs.aio-max-nr = 1048576 net.ipv4.ip_local_port_range = 9000 65500 EOF #修改生效: [root@ehs-rac-01 ~]# /sbin/sysctl -p
7.修改系統限制參數
[root@ehs-rac-01 ~]# cat >> /etc/security/limits.conf << EOF oracle soft nofile 1024 oracle hard nofile 65536 oracle soft nproc 16384 oracle hard nproc 16384 oracle soft stack 10240 oracle hard stack 32768 oracle hard memlock 134217728 oracle soft memlock 134217728 grid soft nofile 1024 grid hard nofile 65536 grid soft nproc 16384 grid hard nproc 16384 grid soft stack 10240 grid hard stack 32768 grid hard memlock 134217728 grid soft memlock 134217728 EOF
8.關閉ZEROCONF
echo "NOZEROCONF=yes" >> /etc/sysconfig/network
9.增加組和用戶
[root@ehs-rac-01 ~]# groupadd -g 54321 oinstall; groupadd -g 54322 dba; \
groupadd -g 54323 oper; groupadd -g 54324 backupdba; \
groupadd -g 54325 dgdba; groupadd -g 54326 kmdba; \
groupadd -g 54327 asmdba; groupadd -g 54328 asmoper; \
groupadd -g 54329 asmadmin; groupadd -g 54330 racdba; \
useradd -u 54321 -g oinstall -G dba,asmdba,backupdba,dgdba,kmdba,racdba,oper oracle; \
useradd -u 54322 -g oinstall -G asmadmin,asmdba,asmoper,dba grid [root@ehs-rac-01 ~]# passwd oracle [root@ehs-rac-01 ~]# passwd grid
10.修改 hosts 文件
[root@ehs-rac-01 ~]# cat >> /etc/hosts << EOF # Public 10.0.7.110 ehs-rac-01.localdomain ehs-rac-01 10.0.7.120 ehs-rac-02.localdomain ehs-rac-02 # Private 10.0.0.110 ehs-rac-01-priv.localdomain ehs-rac-01-priv 10.0.0.120 ehs-rac-02-priv.localdomain ehs-rac-02-priv # Virtual 10.0.7.113 ehs-rac-01-vip.localdomain ehs-rac-01-vip 10.0.7.123 ehs-rac-02-vip.localdomain ehs-rac-02-vip #Scan-IP 10.0.7.101 rac-scan EOF
11.查看網卡信息
[root@ehs-rac-01 ~]# ifconfig ens10: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 10.0.7.111 netmask 255.255.255.0 broadcast 10.0.7.255 inet6 fe80::9de4:f335:6b37:fc6a prefixlen 64 scopeid 0x20<link> ether 52:54:00:40:5c:b4 txqueuelen 1000 (Ethernet) RX packets 106 bytes 4876 (4.7 KiB) RX errors 0 dropped 98 overruns 0 frame 0 TX packets 27 bytes 3955 (3.8 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 10.0.7.110 netmask 255.255.255.0 broadcast 10.0.7.255 inet6 fe80::93fc:53a6:2f8d:69c3 prefixlen 64 scopeid 0x20<link> ether 52:54:00:8c:90:67 txqueuelen 1000 (Ethernet) RX packets 1131 bytes 112292 (109.6 KiB) RX errors 0 dropped 151 overruns 0 frame 0 TX packets 489 bytes 107148 (104.6 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 1 (Local Loopback) RX packets 40 bytes 2870 (2.8 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 40 bytes 2870 (2.8 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 virbr0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 inet 192.168.122.1 netmask 255.255.255.0 broadcast 192.168.122.255 ether 52:54:00:94:02:c3 txqueuelen 1000 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
12.建立文件路徑
[root@ehs-rac-01 /]# mkdir -p /u01/app/12.2.0.1/grid; mkdir -p /u01/app/grid; mkdir -p /u01/app/oracle/product/12.2.0.1/db_1; chown -R grid:oinstall /u01; chmod -R 775 /u01/; chown -R oracle:oinstall /u01/app/oracle
13.修改grid用戶的環境變量
[root@ehs-rac-01 /]# su - grid [grid@ehs-rac-01 ~]$ cd ~ [grid@ehs-rac-01 ~]$ cat >> .bash_profile << EOF export ORACLE_BASE=/u01/app/grid export ORACLE_HOME=/u01/app/12.2.0.1/grid export ORACLE_SID=+ASM1 export PATH=\$ORACLE_HOME/bin:\$PATH export LD_LIBRARY_PATH=\$ORACLE_HOME/lib:/lib:/usr/lib export CLASSPATH=\$ORACLE_HOME/JRE:\$ORACLE_HOME/jlib:\$ORACLE_HOME/rdbms/jlib umask=022 EOF #--注意 ORACLE_SID rac2 節點需要修改 +ASM2
14. 修改oracle用戶的環境變量
[root@ol7-122-rac1 ~]# su - oracle [oracle@ol7-122-rac1 ~]$ cd ~ [oracle@ol7-122-rac1 ~]$ cat >> .bash_profile << EOF export ORACLE_BASE=/u01/app/oracle export ORACLE_HOME=/u01/app/oracle/product/12.2.0.1/db_1 export ORACLE_SID=orcl1 export PATH=\$ORACLE_HOME/bin:\$PATH export LD_LIBRARY_PATH=\$ORACLE_HOME/lib:/lib:/usr/lib export CLASSPATH=\$ORACLE_HOME/JRE:\$ORACLE_HOME/jlib:\$ORACLE_HOME/rdbms/jlib umask=022 EOF #--注意rac2節點需要修改 orcl2
三、配置 iscsi 共享存儲
參考文檔:https://www.cnblogs.com/opma/p/11606875.html
四、綁定 UDEV 共享磁盤(rac1 和 rac2)
1. 生成規則文件
[root@ehs-rac-01 rules.d]# touch /etc/udev/rules.d/99-oracle-asmdevices.rules; cd /etc/udev/rules.d; ll -rw-r--r--. 1 root root 709 8月 24 2016 70-persistent-ipoib.rules -rw-r--r-- 1 root root 0 9月 10 20:16 99-oracle-asmdevices.rules #####或者##### [root@ehs-rac-01 rules.d]# touch /usr/lib/udev/rules.d/99-oracle-asmdevices.rules
2. 生成規則
for i in a b c d e; do echo "KERNEL==\"sd*\", SUBSYSTEM==\"block\", PROGRAM==\"/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/\$name\", RESULT==\"`/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/sd$i`\", SYMLINK+=\"asm-disk$i\", OWNER=\"grid\", GROUP=\"asmadmin\", MODE=\"0660\"" >> /etc/udev/rules.d/99-oracle-asmdevices.rules done [root@ehs-rac-01 rules.d]# cat 99-oracle-asmdevices.rules KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="360000000000000000e00000000010001", SYMLINK+="asm-diska", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="360000000000000000e00000000010002", SYMLINK+="asm-diskb", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="360000000000000000e00000000010003", SYMLINK+="asm-diskc", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="360000000000000000e00000000010004", SYMLINK+="asm-diskd", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="360000000000000000e00000000010005", SYMLINK+="asm-diske", OWNER="grid", GROUP="asmadmin", MODE="0660"
對sdb 進行了分區,執行如下shell腳本 (不需要操作,了解就可以)
for i in b1 b2 do echo "KERNEL==\"sd$i\", SUBSYSTEM==\"block\", PROGRAM==\"/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/\$parent\", RESULT==\"`/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/sd${i:0:1}`\", SYMLINK+=\"asm-disk$i\", OWNER=\"grid\", GROUP=\"asmadmin\", MODE=\"0660\"" >> /etc/udev/rules.d/99-oracle-asmdevices.rules done;
注意:未分區用 $name,分區用 $parent
3. 文件 99-oracle-asmdevices.rules 格式 (不需要操作,了解就可以)
KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="36000c2948ef9d9e4a7937bfc65888bc8", NAME="asm-diskb", OWNER="grid", GROUP="asmadmin", MODE="0660"
手動加載分區
[root@ehs-rac-01 rules.d]# /sbin/partprobe /dev/sda; /sbin/partprobe /dev/sdb; /sbin/partprobe /dev/sdc; /sbin/partprobe /dev/sdd; /sbin/partprobe /dev/sde
[root@ehs-rac-01 rules.d]# /usr/lib/udev/scsi_id -g -u /dev/sdb 360000000000000000e00000000010002
4. 用 udevadm 進行測試
注意: udevadm 命令不接受 /dev/sdc 這樣的掛載設備名,必須是使用 /sys/block/sdb 這樣的原始設備名。
[root@ehs-rac-01 rules.d]# /sbin/udevadm test /sys/block/sda; /sbin/udevadm test /sys/block/sdb; /sbin/udevadm test /sys/block/sdc; /sbin/udevadm test /sys/block/sdd; /sbin/udevadm test /sys/block/sde [root@ehs-rac-01 rules.d]# udevadm info --query=all --path=/sys/block/sda; udevadm info --query=all --path=/sys/block/sdb; udevadm info --query=all --path=/sys/block/sdc; udevadm info --query=all --path=/sys/block/sdd; udevadm info --query=all --path=/sys/block/sde [root@ehs-rac-01 rules.d]# udevadm info --query=all --name=asm-diska; udevadm info --query=all --name=asm-diskb; udevadm info --query=all --name=asm-diskc; udevadm info --query=all --name=asm-diskd; udevadm info --query=all --name=asm-diske
5. 啟動udev
[root@ehs-rac-01 rules.d]# /sbin/udevadm control --reload-rules [root@ehs-rac-01 rules.d]# systemctl status systemd-udevd.service [root@ehs-rac-01 rules.d]# systemctl enable systemd-udevd.service
6. 檢查設備是否正確綁定
[root@ehs-rac-01 rules.d]# ll /dev/asm* lrwxrwxrwx 1 root root 3 9月 10 20:28 /dev/asm-diska -> sda lrwxrwxrwx 1 root root 3 9月 10 20:30 /dev/asm-diskb -> sdb lrwxrwxrwx 1 root root 3 9月 10 20:30 /dev/asm-diskc -> sdc lrwxrwxrwx 1 root root 3 9月 10 20:30 /dev/asm-diskd -> sdd lrwxrwxrwx 1 root root 3 9月 10 20:30 /dev/asm-diske -> sde [root@ehs-rac-01 rules.d]# [root@ehs-rac-01 rules.d]# ll /dev/sd* brw-rw---- 1 grid asmadmin 8, 0 9月 10 20:28 /dev/sda brw-rw---- 1 grid asmadmin 8, 16 9月 10 20:30 /dev/sdb brw-rw---- 1 grid asmadmin 8, 32 9月 10 20:30 /dev/sdc brw-rw---- 1 grid asmadmin 8, 48 9月 10 20:30 /dev/sdd brw-rw---- 1 grid asmadmin 8, 64 9月 10 20:30 /dev/sde
五、設置SSH信任關系(rac1 和 rac2 )
1.Rac1服務器設置
[root@ehs-rac-01 ~]# su - oracle [oracle@ehs-rac-01 ~]$ ssh-keygen -t rsa [oracle@ehs-rac-01 ~]$ ssh-keygen -t dsa [oracle@ehs-rac-01 ~]$ ll .ssh/ total 16 -rw------- 1 oracle oinstall 672 Sep 10 13:51 id_dsa -rw-r--r-- 1 oracle oinstall 607 Sep 10 13:51 id_dsa.pub -rw------- 1 oracle oinstall 1679 Sep 10 13:51 id_rsa -rw-r--r-- 1 oracle oinstall 399 Sep 10 13:51 id_rsa.pub
(2)把rsa和dsa加密都放置到authorized_keys文件中:
[oracle@ehs-rac-01 ~]$ cat .ssh/id_rsa.pub >> .ssh/authorized_keys [oracle@ehs-rac-01 ~]$ cat .ssh/id_dsa.pub >> .ssh/authorized_keys [oracle@ehs-rac-01 ~]$ ll .ssh/ 總用量 20 -rw-r--r-- 1 oracle oinstall 1006 9月 10 20:49 authorized_keys -rw------- 1 oracle oinstall 668 9月 10 20:48 id_dsa -rw-r--r-- 1 oracle oinstall 607 9月 10 20:48 id_dsa.pub -rw------- 1 oracle oinstall 1675 9月 10 20:48 id_rsa -rw-r--r-- 1 oracle oinstall 399 9月 10 20:48 id_rsa.pub
(3)把rac1的authorized_keys拷貝到rac2中
[oracle@ehs-rac-01 ~]$ cd /home/oracle/.ssh [oracle@ehs-rac-01 .ssh]$ scp authorized_keys 10.0.7.120:/home/oracle/.ssh/
2.Rac2服務器設置
[root@ehs-rac-02 ~]# su - oracle [oracle@ehs-rac-02 ~]$ ssh-keygen -t rsa [oracle@ehs-rac-02 ~]$ ssh-keygen -t dsa [oracle@ehs-rac-02 ~]$ ll .ssh/ 總用量 20 -rw-r--r-- 1 oracle oinstall 1006 9月 10 20:51 authorized_keys -rw------- 1 oracle oinstall 668 9月 10 20:50 id_dsa -rw-r--r-- 1 oracle oinstall 607 9月 10 20:50 id_dsa.pub -rw------- 1 oracle oinstall 1679 9月 10 20:50 id_rsa -rw-r--r-- 1 oracle oinstall 399 9月 10 20:50 id_rsa.pub
(2)把rac2的rsa和dsa加密都放置到從rac1拷貝來的authorized_keys文件中:
[oracle@ehs-rac-02 ~]$ cat .ssh/id_rsa.pub >> .ssh/authorized_keys [oracle@ehs-rac-02 ~]$ cat .ssh/id_dsa.pub >> .ssh/authorized_keys
(3)把rac2中的authorized_keys文件拷貝到rac1中,覆蓋之前的authorized_keys文件:
[oracle@ehs-rac-02 ~]$ cd /home/oracle/.ssh [oracle@ehs-rac-02 .ssh]$ scp authorized_keys 10.0.7.110:/home/oracle/.ssh/
3.查看一下 rac1 和 rac2 相同的 authorized_keys 文件
[oracle@ehs-rac-01 .ssh]$ more authorized_keys ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxgKiV0Xu/9T6f2NFrWOSePEQsHHB3Ug7ueUs/ZCGvP1/QlAT8h8RowfuEZ6ZaeqwTpqpjB74j4OKGdg/bvDuDIvmRx5qrE5TRZ5fYPhc114j5nkOG+m6GM1xxgvNmn8Qi/PdXaCfqNP4KMUlTpDEyaoMT2crGzeKzM4MBUedYiAd7fM0MNEcVI0wCS4ec491NQ3TKxfdk/T6ohisNn7ZbM+EXgIXadHS2ph3Pke03rCwUjKOjo+mR4I0T5xcKk8Xdc/V4CdGMx65nQadKFZ9reRgdc7KIJn2bpTyyMrnYkytfuas1ron8HQmhhe73qhNm45ml0RDUlTKxQ8wrf7a7 oracle@ehs-rac-01 ssh-dss AAAAB3NzaC1kc3MAAACBAIby5b4SAyCNIIHH1QI+SYbP6NwfWSaPFhZUYIZfESPpEVgL6s40kksqzp3uDhCz5dqGuwhQdnVDVNUCZtSQKmqPTn4uLZopJtMf82+E8tKUjDF75gjreoVgA9MXvveqZ2sN9iz09c4QIdE5usWFBMLjyFGldrZt9ES0HHMYbfd9AAAAFQCkk/CtZ3iNP8tP0TD8h/bBkFkDYwAAAIBNMY0xwc+x05BhEDjRuetST72qYM6sp3HRfTblQ7p/kBuLAcHpdLw5h8rtc1bR/MFhYNGfZYdQS90IQJzET78zd8QzFqKF4k62Zn95UfFmKksUCmYJL/zV0Ac+TaXS8Lfx6yL1JDmrgN0/ejKgz+OxJR4+hrXSXsqImSldgtJYlAAAAIBH8mOlFgoZYpvKvyWYm59NA1LFJelA61vtUFVaVzmeh1P9J8aFnLWewnGT8RvKjoEw6Kj0lUF39kGjTYJMk8BCkvhrRf4IPlT7gMSk+x1CMao/br5N4B732Grw2LPuIyJNpqg38Oo6HOZi63qLZhUzqpJaO7KGgEYzoD4AtrwMfw== oracle@ehs-rac-01 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYCk6agT6tY4D3b3vdB120HpeH8Pqo6ZHUBuLVjvqCK0QtxQcdleWQXRToULqyu3uYuHlsiJCAG6kbT3w2lbkVJLAtWhYa7u0owuy2zhMYf+2bvvTfOgbA/z6JikRpEGRmUEa6/uatr9EhuVQ2st3peyn/e5Gdnb55hhQ8XYJ4IClhCfmkaeT+arl1BjLDO7SG6cLcedEiJAGpNUtnzLqo86g2T6ar/GbwQzy1FfFcc8PzIgClsCXTJP3oXWUdOlDXs6Tonugs/rVjV22ISkdeWNfQnZouPa9YdX7trJrpw5VzenksVFhAXmfkVo7IhNHvnqyPuU+lhoiMXuGJ13cv oracle@ehs-rac-02 ssh-dss AAAAB3NzaC1kc3MAAACBAKqWe/wM9wHWxTrjsxE7nU/oZ1zzGtVR2o21Na9V6ivZpQYPp2CobiUmw5EyNAHDJndgyaBOS6GH6iPyaCVKhikhr5ZRKqIwxq+xNcZJl+it1Yry7pnGL3V3zdqu433zJU9IXtcHAF1wj0EEvGlm+USSE9oOeixBfplCPWOcz5fFAAAAFQDaFCYj81olTz+HWleGN0RpXaRmAwAAAIEAh9/AuRE1n06nhobrOKGgpEs7G3nGFxuJPZ98apYVpNf28qPwgPm+dfHIfpIIbaLEOpYK1jXLfaH/ZgM9CSy8rlf2bANPKcZ3AfktjLBenuZiJLP9LPOgPXoJlL+zVK2j7lYa+oCmI4lM4BVxhWr/nzCES5kgOfM1DiiuGd8QYEEAAACAMn5GWRD1ZO+k19yjip9HqRlto8MpoUvXQuDkUjIimx7cEBrd0dK0pJ/sBIIZqW0nw/8RvUd2AcMquoq+uR41NSJNWKwQPufLy2p2QQEnTCABP9IXzcHSPORYE/0QGkOuos467HK0JvxbnSsvmTwXSO9ieTXwoQvE2VJBNVsFNKY= oracle@ehs-rac-02
4.測試SSH
Rac1和 rac2分別測試,可以正常返回日期即可: [oracle@ehs-rac-01 .ssh]$ ssh ehs-rac-01 date Tue Sep 10 14:11:55 CST 2019 [oracle@ehs-rac-01 .ssh]$ ssh ehs-rac-01-priv date Tue Sep 10 14:11:59 CST 2019 [oracle@ehs-rac-01 .ssh]$ ssh ehs-rac-02 date Tue Sep 10 14:12:03 CST 2019 [oracle@ehs-rac-01 .ssh]$ ssh ehs-rac-02-priv date Tue Sep 10 14:12:08 CST 2019 [oracle@ehs-rac-02 .ssh]$ ssh ehs-rac-01 date Tue Sep 10 14:10:38 CST 2019 [oracle@ehs-rac-02 .ssh]$ ssh ehs-rac-01-priv date Tue Sep 10 14:10:43 CST 2019 [oracle@ehs-rac-02 .ssh]$ ssh ehs-rac-02 date Tue Sep 10 14:10:51 CST 2019 [oracle@ehs-rac-02 .ssh]$ ssh ehs-rac-02-priv date Tue Sep 10 14:10:56 CST 2019
六、安裝GRID軟件 (rac1)
[root@ehs-rac-01 ~]# su - grid [grid@ehs-rac-01 ~]$ unzip linuxx64_12201_grid_home.zip -d $ORACLE_HOME [grid@ehs-rac-01 ~]$ cd $ORACLE_HOME
1.安裝grid前檢查
[grid@ehs-rac-01 grid]$ ./runcluvfy.sh stage -pre crsinst -n ehs-rac-01,ehs-rac-02 -verbose ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 正在驗證交換空間大小...失敗 ehs-rac-01: PRVF-7573 : 節點 "ehs-rac-01" 上的交換空間大小不足 [應為 = 7.7967GB (8175384.0KB); 找到 = 4GB (4194300.0KB)] 執行 CVU 驗證請求 "stage -pre crsinst" 時失敗。 正在驗證包: ksh...失敗 ehs-rac-01: PRVF-7532 : 節點 "ehs-rac-01" 上缺少程序包 "ksh" 正在驗證包: cvuqdisk-1.0.10-1...失敗 ehs-rac-01: PRVG-11550 : 節點 "ehs-rac-01" 上缺少程序包 "cvuqdisk" 正在驗證守護程序 "avahi-daemon" 未配置且未運行...失敗 ehs-rac-01: PRVG-1360 : 守護程序進程 "avahi-daemon" 正在節點 "ehs-rac-01" 上運行 正在驗證/dev/shm 作為臨時文件系統裝載...失敗 ehs-rac-01: PRVE-0421 : /etc/fstab 中不存在用於裝載 /dev/shm 的條目 正在驗證zeroconf 檢查...失敗 ehs-rac-01: PRVE-10077 : 在節點 "ehs-rac-01" 上的文件 "/etc/sysconfig/network" 中未指定 NOZEROCONF 參數或未將該參數設置為 'yes' ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
解決檢查未通過項:
正在驗證交換空間大小... 節點名 可用 必需 狀態 ------------ ------------------------ ------------------------ ---------- ehs-rac-01 4GB (4194300.0KB) 7.7967GB (8175384.0KB) 失敗 正在驗證交換空間大小...失敗 (PRVF-7573)
解決方法:
[root@ehs-rac-01 /]# free -h total used free shared buff/cache available Mem: 7.8G 554M 6.1G 11M 1.1G 7.0G Swap: 4.0G 0B 4.0G [root@ehs-rac-01 /]# dd if=/dev/zero of=/swap bs=1024 count=12000000 12000000+0 records in 12000000+0 records out 12288000000 bytes (12 GB) copied, 82.7888 s, 148 MB/s [root@ehs-rac-01 /]# mkswap /swap Setting up swapspace version 1, size = 11999996 KiB no label, UUID=d50bfab0-4716-4971-8413-b66d427c65a7 [root@ehs-rac-01 /]# swapon /swap swapon: /swap: insecure permissions 0644, 0600 suggested. [root@ehs-rac-01 /]# vim /etc/fstab UUID=d50bfab0-4716-4971-8413-b66d427c65a7 swap swap default 0 0 [root@ehs-rac-01 /]# free -h total used free shared buff/cache available Mem: 7.8G 568M 129M 9.9M 7.1G 6.9G Swap: 15G 1.5M 15G
(2)正在驗證包: ksh...失敗 (PRVF-7532)
正在驗證包: ksh... 節點名 可用 必需 狀態 ------------ ------------------------ ------------------------ ---------- ehs-rac-01 缺失 ksh 失敗 正在驗證包: ksh...失敗 (PRVF-7532)
解決方法:
yum install -y ksh
(3)正在驗證包: cvuqdisk-1.0.10-1...失敗 (PRVG-11550)
正在驗證包: cvuqdisk-1.0.10-1... 節點名 可用 必需 狀態 ------------ ------------------------ ------------------------ ---------- ehs-rac-01 缺失 cvuqdisk-1.0.10-1 失敗 正在驗證包: cvuqdisk-1.0.10-1...失敗 (PRVG-11550)
解決方法:
[root@ehs-rac-01 shm]# cd /u01/app/12.2.0.1/grid/cv/rpm [root@ehs-rac-01 rpm]# rpm -ivh cvuqdisk-1.0.10-1.rpm
(4)正在驗證(Linux) resolv.conf 完整性...失敗 (PRVG-10048)
正在驗證resolv.conf 完整性... 正在驗證(Linux) resolv.conf 完整性... 節點名 狀態 ------------------------------------ ------------------------ ehs-rac-01 通過 檢查 "/etc/resolv.conf" 中指定的每個名稱服務器對名稱 "ehs-rac-01" 的響應 節點名 源 注釋 狀態 ------------ ------------------------ ------------------------ ---------- ehs-rac-01 114.114.114.114 IPv4 失敗 正在驗證(Linux) resolv.conf 完整性...失敗 (PRVG-10048) 正在驗證resolv.conf 完整性...失敗 (PRVG-10048) 正在驗證DNS/NIS 名稱服務...通過 正在驗證域套接字...通過 正在驗證/boot 裝載...通過
解決方法:
正在驗證守護程序 "avahi-daemon" 未配置且未運行... 節點名 已配置 狀態 ------------ ------------------------ ------------------------ ehs-rac-01 否 通過 節點名 正在運行? 狀態 ------------ ------------------------ ------------------------ ehs-rac-01 是 失敗 正在驗證守護程序 "avahi-daemon" 未配置且未運行...失敗 (PRVG-1360)
解決方法:
[root@ehs-rac-01 rpm]# ps -ef|grep avahi-daemon avahi 700 1 0 9月10 ? 00:00:00 avahi-daemon: running [ehs-rac-01-2.local] avahi 708 700 0 9月10 ? 00:00:00 avahi-daemon: chroot helper root 12861 23175 0 13:56 pts/0 00:00:00 grep --color=auto avahi-daemon [root@ehs-rac-01 rpm]# kill -9 700 708 [root@ehs-rac-01 rpm]# ps -ef|grep avahi-daemon root 12873 23175 0 13:56 pts/0 00:00:00 grep --color=auto avahi-daemon
(6)正在驗證 /dev/shm 作為臨時文件系統裝載...失敗 (PRVE-0421)
echo "NOZEROCONF=yes" >> /etc/sysconfig/network
(8)在所有節點上預檢查 集群服務設置 失敗。
2.檢查網絡和等效性
[grid@ehs-rac-01 grid]$ ./runcluvfy.sh comp nodecon -n ehs-rac-01,ehs-rac-02 -verbose WARNING: 忽略節點 "ehs-rac-01" 的多個規范。將在節點 "ehs-rac-01" 上執行檢查 正在驗證節點連接性... 正在驗證主機文件... 節點名 狀態 ------------------------------------ ------------------------ ehs-rac-01 通過 正在驗證主機文件...通過 節點 "ehs-rac-01" 的接口信息 名稱 IP 地址 子網 網關 默認網關 HW 地址 MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ ens10 10.0.7.111 10.0.7.0 0.0.0.0 10.0.7.1 52:54:00:CA:41:DD 1500 eth0 10.0.7.110 10.0.7.0 0.0.0.0 10.0.7.1 52:54:00:76:BC:32 1500 virbr0 192.168.122.1 192.168.122.0 0.0.0.0 10.0.7.1 52:54:00:F3:67:77 1500 檢查: 子網 "10.0.7.0" 的 MTU 一致性。 節點 名稱 IP 地址 子網 MTU ---------------- ------------ ------------ ------------ ---------------- ehs-rac-01 ens10 10.0.7.111 10.0.7.0 1500 ehs-rac-01 eth0 10.0.7.110 10.0.7.0 1500 檢查: 子網 "192.168.122.0" 的 MTU 一致性。 節點 名稱 IP 地址 子網 MTU ---------------- ------------ ------------ ------------ ---------------- ehs-rac-01 virbr0 192.168.122.1 192.168.122.0 1500 正在驗證檢查流經子網的最大 (MTU) 大小數據包...通過 源 目標 是否已連接? ------------------------------ ------------------------------ ---------------- ehs-rac-01[ens10:10.0.7.111] ehs-rac-01[eth0:10.0.7.110] 是 正在驗證節點連接性...通過 正在驗證多點傳送檢查... 正在檢查子網 "10.0.7.0" 是否能夠與多點傳送組 "224.0.0.251" 進行多點傳送通信 正在驗證多點傳送檢查...通過 節點連接性 的驗證成功。 CVU 操作已執行: 節點連接性 日期: 2019-9-11 10:38:34 CVU 主目錄: /home/grid/linuxx64_12201_grid_home/ 用戶: grid
3.安裝 GRID
[grid@ehs-rac-01 grid]$ ./gridSetup.sh ERROR: Unable to verify the graphical display setup. This application requires X display. Make sure that xdpyinfo exist under PATH variable. No X11 DISPLAY variable was set, but this program performed an operation which requires it.
錯誤:無法驗證圖形顯示設置。 此應用程序需要X顯示。 確保在PATH變量下存在xdpyinfo。
[root@ehs-rac-01 ~]# xhost + access control disabled, clients can connect from any host
這時就需要設置DISPLAY環境變量了,但是現在並不清楚DISPLAY的端口,需要用xdpyinfo命令查看DISPLAY的當前端口是什么,默認是0。
[root@ehs-rac-01 ~]# xdpyinfo | head name of display: :0 version number: 11.0 vendor string: The X.Org Foundation vendor release number: 11905000 X.Org version: 1.19.5 maximum request size: 16777212 bytes motion buffer size: 256 bitmap unit, bit order, padding: 32, LSBFirst, 32 image byte order: LSBFirst number of supported pixmap formats: 7
xdpyinfo的命令輸出過多,這里只需要關注第一行也就是name of display后面的值,然后設置DISPLAY環境變量的值為name of display后面的值即可。
[grid@ehs-rac-01 grid]$ export DISPLAY=:0
或者像下面這樣設置也可以。
[grid@ehs-rac-01 grid]$ export DISPLAY=localhost:0.0
然后就可用解決這個問題打開圖形界面了。
[grid@ehs-rac-01 grid]$ ./gridSetup.sh
注意:SCAN 名稱需要對應 hosts 文件中 scan ip 名稱一致,否則會報下面錯誤;

單擊“Add”,添加其他節點的信息,注意與 /etc/hosts 文件中保持一致。添加完后單擊“OK”。
安裝到 Installation Location 時遇到 [INS-44002] The Oracle home location contains directories or files on following remote nodes 錯誤。

此時到 rac02 節點目錄查看是否有文件
[root@ehs-rac-02 grid]# cd /u01/app/12.2.0.1/grid [root@ehs-rac-02 grid]# ls addnode crs deinstall gpnp inventory ldap OPatch ords precomp rhp rootupgrade.sh.ouibak srvm welcome.html assistants css demo gridSetup.sh javavm lib opmn oss QOpatch root.sh runcluvfy.sh suptools wlm bin cv diagnostics has jdbc linuxx64_12201_grid_home.zip oracore oui qos root.sh.old scheduler tomcat wwg cfgtoollogs dbjava dmu hs jdk md oraInst.loc owm racg root.sh.old.1 slax ucp xag cha dbs env.ora install jlib network ord perl rdbms root.sh.ouibak sqlpatch usm xdk clone dc_ocm evm instantclient JRE nls ordim plsql relnotes rootupgrade.sh sqlplus utl
分別在 rac01 和 rac02 節點上執行腳本
[root@ehs-rac-01 grid]# /u01/app/oraInventory/orainstRoot.sh 更改權限/u01/app/oraInventory. 添加組的讀取和寫入權限。 刪除全局的讀取, 寫入和執行權限。 更改組名/u01/app/oraInventory 到 oinstall. 腳本的執行已完成。 [root@ehs-rac-01 grid]# /u01/app/12.2.0.1/grid/root.sh Performing root user operation. The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /u01/app/12.2.0.1/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: Copying dbhome to /usr/local/bin ... Copying oraenv to /usr/local/bin ... Copying coraenv to /usr/local/bin ... Creating /etc/oratab file... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Relinking oracle with rac_on option Using configuration parameter file: /u01/app/12.2.0.1/grid/crs/install/crsconfig_params The log of current session can be found at: /u01/app/grid/crsdata/ehs-rac-01/crsconfig/rootcrs_ehs-rac-01_2019-09-16_11-39-39AM.log 2019/09/16 11:40:05 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'. 2019/09/16 11:40:07 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector. 2019/09/16 11:40:41 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector. 2019/09/16 11:40:41 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'. 2019/09/16 11:40:52 CLSRSC-363: User ignored prerequisites during installation 2019/09/16 11:40:52 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'. 2019/09/16 11:40:54 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'. 2019/09/16 11:40:56 CLSRSC-594: Executing installation step 5 of 19: 'SaveParamFile'. 2019/09/16 11:41:07 CLSRSC-594: Executing installation step 6 of 19: 'SetupOSD'. 2019/09/16 11:41:08 CLSRSC-594: Executing installation step 7 of 19: 'CheckCRSConfig'. 2019/09/16 11:41:08 CLSRSC-594: Executing installation step 8 of 19: 'SetupLocalGPNP'. 2019/09/16 11:41:38 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'. 2019/09/16 11:41:54 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'. 2019/09/16 11:41:54 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'. 2019/09/16 11:42:03 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'. 2019/09/16 11:42:18 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service' 2019/09/16 11:43:05 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'. 2019/09/16 11:43:14 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'. CRS-2791: 正在啟動用於關閉 'ehs-rac-01' 上 Oracle 高可用性服務管理的資源的操作 CRS-2793: 關閉 'ehs-rac-01' 上 Oracle 高可用性服務管理的資源的操作已完成 CRS-4133: Oracle 高可用性服務已停止。 CRS-4123: Oracle 高可用性服務已啟動。 2019/09/16 11:43:46 CLSRSC-400: A system reboot is required to continue installing. The command '/u01/app/12.2.0.1/grid/perl/bin/perl -I/u01/app/12.2.0.1/grid/perl/lib -I/u01/app/12.2.0.1/grid/crs/install /u01/app/12.2.0.1/grid/crs/install/rootcrs.pl ' execution failed
2019/09/16 11:43:46 CLSRSC-400: A system reboot is required to continue installing. The command '/u01/app/12.2.0.1/grid/perl/bin/perl -I/u01/app/12.2.0.1/grid/perl/lib -I/u01/app/12.2.0.1/grid/crs/install /u01/app/12.2.0.1/grid/crs/install/rootcrs.pl ' execution failed
解決:
[grid@ehs-rac-01 grid]$ acfsdriverstate -orahome $ORACLE_HOME supported ACFS-9459: ADVM/ACFS is not supported on this OS version: '3.10.0-514.el7.x86_64' ACFS-9201: Not Supported
解釋:
我的系統不支持ADVM / ACFS,意味着我無法安裝ACFS驅動程序。 “Doc ID 2025056.1”表示:如果節點不使用或不支持ACFS,則可以忽略這些錯誤消息。 所以我重新啟動系統並刪除安裝的文件。 然后我再次安裝,沒有出現錯誤。 如果節點2出現相同的錯誤,只需重啟系統並再次運行root.sh。
重新執行結果:
[root@ehs-rac-01 grid]# /u01/app/12.2.0.1/grid/root.sh Performing root user operation. The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /u01/app/12.2.0.1/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed. No need to overwrite. The contents of "oraenv" have not changed. No need to overwrite. The contents of "coraenv" have not changed. No need to overwrite. Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Relinking oracle with rac_on option Using configuration parameter file: /u01/app/12.2.0.1/grid/crs/install/crsconfig_params The log of current session can be found at: /u01/app/grid/crsdata/ehs-rac-01/crsconfig/rootcrs_ehs-rac-01_2019-09-16_12-26-25AM.log 2019/09/16 12:26:29 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'. 2019/09/16 12:26:29 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector. 2019/09/16 12:26:29 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector. 2019/09/16 12:26:30 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'. 2019/09/16 12:26:32 CLSRSC-363: User ignored prerequisites during installation 2019/09/16 12:26:32 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'. 2019/09/16 12:26:34 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'. 2019/09/16 12:26:36 CLSRSC-594: Executing installation step 5 of 19: 'SaveParamFile'. 2019/09/16 12:26:38 CLSRSC-594: Executing installation step 6 of 19: 'SetupOSD'. 2019/09/16 12:26:39 CLSRSC-594: Executing installation step 7 of 19: 'CheckCRSConfig'. 2019/09/16 12:26:40 CLSRSC-594: Executing installation step 8 of 19: 'SetupLocalGPNP'. 2019/09/16 12:26:43 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'. 2019/09/16 12:26:44 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'. 2019/09/16 12:27:31 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'. 2019/09/16 12:27:32 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'. 2019/09/16 12:27:33 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'. 2019/09/16 12:27:38 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'. CRS-2791: 正在啟動用於關閉 'ehs-rac-01' 上 Oracle 高可用性服務管理的資源的操作 CRS-2793: 關閉 'ehs-rac-01' 上 Oracle 高可用性服務管理的資源的操作已完成 CRS-4133: Oracle 高可用性服務已停止。 CRS-4123: Oracle 高可用性服務已啟動。 2019/09/16 12:28:11 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'. 2019/09/16 12:28:20 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'. CRS-2791: 正在啟動用於關閉 'ehs-rac-01' 上 Oracle 高可用性服務管理的資源的操作 CRS-2793: 關閉 'ehs-rac-01' 上 Oracle 高可用性服務管理的資源的操作已完成 CRS-4133: Oracle 高可用性服務已停止。 CRS-4123: Oracle 高可用性服務已啟動。 CRS-2672: 嘗試啟動 'ora.evmd' (在 'ehs-rac-01' 上) CRS-2672: 嘗試啟動 'ora.mdnsd' (在 'ehs-rac-01' 上) CRS-2676: 成功啟動 'ora.mdnsd' (在 'ehs-rac-01' 上) CRS-2676: 成功啟動 'ora.evmd' (在 'ehs-rac-01' 上) CRS-2672: 嘗試啟動 'ora.gpnpd' (在 'ehs-rac-01' 上) CRS-2676: 成功啟動 'ora.gpnpd' (在 'ehs-rac-01' 上) CRS-2672: 嘗試啟動 'ora.cssdmonitor' (在 'ehs-rac-01' 上) CRS-2672: 嘗試啟動 'ora.gipcd' (在 'ehs-rac-01' 上) CRS-2676: 成功啟動 'ora.cssdmonitor' (在 'ehs-rac-01' 上) CRS-2676: 成功啟動 'ora.gipcd' (在 'ehs-rac-01' 上) CRS-2672: 嘗試啟動 'ora.cssd' (在 'ehs-rac-01' 上) CRS-2672: 嘗試啟動 'ora.diskmon' (在 'ehs-rac-01' 上) CRS-2676: 成功啟動 'ora.diskmon' (在 'ehs-rac-01' 上) CRS-2676: 成功啟動 'ora.cssd' (在 'ehs-rac-01' 上) 已成功創建磁盤組。有關詳細信息, 請查看 /u01/app/grid/cfgtoollogs/asmca/asmca-190916下午122902.log。 2019/09/16 12:30:46 CLSRSC-482: Running command: '/u01/app/12.2.0.1/grid/bin/ocrconfig -upgrade grid oinstall' CRS-2672: 嘗試啟動 'ora.crf' (在 'ehs-rac-01' 上) CRS-2672: 嘗試啟動 'ora.storage' (在 'ehs-rac-01' 上) CRS-2676: 成功啟動 'ora.storage' (在 'ehs-rac-01' 上) CRS-2676: 成功啟動 'ora.crf' (在 'ehs-rac-01' 上) CRS-2672: 嘗試啟動 'ora.crsd' (在 'ehs-rac-01' 上) CRS-2676: 成功啟動 'ora.crsd' (在 'ehs-rac-01' 上) CRS-4256: 更新概要文件 已成功添加表決磁盤 e81cdfa91e744f09bfdf07f2f234e0b5。 已成功添加表決磁盤 560fa68cb9f44f4fbfdbd9c43496b321。 已成功添加表決磁盤 5afa9cb55c614feebffe7540e6c0f053。 已成功將表決磁盤組替換為 +DG_OCR。 CRS-4256: 更新概要文件 CRS-4266: 已成功替換表決文件 ## STATE File Universal Id File Name Disk group -- ----- ----------------- --------- --------- 1. ONLINE e81cdfa91e744f09bfdf07f2f234e0b5 (/dev/sda) [DG_OCR] 2. ONLINE 560fa68cb9f44f4fbfdbd9c43496b321 (/dev/sdc) [DG_OCR] 3. ONLINE 5afa9cb55c614feebffe7540e6c0f053 (/dev/sdb) [DG_OCR] 找到了 3 個表決磁盤。 CRS-2791: 正在啟動用於關閉 'ehs-rac-01' 上 Oracle 高可用性服務管理的資源的操作 CRS-2673: 嘗試停止 'ora.crsd' (在 'ehs-rac-01' 上) CRS-2677: 成功停止 'ora.crsd' (在 'ehs-rac-01' 上) CRS-2673: 嘗試停止 'ora.storage' (在 'ehs-rac-01' 上) CRS-2673: 嘗試停止 'ora.crf' (在 'ehs-rac-01' 上) CRS-2673: 嘗試停止 'ora.gpnpd' (在 'ehs-rac-01' 上) CRS-2673: 嘗試停止 'ora.mdnsd' (在 'ehs-rac-01' 上) CRS-2677: 成功停止 'ora.crf' (在 'ehs-rac-01' 上) CRS-2677: 成功停止 'ora.gpnpd' (在 'ehs-rac-01' 上) CRS-2677: 成功停止 'ora.storage' (在 'ehs-rac-01' 上) CRS-2673: 嘗試停止 'ora.asm' (在 'ehs-rac-01' 上) CRS-2677: 成功停止 'ora.mdnsd' (在 'ehs-rac-01' 上) CRS-2677: 成功停止 'ora.asm' (在 'ehs-rac-01' 上) CRS-2673: 嘗試停止 'ora.cluster_interconnect.haip' (在 'ehs-rac-01' 上) CRS-2677: 成功停止 'ora.cluster_interconnect.haip' (在 'ehs-rac-01' 上) CRS-2673: 嘗試停止 'ora.ctssd' (在 'ehs-rac-01' 上) CRS-2673: 嘗試停止 'ora.evmd' (在 'ehs-rac-01' 上) CRS-2677: 成功停止 'ora.ctssd' (在 'ehs-rac-01' 上) CRS-2677: 成功停止 'ora.evmd' (在 'ehs-rac-01' 上) CRS-2673: 嘗試停止 'ora.cssd' (在 'ehs-rac-01' 上) CRS-2677: 成功停止 'ora.cssd' (在 'ehs-rac-01' 上) CRS-2673: 嘗試停止 'ora.gipcd' (在 'ehs-rac-01' 上) CRS-2677: 成功停止 'ora.gipcd' (在 'ehs-rac-01' 上) CRS-2793: 關閉 'ehs-rac-01' 上 Oracle 高可用性服務管理的資源的操作已完成 CRS-4133: Oracle 高可用性服務已停止。 2019/09/16 12:32:46 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'. CRS-4123: 正在啟動 Oracle 高可用性服務管理的資源 CRS-2672: 嘗試啟動 'ora.mdnsd' (在 'ehs-rac-01' 上) CRS-2672: 嘗試啟動 'ora.evmd' (在 'ehs-rac-01' 上) CRS-2676: 成功啟動 'ora.mdnsd' (在 'ehs-rac-01' 上) CRS-2676: 成功啟動 'ora.evmd' (在 'ehs-rac-01' 上) CRS-2672: 嘗試啟動 'ora.gpnpd' (在 'ehs-rac-01' 上) CRS-2676: 成功啟動 'ora.gpnpd' (在 'ehs-rac-01' 上) CRS-2672: 嘗試啟動 'ora.gipcd' (在 'ehs-rac-01' 上) CRS-2676: 成功啟動 'ora.gipcd' (在 'ehs-rac-01' 上) CRS-2672: 嘗試啟動 'ora.drivers.acfs' (在 'ehs-rac-01' 上) CRS-2674: 未能啟動 'ora.drivers.acfs' (在 'ehs-rac-01' 上) CRS-2672: 嘗試啟動 'ora.cssdmonitor' (在 'ehs-rac-01' 上) CRS-2676: 成功啟動 'ora.cssdmonitor' (在 'ehs-rac-01' 上) CRS-2672: 嘗試啟動 'ora.cssd' (在 'ehs-rac-01' 上) CRS-2672: 嘗試啟動 'ora.diskmon' (在 'ehs-rac-01' 上) CRS-2676: 成功啟動 'ora.diskmon' (在 'ehs-rac-01' 上) CRS-2676: 成功啟動 'ora.cssd' (在 'ehs-rac-01' 上) CRS-2672: 嘗試啟動 'ora.cluster_interconnect.haip' (在 'ehs-rac-01' 上) CRS-2672: 嘗試啟動 'ora.ctssd' (在 'ehs-rac-01' 上) CRS-2676: 成功啟動 'ora.ctssd' (在 'ehs-rac-01' 上) CRS-2672: 嘗試啟動 'ora.drivers.acfs' (在 'ehs-rac-01' 上) CRS-2674: 未能啟動 'ora.drivers.acfs' (在 'ehs-rac-01' 上) CRS-2676: 成功啟動 'ora.cluster_interconnect.haip' (在 'ehs-rac-01' 上) CRS-2672: 嘗試啟動 'ora.asm' (在 'ehs-rac-01' 上) CRS-2676: 成功啟動 'ora.asm' (在 'ehs-rac-01' 上) CRS-2672: 嘗試啟動 'ora.storage' (在 'ehs-rac-01' 上) CRS-2676: 成功啟動 'ora.storage' (在 'ehs-rac-01' 上) CRS-2672: 嘗試啟動 'ora.crf' (在 'ehs-rac-01' 上) CRS-2676: 成功啟動 'ora.crf' (在 'ehs-rac-01' 上) CRS-2672: 嘗試啟動 'ora.crsd' (在 'ehs-rac-01' 上) CRS-2676: 成功啟動 'ora.crsd' (在 'ehs-rac-01' 上) CRS-6023: 正在啟動 Oracle 集群就緒服務管理的資源 CRS-6017: 正在處理以下服務器的資源自動啟動: ehs-rac-01 CRS-6016: 服務器 ehs-rac-01 的資源自動啟動已完成 CRS-6024: 已完成啟動 Oracle 集群就緒服務管理的資源 CRS-4123: Oracle 高可用性服務已啟動。 2019/09/16 12:35:28 CLSRSC-343: Successfully started Oracle Clusterware stack 2019/09/16 12:35:28 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'. CRS-2672: 嘗試啟動 'ora.ASMNET1LSNR_ASM.lsnr' (在 'ehs-rac-01' 上) CRS-2676: 成功啟動 'ora.ASMNET1LSNR_ASM.lsnr' (在 'ehs-rac-01' 上) CRS-2672: 嘗試啟動 'ora.asm' (在 'ehs-rac-01' 上) CRS-2676: 成功啟動 'ora.asm' (在 'ehs-rac-01' 上) CRS-2672: 嘗試啟動 'ora.DG_OCR.dg' (在 'ehs-rac-01' 上) CRS-2676: 成功啟動 'ora.DG_OCR.dg' (在 'ehs-rac-01' 上) 2019/09/16 12:38:58 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'. 2019/09/16 12:40:25 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded
rac2 執行結果:
[root@ehs-rac-02 grid]# /u01/app/oraInventory/orainstRoot.sh 更改權限/u01/app/oraInventory. 添加組的讀取和寫入權限。 刪除全局的讀取, 寫入和執行權限。 更改組名/u01/app/oraInventory 到 oinstall. 腳本的執行已完成。 [root@ehs-rac-02 grid]# /u01/app/12.2.0.1/grid/root.sh Performing root user operation. The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /u01/app/12.2.0.1/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed. No need to overwrite. The contents of "oraenv" have not changed. No need to overwrite. The contents of "coraenv" have not changed. No need to overwrite. Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Relinking oracle with rac_on option Using configuration parameter file: /u01/app/12.2.0.1/grid/crs/install/crsconfig_params The log of current session can be found at: /u01/app/grid/crsdata/ehs-rac-02/crsconfig/rootcrs_ehs-rac-02_2019-09-16_12-46-13AM.log 2019/09/16 12:46:17 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'. 2019/09/16 12:46:17 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector. 2019/09/16 12:46:18 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector. 2019/09/16 12:46:18 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'. 2019/09/16 12:46:20 CLSRSC-363: User ignored prerequisites during installation 2019/09/16 12:46:20 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'. 2019/09/16 12:46:21 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'. 2019/09/16 12:46:22 CLSRSC-594: Executing installation step 5 of 19: 'SaveParamFile'. 2019/09/16 12:46:23 CLSRSC-594: Executing installation step 6 of 19: 'SetupOSD'. 2019/09/16 12:46:25 CLSRSC-594: Executing installation step 7 of 19: 'CheckCRSConfig'. 2019/09/16 12:46:25 CLSRSC-594: Executing installation step 8 of 19: 'SetupLocalGPNP'. 2019/09/16 12:46:27 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'. 2019/09/16 12:46:29 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'. 2019/09/16 12:47:16 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'. 2019/09/16 12:47:17 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'. 2019/09/16 12:47:18 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'. 2019/09/16 12:47:19 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'. CRS-2791: 正在啟動用於關閉 'ehs-rac-02' 上 Oracle 高可用性服務管理的資源的操作 CRS-2793: 關閉 'ehs-rac-02' 上 Oracle 高可用性服務管理的資源的操作已完成 CRS-4133: Oracle 高可用性服務已停止。 CRS-4123: Oracle 高可用性服務已啟動。 2019/09/16 12:47:38 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'. 2019/09/16 12:47:40 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'. CRS-2791: 正在啟動用於關閉 'ehs-rac-02' 上 Oracle 高可用性服務管理的資源的操作 CRS-2793: 關閉 'ehs-rac-02' 上 Oracle 高可用性服務管理的資源的操作已完成 CRS-4133: Oracle 高可用性服務已停止。 CRS-4123: Oracle 高可用性服務已啟動。 CRS-2791: 正在啟動用於關閉 'ehs-rac-02' 上 Oracle 高可用性服務管理的資源的操作 CRS-2793: 關閉 'ehs-rac-02' 上 Oracle 高可用性服務管理的資源的操作已完成 CRS-4133: Oracle 高可用性服務已停止。 2019/09/16 12:48:02 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'. CRS-4123: 正在啟動 Oracle 高可用性服務管理的資源 CRS-2672: 嘗試啟動 'ora.mdnsd' (在 'ehs-rac-02' 上) CRS-2672: 嘗試啟動 'ora.evmd' (在 'ehs-rac-02' 上) CRS-2676: 成功啟動 'ora.mdnsd' (在 'ehs-rac-02' 上) CRS-2676: 成功啟動 'ora.evmd' (在 'ehs-rac-02' 上) CRS-2672: 嘗試啟動 'ora.gpnpd' (在 'ehs-rac-02' 上) CRS-2676: 成功啟動 'ora.gpnpd' (在 'ehs-rac-02' 上) CRS-2672: 嘗試啟動 'ora.gipcd' (在 'ehs-rac-02' 上) CRS-2676: 成功啟動 'ora.gipcd' (在 'ehs-rac-02' 上) CRS-2672: 嘗試啟動 'ora.cssdmonitor' (在 'ehs-rac-02' 上) CRS-2676: 成功啟動 'ora.cssdmonitor' (在 'ehs-rac-02' 上) CRS-2672: 嘗試啟動 'ora.cssd' (在 'ehs-rac-02' 上) CRS-2672: 嘗試啟動 'ora.diskmon' (在 'ehs-rac-02' 上) CRS-2676: 成功啟動 'ora.diskmon' (在 'ehs-rac-02' 上) CRS-2676: 成功啟動 'ora.cssd' (在 'ehs-rac-02' 上) CRS-2672: 嘗試啟動 'ora.cluster_interconnect.haip' (在 'ehs-rac-02' 上) CRS-2672: 嘗試啟動 'ora.ctssd' (在 'ehs-rac-02' 上) CRS-2676: 成功啟動 'ora.ctssd' (在 'ehs-rac-02' 上) CRS-2672: 嘗試啟動 'ora.crf' (在 'ehs-rac-02' 上) CRS-2676: 成功啟動 'ora.crf' (在 'ehs-rac-02' 上) CRS-2672: 嘗試啟動 'ora.crsd' (在 'ehs-rac-02' 上) CRS-2676: 成功啟動 'ora.crsd' (在 'ehs-rac-02' 上) CRS-2676: 成功啟動 'ora.cluster_interconnect.haip' (在 'ehs-rac-02' 上) CRS-2672: 嘗試啟動 'ora.drivers.acfs' (在 'ehs-rac-02' 上) CRS-2674: 未能啟動 'ora.drivers.acfs' (在 'ehs-rac-02' 上) CRS-2672: 嘗試啟動 'ora.asm' (在 'ehs-rac-02' 上) CRS-2676: 成功啟動 'ora.asm' (在 'ehs-rac-02' 上) CRS-6017: 正在處理以下服務器的資源自動啟動: ehs-rac-02 CRS-2672: 嘗試啟動 'ora.net1.network' (在 'ehs-rac-02' 上) CRS-2672: 嘗試啟動 'ora.ASMNET1LSNR_ASM.lsnr' (在 'ehs-rac-02' 上) CRS-2676: 成功啟動 'ora.net1.network' (在 'ehs-rac-02' 上) CRS-2672: 嘗試啟動 'ora.ons' (在 'ehs-rac-02' 上) CRS-2676: 成功啟動 'ora.ASMNET1LSNR_ASM.lsnr' (在 'ehs-rac-02' 上) CRS-2672: 嘗試啟動 'ora.asm' (在 'ehs-rac-02' 上) CRS-2676: 成功啟動 'ora.ons' (在 'ehs-rac-02' 上) CRS-2676: 成功啟動 'ora.asm' (在 'ehs-rac-02' 上) CRS-6016: 服務器 ehs-rac-02 的資源自動啟動已完成 CRS-6024: 已完成啟動 Oracle 集群就緒服務管理的資源 CRS-4123: Oracle 高可用性服務已啟動。 2019/09/16 12:51:18 CLSRSC-343: Successfully started Oracle Clusterware stack 2019/09/16 12:51:18 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'. 2019/09/16 12:51:45 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'. 2019/09/16 12:51:58 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded
錯誤:

安裝完成后,查看集群狀態
[grid@ehs-rac-01 ~]$ crsctl stat res -t -------------------------------------------------------------------------------- Name Target State Server State details -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- ora.ASMNET1LSNR_ASM.lsnr ONLINE ONLINE ehs-rac-01 STABLE ONLINE ONLINE ehs-rac-02 STABLE ora.DG_MGMT.dg ONLINE ONLINE ehs-rac-01 STABLE ONLINE ONLINE ehs-rac-02 STABLE ora.DG_OCR.dg ONLINE ONLINE ehs-rac-01 STABLE ONLINE ONLINE ehs-rac-02 STABLE ora.LISTENER.lsnr ONLINE ONLINE ehs-rac-01 STABLE ONLINE ONLINE ehs-rac-02 STABLE ora.chad ONLINE ONLINE ehs-rac-01 STABLE ONLINE ONLINE ehs-rac-02 STABLE ora.net1.network ONLINE ONLINE ehs-rac-01 STABLE ONLINE ONLINE ehs-rac-02 STABLE ora.ons ONLINE ONLINE ehs-rac-01 STABLE ONLINE ONLINE ehs-rac-02 STABLE -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE ehs-rac-01 STABLE ora.MGMTLSNR 1 ONLINE ONLINE ehs-rac-01 169.254.96.70 10.0.0 .110,STABLE ora.asm 1 ONLINE ONLINE ehs-rac-01 Started,STABLE 2 ONLINE ONLINE ehs-rac-02 Started,STABLE 3 OFFLINE OFFLINE STABLE ora.cvu 1 ONLINE ONLINE ehs-rac-01 STABLE ora.ehs-rac-01.vip 1 ONLINE ONLINE ehs-rac-01 STABLE ora.ehs-rac-02.vip 1 ONLINE ONLINE ehs-rac-02 STABLE ora.mgmtdb 1 ONLINE ONLINE ehs-rac-01 Open,STABLE ora.qosmserver 1 ONLINE ONLINE ehs-rac-01 STABLE ora.scan1.vip 1 ONLINE ONLINE ehs-rac-01 STABLE --------------------------------------------------------------------------------
七、配置 ASM(rac1)
[grid@ehs-rac-01 ~]$ asmca
在磁盤組上右鍵選擇創建或點擊下方創建按鈕
八、安裝 ORACLE 軟件(rac1)
解壓 oracle 安裝包(解壓到 oracle 家目錄就可以)
[oracle@ehs-rac-02 ~]$ unzip linuxx64_12201_database.zip [oracle@ehs-rac-01 ~]$ cd database/ [oracle@ehs-rac-01 database]$ ls install response rpm runInstaller sshsetup stage welcome.html [oracle@ehs-rac-01 database]$ ./runInstaller
設置環境變量
[oracle@ehs-rac-01 database]$ export DISPLAY=:0
rac01 執行腳本
[root@ehs-rac-01 rules.d]# /u01/app/oracle/product/12.2.0.1/db_1/root.sh Performing root user operation. The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/app/oracle/product/12.2.0.1/db_1 Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed. No need to overwrite. The contents of "oraenv" have not changed. No need to overwrite. The contents of "coraenv" have not changed. No need to overwrite. Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed.
rac02 執行腳本
[root@ehs-rac-02 db_1]# /u01/app/oracle/product/12.2.0.1/db_1/root.sh Performing root user operation. The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/app/oracle/product/12.2.0.1/db_1 Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed. No need to overwrite. The contents of "oraenv" have not changed. No need to overwrite. The contents of "coraenv" have not changed. No need to overwrite. Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed.
九、安裝數據庫(rac1)
[oracle@ehs-rac-01 database]$ dbca
查看集群狀態
[grid@ehs-rac-01 ~]$ srvctl config database -d orcl 數據庫唯一名稱: orcl 數據庫名: orcl Oracle 主目錄: /u01/app/oracle/product/12.2.0.1/db_1 Oracle 用戶: oracle Spfile: +DG_DATA/ORCL/PARAMETERFILE/spfile.272.1019240865 口令文件: +DG_DATA/ORCL/PASSWORD/pwdorcl.256.1019236783 域: hs 啟動選項: open 停止選項: immediate 數據庫角色: PRIMARY 管理策略: AUTOMATIC 服務器池: 磁盤組: DG_DATA 裝載點路徑: 服務: 類型: RAC 啟動並行: 停止並行: OSDBA 組: dba OSOPER 組: oper 數據庫實例: orcl1,orcl2 已配置的節點: ehs-rac-01,ehs-rac-02 CSS 關鍵型: no CPU 計數: 0 內存目標: 0 最大內存: 0 數據庫服務的默認網絡編號: 數據庫是管理員管理的 [grid@ehs-rac-01 ~]$ srvctl status database -d orcl 實例 orcl1 正在節點 ehs-rac-01 上運行 實例 orcl2 正在節點 ehs-rac-02 上運行