RedHat 7.4下Oracle 19c RAC部署
一、主機環境以及配置
虛擬主機環境
RAC主機名:rac1、rac2
CPU:Intel® Xeon® CPU E5-2670 v2 @ 2.50GHz 4core
內存:32G
操作環境: RHEL 7.4 + Oracle 19c + UDEV
SSD:INTEL P3700 1.6T NVME
存儲:Openfiler
1.1 Oracle 軟硬件要求
官方要求操作系統要在RHEL 7.4及以上
https://docs.oracle.com/en/database/oracle/oracle-database/19/cwlin/supported-red-hat-enterprise-linux-7-distributions-for-x86-64.html#GUID-2E11B561-6587-4789-A583-2E33D705E498
禁用透明大頁
rac需要的最少rpm包
https://docs.oracle.com/en/database/oracle/oracle-database/19/cwlin/supported-red-hat-enterprise-linux-7-distributions-for-x86-64.html#GUID-2E11B561-6587-4789-A583-2E33D705E498
asm磁盤大小規划最小要求EXTERNAL
最少要30GNORMAL
冗余最少要60G
https://docs.oracle.com/en/database/oracle/oracle-database/19/cwlin/oracle-clusterware-storage-space-requirements.html#GUID-97FD5D40-A65B-4575-AD12-06C491AF3F41
1.2 ASM磁盤組規划
ASM磁盤組 | 用途 | 大小 | 冗余 |
---|---|---|---|
CRS_GIMR | ocr、voting file、19c grid管理庫 | 30G+30G+30G | NORMAL |
DATA | 數據文件 | 20G | EXTERNAL |
1.3 主機網絡規划
網絡配置 | 節點1 | 節點2 |
---|---|---|
主機名稱 | rac1 | rac2 |
public ip | 192.168.189.211 | 192.168.189.212 |
private ip | 10.10.10.211 | 10.10.10.212 |
vip | 192.168.189.213 | 192.168.189.214 |
scan ip | 192.168.189.215 | 192.168.189.215 |
1.4 操作系統配置部分
兩個節點主機名設置
hostnamectl set-hostname racnnode1
1、關閉防火牆
在兩個節點關閉防火牆
[root@rac1 ~]# systemctl stop firewalld
[root@rac1 ~]# systemctl disabled firewalld
2、關閉selinux
在兩個節點關閉selinux
[root@rac1 ~]# vi /etc/selinux/config SELINUX=disabled [root@rac1 ~]#setenforce 0
3、hosts文件配置
在兩個節點修改hosts文件
[root@rac1 ~]# vi /etc/hosts #public ip ent0 192.168.189.211 rac1 192.168.189.212 rac2 192.168.189.213 rac1-vip 192.168.189.214 rac2-vip 10.10.10.211 rac1-prv 10.10.10.212 rac2-prv 192.168.189.215 rac-scan
4、依賴包配置
在兩個節點安裝rac需要的依賴包
[root@rac1 ~]# yum install -y binutils yum install -y compat-libcap1 yum install -y compat-libstdc++-33 yum install -y compat-libstdc++-33.i686 yum install -y gcc yum install -y gcc-c++ yum install -y glibc yum install -y glibc.i686 yum install -y glibc-devel yum install -y glibc-devel.i686 yum install -y ksh yum install -y libgcc yum install -y libgcc.i686 yum install -y libstdc++ yum install -y libstdc++.i686 yum install -y libstdc++-devel yum install -y libstdc++-devel.i686 yum install -y libaio yum install -y libaio.i686 yum install -y libaio-devel yum install -y libaio-devel.i686 yum install -y libXext yum install -y libXext.i686 yum install -y libXtst yum install -y libXtst.i686 yum install -y libX11 yum install -y libX11.i686 yum install -y libXau yum install -y libXau.i686 yum install -y libxcb yum install -y libxcb.i686 yum install -y libXi yum install -y libXi.i686 yum install -y make yum install -y sysstat yum install -y unixODBC yum install -y unixODBC-devel yum install -y readline yum install -y libtermcap-devel yum install -y bc yum install -y compat-libstdc++ yum install -y elfutils-libelf yum install -y elfutils-libelf-devel yum install -y fontconfig-devel yum install -y libXi yum install -y libXtst yum install -y libXrender yum install -y libXrender-devel yum install -y libgcc yum install -y librdmacm-devel yum install -y libstdc++ yum install -y libstdc++-devel yum install -y net-tools yum install -y nfs-utils yum install -y python yum install -y python-configshell yum install -y python-rtslib yum install -y python-six yum install -y targetcli yum install -y smartmontools rhel7還需單獨安裝一個獨立包rpm -ivh compat-libstdc++-33-3.2.3-72.el7.x86_64.rpm
5、關閉時間同步ntpd服務
檢查兩節點時間,時區是否相同,並禁止ntp
[root@rac1 ~]#systemctl disable ntpd.service [root@rac1 ~]#systemctl stop ntpd.service [root@rac1 ~]#mv /etc/ntp.conf /etc/ntp.conf.orig [root@rac1 ~]#systemctl status ntpd 時區設置 # timedatectl list-timezones |grep Shanghai #查找中國時區的完整名稱 Asia/Shanghai # timedatectl set-timezone Asia/Shanghai
6、創建用戶及用戶組
在兩個節點創建oracle,grid用戶組
[root@rac1 ~] groupadd -g 54321 oinstall groupadd -g 54322 dba groupadd -g 54323 oper groupadd -g 54324 backupdba groupadd -g 54325 dgdba groupadd -g 54326 kmdba groupadd -g 54327 asmdba groupadd -g 54328 asmoper groupadd -g 54329 asmadmin groupadd -g 54330 racdba useradd -u 54321 -g oinstall -G dba,asmdba,backupdba,dgdba,kmdba,racdba,oper oracle useradd -u 54322 -g oinstall -G asmadmin,asmdba,asmoper,dba grid
設置grid、oracle密碼
[root@rac1 ~] echo "oracle" | passwd --stdin oracle echo "oracle" | passwd --stdin grid
7、創建文件目錄
在兩個節點創建grid、oracle文件目錄
[root@rac1 ~] mkdir -p /u01/app/19.0.0/grid mkdir -p /u01/app/grid mkdir -p /u01/app/oracle mkdir -p /u01/app/oracle/product/19.0.0/db_1 chown -R grid:oinstall /u01 chown -R oracle:oinstall /u01/app/oracle chmod -R 775 /u01/
8、修改20-nproc.conf
在兩個節點修改/etc/security/limits.d/20-nproc.conf
[root@rac1 ~]vi /etc/security/limits.d/20-nproc.conf # Change this #注釋這行* soft nproc 1024 # To this增加下面一行 * - nproc 16384
9、修改limits.conf
在兩個節點修改/etc/security/limits.conf
[root@rac1 ~]vi /etc/security/limits.conf #ORACLE SETTING grid soft nproc 16384 grid hard nproc 16384 grid soft nofile 1024 grid hard nofile 65536 grid soft stack 10240 grid hard stack 32768 oracle soft nproc 16384 oracle hard nproc 16384 oracle soft nofile 1024 oracle hard nofile 65536 oracle soft stack 10240 oracle hard stack 32768 oracle hard memlock 3145728 oracle soft memlock 3145728
10、修改login
在兩個節點修改/etc/pam.d/login文件
[root@rac1 ~]vi /etc/pam.d/login #ORACLE SETTING session required pam_limits.so
11、修改sysctl.conf
在兩個節點修改/etc/sysctl.conf
[root@rac1 ~]vi /etc/sysctl.conf #ORACLE SETTING fs.aio-max-nr = 1048576 fs.file-max = 6815744 kernel.shmmax = 15461882265 kernel.shmall = 3774873 kernel.shmmni = 4096 kernel.sem = 250 32000 100 128 net.ipv4.ip_local_port_range = 9000 65500 net.core.rmem_default = 262144 net.core.rmem_max = 4194304 net.core.wmem_default = 262144 net.core.wmem_max = 1048586
12、停止avahi-daemon服務
兩個節點分別操作
[root@rac1 ~]# systemctl disable avahi-daemon.socket [root@rac1 ~]# systemctl disable avahi-daemon.service ps -ef|grep avahi-daemon kill -9 pid avahi-daemon
13、添加NOZEROCONF=yes參數
[root@rac1 rpm]# vi /etc/sysconfig/network NOZEROCONF=yes
14、修改grid用戶環境變量
節點1 grid
[root@rac1 ~]# su - grid [grid@rac1:/home/grid]$vi ~/.bash_profile PS1="[`whoami`@`hostname`:"'$PWD]$' export PS1 umask 022 export TMP=/tmp export LANG=en_US export TMPDIR=$TMP ORACLE_SID=+ASM1; export ORACLE_SID ORACLE_TERM=xterm; export ORACLE_TERM ORACLE_BASE=/u01/app/grid; export ORACLE_BASE ORACLE_HOME=/u01/app/19.0.0/grid; export ORACLE_HOME NLS_DATE_FORMAT="yyyy-mm-dd HH24:MI:SS"; export NLS_DATE_FORMAT PATH=.:$PATH:$HOME/bin:$ORACLE_HOME/bin; export PATH export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib THREADS_FLAG=native; export THREADS_FLAG if [ $USER = "oracle" ] || [ $USER = "grid" ]; then if [ $SHELL = "/bin/ksh" ]; then ulimit -p 16384 ulimit -n 65536 else ulimit -u 16384 -n 65536 fi umask 022 fi
節點2 grid
[root@rac2 ~]# su - grid [grid@rac2:/home/grid]$vi ~/.bash_profile PS1="[`whoami`@`hostname`:"'$PWD]$' export PS1 umask 022 export TMP=/tmp export LANG=en_US export TMPDIR=$TMP ORACLE_SID=+ASM2; export ORACLE_SID ORACLE_TERM=xterm; export ORACLE_TERM ORACLE_BASE=/u01/app/grid; export ORACLE_BASE ORACLE_HOME=/u01/app/19.0.0/grid; export ORACLE_HOME NLS_DATE_FORMAT="yyyy-mm-dd HH24:MI:SS"; export NLS_DATE_FORMAT PATH=.:$PATH:$HOME/bin:$ORACLE_HOME/bin; export PATH export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib THREADS_FLAG=native; export THREADS_FLAG if [ $USER = "oracle" ] || [ $USER = "grid" ]; then if [ $SHELL = "/bin/ksh" ]; then ulimit -p 16384 ulimit -n 65536 else ulimit -u 16384 -n 65536 fi umask 022 fi
15、修改oracle用戶環境變量
節點1 oracle
[root@rac1 ~]# su - oracle [oracle@rac1:/home/oracle]$vi ~/.bash_profile PS1="[`whoami`@`hostname`:"'$PWD]$' export PS1 export TMP=/tmp export LANG=en_US export TMPDIR=$TMP ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE ORACLE_HOME=$ORACLE_BASE/product/19.0.0/db_1; export ORACLE_HOME ORACLE_SID=orcl1; export ORACLE_SID ORACLE_TERM=xterm; export ORACLE_TERM NLS_DATE_FORMAT="yyyy-mm-dd HH24:MI:SS"; export NLS_DATE_FORMAT NLS_LANG=AMERICAN_AMERICA.ZHS16GBK;export NLS_LANG PATH=.:$PATH:$HOME/bin:$ORACLE_BASE/product/19.0.0/db_1/bin:$ORACLE_HOME/bin; export PATH export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib THREADS_FLAG=native; export THREADS_FLAG if [ $USER = "oracle" ] || [ $USER = "grid" ]; then if [ $SHELL = "/bin/ksh" ]; then ulimit -p 16384 ulimit -n 65536 else ulimit -u 16384 -n 65536 fi umask 022 fi
節點2 oracle
[root@rac2 ~]# su - oracle [oracle@rac2:/home/oracle]$vi ~/.bash_profile PS1="[`whoami`@`hostname`:"'$PWD]$' export PS1 export TMP=/tmp export LANG=en_US export TMPDIR=$TMP ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE ORACLE_HOME=$ORACLE_BASE/product/19.0.0/db_1; export ORACLE_HOME ORACLE_SID=orcl2; export ORACLE_SID ORACLE_TERM=xterm; export ORACLE_TERM NLS_DATE_FORMAT="yyyy-mm-dd HH24:MI:SS"; export NLS_DATE_FORMAT NLS_LANG=AMERICAN_AMERICA.ZHS16GBK;export NLS_LANG PATH=.:$PATH:$HOME/bin:$ORACLE_BASE/product/19.0.0/db_1/bin:$ORACLE_HOME/bin; export PATH export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib THREADS_FLAG=native; export THREADS_FLAG if [ $USER = "oracle" ] || [ $USER = "grid" ]; then if [ $SHELL = "/bin/ksh" ]; then ulimit -p 16384 ulimit -n 65536 else ulimit -u 16384 -n 65536 fi umask 022 fi
16、修改root環境變量
加入grid用戶$ORACLE_HOME
[root@rac1 opt]# vi ~/.bash_profile # .bash_profile # Get the aliases and functions if [ -f ~/.bashrc ]; then . ~/.bashrc fi # User specific environment and startup programs PATH=$PATH:/u01/app/19.0.0/grid/bin:$HOME/bin export PATH
17、禁用透明大頁
oracle官方建議關閉透明大頁
1 修改grub文件
cp /etc/default/grub /etc/default/grub.bak vi /etc/default/grub 2 增加一行transparent_hugepage=never到尾部 GRUB_CMDLINE_LINUX= "rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet transparent_hugepage=never 3 執行命令 grub2-mkconfig -o /boot/grub2/grub.cfg 4 不重啟生效 [root@rac1 ~]#echo never > /sys/kernel/mm/transparent_hugepage/enabled 5 查看是否禁用透明大頁 [root@rac1 ~]# cat /sys/kernel/mm/transparent_hugepage/enabled always madvise [never] [root@rac2 ~]# grep AnonHugePages /proc/meminfo AnonHugePages: 0 kB---------->>>返回值若是零,代表成功禁用THP
18、修改共享存儲、udev規則
在節點1和節點2,固定設備名稱並且賦予grid屬性,執行如下命令,用來生成udev規則
1 取得DM_UUID
cd /dev/mapper for i in `ls mpath*`; do printf "%s %s\n" "$i" "$(udevadm info --query=all --name=/dev/mapper/$i |grep -i dm_uuid)"; done
2 生成rules文件
vi /etc/udev/rules.d/99-oracle-asmdevices.rules
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-14f504e46494c455232776b726c352d4557426d2d43776568",SYMLINK+="asm_crs_1_30g",OWNER="grid",GROUP="asmadmin",MODE="0660" KERNEL=="dm-*",ENV{DM_UUID}=="mpath-14f504e46494c4552477a424f6e592d644b49572d56614c4b",SYMLINK+="asm_crs_2_30g",OWNER="grid",GROUP="asmadmin",MODE="0660" KERNEL=="dm-*",ENV{DM_UUID}=="mpath-14f504e46494c45524e5871614c522d74376d482d5a564272",SYMLINK+="asm_crs_3_30g",OWNER="grid",GROUP="asmadmin",MODE="0660" KERNEL=="dm-*",ENV{DM_UUID}=="mpath-14f504e46494c4552386e674a70452d4959334f2d47745667",SYMLINK+="asm_data_1_20g",OWNER="grid",GROUP="asmadmin",MODE="0660"
3重新加載udev
udevadm control --reload-rules
udevadm trigger
4查看asm磁盤
[root@rac1 ~]# ll /dev/asm*
19、節點2拷貝節點1的udev規則文件
scp rac1:/etc/udev/rules.d/99-oracle-asmdevices.rules /etc/udev/rules.d/99-oracle-asmdevices.rules
二、Grid集群軟件安裝部分
1、上傳集群軟件包
在節點1上傳grid安裝包
[root@rac1 grid]# ll -rwxr-xr-x 1 grid oinstall 5.1G Jan 28 15:58 LINUX.X64_180000_grid_home.zip
2、解壓grid安裝包
在19C中需要把grid包解壓放到grid用戶下ORACLE_HOME目錄內
解壓文件到/u01/app/19.0.0/grid
[grid@rac2 grid]$ cd /u01/app/19.0.0/grid [grid@rac2 grid]$ unzip LINUX.X64_193000_grid_home.zip
3、進入grid集群軟件目錄執行安裝
[grid@rac1 grid]$ ./gridSetup.sh
4、GUI安裝步驟
4.1 創建新的集群
4.2 配置集群名稱以及scan名稱
4.3 節點互信
4.4 公網、私網網段選擇
4.5 選擇asm存儲
4.6 選擇配置GIMR
4.7 這里選擇ocr、voting file與gimr放在一起
4.8 選擇asm磁盤組
4.9 輸入密碼
4.10 保持默認
4.11 保持默認
4.12保持默認
4.13 確認base目錄
4.14保持默認
4.15這里可以選擇自動root執行腳本
4.16 預安裝檢查
4.17 解決相關依賴后,忽略如下報錯
4.18 安裝前摘要
4.19 如下警告可以忽略
如上警告是由於沒有使用DNS解析造成可忽略
4.20 grid安裝完畢
三、Oracle DataBase軟件安裝
解壓壓縮包到oracle用戶的ORACLE_HOME目錄,執行安裝
[oracle@rac1 db_1]$ pwd /u01/app/oracle/product/19.0.0/db_1 [oracle@rac1 db_1]$ unzip /opt/LINUX.X64_193000_db_home.zip [oracle@rac1 db_1]$ ./runInstaller
- 1
- 2
- 3
- 4
- 5
1 software only
2 選擇rac
3 ssh互信驗證
4 保持默認
5 保持默認
6 保持默認
7 可選擇自動執行root腳本
8 預安裝前檢查
忽略如下警告
9 安裝摘要
10 開始安裝
11 執行root腳本
12 oracle軟件安裝部分結束
四、DBCA創建實例
在創建實例前先通過grid的asmca創建磁盤組,隨后再用oracle dbca創建實例
1、通過asmca創建oracle數據文件磁盤組
[grid@rac1 grid]$ asmca
data磁盤組
asm磁盤組創建完畢
2、dbca圖形化安裝實例
[oracle@rac1 db_1]$ dbca
常規配置DBCA
DBCA完成
查看集群狀態
[root@rac1 ~]# crsctl stat res -t -------------------------------------------------------------------------------- Name Target State Server State details -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- ora.CRS_GIMR.GHCHKPT.advm OFFLINE OFFLINE rac1 STABLE OFFLINE OFFLINE rac2 STABLE ora.LISTENER.lsnr ONLINE ONLINE rac1 STABLE ONLINE ONLINE rac2 STABLE ora.chad ONLINE ONLINE rac1 STABLE ONLINE ONLINE rac2 STABLE ora.crs_gimr.ghchkpt.acfs OFFLINE OFFLINE rac1 STABLE OFFLINE OFFLINE rac2 STABLE ora.helper OFFLINE OFFLINE rac1 IDLE,STABLE OFFLINE OFFLINE rac2 IDLE,STABLE ora.net1.network ONLINE ONLINE rac1 STABLE ONLINE ONLINE rac2 STABLE ora.ons ONLINE ONLINE rac1 STABLE ONLINE ONLINE rac2 STABLE ora.proxy_advm OFFLINE OFFLINE rac1 STABLE OFFLINE OFFLINE rac2 STABLE -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup) 1 ONLINE ONLINE rac1 STABLE 2 ONLINE ONLINE rac2 STABLE 3 ONLINE OFFLINE STABLE ora.CRS_GIMR.dg(ora.asmgroup) 1 ONLINE ONLINE rac1 STABLE 2 ONLINE ONLINE rac2 STABLE 3 OFFLINE OFFLINE STABLE ora.DATA.dg(ora.asmgroup) 1 ONLINE ONLINE rac1 STABLE 2 ONLINE ONLINE rac2 STABLE 3 OFFLINE OFFLINE STABLE ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE rac1 STABLE ora.MGMTLSNR 1 ONLINE ONLINE rac1 169.254.11.15 10.10. 10.211,STABLE ora.asm(ora.asmgroup) 1 ONLINE ONLINE rac1 Started,STABLE 2 ONLINE ONLINE rac2 Started,STABLE 3 OFFLINE OFFLINE STABLE ora.asmnet1.asmnetwork(ora.asmgroup) 1 ONLINE ONLINE rac1 STABLE 2 ONLINE ONLINE rac2 STABLE 3 OFFLINE OFFLINE STABLE ora.cvu 1 ONLINE ONLINE rac1 STABLE ora.mgmtdb 1 ONLINE ONLINE rac1 Open,STABLE ora.orcl.db 1 ONLINE ONLINE rac1 Open,HOME=/u01/app/o racle/product/19.0.0 /db_1,STABLE 2 ONLINE ONLINE rac2 Open,HOME=/u01/app/o racle/product/19.0.0 /db_1,STABLE ora.qosmserver 1 ONLINE ONLINE rac1 STABLE ora.rac1.vip 1 ONLINE ONLINE rac1 STABLE ora.rac2.vip 1 ONLINE ONLINE rac2 STABLE ora.rhpserver 1 OFFLINE OFFLINE STABLE ora.scan1.vip 1 ONLINE ONLINE rac1 STABLE --------------------------------------------------------------------------------
查看數據庫版本
[oracle@rac1 db_1]$ sqlplus / as sysdba SQL> col banner_full for a120 SQL> select BANNER_FULL from v$version; BANNER_FULL -------------------------------------------------------------------------------- Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production