Oracle 19c RAC 升級RU 從19.3 升級到19.6 操作手冊（使用nonrolling 方式）
2020-03-14 16:09 935
0
原創 Oracle 19c
在之前的博客我們了解了Oracle 19c RAC的安裝，以及單實例的RU升級，如下：
Linux 7.6 平台 Oracle 19c(19.3) RAC 安裝手冊 詳細截圖版
https://www.cndba.cn/dave/article/4073
Oracle 19c 單實例 RU 從19.3.0 升級到19.6.0 操作手冊
https://www.cndba.cn/dave/article/4065
Linux 7.7 平台 Oracle 19c Data Guard 環境 升級RU 19.3 到19.6 操作手冊
https://www.cndba.cn/dave/article/4068
Oracle 12.2 家族的補丁 RU 和 RUR 說明
https://www.cndba.cn/dave/article/4063
本篇我們看下19c RAC 平台如何升級RU.
1升級說明
從MOS上我們可以看到19.6的RU 有GI和DB 2個版本，但從GI Patch的readme上看，GI 的Patch 已經包含了DB HOME和GI HOME的補丁，只需要用root用戶執行opatchauto命令打GI的補丁即可。
根據官方手冊的說明，對於RU的升級，可以采用滾動升級的方式進行。依次在每個節點執行，不能同時在2個節點執行。 但是實際操作有問題，目前可行的操作是采用nonrolling的方式升級RU。
2具體升級過程
2.1 當前環境
[grid@www.cndba.cn_1 ~]$ crsctl query crs softwareversion
Oracle Clusterware version on node [rac1] is [19.0.0.0.0]
[grid@www.cndba.cn_1 ~]$ crsctl query crs releaseversion
Oracle High Availability Services release version on the local node is [19.0.0.0.0]
[grid@www.cndba.cn_1 ~]$ crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
[grid@www.cndba.cn_1 ~]$
[grid@www.cndba.cn_1 OPatch]$ ./opatch lspatches
29585399;OCW RELEASE UPDATE 19.3.0.0.0 (29585399)
29517247;ACFS RELEASE UPDATE 19.3.0.0.0 (29517247)
29517242;Database Release Update : 19.3.0.0.190416 (29517242)
29401763;TOMCAT RELEASE UPDATE 19.0.0.0.0 (29401763)
OPatch succeeded.
[grid@www.cndba.cn_1 OPatch]$
2.2 檢查OPatch工具版本
安裝19.6 的RU，OPatch的版本必須大於12.2.0.1.17. 最新版本的Opatch可以從MOS 6880880上下載。
[grid@www.cndba.cn_1 ~]$ cd $ORACLE_HOME/OPatch
[grid@www.cndba.cn_1 OPatch]$ ./opatch version
OPatch Version: 12.2.0.1.17
OPatch succeeded.
[grid@www.cndba.cn_1 OPatch]$
因為opatch低版本打補丁的時候有bug，所以我這里直接將OPatch 升級到最新的12.2.0.1.19。
[oracle@www.cndba.cn_2 OPatch]$ ./opatch version
OPatch Version: 12.2.0.1.19
OPatch succeeded.
[oracle@www.cndba.cn_2 OPatch]$
12c opatchauto: Prerequisite check “CheckApplicable” failed (Doc ID 1937982.1)
OPatch升級可以參考我的博客：
Oracle 更新 OPatch 工具版本 的方法 說明
https://www.cndba.cn/cndba/dave/article/1353
2.3 驗證Oracle Inventory的有效性
GI HOME 和DB HOME 都需要驗證，分別使用grid和oracle用戶執行如下命令，確保返回SUCCESS：
$ /OPatch/opatch lsinventory -detail -oh
GRID 用戶：
[grid@www.cndba.cn_1 OPatch]$ $ORACLE_HOME/OPatch/opatch lsinventory -detail -oh $ORACLE_HOME
Oracle Interim Patch Installer version 12.2.0.1.17
Copyright (c) 2020, Oracle Corporation. All rights reserved.
Oracle Home : /u01/app/19.3.0/grid
Central Inventory : /u01/app/oraInventory
from : /u01/app/19.3.0/grid/oraInst.loc
OPatch version : 12.2.0.1.17
OUI version : 12.2.0.7.0
Log file location : /u01/app/19.3.0/grid/cfgtoollogs/opatch/opatch2020-03-12_20-29-23PM_1.log
Lsinventory Output file location : /u01/app/19.3.0/grid/cfgtoollogs/opatch/lsinv/lsinventory2020-03-12_20-29-23PM.txt
--------------------------------------------------------------------------------
Local Machine Information::
Hostname: rac1
ARU platform id: 226
ARU platform description:: Linux x86-64
Installed Top-level Products (1):
Oracle Grid Infrastructure 19c 19.0.0.0.0
There are 1 products installed in this Oracle Home.
Installed Products (102):
……
OPatch succeeded.
[grid@www.cndba.cn_1 OPatch]$
Oracle 用戶：
[oracle@www.cndba.cn_1 ~]$ $ORACLE_HOME/OPatch/opatch lsinventory -detail -oh $ORACLE_HOME
Oracle Interim Patch Installer version 12.2.0.1.17
Copyright (c) 2020, Oracle Corporation. All rights reserved.
Oracle Home : /u01/app/oracle/product/19.3.0/db_1
Central Inventory : /u01/app/oraInventory
from : /u01/app/oracle/product/19.3.0/db_1/oraInst.loc
OPatch version : 12.2.0.1.17
OUI version : 12.2.0.7.0
Log file location : /u01/app/oracle/product/19.3.0/db_1/cfgtoollogs/opatch/opatch2020-03-12_20-31-13PM_1.log
Lsinventory Output file location : /u01/app/oracle/product/19.3.0/db_1/cfgtoollogs/opatch/lsinv/lsinventory2020-03-12_20-31-13PM.txt
--------------------------------------------------------------------------------
Local Machine Information::
Hostname: rac1
ARU platform id: 226
ARU platform description:: Linux x86-64
Installed Top-level Products (1):
Oracle Database 19c 19.0.0.0.0
There are 1 products installed in this Oracle Home.
Installed Products (128):
……
Patch Location in Inventory:
/u01/app/oracle/product/19.3.0/db_1/inventory/oneoffs/29517242
Patch Location in Storage area:
/u01/app/oracle/product/19.3.0/db_1/.patch_storage/29517242_Apr_17_2019_23_27_10
--------------------------------------------------------------------------------
OPatch succeeded.
[oracle@www.cndba.cn_1 ~]$
2.4 下載19.6 的RU
直接從MOS下載。
Quick Reference to Patch Numbers for Database/GI PSU, SPU(CPU), Bundle Patches and Patchsets (文檔 ID 1454618.1)
[root@www.cndba.cn_1 30501910]# ll
total 132
drwxr-x---. 5 oracle oinstall 62 Jan 7 13:25 30489227
drwxr-x---. 5 oracle oinstall 62 Jan 7 13:26 30489632
drwxr-x---. 5 oracle oinstall 81 Jan 7 13:22 30557433
drwxr-x---. 4 oracle oinstall 48 Jan 7 13:26 30655595
drwxr-x---. 2 oracle oinstall 4096 Jan 7 13:25 automation
-rw-rw-r--. 1 oracle oinstall 5054 Jan 7 21:43 bundle.xml
-rw-r--r--. 1 oracle oinstall 122266 Jan 7 21:32 README.html
-rw-r--r--. 1 oracle oinstall 0 Jan 7 13:25 README.txt
[root@www.cndba.cn_1 30501910]#
2.5 檢查Patch 是否沖突
19.6 的GI RU里包含GI和DB的補丁，需要分別進行檢測，在檢測之前需要先修改權限，否則會報錯。
GRID 用戶執行：
chown grid:oinstall /u01/software/30501910 -R
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/software/30501910/30557433
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/software/30501910/30489227
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/software/30501910/30489632
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/software/30501910/
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/software/30501910/30655595
[grid@www.cndba.cn_1 ~]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/software/30501910/30655595
Oracle Interim Patch Installer version 12.2.0.1.17
Copyright (c) 2020, Oracle Corporation. All rights reserved.
PREREQ session
Oracle Home : /u01/app/19.3.0/grid
Central Inventory : /u01/app/oraInventory
from : /u01/app/19.3.0/grid/oraInst.loc
OPatch version : 12.2.0.1.17
OUI version : 12.2.0.7.0
Log file location : /u01/app/19.3.0/grid/cfgtoollogs/opatch/opatch2020-03-12_20-43-04PM_1.log
Invoking prereq "checkconflictagainstohwithdetail"
Prereq "checkConflictAgainstOHWithDetail" passed.
OPatch succeeded.
Oracle 用戶執行：
chown oracle:oinstall /u01/software/30501910 -R
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/software/30501910/30557433
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/software/30501910/30489227
[oracle@www.cndba.cn_1 ~]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/software/30501910/30489227
Oracle Interim Patch Installer version 12.2.0.1.17
Copyright (c) 2020, Oracle Corporation. All rights reserved.
PREREQ session
Oracle Home : /u01/app/oracle/product/19.3.0/db_1
Central Inventory : /u01/app/oraInventory
from : /u01/app/oracle/product/19.3.0/db_1/oraInst.loc
OPatch version : 12.2.0.1.17
OUI version : 12.2.0.7.0
Log file location : /u01/app/oracle/product/19.3.0/db_1/cfgtoollogs/opatch/opatch2020-03-12_20-43-37PM_1.log
Invoking prereq "checkconflictagainstohwithdetail"
Prereq "checkConflictAgainstOHWithDetail" passed.
OPatch succeeded.
2.6 檢查系統空間
在打Patch之前需要先確保ORACLE_HOME 所在的文件系統有足夠的空閑空間，可以執行如下命令來檢查。
GRID用戶操作：
創建/tmp/patch_list_gihome.txt 文件，並添加如下內容：
[grid@www.cndba.cn_1 ~]$ cat /tmp/patch_list_gihome.txt
/u01/software/30501910/30557433
/u01/software/30501910/30489227
/u01/software/30501910/30489632
/u01/software/30501910/30655595
[grid@www.cndba.cn_1 ~]$
注意這里內容與官方有所不同，按readme文檔會報如下錯誤：
Oracle OPatch 執行 返回 This command doesn’t support System Patch. 解決方法
https://www.cndba.cn/dave/article/4074
運行opatch命令檢查GI HOME下是否有足夠的空間：
[root@www.cndba.cn_1 software]# chown grid:oinstall /u01/software/30501910 -R
[grid@www.cndba.cn_1 ~]$ $ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /tmp/patch_list_gihome.txt
Oracle Interim Patch Installer version 12.2.0.1.17
Copyright (c) 2020, Oracle Corporation. All rights reserved.
PREREQ session
Oracle Home : /u01/app/19.3.0/grid
Central Inventory : /u01/app/oraInventory
from : /u01/app/19.3.0/grid/oraInst.loc
OPatch version : 12.2.0.1.17
OUI version : 12.2.0.7.0
Log file location : /u01/app/19.3.0/grid/cfgtoollogs/opatch/opatch2020-03-12_21-54-25PM_1.log
Invoking prereq "checksystemspace"
Prereq "checkSystemSpace" passed.
OPatch succeeded.
[grid@www.cndba.cn_1 ~]$
ORACLE 用戶操作：
創建文件/tmp/patch_list_dbhome.txt並添加如下內容：
[oracle@www.cndba.cn_1 ~]$ cat /tmp/patch_list_dbhome.txt
/u01/software/30501910/30557433
/u01/software/30501910/30489227
運行opatch命令檢查DB HOME下是否有足夠的空間：
[root@www.cndba.cn_1 software]# chown oracle:oinstall /u01/software/30501910 -R
[oracle@www.cndba.cn_1 ~]$ $ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /tmp/patch_list_dbhome.txt
Oracle Interim Patch Installer version 12.2.0.1.17
Copyright (c) 2020, Oracle Corporation. All rights reserved.
PREREQ session
Oracle Home : /u01/app/oracle/product/19.3.0/db_1
Central Inventory : /u01/app/oraInventory
from : /u01/app/oracle/product/19.3.0/db_1/oraInst.loc
OPatch version : 12.2.0.1.17
OUI version : 12.2.0.7.0
Log file location : /u01/app/oracle/product/19.3.0/db_1/cfgtoollogs/opatch/opatch2020-03-12_20-58-02PM_1.log
Invoking prereq "checksystemspace"
Prereq "checkSystemSpace" passed.
OPatch succeeded.
[oracle@www.cndba.cn_1 ~]$
Oracle 用戶執行正常。
2.7 安裝前分析
在每個節點執行analyze：
/u01/app/19.0.0/grid/OPatch/opatchauto apply /u01/app/grid/soft/30501910 -analyze
“-analyze”選項可以模擬OPatchauto apply，提前檢查所有檢查項目，但是運行”-analyze”選項不會真正改變系統。
[root@www.cndba.cn_2 software]# /u01/app/19.3.0/grid/OPatch/opatchauto apply /tmp/30501910 -analyze
OPatchauto session is initiated at Fri Mar 13 14:20:49 2020
System initialization log file is /u01/app/19.3.0/grid/cfgtoollogs/opatchautodb/systemconfig2020-03-13_02-21-06PM.log.
Session log file is /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/opatchauto2020-03-13_02-23-32PM.log
The id for this session is AT3D
Executing OPatch prereq operations to verify patch applicability on home /u01/app/19.3.0/grid
Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/19.3.0/db_1
Patch applicability verified successfully on home /u01/app/oracle/product/19.3.0/db_1
Patch applicability verified successfully on home /u01/app/19.3.0/grid
Verifying SQL patch applicability on home /u01/app/oracle/product/19.3.0/db_1
SQL patch applicability verified successfully on home /u01/app/oracle/product/19.3.0/db_1
OPatchAuto successful.
--------------------------------Summary--------------------------------
Analysis for applying patches has completed successfully:
Host:rac2
RAC Home:/u01/app/oracle/product/19.3.0/db_1
Version:19.0.0.0.0
==Following patches were SKIPPED:
Patch: /tmp/30501910/30489632
Reason: This patch is not applicable to this specified target type - "rac_database"
Patch: /tmp/30501910/30655595
Reason: This patch is not applicable to this specified target type - "rac_database"
==Following patches were SUCCESSFULLY analyzed to be applied:
Patch: /tmp/30501910/30489227
Log: /u01/app/oracle/product/19.3.0/db_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-03-13_14-24-10PM_1.log
Patch: /tmp/30501910/30557433
Log: /u01/app/oracle/product/19.3.0/db_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-03-13_14-24-10PM_1.log
Host:rac2
CRS Home:/u01/app/19.3.0/grid
Version:19.0.0.0.0
==Following patches were SUCCESSFULLY analyzed to be applied:
Patch: /tmp/30501910/30489227
Log: /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-03-13_14-24-10PM_1.log
Patch: /tmp/30501910/30489632
Log: /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-03-13_14-24-10PM_1.log
Patch: /tmp/30501910/30655595
Log: /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-03-13_14-24-10PM_1.log
Patch: /tmp/30501910/30557433
Log: /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-03-13_14-24-10PM_1.log
OPatchauto session completed at Fri Mar 13 14:30:24 2020
Time taken to complete the session 9 minutes, 36 seconds
[root@www.cndba.cn_2 software]#
2.8 執行opatchauto 工具應用RU
在DB HOME的版本和GI HOME的版本一致的情況下，可以使用root用戶執行opatchauto命令一次對GI 和 DB HOME進行Patch操作。如果GI 和 DB Home 版本不同，可以使用opatchauto 分別進行patch動作。
注意一點： 這里的RU補丁，一定要用GRID用戶來解壓縮，否則執行opatchauto的時候回報OPatch Prerequisite check “CheckApplicable” failed的錯誤。
其實是根據readme文檔的說明，可以一次性打GI 和DB 的RU，但按文檔操作，出現了很多問題。 一直沒徹底解決的就是：
[Mar 13, 2020 3:54:58 PM] [SEVERE] OUI-67073:UtilSession failed: ApplySession failed in system modification phase... 'ApplySession::apply failed: Copy failed from '/tmp/30501910/30489227/files/bin/crsd.bin' to '/u01/app/19.3.0/grid/bin/crsd.bin'...
Copy failed from '/tmp/30501910/30489227/files/bin/cssdagent' to '/u01/app/19.3.0/grid/bin/cssdagent'...
直接一次性打GI 和 DB HOME 中遇到的問題，參考我的博客：
Oracle 19c RAC 環境升級 19.6 RU OPatch Prerequisite check “CheckApplicable” failed 解決方法
https://www.cndba.cn/dave/article/4081
所以，對於RU的升級，建議按照nonrolling的方式進行，雖然要停機，但可以保證升級的成功。
對GI HOME 進行patch：
opatchauto apply /28660077 -oh -nonrolling
對一個或者多個DB HOME 進行patch：
opatchauto apply /28660077 -oh , -nonrolling
2.8.1打GI 的RU
先在節點1執行：
[root@www.cndba.cn_1 software]# export PATH=$PATH:/u01/app/19.3.0/grid/OPatch
[root@www.cndba.cn_1 software]# chown grid:oinstall 30501910 -R
[root@www.cndba.cn_1 software]# chmod 777 30501910 -R
[root@www.cndba.cn_1 software]# opatchauto apply /tmp/30501910 -oh /u01/app/19.3.0/grid -nonrolling
[root@www.cndba.cn_1 lib]# opatchauto apply /tmp/30501910 -oh /u01/app/19.3.0/grid -nonrolling
OPatchauto session is initiated at Fri Mar 13 20:37:31 2020
System initialization log file is /u01/app/19.3.0/grid/cfgtoollogs/opatchautodb/systemconfig2020-03-13_08-37-39PM.log.
Session log file is /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/opatchauto2020-03-13_08-37-48PM.log
The id for this session is AFS1
Executing OPatch prereq operations to verify patch applicability on home /u01/app/19.3.0/grid
Patch applicability verified successfully on home /u01/app/19.3.0/grid
OPatchAuto successful.
--------------------------------Summary--------------------------------
Patching is completed successfully. Please find the summary as follows:
Host:rac1
CRS Home:/u01/app/19.3.0/grid
Version:19.0.0.0.0
Summary:
==Following patches were SKIPPED:
Patch: /tmp/30501910/30489227
Reason: This patch is already been applied, so not going to apply again.
Patch: /tmp/30501910/30489632
Reason: This patch is already been applied, so not going to apply again.
Patch: /tmp/30501910/30655595
Reason: This patch is already been applied, so not going to apply again.
Patch: /tmp/30501910/30557433
Reason: This patch is already been applied, so not going to apply again.
OPatchauto session completed at Fri Mar 13 20:38:10 2020
Time taken to complete the session 0 minute, 40 seconds
[root@www.cndba.cn_1 lib]#
節點2執行：
[root@www.cndba.cn_2 cfgtoollogs]# crsctl status res -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
ONLINE ONLINE rac2 STABLE
ora.MGMT.GHCHKPT.advm
OFFLINE OFFLINE rac2 STABLE
ora.chad
ONLINE ONLINE rac2 STABLE
ora.helper
OFFLINE OFFLINE rac2 IDLE,STABLE
ora.mgmt.ghchkpt.acfs
OFFLINE OFFLINE rac2 STABLE
ora.net1.network
ONLINE ONLINE rac2 STABLE
ora.ons
ONLINE ONLINE rac2 STABLE
ora.proxy_advm
OFFLINE OFFLINE rac2 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
1 ONLINE OFFLINE STABLE
2 ONLINE ONLINE rac2 STABLE
3 ONLINE OFFLINE STABLE
ora.DATA.dg(ora.asmgroup)
1 OFFLINE OFFLINE STABLE
2 ONLINE ONLINE rac2 STABLE
3 OFFLINE OFFLINE STABLE
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE rac2 STABLE
ora.MGMT.dg(ora.asmgroup)
1 OFFLINE OFFLINE STABLE
2 ONLINE ONLINE rac2 STABLE
3 OFFLINE OFFLINE STABLE
ora.MGMTLSNR
1 ONLINE ONLINE rac2 169.254.21.159 192.1
68.222.181,STABLE
ora.OCR.dg(ora.asmgroup)
1 OFFLINE OFFLINE STABLE
2 ONLINE ONLINE rac2 STABLE
3 OFFLINE OFFLINE STABLE
ora.asm(ora.asmgroup)
1 ONLINE OFFLINE STABLE
2 ONLINE ONLINE rac2 Started,STABLE
3 OFFLINE OFFLINE STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
1 ONLINE OFFLINE STABLE
2 ONLINE ONLINE rac2 STABLE
3 OFFLINE OFFLINE STABLE
ora.cndba.db
1 ONLINE OFFLINE STABLE
2 ONLINE OFFLINE rac2 Instance Shutdown,ST
ARTING
ora.cvu
1 ONLINE ONLINE rac2 STABLE
ora.mgmtdb
1 ONLINE OFFLINE rac2 Instance Shutdown,ST
ARTING
ora.qosmserver
1 ONLINE ONLINE rac2 STABLE
ora.rac1.vip
1 ONLINE INTERMEDIATE rac2 FAILED OVER,STABLE
ora.rac2.vip
1 ONLINE ONLINE rac2 STABLE
ora.rhpserver
1 OFFLINE OFFLINE STABLE
ora.scan1.vip
1 ONLINE ONLINE rac2 STABLE
--------------------------------------------------------------------------------
[root@www.cndba.cn_2 cfgtoollogs]# opatchauto apply /tmp/30501910 -oh /u01/app/19.3.0/grid -nonrolling
OPatchauto session is initiated at Fri Mar 13 21:32:49 2020
System initialization log file is /u01/app/19.3.0/grid/cfgtoollogs/opatchautodb/systemconfig2020-03-13_09-32-56PM.log.
Session log file is /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/opatchauto2020-03-13_09-33-42PM.log
The id for this session is I2XJ
Executing OPatch prereq operations to verify patch applicability on home /u01/app/19.3.0/grid
Patch applicability verified successfully on home /u01/app/19.3.0/grid
Bringing down CRS service on home /u01/app/19.3.0/grid
CRS service brought down successfully on home /u01/app/19.3.0/grid
Start applying binary patch on home /u01/app/19.3.0/grid
Binary patch applied successfully on home /u01/app/19.3.0/grid
Starting CRS service on home /u01/app/19.3.0/grid
CRS service started successfully on home /u01/app/19.3.0/grid
OPatchAuto successful.
--------------------------------Summary--------------------------------
Patching is completed successfully. Please find the summary as follows:
Host:rac2
CRS Home:/u01/app/19.3.0/grid
Version:19.0.0.0.0
Summary:
==Following patches were SUCCESSFULLY applied:
Patch: /tmp/30501910/30489227
Log: /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-03-13_21-37-51PM_1.log
Patch: /tmp/30501910/30489632
Log: /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-03-13_21-37-51PM_1.log
Patch: /tmp/30501910/30557433
Log: /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-03-13_21-37-51PM_1.log
Patch: /tmp/30501910/30655595
Log: /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-03-13_21-37-51PM_1.log
OPatchauto session completed at Fri Mar 13 22:13:32 2020
Time taken to complete the session 40 minutes, 44 seconds
[root@www.cndba.cn_2 cfgtoollogs]#
注意：這里可能會遇到如下錯誤：
Oracle 19c opatchauto Failed to serialize remote node list 錯誤解決方法
https://www.cndba.cn/dave/article/4075
2.8.2打DB 的RU
2.8.2.1先打節點1：（測試RU 目錄權限還是grid）
[root@www.cndba.cn_1 ~]# export PATH=$PATH:/u01/app/oracle/product/19.3.0/db_1
[root@www.cndba.cn_1 ~]# /u01/app/oracle/product/19.3.0/db_1/OPatch/opatchauto apply /tmp/30501910 -oh /u01/app/oracle/product/19.3.0/db_1 -nonrolling
OPatchauto session is initiated at Fri Mar 13 22:23:36 2020
System initialization log file is /u01/app/oracle/product/19.3.0/db_1/cfgtoollogs/opatchautodb/systemconfig2020-03-13_10-23-51PM.log.
Session log file is /u01/app/oracle/product/19.3.0/db_1/cfgtoollogs/opatchauto/opatchauto2020-03-13_10-24-02PM.log
The id for this session is NXT9
Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/19.3.0/db_1
Patch applicability verified successfully on home /u01/app/oracle/product/19.3.0/db_1
Verifying SQL patch applicability on home /u01/app/oracle/product/19.3.0/db_1
No step execution required.........
Preparing home /u01/app/oracle/product/19.3.0/db_1 after database service restarted
No step execution required.........
OPatchAuto successful.
--------------------------------Summary--------------------------------
Patching is completed successfully. Please find the summary as follows:
Host:rac1
RAC Home:/u01/app/oracle/product/19.3.0/db_1
Version:19.0.0.0.0
Summary:
==Following patches were SKIPPED:
Patch: /tmp/30501910/30489632
Reason: This patch is not applicable to this specified target type - "rac_database"
Patch: /tmp/30501910/30655595
Reason: This patch is not applicable to this specified target type - "rac_database"
Patch: /tmp/30501910/30489227
Reason: This patch is already been applied, so not going to apply again.
Patch: /tmp/30501910/30557433
Reason: This patch is already been applied, so not going to apply again.
OPatchauto session completed at Fri Mar 13 22:24:33 2020
Time taken to complete the session 0 minute, 58 seconds
[root@www.cndba.cn_1 ~]#
因此之前測試的時候，節點1已經成功打上了，所以這里跳過了。
2.8.2.2在打節點2：
[root@www.cndba.cn_2 ~]# export PATH=$PATH:/u01/app/oracle/product/19.3.0/db_1
[root@www.cndba.cn_2 ~]# /u01/app/oracle/product/19.3.0/db_1/OPatch/opatchauto apply /tmp/30501910 -oh /u01/app/oracle/product/19.3.0/db_1 -nonrolling
OPatchauto session is initiated at Fri Mar 13 22:49:14 2020
System initialization log file is /u01/app/oracle/product/19.3.0/db_1/cfgtoollogs/opatchautodb/systemconfig2020-03-13_10-49-21PM.log.
Session log file is /u01/app/oracle/product/19.3.0/db_1/cfgtoollogs/opatchauto/opatchauto2020-03-13_10-50-20PM.log
The id for this session is GXFF
Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/19.3.0/db_1
Patch applicability verified successfully on home /u01/app/oracle/product/19.3.0/db_1
Verifying SQL patch applicability on home /u01/app/oracle/product/19.3.0/db_1
SQL patch applicability verified successfully on home /u01/app/oracle/product/19.3.0/db_1
Preparing to bring down database service on home /u01/app/oracle/product/19.3.0/db_1
Successfully prepared home /u01/app/oracle/product/19.3.0/db_1 to bring down database service
Bringing down database service on home /u01/app/oracle/product/19.3.0/db_1
Database service successfully brought down on home /u01/app/oracle/product/19.3.0/db_1
Performing prepatch operation on home /u01/app/oracle/product/19.3.0/db_1
Perpatch operation completed successfully on home /u01/app/oracle/product/19.3.0/db_1
Start applying binary patch on home /u01/app/oracle/product/19.3.0/db_1
Binary patch applied successfully on home /u01/app/oracle/product/19.3.0/db_1
Performing postpatch operation on home /u01/app/oracle/product/19.3.0/db_1
Postpatch operation completed successfully on home /u01/app/oracle/product/19.3.0/db_1
Starting database service on home /u01/app/oracle/product/19.3.0/db_1
Database service successfully started on home /u01/app/oracle/product/19.3.0/db_1
Preparing home /u01/app/oracle/product/19.3.0/db_1 after database service restarted
No step execution required.........
Trying to apply SQL patch on home /u01/app/oracle/product/19.3.0/db_1
SQL patch applied successfully on home /u01/app/oracle/product/19.3.0/db_1
OPatchAuto successful.
--------------------------------Summary--------------------------------
Patching is completed successfully. Please find the summary as follows:
Host:rac2
RAC Home:/u01/app/oracle/product/19.3.0/db_1
Version:19.0.0.0.0
Summary:
==Following patches were SKIPPED:
Patch: /tmp/30501910/30489632
Reason: This patch is not applicable to this specified target type - "rac_database"
Patch: /tmp/30501910/30655595
Reason: This patch is not applicable to this specified target type - "rac_database"
Patch: /tmp/30501910/30489227
Reason: This patch is already been applied, so not going to apply again.
==Following patches were SUCCESSFULLY applied:
Patch: /tmp/30501910/30557433
Log: /u01/app/oracle/product/19.3.0/db_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-03-13_22-51-09PM_1.log
Patching session reported following warning(s):
_________________________________________________
[WARNING] The database instance 'cndba2' from '/u01/app/oracle/product/19.3.0/db_1', in host'rac2' is not running. SQL changes, if any, will not be applied.
To apply. the SQL changes, bring up the database instance and run the command manually from any one node (run as oracle).
Refer to the readme to get the correct steps for applying the sql changes.
OPatchauto session completed at Fri Mar 13 23:00:17 2020
Time taken to complete the session 11 minutes, 4 seconds
[root@www.cndba.cn_2 ~]#
這里可能會遇到如下錯誤：
Oracle 19c RU opatchauto oui-patch.xml (Permission denied) 解決方法
https://www.cndba.cn/dave/article/4076
Oracle 19c 打19.6 的RU opatchauto 報錯OPatchException: Unable to create patchObject 解決方法
https://www.cndba.cn/dave/article/4077
2.9 加載修改后的SQL到數據庫
安裝補丁之后，還需要將有變化的SQL加載到數據庫中，這里可以直接運行Datapatch工具將這些修改的SQL重新加載到數據庫中，RAC環境，只需要在一個節點執行就可以了。
根據readme的說明，操作步驟如下：
sqlplus /nolog
SQL> Connect / as sysdba
SQL> startup
SQL> alter pluggable database all open;
SQL> quit
cd $ORACLE_HOME/OPatch
./datapatch -verbose
datapatch命令只對打開的數據庫生效，所有Oracle建議在執行該命令之前將CDB和所有的PDB都打開，一次更新掉。 但如果有部分PDB沒有打開，也可以在打開之后，重新運行datapatch命令並制定pdb實例名，比如： ./datapatch -pdbs dave
這種情況可以參考之前的博客：
Oracle 18c 單實例 RUR 從18.3.0 升級到18.3.1 操作手冊
https://www.cndba.cn/dave/article/3138
我們這里將所有的PDB 都打開后在執行datapatch工具。
啟動數據庫：
[oracle@www.cndba.cn_2 oneoffs]$ sqlplus / as sysdba
SQL*Plus: Release 19.0.0.0.0 - Production on Fri Mar 13 23:16:06 2020
Version 19.6.0.0.0
Copyright (c) 1982, 2019, Oracle. All rights reserved.
Connected to an idle instance.
SQL> startup
ORACLE instance started.
Total System Global Area 2466250400 bytes
Fixed Size 9137824 bytes
Variable Size 738197504 bytes
Database Buffers 1711276032 bytes
Redo Buffers 7639040 bytes
Database mounted.
Database opened.
SQL> show pdbs
CON_ID CON_NAME OPEN MODE RESTRICTED
---------- ------------------------------ ---------- ----------
2 PDB$SEED READ ONLY NO
3 DAVE READ WRITE NO
SQL>
運行datapatch：這里我們在節點2上執行：
[oracle@www.cndba.cn_2 oneoffs]$ $ORACLE_HOME/OPatch/datapatch -verbose
SQL Patching tool version 19.6.0.0.0 Production on Fri Mar 13 23:19:09 2020
Copyright (c) 2012, 2019, Oracle. All rights reserved.
Log file for this invocation: /u01/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_6158_2020_03_13_23_19_09/sqlpatch_invocation.log
Connecting to database...OK
Gathering database info...done
Note: Datapatch will only apply or rollback SQL fixes for PDBs
that are in an open state, no patches will be applied to closed PDBs.
Please refer to Note: Datapatch: Database 12c Post Patch SQL Automation
(Doc ID 1585822.1)
Bootstrapping registry and package to current versions...done
Determining current state...done
Current state of interim SQL patches:
No interim patches found
Current state of release update SQL patches:
Binary registry:
19.6.0.0.0 Release_Update 191217155004: Installed
PDB CDB$ROOT:
Applied 19.3.0.0.0 Release_Update 190410122720 successfully on 12-MAR-20 04.12.02.535145 PM
PDB DAVE:
Applied 19.3.0.0.0 Release_Update 190410122720 successfully on 12-MAR-20 04.31.06.276026 PM
PDB PDB$SEED:
Applied 19.3.0.0.0 Release_Update 190410122720 successfully on 12-MAR-20 04.31.06.276026 PM
Adding patches to installation queue and performing prereq checks...done
Installation queue:
For the following PDBs: CDB$ROOT PDB$SEED DAVE
No interim patches need to be rolled back
Patch 30557433 (Database Release Update : 19.6.0.0.200114 (30557433)):
Apply from 19.3.0.0.0 Release_Update 190410122720 to 19.6.0.0.0 Release_Update 191217155004
No interim patches need to be applied
Installing patches...
Patch installation complete. Total patches installed: 3
Validating logfiles...done
Patch 30557433 apply (pdb CDB$ROOT): SUCCESS
logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/30557433/23305305/30557433_apply_CNDBA_CDBROOT_2020Mar13_23_21_42.log (no errors)
Patch 30557433 apply (pdb PDB$SEED): SUCCESS
logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/30557433/23305305/30557433_apply_CNDBA_PDBSEED_2020Mar13_23_32_19.log (no errors)
Patch 30557433 apply (pdb DAVE): SUCCESS
logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/30557433/23305305/30557433_apply_CNDBA_DAVE_2020Mar13_23_32_21.log (no errors)
SQL Patching tool complete on Fri Mar 13 23:42:45 2020
2.10 處理無效對象
最后一步就是處理無效對象，因為之前datapatch命令會加載SQL，這個過程可能會產生無效對象。 可以執行@utlrp.sql腳本處理這些無效對象。
[oracle@www.cndba.cn_2 oneoffs]$ cd $ORACLE_HOME/rdbms/admin
[oracle@www.cndba.cn_2 admin]$ sqlplus / as sysdba
SQL*Plus: Release 19.0.0.0.0 - Production on Fri Mar 13 23:44:38 2020
Version 19.6.0.0.0
Copyright (c) 1982, 2019, Oracle. All rights reserved.
Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.6.0.0.0
SQL> @utlrp.sql
Session altered.
2.11 驗證版本
節點2：
SQL> col action for a15
SQL> col status for a15
SQL> select PATCH_ID,PATCH_TYPE,ACTION,STATUS,TARGET_VERSION from dba_registry_sqlpatch;
PATCH_ID PATCH_TYPE ACTION STATUS TARGET_VERSION
---------- -------------------- --------------- --------------- ------------------------------
29517242 RU APPLY SUCCESS 19.3.0.0.0
30557433 RU APPLY SUCCESS 19.6.0.0.0
SQL>
[grid@www.cndba.cn_2 ~]$ /u01/app/19.3.0/grid/bin/kfod op=patches
---------------
List of Patches
===============
30489227
30489632
30557433
30655595
[grid@www.cndba.cn_2 ~]$ /u01/app/19.3.0/grid/bin/kfod op=patchlvl
-------------------
Current Patch level
===================
2701864972
[grid@www.cndba.cn_2 ~]$
[grid@www.cndba.cn_2 ~]$ $ORACLE_HOME/OPatch/opatch lsinventory
Oracle Interim Patch Installer version 12.2.0.1.19
Copyright (c) 2020, Oracle Corporation. All rights reserved.
Oracle Home : /u01/app/19.3.0/grid
Central Inventory : /u01/app/oraInventory
from : /u01/app/19.3.0/grid/oraInst.loc
OPatch version : 12.2.0.1.19
OUI version : 12.2.0.7.0
Log file location : /u01/app/19.3.0/grid/cfgtoollogs/opatch/opatch2020-03-13_23-49-10PM_1.log
Lsinventory Output file location : /u01/app/19.3.0/grid/cfgtoollogs/opatch/lsinv/lsinventory2020-03-13_23-49-10PM.txt
--------------------------------------------------------------------------------
Local Machine Information::
Hostname: rac2
ARU platform id: 226
ARU platform description:: Linux x86-64
Installed Top-level Products (1):
Oracle Grid Infrastructure 19c 19.0.0.0.0
There are 1 products installed in this Oracle Home.
Interim patches (4) :
Patch 30655595 : applied on Fri Mar 13 21:48:16 CST 2020
Unique Patch ID: 23265660
Patch description: "TOMCAT RELEASE UPDATE 19.0.0.0.0 (30655595)"
Created on 12 Dec 2019, 04:55:54 hrs PST8PDT
Bugs fixed:
29286300
Patch 30557433 : applied on Fri Mar 13 21:43:14 CST 2020
Unique Patch ID: 23305305
Patch description: "Database Release Update : 19.6.0.0.200114 (30557433)"
Created on 6 Jan 2020, 19:07:34 hrs PST8PDT
Bugs fixed:
30545281, 8476681, 14735102, 17428816, 19080742, 19697993, 20313356
21374587, 21965541, 23296836, 23606241, 24687075, 25756945, 25806201
25883179, 25986062, 25997810, 26476244, 26611353, 26739322, 26777814
26872233, 27036163, 27044169, 27101798, 27126938, 27195935, 27244999
27254335, 27359766, 27369515, 27406105, 27411022, 27423500, 27439716
……
節點1：
[grid@www.cndba.cn_1 ~]$ /u01/app/19.3.0/grid/bin/kfod op=patches
---------------
List of Patches
===============
30489227
30489632
30557433
30655595
[grid@www.cndba.cn_1 ~]$ /u01/app/19.3.0/grid/bin/kfod op=patchlvl
-------------------
Current Patch level
===================
2701864972
[grid@www.cndba.cn_1 ~]$
2.12 小結說明
之前計划使用滾動升級的方式進行，但測試一直過不去，所以在節點上按 nonrolling的方式進行了。 但是節點因為權限的問題導致CRS 無法啟動。 在解決節點1的權限問題這塊花了不少時間，最終解決方法是將節點2的/u01 目錄打包到了節點2，然后重新執行了root.sh 腳本，詳細過程參考我的博客：
Oracle 19c 升級19.6 RU 導致權限異常 gipcInternalConnectSync: failed sync request 解決方法
https://www.cndba.cn/dave/article/4080
[root@www.cndba.cn_1 crs]# crsctl stat res -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
ONLINE ONLINE rac1 STABLE
ONLINE ONLINE rac2 STABLE
ora.MGMT.GHCHKPT.advm
OFFLINE OFFLINE rac1 STABLE
OFFLINE OFFLINE rac2 STABLE
ora.chad
ONLINE ONLINE rac1 STABLE
ONLINE ONLINE rac2 STABLE
ora.helper
OFFLINE OFFLINE rac1 IDLE,STABLE
OFFLINE OFFLINE rac2 IDLE,STABLE
ora.mgmt.ghchkpt.acfs
OFFLINE OFFLINE rac1 STABLE
OFFLINE OFFLINE rac2 STABLE
ora.net1.network
ONLINE ONLINE rac1 STABLE
ONLINE ONLINE rac2 STABLE
ora.ons
ONLINE ONLINE rac1 STABLE
ONLINE ONLINE rac2 STABLE
ora.proxy_advm
OFFLINE OFFLINE rac1 STABLE
OFFLINE OFFLINE rac2 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
1 ONLINE ONLINE rac1 STABLE
2 ONLINE ONLINE rac2 STABLE
3 ONLINE OFFLINE STABLE
ora.DATA.dg(ora.asmgroup)
1 ONLINE ONLINE rac1 STABLE
2 ONLINE ONLINE rac2 STABLE
3 OFFLINE OFFLINE STABLE
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE rac2 STABLE
ora.MGMT.dg(ora.asmgroup)
1 ONLINE ONLINE rac1 STABLE
2 ONLINE ONLINE rac2 STABLE
3 OFFLINE OFFLINE STABLE
ora.MGMTLSNR
1 ONLINE ONLINE rac2 169.254.21.159 192.1
68.222.181,STABLE
ora.OCR.dg(ora.asmgroup)
1 ONLINE ONLINE rac1 STABLE
2 ONLINE ONLINE rac2 STABLE
3 OFFLINE OFFLINE STABLE
ora.asm(ora.asmgroup)
1 ONLINE ONLINE rac1 Started,STABLE
2 ONLINE ONLINE rac2 Started,STABLE
3 OFFLINE OFFLINE STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
1 ONLINE ONLINE rac1 STABLE
2 ONLINE ONLINE rac2 STABLE
3 OFFLINE OFFLINE STABLE
ora.cndba.db
1 ONLINE ONLINE rac1 Open,HOME=/u01/app/o
racle/product/19.3.0
/db_1,STABLE
2 ONLINE ONLINE rac2 Open,HOME=/u01/app/o
racle/product/19.3.0
/db_1,STABLE
ora.cvu
1 ONLINE ONLINE rac2 STABLE
ora.mgmtdb
1 ONLINE ONLINE rac2 Open,STABLE
ora.qosmserver
1 ONLINE ONLINE rac2 STABLE
ora.rac1.vip
1 ONLINE ONLINE rac1 STABLE
ora.rac2.vip
1 ONLINE ONLINE rac2 STABLE
ora.rhpserver
1 OFFLINE OFFLINE STABLE
ora.scan1.vip
1 ONLINE ONLINE rac2 STABLE
--------------------------------------------------------------------------------
[root@www.cndba.cn_1 crs]#