升級oracle11g集群到19.3
前述環境檢查：
[root@qhdb1 ~]# crsctl status res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA1.dg
ONLINE ONLINE qhdb1
ONLINE ONLINE qhdb2
ora.LISTENER.lsnr
ONLINE ONLINE qhdb1
ONLINE ONLINE qhdb2
ora.OCR.dg
ONLINE ONLINE qhdb1
ONLINE ONLINE qhdb2
ora.asm
ONLINE ONLINE qhdb1 Started
ONLINE ONLINE qhdb2 Started
ora.gsd
OFFLINE OFFLINE qhdb1
OFFLINE OFFLINE qhdb2
ora.net1.network
ONLINE ONLINE qhdb1
ONLINE ONLINE qhdb2
ora.ons
ONLINE ONLINE qhdb1
ONLINE ONLINE qhdb2
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE qhdb1
ora.andb.db
1 ONLINE ONLINE qhdb1 Open
2 ONLINE ONLINE qhdb2 Open
ora.cvu
1 ONLINE ONLINE qhdb1
ora.oc4j
1 ONLINE OFFLINE
ora.qhdb1.vip
1 ONLINE ONLINE qhdb1
ora.qhdb2.vip
1 ONLINE ONLINE qhdb2
ora.scan1.vip
1 ONLINE ONLINE qhdb1
-------------------------------------------------------------------------------
CRS版本檢查：
[root@qhdb1 ~]# crsctl query crs releaseversion
Oracle High Availability Services release version on the local node is [11.2.0.4.0]
[root@qhdb1 ~]# crsctl query crs softwareversion
Oracle Clusterware version on node [qhdb1] is [11.2.0.4.0]
You have mail in /var/spool/mail/root
[root@qhdb1 ~]# crsctl query crs softwareversion -all
Oracle Clusterware version on node [qhdb2] is [11.2.0.4.0]
Oracle Clusterware version on node [qhdb1] is [11.2.0.4.0]
[root@qhdb1 ~]# crsctl check cluster -all
**************************************************************
qhdb1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
qhdb2:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
ASM兼容性檢查：
SQL> set lines 300
SQL> col name for a30
col COMPATIBILITY for a30
col DATABASE_COMPATIBILITY for a40
select GROUP_NUMBER,NAME,TYPE,COMPATIBILITY,DATABASE_COMPATIBILITY from v$asm_diskgroup;
SQL> SQL> SQL>
GROUP_NUMBER NAME TYPE COMPATIBILITY DATABASE_COMPATIBILITY
------------ ------------------------------ ------ ------------------------------ ----------------------------------------
1 OCR NORMAL 11.2.0.0.0 10.1.0.0.0
2 DATA1 EXTERN 11.2.0.0.0 10.1.0.0.0
設置ASM兼容性：
alter diskgroup DATA1 set attribute 'compatible.rdbms'='x';
alter diskgroup DATA1 set attribute 'compatible.asm'='x';
補丁應用情況檢查：
SQL> set lines 300
SQL> col COMP_NAME for a50
SQL> select COMP_ID,comp_name,version,status from dba_registry order by 4;
COMP_ID COMP_NAME VERSION STATUS
------------------------------ -------------------------------------------------- ------------------------------ --------------------------------------------
OWB OWB 11.2.0.4.0 VALID
RAC Oracle Real Application Clusters 11.2.0.4.0 VALID
EM Oracle Enterprise Manager 11.2.0.4.0 VALID
AMD OLAP Catalog 11.2.0.4.0 VALID
SDO Spatial 11.2.0.4.0 VALID
ORDIM Oracle Multimedia 11.2.0.4.0 VALID
XDB Oracle XML Database 11.2.0.4.0 VALID
CONTEXT Oracle Text 11.2.0.4.0 VALID
EXF Oracle Expression Filter 11.2.0.4.0 VALID
RUL Oracle Rules Manager 11.2.0.4.0 VALID
OWM Oracle Workspace Manager 11.2.0.4.0 VALID
COMP_ID COMP_NAME VERSION STATUS
------------------------------ -------------------------------------------------- ------------------------------ --------------------------------------------
CATALOG Oracle Database Catalog Views 11.2.0.4.0 VALID
CATPROC Oracle Database Packages and Types 11.2.0.4.0 VALID
JAVAVM JServer JAVA Virtual Machine 11.2.0.4.0 VALID
XML Oracle XDK 11.2.0.4.0 VALID
CATJAVA Oracle Database Java Packages 11.2.0.4.0 VALID
APS OLAP Analytic Workspace 11.2.0.4.0 VALID
XOQ Oracle OLAP API 11.2.0.4.0 VALID
APEX Oracle Application Express 3.2.1.00.12 VALID
19 rows selected.
SQL>
SQL> col ACTION_TIME for a30
SQL> col ACTION for a15
SQL> col NAMESPACE for a15
SQL> col VERSION for a25
SQL> col COMMENTS for a60
SQL> col BUNDLE_SERIES for a15
set lines 400
select * from dba_registry_history order by 1;
SQL> SQL>
ACTION_TIME ACTION NAMESPACE VERSION ID BUNDLE_SERIES COMMENTS
------------------------------ --------------- --------------- ------------------------- ---------- --------------- ------------------------------------------------------------
24-AUG-13 12.03.45.119862 PM APPLY SERVER 11.2.0.4 0 PSU Patchset 11.2.0.2.0
10-AUG-21 01.45.44.791636 PM APPLY SERVER 11.2.0.4 0 PSU Patchset 11.2.0.2.0
10-AUG-21 03.18.19.828867 PM APPLY SERVER 11.2.0.4 190416 PSU PSU 11.2.0.4.190416
[grid@qhdb1 ~]$ opatch lsinv
Oracle Interim Patch Installer version 11.2.0.3.21
Copyright (c) 2021, Oracle Corporation. All rights reserved.
Oracle Home : /u01/app/11.2.4/grid
Central Inventory : /u01/app/oraInventory
from : /u01/app/11.2.4/grid/oraInst.loc
OPatch version : 11.2.0.3.21
OUI version : 11.2.0.4.0
Log file location : /u01/app/11.2.4/grid/cfgtoollogs/opatch/opatch2021-08-11_10-30-33AM_1.log
Lsinventory Output file location : /u01/app/11.2.4/grid/cfgtoollogs/opatch/lsinv/lsinventory2021-08-11_10-30-33AM.txt
--------------------------------------------------------------------------------
Local Machine Information::
Hostname: qhdb1.zpepc.com.cn
ARU platform id: 226
ARU platform description:: Linux x86-64
Installed Top-level Products (1):
Oracle Grid Infrastructure 11g 11.2.0.4.0
There are 1 products installed in this Oracle Home.
Interim patches (3) :
Patch 28729245 : applied on Tue Aug 10 14:34:05 CST 2021
Unique Patch ID: 22724549
Patch description: "ACFS Patch Set Update : 11.2.0.4.190115 (28729245)"
Created on 2 Nov 2018, 04:47:07 hrs PST8PDT
Bugs fixed:
28069955, 26247490, 21369858, 16318126, 19690653, 17503605, 17203009
17359415, 20140148, 17611362, 25098392, 17270418, 17164243, 19053182
17696547, 17488768, 25078431, 18168684, 21519796, 18143006, 21208140
23321114, 17428148, 17070158, 20438706, 17510275, 17172303, 28142134
18610307, 17376318, 17721778, 22198405, 17699423, 22810422, 18915417
21233961, 18155334, 25381434, 19919907, 18321597, 18185024, 17363999
17636008, 27593263, 20681968, 17475946, 27463879
Patch 29141056 : applied on Tue Aug 10 14:33:07 CST 2021
Unique Patch ID: 22768427
Patch description: "Database Patch Set Update : 11.2.0.4.190416 (29141056)"
Created on 17 Feb 2019, 23:52:26 hrs PST8PDT
Sub-patch 28729262; "Database Patch Set Update : 11.2.0.4.190115 (28729262)"
1.配置19 軟件安裝目錄
grid:節點qhdb1
umask 022
export ORACLE_BASE=/u01/app/grid
export ORACLE_HOME=/u01/app/11.2.4/grid
####19c ORACLE_HOME
##export ORACLE_HOME=/u01/app/19.0.0/grid
export ORACLE_SID=+ASM2
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib:$ORACLE_HOME/rdbms/lib
export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:/usr/bin:/usr/sbin:/usr/local/bin:$PATH
grid:節點qhdb2
umask 022
export ORACLE_BASE=/u01/app/grid
export ORACLE_HOME=/u01/app/11.2.4/grid
####19c ORACLE_HOME
##export ORACLE_HOME=/u01/app/19.0.0/grid
export ORACLE_SID=+ASM1
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib:$ORACLE_HOME/rdbms/lib
export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:/usr/bin:/usr/sbin:/usr/local/bin:$PATH
oracle：節點qhdb1
umask 022
export ORACLE_BASE=/u01/app/oracle
export ORACLE_SID=andb2
export ORACLE_HOME=/u01/app/oracle/product/11.2.4/db_home1
#####19c oracle_home
##export ORACLE_HOME=/u01/app/oracle/product/19.0.0/db_home1
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib:$ORACLE_HOME/rdbms/lib
export PATH=$ORACLE_HOME/bin:/u01/app/11.2.4/grid/bin:/usr/bin:/usr/sbin:/usr/local/bin:$PATH
oracle：節點qhdb2
umask 022
export ORACLE_BASE=/u01/app/oracle
export ORACLE_SID=andb1
export ORACLE_HOME=/u01/app/oracle/product/11.2.4/db_home1
#####19c oracle_home
##export ORACLE_HOME=/u01/app/oracle/product/19.0.0/db_home1
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib:$ORACLE_HOME/rdbms/lib
export PATH=$ORACLE_HOME/bin:/u01/app/11.2.4/grid/bin:/usr/bin:/usr/sbin:/usr/local/bin:$PATH
2.拷貝軟件到ORACLE_HOME
grid:
cp LINUX.X64_193000_grid_home.zip /u01/app/19.0.0/grid/
oracle:
cp LINUX.X64_193000_db_home.zip /u01/app/oracle/product/19.0.0/db_home1/
修改軟件包權限，並解壓：
grid:
[grid@qhdb1 grid]$ logout
You have mail in /var/spool/mail/root
[root@qhdb1 app]# cd /u01/app/19.0.0/grid
[root@qhdb1 grid]# ls -lrt
total 2821472
-rw-r--r-- 1 root root 2889184573 Aug 11 10:16 LINUX.X64_193000_grid_home.zip
[root@qhdb1 grid]#
[root@qhdb1 grid]# chown -R grid:oinstall LINUX.X64_193000_grid_home.zip
[root@qhdb1 grid]# su - grid
Last login: Wed Aug 11 10:30:28 CST 2021 on pts/0
[grid@qhdb1 ~]$ cd /u01/app/19.0.0/grid
[grid@qhdb1 grid]$ ls -lrt
total 2821472
-rw-r--r-- 1 grid oinstall 2889184573 Aug 11 10:16 LINUX.X64_193000_grid_home.zip
[grid@qhdb1 grid]$ unzip LINUX.X64_193000_grid_home.zip
oracle:
[oracle@qhdb1 ~]$ cd /u01/app/oracle/product/19.0.0/db_home1
[oracle@qhdb1 db_home1]$ ls -lrt
total 2987996
-rw-r--r-- 1 root root 3059705302 Aug 11 10:22 LINUX.X64_193000_db_home.zip
[oracle@qhdb1 db_home1]$
[oracle@qhdb1 db_home1]$
[oracle@qhdb1 db_home1]$ logout
[root@qhdb1 ~]# cd /u01/app/oracle/product/19.0.0/db_home1
[root@qhdb1 db_home1]# chown -R oracle:oinstall LINUX.X64_193000_db_home.zip
[root@qhdb1 db_home1]# su - oracle
Last login: Wed Aug 11 10:32:29 CST 2021 on pts/3
[oracle@qhdb1 ~]$ cd /u01/app/oracle/product/19.0.0/db_home1
[oracle@qhdb1 db_home1]$ ls -lrt
total 2987996
-rw-r--r-- 1 oracle oinstall 3059705302 Aug 11 10:22 LINUX.X64_193000_db_home.zip
[oracle@qhdb1 db_home1]$
[oracle@qhdb1 db_home1]$
[oracle@qhdb1 db_home1]$ unzip LINUX.X64_193000_db_home.zip
3.調出VNC，准備升級CRS
[grid@qhdb1 grid]$ export DISPLAY=:1
[grid@qhdb1 grid]$ pwd
/u01/app/19.0.0/grid
[grid@qhdb1 grid]$ ./gridSetup.sh
1.選擇升級GI-
2.直接點擊下一步--若互信有問題則會提示、重新配置互信
3.點擊下一步
4.設置ORACLE_BASE 和ORACLE_HOME目錄，這里ORACLE_HOME目錄是灰色不讓選擇的，但是BASE目錄是可以選擇的
5.升級腳本是自動運行還是手工運行
手工運行uproot.sh腳本就不要選擇
要是自動運行升級腳本，則打勾勾，輸入root密碼
6.需要注意：
嘗試忽略，提示不能夠忽略，所以，接下來 我們需要逐個問題結局：
1.提示需要安裝補丁：28553832
2.設置ASM diskgroup compatibility 兼容性參數設置；
3.time oggset between nodes ：兩個節點時間差太大，需要設置NTP服務；
一：首先解決：NTP
配置NTP服務器：
二：查看ASM磁盤組兼容性參數，並修改：
三：補丁更新：28553832--
注意：我這里直接下載了11.2.0.4的最新補丁包：31718723
接下來就可以忽略升級GI
接下來就是升級GI
執行升級腳本：
節點一執行升級腳本：
[root@qhdb1 ~]# /u01/app/19.0.0/grid/rootupgrade.sh
Performing root user operation.
The following environment variables are set as:
ORACLE_OWNER= grid
ORACLE_HOME= /u01/app/19.0.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y
Copying dbhome to /usr/local/bin ...
The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y
Copying oraenv to /usr/local/bin ...
The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y
Copying coraenv to /usr/local/bin ...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /u01/app/19.0.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
/u01/app/grid/crsdata/qhdb1/crsconfig/rootcrs_qhdb1_2021-08-16_08-15-33AM.log
2021/08/16 08:15:49 CLSRSC-595: Executing upgrade step 1 of 18: 'UpgradeTFA'.
2021/08/16 08:15:49 CLSRSC-4015: Performing install or upgrade action for Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 08:15:49 CLSRSC-595: Executing upgrade step 2 of 18: 'ValidateEnv'.
2021/08/16 08:15:53 CLSRSC-595: Executing upgrade step 3 of 18: 'GetOldConfig'.
2021/08/16 08:15:54 CLSRSC-464: Starting retrieval of the cluster configuration data
2021/08/16 08:16:01 CLSRSC-692: Checking whether CRS entities are ready for upgrade. This operation may take a few minutes.
2021/08/16 08:18:50 CLSRSC-4003: Successfully patched Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 08:18:50 CLSRSC-693: CRS entities validation completed successfully.
2021/08/16 08:18:54 CLSRSC-515: Starting OCR manual backup.
2021/08/16 08:18:59 CLSRSC-516: OCR manual backup successful.
2021/08/16 08:21:21 CLSRSC-486:
At this stage of upgrade, the OCR has changed.
Any attempt to downgrade the cluster after this point will require a complete cluster outage to restore the OCR.
2021/08/16 08:21:21 CLSRSC-541:
To downgrade the cluster:
1. All nodes that have been upgraded must be downgraded.
2021/08/16 08:21:21 CLSRSC-542:
2. Before downgrading the last node, the Grid Infrastructure stack on all other cluster nodes must be down.
2021/08/16 08:21:27 CLSRSC-465: Retrieval of the cluster configuration data has successfully completed.
2021/08/16 08:21:27 CLSRSC-595: Executing upgrade step 4 of 18: 'GenSiteGUIDs'.
2021/08/16 08:21:28 CLSRSC-595: Executing upgrade step 5 of 18: 'UpgPrechecks'.
2021/08/16 08:21:32 CLSRSC-363: User ignored prerequisites during installation
2021/08/16 08:21:38 CLSRSC-595: Executing upgrade step 6 of 18: 'SetupOSD'.
2021/08/16 08:21:38 CLSRSC-595: Executing upgrade step 7 of 18: 'PreUpgrade'.
2021/08/16 08:22:03 CLSRSC-468: Setting Oracle Clusterware and ASM to rolling migration mode
2021/08/16 08:22:03 CLSRSC-482: Running command: '/u01/app/19.0.0/grid/bin/asmca -silent -upgradeNodeASM -nonRolling false -oldCRSHome /u01/app/11.2.4/grid -oldCRSVersion 11.2.0.4.0 -firstNode true -startRolling true '
ASM configuration upgraded in local node successfully.
2021/08/16 08:22:10 CLSRSC-469: Successfully set Oracle Clusterware and ASM to rolling migration mode
2021/08/16 08:25:52 CLSRSC-466: Starting shutdown of the current Oracle Grid Infrastructure stack
2021/08/16 08:26:19 CLSRSC-467: Shutdown of the current Oracle Grid Infrastructure stack has successfully completed.
2021/08/16 08:26:24 CLSRSC-595: Executing upgrade step 8 of 18: 'CheckCRSConfig'.
2021/08/16 08:26:24 CLSRSC-595: Executing upgrade step 9 of 18: 'UpgradeOLR'.
2021/08/16 08:26:35 CLSRSC-595: Executing upgrade step 10 of 18: 'ConfigCHMOS'.
2021/08/16 08:26:36 CLSRSC-595: Executing upgrade step 11 of 18: 'UpgradeAFD'.
2021/08/16 08:26:42 CLSRSC-595: Executing upgrade step 12 of 18: 'createOHASD'.
2021/08/16 08:26:49 CLSRSC-595: Executing upgrade step 13 of 18: 'ConfigOHASD'.
2021/08/16 08:26:49 CLSRSC-329: Replacing Clusterware entries in file 'oracle-ohasd.service'
2021/08/16 08:27:51 CLSRSC-595: Executing upgrade step 14 of 18: 'InstallACFS'.
CRS-41053: checking Oracle Grid Infrastructure for file permission issues
PRVH-0116 : Path "/u01/app/19.0.0/grid/crs/install/cmdllroot.sh" with permissions "rw-r--r--" does not have execute permissions for the owner, file's group, and others on node "qhdb1".
PRVG-2031 : Owner of file "/u01/app/19.0.0/grid/crs/install/cmdllroot.sh" did not match the expected value on node "qhdb1". [Expected = "grid(1002)" ; Found = "root(0)"]
PRVG-2032 : Group of file "/u01/app/19.0.0/grid/crs/install/cmdllroot.sh" did not match the expected value on node "qhdb1". [Expected = "oinstall(1006)" ; Found = "root(0)"]
CRS-4124: Oracle High Availability Services startup failed.
CRS-4000: Command Start failed, or completed with errors.
2021/08/16 08:30:00 CLSRSC-117: Failed to start Oracle Clusterware stack
Died at /u01/app/19.0.0/grid/crs/install/crsutils.pm line 11714.
You have mail in /var/spool/mail/root
[root@qhdb1 ~]#
升級報錯： 權限不足
檢查："/u01/app/19.0.0/grid/crs/install/cmdllroot.sh權限
修改權限：chown -R grid:oinstall * ; chmod -R 775 *
看到這個目錄有些文件權限是root:root ,所以 我們直接修改。----------------------------問題OK！
重新執行升級腳本：
[root@qhdb1 ~]# /u01/app/19.0.0/grid/rootupgrade.sh
Performing root user operation.
The following environment variables are set as:
ORACLE_OWNER= grid
ORACLE_HOME= /u01/app/19.0.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /u01/app/19.0.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
/u01/app/grid/crsdata/qhdb1/crsconfig/rootcrs_qhdb1_2021-08-16_09-15-01AM.log
2021/08/16 09:15:12 CLSRSC-595: Executing upgrade step 1 of 18: 'UpgradeTFA'.
2021/08/16 09:15:12 CLSRSC-4015: Performing install or upgrade action for Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 09:15:12 CLSRSC-4012: Shutting down Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 09:16:52 CLSRSC-4013: Successfully shut down Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 09:16:52 CLSRSC-595: Executing upgrade step 2 of 18: 'ValidateEnv'.
2021/08/16 09:16:53 CLSRSC-595: Executing upgrade step 3 of 18: 'GetOldConfig'.
2021/08/16 09:16:53 CLSRSC-464: Starting retrieval of the cluster configuration data
2021/08/16 09:16:57 CLSRSC-692: Checking whether CRS entities are ready for upgrade. This operation may take a few minutes.
2021/08/16 09:17:00 CLSRSC-693: CRS entities validation completed successfully.
2021/08/16 09:17:04 CLSRSC-4003: Successfully patched Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 09:17:06 CLSRSC-465: Retrieval of the cluster configuration data has successfully completed.
2021/08/16 09:17:06 CLSRSC-595: Executing upgrade step 4 of 18: 'GenSiteGUIDs'.
2021/08/16 09:17:07 CLSRSC-595: Executing upgrade step 5 of 18: 'UpgPrechecks'.
2021/08/16 09:17:07 CLSRSC-595: Executing upgrade step 6 of 18: 'SetupOSD'.
2021/08/16 09:17:07 CLSRSC-595: Executing upgrade step 7 of 18: 'PreUpgrade'.
2021/08/16 09:17:07 CLSRSC-595: Executing upgrade step 8 of 18: 'CheckCRSConfig'.
2021/08/16 09:17:08 CLSRSC-595: Executing upgrade step 9 of 18: 'UpgradeOLR'.
2021/08/16 09:17:09 CLSRSC-595: Executing upgrade step 10 of 18: 'ConfigCHMOS'.
2021/08/16 09:17:44 CLSRSC-595: Executing upgrade step 11 of 18: 'UpgradeAFD'.
2021/08/16 09:17:45 CLSRSC-595: Executing upgrade step 12 of 18: 'createOHASD'.
2021/08/16 09:17:46 CLSRSC-595: Executing upgrade step 13 of 18: 'ConfigOHASD'.
CSKM-1110: error while generating cluster root certificate
CRS-10403: (:CLSCRED0102:)Could not connect to credential storage: PROCL-32: Oracle High Availability Services on the local node is not running Messaging error [gipcretConnectionRefused] [29].
CRS-4000: Command Delete failed, or completed with errors.
問題：執行升級腳本失敗，分析問題原因：提示HAS not running！！！
由於我們Linux版本是7，原數據庫版本是11g，由於11g和Linux7版本的systemd進程服務不相匹配，
所以我們在Linux7版本上安裝oracle11gRAC,需要手工創建ohas.service
以下是Linux上安裝11g，需要配置的OHAS服務。
啟動服務：
systemctl daemon-reload
systemctl enable ohas.service
systemctl start ohas.service
systemctl status ohas.service
/usr/lib/systemd/system/ohas.service
處理方式：ps -fle | grep d.bin 然后kill
關閉ohas服務：
systemctl disable ohas.service
systemctl stop ohas.service
然后在重新執行升級腳本：
[root@qhdb1 ~]# /u01/app/19.0.0/grid/rootupgrade.sh
Performing root user operation.
The following environment variables are set as:
ORACLE_OWNER= grid
ORACLE_HOME= /u01/app/19.0.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /u01/app/19.0.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
/u01/app/grid/crsdata/qhdb1/crsconfig/rootcrs_qhdb1_2021-08-16_09-40-46AM.log
2021/08/16 09:40:57 CLSRSC-595: Executing upgrade step 1 of 18: 'UpgradeTFA'.
2021/08/16 09:40:57 CLSRSC-4015: Performing install or upgrade action for Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 09:40:57 CLSRSC-4012: Shutting down Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 09:42:37 CLSRSC-4013: Successfully shut down Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 09:42:37 CLSRSC-595: Executing upgrade step 2 of 18: 'ValidateEnv'.
2021/08/16 09:42:38 CLSRSC-595: Executing upgrade step 3 of 18: 'GetOldConfig'.
2021/08/16 09:42:38 CLSRSC-464: Starting retrieval of the cluster configuration data
2021/08/16 09:42:43 CLSRSC-692: Checking whether CRS entities are ready for upgrade. This operation may take a few minutes.
2021/08/16 09:42:50 CLSRSC-4003: Successfully patched Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 09:45:40 CLSRSC-693: CRS entities validation completed successfully.
2021/08/16 09:45:46 CLSRSC-465: Retrieval of the cluster configuration data has successfully completed.
2021/08/16 09:45:46 CLSRSC-595: Executing upgrade step 4 of 18: 'GenSiteGUIDs'.
2021/08/16 09:45:47 CLSRSC-595: Executing upgrade step 5 of 18: 'UpgPrechecks'.
2021/08/16 09:45:47 CLSRSC-595: Executing upgrade step 6 of 18: 'SetupOSD'.
2021/08/16 09:45:47 CLSRSC-595: Executing upgrade step 7 of 18: 'PreUpgrade'.
2021/08/16 09:45:47 CLSRSC-595: Executing upgrade step 8 of 18: 'CheckCRSConfig'.
2021/08/16 09:45:48 CLSRSC-595: Executing upgrade step 9 of 18: 'UpgradeOLR'.
2021/08/16 09:45:49 CLSRSC-595: Executing upgrade step 10 of 18: 'ConfigCHMOS'.
2021/08/16 09:45:49 CLSRSC-595: Executing upgrade step 11 of 18: 'UpgradeAFD'.
2021/08/16 09:45:50 CLSRSC-595: Executing upgrade step 12 of 18: 'createOHASD'.
2021/08/16 09:45:51 CLSRSC-595: Executing upgrade step 13 of 18: 'ConfigOHASD'.
2021/08/16 09:45:54 CLSRSC-595: Executing upgrade step 14 of 18: 'InstallACFS'.
2021/08/16 09:48:17 CLSRSC-595: Executing upgrade step 15 of 18: 'InstallKA'.
2021/08/16 09:48:24 CLSRSC-595: Executing upgrade step 16 of 18: 'UpgradeCluster'.
clscfg: EXISTING configuration version 5 detected.
Successfully taken the backup of node specific configuration in OCR.
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
2021/08/16 10:01:59 CLSRSC-343: Successfully started Oracle Clusterware stack
2021/08/16 10:02:05 CLSRSC-595: Executing upgrade step 17 of 18: 'UpgradeNode'.
2021/08/16 10:02:09 CLSRSC-474: Initiating upgrade of resource types
2021/08/16 10:02:58 CLSRSC-475: Upgrade of resource types successfully initiated.
2021/08/16 10:03:07 CLSRSC-595: Executing upgrade step 18 of 18: 'PostUpgrade'.
2021/08/16 10:03:21 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded
看到軟件已經升級成功了！
開始升級節點2：/u01/app/19.0.0/grid/rootupgrade.sh
CRS-1115: Oracle Clusterware has already been upgraded.
2021/08/16 10:36:54 CLSRSC-479: Successfully set Oracle Clusterware active version
2021/08/16 10:37:02 CLSRSC-476: Finishing upgrade of resource types
PRCR-1079 : Failed to start resource ora.qosmserver
TCC-0004: The container was not able to start.
Failed to start connector [Connector[HTTP/1.1-8888]]Failed to initialize component [Connector[HTTP/1.1-8888]]
Protocol handler initialization failed
Address already in use
CRS-2674: Start of 'ora.qosmserver' on 'qhdb2' failed
TCC-0004: The container was not able to start.
Failed to start connector [Connector[HTTP/1.1-8888]]Failed to initialize component [Connector[HTTP/1.1-8888]]
Protocol handler initialization failed
Address already in use
CRS-2674: Start of 'ora.qosmserver' on 'qhdb1' failed
CRS-2632: There are no more servers to try to place resource 'ora.qosmserver' on that would satisfy its placement policy
2021/08/16 10:37:37 CLSRSC-180: An error occurred while executing the command '/u01/app/19.0.0/grid/bin/srvctl upgrade model -s 11.2.0.4.0 -d 19.0.0.0.0 -p last'
2021/08/16 10:37:37 CLSRSC-284: Failed to perform last node tasks for cluster modeling upgrade
Died at /u01/app/19.0.0/grid/crs/install/orasrvm.pm line 570.
解決方式：
[root@qhdb2 ~]# vi /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
#::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 注釋掉
1.查看lo的IPV6地址：
刪除 ：
ip addr del ::1/128 dev lo
netstat -tunlp | grep ':8888'
ps -fle | grep d.bin ---KILL
ps -fle | grep grid ---KILL
然后重新運行rootupgrade.sh 升級腳本
=============================================
[root@qhdb2 app]# /u01/app/19.0.0/grid/rootupgrade.sh
Performing root user operation.
The following environment variables are set as:
ORACLE_OWNER= grid
ORACLE_HOME= /u01/app/19.0.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /u01/app/19.0.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
/u01/app/grid/crsdata/qhdb2/crsconfig/rootcrs_qhdb2_2021-08-16_02-21-22PM.log
2021/08/16 14:21:31 CLSRSC-595: Executing upgrade step 1 of 18: 'UpgradeTFA'.
2021/08/16 14:21:31 CLSRSC-4015: Performing install or upgrade action for Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 14:21:31 CLSRSC-4012: Shutting down Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 14:23:11 CLSRSC-4013: Successfully shut down Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 14:23:11 CLSRSC-595: Executing upgrade step 2 of 18: 'ValidateEnv'.
2021/08/16 14:23:12 CLSRSC-595: Executing upgrade step 3 of 18: 'GetOldConfig'.
2021/08/16 14:23:12 CLSRSC-464: Starting retrieval of the cluster configuration data
2021/08/16 14:23:24 CLSRSC-4003: Successfully patched Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 14:23:48 CLSRSC-465: Retrieval of the cluster configuration data has successfully completed.
2021/08/16 14:23:49 CLSRSC-595: Executing upgrade step 4 of 18: 'GenSiteGUIDs'.
2021/08/16 14:23:49 CLSRSC-595: Executing upgrade step 5 of 18: 'UpgPrechecks'.
2021/08/16 14:23:49 CLSRSC-595: Executing upgrade step 6 of 18: 'SetupOSD'.
2021/08/16 14:23:49 CLSRSC-595: Executing upgrade step 7 of 18: 'PreUpgrade'.
2021/08/16 14:23:49 CLSRSC-595: Executing upgrade step 8 of 18: 'CheckCRSConfig'.
2021/08/16 14:23:50 CLSRSC-595: Executing upgrade step 9 of 18: 'UpgradeOLR'.
2021/08/16 14:23:51 CLSRSC-595: Executing upgrade step 10 of 18: 'ConfigCHMOS'.
2021/08/16 14:24:22 CLSRSC-595: Executing upgrade step 11 of 18: 'UpgradeAFD'.
2021/08/16 14:24:23 CLSRSC-595: Executing upgrade step 12 of 18: 'createOHASD'.
2021/08/16 14:24:24 CLSRSC-595: Executing upgrade step 13 of 18: 'ConfigOHASD'.
2021/08/16 14:24:24 CLSRSC-329: Replacing Clusterware entries in file 'oracle-ohasd.service'
2021/08/16 14:24:38 CLSRSC-595: Executing upgrade step 14 of 18: 'InstallACFS'.
2021/08/16 14:24:40 CLSRSC-595: Executing upgrade step 15 of 18: 'InstallKA'.
2021/08/16 14:24:41 CLSRSC-595: Executing upgrade step 16 of 18: 'UpgradeCluster'.
2021/08/16 14:25:47 CLSRSC-343: Successfully started Oracle Clusterware stack
2021/08/16 14:25:53 CLSRSC-595: Executing upgrade step 17 of 18: 'UpgradeNode'.
CRS-1115: Oracle Clusterware has already been upgraded.
2021/08/16 14:25:58 CLSRSC-478: Setting Oracle Clusterware active version on the last node to be upgraded
2021/08/16 14:25:58 CLSRSC-482: Running command: '/u01/app/19.0.0/grid/bin/crsctl set crs activeversion'
CRS-1115: Oracle Clusterware has already been upgraded.
2021/08/16 14:26:58 CLSRSC-479: Successfully set Oracle Clusterware active version
2021/08/16 14:27:06 CLSRSC-476: Finishing upgrade of resource types
2021/08/16 14:27:45 CLSRSC-477: Successfully completed upgrade of resource types
2021/08/16 14:28:43 CLSRSC-595: Executing upgrade step 18 of 18: 'PostUpgrade'.
Successfully updated XAG resources.
2021/08/16 14:29:13 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded
至此GI升級完成！！！DB的升級其實就是重新安裝一套數據庫軟件而已，這個一般來時不會出現問題的。
-----------------------接下來升級數據庫軟件
節點一 二創建19c數據軟件目錄
mkdir -pv /u01/app/oracle/product/19.0.0/db_home1
chown -R oracle:oinstall /u01/app/oracle/product/19.0.0
chmod -R 775 /u01/app/oracle/product/19.0.0/db_home1
su - oracle
unzip LINUX.X64_193000_db_home.zip
調出圖形化，裝備安裝19c數據庫軟件
僅安裝數據庫軟件
集群數據庫軟件
默認選擇NEXT
默認選擇NEXT
默認選擇NEXT
默認選擇NEXT
默認選擇NEXT--點擊忽略打勾勾
執行完畢后點擊OK 軟件升級完成！！！
------------------------------接下來升級數據庫andb
---拷貝文件11g $ORACLE_HOME/dbs/initandb***.ora 到19c環境下
ORA-00603: ORACLE server session terminated by fatal error
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-39701: database must be mounted EXCLUSIVE for UPGRADE or DOWNGRADE
提示cluster_database 參數需要修改
[oracle@qhdb1 ~]$ sqlplus / as sysdba
SQL*Plus: Release 19.0.0.0.0 - Production on Mon Aug 16 16:42:49 2021
Version 19.3.0.0.0
Copyright (c) 1982, 2019, Oracle. All rights reserved.
Connected to an idle instance.
SQL>
SQL> startup nomount
ORACLE instance started.
Total System Global Area 8589930600 bytes
Fixed Size 12456040 bytes
Variable Size 1593835520 bytes
Database Buffers 6962544640 bytes
Redo Buffers 21094400 bytes
SQL>
SQL> show parameter cluster
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
cdb_cluster boolean FALSE
cdb_cluster_name string
cluster_database boolean TRUE
cluster_database_instances integer 2
cluster_interconnects string
SQL> alter system set cluster_database=false scope=spfile;
System altered.
SQL> shu abort
ORACLE instance shut down.
SQL> startup upgrade
ORACLE instance started.
Total System Global Area 8589930600 bytes
Fixed Size 12456040 bytes
Variable Size 1593835520 bytes
Database Buffers 6962544640 bytes
Redo Buffers 21094400 bytes
Database mounted.
Database opened.
接下來就是升級數據字典了！！！----- @?/rdbms/admin/catupgrd.sql
OK 提示我們按照它提供的方式進行升級！！
cd $ORACLE_HOME/rdbms/admin
$ORACLE_HOME/perl/bin/perl catctl.pl catupgrd.sql
----------------------
cd $ORACLE_HOME/rdbms/admin
$ORACLE_HOME/perl/bin/perl catctl.pl -n 24 catupgrd.sql
-n 24 開並發進行升級！！
[oracle@qhdb1 admin]$ $ORACLE_HOME/perl/bin/perl catctl.pl -n 24 catupgrd.sql
升級后補充運行腳本，要是手動升級數據庫，但是未運行腳本catuppst.sql腳本，則數據庫的性能隨着時間下降。
$ORACLE_HOME/perl/bin/perl $ORACLE_HOME/rdbms/admin/catcon.pl -n 24 -e -b catuppst -d "." catuppst.sql
編譯無效對象：
$ORACLE_HOME/perl/bin/perl $ORACLE_HOME/rdbms/admin/catcon.pl -n 24 -e -b utlrp -d "." utlrp.sql
----------------------升級完后檢查：
1.timezone_file
2.sqlnet.ora
3.ora-20000權限不足檢查
4.創建server_name報錯………………等等問題
檢查腳本：
@?/rdbms/admin/utltz_upg_check.sql
設置timezone_file腳本：
@?/rdbms/admin/utltz_upg_apply.sql
sqlnet.ora
NAMES.DIRECTORY_PATH= (TNSNAMES, ONAMES, HOSTNAME)
SQLNET.ALLOWED_LOGON_VERSION_CLIENT = 8
SQLNET.ALLOWED_LOGON_VERSION_SERVER = 8
SQLNET.ALLOWED_LOGON_VERSION_CLIENT = 10
SQLNET.ALLOWED_LOGON_VERSION_CLIENT = 11
SQLNET.ALLOWED_LOGON_VERSION_CLIENT = 12
--alter system set resource_manager_plan=default_plan;
grant administer database trigger to SYS ;
grant administer database trigger to system ;
--------最后cluster_database=true。