環境：3台虛擬機 RHEL 7.3 + Oracle RAC 11.2.0.4
問題現象：RAC運行正常，ASM磁盤組Normal冗余，節點主機重啟，offline狀態的asm disk短時間內可以直接online。
在《測試一體機ASM failgroup的相關問題處理》之前的文章，描述了重新添加磁盤的場景，其實如果故障發現及時（默認3.6h內），是可以直接online對應磁盤的，這時候現象類似如下：
SQL> select group_number, disk_number, name, path, failgroup, mode_status, voting_file from v$asm_disk order by 1, 2;
GROUP_NUMBER DISK_NUMBER NAME PATH FAILGROUP MODE_STATUS VO
------------ ----------- ------------------------------ ----------------------------------- ------------------------------ -------------- --
0 0 /dev/CELL01-data2 ONLINE N
0 1 /dev/CELL01-data1 ONLINE N
0 2 /dev/CELL01-crs1 ONLINE Y
1 0 CRS_0000 CRS_0000 OFFLINE N
1 1 CRS_0001 /dev/CELL02-crs2 CRS_0001 ONLINE Y
1 2 CRS_0002 /dev/CELL03-crs3 CRS_0002 ONLINE Y
2 0 DATA_0000 /dev/CELL03-data1 CELL03 ONLINE N
2 1 DATA_0001 /dev/CELL03-data2 CELL03 ONLINE N
2 2 DATA_0002 /dev/CELL02-data1 CELL02 ONLINE N
2 3 DATA_0003 /dev/CELL02-data2 CELL02 ONLINE N
2 4 DATA_0004 CELL01 OFFLINE N
2 5 DATA_0005 CELL01 OFFLINE N
12 rows selected.
這種情況就可以直接online對應磁盤:
SQL> alter diskgroup CRS online disk CRS_0000;
Diskgroup altered.
SQL> alter diskgroup DATA online disk DATA_0004,DATA_0005;
alter diskgroup DATA online disk DATA_0004,DATA_0005
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15282: ASM disk "DATA_0005" is not visible cluster-wide
ORA-15282: ASM disk "DATA_0004" is not visible cluster-wide
直接online對應磁盤如果遇到上面這個報錯，因為有其他節點沒有看到要online的磁盤，檢查其他節點：
[root@db02 ~]# ls -l /dev/CELL*
lrwxrwxrwx 1 root root 3 Dec 2 21:16 /dev/CELL01-crs1 -> sdc
lrwxrwxrwx 1 root root 3 Dec 2 21:16 /dev/CELL02-crs2 -> sdi
lrwxrwxrwx 1 root root 3 Dec 2 21:16 /dev/CELL02-data1 -> sdj
lrwxrwxrwx 1 root root 3 Dec 2 21:16 /dev/CELL02-data2 -> sdk
lrwxrwxrwx 1 root root 3 Dec 2 21:16 /dev/CELL03-crs3 -> sdd
lrwxrwxrwx 1 root root 3 Dec 2 21:16 /dev/CELL03-data1 -> sdf
lrwxrwxrwx 1 root root 3 Dec 2 21:16 /dev/CELL03-data2 -> sdh
此時先要確認lsscsi底層正常（如果不正常先解決iscsi層的問題，我這里是正常的）：
[root@db02 ~]# lsscsi
[1:0:0:0] cd/dvd VBOX CD-ROM 1.0 /dev/sr0
[2:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sda
[3:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sdb
[4:0:0:0] disk LIO-ORG disk1 4.0 /dev/sdc
[4:0:0:1] disk LIO-ORG disk2 4.0 /dev/sde
[4:0:0:2] disk LIO-ORG disk3 4.0 /dev/sdg
[5:0:0:0] disk LIO-ORG disk1 4.0 /dev/sdd
[5:0:0:1] disk LIO-ORG disk2 4.0 /dev/sdf
[5:0:0:2] disk LIO-ORG disk3 4.0 /dev/sdh
[6:0:0:0] disk LIO-ORG disk1 4.0 /dev/sdi
[6:0:0:1] disk LIO-ORG disk2 4.0 /dev/sdj
[6:0:0:2] disk LIO-ORG disk3 4.0 /dev/sdk
重新刷新udev規則，確認所有磁盤識別正確：
[root@db02 ~]# udevadm control --reload
[root@db02 ~]# udevadm trigger
[root@db02 ~]# ls -l /dev/CELL*
lrwxrwxrwx 1 root root 3 Dec 2 21:18 /dev/CELL01-crs1 -> sdc
lrwxrwxrwx 1 root root 3 Dec 2 21:18 /dev/CELL01-data1 -> sde
lrwxrwxrwx 1 root root 3 Dec 2 21:18 /dev/CELL01-data2 -> sdg
lrwxrwxrwx 1 root root 3 Dec 2 21:18 /dev/CELL02-crs2 -> sdi
lrwxrwxrwx 1 root root 3 Dec 2 21:18 /dev/CELL02-data1 -> sdj
lrwxrwxrwx 1 root root 3 Dec 2 21:18 /dev/CELL02-data2 -> sdk
lrwxrwxrwx 1 root root 3 Dec 2 21:18 /dev/CELL03-crs3 -> sdd
lrwxrwxrwx 1 root root 3 Dec 2 21:18 /dev/CELL03-data1 -> sdf
lrwxrwxrwx 1 root root 3 Dec 2 21:18 /dev/CELL03-data2 -> sdh
然后再次進行online磁盤成功：
SQL> alter diskgroup DATA online disk DATA_0004,DATA_0005;
Diskgroup altered.
這種可以直接online disk的情況就免去了ASM磁盤組rebalance的大量時間，所以這類問題發現后最好要及時聯系工程師進行處理。