最近有個客戶需求是在某12.2版本的RAC環境上添加心跳網絡,順便考慮將之前的心跳網絡改為asm專用。我目前只有19c的RAC的測試環境(19c是12c的最終穩定版本),直接測試驗證下過程備忘。
1.測試環境概述
首先我的測試環境是這樣的:[grid@db195 ~]$ oifcfg getif
enp0s3 192.168.1.0 global public
enp0s8 10.10.1.0 global cluster_interconnect,asm
2.添加心跳網絡
主機層面添加新的網卡enp0s9,配置對應的IP地址后,可以將enp0s9添加為心跳網絡:[grid@db195 ~]$ oifcfg setif -global enp0s9/10.10.1.0:cluster_interconnect
[grid@db195 ~]$ oifcfg getif
enp0s3 192.168.1.0 global public
enp0s8 10.10.1.0 global cluster_interconnect,asm
enp0s9 10.10.1.0 global cluster_interconnect
3.修改asm網絡
如果確認需要將之前的設置為asm專用,可以再修改:[grid@db195 ~]$ oifcfg setif -global enp0s8/10.10.1.0:asm
[grid@db195 ~]$ oifcfg getif
enp0s3 192.168.1.0 global public
enp0s8 10.10.1.0 global asm
enp0s9 10.10.1.0 global cluster_interconnect
此時查看RAC其他節點的配置確保一樣。
4.重啟has集群
使用ifconfig -a查看新增網卡enp0s9的信息,還是沒有被HAIP接管的(沒有對應169網段的地址信息)enp0s9: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.10.1.196 netmask 255.255.255.0 broadcast 10.10.1.255
inet6 fe80::2549:d14d:7dfd:61c4 prefixlen 64 scopeid 0x20<link>
ether 08:00:27:59:1a:a6 txqueuelen 1000 (Ethernet)
RX packets 66 bytes 19094 (18.6 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 16 bytes 1170 (1.1 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
需要使用crsctl stop has關閉各節點集群:
[root@db193 ~]# /u01/app/19.3.0/grid/bin/crsctl stop has
[root@db195 ~]# /u01/app/19.3.0/grid/bin/crsctl stop has
[root@db193 ~]# /u01/app/19.3.0/grid/bin/crsctl start has
[root@db195 ~]# /u01/app/19.3.0/grid/bin/crsctl start has
5.檢查心跳狀態
重啟后,再次觀察ifconfig -a看到新增網卡enp0s9有對應的HAIP地址了,但是enp0s8沒有了,因為enp0s8只是用的asm:enp0s8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.10.1.195 netmask 255.255.255.0 broadcast 10.10.1.255
inet6 fe80::5a1e:e7de:fe1a:a286 prefixlen 64 scopeid 0x20<link>
ether 08:00:27:1d:85:d4 txqueuelen 1000 (Ethernet)
RX packets 38565 bytes 27624748 (26.3 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 69723 bytes 72018893 (68.6 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
enp0s9: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.10.1.196 netmask 255.255.255.0 broadcast 10.10.1.255
inet6 fe80::2549:d14d:7dfd:61c4 prefixlen 64 scopeid 0x20<link>
ether 08:00:27:59:1a:a6 txqueuelen 1000 (Ethernet)
RX packets 12510 bytes 7500503 (7.1 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 14723 bytes 17562072 (16.7 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
enp0s9:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 169.254.3.55 netmask 255.255.224.0 broadcast 169.254.31.255
ether 08:00:27:59:1a:a6 txqueuelen 1000 (Ethernet)
如果之前的enp0s8網卡沒有修改為asm專用,也就是asm,cluster_interconnect的話,那么enp0s8網卡就會保持有HAIP地址:
--設置enp0s8為asm,cluster_interconnect
[grid@db195 ~]$ oifcfg setif -global enp0s8/10.10.1.0:asm,cluster_interconnect
--重啟集群后,ifconfig -a就會發現enp0s8網卡保持有HAIP地址:
enp0s8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.10.1.195 netmask 255.255.255.0 broadcast 10.10.1.255
inet6 fe80::5a1e:e7de:fe1a:a286 prefixlen 64 scopeid 0x20<link>
ether 08:00:27:1d:85:d4 txqueuelen 1000 (Ethernet)
RX packets 41817 bytes 28559675 (27.2 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 89166 bytes 88323795 (84.2 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
enp0s8:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 169.254.7.148 netmask 255.255.240.0 broadcast 169.254.15.255
ether 08:00:27:1d:85:d4 txqueuelen 1000 (Ethernet)
enp0s9: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.10.1.196 netmask 255.255.255.0 broadcast 10.10.1.255
inet6 fe80::2549:d14d:7dfd:61c4 prefixlen 64 scopeid 0x20<link>
ether 08:00:27:59:1a:a6 txqueuelen 1000 (Ethernet)
RX packets 103509 bytes 77082525 (73.5 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 237436 bytes 290562629 (277.1 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
enp0s9:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 169.254.22.254 netmask 255.255.240.0 broadcast 169.254.31.255
ether 08:00:27:59:1a:a6 txqueuelen 1000 (Ethernet)
注意:如果新增enp0s9心跳網卡和之前心跳網段不一樣,比如10.10.2.195,還要注意和oifcfg setif enp0s9/10.10.2.0:cluster_interconnect,網段與新網卡的網路保持一致,相互匹配。
總結下這個事情就是先確認物理增加心跳網卡和心跳線,然后數據庫層面直接使用oifcfg可增加或修改,需要重啟has集群生效。