Oracle啟動報錯ORA-27102解決

環境：RHEL5.5 + Oracle 10.2.0.4
此錯誤一般是因為數據庫的初始化參數文件的內存設置不當導致。本例是因為操作系統參數設置問題導致。

1. 當前現象：Oracle啟動報錯ORA-27102
2. 檢查各參數的配置情況
3. 定位解決問題
4. 延伸總結

1.當前現象：Oracle啟動報錯ORA-27102

``` [oracle@JYDB1 ~]$ sqlplus / as sysdba

SQL*Plus: Release 10.2.0.4.0 - Production on 星期四 7月 30 19:55:10 2015

Copyright (c) 1982, 2007, Oracle. All Rights Reserved.

已連接到空閑例程。

SQL> startup
ORA-27102: out of memory
Linux-x86_64 Error: 28: No space left on device

<h1 id="2">2.檢查各參數的配置情況</h1>
## 2.1 根據當前的spfile文件創建pfile文件 ##

SQL> create pfile='/tmp/pfile0730.bak' from spfile;

文件已創建。

查看當前的初始化參數配置信息：

[oracle@JYDB1 dbs]$ more /tmp/pfile0730.bak
JYZHAO.__db_cache_size=10938744832
JYZHAO.__java_pool_size=117440512
JYZHAO.__large_pool_size=16777216
JYZHAO.__shared_pool_size=1442840576
JYZHAO.__streams_pool_size=33554432
*.audit_file_dest='/opt/app/oracle//admin/JYZHAO/adump'
*.background_dump_dest='/opt/app/oracle//admin/JYZHAO/bdump'
*.compatible='10.2.0.3.0'
*.control_files='/usr3/oradata1/sysdata/control_file/control01.ctl','/usr3/oradata1/sysdata/control_file/control02.ctl','
/usr3/oradata1/sysdata/control_file/control03.ctl'
*.core_dump_dest='/opt/app/oracle//admin/JYZHAO/cdump'
*.db_block_size=8192
*.db_domain=''
*.db_file_multiblock_read_count=16
*.db_files=2048
*.db_name='JYZHAO'
*.dispatchers='(PROTOCOL=TCP) (SERVICE=JYZHAOXDB)'
*.job_queue_processes=10
*.nls_language='SIMPLIFIED CHINESE'
*.nls_territory='CHINA'
*.open_cursors=300
*.pga_aggregate_target=1572864000
*.processes=600
*.remote_login_passwordfile='EXCLUSIVE'
*.resource_limit=TRUE
*.sessions=665
*.sga_target=12582912000
*.undo_management='AUTO'
*.undo_tablespace='UNDOTBS1'
*.user_dump_dest='/opt/app/oracle//admin/JYZHAO/udump'

可以看到，sga=12G，pga=1.5G，
## 2.2 用pfile文件啟動得到相同報錯 ##
startup pfile='/tmp/pfile0730.bak'

SQL> startup pfile='/tmp/pfile0730.bak'
ORA-27102: out of memory
Linux-x86_64 Error: 28: No space left on device
SQL> !

## 2.3 檢查主機的內存，/dev/shm，ipcs ##
### 2.3.1 內存空閑充足 ###

[oracle@JYDB1 10.2.0]$ free -g
total used free shared buffers cached
Mem: 31 0 30 0 0 0
-/+ buffers/cache: 0 31
Swap: 31 0 31

### 2.3.2 /dev/shm設置為16G，符合當前需求 ###

[oracle@JYDB1 10.2.0]$ df -h /dev/shm
文件系統 容量 已用 可用 已用% 掛載點
tmpfs 16G 0 16G 0% /dev/shm

### 2.3.3 ipcs -a也沒有未釋放的共享內存 ###

[oracle@JYDB1 10.2.0]$ ipcs -a

------ Shared Memory Segments --------
key shmid owner perms bytes nattch status

------ Semaphore Arrays --------
key semid owner perms nsems

------ Message Queues --------
key msqid owner perms used-bytes messages

查到這里我們發現系統的硬件完全可以支持sga=12G，pga=1.5G的配置。
但此時實驗了下，大概是只能以sga=6.5G的大小啟動數據庫。sga再大都會報錯：ORA-27102。

## 2.4 檢查系統配置文件/etc/sysctl.conf ##
more /etc/sysctl.conf

fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmax = 25769803776
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048586

可以看到是按照11g 安裝文檔建議配置項配置的，其中kernel.shmmax是根據主機內存的75%計算來的。其他參數沒有改變。
因為這里的環境是Oracle 10g，所以我們還是按照10g官檔的建議，修改為10g版本的安裝文檔配置項：

kernel.shmall = 2097152
kernel.shmmax = 25769803776
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 65536
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default = 262144
net.core.rmem_max = 262144
net.core.wmem_default = 262144
net.core.wmem_max = 262144

sysctl -p 生效配置后，此時嘗試啟動數據庫，結果很不幸，依舊報錯ORA-27102。

<h1 id="3">3.定位解決問題</h1>
Google了一下，發現還有可能是kernel.shmall這個參數導致的。
查看當前的shmmax和shmall配置

[root@JYDB1 ~]# cat /proc/sys/kernel/shmmax
25769803776
[root@JYDB1 ~]# cat /proc/sys/kernel/shmall
2097152

注釋掉新增的kernel.shmall參數的配置（原/etc/sysctl.conf中有kernel.shmall的配置值為4294967296）

kernel.shmall = 2097152

kernel.shmmax = 25769803776
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 65536
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default = 262144
net.core.rmem_max = 262144
net.core.wmem_default = 262144
net.core.wmem_max = 262144

再次查看當前的shmmax和shmall配置

[root@JYDB1 ~]# cat /proc/sys/kernel/shmmax
25769803776
[root@JYDB1 ~]# cat /proc/sys/kernel/shmall
4294967296

此時重啟數據庫就可以sga=12G，pga=1.5G正常啟動，那么之前的報錯就是因為kernel.shmall參數的配置項過小導致了。

<h1 id="4">4.延伸總結</h1>
Oracle官檔中，無論是10g還是11g，kernel.shmall參數的配置建議就是kernel.shmall = 2097152，那么在這里的環境有什么問題呢？

**下面我們來弄清楚kernel.shmall這個參數的意義**

[root@JYDB1 ~]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.5 (Tikanga)

在RedHat 5的[文檔](https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/5/html/Tuning_and_Optimizing_Red_Hat_Enterprise_Linux_for_Oracle_9i_and_10g_Databases/sect-Oracle_9i_and_10g_Tuning_Guide-Setting_Shared_Memory-Setting_SHMALL_Parameter.html)中，關於kernel.shmall這個參數有這樣一段描述：

> **7.3. Setting SHMALL Parameter**
> This parameter sets the total amount of shared memory pages that can be used system wide. Hence, SHMALL should always be at least ceil(shmmax/PAGE_SIZE).
> The default size for SHMALL in Red Hat Enterprise Linux 2.1, 3, 4 and 5 is 2097152 which is also Oracle's recommended minimum setting for 9i and 10g on x86 and x86-64 platforms. In most cases this setting should be sufficient since it means that the total amount of shared memory available on the system is 2097152\*4096 bytes (shmall\*PAGE_SIZE) which is 8 GB. PAGE_SIZE is usually 4096 bytes unless you use Chapter 14, Large Memory Optimization, Big Pages, and Huge Pages which supports the configuration of larger memory pages.

**查看系統默認的PAGE_SIZE值：**

[root@JYDB1 ~]# getconf PAGESIZE
4096

計算shmmax/PAGE_SIZE的值`shmmax/PAGE_SIZE=25769803776/4096=6291456`, 
這樣問題根本原因找到了，在當前環境，kernel.shmall的值至少應該被設置為6291456。

**總結：**Oracle部署過程中，還要對建議的系統參數值有一些深入的了解。有些官檔建議的參數可能不符合實際情況，這時候可以結合系統主機的文檔來綜合定位問題。