這個是測試庫的問題，但是過了兩天生產庫也出現同樣的問題了，所以大家徹查了一下原因。
目前了解的情況是與CPU核數過高、數據庫的並行參數（paraller_max_server）設置過高（256）等有關，ORACLE的算法在低版本有BUG（在cpu核數超過128、並行參數設置過高時容易觸發）。
Alert日志顯示在進行一個insert語句時觸發了該bug引起宕機。之后重啟時由於並行恢復參數設置問題，同樣觸發了BUG，導致無法啟動。經查詢metalink，通過降低CPU核數繞過該bug，完成啟動。
目前metalink上未找到當前版本的補丁，后續將開展升級整改。
那么我們就來看看paraller_max_server這個東西是怎么計算的？
metlink上記錄：
With 11.2.0.2 there is a new method to compute the default for PARALLEL_MAX_SERVERS.
In the Oracle Rdbms Reference Guide we find:
parallel_max_servers = PARALLEL_THREADS_PER_CPU * CPU_COUNT * concurrent_parallel_users * 5
In the formula, the value assigned to concurrent_parallel_users running at the default degree of parallelism on an
instance is dependent on the memory management setting.
- If automatic memory management is disabled (manual mode), then the value of concurrent_parallel_users is 1.
- If PGA automatic memory management is enabled, then the value of concurrent_parallel_users is 2.
- If global memory management or SGA memory target is used in addition to PGA automatic memory management,
then the value of concurrent_parallel_users is 4.
The value is capped by processes -15 (this is true for versions prior 11.2.0.2 as well).
As example we have the following values
parallel_threads_per_cpu = 2
cpu_count = 4
pga_aggregate_target = 500M
sga_target = 900M
processes = 150
parallel_max_servers = 2 * 4 * 4 * 5 = 160
parallel_max_servers = min( 150-15 , 160 ) = 135
So with these values we get a default of 135 for parallel_max_servers.
Note if the parallel_max_servers is reduced due to value of processes, then you see similar to the following in alert log (e.g. at instance start up):
Mon May 06 18:43:06 2013
Adjusting the default value of parameter parallel_max_servers
from 160 to 135 due to the value of parameter processes (150)
Starting ORACLE instance (normal)
所以如果這個參數太高，並行的進程就太大了，導數數據庫無法承受。
這個參數值可以通過下面的語句查詢：
select * from dba_hist_parameter b where b.parameter_name='parallel_max_servers' order by b.snap_id desc
從上面的內容看出oracle並行進程不能設置的太高。
詳解
Oracle數據庫並行操作，特別是在RAC環境，一定程度上能夠提升數據庫的性能，所以對相關的初始化參數的了解是必要的，這篇文章將根據實際的案例討論Oracle數據庫的部分並行參數。
Oracle數據庫相關的並行參數：
SQL> show parameter parallel
NAME TYPE VALUE
------------------------------------ ---------------------- --------------------
fast_start_parallel_rollback string LOW
parallel_adaptive_multi_user boolean TRUE
parallel_automatic_tuning boolean FALSE
parallel_degree_limit string CPU
parallel_degree_policy string MANUAL
parallel_execution_message_size integer 16384
parallel_force_local boolean FALSE
parallel_instance_group string
parallel_io_cap_enabled boolean FALSE
parallel_max_servers integer 135
parallel_min_percent integer 0
NAME TYPE VALUE
------------------------------------ ---------------------- --------------------
parallel_min_servers integer 0
parallel_min_time_threshold string AUTO
parallel_server boolean FALSE
parallel_server_instances integer 1
parallel_servers_target integer 64
parallel_threads_per_cpu integer 2
recovery_parallelism integer 0
下面是實際Oracle RAC環境下，Oracle並行參數的設置，我們將優先討論這些參數：
*.parallel_adaptive_multi_user=FALSE
*.parallel_execution_message_size=16384
*.parallel_max_servers=240
*.parallel_min_servers=0
*.parallel_threads_per_cpu=1
PARALLEL_ADAPTIVE_MULTI_USER
Property Description
Parameter type Boolean
Default value true
Modifiable ALTER SYSTEM
Range of values true | false
PARALLEL_ADAPTIVE_MULTI_USER, when set to true, enables an adaptive algorithm designed to improve performance in multiuser environments that use parallel execution. The algorithm automatically reduces the requested degree of parallelism based on the system load at query startup time. The effective degree of parallelism is based on the default degree of parallelism, or the degree from the table or hints, divided by a reduction factor.
當PARALLEL_ADAPTIVE_MULTI_USER參數設置為TRUE，啟用設計的適當算法，在多用戶環境下使用並行執行提升性能。這個算法基於查詢開始時的系統負載自動減少請求的並行度。有效的並行度是基於默認的並行度，或者來自表或HINT的並行度，通過減少系數進行分割。
The algorithm assumes that the system has been tuned for optimal performance in a single-user environment.
算法假定系統在單用戶環境下按照最優性能被調整。
Tables and hints use the default degree of parallelism.
表和HINT使用默認的並行度。
PARALLEL_MAX_SERVERS
Property Description
Parameter type Integer
Default value PARALLEL_THREADS_PER_CPU * CPU_COUNT * concurrent_parallel_users * 5
Modifiable ALTER SYSTEM
Range of values 0 to 3600
Oracle RAC Multiple instances can have different values.
Note:
This parameter applies to parallel execution in exclusive mode as well as in a Real Application Clusters environment.
PARALLEL_MAX_SERVERS specifies the maximum number of parallel execution processes and parallel recovery processes for an instance. As demand increases, Oracle Database increases the number of processes from the number created at instance startup up to this value.
PARALLEL_MAX_SERVERS指定實例最大並行執行進程和並行恢復進程數。隨着增長需求，Oracle數據庫需要增加進程數，從實例啟動時創建的數目到增長值。
In the formula, the value assigned to concurrent_parallel_users running at the default degree of parallelism on an instance is dependent on the memory management setting. If automatic memory management is disabled (manual mode), then the value of concurrent_parallel_users is 1. If PGA automatic memory management is enabled, then the value of concurrent_parallel_users is 2. If global memory management or SGA memory target is used in addition to PGA automatic memory management, then the value of concurrent_parallel_users is 4.
根據上面的公式，分配給concurrent_parallel_users的值，運行在實例的默認並行度依賴於內存管理設置。如果禁用自動內存管理（手動模式），那么concurrent_parallel_user的值是1，如果啟用PGA自動內存管理，那么concurrent_parallel_users的值是2。如果除了PGA自動內存管理外，還使用了全局內存管理或者SGA內存target，那么concurrent_parallel_users的值是4。
If you set this parameter too low, then some queries may not have a parallel execution process available to them during query processing. If you set it too high, then memory resource shortages may occur during peak periods, which can degrade performance.
如果設置這個參數過小，那么某些查詢在查詢過程中可能沒有並行執行進程活動。如果設置這個參數過大，那么在峰值期間內存資源可能不足，導致性能下降。
PARALLEL_MIN_SERVERS
Property Description
Parameter type Integer
Default value 0
Modifiable ALTER SYSTEM
Range of values 0 to value of PARALLEL_MAX_SERVERS
Oracle RAC Multiple instances can have different values.
Note:
This parameter applies to parallel execution in exclusive mode as well as in a Real Application Clusters environment.
PARALLEL_MIN_SERVERS specifies the minimum number of parallel execution processes for the instance. This value is the number of parallel execution processes Oracle creates when the instance is started.
PARALLEL_MIN_SERVERS指定實例並行執行進程數的最小值。這個值是實例在啟動時，Oracle創建的並行執行進程數。
PARALLEL_THREADS_PER_CPU
Property Description
Parameter type Integer
Default value Operating system-dependent, usually 2
Modifiable ALTER SYSTEM
Range of values Any nonzero number
Note:
This parameter applies to parallel execution in exclusive mode as well as in a Real Application Clusters environment.
PARALLEL_THREADS_PER_CPU specifies the default degree of parallelism for the instance and determines the parallel adaptive and load balancing algorithms. The parameter describes the number of parallel execution processes or threads that a CPU can handle during parallel execution.
PARALLEL_THREADS_PER_CPU指定實例默認的並行度，確定合適的並行和負載均衡算法。這個參數描述並行執行進程數，或者在並行執行期間CPU能處理的線程數。
The default is platform-dependent and is adequate in most cases. You should decrease the value of this parameter if the machine appears to be overloaded when a representative parallel query is executed. You should increase the value if the system is I/O bound.
默認值依賴於平台，在大多數情況下都是合適的。當執行一個典型的並行查詢時，服務器出現過載的情況，應該減少這個參數的值。如果系統在I/O的邊界應該增加這個值。
在並行參數方面，有以下最佳實踐：
確保監控活動並行服務器進程的數量並計算要應用於 PARALLEL_MIN_SERVERS 的平均值。可通過以下操作完成：
Select * from v$pq_syssstat;
Then: Get/save the value for row "Servers Highwater"
根據您的硬件情況優化 PARALLEL_MAX_SERVERS的值。最開始可以使用 (2 * ( 2 個線程 ) *(CPU_COUNT)) = 4 x CPU 計算，然后使用測試數據對更高的值重復測試。
考慮設置 FAST_START_PARALLEL_ROLLBACK。此參數可確定將有多少個進程用於事務恢復（在 redo 應用后執行）。為了確保在出現計划外故障后仍能獲得高效的工作負載，優化事務恢復顯得非常重要。只要系統不大量占用 CPU，最佳實踐是將此參數設置為值“HIGH”。這會導致 Oracle 使用四倍於 CPU 個數 (4 X cpu_count) 的並行進程進行事務恢復。此參數的默認值是“LOW”，或兩倍的 CPU 計數 (2 X cpu_count)。
對於 11gR2 之前的版本，將 PARALLEL_EXECUTION_MESSAGE_SIZE 從默認值（通常為 2048）增加到 8192。對於基於數據倉庫的系統（通過 PQ 傳輸大量數據），可以將其設置的更高。在版本 11gR2 中，PARALLEL_EXECUTION_MESSAGE_SIZE 的默認值是 16K，經證明，該值在大多數情況下都能夠滿足要求。
參考文章：《RAC 和 Oracle Clusterware 最佳實踐和初學者指南（平台無關部分） [ID 1526083.1]》
告警日志：
在某些數據庫啟動的時候還能從告警日志的最開始位置看到以下的信息：
Tue May 07 23:38:27 2013
Adjusting the default value of parameter parallel_max_servers
from 1280 to 985 due to the value of parameter processes (1000)
Starting ORACLE instance (normal)
出現此告警的原因是默認計算出的parallel_max_server的值1280超過了process的最大值1000，動態調整到小於process的值。
--end--