當收到告警信息ORA-01652: unable to extend temp segment by 128 in tablespace xxxx 時，如何Troubleshooting ORA-1652這樣的問題呢？ 當然一般xxx是臨時表空間，也有可能是用戶表空間。
我們先來模擬一下這個情況，在兩個會話窗口執行下面SQL語句，這個視圖比較特殊（因為比較懶，不想去構造一個大量消耗臨時段的SQL，便使用手頭的一個案例腳本），它里面有一個DISTINCT操作會消耗TEMP表空間中大量的臨時段
SQL> select count(*) from v_ies_go_information;
開啟兩個會話窗口執行上面這個SQL，此時這兩個會話會耗大量臨時段，那么你用下面SQL語句就能捕獲到這個SQL，如下所示：
For 8.1.7 to 9.2
SELECT A.USERNAME, A.SID, A.SERIAL#, A.OSUSER, B.TABLESPACE, B.BLOCKS, C.SQL_TEXT
FROM V$SESSION A, V$SORT_USAGE B, V$SQLAREA C
WHERE A.SADDR = B.SESSION_ADDR
AND C.ADDRESS= A.SQL_ADDRESS
AND C.HASH_VALUE = A.SQL_HASH_VALUE
ORDER BY B.TABLESPACE, B.BLOCKS;
For 10.1 and above:
COL USERNAME FOR A16;
COL OSUSER FOR A16;
COL TABLESPACE FOR A10;
COL SQL_TEXT FOR A160;
SELECT A.USERNAME, A.SID, A.SERIAL#, A.OSUSER, B.TABLESPACE, B.BLOCKS, C.SQL_TEXT
FROM GV$SESSION A, GV$TEMPSEG_USAGE B, GV$SQLAREA C
WHERE A.SADDR = B.SESSION_ADDR
AND C.ADDRESS= A.SQL_ADDRESS
AND C.HASH_VALUE = A.SQL_HASH_VALUE
ORDER BY B.TABLESPACE, B.BLOCKS;
當然消耗臨時表空間的BLOCKS是一直變化的，下面只是其中一次查詢結果的截圖
當然這個也可以通過下面SQL查詢當前消耗TEMP臨時段的SQL_ID以及具體大小信息。這些信息都是實時變化的。
SQL> SELECT SQL_ID,SUM(BLOCKS) FROM GV$TEMPSEG_USAGE GROUP BY SQL_ID ORDER BY 2 DESC;
SQL_ID SUM(BLOCKS)
------------- -----------
cw4d8h5fudg6b 456704
SQL> SELECT TABLESPACE_NAME,TOTAL_BLOCKS,USED_BLOCKS,FREE_BLOCKS FROM V$SORT_SEGMENT;
TABLESPACE_NAME TOTAL_BLOCKS USED_BLOCKS FREE_BLOCKS
------------------------------- ------------ ----------- -----------
TEMPSCM2 1048320 506368 541952
SQL> SELECT TABLESPACE_NAME,TOTAL_BLOCKS,USED_BLOCKS,FREE_BLOCKS FROM V$SORT_SEGMENT;
TABLESPACE_NAME TOTAL_BLOCKS USED_BLOCKS FREE_BLOCKS
------------------------------- ------------ ----------- -----------
TEMPSCM2 1048320 1030144 18176
在另外一個窗口，不時執行下面SQL語句觀察臨時表空間的消耗使用情況，也能看到臨時表空間的消耗變化情況， 如下所示：
SELECT D.TABLESPACE_NAME,
SPACE "SUM_SPACE(M)",
BLOCKS "SUM_BLOCKS",
USED_SPACE "USED_SPACE(M)",
ROUND(NVL(USED_SPACE, 0) / SPACE * 100, 2) "USED_RATE(%)",
SPACE - USED_SPACE "FREE_SPACE(M)"
FROM (SELECT TABLESPACE_NAME,
ROUND(SUM(BYTES) / ( 1024 * 1024 ), 2) SPACE,
SUM(BLOCKS) BLOCKS
FROM DBA_TEMP_FILES
GROUP BY TABLESPACE_NAME) D,
(SELECT TABLESPACE,
ROUND(SUM(BLOCKS * 8192) / ( 1024 * 1024 ), 2) USED_SPACE
FROM V$SORT_USAGE
GROUP BY TABLESPACE) F
WHERE D.TABLESPACE_NAME = F.TABLESPACE(+)
AND D.TABLESPACE_NAME='TEMPSCM2'
但是很多時候，當我們收到告警日志的告警郵件時，其實該SQL語句其實已經結束了。就像我這個測試會話中，如果已經收到ORA-1652 錯誤提示，其實會話已經結束，返回錯誤提示了。
Mon Aug 07 22:23:40 CST 2017
ORA-1652: unable to extend temp segment by 128 in tablespace TEMPSCM2
Mon Aug 07 22:23:40 CST 2017
ORA-1652: unable to extend temp segment by 128 in tablespace TEMPSCM2
此時你用上面SQL其實已經不能捕獲到相關信息了，因為PMON已經釋放、回收了這些會話占用的臨時段，如下所示，測試環境已經查不到任何信息，如果是生產環境，那么有可能查到是不准確的信息（查到的是非引起問題的SQL）。上面只適合查詢當前臨時表空間的使用情況，而不適合用來追查已經出現的ORA-1652錯誤。
那么此時我們應該怎么辦呢？ 其實我們可以使用ASH報告來幫忙定位消耗了大量臨時段的SQL語句，如果收到ORA-01652告警后，最好及時生成一個快照，然后根據告警日志里面ORA-01652出現的時間，生成ASH報表，例如，此次試驗ORA-01652出錯的時間為22:23:40，那么我們生成22：20 ~ 22:25這個時間段的ASH報告。當然這個時間適當調整，盡量縮小范圍，可以精准定位問題SQL。
SQL> @?/rdbms/admin/ashrpt.sql
然后從ASH報告的TOP SQL里面找到對應的TOP SQL的SQL ID,然后使用awrsqrpt報告找到具體SQL的執行計划， 如下所示， 然后分析SQL是否耗用了大量的臨時段，當然生產環境肯定會復雜很多，TOP SQL里面肯定有多個，我們需要仔細甄別。這個分析也是一個耗時費力的體力活，所以上述ASH報告的時間段非常關鍵。
如下所示，通過awrsqrpt找到對應SQL_ID的具體執行計划，發現HASH UNIQUE這個DISTINCT操作使用了接近3G的臨時段排序。再加上一些其他的操作需要消耗臨時段，所以兩個會話的同時執行就引起了ORA-1652的錯誤。
使用ASH報告基本上都能定位到具體消耗大量臨時段的SQL語句，但是這個分析，有時候需要耗費很長時間，在How Can Temporary Segment Usage Be Monitored Over Time? (文檔 ID 364417.1)里面介紹了如何監控臨時段的使用情況。如下所示：
-- Create a table to hold your temporary space monitoring
-- 最好根據具體情況放入一個表空間，不要放入系統表空間
CREATE TABLE MONITOR_TEMP_SEG_USAGE
(
DATE_TIME DATE,
USERNAME VARCHAR2(30),
SID VARCHAR2(6),
SERIAL# VARCHAR2(6),
OS_USER VARCHAR2(30),
SPACE_USED NUMBER,
SQL_TEXT VARCHAR2(1000)
);
--創建存儲過程，將消耗臨時段超過閥值的SQL插入MONITOR_TEMP_SEG_USAGE
CREATE OR REPLACE PROCEDURE MONITOR_TEMP_SEG_USAGE_INSERT IS
BEGIN
INSERT INTO MONITOR_TEMP_SEG_USAGE
SELECT sysdate,a.username, a.sid, a.serial#, a.osuser, b.blocks, c.sql_text
FROM v$session a, v$sort_usage b, v$sqlarea c
WHERE b.tablespace = 'TEMP' --輸入具體臨時表空間
AND a.saddr = b.session_addr
AND c.address= a.sql_address
AND c.hash_value = a.sql_hash_value
AND b.blocks*(select block_size from dba_tablespaces where tablespace_name = b.tablespace) > 1024;
COMMIT;
END;
/
--創建作業,每5分鍾運行一次，捕獲消耗臨時段超過閥值的SQL語句。
SQL> SELECT JOB FROM DBA_JOBS;
JOB
----------
141
142
BEGIN
DBMS_JOB.ISUBMIT(JOB => 20,
WHAT => 'MONITOR_TEMP_SEG_USAGE_INSERT;',
NEXT_DATE => SYSDATE,
INTERVAL => 'SYSDATE + (5/1440)');
COMMIT;
END;
/
另外，ORACLE 11.2 或后面的版本，可以使用下面SQL 語句查詢出消耗臨時段超過一定閥值的SQL語句，這樣基本也能通過控制條件找到引起ORA-01652錯誤的SQL
SELECT SQL_ID,MAX(TEMP_SPACE_ALLOCATED)/(1024*1024*1024) GIG
FROM DBA_HIST_ACTIVE_SESS_HISTORY
WHERE
SAMPLE_TIME > SYSDATE-2 AND
TEMP_SPACE_ALLOCATED > (1024*1024*1024)
GROUP BY SQL_ID ORDER BY SQL_ID;
上面的一些介紹，基本已經涵蓋如何Troubleshooting ORA-1652這個問題了，那么下面介紹一下ORA-1652出現的場景，其實這個對理解ORA-1652出現的前因后果非常有幫助！
EXAMPLE 1:
Temporary tablespace TEMP is being used and is 50gb in size (a recommended minimum for 11g)
TIME 1 : Session 1 starts a long running query
TIME 2 : Session 2 starts a query and at this point in time Session 1 has consumed 48gb of TEMP's free space
TIME 3 : Session 1 and Session 2 receive an ORA-1652 because the tablespace has exhausted of of its free space
Both sessions fail .. and all temp space used by the sessions are freed (the segments used are marked FREE for reuse)
TIME 4 : SMON cleans up the temporary segments used by Session 1 and Session 2 (deallocates the storage)
TIME 5 : Queries are run against the views V$SORTSEG_USAGE or V$TEMSEG_USAGE and V$SORT_SEGMENT ... and it is found that no space is being used (this is normal)
EXAMPLE 2:
Permanent tablespace INDEX_TBS is being used and has 20gb of space free
TIME 1 : Session 1 begins a CREATE INDEX command with the index stored in INDEX_TBS
TIME 2 : Session 1 exhausts all of the free space in INDEX_TBS as a result the CREATE INDEX abends
TIME 3 : SMON cleans up the temporary segments that were used to attempt to create the index
TIME 4 : Queries are run against the views V$SORTSEG_USAGE or V$TEMSEG_USAGE ... and it is found that the INDEX_TBS has no space used (this is normal)
In some cases, you may find that the ORA-1652 is not reported for a temporary tablespace, but a permanent one. This is not an abnormal behaviour and it can occur for example while creating or dropping objects like tables and indexes in permanent tablespaces. Reference : Note 19047.1 - OERR: ORA 1652 unable to extend temp segment by %s in tablespace %s
In such cases the following note will be of use :
Note 100492.1 - ORA-01652: Estimate Space Needed to CREATE INDEX
If the tablespace in which the TEMPORARY segment resides is of type PERMANENT, also check that the following events are not set in the initialization parameter file:
event="10061 trace name context forever, level 10"
event="10269 trace name context forever, level 10"
If they are set, unset them and restart database.
These two events prevent SMON from cleaning up.
Reference : Note 1039341.6 - Temporary Segments Are Not Being De-Allocated After a Sort
最后就是要給出一個解決方案，對於ORA-01652 這個錯誤有兩個解決方案：
1： 如果臨時表空間確實比較小，那么必須擴展臨時表空間，增加臨時數據文件或設置現有臨時數據文件自動擴展。
2： 優化消耗大量臨時段的SQL語句。減少臨時段的消耗。
另外在RAC環境中，情況又有所不同，可以參考NOTE:280578.1 - Troubleshooting ORA-1652 Errors in RAC
參考資料：
https://support.oracle.com/epmos/faces/DocumentDisplay?_afrLoop=144013041361565&id=793380.1&displayIndex=4&_afrWindowMode=0&_adf.ctrl-state=sucf6uzjm_402
https://support.oracle.com/epmos/faces/DocumentDisplay?_afrLoop=147317477297631&id=100492.1&_afrWindowMode=0&_adf.ctrl-state=sucf6uzjm_672
https://support.oracle.com/epmos/faces/DocumentDisplay?_afrLoop=147331987541915&id=793380.1&_afrWindowMode=0&_adf.ctrl-state=sucf6uzjm_721