早上收到一台Linux服務器磁盤告警郵件以及監控告警日志程序發來的郵件。檢查過后，發現Linux服務器中一個分區沒有空間了。主要原因是由於昨晚程序員做升級時，產生了大量的歸檔日志，導致聯機重做日志無法歸檔，出現下面錯誤：
363 | Wed Mar 19 02:33:16 2014 | ORA-00312: online log 2 thread 1: '/u01/app/oracle/oradata/mes/redo02.log'
369 | Wed Mar 19 02:34:16 2014 | ORA-16014: log 2 sequence# 75999 not archived, no available destinations
370 | Wed Mar 19 02:34:16 2014 | ORA-00312: online log 2 thread 1: '/u01/app/oracle/oradata/mes/redo02.log'
373 | Wed Mar 19 02:34:16 2014 | ORA-16014: log 2 sequence# 75999 not archived, no available destinations
374 | Wed Mar 19 02:34:16 2014 | ORA-00312: online log 2 thread 1: '/u01/app/oracle/oradata/mes/redo02.log'
383 | Wed Mar 19 02:39:16 2014 | ORA-19502: write error on file "/u03/flash_recovery_area/mes/archivelog/2014_03_19/o1_mf_1_75999_9lk4pngn_.arc", blockno 108545 (blocksize=512)
384 | Wed Mar 19 02:39:16 2014 | ORA-27072: File I/O error
由於當時比較急，於是手工刪除了昨天的歸檔日志，釋放了大量空間，結果發現告警日志出現了如下錯誤：
ORA-19815: WARNING: db_recovery_file_dest_size of 42949672960 bytes is 100.00% used, and has 7684830208 remaining bytes available.
Wed Mar 19 08:48:48 2014
************************************************************************
You have following choices to free up space from flash recovery area:
1. Consider changing RMAN RETENTION POLICY. If you are using Data Guard,
then consider changing RMAN ARCHIVELOG DELETION POLICY.
2. Back up files to tertiary device such as tape using RMAN
BACKUP RECOVERY AREA command.
3. Add disk space and increase db_recovery_file_dest_size parameter to
reflect the new space.
4. Delete unnecessary files using RMAN DELETE command. If an operating
system command was used to delete files, then use RMAN CROSSCHECK and
DELETE EXPIRED commands.
************************************************************************
Wed Mar 19 08:48:54 2014
Thread 1 advanced to log sequence 76076 (LGWR switch)
出現這個錯誤時因為：當手工刪除了歸檔日志以后，Rman備份會檢測到日志缺失，從而無法進一步繼續執行。所以此時需要手工執行crosscheck過程，之后Rman備份可以恢復正常。
如上所示，Oracle 提供了4中方法解決這個問題：
You have following choices to free up space from flash recovery area:
1. Consider changing RMAN RETENTION POLICY. If you are using Data Guard,
then consider changing RMAN ARCHIVELOG DELETION POLICY.
2. Back up files to tertiary device such as tape using RMAN
BACKUP RECOVERY AREA command.
3. Add disk space and increase db_recovery_file_dest_size parameter to
reflect the new space.
4. Delete unnecessary files using RMAN DELETE command. If an operating
system command was used to delete files, then use RMAN CROSSCHECK and
DELETE EXPIRED commands.
翻譯如下：
你可以采用下面選項釋放閃回恢復區的空間：
1：考慮修改RMAN備份保留策略，如果你使用的Data Guard，應該考慮修改RMAN歸檔日志刪除策略。
2：使用RMAN BACKUP RECOVERY AREA命令將備份至於第三方設備(例如磁帶)
3：增加磁盤空間並增加db_recovery_file_dest_size大小。
1: SQL> show parameter db_recovery_file_dest
2:
3: NAME TYPE VALUE
4:
5: ------------------------------------ ----------- ------------------------------
6:
7: db_recovery_file_dest string /u03/flash_recovery_area
8:
9: db_recovery_file_dest_size big integer 40G
10:
11: SQL>alter system set db_recovery_file_dest_size=48G scope=both;
12:
4: 使用RMAN DELETE命令刪除不需要的文件，如果使用如果操作系統命令來刪除文件，則必須使用RMAN CROSSCHECK和
DELETE EXPIRED 的命令
$ rman target /
Recovery Manager: Release 10.2.0.4.0 - Production on Wed Mar 19 14:51:09 2014
Copyright (c) 1982, 2007, Oracle. All rights reserved.
connected to target database: EPPS (DBID=2179379665)
RMAN> crosscheck archivelog all;
RMAN> delete expired archivelog all;
RMAN> delete obsolete;
參考資料：http://www.eygle.com/archives/2004/12/rman_crosscheck.html