oracle index build online與offline
測試環境為oracle 11.2.0.4
--sql test SQL> conn test/test create table test.rb_test(id number,con varchar2(20)); begin for i in 1..1000000 loop insert into test.rb_test values(i,'ok'); commit; end loop; end; / SQL> select count(*) from test.rb_test; COUNT(*) ---------- 1000000 SQL> create index idx_rb_test on test.rb_test(id);
1 測試默認的rebuild,即rebuild offline
SQL> alter index idx_rb_test rebuild; Index altered. SQL> select sid,type,id1,id2,lmode,request,block from v$lock where type in('DL','TM','TX'); SQL> / SID TY ID1 ID2 LMODE REQUEST BLOCK ---------- -- ---------- ---------- ---------- ---------- ---------- 133 DL 87286 0 3 0 0 133 DL 87286 0 3 0 0 133 TM 87286 0 4 0 0 133 TX 524318 7858 6 0 0 這時候持有的LMODE 4的鎖，即在rebuild 期間我們不能執行DML 操作
2 對表進行dml操作，不提交，在rebuild
--sesion 1 SQL> update rb_test set con='hello test!' where id=1828; 1 row updated. --session 2 SQL> alter index idx_rb_test rebuild; ##這時候rebuild報錯 alter index idx_rb_test rebuild * ERROR at line 1: ORA-00054: resource busy and acquire with NOWAIT specified or timeout expired --session 1 SQL> commit; Commit complete. --session 2 SQL> / ##執行成功 Index altered
3 在索引rebuild 期間，我們去update table，並查看v$lock
--session 1 SQL> set timing on SQL> update rb_test set con='hello test!' where id=1830; 1 row updated. Elapsed: 00:00:00.00 SQL> commit; Commit complete. Elapsed: 00:00:00.01 SQL> alter index idx_rb_test rebuild; Index altered. Elapsed: 00:00:08.74 --session 2 SQL> update rb_test set con='hello test!' where id=1830; ##會被133阻塞，該144等待REQUEST mode為3的lock 2 rows updated. Elapsed: 00:00:07.67 --session 3 SQL> / SID TY ID1 ID2 LMODE REQUEST BLOCK ---------- -- ---------- ---------- ---------- ---------- ---------- 133 DL 87286 0 3 0 0 133 DL 87286 0 3 0 0 144 TM 87286 0 0 3 0 ---REQUEST mode為3的lock 133 TM 87286 0 4 0 1 133 TX 458785 17460 6 0 0 Elapsed: 00:00:00.10
##session1 執行完成之后，只有session2的dml操作，持有2個鎖，對表加模式為3的tm鎖，對數據行的模式為6的tx鎖
SQL> / SID TY ID1 ID2 LMODE REQUEST BLOCK ---------- -- ---------- ---------- ---------- ---------- ---------- 144 TM 87286 0 3 0 0 144 TX 262164 17266 6 0 0 Elapsed: 00:00:00.07
4 rebuild online
step 1: update rb_test set con='hello test!' where id=1833; no commit;
step 2: rebuild online
step 3: update
SQL> select sys_context('userenv','sid') from dual;
--session 1,sid=133
SQL> update rb_test set con='hello test!' where id=1833; 2 rows updated. Elapsed: 00:00:00.01 SQL> select sid,type,id1,id2,lmode,request,block from v$lock where type in('DL','TM','TX'); SID TY ID1 ID2 LMODE REQUEST BLOCK ---------- -- ---------- ---------- ---------- ---------- ---------- 133 TM 87286 0 3 0 0 133 TX 65554 17262 6 0 0 Elapsed: 00:00:00.06 --session 2,sid=144 SQL> alter index idx_rb_test rebuild online; ##hang住 --session 3，sid=146 SQL> update rb_test set con='hello test!' where id=1837; 2 rows updated. 這里session 3 順利執行，沒有什么鎖等待（11.2.0.4） 查看鎖情況 SQL> / SID TY ID1 ID2 LMODE REQUEST BLOCK ---------- -- ---------- ---------- ---------- ---------- ---------- 144 TX 65554 17262 0 4 0 ##這里144即session2在等待請求鎖為4的鎖 144 DL 87286 0 3 0 0 144 DL 87286 0 3 0 0 144 TM 87286 0 2 0 0 144 TM 87308 0 4 0 0 133 TM 87286 0 3 0 0 144 TX 131102 30599 6 0 0 133 TX 65554 17262 6 0 1 8 rows selected.
當前有大量事務在操作，這時候去rebuild online，這個操作就會需要去拿TX 4的lock，
在oracle 11g之前，這個rebuild online之后的dml操作都會鎖等待，之后的事務沒有辦法進行，解決方法就是找到阻止rebuild online的拿到想tx 4的鎖。
把它kill掉。這樣rebuild online操作就會繼續執行，后續的dml操作就可以繼續。
這點和rebuild offline的區別，后者直接報錯返回ORA-00054: resource busy and acquire with NOWAIT specified or timeout expired
--session 1 提交 SQL> commit; Commit complete. Elapsed: 00:00:00.01 SQL> select sid,type,id1,id2,lmode,request,block from v$lock where type in('DL','TM','TX'); SID TY ID1 ID2 LMODE REQUEST BLOCK ---------- -- ---------- ---------- ---------- ---------- ---------- 144 TX 393247 22992 0 4 0 144 DL 87286 0 3 0 0 144 DL 87286 0 3 0 0 146 TM 87286 0 3 0 0 144 TM 87286 0 2 0 0 144 TM 87308 0 4 0 0 146 TX 393247 22992 6 0 1 144 TX 131102 30599 6 0 0 133支持的鎖已經釋放掉，144等待146的tx 4的鎖 --session 3 提交 SQL> commit; Commit complete. Elapsed: 00:00:00.00 --sesion 2完成 SQL> alter index idx_rb_test rebuild online; Index altered. Elapsed: 00:02:29.98 由此可見rebuild online 是個需要謹慎使用的命令
引用 參考
--https://blog.csdn.net/tianlesoftware/article/details/6538928
--http://www.itpub.net/thread-1445427-1-1.html
kill 掉正在rebuild online 的session。 可能會導致在下次rebuild index或者drop，analyze 的時候報ORA-08104的錯誤。 因為在異常終止online rebuild操作的時候，oracle沒來得及清理相應的臨時段和標志位，系統認為online rebuild操作還在進行造成的 SQL> alter index idx_rb_test rebuild online; ##CTRL+C ^Calter index idx_rb_test rebuild online * ERROR at line 1: ORA-01013: user requested cancel of current operation Elapsed: 00:00:03.08 SQL> exec dbms_stats.gather_table_stats('TEST','RB_TEST'); PL/SQL procedure successfully completed. Elapsed: 00:00:01.09 SQL> alter index idx_rb_test rebuild online; Index altered. Elapsed: 00:00:03.99 SQL> / ##執行期間用另外的session kill掉該session alter index idx_rb_test rebuild online * ERROR at line 1: ORA-00028: your session has been killed ORA-00028: your session has been killed --sesion 2 SQL> alter system kill session '133,56593'; System altered. Elapsed: 00:00:01.01 SQL> conn test/test Connected. SQL> alter index idx_rb_test rebuild online; Index altered. Elapsed: 00:00:05.58 還是沒有報錯 SQL> alter index idx_rb_test rebuild online; ##這次關閉ssh 再次連接執行 SQL> conn test/test Connected. SQL> alter index idx_rb_test rebuild online; Index altered. SQL> 還是沒有報錯 如果報錯的話 ERROR at line 1: ORA-08104: this index object 53367 is being online built or rebuilt SQL> select obj#,flags from ind$ where obj#=53367; OBJ# FLAGS ---------- ---------- 53367 514
sql.bsq 是個總的說明，在dcore.bsq 里找到了ind$的創建SQL： /* mutable flags: anything permanent should go into property */ /* unusable (dls) : 0×01 */ /* analyzed : 0×02 */ /* no logging : 0×04 */ /* index is currently being built : 0×08 */ /* index creation was incomplete : 0×10 */ /* key compression enabled : 0×20 */ /* user-specified stats : 0×40 */ /* secondary index on IOT : 0×80 */ /* index is being online built : 0×100 */ /* index is being online rebuilt : 0×200 */ /* index is disabled : 0×400 */ /* global stats : 0×800 */ /* fake index(internal) : 0×1000 */ /* index on UROWID column(s) : 0×2000 */ /* index with large key : 0×4000 */ /* move partitioned rows in base table : 0×8000 */ /* index usage monitoring enabled : 0×10000 */ 這里的0×200 等是十六進制來表示的。 Flags 是514， 其16進制是是202，514=0×202，表示該索引狀態為index is being online rebuilt : 0×200 + analyzed : 0×02
SQL> select to_char(514,'xxxxxxxxxxxxxxx') from dual; TO_CHAR(514,'XXX ---------------- 202 在上面，我們說減去512. 512 的16進制是200. 對應的是：/* index is being online rebuilt : 0×200 */。 所以，我們在rebuild的時候，會對flags 加上512. MOS 803008.1 上的說明： SMON should cleanup the failed online index rebuild operation and so correct this. However, if the table is highly active with transactions, SMON may not be able to get the required lock and so the index will not get cleaned up. In such situations, you can manually cleanup the failed index rebuild using the DBMS_REPAIR.ONLINE_INDEX_CLEAN procedure. To do this, if activity on the problem table can be stopped, then simply execute: connect / as sysdba select dbms_repar.online_index_clean(<problem index object_id>) from dual; exit select dbms_repair.online_index_clean(53367) from dual; DECLARE RetVal BOOLEAN; OBJECT_ID BINARY_INTEGER; WAIT_FOR_LOCK BINARY_INTEGER; BEGIN OBJECT_ID := 53367; WAIT_FOR_LOCK := NULL; RetVal := SYS.DBMS_REPAIR.ONLINE_INDEX_CLEAN (); COMMIT; END; / select obj#,flags from ind$ where obj#=53367; OBJ# FLAGS ---------- ---------- 53367 2 在執行clean命令的時候，可能會遇到： ORA-00054: resource busy and acquire with NOWAIT specified 多執行幾次就ok了。 應該也是和這個鎖有關。
========
- Online Index rebuild takes a long time.
- ONLINE INDEX REBUILD SCANS THE BASE TABLE AND NOT THE INDEX
Symptoms:
=========
Performance issues while rebuilding very large indexes.
- The offline rebuilds of their index is relatively quick -finishes in 15 minutes.
- Issuing index rebuild ONLINE statement => finishes in about an hour.
- This behavior of ONLINE index rebuilds makes it a non-option for large tables
as it just takes too long to scan the table to rebuild the index. The offline may not be feasible due to due to the 24/7 nature of the database.
- This may be a loss of functionality for such situations.
- If we attempt to simultaneously ONLINE rebuild the same indexes we may encounter hanging behavior indefinitely (or more than 6 hours).
DIAGNOSTIC ANALYSIS:
--------------------
We can trace the sessions rebuilding the indexes with 10046 level 12.
Comparing the IO reads for the index-rebuild and the index-rebuild-online reveals the following:
-ONLINE index rebuilds
It scans the base table and it doesn't scan the blocks of the index.
-OFFLINE index rebuilds
It scans the index for the build operation.
- This behaviour is across all versions.
Cause
Cause/Explanation
=============
When you rebuild index online,
- it will do a full tablescan on the base table.
- At the same time it will maintain a journal table for DML data, which has
changed during this index rebuilding operation.
So it should take longer time, specially if you do lots of DML on the same table,while rebuilding index online.
On the other hand, while rebuilding the index without online option, Oracle will grab the index in X-mode and rebuild a new index segment by
selecting the data from the old index. So here we are
- not allowing any DML on the table hence there is no journal table involved
- and it is doing an index scan
Hence it will be pretty fast.
Fix
Solution/Conclusion:
===========
- The ONLINE index rebuild reads the base table, and this is by design.
- Rebuilding index ONLINE is pretty slow.
- Rebuilding index offline is very fast, but it prevents any DML on the base table.
-- rebuild index online的時候，會選擇全表掃描，同時會維護一個中間日志表，用來記錄在rebuild 期間的增量數據，原理類似於物化視圖日志，
日志表是一個索引組織表(IOT)，這張中間表只有插入，不會有刪除和修改操作，而且只有主鍵條件查詢，正是IOT最合適的場景。
--rebuild offline時，選擇的6模式的X 鎖，它根據old index 來rebuild。 因此不允許進行DML，也就沒有中間表。因此也比較塊。
DML操作一般要加兩個鎖，一個是對表加模式為3的TM鎖，一個是對數據行的模式為6的TX鎖。只要操作的不是同一行數據，是互不阻塞的
在rebuild index online 的開始和結束階段時，需要短暫的對表持有模式為4的TM鎖的，當獲取到4級別的鎖之后，才降為2級。
如果rebuild online一直沒獲取到4級別的鎖,那么相關的DML全部產生等待。 在執行期間只持有模式2的TM鎖，不會阻塞DML操作。
在Oracle 11g之后,oracle做了特殊處理,后續的dml不會被rebuild online的4級別鎖阻塞.
所以如果在執行rebuild index online前長事務，並且並發量比較大，則一旦執行alter index rebuild online，可能因為長事務阻塞，
可能導致系統瞬間出現大量的鎖，對於壓力比較大的系統，這是一個不小的風險。這是需要迅速找出導致阻塞的會話kill掉，
rebuild index online一旦執行，不可輕易中斷，否則可能遇到ORA-08104。