一、疑問
Oracle OGG是如何正確的進行數據同步呢？初始化一個表,完成一個數據的同步？
1)假設同步的表有靜態時間2小時;
2小時內，通過db link,exp,expdp 數據導入在目標庫新建表,完成第一次的數據初始化;
隨后，源端目標端使用OGG追加這個表進行數據同步；
2)假設同步的表數據隨時可能修改；
一般選擇，源端對表開啟最小補充日志,源端指定scn導出，目標端導入后,完成數據初始化;
隨后，目標端OGG指定scn,復制進程級別或者表級別進行csn過濾,過濾ogg復制進程trail csn ~ 源端導出scn之間的數據變換！
如下是MOS說明，通過csn完成數據同步，比使用參數handlecollision可靠！
Oracle GoldenGate Best Practices: Instantiation from an Oracle Source Database (Doc ID 1276058.1) This document is an introduction to Oracle GoldenGate’s best practices and guidelines for instantiation of a target database from
an Oracle Source Database. This method will allow the customer to start Replicat at a specific point in the trail file created by
Extract by using CSN number so the Replicat will not hit any "duplicate" records. This is a much better initial load methodology than
the traditional "handlecollision" solution.
3）我們認為數據都可能改變，但是不想使用方法二？ 咋整？
Oracle對OGG12.2 又來新特性了，使用參數，Oracle自動追蹤數據泵導出的scn，OGG啟動時查詢目標庫的信息自動進行過濾！
DBOPTIONS 參考 https://blog.csdn.net/diaokan1920/article/details/101762900 https://blog.dbi-services.com/oracle-data-pump-integration-for-table-instantiation-with-oracle-golden-gate/ ENABLE_INSTANTIATION_FILTERING Valid for Oracle. Enables automatic per table instantiation CSN filtering on tables imported using Oracle data pump or manually instantiated
using the SET_INSTANTIATION_CSN command. SOURCE_DB_NAME src_dbase_global_name Valid for Oracle. Indicates the Global Name of the Trail Source Database. It is used to query the relevant instantiation information when
DBOPTIONS ENABLE_INSTANTIATION_FILTERING is enabled. This option is optional for instantiation filtering in a 12.2. trail file with
metadata enabled. When the source has no DOMAIN, do not specify a DOMAIN for the downstream database.
二、學習測試
2.1 測試鏈路數據同步測試
1）源端測試表新建、最小補充日志開啟
備注：源端目標端DB 11.2.0.4,OGG19.1
Source SQL> conn yz/yz SQL> create table a as select * from dba_objects; SQL> create table b as select * from dba_objects; GGSCI (t1) 1> dblogin USERID ogg,PASSWORD OGG > add trandata yz.a > add trandata yz.b alter table a rename to test00a1; alter table b rename to test00a2; info trandata yz.test00a2 Prepared CSN for table YZ.TEST00A1: 1937981 Prepared CSN for table YZ.TEST00A2: 1938904
本次測試附帶測試了一波rename 最小補充日志是否生效,oracle給力，自動內部進行重命名，還是有效的！
2）視圖說明
DBA_CAPTURE_PREPARED_TABLES displays information about all tables prepared for instantiation at the local database DBA_APPLY_INSTANTIATED_OBJECTS displays information about objects for which an instantiation SCN has been set. INSTANTIATION_SCN Instantiation SCN for the object. Only changes committed after this SCN are applied by an apply process. IGNORE_SCN SCN below which the instantiation SCN cannot be set. This value corresponds to the SCN value at the source database at the time
when the object was prepared for instantiation.
SQL> select table_name, scn from dba_capture_prepared_tables where table_owner = 'YZ' ;
TABLE_NAME SCN
------------------------------ ----------
TEST00A1 1937981
GBK1 1799996
TEST00A2 1938904
Target
SQL> select source_object_name, instantiation_scn, ignore_scn from dba_apply_instantiated_objects where source_object_owner = 'YZ' ;
null
以上可知，最小補充日志的添加，對應的SCN=>dba_capture_prepared_tables=>SCN
3)OGG進程鏈路關系
>ext_gbk EXTTRAIL /u01/ogg/base/dirdat/ext_gbk/cc > view param ext_gbk TABLE yz.gbk1,cols(id); --add TABLE YZ.TEST00A1; TABLE YZ.TEST00A2; > info D_GBK_A,detail Trail Name /u01/ogg/base/dirdat/gbk/aa /u01/ogg/base/dirdat/ext_gbk/cc000000021 --add TABLE YZ.TEST00A1; TABLE YZ.TEST00A2;
4）數據初始化同步，觀察視圖SCN變換
--expdp select * from dba_directories; create directory dump as '/home/oracle'; grant read,write on directory dump to sys; expdp \'/ as sysdba\' directory=dump dumpfile=TEST00A1.dmp logfile=TEST00A1.log tables=yz.TEST00A1 SQL> select current_scn from v$database; CURRENT_SCN ----------- 1943827 expdp \'/ as sysdba\' directory=dump dumpfile=TEST00A2.dmp logfile=TEST00A2.log tables=yz.TEST00A2 FLASHBACK_SCN=1943827 $ scp TEST00A*.dmp oracle@10.0.0.32:/home/oracle/. impdp \'/ as sysdba\' directory=dump dumpfile=TEST00A1.dmp logfile=TEST00A1.log REMAP_SCHEMA=YZ:BAK_YZ impdp \'/ as sysdba\' directory=dump dumpfile=TEST00A2.dmp logfile=TEST00A2.log REMAP_SCHEMA=YZ:BAK_YZ select source_object_name, instantiation_scn, ignore_scn from dba_apply_instantiated_objects where source_object_owner = 'YZ' SOURCE_OBJECT_NAME INSTANTIATION_SCN IGNORE_SCN ------------------------------ ----------------- ---------- TEST00A2 1943827 0 TEST00A1 1944375 1937981
我們可以發現當源端指定了SCN，則目標端OGG INSTANTIATION_SCN = 指定的導出SCN
沒有指定SCN則INSTANTIATION_SCN 暫時無法判定！IGNORE_SCN則=源端對表開啟最小補充日志的SCN
5）測試OGG鏈路同步OK
數據初始化同步時，OGG進程是源端抽取、投遞正常，目標端復制進程未新建、未處理！
新增復制進程 >info REP_TA,detai > add replicat REP_GBKA,exttrail /u01/ogg/base/dirdat/gbk/aa >edit param REP_GBKA replicat REP_GBKA userid ogg,password ogg assumetargetdefs discardfile /u01/ogg/base/dirrpt/REP_GBKA.dsc MAP YZ.TEST00A1 ,TARGET BAK_YZ.TEST00A1; MAP YZ.TEST00A2 ,TARGET BAK_YZ.TEST00A2; alter replicat REP_GBKA,extseqno 9 extrba 0 start rep_gbka
確認數據同步正常！
2.2新增參數測試
1）環境清理准備
目標端清理數據，停止復制進程！ GGSCI (t2) 9> stop rep_gbka Log Read Checkpoint File /u01/ogg/base/dirdat/gbk/aa000000013 2021-07-01 10:56:08.685040 RBA 4034 SQL> drop table TEST00A1; SQL> drop table TEST00A2;
2）數據泵重新數據初始化
Source expdp \'/ as sysdba\' directory=dump dumpfile=TEST00A1.dmp logfile=TEST00A1.log tables=yz.TEST00A1 SQL> select current_scn from v$database; CURRENT_SCN ----------- 1948959 expdp \'/ as sysdba\' directory=dump dumpfile=TEST00A2.dmp logfile=TEST00A2.log tables=yz.TEST00A2 FLASHBACK_SCN=1948959 $ scp TEST00A*.dmp oracle@10.0.0.32:/home/oracle/. SQL> select table_name, scn from dba_capture_prepared_tables where table_owner = 'YZ' ; TABLE_NAME SCN ------------------------------ ---------- TEST00A1 1937981 GBK1 1799996 TEST00A2 1938904
Target impdp 之前 select source_object_name, instantiation_scn, ignore_scn from dba_apply_instantiated_objects where source_object_owner = 'YZ'; SOURCE_OBJECT_NAME INSTANTIATION_SCN IGNORE_SCN ------------------------------ ----------------- ---------- TEST00A2 1943827 0 TEST00A1 1944375 1937981 impdp \'/ as sysdba\' directory=dump dumpfile=TEST00A1.dmp logfile=TEST00A1.log REMAP_SCHEMA=YZ:BAK_YZ impdp \'/ as sysdba\' directory=dump dumpfile=TEST00A2.dmp logfile=TEST00A2.log REMAP_SCHEMA=YZ:BAK_YZ select source_object_name, instantiation_scn, ignore_scn from dba_apply_instantiated_objects where source_object_owner = 'YZ'; SOURCE_OBJECT_NAME INSTANTIATION_SCN IGNORE_SCN ------------------------------ ----------------- ---------- GBK1 1949541 1799996 TEST00A2 1948959 0 TEST00A1 1949541 1937981
3）復制進程添加參數，觀察效果！
source db dml 刪除一條記錄后source TEST00A1 86583 TEST00A2 86587 target ogg 啟動！ DBOPTIONS ENABLE_INSTANTIATION_FILTERING GGSCI (t2) 12> start rep_gbka 2021-07-01 11:05:41 INFO OGG-10155 Instantiation CSN filtering is enabled on table YZ.TEST00A1 at CSN 1949541. 2021-07-01 11:05:41 INFO OGG-10155 Instantiation CSN filtering is enabled on table YZ.TEST00A2 at CSN 1948959. 其實沒必要確認測試數據，讀取日志我們可以清晰的發現，是等同於最開始說的方法二、表或進程級別filtering csn一樣的效果
這個scn選擇的是INSTANTIATION_SCN
Export: Release 11.2.0.4.0 - Production on Thu Jul 1 10:59:03 2021
select timestamp_to_scn(to_date('2021-07-01 10:59:03','yyyy-mm-dd hh24:mi:ss')) from v$database;
TIMESTAMP_TO_SCN(TO_DATE('2021-07-0110:59:03','YYYY-MM-DDHH24:MI:SS'))
----------------------------------------------------------------------
1947745 --因為數據泵導出的日志時間不一定完全准確！ 但是可以認為是相近的了。
三、結論
復制進程使用如下參數 DBOPTIONS ENABLE_INSTANTIATION_FILTERING 本質上是使用dba_apply_instantiated_objects INSTANTIATION_SCN 進行過濾； 建議：1.源端指定scn進行導出這個scn= dba_apply_instantiated_objects INSTANTIATION_SCN 2.源端導出不知道scn ，根據導出日志的時間與dba_apply_instantiated_objects INSTANTIATION_SCN 基本吻合，但是不知道Oracle內部如何實現的不敢確認。
建議生產環境源端導出還是使用統一scn導出，目標端導入可以自動處理。