ogg一般來說有4個進程,Manager進程,Extract進程,Pump進程,Replicat進程
Manager進程:
是GoldenGate的控制進程,它主要作用有以下幾個方面:啟動、監控、重啟GoldenGate的其他進程,報告錯誤及時間,分配數據存儲空間,發布閥
值報告等。
Extract進程:
運行在數據庫源端,負責從源端數據表或日志中捕獲數據。Extract進程利用其內在的checkpoint機制,周期性地檢查 並記錄其讀寫的位置,通常
是寫入到本地的trail文件。這種機制是為了保證如果Extract進程終止或者操作系統宕機,我們重啟Extract進程 后,GoldenGate能夠恢復到以前
的狀態,從上一個斷點處繼續往下運行,而不會有任何數據損失。
Pump進程:(如果trail文件落本地才會用這個進程,一般OGG都會把traile文件落地)
運行在數據庫源端,其作用非常簡單。如果源端使用了本地trail文件,那么Pump進程就會把Trail文件以數據塊的形式通過 TCP/IP協議發送到目
標端,我們下面的配置都是這種方式。Pump進程本質是Extract進程的一種特殊形式,如果不使用Trail文件,那么 Extract進程在抽取完數據后,
直接投遞到目標端。
與Pump進程相對應的叫Server Collector進程,這個進程不需要引起我們關注,因為在實際操作過程中無需對其進行任何配置,它運行在目標端,
任務就是把Extract/Pump進程投遞過來的數據塊重新組裝成Trail文件。
Replicat進程:
運行在目標端,是數據投遞的最后一站,負責讀取目標端Trail文件中的內容,並將解析其解析為DML或DDL語句,然后應用到目標數據庫中。
測試環境
源端:
IP:172.16.0.30 SID:prod
目標;
IP: 172.16.0.40 SID:prod(實例名可以不一致)
1、安裝ogg軟件,安裝方式如下(兩端均執行)
oracle用戶下創建ogg目錄
mkdir /home/oracle/ogg
[oracle@localhost ~]$ cd ogg/
[oracle@localhost ogg]$ unzip ogg112101_fbo_ggs_Linux_x64_ora11g_64bit.zip
[oracle@localhost ogg]$ tar -xvf fbo_ggs_Linux_x64_ora11g_64bit.tar
設置LD_LIBRARY_PATH參數
export LD_LIBRARY_PATH=$ORACLE_HOME/lib
查看運行./ggsci命令行接口所缺是的庫文件,可能會出現libnnz11.so,libclntsh.so.11.1缺失,只需要將libnnz10.so,libclntsh.so.10.1軟
解析到缺失的文件即可
[oracle@localhost ogg]$ ldd ./ggsci
2、數據庫環境准備(兩端均執行)
打開最小附加日志
SQL>alter database add supplemental log data;
SQL>select supplemental_log_data_min from v$database;
創建數據庫ogg用戶
SQL>CREATE USER ogg IDENTIFIED BY ogg DEFAULT TABLESPACE USERS ;
授權ogg用戶,如果對用戶權限要求不嚴格可以直接授予dba權限,否則授予如下權限
源端:
GRANT CONNECT TO goldengate;
GRANT CREATE SESSION TO goldengate;
GRANT ALTER SESSION TO goldengate;
GRANT RESOURCE TO goldengate;
GRANT SELECT ANY DICTIONARY TO goldengate;
GRANT SELECT ANY TABLE TO goldengate;
GRANT FLASHBACK ANY TABLE TO goldengate;
GRANT ALTER ANY TABLE TO goldengate;
目標端:
GRANT ALTER SESSION TO goldengate;
GRANT CREATE SESSION TO goldengate;
GRANT CONNECT TO goldengate;
GRANT RESOURCE TO goldengate;
GRANT SELECT ANY DICTIONARY TO goldengate;
GRANT SELECT ANY TABLE TO goldengate;
GRANT FLASHBACK ANY TABLE TO goldengate;
GRANT INSERT ANY TABLE TO goldengate;
GRANT UPDATE ANY TABLE TO goldengate;
GRANT DELETE ANY TABLE TO goldengate;
3、主備庫創建mgr管理進程,添加端口7809(兩端均執行)
創建ogg子目錄
[oracle@localhost ogg]$ ./ggsci
GGSCI>create subdirs
GGSCI (localhost.localdomain) 10> edit params mgr
port 7809
GGSCI (localhost.localdomain) 11> start mgr
GGSCI (localhost.localdomain) 11> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
mgr可以添加如下參數:
DYNAMICPORTLIST 7820-7830, 7833, 7835
autostart er *
autorestart er *, retries 5, waitminutes 3 (自動啟動所有參數,會嘗試5次,每次等待3分鍾)
purgeoldextracts ./dirdat/*,usecheckpoints, minkeepdays 7(自動清理過期的trail文件,會保留7天)
4、添加表級附加日志
ggsci>dblogin userid ogg, password ogg
ggsci>add trandata hzq.*
5、添加抽取進程 (源端)
GGSCI> add ext exta, tranlog, begin now
exta為進程名,一般為ext開頭表示是extract進程,后面可以加1-2位字符標識;begin可以是now也可以是指定的時間
為exta進程配置本地隊列,本地的trail文件以la開頭
GGSCI> add exttrail ./dirdat/lc, ext exta, MEGABYTES 20
添加投遞進程pmpa,將本地的./dirdat/la文件傳送到目標端
GGSCI> add extract pmpa, EXTTRAILSOURCE ./dirdat/lc
為pmpa進程配置遠程隊列,隊列的位置是源端的主機目錄,傳送到目標斷后命名以rc開頭。
GGSCI> add rmttrail ./dirdat/rc, ext dpea, MEGABYTES 20
GGSCI (localhost.localdomain) 23> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
EXTRACT STOPPED EXTA 00:00:00 00:00:02
EXTRACT STOPPED PMPA 00:00:00 00:00:01
6、配置進程參數(源端)
GGSCI (localhost.localdomain) 25> edit params exta
EXTRACT exta
setenv ( NLS_LANG = "AMERICAN_AMERICA.ZHS16GBK" )
setenv (ORACLE_SID = prod)
USERID ogg, PASSWORD ogg
EXTTRAIL ./dirdat/lc
dynamicresolution
--GETUPDATEBEFORES
--DDL INCLUDE ALL (DDL復制參數)
--DDLOPTIONS ADDTRANDATA NOCROSSRENAME REPORT (ddl復制參數)
--NOCOMPRESSDELETES
table hzq.*;
GGSCI (localhost.localdomain) 28> edit params pmpa
extract pmpa
setenv ( NLS_LANG = AMERICAN_AMERICA.ZHS16GBK )
passthru
--REPORT AT 01:59
--reportrollover at 02:00
rmthost 172.16.0.40, mgrport 7809, compress
rmttrail ./dirdat/rc
dynamicresolution
table hzq.*;
啟動進程
GGSCI>start ext *
GGSCI (localhost.localdomain) 29> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
EXTRACT RUNNING EXTA 00:00:00 00:00:04
EXTRACT RUNNING PMPA 00:00:00 00:00:05
7、配置目標端進程和參數(目標端)
GGSCI> add rep rep1a, exttrail ./dirdat/rc, nodbcheckpoint
GGSCI>edit param rep1a
replicat rep1a
setenv ( NLS_LANG = AMERICAN_AMERICA.ZHS16GBK )
setenv (ORACLE_SID = prod)
userid ogg, password ogg
reperror default,abend
discardfile ./dirrpt/rep1a.dsc,append, megabytes 10
assumetargetdefs
--allownoopupdates
dynamicresolution
--DDL include all
--ddloptions report
--DDLERROR DEFAULT IGNORE RETRYOP MAXRETRIES 3 RETRYDELAY 5
ASSUMETARGETDEFS
--INSERTALLRECORDS
map hzq.*, target hzq.*;
啟動參數
GGSCI (localhost.localdomain) 12> start rep1a
GGSCI (localhost.localdomain) 11> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
REPLICAT RUNNING REP1A 00:00:00 00:00:06
ogg一般來說有4個進程,Manager進程,Extract進程,Pump進程,Replicat進程
Manager進程:
是GoldenGate的控制進程,它主要作用有以下幾個方面:啟動、監控、重啟GoldenGate的其他進程,報告錯誤及時間,分配數據存儲空間,發布閥
值報告等。
Extract進程:
運行在數據庫源端,負責從源端數據表或日志中捕獲數據。Extract進程利用其內在的checkpoint機制,周期性地檢查 並記錄其讀寫的位置,通常
是寫入到本地的trail文件。這種機制是為了保證如果Extract進程終止或者操作系統宕機,我們重啟Extract進程 后,GoldenGate能夠恢復到以前
的狀態,從上一個斷點處繼續往下運行,而不會有任何數據損失。
Pump進程:(如果trail文件落本地才會用這個進程,一般OGG都會把traile文件落地)
運行在數據庫源端,其作用非常簡單。如果源端使用了本地trail文件,那么Pump進程就會把Trail文件以數據塊的形式通過 TCP/IP協議發送到目
標端,我們下面的配置都是這種方式。Pump進程本質是Extract進程的一種特殊形式,如果不使用Trail文件,那么 Extract進程在抽取完數據后,
直接投遞到目標端。
與Pump進程相對應的叫Server Collector進程,這個進程不需要引起我們關注,因為在實際操作過程中無需對其進行任何配置,它運行在目標端,
任務就是把Extract/Pump進程投遞過來的數據塊重新組裝成Trail文件。
Replicat進程:
運行在目標端,是數據投遞的最后一站,負責讀取目標端Trail文件中的內容,並將解析其解析為DML或DDL語句,然后應用到目標數據庫中。
測試環境
源端:
IP:172.16.0.30 SID:prod
目標;
IP: 172.16.0.40 SID:prod(實例名可以不一致)
1、安裝ogg軟件,安裝方式如下(兩端均執行)
oracle用戶下創建ogg目錄
mkdir /home/oracle/ogg
[oracle@localhost ~]$ cd ogg/
[oracle@localhost ogg]$ unzip ogg112101_fbo_ggs_Linux_x64_ora11g_64bit.zip
[oracle@localhost ogg]$ tar -xvf fbo_ggs_Linux_x64_ora11g_64bit.tar
設置LD_LIBRARY_PATH參數
export LD_LIBRARY_PATH=$ORACLE_HOME/lib
查看運行./ggsci命令行接口所缺是的庫文件,可能會出現libnnz11.so,libclntsh.so.11.1缺失,只需要將libnnz10.so,libclntsh.so.10.1軟
解析到缺失的文件即可
[oracle@localhost ogg]$ ldd ./ggsci
2、數據庫環境准備(兩端均執行)
打開最小附加日志
SQL>alter database add supplemental log data;
SQL>select supplemental_log_data_min from v$database;
創建數據庫ogg用戶
SQL>CREATE USER ogg IDENTIFIED BY ogg DEFAULT TABLESPACE USERS ;
授權ogg用戶,如果對用戶權限要求不嚴格可以直接授予dba權限,否則授予如下權限
源端:
GRANT CONNECT TO goldengate;
GRANT CREATE SESSION TO goldengate;
GRANT ALTER SESSION TO goldengate;
GRANT RESOURCE TO goldengate;
GRANT SELECT ANY DICTIONARY TO goldengate;
GRANT SELECT ANY TABLE TO goldengate;
GRANT FLASHBACK ANY TABLE TO goldengate;
GRANT ALTER ANY TABLE TO goldengate;
目標端:
GRANT ALTER SESSION TO goldengate;
GRANT CREATE SESSION TO goldengate;
GRANT CONNECT TO goldengate;
GRANT RESOURCE TO goldengate;
GRANT SELECT ANY DICTIONARY TO goldengate;
GRANT SELECT ANY TABLE TO goldengate;
GRANT FLASHBACK ANY TABLE TO goldengate;
GRANT INSERT ANY TABLE TO goldengate;
GRANT UPDATE ANY TABLE TO goldengate;
GRANT DELETE ANY TABLE TO goldengate;
3、主備庫創建mgr管理進程,添加端口7809(兩端均執行)
創建ogg子目錄
[oracle@localhost ogg]$ ./ggsci
GGSCI>create subdirs
GGSCI (localhost.localdomain) 10> edit params mgr
port 7809
GGSCI (localhost.localdomain) 11> start mgr
GGSCI (localhost.localdomain) 11> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
mgr可以添加如下參數:
DYNAMICPORTLIST 7820-7830, 7833, 7835
autostart er *
autorestart er *, retries 5, waitminutes 3 (自動啟動所有參數,會嘗試5次,每次等待3分鍾)
purgeoldextracts ./dirdat/*,usecheckpoints, minkeepdays 7(自動清理過期的trail文件,會保留7天)
4、添加表級附加日志
ggsci>dblogin userid ogg, password ogg
ggsci>add trandata hzq.*
5、添加抽取進程 (源端)
GGSCI> add ext exta, tranlog, begin now
exta為進程名,一般為ext開頭表示是extract進程,后面可以加1-2位字符標識;begin可以是now也可以是指定的時間
為exta進程配置本地隊列,本地的trail文件以la開頭
GGSCI> add exttrail ./dirdat/lc, ext exta, MEGABYTES 20
添加投遞進程pmpa,將本地的./dirdat/la文件傳送到目標端
GGSCI> add extract pmpa, EXTTRAILSOURCE ./dirdat/lc
為pmpa進程配置遠程隊列,隊列的位置是源端的主機目錄,傳送到目標斷后命名以rc開頭。
GGSCI> add rmttrail ./dirdat/rc, ext dpea, MEGABYTES 20
GGSCI (localhost.localdomain) 23> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
EXTRACT STOPPED EXTA 00:00:00 00:00:02
EXTRACT STOPPED PMPA 00:00:00 00:00:01
6、配置進程參數(源端)
GGSCI (localhost.localdomain) 25> edit params exta
EXTRACT exta
setenv ( NLS_LANG = "AMERICAN_AMERICA.ZHS16GBK" )
setenv (ORACLE_SID = prod)
USERID ogg, PASSWORD ogg
EXTTRAIL ./dirdat/lc
dynamicresolution
--GETUPDATEBEFORES
--DDL INCLUDE ALL (DDL復制參數)
--DDLOPTIONS ADDTRANDATA NOCROSSRENAME REPORT (ddl復制參數)
--NOCOMPRESSDELETES
table hzq.*;
GGSCI (localhost.localdomain) 28> edit params pmpa
extract pmpa
setenv ( NLS_LANG = AMERICAN_AMERICA.ZHS16GBK )
passthru
--REPORT AT 01:59
--reportrollover at 02:00
rmthost 172.16.0.40, mgrport 7809, compress
rmttrail ./dirdat/rc
dynamicresolution
table hzq.*;
啟動進程
GGSCI>start ext *
GGSCI (localhost.localdomain) 29> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
EXTRACT RUNNING EXTA 00:00:00 00:00:04
EXTRACT RUNNING PMPA 00:00:00 00:00:05
7、配置目標端進程和參數(目標端)
GGSCI> add rep rep1a, exttrail ./dirdat/rc, nodbcheckpoint
GGSCI>edit param rep1a
replicat rep1a
setenv ( NLS_LANG = AMERICAN_AMERICA.ZHS16GBK )
setenv (ORACLE_SID = prod)
userid ogg, password ogg
reperror default,abend
discardfile ./dirrpt/rep1a.dsc,append, megabytes 10
assumetargetdefs
--allownoopupdates
dynamicresolution
--DDL include all
--ddloptions report
--DDLERROR DEFAULT IGNORE RETRYOP MAXRETRIES 3 RETRYDELAY 5
ASSUMETARGETDEFS
--INSERTALLRECORDS
map hzq.*, target hzq.*;
啟動參數
GGSCI (localhost.localdomain) 12> start rep1a
GGSCI (localhost.localdomain) 11> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
REPLICAT RUNNING REP1A 00:00:00 00:00:06