bucardo是PostgreSQL數據庫中實現雙向同步的軟件,可以實現PostgreSQL數據庫的雙master的方案,不過bucardo中的同步都是異步的,它是通過觸發器記錄變化,程序是perl寫的.bucardo可以實現postgresql的多主復制、主從同步,甚至可以以postgresql為源庫,可以和oracle、mysql、mongodb等很多數據庫進行數據異步同步。
而pg原生的stream replication;雖可以同步;只能單向同步。只能是只讀操作。而bucardo不支持DDL的同步; 所以說 相當於對stream replication的關系;相當於同oracle中 ogg對dg。
一 , 安裝軟件包
1. 首先安裝依賴包:
yum install perl-boolean
2. 安裝Bucardo
對上面perl安裝包;安裝過程
perl Makefile.PL
make
make install
3. 上面安裝好了軟件;下面實例化Bucardo;在postgres用戶下面新建.bucardorc 文件
log_conflict_file = /home/postgres/log/bucardo_conflict.log piddir = /home/postgres/run reason_file = /home/postgres/log/bucardo.restart.reason.log warning_file = /home/postgres/log/bucardo.warning.log
4. 初始化bucardo
[postgres@sdserver40_222 ~]$ bucardo install This will install the bucardo database into an existing Postgres cluster. Postgres must have been compiled with Perl support, and you must connect as a superuser Current connection settings: 1. Host: <none> 2. Port: 5432 3. User: postgres 4. Database: bucardo 5. PID directory: /home/postgres/run Enter a number to change it, P to proceed, or Q to quit: 1 Change the host to: 127.0.0.1 Changed host to: 127.0.0.1 Current connection settings: 1. Host: 127.0.0.1 2. Port: 5432 3. User: postgres 4. Database: bucardo 5. PID directory: /home/postgres/run Enter a number to change it, P to proceed, or Q to quit: 4 Change the database name to: mydb Changed database name to: mydb Current connection settings: 1. Host: 127.0.0.1 2. Port: 5432 3. User: postgres 4. Database: mydb 5. PID directory: /home/postgres/run Enter a number to change it, P to proceed, or Q to quit: P Postgres version is: 9.5 Attempting to create and populate the bucardo database and schema Database creation is complete Updated configuration setting "piddir" Installation is now complete. If you see errors or need help, please email bucardo-general@bucardo.org You may want to check over the configuration variables next, by running: bucardo show all Change any setting by using: bucardo set foo=bar
這步可能會出現問題;大多數是依賴包沒裝的問題;現后面FAQ
二, 配置bucardo環境
下面的配置的環境:
主機1: 192.168.1.201
主機2: 192.168.1.202
1 添加源數據庫
在192.168.1.201 執行
[postgres@sdserver40_222 ~]$ bucardo add db db1 dbname=mydb host=127.0.0.1 port=5432 user=postgres Added database "db1" [postgres@sdserver40_222 ~]$ bucardo add db db2 dbname=mydb host=192.168.1.202 port=5432 user=postgres Added database "db2"
這里需要注意的是;psql -h 192.168.1.202是否可以連通;看是不是防火牆;在主機192.168.1.202上pg_hba.conf是否配置。
2. 添加表:
[
postgres@sdserver40_222 ~]$ bucardo add all tables
New tables added: 0
Already added: 13
[
postgres@sdserver40_222 ~]$ bucardo add all sequences
Sorry, no sequences were found
New sequences added: 0
如果要添加某一個表,則可以bucardo add table tablename
3. 添加復制群組(即復制一組表):
[
postgres@sdserver40_222 ~]$ bucardo add relgroup relgroup1 lottu03 lottu01;
Created relgroup "relgroup1"
The following tables or sequences are now part of the relgroup "relgroup1":
public.lottu01
public.lottu03
[
postgres@sdserver40_222 ~]$ bucardo add dbgroup dbgroup1 db1:source db2:target
Created dbgroup "dbgroup1"
Added database "db1" to dbgroup "dbgroup1" as source
Added database "db2" to dbgroup "dbgroup1" as target
1)若添加若有表則用 all tables;
2) 只能同步存在主鍵的table;像這里的lottu01我是沒建主鍵的;下面添加同步步驟會出現錯誤。
4. 添加同步
[postgres@sdserver40_222 ~]$ bucardo add sync sync1 relgroup=relgroup1 dbs=dbgroup1
Added sync "sync1"
1)這里注意的是;同步到表需要主鍵約束;不然同步是添加不了的。
會報如下錯誤;這個是跟觸發器有關系;bucardo是建立在觸發器之上的。
Failed to add sync: DBD::Pg::st execute failed: ERROR: Table "public.lottu01" must specify a primary key! at line 117. at line 30. CONTEXT: PL/Perl function "validate_sync" at /usr/local/bin/bucardo line 4612
2)刪除同步 :bucardo add sync syncname
3 ) bucardo 添加同步;參數詳情 如下;
add sync: bucardo add sync name relgroup=xxx dbs=xxx Adds a sync, which is a named replication event containing information about what to replicate from where to where. The supported parameters are: "dbs" The name of a dbgroup or comma-delimited list of databases. All of the specified databases will be synchronized. Required. "dbgroup" The name of a dbgroup. All of the databases within this group will be part of the sync. If the dbgroup does not exists and a separate list of databases is given, the group will be created and populated. "relgroup" The name of a relgroup to synchronize. All of the tables and/or sequences in the relgroup will be synchronized. Required unless "tables" is specified. "tables" List of tables to add to the sync. This implicitly creates a relgroup with the same name as the sync. Required unless "relgroup" is specified. "status" Indicates whether or not the sync is active. Must be either "active" or "inactive". Defaults to "active". "rebuild_index" Boolean indicating whether or not to rebuild indexes after every sync. Defaults to off. "lifetime" Number of seconds a KID can live before being reaped. No limit by default. "maxkicks" Number of times a KID may be kicked before being reaped. No limit by default. "conflict_strategy" The conflict resolution strategy to use in the sync. Supported values: "bucardo_source" The rows on the "source" database always "win". In other words, in a conflict, Bucardo copies rows from source to target. "bucardo_target" The rows on the "target" database always win. "bucardo_skip" Any conflicting rows are simply not replicated. Not recommended for most cases. "bucardo_random" Each database has an equal chance of winning each time. This is the default. "bucardo_latest" The row that was most recently changed wins. "bucardo_abort" The sync is aborted on a conflict. "onetimecopy" Determines whether or not a sync should switch to a full copy mode for a single run. Supported values are: 0: off 1: always full copy 2: only copy tables that are empty on the target "stayalive" Boolean indicating whether or not the sync processes (CTL) should be persistent. Defaults to false. "kidsalive" Boolean indicating whether or not the sync child processes (KID) should be persistent. Defaults to false. "autokick" Boolean indicating whether or not tables in the sync should automatically send kick messages when they're modified. May be overridden by the "autokick" parameter of individual tables. "checktime" An interval specifying the maximum time a sync should go before being kicked. Useful for busy systems where you don't want the overhead of notify triggers. "priority" An integer indicating the priority of the sync. Lower numbers are higher priority. Currently used only for display purposes. "analyze_after_copy" Boolean indicating whether or not to analyze tables after every sync. Off by default. Optional. "overdue" An interval specifying the amount of time after which the sync has not run that it should be considered overdue. "check_bucardo_sync" issues a warning when a sync has not been run in this amount of time. "expired" An interval specifying the amount of time after which the sync has not run that it should be considered expired. "check_bucardo_sync" issues a critical message when a sync has not been run in this amount of time. "track_rates" Boolean indicating whether or not to track synchronization rates. "rebuild_index" Boolean indicating whether or not to rebuild indexes after every sync. Off by default. Optional. "strict_check" Boolean indicating whether or not to be strict when comparing tables in the sync. If the columns have different names or data types, the validation will fail. But perhaps the columns are allowed to have different names or data types. If so, disable "strict_check" and column differences will result in warnings rather than failing the validation. Defaults to true.
5. 啟動bucardo
bucardo start
bucardo stop
bucardo restart
三,驗證操作
在原庫操作
[postgres@sdserver40_222 ~]$ psql mydb psql (9.5.0) Type "help" for help. mydb=# delete from lottu03; DELETE 998 mydb=# insert into lottu03 values (1001,'bucardo'); INSERT 0 1
在目標庫查看
[postgres@sdserver40_210 ~]$ psql mydb psql (9.5.0) Type "help" for help. mydb=# select * from lottu03; id | name ------+--------- 1001 | bucardo (1 row)
四, FAQ
Q1. 在執行bucardo install 操作出現
Creating superuser 'bucardo' Attempting to create and populate the bucardo database and schema INSTALLATION FAILED! (psql:/usr/local/share/bucardo/bucardo.schema:46: ERROR: language "plperlu" does not exist HINT: Use CREATE LANGUAGE to load the language into the database.)
解決方法;看提示知道需要“create language plperlu;”但是在我的環境之下
mydb=# create language plperlu; ERROR: could not access file "$libdir/plperl": No such file or directory
出現這種問題不奇怪;因在我的博客
http://www.cnblogs.com/lottu/p/5149191.html 安裝postgres中初始化時未加--
with-perl;所以安裝后lib目錄下缺少plperl.so文件;
解決方法:再來一邊初始化postgrs; 把plperl.so復制到對應的lib目錄下即可。
postgres=# CREATE LANGUAGE plperlu; CREATE LANGUAG
Q2: 解決Q1;再一次執行bucardo install 操作出現
install_driver(Pg) failed: DBI version 1.614 required--this is only version 1.609 at /usr/share/perl5/Exporter/Heavy.pm line 240, <> line 5. Compilation failed in require at (eval 9) line 3, <> line 5.
解決方法: 這個是我開始安裝依賴包時;偷懶未安裝DBI 包; 提示是需要更新
DBI 包;目前最新的是1.633版本;這里我們安裝16.30就行
Q3: 安裝bucardo install 之后錯誤提示
perl: symbol lookup error: /usr/local/lib64/perl5/auto/DBD/Pg/Pg.so: undefined symbol: DBIc_AIADESTROY
解決方法: 這個缺少symbol: DBIc_AIADESTROY; 這個是Q2問題遺留下來的; 之前安裝DBD_pg時是之前的DBI;現DBI更新了;需要重新安裝DBD_pg
Q4:在啟動bucardo start出現錯誤提示
Checking for existing processes Can't locate boolean.pm in @INC (@INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at /usr/local/share/perl5/Bucardo.pm line 34. BEGIN failed--compilation aborted at /usr/local/share/perl5/Bucardo.pm line 34. Compilation failed in require at /usr/local/bin/bucardo line 833.
解決方法: yum install perl-boolean