PostgreSQL的OGG -- bucardo


bucardo是PostgreSQL數據庫中實現雙向同步的軟件,可以實現PostgreSQL數據庫的雙master的方案,不過bucardo中的同步都是異步的,它是通過觸發器記錄變化,程序是perl寫的.bucardo可以實現postgresql的多主復制、主從同步,甚至可以以postgresql為源庫,可以和oracle、mysql、mongodb等很多數據庫進行數據異步同步。
 
而pg原生的stream replication;雖可以同步;只能單向同步。只能是只讀操作。而bucardo不支持DDL的同步; 所以說 相當於對stream replication的關系;相當於同oracle中 ogg對dg。
 
一 , 安裝軟件包
 
 
    對上面perl安裝包;安裝過程
perl Makefile.PL
make
make install
3. 上面安裝好了軟件;下面實例化Bucardo;在postgres用戶下面新建.bucardorc 文件
log_conflict_file         = /home/postgres/log/bucardo_conflict.log
piddir                    = /home/postgres/run
reason_file               = /home/postgres/log/bucardo.restart.reason.log
warning_file              = /home/postgres/log/bucardo.warning.log

4. 初始化bucardo

[postgres@sdserver40_222 ~]$ bucardo install
This will install the bucardo database into an existing Postgres cluster.
Postgres must have been compiled with Perl support,
and you must connect as a superuser

Current connection settings:
1. Host:           <none>
2. Port:           5432
3. User:           postgres
4. Database:       bucardo
5. PID directory:  /home/postgres/run
Enter a number to change it, P to proceed, or Q to quit: 1

Change the host to: 127.0.0.1

Changed host to: 127.0.0.1
Current connection settings:
1. Host:           127.0.0.1
2. Port:           5432
3. User:           postgres
4. Database:       bucardo
5. PID directory:  /home/postgres/run
Enter a number to change it, P to proceed, or Q to quit: 4

Change the database name to: mydb

Changed database name to: mydb
Current connection settings:
1. Host:           127.0.0.1
2. Port:           5432
3. User:           postgres
4. Database:       mydb
5. PID directory:  /home/postgres/run
Enter a number to change it, P to proceed, or Q to quit: P

Postgres version is: 9.5
Attempting to create and populate the bucardo database and schema
Database creation is complete

Updated configuration setting "piddir"
Installation is now complete.
If you see errors or need help, please email bucardo-general@bucardo.org

You may want to check over the configuration variables next, by running:
bucardo show all
Change any setting by using: bucardo set foo=bar 
這步可能會出現問題;大多數是依賴包沒裝的問題;現后面FAQ
 
二, 配置bucardo環境
 
下面的配置的環境:
主機1: 192.168.1.201 
主機2: 192.168.1.202
 
1 添加源數據庫
在192.168.1.201 執行
[postgres@sdserver40_222 ~]$ bucardo add db db1 dbname=mydb host=127.0.0.1 port=5432  user=postgres
Added database "db1"
[postgres@sdserver40_222 ~]$ bucardo add db db2 dbname=mydb host=192.168.1.202 port=5432 user=postgres
Added database "db2"
這里需要注意的是;psql -h 192.168.1.202是否可以連通;看是不是防火牆;在主機192.168.1.202上pg_hba.conf是否配置。
 
2. 添加表:
[ postgres@sdserver40_222 ~]$  bucardo add all tables
New tables added: 0
Already added: 13
[ postgres@sdserver40_222 ~]$ bucardo add all sequences
Sorry, no sequences were found
New sequences added: 0
如果要添加某一個表,則可以bucardo add  table tablename
 
3.  添加復制群組(即復制一組表):
[ postgres@sdserver40_222 ~]$ bucardo add relgroup relgroup1 lottu03 lottu01;
Created relgroup "relgroup1"
The following tables or sequences are now part of the relgroup "relgroup1":
  public.lottu01
  public.lottu03
[ postgres@sdserver40_222 ~]$ bucardo add dbgroup dbgroup1 db1:source db2:target
Created dbgroup "dbgroup1"
Added database "db1" to dbgroup "dbgroup1" as source
Added database "db2" to dbgroup "dbgroup1" as target

  1)若添加若有表則用 all tables;

  2) 只能同步存在主鍵的table;像這里的lottu01我是沒建主鍵的;下面添加同步步驟會出現錯誤。

4. 添加同步

[postgres@sdserver40_222 ~]$ bucardo add sync sync1 relgroup=relgroup1 dbs=dbgroup1
Added sync "sync1"
   1)這里注意的是;同步到表需要主鍵約束;不然同步是添加不了的。
會報如下錯誤;這個是跟觸發器有關系;bucardo是建立在觸發器之上的。
Failed to add sync: DBD::Pg::st execute failed: ERROR:  Table "public.lottu01" must specify a primary key! at line 117. at line 30.
CONTEXT:  PL/Perl function "validate_sync" at /usr/local/bin/bucardo line 4612
  2)刪除同步 :bucardo add sync syncname
  3 ) bucardo 添加同步;參數詳情 如下;
  add sync:
      bucardo add sync name relgroup=xxx dbs=xxx

    Adds a sync, which is a named replication event containing information
    about what to replicate from where to where. The supported parameters
    are:

    "dbs"
        The name of a dbgroup or comma-delimited list of databases. All of
        the specified databases will be synchronized. Required.

    "dbgroup"
        The name of a dbgroup. All of the databases within this group will
        be part of the sync. If the dbgroup does not exists and a separate
        list of databases is given, the group will be created and populated.

    "relgroup"
        The name of a relgroup to synchronize. All of the tables and/or
        sequences in the relgroup will be synchronized. Required unless
        "tables" is specified.

    "tables"
        List of tables to add to the sync. This implicitly creates a
        relgroup with the same name as the sync. Required unless "relgroup"
        is specified.

    "status"
        Indicates whether or not the sync is active. Must be either "active"
        or "inactive". Defaults to "active".

    "rebuild_index"
        Boolean indicating whether or not to rebuild indexes after every
        sync. Defaults to off.

    "lifetime"
        Number of seconds a KID can live before being reaped. No limit by
        default.

    "maxkicks"
        Number of times a KID may be kicked before being reaped. No limit by
        default.

    "conflict_strategy"
        The conflict resolution strategy to use in the sync. Supported
        values:

        "bucardo_source"
            The rows on the "source" database always "win". In other words,
            in a conflict, Bucardo copies rows from source to target.

        "bucardo_target"
            The rows on the "target" database always win.

        "bucardo_skip"
            Any conflicting rows are simply not replicated. Not recommended
            for most cases.

        "bucardo_random"
            Each database has an equal chance of winning each time. This is
            the default.

        "bucardo_latest"
            The row that was most recently changed wins.

        "bucardo_abort"
            The sync is aborted on a conflict.

    "onetimecopy"
        Determines whether or not a sync should switch to a full copy mode
        for a single run. Supported values are:

        0: off
        1: always full copy
        2: only copy tables that are empty on the target

    "stayalive"
        Boolean indicating whether or not the sync processes (CTL) should be
        persistent. Defaults to false.

    "kidsalive"
        Boolean indicating whether or not the sync child processes (KID)
        should be persistent. Defaults to false.

    "autokick"
        Boolean indicating whether or not tables in the sync should
        automatically send kick messages when they're modified. May be
        overridden by the "autokick" parameter of individual tables.

    "checktime"
        An interval specifying the maximum time a sync should go before
        being kicked. Useful for busy systems where you don't want the
        overhead of notify triggers.

    "priority"
        An integer indicating the priority of the sync. Lower numbers are
        higher priority. Currently used only for display purposes.

    "analyze_after_copy"
        Boolean indicating whether or not to analyze tables after every
        sync. Off by default. Optional.

    "overdue"
        An interval specifying the amount of time after which the sync has
        not run that it should be considered overdue. "check_bucardo_sync"
        issues a warning when a sync has not been run in this amount of
        time.

    "expired"
        An interval specifying the amount of time after which the sync has
        not run that it should be considered expired. "check_bucardo_sync"
        issues a critical message when a sync has not been run in this
        amount of time.

    "track_rates"
        Boolean indicating whether or not to track synchronization rates.

    "rebuild_index"
        Boolean indicating whether or not to rebuild indexes after every
        sync. Off by default. Optional.

    "strict_check"
        Boolean indicating whether or not to be strict when comparing tables
        in the sync. If the columns have different names or data types, the
        validation will fail. But perhaps the columns are allowed to have
        different names or data types. If so, disable "strict_check" and
        column differences will result in warnings rather than failing the
        validation. Defaults to true.

5. 啟動bucardo

bucardo start
bucardo stop
bucardo restart
 
三,驗證操作
  在原庫操作
[postgres@sdserver40_222 ~]$ psql mydb
psql (9.5.0)
Type "help" for help.
mydb=# delete from lottu03;
DELETE 998
                                                  
mydb=# insert into lottu03 values (1001,'bucardo');
INSERT 0 1

在目標庫查看

[postgres@sdserver40_210 ~]$ psql mydb
psql (9.5.0)
Type "help" for help.

mydb=# select * from lottu03;
  id  |  name   
------+---------
 1001 | bucardo
(1 row)
 
四, FAQ
 Q1. 在執行bucardo install 操作出現
 Creating superuser 'bucardo'
Attempting to create and populate the bucardo database and schema

INSTALLATION FAILED! (psql:/usr/local/share/bucardo/bucardo.schema:46: ERROR:  language "plperlu" does not exist
HINT:  Use CREATE LANGUAGE to load the language into the database.)  

 解決方法;看提示知道需要“create language plperlu;”但是在我的環境之下

mydb=# create language plperlu;
ERROR:  could not access file "$libdir/plperl": No such file or directory
 出現這種問題不奇怪;因在我的博客 http://www.cnblogs.com/lottu/p/5149191.html  安裝postgres中初始化時未加-- with-perl;所以安裝后lib目錄下缺少plperl.so文件; 
解決方法:再來一邊初始化postgrs; 把plperl.so復制到對應的lib目錄下即可。
postgres=# CREATE LANGUAGE plperlu;
CREATE LANGUAG
 
Q2: 解決Q1;再一次執行bucardo install 操作出現
install_driver(Pg) failed: DBI version 1.614 required--this is only version 1.609 at /usr/share/perl5/Exporter/Heavy.pm line 240, <> line 5.
Compilation failed in require at (eval 9) line 3, <> line 5.
 解決方法: 這個是我開始安裝依賴包時;偷懶未安裝DBI 包; 提示是需要更新  DBI 包;目前最新的是1.633版本;這里我們安裝16.30就行 
 
Q3: 安裝bucardo install 之后錯誤提示
perl: symbol lookup error: /usr/local/lib64/perl5/auto/DBD/Pg/Pg.so: undefined symbol: DBIc_AIADESTROY
解決方法: 這個缺少symbol: DBIc_AIADESTROY; 這個是Q2問題遺留下來的; 之前安裝DBD_pg時是之前的DBI;現DBI更新了;需要重新安裝DBD_pg
 
Q4:在啟動bucardo start出現錯誤提示
Checking for existing processes
Can't locate boolean.pm in @INC (@INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at /usr/local/share/perl5/Bucardo.pm line 34.
BEGIN failed--compilation aborted at /usr/local/share/perl5/Bucardo.pm line 34.
Compilation failed in require at /usr/local/bin/bucardo line 833.

解決方法: yum install perl-boolean


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM