介紹
pg_rman是一款專門為postgresql設計的在線備份恢復的工具。其支持在線和基於時間點備份方式，還可以通過創建backup catalog來維護DB cluster備份信息。
pg_rman特點：
- 使用簡單.一個命令即可完成備份和恢復.
- 支持在線全備,增量備份，歸檔備份.
- 支持備份壓縮.通過gzip工具實現頁內壓縮.
- 自動備份維護.自動刪除過期的WAL備份文件.
- 支持備份驗證.
- 恢復期間無事務丟失.支持基於PITR的配置文件生成器.
pg_rman的用法
pg_rman manage backup/recovery of PostgreSQL database. Usage: pg_rman OPTION init pg_rman OPTION backup pg_rman OPTION restore pg_rman OPTION show [DATE] pg_rman OPTION show detail [DATE] pg_rman OPTION validate [DATE] pg_rman OPTION delete DATE pg_rman OPTION purge Common Options: -D, --pgdata=PATH location of the database storage area -A, --arclog-path=PATH location of archive WAL storage area -S, --srvlog-path=PATH location of server log storage area -B, --backup-path=PATH location of the backup storage area -c, --check show what would have been done -v, --verbose show what detail messages -P, --progress show progress of processed files Backup options: -b, --backup-mode=MODE full, incremental, or archive -s, --with-serverlog also backup server log files -Z, --compress-data compress data backup with zlib -C, --smooth-checkpoint do smooth checkpoint before backup -F, --full-backup-on-error switch to full backup mode if pg_rman cannot find validate full backup on current timeline NOTE: this option is only used in --backup-mode=incremental or archive. --keep-data-generations=NUM keep NUM generations of full data backup --keep-data-days=NUM keep enough data backup to recover to N days ago --keep-arclog-files=NUM keep NUM of archived WAL --keep-arclog-days=DAY keep archived WAL modified in DAY days --keep-srvlog-files=NUM keep NUM of serverlogs --keep-srvlog-days=DAY keep serverlog modified in DAY days --standby-host=HOSTNAME standby host when taking backup from standby --standby-port=PORT standby port when taking backup from standby Restore options: --recovery-target-time time stamp up to which recovery will proceed --recovery-target-xid transaction ID up to which recovery will proceed --recovery-target-inclusive whether we stop just after the recovery target --recovery-target-timeline recovering into a particular timeline --hard-copy copying archivelog not symbolic link Catalog options: -a, --show-all show deleted backup too Delete options: -f, --force forcibly delete backup older than given DATE Connection options: -d, --dbname=DBNAME database to connect -h, --host=HOSTNAME database server host or socket directory -p, --port=PORT database server port -U, --username=USERNAME user name to connect as -w, --no-password never prompt for password -W, --password force password prompt Generic options: -q, --quiet don't show any INFO or DEBUG messages --debug show DEBUG messages --help show this help, then exit --version output version information, then exit
使用前提
- 開啟歸檔
- 配置csvlog
安裝
下載地址：
https://github.com/ossc-db/pg_rman/releases
https://yum.postgresql.org/9.6/redhat/rhel-6.5-x86_64/
[root@racnode1 ~]# rpm --force -ivh openssl10-libs-1.0.1e-1.ius.el6.x86_64.rpm warning: openssl10-libs-1.0.1e-1.ius.el6.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 9cd4953f: NOKEY Preparing... ########################################### [100%] 1:openssl10-libs ########################################### [100%] [root@racnode1 ~]# rpm -ivh postgresql96-libs-9.6.4-1PGDG.rhel6.x86_64.rpm warning: postgresql96-libs-9.6.4-1PGDG.rhel6.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 442df0f8: NOKEY Preparing... ########################################### [100%] 1:postgresql96-libs ########################################### [100%] [root@racnode1 ~]# rpm -ivh pg_rman-1.3.5-1.pg96.el6.x86_64.rpm Preparing... ########################################### [100%] 1:pg_rman ########################################### [100%]
安裝完成后，pg_rman會安裝在默認的/usr/psql-9.6/目錄下
切換用戶到postgres用戶下
配置環境變量
export PG_RMAN=/usr/pgsql-9.6 export PATH=$PATH:$HOME/bin:/opt/pgsql95/bin:$PG_RMAN/bin export BACKUP_PATH=/data/pg_rman
初始化備份目錄
首先需要初始化一個backup catalog，實際上就是需要一個目錄，這個目錄將用於存放備份的文件。 同時這個目錄也會存放一些元數據，例如備份的配置文件，數據庫的systemid，時間線文件歷史等等。 初始化命令需要兩個參數，分別為備份目標目錄，以及數據庫的$PGDATA
[postgres@racnode1 ~]$ pg_rman init -B /data/pg_rman INFO: ARCLOG_PATH is set to '/home/postgres/arch' INFO: SRVLOG_PATH is set to '/home/postgres/data/pg_log' [root@racnode1 pg_rman]# ll total 16 drwx------. 4 postgres postgres 4096 Aug 30 00:06 backup -rw-rw-r--. 1 postgres postgres 76 Aug 30 00:06 pg_rman.ini -rw-rw-r--. 1 postgres postgres 40 Aug 30 00:06 system_identifier drwx------. 2 postgres postgres 4096 Aug 30 00:06 timeline_history
你可以把將來要使用的配置寫在這個配置文件中，或者寫在pg_rman的命令行中。
[postgres@racnode1 pg_rman]$ cat pg_rman.ini ARCLOG_PATH='/home/postgres/arch' SRVLOG_PATH='/home/postgres/data/pg_log' COMPRESS_DATA = YES KEEP_ARCLOG_FILES = 10 KEEP_ARCLOG_DAYS = 10 KEEP_DATA_GENERATIONS = 3 KEEP_DATA_DAYS = 10 KEEP_SRVLOG_FILES = 10 KEEP_SRVLOG_DAYS = 10
備份操作
全量備份 [postgres@racnode1 ~]$ pg_rman backup -b full INFO: copying database files INFO: copying archived WAL files INFO: backup complete INFO: Please execute 'pg_rman validate' to verify the files are correctly copied. INFO: start deleting old archived WAL files from ARCLOG_PATH (keep files = 10, keep days = 10) INFO: the threshold timestamp calculated by keep days is "2017-08-20 00:00:00" INFO: start deleting old backup (keep generations = 3 AND keep after = 2017-08-20 00:00:00) INFO: does not include the backup just taken WARNING: backup "2017-08-30 17:09:31" is not taken into account DETAIL: This is not a valid backup. 增量備份 [postgres@racnode1 ~]$ psql psql (9.6.0) Type "help" for help. postgres=# \c lottu lottu You are now connected to database "lottu" as user "lottu". lottu=# create table lottu as select generate_series(1,10) id; SELECT 10 [postgres@racnode1 ~]$ pg_rman backup -b incremental INFO: copying database files INFO: copying archived WAL files INFO: backup complete INFO: Please execute 'pg_rman validate' to verify the files are correctly copied. ... DETAIL: This is not valid backup.
備份集校驗
每次備份完，必須要做一次校驗，否則備份集不可用用來恢復，增量備份時也不會用它來做增量比較 [postgres@racnode1 ~]$ pg_rman validate INFO: validate: "2017-08-30 17:34:18" backup and archive log files by CRC INFO: backup "2017-08-30 17:34:18" is valid INFO: validate: "2017-08-30 17:37:36" backup and archive log files by CRC INFO: backup "2017-08-30 17:37:36" is valid [postgres@racnode1 ~]$ pg_rman show ===================================================================== StartTime EndTime Mode Size TLI Status ===================================================================== 2017-08-30 17:37:36 2017-08-30 17:37:45 FULL 3710kB 3 OK 2017-08-30 17:34:18 2017-08-30 17:34:27 FULL 3656kB 3 OK 2017-08-30 17:33:33 2017-08-30 17:33:45 FULL 3820kB 3 OK 2017-08-30 17:27:49 2017-08-30 17:27:58 FULL 3766kB 3 OK 2017-08-30 17:27:33 2017-08-30 17:27:43 FULL 3712kB 3 OK
查看備份集
[postgres@racnode1 ~]$ pg_rman show ===================================================================== StartTime EndTime Mode Size TLI Status ===================================================================== 2017-08-30 17:37:36 2017-08-30 17:37:45 FULL 3710kB 3 OK 2017-08-30 17:34:18 2017-08-30 17:34:27 FULL 3656kB 3 OK 2017-08-30 17:33:33 2017-08-30 17:33:45 FULL 3820kB 3 OK 2017-08-30 17:27:49 2017-08-30 17:27:58 FULL 3766kB 3 OK 2017-08-30 17:27:33 2017-08-30 17:27:43 FULL 3712kB 3 OK
刪除備份集
1. 按指定時間從catalog刪除備份集 例如我只需要我的備份集能恢復到2017-08-30 17:27:49，在這個時間點以前，不需要用來恢復到這個時間點的備份全刪掉。 [postgres@racnode1 ~]$ pg_rman delete "2017-08-30 17:27:49" WARNING: cannot delete backup with start time "2017-08-30 17:27:49" DETAIL: This is the latest full backup necessary for successful recovery. INFO: delete the backup with start time: "2017-08-30 17:27:33" 2. 根據備份策略來刪除備份集 KEEP_DATA_GENERATIONS = 3 -- 備份集冗余度是3 KEEP_DATA_DAYS = 10 -- 備份集保留日期是10d 在備份時；假如備份時間是2017-08-30。兩者條件是“和” INFO: start deleting old backup (keep generations = 3 AND keep after = 2017-08-20 00:00:00)
清除備份集
物理刪除已從catalog刪除的備份集 上面從catalog刪除的備份集;備份集文件夾並沒有一起刪除。只是把arclog/ database/ srvlog/刪除了。 [postgres@racnode1 ~]$ pg_rman purge INFO: DELETED backup "2017-08-30 17:27:33" is purged INFO: DELETED backup "2017-08-30 17:26:22" is purged INFO: DELETED backup "2017-08-30 17:15:26" is purged
恢復操作
pg_rman restore [options:] 命令的選項也很簡單，甚至可以不指定任何option Restore options: The parameters which are started with –recovery are same as parameters in recovery.conf. See also “Recovery Configuration” for details. 接下來的幾個配置，與recovery.conf的意思對齊。 如果不指定時間線，則使用$PGDATA/global/pg_control，如果沒有$PGDATA/global/pg_control，則使用最新的全量備份集的時間線。 --recovery-target-timeline TIMELINE Specifies recovering into a particular timeline. If not specified, the current timeline from ($PGDATA/global/pg_control) is used. 如果不指定，則恢復到最新時間 --recovery-target-time TIMESTAMP This parameter specifies the time stamp up to which recovery will proceed. If not specified, continue recovery to the latest time. 如果不指定，則恢復到最新xid --recovery-target-xid XID This parameter specifies the transaction ID up to which recovery will proceed. If not specified, continue recovery to the latest xid. 如果不指定，則默認使用true，即恢復到包含恢復目標XID的commit record為止，或者第一筆commit record ts>指定ts的 commit redo record為止； 如果是false則不apply恢復目標XID的commit record，或者不apply第一筆commit record ts>=指定ts的 commit redo record。 --recovery-target-inclusive Specifies whether we stop just after the specified recovery target (true), or just before the recovery target (false). Default is true. 是否使用硬鏈接復制archive log，而不需要拷貝文件 The following parameter determines the behavior of restore. --hard-copy The archive WAL are copied to archive WAL storage area. If not specified, pg_rman makes symbolic link to archive WAL where are in the backup catalog directory. 恢復時，需要注意，你可以選擇原地恢復（覆蓋式），或者使用新的$PGDATA作為恢復目標。 但是請注意，不管是哪種恢復方式，如果在本機恢復的話，pg_rman可能會覆蓋原有的數據文件，arch, pg_xlog目錄中的文件，所以，如果你要保留原數據，建議先將原數據目錄重命名。 1. 原地恢復 2. 使用新的$PGDATA恢復 [postgres@racnode1 ~]$ pg_rman restore --recovery-target-time "2017-08-30 19:48:00"; INFO: the recovery target timeline ID is not given INFO: use timeline ID of current database cluster as recovery target: 5 INFO: calculating timeline branches to be used to recovery target point INFO: searching latest full backup which can be used as restore start point INFO: found the full backup can be used as base in recovery: "2017-08-30 17:37:36" INFO: copying online WAL files and server log files INFO: clearing restore destination INFO: validate: "2017-08-30 17:37:36" backup and archive log files by SIZE INFO: backup "2017-08-30 17:37:36" is valid INFO: restoring database files from the full mode backup "2017-08-30 17:37:36" INFO: searching incremental backup to be restored INFO: validate: "2017-08-30 17:59:34" backup and archive log files by SIZE INFO: backup "2017-08-30 17:59:34" is valid INFO: restoring database files from the incremental mode backup "2017-08-30 17:59:34" INFO: searching backup which contained archived WAL files to be restored INFO: backup "2017-08-30 17:59:34" is valid INFO: restoring WAL files from backup "2017-08-30 17:59:34" INFO: restoring online WAL files and server log files INFO: generating recovery.conf INFO: restore complete HINT: Recovery will start automatically when the PostgreSQL server is started.
附上備份腳本
##=========================================================== ## pg_rman.sh ## created by lottu ## 2017/08/07 ## usage: pg_rman.sh ##============================================================ #!/bin/bash source /home/postgres/.bash_profile DATE=`date +%Y%m%d`; PG_HOME=/home/postgres BACK_LOG=/home/postgres/log/pg_rman_${DATE}.log #START BACKUP echo "START BACKUP............................................" > $BACK_LOG #執行備份命令 pg_rman backup -b full >> $BACK_LOG #備份集校驗 pg_rman validate >> $BACK_LOG #檢查備份是否成功 error_num=`pg_rman show | awk 'BEGIN{n=0}{if(NR > 3 && $8 != "OK")n++}END{print n}'` if [ $error_num > 0 ];then # message="Postgres 數據庫服務器${hostname}在${DATE}備份失敗" # 報警方式：微信，短信，郵件..... # wechat.sh $message echo "0" fi #清理無效備份集 pg_rman purge >> $BACK_LOG echo "BACKUP END............................................" >> $BACK_LOG
參考文獻：--https://yq.aliyun.com/articles/71929?spm=5176.100239.blogcont2274.11.N0xZJ5