PG-pg_probackup備份恢復工具

pg_probackup

pg_probackup是由Postgres Professional發布的一個用於管理PostgreSQL 9.5 及以上版本數據庫備份和恢復的工具。pg_probackup 實用程序基於 pg_arman，它最初由 NTT 編寫，然后由 Michael Paquier 開發和維護。

安裝配置

軟件安裝

源碼安裝

wget https://codeload.github.com/postgrespro/pg_probackup/tar.gz/refs/tags/2.5.1

make USE_PGXS=1 PG_CONFIG=<path_to_pg_config> top_srcdir=<path_to_PostgreSQL_source_tree>
make USE_PGXS=1 PG_CONFIG=<path_to_pg_config> top_srcdir=<path_to_PostgreSQL_source_tree> install

# 不使用 PGXS
cd <path_to_PostgreSQL_source_tree> && git clone https://github.com/postgrespro/pg_probackup contrib/pg_probackup && cd contrib/pg_probackup && make

rpm包

# pg_probackup for PostgresPro Standard and Enterprise
#DEB Ubuntu|Debian Packages
sudo sh -c 'echo "deb [arch=amd64] https://repo.postgrespro.ru/pg_probackup-forks/deb/ $(lsb_release -cs) main-$(lsb_release -cs)" > /etc/apt/sources.list.d/pg_probackup-forks.list'
sudo wget -O - https://repo.postgrespro.ru/pg_probackup-forks/keys/GPG-KEY-PG_PROBACKUP | sudo apt-key add - && sudo apt-get update
sudo apt-get install pg-probackup-{std,ent}-{12,11,10,9.6}
sudo apt-get install pg-probackup-{std,ent}-{12,11,10,9.6}-dbg

#DEB Astra Linix Orel
sudo sh -c 'echo "deb [arch=amd64] https://repo.postgrespro.ru/pg_probackup-forks/deb/ stretch main-stretch" > /etc/apt/sources.list.d/pg_probackup.list'
sudo wget -O - https://repo.postgrespro.ru/pg_probackup-forks/keys/GPG-KEY-PG_PROBACKUP | sudo apt-key add - && sudo apt-get update
sudo apt-get install pg-probackup-{std,ent}-{12,11,10,9.6}{-dbg,}


#RPM Centos Packages
rpm -ivh https://repo.postgrespro.ru/pg_probackup-forks/keys/pg_probackup-repo-forks-centos.noarch.rpm
yum install pg_probackup-{std,ent}-{12,11,10,9.6}
yum install pg_probackup-{std,ent}-{12,11,10,9.6}-debuginfo

#RPM RHEL Packages
rpm -ivh https://repo.postgrespro.ru/pg_probackup-forks/keys/pg_probackup-repo-forks-rhel.noarch.rpm
yum install pg_probackup-{std,ent}-{12,11,10,9.6}
yum install pg_probackup-{std,ent}-{12,11,10,9.6}-debuginfo

#RPM Oracle Linux Packages
rpm -ivh https://repo.postgrespro.ru/pg_probackup-forks/keys/pg_probackup-repo-forks-oraclelinux.noarch.rpm
yum install pg_probackup-{std,ent}-{12,11,10,9.6}
yum install pg_probackup-{std,ent}-{12,11,10,9.6}-debuginfo

#RPM ALT Linux 7
sudo sh -c 'echo "rpm https://repo.postgrespro.ru/pg_probackup-forks/rpm/latest/altlinux-p7 x86_64 forks" > /etc/apt/sources.list.d/pg_probackup_forks.list'
sudo apt-get update
sudo apt-get install pg_probackup-{std,ent}-{12,11,10,9.6}
sudo apt-get install pg_probackup-{std,ent}-{12,11,10,9.6}-debuginfo

#RPM ALT Linux 8
sudo sh -c 'echo "rpm https://repo.postgrespro.ru/pg_probackup-forks/rpm/latest/altlinux-p8 x86_64 forks" > /etc/apt/sources.list.d/pg_probackup_forks.list'
sudo apt-get update
sudo apt-get install pg_probackup-{std,ent}-{12,11,10,9.6}
sudo apt-get install pg_probackup-{std,ent}-{12,11,10,9.6}-debuginfo

#RPM ALT Linux 9
sudo sh -c 'echo "rpm https://repo.postgrespro.ru/pg_probackup-forks/rpm/latest/altlinux-p9 x86_64 forks" > /etc/apt/sources.list.d/pg_probackup_forks.list' && sudo apt-get update
sudo apt-get install pg_probackup-{std,ent}-{12,11,10,9.6}
sudo apt-get install pg_probackup-{std,ent}-{12,11,10,9.6}-debuginfo

配置

用法

pg_probackup version

pg_probackup help [command]

pg_probackup init -B backup_dir

pg_probackup add-instance -B backup_dir -D data_dir --instance instance_name

pg_probackup del-instance -B backup_dir --instance instance_name

pg_probackup set-config -B backup_dir --instance instance_name [option...]

pg_probackup set-backup -B backup_dir --instance instance_name -i backup_id [option...]

pg_probackup show-config -B backup_dir --instance instance_name [--format=format]

pg_probackup show -B backup_dir [option...]

pg_probackup backup -B backup_dir --instance instance_name -b backup_mode [option...]

pg_probackup restore -B backup_dir --instance instance_name [option...]

pg_probackup checkdb -B backup_dir --instance instance_name -D data_dir [option...]

pg_probackup validate -B backup_dir [option...]

pg_probackup merge -B backup_dir --instance instance_name -i backup_id [option...]

pg_probackup delete -B backup_dir --instance instance_name { -i backup_id | --delete-wal | --delete-expired | --merge-expired } [option...]

pg_probackup archive-push -B backup_dir --instance instance_name --wal-file-name wal_file_name [option...]

pg_probackup archive-get -B backup_dir --instance instance_name --wal-file-path wal_file_path --wal-file-name wal_file_name [option...]

pg_probackup catchup -b catchup_mode --source-pgdata=path_to_pgdata_on_remote_server --destination-pgdata=path_to_local_dir [option...]

初始化catalog

用於存放備份文件和歸檔日志文件

pg_probackup init -B /data/postgres/probackup

完成上面命令后，它會創建下面的子目錄

• wal/ — directory for WAL files.
• backups/ — directory for backup files.

向備份目錄添加新的備份實例

pg_probackup add-instance -B backup_dir -D data_dir --instance instance_name [remote_options]

#-- 本地添加備份instance 示例
pg_probackup add-instance -B /data/postgres/probackup -D $PGDATA --instance local5432

• data_dir是要備份的群集的數據目錄。要設置和使用pg_probackup，需要對此目錄的寫訪問權限。
• instance_name是將存儲此群集的WAL和備份文件的子目錄的名稱。
• remote_options是可選參數，僅當data_dir位於遠程系統上時，才需要指定這些參數。

配置PostgreSQL集群

創建用戶

建議創建一個具有所選備份策略所需的最低權限的單獨角色運行pg_probackup使用程序

• For Postgres Pro 9.5

  BEGIN;
  CREATE ROLE backup WITH LOGIN;
  GRANT USAGE ON SCHEMA pg_catalog TO backup;
  GRANT EXECUTE ON FUNCTION pg_catalog.current_setting(text) TO backup;
  GRANT EXECUTE ON FUNCTION pg_catalog.pg_is_in_recovery() TO backup;
  GRANT EXECUTE ON FUNCTION pg_catalog.pg_start_backup(text, boolean) TO backup;
  GRANT EXECUTE ON FUNCTION pg_catalog.pg_stop_backup() TO backup;
  GRANT EXECUTE ON FUNCTION pg_catalog.pg_create_restore_point(text) TO backup;
  GRANT EXECUTE ON FUNCTION pg_catalog.pg_switch_xlog() TO backup;
  GRANT EXECUTE ON FUNCTION pg_catalog.txid_current() TO backup;
  GRANT EXECUTE ON FUNCTION pg_catalog.txid_current_snapshot() TO backup;
  GRANT EXECUTE ON FUNCTION pg_catalog.txid_snapshot_xmax(txid_snapshot) TO backup;
  COMMIT;
  

• For Postgres Pro 9.6

  BEGIN;
  CREATE ROLE backup WITH LOGIN;
  GRANT USAGE ON SCHEMA pg_catalog TO backup;
  GRANT EXECUTE ON FUNCTION pg_catalog.current_setting(text) TO backup;
  GRANT EXECUTE ON FUNCTION pg_catalog.pg_is_in_recovery() TO backup;
  GRANT EXECUTE ON FUNCTION pg_catalog.pg_start_backup(text, boolean, boolean) TO backup;
  GRANT EXECUTE ON FUNCTION pg_catalog.pg_stop_backup(boolean) TO backup;
  GRANT EXECUTE ON FUNCTION pg_catalog.pg_create_restore_point(text) TO backup;
  GRANT EXECUTE ON FUNCTION pg_catalog.pg_switch_xlog() TO backup;
  GRANT EXECUTE ON FUNCTION pg_catalog.pg_last_xlog_replay_location() TO backup;
  GRANT EXECUTE ON FUNCTION pg_catalog.txid_current() TO backup;
  GRANT EXECUTE ON FUNCTION pg_catalog.txid_current_snapshot() TO backup;
  GRANT EXECUTE ON FUNCTION pg_catalog.txid_snapshot_xmax(txid_snapshot) TO backup;
  GRANT EXECUTE ON FUNCTION pg_catalog.pg_control_checkpoint() TO backup;
  COMMIT;
  

• For Postgres Pro 10 or higher:

BEGIN;
CREATE ROLE backup WITH LOGIN;
GRANT USAGE ON SCHEMA pg_catalog TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.current_setting(text) TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.pg_is_in_recovery() TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.pg_start_backup(text, boolean, boolean) TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.pg_stop_backup(boolean, boolean) TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.pg_create_restore_point(text) TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.pg_switch_wal() TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.pg_last_wal_replay_lsn() TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.txid_current() TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.txid_current_snapshot() TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.txid_snapshot_xmax(txid_snapshot) TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.pg_control_checkpoint() TO backup;
COMMIT;

• 使用流復制配置

  ALTER ROLE backup WITH REPLICATION;
  

服務端參數配置

確保有連續 WAL 歸檔配置

• 確保wal_level參數高於minimal

• 如果要在primary上配置存檔，則存檔模式必須設置為“on”或“always”。要在standby下執行存檔，請將此參數設置為“always"

• archive_command 參數配置

  archive_command = '"install_dir/pg_probackup" archive-push -B "backup_dir" --instance instance_name --wal-file-name=%f [remote_options]'
  # -- 示例
  /opt/pg12/bin/pg_probackup archive-push -B /data/postgres/probackup --instance local_6000 --wal-file-path=%p --wal-file-name=%f
  

設置集群驗證

• 在集群的每個數據庫中安裝 amcheck 或 amcheck_next 擴展：

CREATE EXTENSION amcheck;

• 為集群的每個數據庫中的備份角色授予以下權限：

  GRANT SELECT ON TABLE pg_catalog.pg_am TO backup;
  GRANT SELECT ON TABLE pg_catalog.pg_class TO backup;
  GRANT SELECT ON TABLE pg_catalog.pg_database TO backup;
  GRANT SELECT ON TABLE pg_catalog.pg_namespace TO backup;
  GRANT SELECT ON TABLE pg_catalog.pg_extension TO backup;
  GRANT EXECUTE ON FUNCTION bt_index_check(regclass) TO backup;
  GRANT EXECUTE ON FUNCTION bt_index_check(regclass, bool) TO backup;
  

使用Partial Restore（部分還原）配置

將 pg_catalog.pg_database的只讀權限給backup用戶角色

GRANT SELECT ON TABLE pg_catalog.pg_database TO backup;

使用遠程備份模式

• 配置SSH互信

使用PTRACK 備份配置

如果要使用 PTRACK 備份，將執行 PTRACK 備份的角色必須有權訪問集群的所有數據庫。

• 創建插件

  CREATE EXTENSION ptrack;
  

• 啟用跟蹤頁面更新參數（tracking page updates）

  # N 是 Postgres Pro 集群的大小，以 MB 為單位
  ptrack.map_size =  N / 1024
  

使用

創建一個備份

pg_probackup backup -B backup_dir --instance instance_name -b backup_mode

backup_mode 取值：

• FULL：完整備份
• DELTA ：自上次備份以來發生更改的頁面創建增量備份
• PAGE ：基於自上次完整或增量備份后生成的 WAL 文件創建增量備份。僅從數據文件中讀取更改的塊。
• PTRACK ：動態創建增量備份跟蹤頁面更改

查看備份

pg_probackup show -B /data/postgres/probackup

恢復備份

全量恢復

pg_probackup restore -B backup_dir --instance instance_name -i backup_id

Partial Restore

僅恢復指定的數據庫

pg_probackup restore -B backup_dir --instance instance_name --db-include=database_name

pg_probackup restore -B backup_dir --instance instance_name --db-include=db1 --db-include=db2

從還原中排除一個或多個數據庫

pg_probackup restore -B backup_dir --instance instance_name --db-exclude=database_name

pg_probackup restore -B backup_dir --instance instance_name --db-exclude=db1 --db-exclude=db2

基於時間點 (PITR) 恢復

要在准確時間恢復集群

pg_probackup restore -B backup_dir --instance instance_name --recovery-target-time="2021-05-18 14:18:11+03"

將集群恢復到特定事務 ID

pg_probackup restore -B backup_dir --instance instance_name --recovery-target-xid=687

恢復到特定 LSN

pg_probackup restore -B backup_dir --instance instance_name --recovery-target-lsn=16/B374D848

還原到特定命名的還原點

pg_probackup restore -B backup_dir --instance instance_name --recovery-target-name="before_app_upgrade"

將備份恢復到 WAL 存檔中可用的最新狀態

pg_probackup restore -B backup_dir --instance instance_name --recovery-target="latest"

將集群恢復到最早的一致性點

pg_probackup restore -B backup_dir --instance instance_name --recovery-target='immediate'

使用遠程備份恢復模式

遠程備份

# postgres 用戶通過端口 2302 的 SSH 連接創建位於遠程系統上的 Postgres Pro 集群的歸檔完整備份，主機地址為 192.168.0.2
pg_probackup backup -B backup_dir --instance instance_name -b FULL --remote-user=postgres --remote-host=192.168.0.2 --remote-port=2302

使用遠程恢復

# postgres 用戶通過端口 2302 的 SSH 連接在主機地址為 192.168.0.2 的遠程系統上恢復最新的可用備份
pg_probackup restore -B backup_dir --instance instance_name --remote-user=postgres --remote-host=192.168.0.2 --remote-port=2302