clickhouse-backup備份數據應用

版本:1.2.1

一、安裝

1.下載

wget https://github.com/AlexAkulov/clickhouse-backup/releases/download/v1.2.1/clickhouse-backup.tar.gz

下載地址
https://github.com/AlexAkulov/clickhouse-backup/releases

 注意 建議用root用戶操作，普通用戶注意設置文件目錄權限。

2.解壓即可使用

tar -xf clickhouse-backup.tar.gz
 cd clickhouse-backup/
 cp clickhouse-backup /usr/local/bin/

#查看版本
clickhouse-backup -v

Version: 1.2.1
Git Commit: 38cac6b647f46c3e076650d574eb1f2fb8c3ecf0
Build Date: 2021-10-30

 

3.編輯配置文件

mkdir -p /etc/clickhouse-backup/
vi /etc/clickhouse-backup/config.yml

 

#加一個必要設置即可【可不用指定數據目錄，注意配置文件格式，避免加載失敗】

general:
  remote_storage: none

【如需要更改ck服務器的賬號密碼，建議參照配置項調整。不設置使用default用戶以及初始密碼】

 

4.查看全部默認的配置項，可根據需求更改。

clickhouse-backup default-config

 

general:
  remote_storage: none
  max_file_size: 107374182400
  disable_progress_bar: true
  backups_to_keep_local: 0
  backups_to_keep_remote: 0
  log_level: info
  allow_empty_backups: false
  download_concurrency: 4
  upload_concurrency: 4
clickhouse:
  username: default
  password: ""
  host: localhost
  port: 9000
  disk_mapping: {}
  skip_tables:
  - system.*
  timeout: 5m
  freeze_by_part: false
  secure: false
  skip_verify: false
  sync_replicated_tables: false
  log_sql_queries: false
  config_dir: /etc/clickhouse-server/
  restart_command: systemctl restart clickhouse-server
  debug: false
s3:
  access_key: ""
  secret_key: ""
  bucket: ""
  endpoint: ""
  region: us-east-1
  acl: private
  assume_role_arn: ""
  force_path_style: false
  path: ""
  disable_ssl: false
  compression_level: 1
  compression_format: tar
  sse: ""
  disable_cert_verification: false
  storage_class: STANDARD
  concurrency: 1
  part_size: 0
  debug: false
gcs:
  credentials_file: ""
  credentials_json: ""
  bucket: ""
  path: ""
  compression_level: 1
  compression_format: tar
  debug: false
  endpoint: ""
cos:
  url: ""
  timeout: 2m
  secret_id: ""
  secret_key: ""
  path: ""
  compression_format: tar
  compression_level: 1
api:
  listen: localhost:7171
  enable_metrics: true
  enable_pprof: false
  username: ""
  password: ""
  secure: false
  certificate_file: ""
  private_key_file: ""
  create_integration_tables: false
ftp:
  address: ""
  timeout: 2m
  username: ""
  password: ""
  tls: false
  path: ""
  compression_format: tar
  compression_level: 1
  concurrency: 4
  debug: false
sftp:
  address: ""
  port: 22
  username: ""
  password: ""
  key: ""
  path: ""
  compression_format: tar
  compression_level: 1
  concurrency: 1
azblob:
  endpoint_suffix: core.windows.net
  account_name: ""
  account_key: ""
  sas: ""
  use_managed_identity: false
  container: ""
  path: ""
  compression_level: 1
  compression_format: tar
  sse_key: ""

 

# 查看可備份的表【可調整配置文件過濾不想備份的表】

 clickhouse-backup tables

 

二、備份

1.創建備份

clickhouse-backup create
#備份存儲在中 $data_path/backup 下，備份名稱默認為時間戳，可手動指定備份名稱。例如：
clickhouse-backup create bak20220309

 

備份包含兩個目錄一個文件：

‘metadata.json’文件：備份的庫表等信息
'metadata'目錄: 包含重新創建所需的DDL SQL
'shadow'目錄: 包含作為ALTER TABLE ... FREEZE操作結果的數據.【即原data文件中的表存儲的結構文件】

 

2.查看備份文件

clickhouse-backup list
2022-03-08T02-58-42 233.389.25KiB 08/03/2022 08:26:42 local
bak20220308   389.25KiB   08/03/2022 08:27:11   local

 

3.刪除備份

clickhouse-backup delete local 2022-03-08T02-58-42
clickhouse-backup delete local bak20220308   

 

三、恢復

1.本機恢復

1）刪除表

DROP TABLE test.tb_test_replacing2

Query id: b982e131-b64e-4ff0-b42b-2718ad669766

Ok.

0 rows in set. Elapsed: 0.002 sec. 

 

2）恢復

clickhouse-backup restore bak20220308 -table test.tb_test_replacing2
2022/03/09 09:33:36  info done                      backup=bak20220308 operation=restore table=test.tb_test_replacing2
2022/03/09 09:33:36  info done                      backup=bak20220308 duration=87ms operation=restore
2022/03/09 09:33:36  info done                      backup=bak20220308 operation=restore

 

2.異機恢復

1）將源端備份目錄打包並傳到恢復機器

tar -cvf bak20220308.tar ./bak20220308   
#scp rz 都可

 

2）恢復機器上創建備份目錄

 cd /data/clickhouse #建議和備份的目錄一致
 mkdir backup

#將源端備份的文件解壓並將目錄拷貝到恢復機器的備份目錄
tar -xvf bak20220308.tar
mv /tmp/bak20220308  /data/clickhouse/backup/

 

3）恢復

【注意這里需要配置/etc/clickhouse-backup/config.yml 指定正確的賬號密碼】

 clickhouse-backup list
bak20220308   389.25KiB   09/03/2022 08:47:11   local

 

再次執行恢復

clickhouse-backup restore bak20220308 -table test.tb_test_replacing2
2022/03/09 09:40:21  info done                      backup=bak20220308 operation=restore table=test.tb_test_replacing2
2022/03/09 09:40:21  info done                      backup=bak20220308 duration=172ms operation=restore
2022/03/09 09:40:21  info done                      backup=bak20220308 operation=restore

3.查看恢復情況

clickhouse-client --m