ElasticSearch 快照 備份、恢復數據

目錄

• ElasticSearch 設置備份文件地址
• 注冊快照存儲庫
• 查看快照存儲庫保存結果
• 創建快照
  • 異步創建
  • 指定索引進行快照
• 在服務器查看備份的數據
• 恢復數據
  • 本機恢復
  • 其他服務器恢復
• 常見問題
  • 報錯 doesn't match any of the locations specified by path.repo because this setting is empty

ElasticSearch 設置備份文件地址

• 在elasticsearch.yml 新增

path.repo: ["/home/es/snapshot"]

注冊快照存儲庫

PUT _snapshot/ecopherebbs
{
    "type": "fs",
    "settings": {
        "location": "/home/es/snapshot"
    }
}

保存成功

查看快照存儲庫保存結果

GET _snapshot

返回結果

{
  "ecopherebbs" : {
    "type" : "fs",
    "settings" : {
      "location" : "/home/es/snapshot"
    }
  }
}

創建快照

異步創建

• 創建

PUT _snapshot/ecopherebbs/snapshot_20210631
{
  "ignore_unavailable": true,
   "include_global_state": true
}

• 查看狀態

GET _snapshot/ecopherebbs/snapshot_20210631

處理中

{
  "snapshots" : [
    {
      "snapshot" : "snapshot_20210631",
      "uuid" : "PTBkWV-pQ8CQ7l4RNxISfA",
      "version_id" : 7090399,
      "version" : "7.9.3",
      "indices" : [
        "t_e_s_t___2_0_2_1_0_4_2_1_0_9_2_9_index",
        "test0627",
        "test110",
        ".kibana-event-log-7.9.3-000001",
        "fang_fa_index",
        "filebeat-7.9.3-2021.06.29-000001",
        "gateway_requests",
        "my_index",
        "ilm-history-2-000003",
        ".kibana-event-log-7.9.3-000002",
        ".kibana_1"
      ],
      "data_streams" : [ ],
      "include_global_state" : true,
      "state" : "IN_PROGRESS",
      "start_time" : "2021-06-30T05:26:10.530Z",
      "start_time_in_millis" : 1625030770530,
      "end_time" : "1970-01-01T00:00:00.000Z",
      "end_time_in_millis" : 0,
      "duration_in_millis" : 0,
      "failures" : [ ],
      "shards" : {
        "total" : 0,
        "failed" : 0,
        "successful" : 0
      }
    }
  ]
}

處理完成

{
  "snapshots" : [
    {
      "snapshot" : "snapshot_20210631",
      "uuid" : "PTBkWV-pQ8CQ7l4RNxISfA",
      "version_id" : 7090399,
      "version" : "7.9.3",
      "indices" : [
        "t_e_s_t___2_0_2_1_0_4_2_1_0_9_2_9_index",
        "test0627",
        "test110",
        ".kibana-event-log-7.9.3-000001",
        "fang_fa_index",
        "filebeat-7.9.3-2021.06.29-000001",
        "gateway_requests",
        "my_index",
        "ilm-history-2-000003",
        ".kibana-event-log-7.9.3-000002",
        ".kibana_1"
      ],
      "data_streams" : [ ],
      "include_global_state" : true,
      "state" : "SUCCESS",
      "start_time" : "2021-06-30T05:26:10.530Z",
      "start_time_in_millis" : 1625030770530,
      "end_time" : "2021-06-30T05:26:16.937Z",
      "end_time_in_millis" : 1625030776937,
      "duration_in_millis" : 6407,
      "failures" : [ ],
      "shards" : {
        "total" : 200,
        "failed" : 0,
        "successful" : 200
      }
    }
  ]
}

指定索引進行快照

PUT _snapshot/ecopherebbs/snapshot_20210630?wait_for_completion=true
{
  "indices": "index1,index2",
  "ignore_unavailable": true,
   "include_global_state": true
}

1. wait_for_completion=true時會一直阻塞直到快照完成，kibana執行時超時為30秒，超時后快照依然會在后台執行
2. snapshot_20200729 就是指定的備份快照命中
3. 請求體重indices指定需要備份的索引（多個用逗號隔開），也可以不指定，這樣會備份所有的es索引庫數據

在服務器查看備份的數據

每次執行快照都會生成一個 meta和一個snap文件

[es@test12 snapshot]$ ls -lrt
總用量 1404
drwxrwxr-x. 62 es es   4096 6月  30 13:23 indices
-rw-rw-r--.  1 es es 460329 6月  30 13:24 meta--WCM58RpQNm5bEywCjO_cg.dat
-rw-rw-r--.  1 es es   1596 6月  30 13:24 snap--WCM58RpQNm5bEywCjO_cg.dat
-rw-rw-r--.  1 es es   1596 6月  30 13:26 snap-PTBkWV-pQ8CQ7l4RNxISfA.dat
-rw-rw-r--.  1 es es 460329 6月  30 13:26 meta-PTBkWV-pQ8CQ7l4RNxISfA.dat
-rw-rw-r--.  1 es es    302 6月  30 13:52 snap-N3zRMDnDTEqoiShEwjKosA.dat
-rw-rw-r--.  1 es es 460329 6月  30 13:52 meta-N3zRMDnDTEqoiShEwjKosA.dat
-rw-rw-r--.  1 es es  24990 6月  30 13:52 index-2
-rw-rw-r--.  1 es es      8 6月  30 13:52 index.latest

恢復數據

本機恢復

POST _snapshot/ecopherebbs/snapshot_20210631_02/_restore

執行恢復要將之前的索引刪除，否則會報錯

• 或者在恢復時執行新索引名稱 官網描述

POST /_snapshot/my_backup/snapshot_1/_restore
{
    "indices": "index_1", 
    "rename_pattern": "index_(.+)", 
    "rename_replacement": "restored_index_$1" 
}

其他服務器恢復

1. 配置第一步設置的目錄
   path.repo: ["/home/es/snapshot"]
2. 創建倉庫

PUT _snapshot/ecopherebbs
{
    "type": "fs",
    "settings": {
        "location": "/home/es/snapshot-test"
    }
}

3. 執行恢復索引

POST _snapshot/ecopherebbs/snapshot_20210631_02/_restore

常見問題

報錯 doesn't match any of the locations specified by path.repo because this setting is empty

{
  "error" : {
    "root_cause" : [
      {
        "type" : "repository_exception",
        "reason" : "[ecopherebbs] location [/home/es/snapshot] doesn't match any of the locations specified by path.repo because this setting is empty"
      }
    ],
    "type" : "repository_exception",
    "reason" : "[ecopherebbs] failed to create repository",
    "caused_by" : {
      "type" : "repository_exception",
      "reason" : "[ecopherebbs] location [/home/es/snapshot] doesn't match any of the locations specified by path.repo because this setting is empty"
    }
  },
  "status" : 500
}

• 解決方法：

在為es集群環境做災備和恢復時候，首先需要創建創建一個倉庫，並往倉庫中存放快照（每個快照中會區分不同的索引）。但是在創建倉庫的時候，要求倉庫的地址必須在每個集群環境中的elasticsearch.yml中進行配置

個人公眾號(大數據學習交流): hadoopwiki