etcd集群數據遷移至新集群

舊ETCD環境數據備份

備份V2:

etcdctl  backup --data-dir /var/lib/etcd --backup-dir /opt/etcdv2

注：此處的數據目錄為： /var/lib/etcd ，備份路徑為：/opt/etcdv2

備份V3:

ETCDCTL_API=3 etcdctl snapshot save /opt/etcdv2/member/snap/db

注：此處的數據備份目錄為 /opt/etcdv2/member/snap/db，路徑和v2的備份路徑相關聯，具體關聯如下：<v2-backdir>/member/snap/db

數據拷貝至新節點

舊節點數據打包：

zip -r etcdv2.zip  /opt/etcdv2

傳送至新節點：

scp  etcdv2.zip root@xxxx:/opt  # scp至新機器（一台機器即可,這里傳到了new-01節點上）

新集群恢復

1.解壓備份文件並放在etcd數據目錄下

unzip /opt/etcdv2 && mv /opt/etcdv2/member /var/lib/etcd/infra1.etcd/

2.啟動新節點(new-01節點)

因為備份的數據中，存在舊服務的集群信息，因為我們進行了遷移，需要將原本的集群信息覆蓋掉(不影響用戶數據)，啟動參數中添加配置--force-new-cluster，等服務成功啟動后，舊集群信息已被覆蓋，然后去掉此配置，重啟服務即可

注：節點配置中，請勿過早添加其他節點信息，只需配置當前節點的信息即可，后面會依次加入新節點信息

new-01節點 etcd配置預覽

etcd.service  折疊源碼

[root@prod-k8s-01 ~] # cat /usr/lib/systemd/system/etcd.service

[Unit]

Description=Etcd Server

After=network.target

After=network-online.target

Wants=network-online.target

  

[Service]

Type=notify

EnvironmentFile=- /etc/etcd/etcd .conf

ExecStart= /usr/bin/etcd  \

--name=${ETCD_NAME} \

--data- dir =${ETCD_DATA_DIR} \

--listen-peer-urls=${ETCD_LISTEN_PEER_URLS} \

--listen-client-urls=${ETCD_LISTEN_CLIENT_URLS} \

--advertise-client-urls=${ETCD_ADVERTISE_CLIENT_URLS} \

--initial-advertise-peer-urls=${ETCD_INITIAL_ADVERTISE_PEER_URLS} \

--initial-cluster=${ETCD_INITIAL_CLUSTER} \

--initial-cluster-token=${ETCD_INITIAL_CLUSTER_TOKEN} \

--initial-cluster-state=${ETCD_INITIAL_CLUSTER_STATE} \

--force-new-cluster

Restart=on-failure

LimitNOFILE=65536

  

[Install]

WantedBy=multi-user.target

etcd.conf  折疊源碼

[root@prod-k8s-01 ~] # cat /etc/etcd/etcd.conf

# configure file for etcd.service

# [member]

ETCD_NAME=infra1

ETCD_DATA_DIR= "/var/lib/etcd/infra1.etcd"

ETCD_LISTEN_PEER_URLS= "http://10.94.19.179:2380"

ETCD_LISTEN_CLIENT_URLS= "http://10.94.19.179:2379,http://127.0.0.1:2379"

#

# [cluster]

ETCD_INITIAL_ADVERTISE_PEER_URLS= "http://10.94.19.179:2380"

ETCD_INITIAL_CLUSTER= "infra1=http://10.94.19.179:2380"

ETCD_INITIAL_CLUSTER_STATE= "new"

ETCD_INITIAL_CLUSTER_TOKEN= "etcd-cluster"

ETCD_ADVERTISE_CLIENT_URLS= "http://10.94.19.179:2379"

啟動服務

[root@prod-k8s-01 ~]#systemctl  start etcd

服務啟動成功后，把/usr/lib/systemd/system/etcd.service中的--force-new-cluster參數刪除，再次重啟etcd即可。

 

3.修正當前節點的peerURLs
在遷移過程中，出現了當前節點的peerURLs錯誤的問題，需要修正下

查看節點信息：

[root@prod-k8s-01 ~]# etcdctl member list
76926a56d901: name=infra1 peerURLs=http://10.94.19.179:2379 clientURLs=http://10.94.19.179:2379 isLeader=false

其中peerURLs=http://10.94.19.179:2379和配置中不相同，需要重新設置：

[root@prod-k8s-01 ~]# etcdctl member update 76926a56d901 http://10.94.19.179:2380 # 更改節點peerurls

至此，我們已經成功在新集群恢復了舊集群的數據，但是服務只有一個節點，不符合高可用要求，需要我們添加更多節點，以滿足高可用

 

加入新節點

1.加入節點2

[root@prod-k8s-01 ~]# etcdctl member add infra2 http://10.94.19.180:2380

Added member named infra2 with ID ee0b43bac93847e to cluster ETCD_NAME="infra2" 

ETCD_INITIAL_CLUSTER="infra1=http://10.94.19.179:2380,infra2=http://10.94.19.180:2380" 

ETCD_INITIAL_CLUSTER_STATE="existing"

 

new-02節點 etcd配置預覽

etcd.conf  折疊源碼

[root@prod-k8s-02 ~] # cat /etc/etcd/etcd.conf

# configure file for etcd.service

# [member]

ETCD_NAME=infra2

ETCD_DATA_DIR= "/var/lib/etcd/infra2.etcd"

ETCD_LISTEN_PEER_URLS= "http://10.94.19.180:2380"

ETCD_LISTEN_CLIENT_URLS= "http://10.94.19.180:2379,http://127.0.0.1:2379"

#

# [cluster]

ETCD_INITIAL_ADVERTISE_PEER_URLS= "http://10.94.19.180:2380"

ETCD_INITIAL_CLUSTER= "infra1=http://10.94.19.179:2380,infra2=http://10.94.19.180:2380"

ETCD_INITIAL_CLUSTER_STATE= "existing"

ETCD_INITIAL_CLUSTER_TOKEN= "etcd-cluster"

ETCD_ADVERTISE_CLIENT_URLS= "http://10.94.19.180:2379"

etcd.service  折疊源碼

[root@prod-k8s-02 ~] # cat /usr/lib/systemd/system/etcd.service

[Unit]

Description=Etcd Server

After=network.target

After=network-online.target

Wants=network-online.target

  

[Service]

Type=notify

EnvironmentFile=- /etc/etcd/etcd .conf

ExecStart= /usr/bin/etcd  \

--name=${ETCD_NAME} \

--data- dir =${ETCD_DATA_DIR} \

--listen-peer-urls=${ETCD_LISTEN_PEER_URLS} \

--listen-client-urls=${ETCD_LISTEN_CLIENT_URLS} \

--advertise-client-urls=${ETCD_ADVERTISE_CLIENT_URLS} \

--initial-advertise-peer-urls=${ETCD_INITIAL_ADVERTISE_PEER_URLS} \

--initial-cluster=${ETCD_INITIAL_CLUSTER} \

--initial-cluster-token=${ETCD_INITIAL_CLUSTER_TOKEN} \

--initial-cluster-state=${ETCD_INITIAL_CLUSTER_STATE}

Restart=on-failure

LimitNOFILE=65536

  

[Install]

WantedBy=multi-user.target

啟動02節點，其中關鍵配置需要設置成上面輸出的信息。

查看節點列表

[root@prod-k8s-01 ~]# etcdctl member list
76926a56d901: name=infra1 peerURLs=http://10.94.19.179:2380 clientURLs=http://10.94.19.179:2379 isLeader=false
ee0b43bac93847e: name=infra2 peerURLs=http://10.94.19.180:2380 clientURLs=http://10.94.19.180:2379 isLeader=true

 

2.加入節點3

[root@prod-k8s-01 ~]# etcdctl member add infra3 http://10.94.19.181:2380
Added member named infra4 with ID 58805300c0ea60c2 to cluster

ETCD_NAME="infra3"
ETCD_INITIAL_CLUSTER="infra1=http://10.94.19.179:2380,infra2=http://10.94.19.180:2380,infra3=http://10.94.19.181:2380"
ETCD_INITIAL_CLUSTER_STATE="existing"

new-03節點 etcd配置預覽

etcd.conf  折疊源碼

[root@prod-k8s-03 ~] # cat /etc/etcd/etcd.conf

# configure file for etcd.service

# [member]

ETCD_NAME=infra3

ETCD_DATA_DIR= "/var/lib/etcd/infra3.etcd"

ETCD_LISTEN_PEER_URLS= "http://10.94.19.181:2380"

ETCD_LISTEN_CLIENT_URLS= "http://10.94.19.181:2379,http://127.0.0.1:2379"

#

# [cluster]

ETCD_INITIAL_ADVERTISE_PEER_URLS= "http://10.94.19.181:2380"

ETCD_INITIAL_CLUSTER= "infra1=http://10.94.19.179:2380,infra2=http://10.94.19.180:2380,infra3=http://10.94.19.181:2380"

ETCD_INITIAL_CLUSTER_STATE= "existing"

ETCD_INITIAL_CLUSTER_TOKEN= "etcd-cluster"

ETCD_ADVERTISE_CLIENT_URLS= "http://10.94.19.181:2379"

etcd.service  折疊源碼

[root@prod-k8s-03 ~] # cat /usr/lib/systemd/system/etcd.service

[Unit]

Description=Etcd Server

After=network.target

After=network-online.target

Wants=network-online.target

  

[Service]

Type=notify

EnvironmentFile=- /etc/etcd/etcd .conf

ExecStart= /usr/bin/etcd  \

--name=${ETCD_NAME} \

--data- dir =${ETCD_DATA_DIR} \

--listen-peer-urls=${ETCD_LISTEN_PEER_URLS} \

--listen-client-urls=${ETCD_LISTEN_CLIENT_URLS} \

--advertise-client-urls=${ETCD_ADVERTISE_CLIENT_URLS} \

--initial-advertise-peer-urls=${ETCD_INITIAL_ADVERTISE_PEER_URLS} \

--initial-cluster=${ETCD_INITIAL_CLUSTER} \

--initial-cluster-token=${ETCD_INITIAL_CLUSTER_TOKEN} \

--initial-cluster-state=${ETCD_INITIAL_CLUSTER_STATE}

Restart=on-failure

LimitNOFILE=65536

  

[Install]

WantedBy=multi-user.target

啟動03節點，其中關鍵配置需要設置成上面輸出的信息。

查看節點列表

[root@prod-k8s-01 ~]# etcdctl member list
76926a56d901: name=infra1 peerURLs=http://10.94.19.179:2380 clientURLs=http://10.94.19.179:2379 isLeader=false
ee0b43bac93847e: name=infra2 peerURLs=http://10.94.19.180:2380 clientURLs=http://10.94.19.180:2379 isLeader=true
58805300c0ea60c2: name=infra3 peerURLs=http://10.94.19.181:2380 clientURLs=http://10.94.19.181:2379 isLeader=false

 

調整new-01、new-02節點配置

修改new-01、new-02節點etcd.conf配置， 把ETCD_INITIAL_CLUSTER參數的值改成和new-03節點ETCD_INITIAL_CLUSTER參數一致，然后重啟new-01、new-02 etcd服務。

 

查看新集群狀態

[root@prod-k8s-01 ~]# etcdctl cluster-health
member 76926a56d901 is healthy: got healthy result from http://10.94.19.179:2379
member ee0b43bac93847e is healthy: got healthy result from http://10.94.19.180:2379
member 58805300c0ea60c2 is healthy: got healthy result from http://10.94.19.181:2379
cluster is healthy