Etcd 基礎維護


本文所有命令均在 TLS 環境下運行,如需參考,請自行更改為您的環境(節點IP,證書路徑),無證書環境請刪除證書相關指令

本文所有命令均在 etcdctl 默認api ,即 etcd api v2 下操作,v3 指令略有改動可能不匹配,詳情請查閱官方文檔:https://etcd.io/docs/

Etcd 使用

  • 舉例:創建、查詢、刪除 key ( /test/ok,值為 11)

# Etcd 錄入數據示例
ETCDCTL_API=3 etcdctl \
--endpoints=https://172.16.10.70:2379 \
--cacert=/etc/kubernetes/ssl/ca.pem \
--cert=/etc/etcd/ssl/etcd.pem \
--key=/etc/etcd/ssl/etcd-key.pem \
put /test/ok 11

# Etcd 查詢數據示例 ETCDCTL_API=3 etcdctl \ --endpoints=https://172.16.10.70:2379 \ --cacert=/etc/kubernetes/ssl/ca.pem \ --cert=/etc/etcd/ssl/etcd.pem \ --key=/etc/etcd/ssl/etcd-key.pem \ get /test/ok

# Etcd 刪除數據示例 ETCDCTL_API
=3 etcdctl \ --endpoints=https://172.16.10.70:2379 \ --cacert=/etc/kubernetes/ssl/ca.pem \ --cert=/etc/etcd/ssl/etcd.pem \ --key=/etc/etcd/ssl/etcd-key.pem \ del /test/ok

 

通過 Curl 來維護 Etcd

  • 查看版本

curl ‐k ‐‐cert /etc/etcd/ssl/etcd.pem ‐‐key /etc/etcd/ssl/etcd‐key.pem https://127.0.0.1:2379/version

 

  • 查看 Etcd 暴露出來的 prometheus 指標,在 prometheus 對其監控時可調用

curl ‐k ‐‐cert /etc/etcd/ssl/etcd.pem ‐‐key /etc/etcd/ssl/etcd‐key.pem https://127.0.0.1:2379/metrics

 

通過 Etcdctl 查看 版本

  • 查看 etcd、etcd api v2 版本

etcdctl -v

 

  • 查看 etcd、etcd api v3 版本

ETCDCTL_API=3 etcdctl version

 

查看集群健康狀態

etcdctl \
--endpoints=https://172.16.10.70:2379 \
--ca-file=/etc/kubernetes/ssl/ca.pem \
--cert-file=/etc/etcd/ssl/etcd.pem \
--key-file=/etc/etcd/ssl/etcd-key.pem \
cluster-health

 

查看集群成員、哪個是leader節點

etcdctl \
--endpoints=https://172.16.10.70:2379 \
--ca-file=/etc/kubernetes/ssl/ca.pem \
--cert-file=/etc/etcd/ssl/etcd.pem \
--key-file=/etc/etcd/ssl/etcd-key.pem \
member list

 

刪除 Etcd 節點

  • 查詢節點 ID

etcdctl \
‐‐endpoints=https://172.16.10.70:2379 \
‐‐ca‐file=/etc/kubernetes/ssl/ca.pem \
‐‐cert‐file=/etc/etcd/ssl/etcd.pem \
‐‐key‐file=/etc/etcd/ssl/etcd‐key.pem \
member list

340acbd004e6bcdb: name=etcd3 peerURLs=https://172.16.10.72:2380 clientURLs=https://172.16.10.72:2379
isLeader=false
9784cb04cceb3a48: name=etcd1 peerURLs=https://172.16.10.70:2380 clientURLs=https://172.16.10.70:2379
isLeader=true
ba343177666dd96e: name=etcd2 peerURLs=https://172.16.10.71:2380 clientURLs=https://172.16.10.71:2379
isLeader=false

 

  • 刪除節點,如刪除 Eecd3

etcdctl \
‐‐endpoints=https://172.16.10.70:2379 \
‐‐ca‐file=/etc/kubernetes/ssl/ca.pem \
‐‐cert‐file=/etc/etcd/ssl/etcd.pem \
‐‐key‐file=/etc/etcd/ssl/etcd‐key.pem \
member remove 340acbd004e6bcdb

 

  • 修改配置文件 etcd.conf,修改參數 ETCD_INITIAL_CLUSTER 並移除節點信息,重啟etcd服務

 

加入 Etcd 節點

已存在的 Etcd 節點故障重新添加(例 etcd3 重新添加)

1)在群集中刪除故障節點

  • 在任意一 etcd 節點服務器查詢該節點 ID,通過ID刪除故障節點,操作步驟如上

  • 刪除目標節點的數據

# 停止目標節點 etcd 服務
systemctl stop etcd

# 刪除前備份
cd /var/lib/ && mkdir ‐p etcd_bak && tar ‐czvf etcd_bak/etcd_`date +%Y%m%d%H%M%S`.tar.gz etcd

# 刪除節點數據
rm ‐rf /var/lib/etcd/*

 

2)編輯目標節點配置文件,將 --initial-cluster-state值為 existing (否則會生成新的ID,與原ID不匹配將無法加入集群)

vim /etc/etcd/etcd.conf

[member]
ETCD_NAME=etcd3
ETCD_DATA_DIR="/var/lib/etcd/"
ETCD_SNAPSHOT_COUNT="100"
ETCD_HEARTBEAT_INTERVAL="100"
ETCD_ELECTION_TIMEOUT="1000"
ETCD_LISTEN_PEER_URLS="https://172.16.10.72:2380"
ETCD_LISTEN_CLIENT_URLS="https://172.16.10.72:2379,https://127.0.0.1:2379"
ETCD_MAX_SNAPSHOTS="5"
ETCD_MAX_WALS="5"
# [cluster]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://172.16.10.72:2380"
ETCD_INITIAL_CLUSTER="etcd1=https://172.16.10.70:2380,etcd2=https://172.16.10.71:2380,etcd3=https://172.16.10.72:2380" ETCD_INITIAL_CLUSTER_STATE="existing"
ETCD_INITIAL_CLUSTER_TOKEN="k8s-etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="https://172.16.10.72:2379"
# [security]
ETCD_CERT_FILE="/etc/etcd/ssl/etcd.pem"
ETCD_KEY_FILE="/etc/etcd/ssl/etcd-key.pem"
ETCD_CLIENT_CERT_AUTH="true"
ETCD_TRUSTED_CA_FILE="/etc/kubernetes/ssl/ca.pem"
ETCD_AUTO_TLS="true"
ETCD_PEER_CERT_FILE="/etc/etcd/ssl/etcd.pem"
ETCD_PEER_KEY_FILE="/etc/etcd/ssl/etcd-key.pem"
ETCD_PEER_CLIENT_CERT_AUTH="true"
ETCD_PEER_TRUSTED_CA_FILE="/etc/kubernetes/ssl/ca.pem"
ETCD_PEER_AUTO_TLS="true"

 

3)加入節點至集群,需輸入目標節點的 etcd name 和 PEER_URLS

etcdctl \
‐‐endpoints=https://172.16.10.70:2379 \
‐‐ca‐file=/etc/kubernetes/ssl/ca.pem \
‐‐cert‐file=/etc/etcd/ssl/etcd.pem \
‐‐key‐file=/etc/etcd/ssl/etcd‐key.pem \
member add etcd3 https://172.16.10.72:2380

 

4)啟動目標節點 etcd 服務

systemctl start etcd && systemctl status etcd

 

5)查看集群健康狀態

etcdctl \
‐‐endpoints=https://172.16.10.70:2379 \
‐‐ca‐file=/etc/kubernetes/ssl/ca.pem \
‐‐cert‐file=/etc/etcd/ssl/etcd.pem \
‐‐key‐file=/etc/etcd/ssl/etcd‐key.pem \
cluster‐health

 

對 Etcd 進行快照備份

ETCDCTL_API=3 etcdctl \
‐‐endpoints=https://172.16.10.70:2379 \
‐‐cacert=/etc/kubernetes/ssl/ca.pem \
‐‐cert=/etc/etcd/ssl/etcd.pem \
‐‐key=/etc/etcd/ssl/etcd‐key.pem \
snapshot save /tmp/snapshot_`date +%Y%m%d%H%M%S`.db

ETCDCTL_API=3:表示使用etcd的v3版本的API接口
注:一定要添加ETCDCTL_API=3才能正常備份;如果不添加將無法備份

 

通過快照備份恢復 Etcd集群;(每個節點都要執行)

  • 停止 Etcd 服務

systemctl stop etcd

 

  • 備份並刪除當前 Etcd 數據 

cd /var/lib/ && mkdir ‐p etcd_bak && tar ‐czvf etcd_bak/etcd_`date +%Y%m%d%H%M%S`.tar.gz etcd ‐‐remov
e‐files

 

  • 還原快照鏡像

ETCDCTL_API=3 etcdctl \
‐‐cacert=/etc/kubernetes/ssl/ca.pem \
‐‐cert=/etc/etcd/ssl/etcd.pem \
‐‐key=/etc/etcd/ssl/etcd‐key.pem \
‐‐name etcd1 \
‐‐data‐dir=/var/lib/etcd \
‐‐initial‐cluster etcd1=https://172.16.10.70:2380,etcd2=https://172.16.10.71:2380,etcd3=https://172.16.10.72:2380 \
‐‐initial‐cluster‐token k8s‐etcd‐cluster \
‐‐initial‐advertise‐peer‐urls https://172.16.10.70:2380 \
snapshot restore /tmp/201912‐18_snapshot.db

‐‐name:表示當前etcd節點的名字(非主機名) ‐‐data‐dir:表示當前 etcd 節點的數據目錄 ‐‐initial‐cluster:集群中所有節點的peer訪問地址;例:etcd1
=https:///172.16.10.70:2380,etcd2=https:///172.16.10.71:2380,etcd3=https:///172.16.10.72:2380
‐‐initial‐cluster‐token:集群中各節點通信的token ‐‐initial‐advertise‐peer‐urls:當前節點對其它節點的通信地址 

 

  • 啟動 所有 Etcd 節點服務器

systemctl start etcd

 

  • 查看集群健康狀態

etcdctl \
‐‐endpoints=https://172.16.10.70:2379 \
‐‐ca‐file=/etc/kubernetes/ssl/ca.pem \
‐‐cert‐file=/etc/etcd/ssl/etcd.pem \
‐‐key‐file=/etc/etcd/ssl/etcd‐key.pem \
cluster‐health

 

沒有進行快照備份,通過數據目錄的 db 恢復

注意:此方法恢復數據可能不完整,僅建議極端環境下使用,常規數據恢復請使用快照

  • 如果當前 Etcd 集群故障,且沒有快照備份文件,可通過數據目錄的 db 恢復數據;

  • 從數據目錄 db 復制而來數據源,沒有完整性hash,需要 --skip-hash-check=true 參數跳過完整性檢查。

ETCDCTL_API=3 etcdctl \
--cacert=/etc/kubernetes/ssl/ca.pem \
--cert=/etc/etcd/ssl/etcd.pem \
--key=/etc/etcd/ssl/etcd-key.pem \
--name etcd3 \
--data-dir=/var/lib/etcd \
--initial-cluster etcd1=https://172.16.10.70:2380,etcd2=https://172.16.10.71:2380,etcd3=https://172.16.10.72:2380 \
--initial-cluster-token k8s-etcd-cluster \
--initial-advertise-peer-urls https://172.16.10.72:2380 \
--skip-hash-check=true \
snapshot restore /var/lib/etcd_bak/etcd/member/snap/db

--name:表示當前etcd節點的名字(非主機名)
--data-dir:表示當前 etcd 節點的數據目錄
--initial-cluster:集群中所有節點的peer訪問地址;例:https://172.16.10.70:2380,etcd2=https://172.16.10.71:2380,etcd3=https://172.16.10.72:2380
--initial-cluster-token:集群中各節點通信的token --initial-advertise-peer-urls:當前節點對其它節點的通信地址

 


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM