k8s集群安裝etcd組件部署問題

https://www.jianshu.com/p/1f9ba144ef34

etcd v3.4.9

使用member list查詢etcd狀態或者使用endpoint health查詢群集狀態時

#etcdctl member list

出現如下信息，切記不是報錯信息，只是通過客戶端訪問的時候需要帶上證書訪問

{"level":"warn","ts":"2021-02-23T02:42:32.148-0500","caller":"clientv3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-633d3464-2a3d-432c-a269-01eb26d31ba0/127.0.0.1:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest balancer error: all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused\""}

Error: context deadline exceeded

#etcdctl endpoint health

{"level":"warn","ts":"2021-02-23T02:42:32.148-0500","caller":"clientv3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-633d3464-2a3d-432c-a269-01eb26d31ba0/127.0.0.1:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest balancer error: all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused\""}

Error: context deadline exceeded

 

正確的訪問方法：

#etcdctl --cacert=/opt/etcd/ssl/ca.pem --cert=/opt/etcd/ssl/server.pem --key=/opt/etcd/ssl/server-key.pem \

>--endpoints="https://192.168.100.71:2379,https://192.168.100.72:2379,https://192.168.100.73:2379" member list

結果：

41077e602e1d7711, started, etcd-3, https://192.168.100.73:2380, https://192.168.100.73:2379, false

e3dca3d7a066519b, started, etcd-2, https://192.168.100.72:2380, https://192.168.100.72:2379, false

e8e1060c65b6e78b, started, etcd-1, https://192.168.100.71:2380, https://192.168.100.71:2379, false

作者：Landely
鏈接：https://www.jianshu.com/p/1f9ba144ef34
來源：簡書
著作權歸作者所有。商業轉載請聯系作者獲得授權，非商業轉載請注明出處。

 

 

 

https://blog.csdn.net/snipercai/article/details/101012124

 

主機列表
本次實驗選擇5台主機，3台作為master主機，2台作為node節點

節點ip OS版本 hostname -f 安裝軟件
192.168.0.1 RHEL7.4 k8s-master01 docker,etcd,flanneld,kube-apiserver,kube-controller-manager,kube-scheduler
192.168.0.2 RHEL7.4 k8s-master02 docker,etcd,flanneld,kube-apiserver,kube-controller-manager,kube-scheduler
192.168.0.3 RHEL7.4 k8s-master03 docker,etcd,flanneld,kube-apiserver,kube-controller-manager,kube-scheduler
192.168.0.4 RHEL7.4 k8s-node01 docker,flanneld,kubelet,kube-proxy
192.168.0.5 RHEL7.4 k8s-node02 docker,flanneld,kubelet,kube-proxy
下載安裝包（etcd最新版本3.4.0）
wget https://github.com/etcd-io/etcd/releases/download/v3.4.0/etcd-v3.4.0-linux-amd64.tar.gz
tar -xvf etcd-v3.3.10-linux-amd64.tar.gz
cp etcd etcdctl /k8s/etcd/bin/

修改配置文件
cat << EOF > /k8s/etcd/cfg/etcd
#[Member]
ETCD_NAME="etcd01"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.0.1:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.0.1:2379"

#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.0.1:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.0.1:2379"
ETCD_INITIAL_CLUSTER="etcd01=https://192.168.0.1:2380,etcd02=https://192.168.0.2:2380,etcd03=https://192.168.0.3:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_ENABLE_V2="true"
EOF

 提示：

其他etcd節點按照以上配置文件修改其中紅字部分即可
flannel操作etcd使用的是v2的API，而kubernetes操作etcd使用的v3的API，為了兼容flannel，將默認開啟v2版本，故配置文件中設置 ETCD_ENABLE_V2="true"
創建TLS 密鑰和證書
為了保證通信安全，客戶端(如etcdctl)與etcd 集群、etcd 集群之間的通信需要使用TLS 加密。
創建etcd 證書簽名請求：

cat > etcd-csr.json <<EOF
{
  "CN": "etcd",
  "hosts": [
    "192.168.0.1",
    "192.168.0.2",
    "192.168.0.3"
  ],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "System"
    }
  ]
}
EOF

生成etcd證書和私鑰
# cfssl gencert -ca=/k8s/kubernetes/ssl/ca.pem -ca-key=/k8s/kubernetes/ssl/ca-key.pem  -config=/k8s/kubernetes/ssl/ca-config.json  -profile=kubernetes etcd-csr.json | cfssljson -bare etcd

# ls etcd*
etcd.csr  etcd-csr.json  etcd-key.pem  etcd.pem

創建 etcd的 systemd unit 文件
cat << EOF > /lib/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
EnvironmentFile=/k8s/etcd/cfg/etcd
ExecStart=/k8s/etcd/bin/etcd \
--cert-file=/k8s/etcd/ssl/etcd.pem \
--key-file=/k8s/etcd/ssl/etcd-key.pem \
--peer-cert-file=/k8s/etcd/ssl/etcd.pem \
--peer-key-file=/k8s/etcd/ssl/etcd-key.pem \
--trusted-ca-file=/k8s/kubernetes/ssl/ca.pem \
--peer-trusted-ca-file=/k8s/kubernetes/ssl/ca.pem
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
EOF

分發文件至其他節點
cd /k8s/ 
scp -r etcd/ 192.168.0.2:/k8s/
scp -r etcd/ 192.168.0.3:/k8s/

scp /lib/systemd/system/etcd.service 192.168.0.2:/lib/systemd/system/etcd.service
scp /lib/systemd/system/etcd.service 192.168.0.3:/lib/systemd/system/etcd.service

啟動etcd 服務
systemctl daemon-reload
systemctl enable etcd
systemctl start etcd

# /k8s/etcd/bin/etcdctl --cacert=/k8s/kubernetes/ssl/ca.pem --cert=/k8s/etcd/ssl/etcd.pem --key=/k8s/etcd/ssl/etcd-key.pem --endpoints="https://192.168.0.3:2379,https://192.168.0.2:2379,https://192.168.0.1:2379" endpoint health
https://192.168.0.2:2379 is healthy: successfully committed proposal: took = 24.271259ms
https://192.168.0.3:2379 is healthy: successfully committed proposal: took = 31.633027ms
https://192.168.0.1:2379 is healthy: successfully committed proposal: took = 37.463262ms

etcd 3.4注意事項
ETCD3.4版本ETCDCTL_API=3 etcdctl 和 etcd --enable-v2=false 成為了默認配置，如要使用v2版本，執行etcdctl時候需要設置ETCDCTL_API環境變量，例如：ETCDCTL_API=2 etcdctl
ETCD3.4版本會自動讀取環境變量的參數，所以EnvironmentFile文件中有的參數，不需要再次在ExecStart啟動參數中添加，二選一，如同時配置，會觸發以下類似報錯“etcd: conflicting environment variable "ETCD_NAME" is shadowed by corresponding command-line flag (either unset environment variable or disable flag)”
flannel操作etcd使用的是v2的API，而kubernetes操作etcd使用的v3的API
————————————————
版權聲明：本文為CSDN博主「snipercai」的原創文章，遵循CC 4.0 BY-SA版權協議，轉載請附上原文出處鏈接及本聲明。
原文鏈接：https://blog.csdn.net/snipercai/article/details/101012124