多節點集群化部署

針對不同的情況，比如當節點數和各節點上的地址已知使用靜態配置，但是當各個節點的地址無法知曉的情況下，使用服務發現進行配置。

一、靜態配置

三台機器信息：

name	ip
etcd1	192.168.159.128
etcd2	192.168.159.129
etcd3	192.168.159.130

進行靜態配置需要滿足以下條件：

• 集群節點個數已知
• 集群各節點上地址已知

 etcd1配置文件conf.yml如下：

name: etcd1
data-dir: /project/etcd_cluster/etcd1/data
listen-client-urls: http://0.0.0.0:2379
advertise-client-urls: http://192.168.159.128:2379
listen-peer-urls: http://0.0.0.0:2380
initial-advertise-peer-urls: http://192.168.159.128:2380
initial-cluster: etcd1=http://192.168.159.128:2380,etcd2=http://192.168.159.129:2380,etcd3=http://192.168.159.130:2380
initial-cluster-token: etcd-cluster-1
initial-cluster-state: new

 etcd2配置文件conf.yml如下：

name: etcd2
data-dir: /project/etcd_cluster/etcd2/data
listen-client-urls: http://0.0.0.0:2379
advertise-client-urls: http://192.168.159.129:2379
listen-peer-urls: http://0.0.0.0:2380
initial-advertise-peer-urls: http://192.168.159.129:2380
initial-cluster: etcd1=http://192.168.159.128:2380,etcd2=http://192.168.159.129:2380,etcd3=http://192.168.159.130:2380
initial-cluster-token: etcd-cluster-1
initial-cluster-state: new

etcd3配置文件conf.yml如下：

name: etcd3
data-dir: /project/etcd_cluster/etcd3/data
listen-client-urls: http://0.0.0.0:2379
advertise-client-urls: http://192.168.159.130:2379
listen-peer-urls: http://0.0.0.0:2380
initial-advertise-peer-urls: http://192.168.159.130:2380
initial-cluster: etcd1=http://192.168.159.128:2380,etcd2=http://192.168.159.129:2380,etcd3=http://192.168.159.130:2380
initial-cluster-token: etcd-cluster-1
initial-cluster-state: new

注意，上面配置文件中listen-client-urls和listen-peer-urls參數監聽的的是所有的地址。

接下來通過etcd（此時etcd壓縮包已經拷貝至另外兩台機器）來啟動各個節點：

• 啟動etcd1

# 192.168.159.128
[root@localhost etcd-v3.3.10-linux-amd64]# ./etcd --config-file=/project/etcd_cluster/etcd1/conf.yml

• 啟動etcd2

# 192.168.159.129
[root@localhost etcd-v3.3.10-linux-amd64]# ./etcd --config-file=/project/etcd_cluster/etcd2/conf.yml

• 啟動etcd3

# 192.168.159.130
[root@localhost etcd-v3.3.10-linux-amd64]# ./etcd --config-file=/project/etcd_cluster/etcd3/conf.yml

啟動完畢后對集群的狀態進行查看：

• 查看成員

[root@localhost etcd-v3.3.10-linux-amd64]# ./etcdctl member list
4c14bc06668e9505: name=etcd3 peerURLs=http://192.168.159.130:2380 clientURLs=http://192.168.159.130:2379 isLeader=false
57bf4d2527966724: name=etcd2 peerURLs=http://192.168.159.129:2380 clientURLs=http://192.168.159.129:2379 isLeader=false
a11e107c0081dbf8: name=etcd1 peerURLs=http://192.168.159.128:2380 clientURLs=http://192.168.159.128:2379 isLeader=true

• 查看集群健康狀態

[root@localhost etcd-v3.3.10-linux-amd64]# ./etcdctl cluster-health
member 4c14bc06668e9505 is healthy: got healthy result from http://192.168.159.130:2379
member 57bf4d2527966724 is healthy: got healthy result from http://192.168.159.129:2379
member a11e107c0081dbf8 is healthy: got healthy result from http://192.168.159.128:2379
cluster is healthy

測試：

• 讀寫測試

[root@localhost etcd-v3.3.10-linux-amd64]# ETCDCTL_API=3 ./etcdctl  put foo bar
OK
[root@localhost etcd-v3.3.10-linux-amd64]# ETCDCTL_API=3 ./etcdctl  get foo
foo
bar

二、服務發現

　　實際應用中，可能無法知道各個節點ip地址的情況，所以就不能使用上述的靜態配置，而是需要使用服務發現，實質上就是利用現有的etcd集群來啟動一個新的etcd集群。我們可以使用之前搭建好的集群來搭建。

1、創建發現url（集群標識）

使用現有的集群創建發現url：

[root@localhost etcd-v3.3.10-linux-amd64]# curl -X PUT http://192.168.159.128:2379/v2/keys/discovery/6c007a14875d53d9bf0ef5a6fc0257c817f0fb83/_config/size -d value=3
{"action":"set","node":{"key":"/discovery/6c007a14875d53d9bf0ef5a6fc0257c817f0fb83/_config/size","value":"3","modifiedIndex":9,"createdIndex":9}}

當然，如果沒有自己的已有集群，也可以通過etcd提供的公共服務獲取發現url：

# size代表要創建的集群大小
[root@localhost etcd-v3.3.10-linux-amd64]# curl -w "\n" 'https://discovery.etcd.io/new?size=3'
https://discovery.etcd.io/8c66ef7d7bdccfa2e83f277b2407bd44

2、啟動新的集群

　　服務發現的url是http://192.168.159.128:2379/v2/keys/discovery/6c007a14875d53d9bf0ef5a6fc0257c817f0fb83，需要將其作為--discovery參數來啟動etcd，新的etcd實例會自動使用服務發現url的目錄來啟動注冊。三個節點的配置文件如下：

• infra0

name: infra0
listen-client-urls: http://0.0.0.0:12379
advertise-client-urls: http://192.168.159.128:12379
listen-peer-urls: http://0.0.0.0:12380
initial-advertise-peer-urls: http://192.168.159.128:12380
discovery: http://192.168.159.128:2379/v2/keys/discovery/6c007a14875d53d9bf0ef5a6fc0257c817f0fb83

• infra1

name: infra1
listen-client-urls: http://0.0.0.0:12379
advertise-client-urls: http://192.168.159.129:12379
listen-peer-urls: http://0.0.0.0:12380
initial-advertise-peer-urls: http://192.168.159.129:12380
discovery: http://192.168.159.128:2379/v2/keys/discovery/6c007a14875d53d9bf0ef5a6fc0257c817f0fb83

• infra2

name: infra2
listen-client-urls: http://0.0.0.0:12379
advertise-client-urls: http://192.168.159.130:12379
listen-peer-urls: http://0.0.0.0:12380
initial-advertise-peer-urls: http://192.168.159.130:12380
discovery: http://192.168.159.128:2379/v2/keys/discovery/6c007a14875d53d9bf0ef5a6fc0257c817f0fb83

　　可以看到這與之前靜態配置的區別是不需要initial-cluster、initial-cluster-token、initial-cluster-state三個參數，但是需要discovery參數用來獲取集群標識。上面的配置沒有加上data-dir，這會在當前目錄下創建一個數據目錄infra0.etcd（或者infra1.etcd、infra2.etcd）。

　　然后，可以進行啟動集群，與靜態配置方式啟動一樣，不過配置文件不同。

# infra0
[root@localhost etcd-v3.3.10-linux-amd64]# ./etcd --config-file=/project/etcd_cluster/infra0/conf.yml

# infra1
[root@localhost etcd-v3.3.10-linux-amd64]# ./etcd --config-file=/project/etcd_cluster/infra1/conf.yml


# infra2
[root@localhost etcd-v3.3.10-linux-amd64]# ./etcd --config-file=/project/etcd_cluster/infra2/conf.yml

獲取集群信息：

[root@localhost etcd-v3.3.10-linux-amd64]# ./etcdctl -endpoints=http://192.168.159.128:12379 member list
4b44e845d0f3585d: name=infra0 peerURLs=http://192.168.159.128:12380 clientURLs=http://192.168.159.128:12379 isLeader=true
6b9a9ac81c9c5d07: name=infra2 peerURLs=http://192.168.159.130:12380 clientURLs=http://192.168.159.130:12379 isLeader=false
957577be1117b1b1: name=infra1 peerURLs=http://192.168.159.129:12380 clientURLs=http://192.168.159.129:12379 isLeader=false