redis集群節點重啟后恢復

服務器重啟后，集群報錯：

[root@SHH-HQ-NHS11S nhsuser]# redis-cli -c -h ip -p 7000
ip:7000> set cc dd
(error) CLUSTERDOWN Hash slot not served

檢查集群節點情況

> cluster nodes
c0767666bee76e5e0dc67f24031a3e1b574235cc :7000@17000 myself,master - 0 0 0 connected

發現集群的節點只剩一個，其余節點都不見了。

將集群節點加入：

# redis-cli --cluster add-node ip:7005 ip:7000
>>> Adding node ip:7005 to cluster ip:7000
>>> Performing Cluster Check (using node ip:7000)
M: c0767666bee76e5e0dc67f24031a3e1b574235cc ip:7000
   slots: (0 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[ERR] Not all 16384 slots are covered by nodes.

這個往往是由於主node移除了，但是並沒有移除node上面的slot，從而導致了slot總數沒有達到16384，其實也就是slots分布不正確。所以在刪除節點的時候一定要注意刪除的是否是Master主節點。現在這樣情況，我們先修復集群：

# redis-cli --cluster fix IP:7000

>>> Covering slot 9675 with ip:7000

有點久,稍等下

然后檢查修復情況:

# redis-cli --cluster check ip:7000
ip:7000 (c0767666...) -> 0 keys | 16384 slots | 0 slaves.
[OK] 0 keys in 1 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node ip:7000)
M: c0767666bee76e5e0dc67f24031a3e1b574235cc IP:7000
   slots:[0-16383] (16384 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

將節點加入集群:

[root@SHH-HQ-NHS11S nhsuser]# redis-cli --cluster add-node ip:7001 ip:7000
>>> Adding node ip:7001 to cluster ip:7000
>>> Performing Cluster Check (using node ip:7000)
M: c0767666bee76e5e0dc67f24031a3e1b574235cc ip:7000
   slots:[0-16383] (16384 slots) master
M: a98432e520af41117f5d8a81e2c0e2a430940c7e ip:7003
   slots: (0 slots) master
M: 2ff0c7ee051e26f484b6826d6aa8a5a3fbc93d17 ip:7002
   slots: (0 slots) master
M: 26bf8ce786d7c9d0bbb9d9d78978497f86fbbe0a ip:7005
   slots: (0 slots) master
M: 0a7718d2c53a16debdbfb391a7acd23a50a05faa ip:7004
   slots: (0 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node ip:7001 to make it join the cluster.
[OK] New node added correctly.

測試一把：

[root@SHH-HQ-NHS11S nhsuser]# redis-cli -c -h IP -p 7000
IP:7000> set aa dd
OK
IP:7000> get aa
"dd"
IP:7000>