redis集群熱擴展（基於4.0.9）

1：環境說明，首先說一下要做的事情，我們要遷移redis集群槽位，現有redis集群環境如下

172,16.128.240:7003
172,16.128.240:7004
172,16.128.241:7003
172,16.128.241:7004
172,16.128.242:7003
172,16.128.242:7004

 

我們看一下集群的基本信息：

172,16.128.240:7003> cluster nodes
8ea64a0049e0b193296aeba868391e7b009ed562 172,16.128.241:7004@17004 slave 7a35e29ebffb7fd42a9533811b8fcf67955635f2 0 1528448456000 4 connected
1cbccbfa9a6ee62d2f744af14fddfe97cc491926 172,16.128.242:7003@17003 master - 0 1528448456095 2 connected 10923-16383
d4fac62706aa4def7611a495676ebc2151fc07ba 172,16.128.240:7003@17003 myself,master - 0 1528448456000 0 connected 0-5461
3801a25ac04c7e2c610c1df9588c107e0ed62fa1 172,16.128.242:7004@17004 slave 1cbccbfa9a6ee62d2f744af14fddfe97cc491926 0 1528448457096 5 connected
7a35e29ebffb7fd42a9533811b8fcf67955635f2 172,16.128.241:7003@17003 master - 0 1528448455000 1 connected 5462-10922
7f2dbd47e72c7b15b160e0f11d9f0ef86f225c27 172,16.128.240:7004@17004 slave d4fac62706aa4def7611a495676ebc2151fc07ba 0 1528448458098 3 connected

 

2：在172,16.128.242上創建7013和7014實例

[root@mxq-ss1 redis7014]# redis-server /home/redis/redis7013/redis7013.conf 
[root@mxq-ss1 redis7014]# redis-server /home/redis/redis7014/redis7014.conf 
[root@mxq-ss1 redis7004]# ps -ef|grep redis
root     41550     1  0 01:21 ?        00:00:09 redis-server 172,16.128.242:7003 [cluster]
root     41559     1  0 01:22 ?        00:00:09 redis-server 172,16.128.242:7004 [cluster]
root     45182     1  0 04:54 ?        00:00:00 redis-server 172,16.128.242:7013 [cluster]
root     45187     1  0 04:54 ?        00:00:00 redis-server 172,16.128.242:7014 [cluster]
root     45192 34163  0 04:54 pts/3    00:00:00 grep --color=auto redis

3：正式進行熱拓展測試（以下腳本將要應用到生產機房，請審視）

（1）server172,16.128.240

[root@mxq-ss3 data]# redis-cli -h 172,16.128.240 -p 7003 -a Rmbe7q3btATOE2Xyvmv
172,16.128.240:7003> cluster nodes
8ea64a0049e0b193296aeba868391e7b009ed562 172,16.128.241:7004@17004 slave 7a35e29ebffb7fd42a9533811b8fcf67955635f2 0 1528449386696 4 connected
1cbccbfa9a6ee62d2f744af14fddfe97cc491926 172,16.128.242:7003@17003 master - 0 1528449385694 2 connected 10923-16383
d4fac62706aa4def7611a495676ebc2151fc07ba 172,16.128.240:7003@17003 myself,master - 0 1528449384000 0 connected 0-5461
3801a25ac04c7e2c610c1df9588c107e0ed62fa1 172,16.128.242:7004@17004 slave 1cbccbfa9a6ee62d2f744af14fddfe97cc491926 0 1528449384692 5 connected
7a35e29ebffb7fd42a9533811b8fcf67955635f2 172,16.128.241:7003@17003 master - 0 1528449384000 1 connected 5462-10922
7f2dbd47e72c7b15b160e0f11d9f0ef86f225c27 172,16.128.240:7004@17004 slave d4fac62706aa4def7611a495676ebc2151fc07ba 0 1528449384000 3 connected
172,16.128.240:7003> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:5
cluster_my_epoch:0
cluster_stats_messages_ping_sent:4212
cluster_stats_messages_pong_sent:4031
cluster_stats_messages_meet_sent:5
cluster_stats_messages_sent:8248
cluster_stats_messages_ping_received:4031
cluster_stats_messages_pong_received:4217
cluster_stats_messages_received:8248
172,16.128.240:7003> cluster meet 172,16.128.242 7013
OK
172,16.128.240:7003> cluster meet 172,16.128.242 7014
OK
172,16.128.240:7003> cluster nodes
8ea64a0049e0b193296aeba868391e7b009ed562 172,16.128.241:7004@17004 slave 7a35e29ebffb7fd42a9533811b8fcf67955635f2 0 1528449505892 4 connected
1cbccbfa9a6ee62d2f744af14fddfe97cc491926 172,16.128.242:7003@17003 master - 0 1528449506000 2 connected 10923-16383
c73b4e70468d091e40123122ce140422f027d0d2 172,16.128.242:7013@17013 master - 0 1528449506593 6 connected
d4fac62706aa4def7611a495676ebc2151fc07ba 172,16.128.240:7003@17003 myself,master - 0 1528449503000 0 connected 0-5461
3801a25ac04c7e2c610c1df9588c107e0ed62fa1 172,16.128.242:7004@17004 slave 1cbccbfa9a6ee62d2f744af14fddfe97cc491926 0 1528449504000 5 connected
7a35e29ebffb7fd42a9533811b8fcf67955635f2 172,16.128.241:7003@17003 master - 0 1528449504891 1 connected 5462-10922
7f2dbd47e72c7b15b160e0f11d9f0ef86f225c27 172,16.128.240:7004@17004 slave d4fac62706aa4def7611a495676ebc2151fc07ba 0 1528449505000 3 connected
bd40901964330046ff5752fe62c7e11abba7ab06 172,16.128.242:7014@17014 master - 0 1528449506892 7 connected
172,16.128.240:7003> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:8
cluster_size:3
cluster_current_epoch:7
cluster_my_epoch:0
cluster_stats_messages_ping_sent:4341
cluster_stats_messages_pong_sent:4147
cluster_stats_messages_meet_sent:7
cluster_stats_messages_sent:8495
cluster_stats_messages_ping_received:4147
cluster_stats_messages_pong_received:4348
cluster_stats_messages_received:8495

 

（2）執行過腳本如下：

[root@mxq-ss3 data]# redis-cli -h 172,16.128.240 -p 7003 -a Rmbe7q3btATOE2Xyvmv
172,16.128.240:7003> cluster nodes
172,16.128.240:7003> cluster info
172,16.128.240:7003> cluster meet 172,16.128.242 7013
OK
172,16.128.240:7003> cluster meet 172,16.128.242 7014
OK
172,16.128.240:7003> cluster nodes
172,16.128.240:7003> cluster info

 

我們從上面信息已經看出已經將新建節點加入到了集群當中。下面開始將新加入的節點作為主從 server172,16.128.242

[root@mxq-ss1 redis7014]# redis-cli -h 172,16.128.242 -p 7014 -a Rmbe7q3btATOE2Xyvmv
172,16.128.242:7014> cluster replicate c73b4e70468d091e40123122ce140422f027d0d2
OK
172,16.128.242:7014> CLUSTER NODES
8ea64a0049e0b193296aeba868391e7b009ed562 172,16.128.241:7004@17004 slave 7a35e29ebffb7fd42a9533811b8fcf67955635f2 0 1528449970993 1 connected
d4fac62706aa4def7611a495676ebc2151fc07ba 172,16.128.240:7003@17003 master - 0 1528449971994 0 connected 0-5461
7f2dbd47e72c7b15b160e0f11d9f0ef86f225c27 172,16.128.240:7004@17004 slave d4fac62706aa4def7611a495676ebc2151fc07ba 0 1528449968000 0 connected
3801a25ac04c7e2c610c1df9588c107e0ed62fa1 172,16.128.242:7004@17004 slave 1cbccbfa9a6ee62d2f744af14fddfe97cc491926 0 1528449972000 2 connected
7a35e29ebffb7fd42a9533811b8fcf67955635f2 172,16.128.241:7003@17003 master - 0 1528449972996 1 connected 5462-10922
c73b4e70468d091e40123122ce140422f027d0d2 172,16.128.242:7013@17013 master - 0 1528449970000 6 connected
bd40901964330046ff5752fe62c7e11abba7ab06 172,16.128.242:7014@17014 myself,slave c73b4e70468d091e40123122ce140422f027d0d2 0 1528449971000 7 connected
1cbccbfa9a6ee62d2f744af14fddfe97cc491926 172,16.128.242:7003@17003 master - 0 1528449969000 2 connected 10923-16383

 

4:執行完以后進行熱拓展

先說明一下現在有三台主從，做熱拓展的話，每個槽位都要遷移出1365個槽位到172,16.128.242:7013的redis實例上。以下是遷移步驟 執行機器：172,16.128.240

執行以前，我們要確定以下配置文件
vi /usr/local/rvm/gems/ruby-2.4.1/gems/redis-4.0.1/lib/redis/client.rb
里面配置和redis的配置要一致，不然連不上redis報錯。這個要確認。下面執行遷移過程
[root@mxq-ss3 ~]# redis-trib.rb reshard 172,16.128.240:7003
How many slots do you want to move (from 1 to 16384)? 1364
What is the receiving node ID? 363ecec54c92c2548dcab016146bdb4c104e5e84
Please enter all the source node IDs.
  Type 'all' to use all the nodes as source nodes for the hash slots.
  Type 'done' once you entered all the source nodes IDs.
Source node #1:c73b4e70468d091e40123122ce140422f027d0d2
Source node #2:done

 

查看節點信息：

172,16.128.240:7003> cluster nodes
8ea64a0049e0b193296aeba868391e7b009ed562 172,16.128.241:7004@17004 slave 7a35e29ebffb7fd42a9533811b8fcf67955635f2 0 1528535455177 4 connected
1cbccbfa9a6ee62d2f744af14fddfe97cc491926 172,16.128.242:7003@17003 master - 0 1528535457181 2 connected 10923-16383
c73b4e70468d091e40123122ce140422f027d0d2 172,16.128.242:7013@17013 master - 0 1528535456180 6 connected
d4fac62706aa4def7611a495676ebc2151fc07ba 172,16.128.240:7003@17003 myself,master - 0 1528535455000 0 connected 0-5461
3801a25ac04c7e2c610c1df9588c107e0ed62fa1 172,16.128.242:7004@17004 slave 1cbccbfa9a6ee62d2f744af14fddfe97cc491926 0 1528535455000 5 connected
7a35e29ebffb7fd42a9533811b8fcf67955635f2 172,16.128.241:7003@17003 master - 0 1528535453172 1 connected 5462-10922
7f2dbd47e72c7b15b160e0f11d9f0ef86f225c27 172,16.128.240:7004@17004 slave d4fac62706aa4def7611a495676ebc2151fc07ba 0 1528535453000 3 connected
bd40901964330046ff5752fe62c7e11abba7ab06 172,16.128.242:7014@17014 slave c73b4e70468d091e40123122ce140422f027d0d2 0 1528535453000 7 connected
172,16.128.240:7003> cluster nodes
8ea64a0049e0b193296aeba868391e7b009ed562 172,16.128.241:7004@17004 slave 7a35e29ebffb7fd42a9533811b8fcf67955635f2 0 1528535856000 4 connected
1cbccbfa9a6ee62d2f744af14fddfe97cc491926 172,16.128.242:7003@17003 master - 0 1528535852000 2 connected 10923-16383
c73b4e70468d091e40123122ce140422f027d0d2 172,16.128.242:7013@17013 master - 0 1528535856884 8 connected 0-1364
d4fac62706aa4def7611a495676ebc2151fc07ba 172,16.128.240:7003@17003 myself,master - 0 1528535854000 0 connected 1365-5461
3801a25ac04c7e2c610c1df9588c107e0ed62fa1 172,16.128.242:7004@17004 slave 1cbccbfa9a6ee62d2f744af14fddfe97cc491926 0 1528535855883 5 connected
7a35e29ebffb7fd42a9533811b8fcf67955635f2 172,16.128.241:7003@17003 master - 0 1528535856000 1 connected 5462-10922
7f2dbd47e72c7b15b160e0f11d9f0ef86f225c27 172,16.128.240:7004@17004 slave d4fac62706aa4def7611a495676ebc2151fc07ba 0 1528535855000 3 connected
bd40901964330046ff5752fe62c7e11abba7ab06 172,16.128.242:7014@17014 slave c73b4e70468d091e40123122ce140422f027d0d2 0 1528535854000 8 connected

 

發現執行完畢，已經遷移了。接下來同樣方式遷移其他兩台master相同槽位到新節點上。 執行完查看節點信息，發現已經完全遷移上去了。

[root@mxq-ss3 ~]# redis-cli -h 172,16.128.240 -p 7003 -a ***********cluster nodes
8ea64a0049e0b193296aeba868391e7b009ed562 172,16.128.241:7004@17004 slave 7a35e29ebffb7fd42a9533811b8fcf67955635f2 0 1528536351000 4 connected
1cbccbfa9a6ee62d2f744af14fddfe97cc491926 172,16.128.242:7003@17003 master - 0 1528536352764 2 connected 12287-16383
c73b4e70468d091e40123122ce140422f027d0d2 172,16.128.242:7013@17013 master - 0 1528536349759 8 connected 0-1364 5462-6825 10923-12286
d4fac62706aa4def7611a495676ebc2151fc07ba 172,16.128.240:7003@17003 myself,master - 0 1528536351000 0 connected 1365-5461
3801a25ac04c7e2c610c1df9588c107e0ed62fa1 172,16.128.242:7004@17004 slave 1cbccbfa9a6ee62d2f744af14fddfe97cc491926 0 1528536349000 5 connected
7a35e29ebffb7fd42a9533811b8fcf67955635f2 172,16.128.241:7003@17003 master - 0 1528536350759 1 connected 6826-10922
7f2dbd47e72c7b15b160e0f11d9f0ef86f225c27 172,16.128.240:7004@17004 slave d4fac62706aa4def7611a495676ebc2151fc07ba 0 1528536351761 3 connected
bd40901964330046ff5752fe62c7e11abba7ab06 172,16.128.242:7014@17014 slave c73b4e70468d091e40123122ce140422f027d0d2 0 1528536350000 8 connected

 

整體遷移腳本如下：

腳本步驟：
腳本步驟： 11：server10：server172,16.128.240上執行
[root@mxq-ss3 data]# redis-cli -h 172,16.128.240 -p 7003 -a Rmbe7q3btATOE2Xyvmv
172,16.128.240:7003> cluster nodes
172,16.128.240:7003> cluster info
172,16.128.240:7003> cluster meet 172,16.128.242 7013
OK
172,16.128.240:7003> cluster meet 172,16.128.242 7014
OK
172,16.128.240:7003> cluster nodes
172,16.128.240:7003> cluster info
新加入節點做主從：
[root@mxq-ss1 redis7014]# redis-cli -h 172,16.128.242 -p 7014 -a Rmbe7q3btATOE2Xyvmv
172,16.128.242:7014> cluster replicate c73b4e70468d091e40123122ce140422f027d0d2
OK
172,16.128.242:7014> CLUSTER NODES
下面進行遷移（注意我們要執行三次，手動從240,241,242上7003實例遷移1364個slot到新實例上，並且每次遷移完成都要查看一下節點狀態）：
[root@mxq-ss3 ~]# redis-trib.rb reshard 172,16.128.240:7003
How many slots do you want to move (from 1 to 16384)? 1364
What is the receiving node ID? 363ecec54c92c2548dcab016146bdb4c104e5e84
Please enter all the source node IDs.
  Type 'all' to use all the nodes as source nodes for the hash slots.
  Type 'done' once you entered all the source nodes IDs.
Source node #1:c73b4e70468d091e40123122ce140422f027d0d2
Source node #2:done
[root@mxq-ss3 ~]# redis-cli -h 172,16.128.240 -p 7003 -a ***********cluster nodes
[root@mxq-ss3 ~]# redis-cli -h 172,16.128.240 -p 7003 -a ***********cluster info