遷移背景：
1、原來redis集群在A機房，需要把其遷移到新機房B上來.
2、保證現有環境穩定。
3、采用在線遷移方式，因為原有redis集群內有大量數據。
4、如果是一個全新的redis集群搭建會簡單很多。
5、10.128.51.14(ht4)是A機房，10.121.51.30(ht20)在B機房。
首先介紹下redis集群
Redis Cluster在多個節點之間進行數據共享，即使部分節點失效或無法進行通訊時，Cluster仍然可以繼續處理請求
Redis 3.0之后，官方版本支持了Cluster
如果每個主節點都有一個從節點支持，在主節點下線或無法與集群的大多數節點進行通訊的情況下，
從節點提升為主節點，並提供服務，保證Cluster正常運行
我這里redis采用單機，多實例方式部署
安裝之前，安裝集群管理工具redis-trib.rb，需要安裝ruby環境：
#首先安裝ruby相關依賴。 [root@ht20 redis]# cat /etc/redhat-release CentOS Linux release 7.9.2009 (Core) [root@ht20 redis]# uname -r 3.10.0-1160.45.1.el7.x86_64 [root@node1 ~]# yum install ruby rubygems -y [root@node1 ~]# cd /usr/local/src/ [root@ht20 src]# wget https://rubygems.org/downloads/redis-3.3.3.gem --2022-03-01 10:53:14-- https://rubygems.org/downloads/redis-3.3.3.gem Resolving rubygems.org (rubygems.org)... 151.101.65.227, 151.101.1.227, 151.101.129.227, ... Connecting to rubygems.org (rubygems.org)|151.101.65.227|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 92672 (90K) [application/octet-stream] Saving to: ‘redis-3.3.3.gem’ 100%[============================================================================================>]
92,672 428KB/s in 0.2s 2022-03-01 10:53:15 (428 KB/s) - ‘redis-3.3.3.gem’ saved [92672/92672] [root@ht20 src]# gem install redis-3.3.3.gem Successfully installed redis-3.3.3 Parsing documentation for redis-3.3.3 Installing ri documentation for redis-3.3.3 1 gem installed
一、首先建立目錄：
/data/redis_mai1/redis
/data/redis_mai2/redis
/data/redis_mai3/redis
/data/redis_mai4/redis
/data/redis_mai5/redis
/data/redis_mai6/redis
二、復制別的機器上的redis,內容就是下面的列表，這些內容是我從別的現成的redis集群直接scp 過來的，到6個目錄下
[root@ht20 redis]# ll total 77196 drwxr-xr-x 2 root root 59 Mar 1 09:51 data -rw-r--r-- 1 root root 158 Mar 1 09:29 keys.redis -rwxr-xr-x 1 root root 2451134 Mar 1 09:29 redis-benchmark -rwxr-xr-x 1 root root 5777399 Mar 1 09:29 redis-check-aof -rwxr-xr-x 1 root root 5777399 Mar 1 09:29 redis-check-rdb -rwxr-xr-x 1 root root 2617215 Mar 1 09:29 redis-cli -rw-r--r-- 1 root root 1558 Mar 1 09:43 redis.conf -rw-r--r-- 1 root root 50779877 Mar 1 09:51 redis.log -rwxr-xr-x 1 root root 5777399 Mar 1 09:29 redis-sentinel -rwxr-xr-x 1 root root 5777399 Mar 1 09:29 redis-server -rwxr-xr-x 1 root root 65991 Mar 1 09:29 redis-trib.rb
查看版本信息，不用啟動
[root@ht20 redis]# ./redis-cli -v
redis-cli 4.0.14
Redis 3.0之后，官方版本支持了Cluster.
三、對應修改配置文件(6個目錄下的redis.conf都需要進行修改)
bind 10.121.51.30 127.0.0.1 //bind用於指定本機網卡對應的IP地址 protected-mode yes port 7736 // 定義啟動端口 tcp-backlog 511 timeout 0 tcp-keepalive 300 daemonize yes //redis后台運行 supervised no pidfile /var/run/redis_7736.pid //socket pid loglevel notice logfile "/data/redis_mai6/redis/redis.log" //設定日志 databases 16 always-show-logo yes save 900 1 save 300 10 save 60 10000 stop-writes-on-bgsave-error yes rdbcompression yes rdbchecksum yes dbfilename dump.rdb dir /data/redis_mai6/redis/data/ slave-serve-stale-data yes slave-read-only yes repl-diskless-sync no repl-diskless-sync-delay 5 repl-disable-tcp-nodelay no slave-priority 100 slave-announce-ip 10.121.51.30 slave-announce-port 7736 lazyfree-lazy-expire no lazyfree-lazy-server-del no slave-lazy-flush no appendonly yes appendfilename "appendonly.aof" appendfsync everysec no-appendfsync-on-rewrite no auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 64mb aof-load-truncated yes aof-use-rdb-preamble no lua-time-limit 5000 //5秒 cluster-enabled yes //啟動集群模式 cluster-config-file nodes.conf //集群中的實例啟動時，自動生成 cluster-node-timeout 15000 //節點超時時間，單位毫秒，設置一個較小的超時時間（15秒） cluster-announce-ip 10.121.51.30 cluster-announce-port 7736 slowlog-log-slower-than 10000 slowlog-max-len 128 latency-monitor-threshold 0 notify-keyspace-events "" hash-max-ziplist-entries 512 hash-max-ziplist-value 64 list-max-ziplist-size -2 list-compress-depth 0 set-max-intset-entries 512 zset-max-ziplist-entries 128 zset-max-ziplist-value 64 hll-sparse-max-bytes 3000 activerehashing yes client-output-buffer-limit normal 0 0 0 client-output-buffer-limit slave 256mb 64mb 60 client-output-buffer-limit pubsub 32mb 8mb 60 hz 10 aof-rewrite-incremental-fsync yes
copy 過來的 nodes.conf的樣子，這個文件在重新安裝集群時，需要刪除掉，因為redis node節點啟動的時候會自動生成。
41767372c36cc268872e86d96660a32bf540624f 10.121.51.30:7732@17732 slave b9a55f739a65e6277e40cdbb806b5443c8aad66e 0 1636343114419 43 connected ff34a5fadf42078332445bcf41a2b59416fd92da 10.121.51.30:7734@17734 slave fb28e95eb37b8d827c0f48800c4d08e64c6fc335 0 1636343115000 44 connected 4bee54a5049524a6280fdb8a8de00a5bb94ccaa1 10.121.51.30:7733@17733 slave 07c99eae8176eb0e985c5a9914f42900d18fb39b 0 1636343116780 46 connected 07c99eae8176eb0e985c5a9914f42900d18fb39b 10.121.51.30:7736@17736 myself,master - 0 1636343113000 46 connected 10923-16383 b9a55f739a65e6277e40cdbb806b5443c8aad66e 10.121.51.30:7735@17735 master - 0 1636343115654 43 connected 5461-10922 fb28e95eb37b8d827c0f48800c4d08e64c6fc335 10.121.51.30:7731@17731 master - 0 1636343116656 44 connected 0-5460 vars currentEpoch 47 lastVoteEpoch 44
//這個里面標記了，節點名稱、ip、端口號、角色、連接狀態等。
由於整個redis是copy 過來的，需要進入到每個節點，刪除data下的幾個文件，保證生成的時候不受干擾，因為對應3個文件也會自動生成。
四、刪除data目錄下的3個文件。
[root@ht20 data]#cd /data/redis_mai1/redis/data/
[root@ht20 data]# ll
total 2296
-rw-r--r-- 1 root root 1442571 Mar 1 12:52 appendonly.aof
-rw-r--r-- 1 root root 897617 Mar 1 12:49 dump.rdb
//默認情況下 每隔一段時間redis服務器程序會自動對數據庫做一次遍歷,把內存快照寫在一個叫做“dump.rdb”的文件里,這個持久化機制叫做SNAPSHOT
-rw-r--r-- 1 root root 1432 Mar 1 12:49 nodes.conf
到6個實例的各自的data目錄下刪除掉對應的三個文件.
大家該說了，怎么這么麻煩你直接用一個新的redis包不就可以了，是的，我們可以用新的，是一樣的，只是我為了和現在的兼容。
創建集群，你會發現會有這種錯誤，是因為沒有刪掉data下的內容, 我這里只起了2個節點， 測試了一下。
(加入集群的節點必須是空節點，不包含槽/數據信息，否則不能加入集群)
[root@ht20 redis]# ./redis-trib.rb create --replicas 1 10.121.51.30:7731 10.129.51.30:7732 >>> Creating cluster [ERR] Node 10.121.51.30:7732 is not empty. Either the node already knows other nodes
(check with CLUSTER NODES) or contains some key in database 0.
//我現在只啟動了2個,11,12 ，會報至少需要6個實例的提示信息，或者叫6個節點（我們可以設置node，放置在不同服務器上）
[root@ht20 redis]# ./redis-trib.rb create --replicas 1 10.129.51.30:7731 10.129.51.30:7732
>>> Creating cluster
*** ERROR: Invalid configuration for cluster creation.
*** Redis Cluster requires at least 3 master nodes. //至少3個 master節點
*** This is not possible with 2 nodes and 1 replicas per node.
*** At least 6 nodes are required. //至少需要6個節點
好了，說下整個過程，現在正式說安裝啟動6個實例過程(一台機器上，啟動6個實例方式)
1、cp 過來整個redis ,6個實例目錄已經規划好了 /data/redis_mai* 方式
2、修改redis.conf(如端口，ip等)
3、刪除掉data下的3個文件 appendonly.aof dump.rdb nodes.conf
4、啟動各個實例。
[root@ht20 redis]#/data/redis_mai1/redis/redis-server /data/redis_mai1/redis/redis.conf [root@ht20 redis]#/data/redis_mai2/redis/redis-server /data/redis_mai2/redis/redis.conf [root@ht20 redis]#/data/redis_mai3/redis/redis-server /data/redis_mai3/redis/redis.conf [root@ht20 redis]#/data/redis_mai4/redis/redis-server /data/redis_mai4/redis/redis.conf [root@ht20 redis]#/data/redis_mai5/redis/redis-server /data/redis_mai5/redis/redis.conf [root@ht20 redis]#/data/redis_mai6/redis/redis-server /data/redis_mai6/redis/redis.conf
查看6個實例進程啟動情況，都正常啟動
[root@ht20 data]# ps -ef | grep redis
root 57063 1 0 12:39 ? 00:00:01 /data/redis_mai1/redis/redis-server 10.121.51.30:7731 [cluster]
root 58394 1 0 12:45 ? 00:00:00 /data/redis_mai2/redis/redis-server 10.121.51.30:7732 [cluster]
root 58881 1 0 12:47 ? 00:00:00 /data/redis_mai3/redis/redis-server 10.121.51.30:7733 [cluster]
root 58893 1 0 12:47 ? 00:00:00 /data/redis_mai4/redis/redis-server 10.121.51.30:7734 [cluster]
root 58905 1 0 12:47 ? 00:00:00 /data/redis_mai5/redis/redis-server 10.121.51.30:7735 [cluster]
root 58939 1 0 12:47 ? 00:00:00 /data/redis_mai6/redis/redis-server 10.121.51.30:7736 [cluster]
//這個時候他們雖然各自啟動了，但是還相互發現不了對方。需要進行集群的配置
5、集群配置(這里注意下，由於14和30是部署在兩個機房的，但是網絡是相通的，結果執行創建后，這些都和原來的redis集群混到了一塊)
說明： "--replicas 1"參數，指定每個主節點配置的從節點數量，如果這里設置為1（1:1比例，一個主，一個從），
最少3個主節點的情況下，總節點數不能低於6，否則集群不能成功創建
因為在 Redis 集群中有且僅有 16383 個 solt ，默認情況會給我們平均分配，
當然你可以指定，后續的增減節點也可以重新分配
[root@ht20 redis]# ./redis-trib.rb create --replicas 1 10.121.51.30:7731
10.121.51.30:7732 10.121.51.30:7733 10.121.51.30:7734
10.121.51.30:7735 10.121.51.30:7736
>>> Creating cluster >>> Performing hash slots allocation on 6 nodes... Using 3 masters: 10.121.51.30:7731 10.121.51.30:7732 10.121.51.30:7733 Adding replica 10.121.51.30:7735 to 10.121.51.30:7731 Adding replica 10.121.51.30:7736 to 10.121.51.30:7732 Adding replica 10.121.51.30:7734 to 10.121.51.30:7733 >>> Trying to optimize slaves allocation for anti-affinity [WARNING] Some slaves are in the same host as their master M: d46f032ea50763de8353fd530535412df6ffdc00 10.121.51.30:7731 slots:0-5460 (5461 slots) master M: 91b7f8c79c91c7edd77458c332f0b9299bdb94d4 10.121.51.30:7732 slots:5461-10922 (5462 slots) master M: d1c7d99e13a2d7317baf883dffa906470a606641 10.121.51.30:7733 slots:10923-16383 (5461 slots) master S: 27f7e8dede5b68581486ce8cefdd656032baed70 10.121.51.30:7734 replicates d46f032ea50763de8353fd530535412df6ffdc00 S: 4d9500ff2ee1432094178dad628007858e876e99 10.121.51.30:7735 replicates 91b7f8c79c91c7edd77458c332f0b9299bdb94d4 S: bc5051aef1a756c274907d23198fbb932c384151 10.121.51.30:7736 replicates d1c7d99e13a2d7317baf883dffa906470a606641 Can I set the above configuration? (type 'yes' to accept): yes 同意分配計划，輸入"yes"后，各節點開始通訊，並協商哈希槽的分配， 最后輸出報告指出各主節點分配的哈希槽，16384個哈希槽全部分配完畢，集群創建成功。 一旦回車之后 >>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join................ >>> Performing Cluster Check (using node 10.129.51.30:7731) S: d46f032ea50763de8353fd530535412df6ffdc00 10.129.51.30:7731 slots: (0 slots) slave replicates fb28e95eb37b8d827c0f48800c4d08e64c6fc335 S: d1c7d99e13a2d7317baf883dffa906470a606641 10.129.51.30:7733 slots: (0 slots) slave replicates 07c99eae8176eb0e985c5a9914f42900d18fb39b M: 07c99eae8176eb0e985c5a9914f42900d18fb39b 10.128.51.14:7736 slots:10923-16383 (5461 slots) master 2 additional replica(s) M: 27f7e8dede5b68581486ce8cefdd656032baed70 10.129.51.30:7734 slots: (0 slots) master 0 additional replica(s) S: ff34a5fadf42078332445bcf41a2b59416fd92da 10.128.51.14:7734 slots: (0 slots) slave replicates fb28e95eb37b8d827c0f48800c4d08e64c6fc335 S: 41767372c36cc268872e86d96660a32bf540624f 10.128.51.14:7732 slots: (0 slots) slave replicates b9a55f739a65e6277e40cdbb806b5443c8aad66e S: 91b7f8c79c91c7edd77458c332f0b9299bdb94d4 10.129.51.30:7732 slots: (0 slots) slave replicates b9a55f739a65e6277e40cdbb806b5443c8aad66e M: fb28e95eb37b8d827c0f48800c4d08e64c6fc335 10.128.51.14:7731 slots:0-5460 (5461 slots) master 2 additional replica(s) M: bc5051aef1a756c274907d23198fbb932c384151 10.129.51.30:7736 slots: (0 slots) master 0 additional replica(s) S: 4bee54a5049524a6280fdb8a8de00a5bb94ccaa1 10.128.51.14:7733 slots: (0 slots) slave replicates 07c99eae8176eb0e985c5a9914f42900d18fb39b M: b9a55f739a65e6277e40cdbb806b5443c8aad66e 10.128.51.14:7735 slots:5461-10922 (5462 slots) master 2 additional replica(s) [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
//注意由於其他環境下也有一組集群，他們自動都進行相互關聯了。
6、登錄控制台查看
//登錄集群節點（隨便哪個節點都可以，查看和測試集群內的數據）
[root@ht20 redis]# ./redis-cli -c -h 10.129.51.30 -p 7731
10.121.51.30:7731> get kingking
-> Redirected to slot [5153] located at 10.128.51.14:7731
(nil)
10.128.51.14:7731> set a 1111
-> Redirected to slot [15495] located at 10.128.51.14:7736
OK
10.128.51.14:7736> get a
"1111"
10.128.51.14:7736> exit
//登錄控制台，必須是-c 集群命令你可以執行，但是例如：set key value 等操作就會出現錯誤。
[root@ht20 redis]# ./redis-cli -c -h 10.129.51.30 -p 7732
10.129.51.30:7732> set bb aaaaaa
-> Redirected to slot [8620] located at 10.128.51.14:7735
OK
10.128.51.14:7735> get bb
"aaaaaa"
//我們看到有些數據設置到了,另一組redis集群里面了
//登錄單節點，不加-c , 這種方式由於不加-c 就啟動不了集群模式，是沒有意義的，會報錯。
[root@ht20 redis]# ./redis-cli -h 10.121.51.30 -p 7732
10.121.51.30:7732> get bb
(error) MOVED 8620 10.128.51.14:7735
其他cluster命令,看看有哪些可以查看的
CLUSTER info：打印集群的信息。 CLUSTER nodes：列出集群當前已知的所有節點（node）的相關信息。 CLUSTER meet <ip> <port>：將ip和port所指定的節點添加到集群當中。 CLUSTER addslots <slot> [slot ...]：將一個或多個槽（slot）指派（assign）給當前節點。 CLUSTER delslots <slot> [slot ...]：移除一個或多個槽對當前節點的指派。 CLUSTER slots：列出槽位、節點信息。 CLUSTER slaves <node_id>：列出指定節點下面的從節點信息。 比如： cluster slaves 11f9169577352c33d85ad0d1ca5f5bf0deba3209
這個實際查的是nodes.conf
CLUSTER replicate <node_id>：將當前節點設置為指定節點的從節點。 CLUSTER saveconfig： 手動執行命令保存保存集群的配置文件，集群默認在配置修改的時候會自動保存配置文件。 CLUSTER keyslot <key>： 列出key被放置在哪個槽上。 例如：cluster keyslot 9223372036854742675 CLUSTER flushslots： 移除指派給當前節點的所有槽，讓當前節點變成一個沒有指派任何槽的節點。 CLUSTER countkeysinslot <slot>： 返回槽目前包含的鍵值對數量。 CLUSTER getkeysinslot <slot槽號> <key的數量>：返回count個槽中的鍵。 比如：cluster getkeysinslot 202 3 CLUSTER setslot <slot> node <node_id> 將槽指派給指定的節點，如果槽已經指派給另一個節點，那么先讓另一個節點刪除該槽，然后再進行指派。 CLUSTER setslot <slot> migrating <node_id> 將本節點的槽遷移到指定的節點中。 CLUSTER setslot <slot> importing <node_id> 從 node_id 指定的節點中導入槽 slot 到本節點。 CLUSTER setslot <slot> stable 取消對槽 slot 的導入（import）或者遷移（migrate）。 CLUSTER failover： 手動進行故障轉移。 CLUSTER forget <node_id>： 從集群中移除指定的節點，這樣就無法完成握手，過期時為60s，60s后兩節點又會繼續完成握手。 CLUSTER reset [HARD|SOFT]：重置集群信息，soft是清空其他節點的信息，但不修改自己的id，hard還會修改自己的id，不傳該參數則使用soft方式。 CLUSTER count-failure-reports <node_id>：列出某個節點的故障報告的長度。 CLUSTER SET-CONFIG-EPOCH： 設置節點epoch，只有在節點加入集群前才能設置。
//使用集群工具來查看集群是否正常
[root@ht20 redis]# ./redis-trib.rb check 10.121.51.30:7731
[root@ht20 redis]# ./redis-trib.rb check 10.121.51.30:7731 >>> Performing Cluster Check (using node 10.121.51.30:7731) S: d46f032ea50763de8353fd530535412df6ffdc00 10.121.51.30:7731 //從1 slots: (0 slots) slave replicates fb28e95eb37b8d827c0f48800c4d08e64c6fc335 S: d1c7d99e13a2d7317baf883dffa906470a606641 10.121.51.30:7733 //從2 slots: (0 slots) slave replicates 07c99eae8176eb0e985c5a9914f42900d18fb39b M: 07c99eae8176eb0e985c5a9914f42900d18fb39b 10.128.51.14:7736 //主5 slots:10923-16383 (5461 slots) master 2 additional replica(s) M: 27f7e8dede5b68581486ce8cefdd656032baed70 10.121.51.30:7734 //主4 slots: (0 slots) master 0 additional replica(s) S: ff34a5fadf42078332445bcf41a2b59416fd92da 10.128.51.14:7734 //從3 slots: (0 slots) slave replicates fb28e95eb37b8d827c0f48800c4d08e64c6fc335 S: 41767372c36cc268872e86d96660a32bf540624f 10.128.51.14:7732 //從4 slots: (0 slots) slave replicates b9a55f739a65e6277e40cdbb806b5443c8aad66e S: 91b7f8c79c91c7edd77458c332f0b9299bdb94d4 10.121.51.30:7732 //從5 slots: (0 slots) slave replicates b9a55f739a65e6277e40cdbb806b5443c8aad66e M: fb28e95eb37b8d827c0f48800c4d08e64c6fc335 10.128.51.14:7731 //主3 slots:0-5460 (5461 slots) master 2 additional replica(s) M: bc5051aef1a756c274907d23198fbb932c384151 10.121.51.30:7736 //主2 slots: (0 slots) master 0 additional replica(s) S: 4bee54a5049524a6280fdb8a8de00a5bb94ccaa1 10.128.51.14:7733 //從6 slots: (0 slots) slave replicates 07c99eae8176eb0e985c5a9914f42900d18fb39b M: b9a55f739a65e6277e40cdbb806b5443c8aad66e 10.128.51.14:7735 //主1 slots:5461-10922 (5462 slots) master 2 additional replica(s) [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
//數了一下發現少了一個實例5，后來查看之后發現是第5個實例下的redis.conf配置ip錯誤，這里就不改了。
集群相關命令
查看集群，非常有用的cluster info
[root@ht20 redis]# ./redis-cli -c -p 7731 cluster info cluster_state:ok cluster_slots_assigned:16384 //#被分配的槽位數 cluster_slots_ok:16384 //#正確分配的槽位 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 //當前是6個節點 cluster_size:3 //當前集群下的有槽位分配的節點，即主節點 cluster_current_epoch:48 cluster_my_epoch:44 cluster_stats_messages_ping_sent:295 cluster_stats_messages_pong_sent:336 cluster_stats_messages_sent:631 cluster_stats_messages_ping_received:336 cluster_stats_messages_pong_received:295 cluster_stats_messages_received:631
如果集群失敗，則信息如下
[root@ht20 redis]# ./redis-cli -c -p 7731 cluster info
cluster_state:fail
cluster_slots_assigned:2380
cluster_slots_ok:2380
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:1
cluster_size:1
cluster_current_epoch:0
cluster_my_epoch:0
cluster_stats_messages_sent:0
cluster_stats_messages_received:0
//登錄單節點，沒有-c參數，集群單節點顯示失敗的相關信息
[root@ht20 redis]# ./redis-cli -h 10.121.51.30 -p 7731
10.121.51.30:7731> get king
(error) CLUSTERDOWN Hash slot not served
//我們可以查看所有的節點信息
[root@ht20 redis]# ./redis-cli -c -h 10.129.51.30 -p 7731 10.129.51.30:7731> cluster nodes d1c7d99e13a2d7317baf883dffa906470a606641 10.129.51.30:7733@17733 slave 07c99eae8176eb0e985c5a9914f42900d18fb39b 0 1646112764069 46 connected d46f032ea50763de8353fd530535412df6ffdc00 10.129.51.30:7731@17731 myself,slave fb28e95eb37b8d827c0f48800c4d08e64c6fc335 0
1646112761000 1 connected 07c99eae8176eb0e985c5a9914f42900d18fb39b 10.128.51.14:7736@17736 master - 0 1646112766077 46 connected 10923-16383 //最后的 10923-16383 就是槽位 27f7e8dede5b68581486ce8cefdd656032baed70 10.129.51.30:7734@17734 master - 0 1646112764000 4 connected ff34a5fadf42078332445bcf41a2b59416fd92da 10.128.51.14:7734@17734 slave fb28e95eb37b8d827c0f48800c4d08e64c6fc335 0 1646112766000 44 connected 41767372c36cc268872e86d96660a32bf540624f 10.128.51.14:7732@17732 slave b9a55f739a65e6277e40cdbb806b5443c8aad66e 0 1646112762071 43 connected 91b7f8c79c91c7edd77458c332f0b9299bdb94d4 10.129.51.30:7732@17732 slave b9a55f739a65e6277e40cdbb806b5443c8aad66e 0 1646112766000 43 connected fb28e95eb37b8d827c0f48800c4d08e64c6fc335 10.128.51.14:7731@17731 master - 0 1646112763073 44 connected 0-5460 //槽位 bc5051aef1a756c274907d23198fbb932c384151 10.129.51.30:7736@17736 master - 0 1646112767573 6 connected 4bee54a5049524a6280fdb8a8de00a5bb94ccaa1 10.128.51.14:7733@17733 slave 07c99eae8176eb0e985c5a9914f42900d18fb39b 0 1646112767078 46 connected b9a55f739a65e6277e40cdbb806b5443c8aad66e 10.128.51.14:7735@17735 master - 0 1646112767578 43 connected 5461-10922 //槽位
//實際讀取的是 data下的nodes.conf文件
//可以查看槽位與對應節點的信息
[root@ht20 redis]# ./redis-cli -c -h 10.121.51.30 -p 7731
10.121.51.30:7731> cluster slots
1) 1) (integer) 10923
2) (integer) 16383
3) 1) "10.128.51.14"
2) (integer) 7736
3) "07c99eae8176eb0e985c5a9914f42900d18fb39b"
4) 1) "10.128.51.14"
2) (integer) 7733
3) "4bee54a5049524a6280fdb8a8de00a5bb94ccaa1"
5) 1) "10.121.51.30"
2) (integer) 7733
3) "d1c7d99e13a2d7317baf883dffa906470a606641"
2) 1) (integer) 0 //槽位范圍 （0-5460）
2) (integer) 5460
3) 1) "10.128.51.14"
2) (integer) 7731
3) "fb28e95eb37b8d827c0f48800c4d08e64c6fc335"
4) 1) "10.121.51.30"
2) (integer) 7731
3) "d46f032ea50763de8353fd530535412df6ffdc00"
5) 1) "10.128.51.14"
2) (integer) 7734
3) "ff34a5fadf42078332445bcf41a2b59416fd92da"
3) 1) (integer) 5461 //解釋槽位（5461-10923）
2) (integer) 10922
3) 1) "10.128.51.14"
2) (integer) 7735
3) "b9a55f739a65e6277e40cdbb806b5443c8aad66e"
4) 1) "10.128.51.14"
2) (integer) 7732
3) "41767372c36cc268872e86d96660a32bf540624f"
5) 1) "10.121.51.30"
2) (integer) 7732
3) "91b7f8c79c91c7edd77458c332f0b9299bdb94d4"
10.121.51.30:7731>
10.129.51.30:7731> get bb -> Redirected to slot [8620] located at 10.128.51.14:7735 "aaaaaa"
//cluster countkeysinslot 數字 可以獲取指定哈希槽的key的數量
10.128.51.14:7735> cluster countkeysinslot 7735 (integer) 0
//info信息比較豐富,查看主從關系.
[root@ht20 redis]# ./redis-cli -c -h 10.121.51.30 -p 7731 10.121.51.30:7731> info # Server redis_version:4.0.14 redis_git_sha1:00000000 redis_git_dirty:0 redis_build_id:a754ba616e326a74 redis_mode:cluster os:Linux 3.10.0-1160.45.1.el7.x86_64 x86_64 arch_bits:64 multiplexing_api:epoll atomicvar_api:atomic-builtin gcc_version:4.8.5 process_id:57063 run_id:409144476e070fcf4813ccacef38effdc9952534 tcp_port:7731 uptime_in_seconds:15517 uptime_in_days:0 hz:10 lru_clock:1957924 executable:/data/redis_fpmai1/redis/redis-server config_file:/data/redis_fpmai1/redis/redis.conf # Clients connected_clients:129 client_longest_output_list:0 client_biggest_input_buf:0 blocked_clients:0 # Memory used_memory:7421344 used_memory_human:7.08M used_memory_rss:15884288 used_memory_rss_human:15.15M used_memory_peak:9244376 used_memory_peak_human:8.82M used_memory_peak_perc:80.28% used_memory_overhead:4937420 used_memory_startup:1445024 used_memory_dataset:2483924 used_memory_dataset_perc:41.56% total_system_memory:267306831872 total_system_memory_human:248.95G used_memory_lua:39936 used_memory_lua_human:39.00K maxmemory:0 maxmemory_human:0B maxmemory_policy:noeviction mem_fragmentation_ratio:2.14 mem_allocator:jemalloc-4.0.3 active_defrag_running:0 lazyfree_pending_objects:0 # Persistence loading:0 rdb_changes_since_last_save:61 rdb_bgsave_in_progress:0 rdb_last_save_time:1646124910 rdb_last_bgsave_status:ok rdb_last_bgsave_time_sec:0 rdb_current_bgsave_time_sec:-1 rdb_last_cow_size:4972544 aof_enabled:1 aof_rewrite_in_progress:0 aof_rewrite_scheduled:0 aof_last_rewrite_time_sec:0 aof_current_rewrite_time_sec:-1 aof_last_bgrewrite_status:ok aof_last_write_status:ok aof_last_cow_size:12689408 aof_current_size:4539890 aof_base_size:1509571 aof_pending_rewrite:0 aof_buffer_length:0 aof_rewrite_buffer_length:0 aof_pending_bio_fsync:0 aof_delayed_fsync:0 # Stats total_connections_received:1705 total_commands_processed:25957 instantaneous_ops_per_sec:1 total_net_input_bytes:4558664 total_net_output_bytes:4948669 instantaneous_input_kbps:0.04 instantaneous_output_kbps:6.26 rejected_connections:0 sync_full:2 sync_partial_ok:0 sync_partial_err:2 expired_keys:0 expired_stale_perc:0.00 expired_time_cap_reached_count:0 evicted_keys:0 keyspace_hits:51 keyspace_misses:26 pubsub_channels:0 pubsub_patterns:0 latest_fork_usec:1254 migrate_cached_sockets:0 slave_expires_tracked_keys:0 active_defrag_hits:0 active_defrag_misses:0 active_defrag_key_hits:0 active_defrag_key_misses:0 # Replication role:slave master_host:10.128.51.14 master_port:7731 master_link_status:up master_last_io_seconds_ago:2 master_sync_in_progress:0 slave_repl_offset:1542999173 slave_priority:100 slave_read_only:1 connected_slaves:1 slave0:ip=10.128.51.14,port=7734,state=online,offset=1542999173,lag=0 master_replid:2dd82c47c221ccb5b19871366f533f5856413f74 master_replid2:0000000000000000000000000000000000000000 master_repl_offset:1542999173 second_repl_offset:-1 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:1541950598 repl_backlog_histlen:1048576 # CPU used_cpu_sys:9.87 used_cpu_user:5.31 used_cpu_sys_children:0.25 used_cpu_user_children:0.60 # Cluster cluster_enabled:1 # Keyspace db0:keys=3189,expires=708,avg_ttl=0 10.121.51.30:7731>
//這里補充下，由於第5個實例，添加失敗，是由於redis.conf配置ip出錯，查看/data下的nodes.conf出現問題。
[root@ht20 data]# pwd
/data/redis_fpmai5/redis/data
[root@ht20 data]# cat nodes.conf
32ee19af1f6a534c4014b9c41d387666f049354b :0@0 myself,master - 0 0 0 connected
vars currentEpoch 0 lastVoteEpoch 0
//下面把實例5節點加入到集群
//增加一個節點到集群 [root@ht20 redis]# ./redis-trib.rb add-node 10.129.51.30:7735 10.129.51.30:7731 >>> Adding node 10.129.51.30:7735 to cluster 10.129.51.30:7731 >>> Performing Cluster Check (using node 10.129.51.30:7731) S: d46f032ea50763de8353fd530535412df6ffdc00 10.129.51.30:7731 slots: (0 slots) slave replicates fb28e95eb37b8d827c0f48800c4d08e64c6fc335 S: d1c7d99e13a2d7317baf883dffa906470a606641 10.129.51.30:7733 slots: (0 slots) slave replicates 07c99eae8176eb0e985c5a9914f42900d18fb39b M: 07c99eae8176eb0e985c5a9914f42900d18fb39b 10.128.51.14:7736 slots:10923-16383 (5461 slots) master 2 additional replica(s) M: 27f7e8dede5b68581486ce8cefdd656032baed70 10.129.51.30:7734 slots: (0 slots) master 0 additional replica(s) S: ff34a5fadf42078332445bcf41a2b59416fd92da 10.128.51.14:7734 slots: (0 slots) slave replicates fb28e95eb37b8d827c0f48800c4d08e64c6fc335 S: 41767372c36cc268872e86d96660a32bf540624f 10.128.51.14:7732 slots: (0 slots) slave replicates b9a55f739a65e6277e40cdbb806b5443c8aad66e S: 91b7f8c79c91c7edd77458c332f0b9299bdb94d4 10.129.51.30:7732 slots: (0 slots) slave replicates b9a55f739a65e6277e40cdbb806b5443c8aad66e M: fb28e95eb37b8d827c0f48800c4d08e64c6fc335 10.128.51.14:7731 slots:0-5460 (5461 slots) master 2 additional replica(s) M: bc5051aef1a756c274907d23198fbb932c384151 10.129.51.30:7736 slots: (0 slots) master 0 additional replica(s) S: 4bee54a5049524a6280fdb8a8de00a5bb94ccaa1 10.128.51.14:7733 slots: (0 slots) slave replicates 07c99eae8176eb0e985c5a9914f42900d18fb39b M: b9a55f739a65e6277e40cdbb806b5443c8aad66e 10.128.51.14:7735 slots:5461-10922 (5462 slots) master 2 additional replica(s) [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. >>> Send CLUSTER MEET to node 10.129.51.30:7735 to make it join the cluster. [OK] New node added correctly.
槽位補充知識(由於要進行槽位遷移，所以這些還是要了解,槽位可以理解為數據占位，
集群就這么多個位置，每個位置在集群中唯一的，這里是三個master,給了3個槽段)
這個也大大方便我們進行相互遷移。
槽位機制(slot)
分配槽位：cluster addslots 槽位，一個槽位只能分配一個節點，16384個槽位必須分配完，不同節點不能沖突,例如
10.121.51.30:7730> cluster addslots 0
OK 10.121.51.30:7730> cluster addslots 0 #沖突 (error) ERR Slot 0 is already busy
批量添加槽位的腳本
node1： #!/bin/bash n=0 for ((i=n;i<=5461;i++)) do /usr/local/bin/redis-cli -h 192.168.100.134 -p 17021 -a dxy CLUSTER ADDSLOTS $i done node2： #!/bin/bash n=5462 for ((i=n;i<=10922;i++)) do /usr/local/bin/redis-cli -h 192.168.100.135 -p 17021 -a dxy CLUSTER ADDSLOTS $i done node3： #!/bin/bash n=10923 for ((i=n;i<=16383;i++)) do /usr/local/bin/redis-cli -h 192.168.100.136 -p 17021 -a dxy CLUSTER ADDSLOTS $i done
移除一個槽位看看：cluster delslots 槽位
10.121.51.31:7731> cluster delslots 0 OK 10.121.51.31:7731> cluster info cluster_state:fail cluster_slots_assigned:16383 //原來是16384 cluster_slots_ok:16383 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:3 cluster_size:3 cluster_current_epoch:2 cluster_my_epoch:1 cluster_stats_messages_sent:4482 cluster_stats_messages_received:4482
如果16384 個槽位如果沒有分配完全，集群是不成功的。
如何進行遷移槽位，目前是槽位都在10.128.51.14上
[root@ht20 redis]# ./redis-cli -c -h 10.129.51.30 -p 7731 10.129.51.30:7731> cluster nodes d1c7d99e13a2d7317baf883dffa906470a606641 10.129.51.30:7733@17733 slave 07c99eae8176eb0e985c5a9914f42900d18fb39b 0 1646112764069 46 connected d46f032ea50763de8353fd530535412df6ffdc00 10.129.51.30:7731@17731 myself,slave fb28e95eb37b8d827c0f48800c4d08e64c6fc335 0 1646112761000 1 connected 07c99eae8176eb0e985c5a9914f42900d18fb39b 10.128.51.14:7736@17736 master - 0 1646112766077 46 connected 10923-16383
//最后的 10923-16383 就是槽位
27f7e8dede5b68581486ce8cefdd656032baed70 10.129.51.30:7734@17734 master - 0 1646112764000 4 connected ff34a5fadf42078332445bcf41a2b59416fd92da 10.128.51.14:7734@17734 slave fb28e95eb37b8d827c0f48800c4d08e64c6fc335 0 1646112766000 44 connected 41767372c36cc268872e86d96660a32bf540624f 10.128.51.14:7732@17732 slave b9a55f739a65e6277e40cdbb806b5443c8aad66e 0 1646112762071 43 connected 91b7f8c79c91c7edd77458c332f0b9299bdb94d4 10.129.51.30:7732@17732 slave b9a55f739a65e6277e40cdbb806b5443c8aad66e 0 1646112766000 43 connected fb28e95eb37b8d827c0f48800c4d08e64c6fc335 10.128.51.14:7731@17731 master - 0 1646112763073 44 connected 0-5460
//槽位,這個要先記住當前是master，后面很多操作之后，變成slave了。
bc5051aef1a756c274907d23198fbb932c384151 10.129.51.30:7736@17736 master - 0 1646112767573 6 connected 4bee54a5049524a6280fdb8a8de00a5bb94ccaa1 10.128.51.14:7733@17733 slave 07c99eae8176eb0e985c5a9914f42900d18fb39b 0 1646112767078 46 connected b9a55f739a65e6277e40cdbb806b5443c8aad66e 10.128.51.14:7735@17735 master - 0 1646112767578 43 connected 5461-10922 //槽位 //實際讀取的是 data下的nodes.conf文件
//14機器本身在另外一個機房，而30在一個機房。
計划遷移槽位有兩個目的：
1）、由於我們要遷移機房，所以14這台機器要釋放掉，所以集群要遷移到30上去
2）、操作當中由於一次失誤，cluster nodes命令不識別 第5台實例需要修復
//把實例5節點補充增加至集群里面，由於之前集群本身是11個，5主6從，實例5加入后自動變成主，從而變成了6主，6從 [root@ht20 redis]# ./redis-trib.rb add-node 10.129.51.30:7735 10.129.51.30:7731 //這里必須是兩台機器. >>> Adding node 10.129.51.30:7735 to cluster 10.129.51.30:7731 >>> Performing Cluster Check (using node 10.129.51.30:7731) S: d46f032ea50763de8353fd530535412df6ffdc00 10.129.51.30:7731 slots: (0 slots) slave replicates fb28e95eb37b8d827c0f48800c4d08e64c6fc335 S: d1c7d99e13a2d7317baf883dffa906470a606641 10.129.51.30:7733 slots: (0 slots) slave replicates 07c99eae8176eb0e985c5a9914f42900d18fb39b M: 07c99eae8176eb0e985c5a9914f42900d18fb39b 10.128.51.14:7736 slots:10923-16383 (5461 slots) master 2 additional replica(s) M: 27f7e8dede5b68581486ce8cefdd656032baed70 10.129.51.30:7734 slots: (0 slots) master 0 additional replica(s) S: ff34a5fadf42078332445bcf41a2b59416fd92da 10.128.51.14:7734 slots: (0 slots) slave replicates fb28e95eb37b8d827c0f48800c4d08e64c6fc335 S: 41767372c36cc268872e86d96660a32bf540624f 10.128.51.14:7732 slots: (0 slots) slave replicates b9a55f739a65e6277e40cdbb806b5443c8aad66e S: 91b7f8c79c91c7edd77458c332f0b9299bdb94d4 10.129.51.30:7732 slots: (0 slots) slave replicates b9a55f739a65e6277e40cdbb806b5443c8aad66e M: fb28e95eb37b8d827c0f48800c4d08e64c6fc335 10.128.51.14:7731 slots:0-5460 (5461 slots) master 2 additional replica(s) M: bc5051aef1a756c274907d23198fbb932c384151 10.129.51.30:7736 slots: (0 slots) master 0 additional replica(s) S: 4bee54a5049524a6280fdb8a8de00a5bb94ccaa1 10.128.51.14:7733 slots: (0 slots) slave replicates 07c99eae8176eb0e985c5a9914f42900d18fb39b M: b9a55f739a65e6277e40cdbb806b5443c8aad66e 10.128.51.14:7735 slots:5461-10922 (5462 slots) master 2 additional replica(s) [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. >>> Send CLUSTER MEET to node 10.129.51.30:7735 to make it join the cluster. [OK] New node added correctly.
槽位遷移： 從14的master，遷移至30上的master上
步驟1、首先查下cluster nodes ，得到節點信息后整理如下
10.129.51.30:7735> cluster nodes
//30機器主
27f7e8dede5b68581486ce8cefdd656032baed70 10.121.51.30:7734@17734 master - 0 1646125938000 4 connected
32ee19af1f6a534c4014b9c41d387666f049354b 10.121.51.30:7735@17735 myself,master - 0 1646125936000 0 connected
bc5051aef1a756c274907d23198fbb932c384151 10.121.51.30:7736@17736 master - 0 1646125939755 6 connected
//30機器從 91b7f8c79c91c7edd77458c332f0b9299bdb94d4 10.129.51.30:7732@17732 slave b9a55f739a65e6277e40cdbb806b5443c8aad66e 0 1646125940000 43 connected d46f032ea50763de8353fd530535412df6ffdc00 10.121.51.30:7731@17731 slave fb28e95eb37b8d827c0f48800c4d08e64c6fc335 0 1646125941758 44 connected d1c7d99e13a2d7317baf883dffa906470a606641 10.121.51.30:7733@17733 slave 07c99eae8176eb0e985c5a9914f42900d18fb39b 0 1646125940757 46 connected //14機器從 41767372c36cc268872e86d96660a32bf540624f 10.128.51.14:7732@17732 slave b9a55f739a65e6277e40cdbb806b5443c8aad66e 0 1646125942263 43 connected 4bee54a5049524a6280fdb8a8de00a5bb94ccaa1 10.128.51.14:7733@17733 slave 07c99eae8176eb0e985c5a9914f42900d18fb39b 0 1646125943767 46 connected ff34a5fadf42078332445bcf41a2b59416fd92da 10.128.51.14:7734@17734 slave fb28e95eb37b8d827c0f48800c4d08e64c6fc335 0 1646125938000 44 connected //14機器主 fb28e95eb37b8d827c0f48800c4d08e64c6fc335 10.128.51.14:7731@17731 master - 0 1646125942765 44 connected 0-5460
b9a55f739a65e6277e40cdbb806b5443c8aad66e 10.128.51.14:7735@17735 master - 0 1646125939560 43 connected 5461-10922
07c99eae8176eb0e985c5a9914f42900d18fb39b 10.128.51.14:7736@17736 master - 0 1646125937757 46 connected 10923-16383
一共就是0-16383個槽位，需要導入到30機器對應上（這里本來整理好了，但是我特意把某台master shutdown了，
但是這種比較清晰看着，所以還是放在這了, 重啟之后會發現master變了）
步驟2）規划到源節點進行遷移操作,要做三個master的轉移工作
一、7734 master的遷移工作。
[root@ht20 redis]# ./redis-cli -c -h 10.121.51.30 -p 7734 10.121.51.30:7734> cluster nodes 07c99eae8176eb0e985c5a9914f42900d18fb39b 10.128.51.14:7736@17736 master - 0 1646130359542 46 connected 10923-16383 准備要遷出的 b9a55f739a65e6277e40cdbb806b5443c8aad66e 10.128.51.14:7735@17735 master - 0 1646130363549 43 connected 5461-10922 准備要遷出的 91b7f8c79c91c7edd77458c332f0b9299bdb94d4 10.121.51.30:7732@17732 slave b9a55f739a65e6277e40cdbb806b5443c8aad66e 0 1646130362544 43 connected fb28e95eb37b8d827c0f48800c4d08e64c6fc335 10.128.51.14:7731@17731 slave ff34a5fadf42078332445bcf41a2b59416fd92da 0 1646130363549 49 connected ff34a5fadf42078332445bcf41a2b59416fd92da 10.128.51.14:7734@17734 master - 0 1646130362000 49 connected 0-5460 要遷移的源 27f7e8dede5b68581486ce8cefdd656032baed70 10.121.51.30:7734@17734 myself,master - 0 1646130357000 4 connected 要遷入的目標 4bee54a5049524a6280fdb8a8de00a5bb94ccaa1 10.128.51.14:7733@17733 slave 07c99eae8176eb0e985c5a9914f42900d18fb39b 0 1646130358541 46 connected 41767372c36cc268872e86d96660a32bf540624f 10.128.51.14:7732@17732 slave b9a55f739a65e6277e40cdbb806b5443c8aad66e 0 1646130360544 43 connected bc5051aef1a756c274907d23198fbb932c384151 10.121.51.30:7736@17736 master - 0 1646130359037 6 connected 32ee19af1f6a534c4014b9c41d387666f049354b 10.121.51.30:7735@17735 master - 0 1646130358000 0 connected d1c7d99e13a2d7317baf883dffa906470a606641 10.121.51.30:7733@17733 slave 07c99eae8176eb0e985c5a9914f42900d18fb39b 0 1646130363000 46 connected d46f032ea50763de8353fd530535412df6ffdc00 10.121.51.30:7731@17731 slave ff34a5fadf42078332445bcf41a2b59416fd92da 0 1646130364546 49 connected
// 27f7e8dede5b68581486ce8cefdd656032baed70 是導入目標id,槽位是5460
10.128.51.14:7734> cluster setslot 5460 MIGRATING 27f7e8dede5b68581486ce8cefdd656032baed70
OK
//查看下執行情況，如果想看到剪頭-> ，必須在要遷出的實例上登錄
10.128.51.14:7734> cluster nodes //必須在master節點或者被遷移的master上才能看到 箭頭->表示在遷移，別的節點看不到這種變化。
91b7f8c79c91c7edd77458c332f0b9299bdb94d4 10.121.51.30:7732@17732 slave b9a55f739a65e6277e40cdbb806b5443c8aad66e 0 1646132546000 43 connected
27f7e8dede5b68581486ce8cefdd656032baed70 10.121.51.30:7734@17734 master - 0 1646132548000 4 connected //遷入目標
d1c7d99e13a2d7317baf883dffa906470a606641 10.121.51.30:7733@17733 slave 07c99eae8176eb0e985c5a9914f42900d18fb39b 0 1646132546000 46 connected
41767372c36cc268872e86d96660a32bf540624f 10.128.51.14:7732@17732 slave b9a55f739a65e6277e40cdbb806b5443c8aad66e 0 1646132546811 43 connected
bc5051aef1a756c274907d23198fbb932c384151 10.121.51.30:7736@17736 master - 0 1646132547518 6 connected
07c99eae8176eb0e985c5a9914f42900d18fb39b 10.128.51.14:7736@17736 master - 0 1646132545000 46 connected 10923-16383 //准備遷出的
fb28e95eb37b8d827c0f48800c4d08e64c6fc335 10.128.51.14:7731@17731 slave ff34a5fadf42078332445bcf41a2b59416fd92da 0 1646132546000 49 connected
4bee54a5049524a6280fdb8a8de00a5bb94ccaa1 10.128.51.14:7733@17733 slave 07c99eae8176eb0e985c5a9914f42900d18fb39b 0 1646132547312 46 connected
b9a55f739a65e6277e40cdbb806b5443c8aad66e 10.128.51.14:7735@17735 master - 0 1646132548814 43 connected 5461-10922 //准備遷出的
d46f032ea50763de8353fd530535412df6ffdc00 10.121.51.30:7731@17731 slave ff34a5fadf42078332445bcf41a2b59416fd92da 0 1646132548000 49 connected
32ee19af1f6a534c4014b9c41d387666f049354b 10.121.51.30:7735@17735 master - 0 1646132545514 0 connected
//這里有變化，看到正在遷移。
ff34a5fadf42078332445bcf41a2b59416fd92da 10.128.51.14:7734@17734 myself,master - 0 1646132543000 49 connected 0-5460 [5460->-27f7e8dede5b68581486ce8cefdd656032baed70]
//ff34a5fadf42078332445bcf41a2b59416fd92da 導出目標id
//我們再用查看槽位是否遷移成功，cluster getkeysinslot 該指令表示 返回存儲在連接節點的指定插槽即槽位里面的是否有key名稱列表，應該是多個。
10.128.51.14:7734> cluster getkeysinslot 100 2 (empty list or set) 10.128.51.14:7734> cluster getkeysinslot 201 3 (empty list or set) 10.128.51.14:7734> cluster getkeysinslot 302 2 (empty list or set)
我們看到都為空
如果當前沒有數據，也不一定是沒有遷移，我們看別的master,//標記號為 07c99eae8176eb0e985c5a9914f42900d18fb39b，找上面的對應
[root@ht20 redis]# ./redis-cli -c -h 10.128.51.14 -p 7736 10.128.51.14:7736> cluster getkeysinslot 10923 10926 1) "4240247369" //這里有數據。
//大約過了幾個小時之后，我覺得比較安全了，我停掉了10.128.14上的redis 7734 實例。
[root@ht4 redis]# /data/redis_mai1/redis/redis-cli -c -h 10.128.51.14 -p 7733 //進入7733
10.128.51.14:7733> cluster nodes
d46f032ea50763de8353fd530535412df6ffdc00 10.121.51.30:7731@17731 master - 0 1646146691954 60 connected 0-5460 //已經轉移過來了
32ee19af1f6a534c4014b9c41d387666f049354b 10.121.51.30:7735@17735 master - 0 1646146691554 0 connected
bc5051aef1a756c274907d23198fbb932c384151 10.121.51.30:7736@17736 master - 0 1646146687000 6 connected
91b7f8c79c91c7edd77458c332f0b9299bdb94d4 10.121.51.30:7732@17732 slave b9a55f739a65e6277e40cdbb806b5443c8aad66e 0 1646146690000 43 connected
fb28e95eb37b8d827c0f48800c4d08e64c6fc335 10.128.51.14:7731@17731 slave d46f032ea50763de8353fd530535412df6ffdc00 0 1646146690947 60 connected
41767372c36cc268872e86d96660a32bf540624f 10.128.51.14:7732@17732 slave b9a55f739a65e6277e40cdbb806b5443c8aad66e 0 1646146689945 43 connected
ff34a5fadf42078332445bcf41a2b59416fd92da 10.128.51.14:7734@17734 master,fail - 1646146668988 1646146665880 49 disconnected //已停掉了
4bee54a5049524a6280fdb8a8de00a5bb94ccaa1 10.128.51.14:7733@17733 myself,slave 07c99eae8176eb0e985c5a9914f42900d18fb39b 0 1646146688000 45 connected
d1c7d99e13a2d7317baf883dffa906470a606641 10.121.51.30:7733@17733 slave 07c99eae8176eb0e985c5a9914f42900d18fb39b 0 1646146693959 46 connected
b9a55f739a65e6277e40cdbb806b5443c8aad66e 10.128.51.14:7735@17735 master - 0 1646146688000 43 connected 5461-10922 //准備遷出的
27f7e8dede5b68581486ce8cefdd656032baed70 10.121.51.30:7734@17734 master - 0 1646146692957 4 connected
07c99eae8176eb0e985c5a9914f42900d18fb39b 10.121.51.14:7736@17736 master - 0 1646146687936 46 connected 10923-16383 //准備遷出的
10.128.51.14:7733>
我類似的執行了（這里注意必須對應進入他們的控制台）
二、7735 master的遷移工作
[root@ht4 redis]# /data/redis_mai1/redis/redis-cli -c -h 10.128.51.14 -p 7735 //進入到7735
10.121.51.30:7735> set aaaaaa kingking //設置key-->value
-> Redirected to slot [11924] located at 10.128.51.14:7736 //11922即slot號
OK
[root@ht4 redis]# /data/redis_mai1/redis/redis-cli -c -h 10.128.51.14 -p 7736
10.128.51.14:7736> cluster getkeysinslot 11924 2 //slot槽號上輸出2個key.
1) "a2707b52fb94381977b22720f7dc752b63699adbffad9d9bd3b8b2b85349cffa79999ca07c7a8662f6d8afd58089f072c8f36b3772f74c4a093a06f4c3
5eb479059e293a83a10011"
2) "aaaaaa" //這個就是我們設定的key。
//我們在進入7735
[root@ht4 redis]# /data/redis_mai1/redis/redis-cli -c -h 10.128.51.14 -p 7735
10.128.51.14:7735> cluster setslot 10922 MIGRATING 32ee19af1f6a534c4014b9c41d387666f049354b //遷出到10.121.51.30：7735
10.128.51.14:7735> cluster getkeysinslot 5451 2
(empty list or set)
10.128.51.14:7735> cluster getkeysinslot 5580 3
(empty list or set)
10.128.51.14:7735> cluster getkeysinslot 5490 4
(empty list or set)
10.128.51.14:7735> cluster getkeysinslot 5482 3
//暫時沒有數據了。
10.128.51.14:7735> cluster nodes
27f7e8dede5b68581486ce8cefdd656032baed70 10.121.51.30:7734@17734 master - 0 1646148888438 4 connected
d46f032ea50763de8353fd530535412df6ffdc00 10.121.51.30:7731@17731 master - 0 1646148887435 60 connected 0-5460 //已遷出完畢
32ee19af1f6a534c4014b9c41d387666f049354b 10.121.51.30:7735@17735 master - 0 1646148888000 0 connected
4bee54a5049524a6280fdb8a8de00a5bb94ccaa1 10.128.51.14:7733@17733 slave 07c99eae8176eb0e985c5a9914f42900d18fb39b 0 1646148889434 46 connected
d1c7d99e13a2d7317baf883dffa906470a606641 10.121.51.30:7733@17733 slave 07c99eae8176eb0e985c5a9914f42900d18fb39b 0 1646148886000 46 connected
41767372c36cc268872e86d96660a32bf540624f 10.128.51.14:7732@17732 slave b9a55f739a65e6277e40cdbb806b5443c8aad66e 0 1646148889000 43 connected
07c99eae8176eb0e985c5a9914f42900d18fb39b 10.128.51.14:7736@17736 master - 0 1646148889000 46 connected 10923-16383 //准備遷出
fb28e95eb37b8d827c0f48800c4d08e64c6fc335 10.128.51.14:7731@17731 slave d46f032ea50763de8353fd530535412df6ffdc00 0 1646148886427 60 connected
bc5051aef1a756c274907d23198fbb932c384151 10.121.51.30:7736@17736 master - 0 1646148890941 6 connected
91b7f8c79c91c7edd77458c332f0b9299bdb94d4 10.121.51.30:7732@17732 slave b9a55f739a65e6277e40cdbb806b5443c8aad66e 0 1646148890441 43 connected
//正在遷出中
b9a55f739a65e6277e40cdbb806b5443c8aad66e 10.128.51.14:7735@17735 myself,master - 0 1646148882000 43 connected 5461-10922 [10922->-32ee19af1f6a534c4014b9c41d387666f049354b]
ff34a5fadf42078332445bcf41a2b59416fd92da :0@0 master,fail,noaddr - 1646146669049 1646146665845 49 disconnected
三、7736 master的遷移工作
[root@ht4 redis]# /data/redis_mai1/redis/redis-cli -c -h 10.128.51.14 -p 7736 //進入到7736
10.128.51.14:7736> cluster setslot 16383 MIGRATING bc5051aef1a756c274907d23198fbb932c384151 //導出到10.121.51.30：7736
//我們看看7736節點的變化情況。...
10.128.51.14:7736> cluster nodes //登錄7736才能看到 ---> 箭頭在工作。 41767372c36cc268872e86d96660a32bf540624f 10.128.51.14:7732@17732 slave b9a55f739a65e6277e40cdbb806b5443c8aad66e 0 1646148060771 43 connected b9a55f739a65e6277e40cdbb806b5443c8aad66e 10.128.51.14:7735@17735 master - 0 1646148059000 43 connected 5461-10922 //遷出中 d46f032ea50763de8353fd530535412df6ffdc00 10.129.51.30:7731@17731 master - 0 1646148057770 60 connected 0-5460 //已遷入,之前操作完了 bc5051aef1a756c274907d23198fbb932c384151 10.129.51.30:7736@17736 master - 0 1646148062779 6 connected //遷入中，當前正在操作，在寫本文時。
//遷出中 07c99eae8176eb0e985c5a9914f42900d18fb39b 10.128.51.14:7736@17736 myself,master - 0 1646148053000 46 connected 10923-16383
[16383->-bc5051aef1a756c274907d23198fbb932c384151]
d1c7d99e13a2d7317baf883dffa906470a606641 10.129.51.30:7733@17733 slave 07c99eae8176eb0e985c5a9914f42900d18fb39b 0 1646148056000 46 connected 32ee19af1f6a534c4014b9c41d387666f049354b 10.129.51.30:7735@17735 master - 0 1646148058000 0 connected //等待遷入 27f7e8dede5b68581486ce8cefdd656032baed70 10.129.51.30:7734@17734 master - 0 1646148060576 4 connected //等待遷入 fb28e95eb37b8d827c0f48800c4d08e64c6fc335 10.128.51.14:7731@17731 slave d46f032ea50763de8353fd530535412df6ffdc00 0 1646148059570 60 connected ff34a5fadf42078332445bcf41a2b59416fd92da :0@0 master,fail,noaddr - 1646146669048 1646146666847 49 disconnected //已停掉了 91b7f8c79c91c7edd77458c332f0b9299bdb94d4 10.129.51.30:7732@17732 slave b9a55f739a65e6277e40cdbb806b5443c8aad66e 0 1646148060000 43 connected 4bee54a5049524a6280fdb8a8de00a5bb94ccaa1 10.128.51.14:7733@17733 slave 07c99eae8176eb0e985c5a9914f42900d18fb39b 0 1646148061772 46 connected
//遷出中，看到數據，
//語法: CLUSTER getkeysinslot <slot槽號> <key的數量>：返回槽中N個key。 比如：cluster getkeysinslot 202 3
10.128.51.14:7736> cluster getkeysinslot 10924 3 1) "card_bin_data_623022" 2) "login_eyJ0eXAiOiJsb2dpbkpXVCibG9naW50b2tlbiIsImFsZyI6IkhTMjU2In0.eyJ1c2VyaWQiOiI5MDU4NjgyNDk2NTgyNDEwMzIiLCJleHAiOjE2M
zg5Mzk4MTYsImlhdCI6MTYzODkzOTgxNn0.1KV4uLqavTRBD4U_549j-7BehiXlA8HtFyHJbg6g3cc_bnz_user_type"
10.128.51.14:7736> cluster getkeysinslot 11924 3
1) "a2707b52fb94381977b22720f7dc752b63699adbffad9d9bd3b8b2b85349cffa79999ca07c7a8662f6d8afd58089f072c8f36b3772f74c4
a093a06f4c35eb479059e293a83a10 011"
2) "aaaaaa"
3) "access:admin78.news.delete"
后面像7734一樣操作即可，完成遷移工作。
最后結果如下：
10.129.51.30:7736> cluster nodes 27f7e8dede5b68581486ce8cefdd656032baed70 10.129.51.30:7734@17734 master - 0 1646180960000 4 connected 4bee54a5049524a6280fdb8a8de00a5bb94ccaa1 10.128.51.14:7733@17733 slave d1c7d99e13a2d7317baf883dffa906470a606641 0 1646180963601 62 connected 07c99eae8176eb0e985c5a9914f42900d18fb39b 10.128.51.14:7736@17736 master - 0 1646180963254 46 connected fb28e95eb37b8d827c0f48800c4d08e64c6fc335 10.128.51.14:7731@17731 slave d46f032ea50763de8353fd530535412df6ffdc00 0 1646180966505 60 connected 41767372c36cc268872e86d96660a32bf540624f 10.128.51.14:7732@17732 slave 91b7f8c79c91c7edd77458c332f0b9299bdb94d4 0 1646180965504 63 connected 32ee19af1f6a534c4014b9c41d387666f049354b 10.129.51.30:7735@17735 master - 0 1646180963497 0 connected d1c7d99e13a2d7317baf883dffa906470a606641 10.129.51.30:7733@17733 master - 0 1646180964497 62 connected 10923-16383 ff34a5fadf42078332445bcf41a2b59416fd92da 10.128.51.14:7734@17734 slave d46f032ea50763de8353fd530535412df6ffdc00 0 1646180966005 60 connected b9a55f739a65e6277e40cdbb806b5443c8aad66e 10.128.51.14:7735@17735 slave 91b7f8c79c91c7edd77458c332f0b9299bdb94d4 0 1646180961496 63 connected d46f032ea50763de8353fd530535412df6ffdc00 10.129.51.30:7731@17731 master - 0 1646180962494 60 connected 0-5460 bc5051aef1a756c274907d23198fbb932c384151 10.129.51.30:7736@17736 myself,master - 0 1646180962000 6 connected 91b7f8c79c91c7edd77458c332f0b9299bdb94d4 10.129.51.30:7732@17732 master - 0 1646180965000 63 connected 5461-10922
槽位已經全部轉移到新的集群節點上。
再測試一下。
10.129.51.30:7736> set test1 testtest
-> Redirected to slot [4768] located at 10.129.51.30:7731
OK
10.129.51.30:7731> cluster getkeysinslot 4768 2
1) "login_eyJ0eXAiOiJsb2dpbkpXVCIsImxvZ2luX3Rva2VuIjoibG9naW50b2tlbiIsImFsZyI6IkhTMjU2In0.eyJ1c2VyaWQiOiI0MjM5OTI5NTI4IiwiZX
hwIjoxNjQ2NDQzOTcwLCJpYXQiOjE2NDU1Nzk5NzB9.nc-ESGj4Od8A61D_90bNfxKSvlJ1mu2hFXPXym6cxh4stp_org"
2) "test1"
我們用另外的方式驗證下，k8s pod里面程序端輸出：
//遷移前 d1c7d99e13a2d7317baf883dffa906470a606641 10.129.51.30:7733@17733 slave 07c99eae8176eb0e985c5a9914f42900d18fb39b 0 1646131344055 46 connected d46f032ea50763de8353fd530535412df6ffdc00 10.129.51.30:7731@17731 myself,slave ff34a5fadf42078332445bcf41a2b59416fd92da 0 1646131343000 1 connected 07c99eae8176eb0e985c5a9914f42900d18fb39b 10.128.51.14:7736@17736 master - 0 1646131341857 46 connected 10923-16383 27f7e8dede5b68581486ce8cefdd656032baed70 10.129.51.30:7734@17734 master - 0 1646131341000 4 connected ff34a5fadf42078332445bcf41a2b59416fd92da 10.128.51.14:7734@17734 master - 0 1646131340000 49 connected 0-5460 41767372c36cc268872e86d96660a32bf540624f 10.128.51.14:7732@17732 slave b9a55f739a65e6277e40cdbb806b5443c8aad66e 0 1646131343859 43 connected 91b7f8c79c91c7edd77458c332f0b9299bdb94d4 10.129.51.30:7732@17732 slave b9a55f739a65e6277e40cdbb806b5443c8aad66e 0 1646131342000 43 connected fb28e95eb37b8d827c0f48800c4d08e64c6fc335 10.128.51.14:7731@17731 slave ff34a5fadf42078332445bcf41a2b59416fd92da 0 1646131342859 49 connected 32ee19af1f6a534c4014b9c41d387666f049354b 10.129.51.30:7735@17735 master - 0 1646131342554 0 connected bc5051aef1a756c274907d23198fbb932c384151 10.129.51.30:7736@17736 master - 0 1646131343000 6 connected 4bee54a5049524a6280fdb8a8de00a5bb94ccaa1 10.128.51.14:7733@17733 slave 07c99eae8176eb0e985c5a9914f42900d18fb39b 0 1646131341000 46 connected b9a55f739a65e6277e40cdbb806b5443c8aad66e 10.128.51.14:7735@17735 master - 0 1646131343000 43 connected 5461-10922 //遷移后，所有的h4上的節點全部變成slave, ht20上的節點變成master,這個輸出結果是我整體排序過了 d1c7d99e13a2d7317baf883dffa906470a606641 10.129.51.30:7733@17733 master - 0 1646182634648 62 connected 10923-16383 91b7f8c79c91c7edd77458c332f0b9299bdb94d4 10.129.51.30:7732@17732 master - 0 1646182632142 63 connected 5461-10922 d46f032ea50763de8353fd530535412df6ffdc00 10.129.51.30:7731@17731 myself,master - 0 1646182631000 60 connected 0-5460 27f7e8dede5b68581486ce8cefdd656032baed70 10.129.51.30:7734@17734 master - 0 1646182631000 4 connected 32ee19af1f6a534c4014b9c41d387666f049354b 10.129.51.30:7735@17735 master - 0 1646182630538 0 connected bc5051aef1a756c274907d23198fbb932c384151 10.129.51.30:7736@17736 master - 0 1646182630000 6 connected 07c99eae8176eb0e985c5a9914f42900d18fb39b 10.128.51.14:7736@17736 slave d1c7d99e13a2d7317baf883dffa906470a606641 0 1646182634151 62 connected ff34a5fadf42078332445bcf41a2b59416fd92da 10.128.51.14:7734@17734 slave d46f032ea50763de8353fd530535412df6ffdc00 0 1646182632000 60 connected 41767372c36cc268872e86d96660a32bf540624f 10.128.51.14:7732@17732 slave 91b7f8c79c91c7edd77458c332f0b9299bdb94d4 0 1646182633149 63 connected fb28e95eb37b8d827c0f48800c4d08e64c6fc335 10.128.51.14:7731@17731 slave d46f032ea50763de8353fd530535412df6ffdc00 0 1646182630000 60 connected 4bee54a5049524a6280fdb8a8de00a5bb94ccaa1 10.128.51.14:7733@17733 slave d1c7d99e13a2d7317baf883dffa906470a606641 0 1646182636154 62 connected b9a55f739a65e6277e40cdbb806b5443c8aad66e 10.128.51.14:7735@17735 slave 91b7f8c79c91c7edd77458c332f0b9299bdb94d4 0 1646182635153 63 connected
我們回到控制台，驗證是對應驗證，否則，你在ht20的7735上執行下面語句是空的，因為槽位(slot)不對。
下面所有槽位都分配在了新的集群節點上了， 他們兩個節點是互為主從關系。
[root@ht4 redis]# /data/redis_fpmai1/redis/redis-cli -c -h 10.128.51.14 -p 7736
10.128.51.14:7736> cluster getkeysinslot 10924 3
1) "card_bin_data_623022"
2) "login_eyJ0eXAiOiJsb2dpbkpXVCIsImxvZ2luX3Rva2VuIjoibG9naW50b2tlbiIsImFsZyI6IkhTMjU2In0.eyJ1c2VyaWDk2NTgyNDEwMzIiLC
JleHAiOjE2Mzg5Mzk4MTYsImlhdCI6MTYzODkzOTgxNn0.1KV4uLqavTRBD4U_549j-7BehiXlA8HtFyHJbg6g3cc_bnz_user_type
[root@ht20 redis]# ./redis-cli -c -h 10.129.51.30 -p 7733
10.129.51.30:7733> cluster getkeysinslot 10924 3
1) "card_bin_data_623022"
2) "login_eyJ0eXAiOiJsb2dpbkpXVCIsImxvZ2luX3Rva2VuIjoibG9naW50b2tlbiIsImFsZyI6IkhTMjU2In0.eyJ1c2VyaWQiOiI5MDU4NjgyNDk2NTgyNDEwMzIiLC
JleHAiOjE2Mzg5Mzk4MTYsImlhdCI6MTYzODkzOTgxNn0.1KV4uLqavTRBD4U_549j-7BehiXlA8HtFyHJbg6g3cc_bnz_user_type"
程序端因為是用pod部署，所以重新部署pod即可, 我們現在使用的就是新的集群，老的數據了。