redis集群同步遷移方法(一):通過redis replication實現


       講到redis的遷移,一般會使用rdb或者aof在主庫做自動重載到目標庫方法。但該方法有個問題就是無法保證源節點數據和目標節點數據保持一致,一般線上環境也不允許源庫停機,所以要在遷移過程后還要實現同步達到數據的一致性。公司線上環境使用的是redis自己的cluster,每個節點都擁有多個rdb和aof文件,使用原始方法無疑是難上加難。本文主要討論兩種方法來實現不停機源庫前提下,實現源庫(redis cluster)到目標庫(cluster或者單實例)的遷移:
  • 采用redis replication實現
  • 使用開源同步開源工具
方法一:通過redis復制機制,將目標節點作為源節點的從節點,然后關閉源節點,進行主從自動fail over,最后再關閉並刪除源節點實例
1.運行環境:
源節點實例:127.0.0.1:12000/127.0.0.1:12001/127.0.0.1:12002
[root@10_86_30_37_10.86.30.37 mycluster_export1]# redis-cli -p 12000 cluster nodes
e5ce695f7c5745ca81b4239fb5666b6a71fbb4ea 127.0.0.1:12000 myself,master - 0 0 1 connected 0-5000
f63f0d52372ad8b5c414c47e9318717b6aa113cc 127.0.0.1:12001 master - 0 1463025774035 2 connected 5001-10000
fdeb68f696290a91f08a5da3b8a3c585aaa35856 127.0.0.1:12002 master - 0 1463025775037 0 connected 10001-16383
遷移目標節點實例:
127.0.0.1:13000/127.0.0.1:13001/127.0.0.1:13002
2.遷移過程
  • 啟動三個目標節點,配置了redis集群模式的實例
redis-server redis13000.conf 
redis-server redis13001.conf 
redis-server redis13002.conf
  • 將這三個節點做已有集群實例的slave
redis-cli -p 13000 cluster meet 127.0.0.1 12000                                         
redis-cli -p 13000 cluster replicate e5ce695f7c5745ca81b4239fb5666b6a71fbb4ea
redis-cli -p 13001 cluster meet 127.0.0.1 12001
redis-cli -p 13001 cluster replicate f63f0d52372ad8b5c414c47e9318717b6aa113cc
redis-cli -p 13002 cluster meet 127.0.0.1 12002
redis-cli -p 13002 cluster replicate fdeb68f696290a91f08a5da3b8a3c585aaa35856
  • 查看集群情況:
redis-cli -p 13002 cluster slots
1) 1) (integer) 5001
   2) (integer) 10000
   3) 1) "127.0.0.1"
      2) (integer) 12001
   4) 1) "127.0.0.1"
      2) (integer) 13001
2) 1) (integer) 10001
   2) (integer) 16383
   3) 1) "127.0.0.1"
      2) (integer) 12002
   4) 1) "127.0.0.1"
      2) (integer) 13002
3) 1) (integer) 0
   2) (integer) 5000
   3) 1) "127.0.0.1"
      2) (integer) 12000
   4) 1) "127.0.0.1"
      2) (integer) 13000
  • 將其中一個主節點下線,10s后觀察情況:
redis-cli -p 12000 shutdown
redis-cli -p 13000 cluster nodes
fdeb68f696290a91f08a5da3b8a3c585aaa35856 127.0.0.1:12002 master - 0 1463042318423 0 connected 10001-16383
f63f0d52372ad8b5c414c47e9318717b6aa113cc 127.0.0.1:12001 master - 0 1463042319426 2 connected 5001-10000
e39a73c30dfff3139242e66f8e0a41178b39e280 127.0.0.1:13000 myself,master - 0 0 6 connected 0-5000
1f552bdea453caeaa64b4b33a05e4eedeb3f3dd2 127.0.0.1:13001 slave f63f0d52372ad8b5c414c47e9318717b6aa113cc 0 1463042317421 4 connected
6a70a82c6f07dc4e61a97b6aee7a2994365642cc 127.0.0.1:13002 slave fdeb68f696290a91f08a5da3b8a3c585aaa35856 0 1463042320429 5 connected
e5ce695f7c5745ca81b4239fb5666b6a71fbb4ea 127.0.0.1:12000 master,fail - 1463042217663 1463042214056 1 disconnected
  • 刪除已經下線的主節點,一個一個操作,操作中間檢查操作是否成功,因為留言協議和failover需要一段時間
redis-cli -p 13000 cluster forget e5ce695f7c5745ca81b4239fb5666b6a71fbb4ea
redis-cli -p 13001 cluster forget e5ce695f7c5745ca81b4239fb5666b6a71fbb4ea
redis-cli -p 13002 cluster forget e5ce695f7c5745ca81b4239fb5666b6a71fbb4ea
redis-cli -p 13000 cluster nodes
fdeb68f696290a91f08a5da3b8a3c585aaa35856 127.0.0.1:12002 master - 0 1463043284365 0 connected 10001-16383
f63f0d52372ad8b5c414c47e9318717b6aa113cc 127.0.0.1:12001 master - 0 1463043283364 2 connected 5001-10000
e39a73c30dfff3139242e66f8e0a41178b39e280 127.0.0.1:13000 myself,master - 0 0 6 connected 0-5000
1f552bdea453caeaa64b4b33a05e4eedeb3f3dd2 127.0.0.1:13001 slave f63f0d52372ad8b5c414c47e9318717b6aa113cc 0 1463043286369 4 connected
6a70a82c6f07dc4e61a97b6aee7a2994365642cc 127.0.0.1:13002 slave fdeb68f696290a91f08a5da3b8a3c585aaa35856 0 1463043285368 5 connected
 
redis-cli -p 13000 cluster nodes                                         
e39a73c30dfff3139242e66f8e0a41178b39e280 127.0.0.1:13000 myself,master - 0 0 6 connected 0-5000
1f552bdea453caeaa64b4b33a05e4eedeb3f3dd2 127.0.0.1:13001 slave - 0 1463043457854 4 connected
6a70a82c6f07dc4e61a97b6aee7a2994365642cc 127.0.0.1:13002 slave - 0 1463043458857 5 connected
3.重點細節:
  • 刪除的主節點,如果重新啟動,他自身會重新加載集群配置文件,造成集群混亂,建議如果想重啟該實例,刪掉集群配置文件,進行重新配置。
  • 必須先關閉master節點后,再刪除。一次不能將所有實例都關閉,逐個操作,否則會造成整個集群down掉
  • 刪除forget節點時,要在所有其他節點上執行cluster forget 命令,貌似這個命令不會通過留言協議傳播到所有節點


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM