碼上歡樂
相關內容    簡體    繁體

redis集群搭建

本文轉載自   查看原文   2019-01-22 11:44   611    Java/ Linux

 
需求
搭建一個redis集群，支持故障恢復,。
Redis內置的一個集群需要至少3個主節點，所以搭建規模為3主3個從的集群用於測試。節點信息列表：
 
role  ip port bus port remark
master1 192.168.201.209 7000 17000 主節點1
master2 192.168.201.209 7001 17001 主節點2
master3 192.168.201.209 7002 17002 主節點3
replica1 192.168.201.209 7003 17003 從節點1
replica1 192.168.201.209 7004 17004 從節點2
replica1 192.168.201.209 7005 17005 從節點3
sentinel1 192.168.201.209 6000 - 哨兵1
sentinel2 192.168.201.209 6001 - 哨兵2
sentinel3 192.168.201.209 6002 - 哨兵3
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
環境：
  • 鏡像redis:alpine，redis的版本為5.0.3
  • macOS 10.13.4
  • Docker 18.03.1-ce
 
 
 
docker中用到的配置文件命名 
➜  config pwd
/Users/cymin/redis/config
➜  config tree .
.
├── m1.conf
├── m2.conf
├── m3.conf
├── r1.conf
├── r2.conf
├── r3.conf
├── s1.conf
├── s2.conf
└── s3.conf
 
0 directories, 9 files

　　

Note that:
  • Masters are called M1, M2, M3, ..., Mn.
  • Slaves are called R1, R2, R3, ..., Rn (R stands for replica).
  • Sentinels are called S1, S2, S3, ..., Sn.
 
 
3個redis master實例配置 
➜  config cat m1.conf
cluster-announce-ip 192.168.201.209
cluster-announce-port 7000
cluster-announce-bus-port 17000
 
cluster-enabled yes
cluster-config-file "/tmp/nodes.conf"
cluster-node-timeout 5000
appendonly yes

　　

剩余的master端口依次+1, cluster-announce-bus-port的配置+10000.
說明：
  • cluster-announce-ip 告訴redis實例外部映射地址
  • cluster-announce-port 告訴redis實例外部映射端口
  • cluster-announce-bus-port 開啟集群總線端口,值=cluster-announce-port +10000
  • masterauth | requirepass 密碼，master和replica需要配置為一樣的密碼
  • 其他都是基本配置,相關文檔： https://redis.io/topics/cluster-tutorial
 
 
 
3個redis replica(slave)實例配置 
➜  config cat r1.conf
cluster-announce-ip 192.168.201.209
cluster-announce-port 7003
cluster-announce-bus-port 17003
 
cluster-enabled yes
cluster-config-file "/tmp/nodes.conf"
cluster-node-timeout 5000
appendonly yes

　　

剩余的cluster-announce-port端口依次+1，cluster-announce-bus-port的配置+10000。
sentinel實例之間的通訊端口默認是26379.
 
 
 
3個哨兵實例配置 
➜  config cat s1.conf
sentinel announce-ip 192.168.201.209
sentinel announce-port 6000
 
sentinel monitor master1 192.168.201.209 7000 2
sentinel down-after-milliseconds master1 60000
sentinel failover-timeout master1 180000
sentinel parallel-syncs master1 1
 
sentinel monitor master2 192.168.201.209 7001 2
sentinel down-after-milliseconds master2 60000
sentinel failover-timeout master2 180000
sentinel parallel-syncs master2 1
 
sentinel monitor master3 192.168.201.209 7002 2
sentinel down-after-milliseconds master3 60000
sentinel failover-timeout master3 180000
sentinel parallel-syncs master3 1　
剩余的announce-port配置依次加1.
說明：每個哨兵只需要監控master節點即可，配置項說明見文檔： https://redis.io/topics/sentinel
 
 
 
啟動redis實例和哨兵
run.sh 
# 3個master
docker run --name master1 -tid --network service -v $HOME/redis/config:/opt/config -p 7000:6379 -p 17000:16379  redis:alpine redis-server /opt/config/m1.conf
docker run --name master2 -tid --network service -v $HOME/redis/config:/opt/config -p 7001:6379 -p 17001:16379  redis:alpine redis-server /opt/config/m2.conf
docker run --name master3 -tid --network service -v $HOME/redis/config:/opt/config -p 7002:6379 -p 17002:16379  redis:alpine redis-server /opt/config/m3.conf
 
# 3個replica
docker run --name replica1 -tid --network service -v $HOME/redis/config:/opt/config -p 7003:6379 -p 17003:16379  redis:alpine redis-server /opt/config/r1.conf
docker run --name replica2 -tid --network service -v $HOME/redis/config:/opt/config -p 7004:6379 -p 17004:16379  redis:alpine redis-server /opt/config/r2.conf
docker run --name replica3 -tid --network service -v $HOME/redis/config:/opt/config -p 7005:6379 -p 17005:16379  redis:alpine redis-server /opt/config/r3.conf
 
# 3個sentinel
docker run --name sentinel1 -tid --network service -v $HOME/redis/config:/opt/config -p 6000:26379 redis:alpine redis-sentinel /opt/config/s1.conf
docker run --name sentinel2 -tid --network service -v $HOME/redis/config:/opt/config -p 6001:26379 redis:alpine redis-sentinel /opt/config/s2.conf
docker run --name sentinel3 -tid --network service -v $HOME/redis/config:/opt/config -p 6002:26379 redis:alpine redis-sentinel /opt/config/s3.conf

　　

 
 
創建集群 
docker exec -it master1 sh
/data # redis-cli --cluster create 192.168.201.209:7000 192.168.201.209:7001 192.168.201.209:7002 192.168.201.209:7003 192.168.201.209:7004 192.168.201.209:7005 --cluster-replicas 1
最后收到"[OK] All 16384 slots covered.”的日志時，集群搭建成功。
 
 
 
連接集群中某個redis實例 
redis-cli -c -h 192.168.201.209 -p 7000

　　

查看集群狀態 
redis-cli cluster nodes
 
 
查看某個節點的主從狀態 
redis-cli -h 192.168.201.209 -p 7000 info Replication

 

 
 
查看某個哨兵的狀態 
redis-cli -h 192.168.201.209 -p 6000 info Sentinel

　　

 
故障自動恢復測試 
# 集群節點狀態
/data # redis-cli cluster nodes
b22123911ffed94e6827a94c55891a4488dbbf5c 192.168.201.209:7003@17003 slave 507f3ec4b24d8d380b4da0c107214dae5b4c08b5 0 1547716358000 4 connected
da63ff7af84ba911d905a8a2e0494520e1eaa6c9 192.168.201.209:7004@17004 slave 4de60b987e84de81a6fce10d68f4b1ba0d2ac28a 0 1547716357525 5 connected
507f3ec4b24d8d380b4da0c107214dae5b4c08b5 192.168.201.209:7001@17001 myself,master - 0 1547716357000 2 connected 5461-10922
e9f1294a466d3c2e4c3b40d59c8237c346c7fed7 192.168.201.209:7005@17005 slave 7d0d00d7710fc416ac2c3f44ea86b306b498038e 0 1547716357014 6 connected
4de60b987e84de81a6fce10d68f4b1ba0d2ac28a 192.168.201.209:7002@17002 master - 0 1547716358535 3 connected 10923-16383
7d0d00d7710fc416ac2c3f44ea86b306b498038e 192.168.201.209:7000@17000 master - 0 1547716356511 1 connected 0-5460
 
# 停止master1
docker stop master1
 
# 集群節點狀態
/data # redis-cli cluster nodes
b22123911ffed94e6827a94c55891a4488dbbf5c 192.168.201.209:7003@17003 slave 507f3ec4b24d8d380b4da0c107214dae5b4c08b5 0 1547716754542 4 connected
da63ff7af84ba911d905a8a2e0494520e1eaa6c9 192.168.201.209:7004@17004 slave 4de60b987e84de81a6fce10d68f4b1ba0d2ac28a 0 1547716754542 5 connected
507f3ec4b24d8d380b4da0c107214dae5b4c08b5 192.168.201.209:7001@17001 myself,master - 0 1547716753000 2 connected 5461-10922
e9f1294a466d3c2e4c3b40d59c8237c346c7fed7 192.168.201.209:7005@17005 master - 0 1547716754000 7 connected 0-5460
4de60b987e84de81a6fce10d68f4b1ba0d2ac28a 192.168.201.209:7002@17002 master - 0 1547716754946 3 connected 10923-16383
7d0d00d7710fc416ac2c3f44ea86b306b498038e 192.168.201.209:7000@17000 master,fail - 1547716421213 1547716420405 1 disconnected
 
# 重新啟動master1
docker restart master1
 
# 集群節點狀態
/data # redis-cli cluster nodes
b22123911ffed94e6827a94c55891a4488dbbf5c 192.168.201.209:7003@17003 slave 507f3ec4b24d8d380b4da0c107214dae5b4c08b5 0 1547716837000 4 connected
da63ff7af84ba911d905a8a2e0494520e1eaa6c9 192.168.201.209:7004@17004 slave 4de60b987e84de81a6fce10d68f4b1ba0d2ac28a 0 1547716837000 5 connected
507f3ec4b24d8d380b4da0c107214dae5b4c08b5 192.168.201.209:7001@17001 myself,master - 0 1547716836000 2 connected 5461-10922
e9f1294a466d3c2e4c3b40d59c8237c346c7fed7 192.168.201.209:7005@17005 master - 0 1547716838053 7 connected 0-5460
4de60b987e84de81a6fce10d68f4b1ba0d2ac28a 192.168.201.209:7002@17002 master - 0 1547716837138 3 connected 10923-16383
7d0d00d7710fc416ac2c3f44ea86b306b498038e 192.168.201.209:7000@17000 slave e9f1294a466d3c2e4c3b40d59c8237c346c7fed7 0 1547716838153 7 connected

觀察當id為7d0d00d7710fc416ac2c3f44ea86b306b498038e的master節點掛掉的時候，其下id為e9f1294a466d3c2e4c3b40d59c8237c346c7fed7的slave的節點自動升級為master節點，當之前的master又重新啟動的時候自動變成slave節點。

 
 
增加一個節點 
# 增加一個redis實例，replica4端口映射到本地為7006
docker run --name replica4 -tid --network service -v $HOME/redis/config:/opt/config -p 7006:6379 -p 17006:16379  redis:alpine redis-server /opt/config/r4.conf
 
# 給id為4de60b987e84de81a6fce10d68f4b1ba0d2ac28a增加一個slave
redis-cli --cluster add-node 192.168.201.209:7006 192.168.201.209:7002 --cluster-slave --cluster-master-id 4de60b987e84de81a6fce10d68f4b1ba0d2ac28a
 
# 查看集群節點狀態
/data # redis-cli cluster nodes
b22123911ffed94e6827a94c55891a4488dbbf5c 192.168.201.209:7003@17003 slave 507f3ec4b24d8d380b4da0c107214dae5b4c08b5 0 1547718761000 4 connected
da63ff7af84ba911d905a8a2e0494520e1eaa6c9 192.168.201.209:7004@17004 slave 4de60b987e84de81a6fce10d68f4b1ba0d2ac28a 0 1547718761820 5 connected
507f3ec4b24d8d380b4da0c107214dae5b4c08b5 192.168.201.209:7001@17001 myself,master - 0 1547718761000 2 connected 5461-10922
e9f1294a466d3c2e4c3b40d59c8237c346c7fed7 192.168.201.209:7005@17005 master - 0 1547718761000 7 connected 0-5460
435a5e171bd4ffc02e862d174609fa0d07045925 192.168.201.209:7006@17006 slave 4de60b987e84de81a6fce10d68f4b1ba0d2ac28a 0 1547718761312 3 connected
4de60b987e84de81a6fce10d68f4b1ba0d2ac28a 192.168.201.209:7002@17002 master - 0 1547718760000 3 connected 10923-16383
7d0d00d7710fc416ac2c3f44ea86b306b498038e 192.168.201.209:7000@17000 slave e9f1294a466d3c2e4c3b40d59c8237c346c7fed7 0 1547718761517 7 connected

　　

 
 
 
刪除一個節點 
/data # redis-cli --cluster del-node 192.168.201.209:7000 435a5e171bd4ffc02e862d174609fa0d07045925
>>> Removing node 435a5e171bd4ffc02e862d174609fa0d07045925 from cluster 192.168.201.209:7000
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.

# 查看集群節點狀態
/data # redis-cli cluster nodes
b22123911ffed94e6827a94c55891a4488dbbf5c 192.168.201.209:7003@17003 slave 507f3ec4b24d8d380b4da0c107214dae5b4c08b5 0 1547719609601 4 connected
da63ff7af84ba911d905a8a2e0494520e1eaa6c9 192.168.201.209:7004@17004 slave 4de60b987e84de81a6fce10d68f4b1ba0d2ac28a 0 1547719610512 5 connected
507f3ec4b24d8d380b4da0c107214dae5b4c08b5 192.168.201.209:7001@17001 myself,master - 0 1547719611000 2 connected 5461-10922
e9f1294a466d3c2e4c3b40d59c8237c346c7fed7 192.168.201.209:7005@17005 master - 0 1547719611626 7 connected 0-5460
4de60b987e84de81a6fce10d68f4b1ba0d2ac28a 192.168.201.209:7002@17002 master - 0 1547719610512 3 connected 10923-16383
7d0d00d7710fc416ac2c3f44ea86b306b498038e 192.168.201.209:7000@17000 slave e9f1294a466d3c2e4c3b40d59c8237c346c7fed7 0 1547719610613 7 connected

　　

 
 
數據同步測試 
/data # redis-cli -c -h 192.168.201.209 -p 7000
192.168.201.209:7000> keys *
(empty list or set)
192.168.201.209:7000> set name cymin
-> Redirected to slot [5798] located at 192.168.201.209:7001
OK
192.168.201.209:7001>
/data # redis-cli -c -h 192.168.201.209 -p 7001
192.168.201.209:7001> keys *
1) "name"
192.168.201.209:7001>
/data # redis-cli -c -h 192.168.201.209 -p 7002
192.168.201.209:7002> keys *
(empty list or set)
192.168.201.209:7002> get name
-> Redirected to slot [5798] located at 192.168.201.209:7001
“cymin”
192.168.201.209:7001>
192.168.201.209:7001> get afawfgawfgw
-> Redirected to slot [15081] located at 192.168.201.209:7002
(nil)
192.168.201.209:7002> get feawgwegewgew
-> Redirected to slot [7926] located at 192.168.201.209:7001
(nil)
192.168.201.209:7001>

觀察數據讀寫均在master節點，如果在replica節點中讀寫key，則會自動重定向到對應的master節點中。

 
 
集群密碼設置
進入各個(6個)實例進行依次設置: 
redis-cli -c -h 192.168.201.209 -p 7000
127.0.0.1:7000> config set masterauth passwd123
127.0.0.1:7000> config set requirepass passwd123
 
# 重新進入實例，寫入配置 
redis-cli -a abc123 -c -h 192.168.201.209 -p 7000
127.0.0.1:7000> config rewrite

 

 
 
 
java連接集群事例
Jar: 
compile group: 'redis.clients', name: 'jedis', version: '3.0.1'

　　 

使用jedis在代碼中連接redis集群，因為是集群，可以自動發現，也同樣只要一個地址就行了 
import org.apache.commons.pool2.impl.GenericObjectPoolConfig;
import redis.clients.jedis.HostAndPort;
import redis.clients.jedis.JedisCluster;
 
import java.util.HashSet;
import java.util.Set;
 
public class RedisTest {
    public static void main(String[] args) {
        testWithPassword();
    }
 
    public static void test() {
        Set<HostAndPort> jedisClusterNodes = new HashSet<>();
        jedisClusterNodes.add(new HostAndPort("192.168.201.208", 7000));
        JedisCluster jc = new JedisCluster(jedisClusterNodes);
        String value = jc.get("name");
        System.out.println(value);
    }
 
 
    public static void testWithPassword() {
        Set<HostAndPort> jedisClusterNodes = new HashSet<>();
        jedisClusterNodes.add(new HostAndPort("192.168.201.208", 7000));
        GenericObjectPoolConfig gopc = new GenericObjectPoolConfig();
        gopc.setMaxTotal(32);
        gopc.setMaxIdle(4);
        JedisCluster jc = new JedisCluster(jedisClusterNodes, 10000, 10000, 3,"abc123", gopc);
        String value = jc.get("name");
        System.out.println(value);
    }
}

　　

 
 
 
 
The end. 


免責聲明！

本站轉載的文章為個人學習借鑒使用，本站對版權不負任何法律責任。如果侵犯了您的隱私權益，請聯系本站郵箱yoyou2525@163.com刪除。



猜您在找 Redis集群(五)：集群搭建 Linux搭建redis集群 redis集群部署和搭建 redis sentinel集群的搭建 redis單機及其集群的搭建 CentOS搭建Redis集群 redis 偽集群搭建 Redis集群搭建 Redis集群環境搭建 redis集群搭建
 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM