第一步:環境准備
我們搞兩台虛擬機
局域網IP 分別是 192.168.1.109和192.168.1.110
我們約定把192.168.1.109作為集群控制端,需要安裝redis-trib.rb
第二步:安裝Redis
前面已經安裝過了 不解釋,
192.168.1.109虛擬機里,Reids安裝包里有個集群工具,要復制到/usr/local/bin里去
cp redis-3.2.9/src/redis-trib.rb /usr/local/bin
第三步:修改配置,創建節點
首先我們在192.168.1.109虛擬機里創建三個節點,端口分別是7001,7002,7003
我們先在root目錄下新建一個redis_cluster目錄,然后該目錄下再創建3個目錄,
分別是7001,7002,7003,用來存redis配置文件;
這里我們要使用redis集群,要先修改redis的配置文件redis.conf
mkdir redis_cluster 新建目錄
[root@localhost ~]# cd redis_cluster/
[root@localhost redis_cluster]# mkdir 7001 7002 7003
[root@localhost redis_cluster]# ll
總用量 0
drwxr-xr-x. 2 root root 6 7月 27 19:49 7001
drwxr-xr-x. 2 root root 6 7月 27 19:49 7002
drwxr-xr-x. 2 root root 6 7月 27 19:49 7003
[root@localhost redis_cluster]#
先復制一份配置文件到7001目錄下
[root@localhost redis_cluster]# cd
[root@localhost ~]# cp redis-3.2.9/redis.conf redis_cluster/7001/
我們修改下這個配置文件
vi redis_cluster/7001/redis.conf
修改一下幾個
port 7001 //六個節點配置文件分別是7001-7003
bind 192.168.1.109 //默認ip為127.0.0.1 需要改為其他節點機器可訪問的ip 否則創建集群時無法訪,和單機集群有區別
daemonize yes //redis后台運行
pidfile /var/run/redis_7001.pid //pidfile文件對應7001-7003
cluster-enabled yes //開啟集群
cluster-config-file nodes_7001.conf //保存節點配置,自動創建,自動更新對應7001-7003
cluster-node-timeout 5000 //集群超時時間,節點超過這個時間沒反應就斷定是宕機
appendonly yes //存儲方式,aof,將寫操作記錄保存到日志中
7001下的修改完后,我們把7001下的配置分別復制到7002-7003 然后對應的再修改下配置即可;
[root@localhost ~]# cp redis_cluster/7001/redis.conf redis_cluster/7002/
[root@localhost ~]# cp redis_cluster/7001/redis.conf redis_cluster/7003/
[root@localhost ~]# vi redis_cluster/7002/redis.conf
[root@localhost ~]# vi redis_cluster/7003/redis.conf
[root@localhost ~]# vi redis_cluster/7004/redis.conf
編輯后面5個配置文件,把 port ,pidfile,cluster-config-file 分別修改下即可;
同理 192.168.1.110機器,也搞一個redis_cluster目錄,然后再新建7004,7005,7006目錄,
復制三份配置文件 其他一樣,就端口和bind地址改下即可;這里不啰嗦了;
第四步:啟動兩台機器的六個節點
192.168.1.109機器
[root@localhost ~]# /usr/local/redis/bin/redis-server redis_cluster/7001/redis.conf
[root@localhost ~]# /usr/local/redis/bin/redis-server redis_cluster/7002/redis.conf
[root@localhost ~]# /usr/local/redis/bin/redis-server redis_cluster/7003/redis.conf
[root@localhost ~]# ps -ef | grep redis
root 2242 1 0 19:55 ? 00:00:00 /usr/local/redis/bin/redis-server 192.168.1.109:7001 [cluster]
root 2252 1 0 19:59 ? 00:00:00 /usr/local/redis/bin/redis-server 192.168.1.109:7002 [cluster]
root 2256 1 0 19:59 ? 00:00:00 /usr/local/redis/bin/redis-server 192.168.1.109:7003 [cluster]
root 2260 2214 0 19:59 pts/0 00:00:00 grep --color=auto redis
192.168.1.110機器
[root@localhost ~]# /usr/local/redis/bin/redis-server redis_cluster/7004/redis.conf
[root@localhost ~]# /usr/local/redis/bin/redis-server redis_cluster/7005/redis.conf
[root@localhost ~]# /usr/local/redis/bin/redis-server redis_cluster/7006/redis.conf
[root@localhost ~]# ps -ef | grep redis
root 2347 1 0 20:31 ? 00:00:00 /usr/local/redis/bin/redis-server 192.168.1.110:7004 [cluster]
root 2351 1 0 20:31 ? 00:00:00 /usr/local/redis/bin/redis-server 192.168.1.110:7005 [cluster]
root 2355 1 0 20:31 ? 00:00:00 /usr/local/redis/bin/redis-server 192.168.1.110:7006 [cluster]
root 2363 2270 0 20:32 pts/0 00:00:00 grep --color=auto redis
說明都啟動OK
第五步:設置防火牆,開放集群端口
兩台機器的防火牆我們直接關掉
systemctl stop firewalld.service
第六步:創建集群
192.168.1.109機器作為集群控制端
redis官方提供了redis-trib.rb工具,第一步里已經房到里bin下 ;
但是在使用之前 需要安裝ruby,以及redis和ruby連接
yum -y install ruby ruby-devel rubygems rpm-build
gem install redis
redis-trib.rb create --replicas 1 192.168.1.109:7001 192.168.1.109:7002 192.168.1.109:7003 192.168.1.110:7004 192.168.1.110:7005 192.168.1.110:7006
創建集群
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
192.168.1.109:7001
192.168.1.110:7004
192.168.1.109:7002
Adding replica 192.168.1.110:7005 to 192.168.1.109:7001
Adding replica 192.168.1.109:7003 to 192.168.1.110:7004
Adding replica 192.168.1.110:7006 to 192.168.1.109:7002
M: 08d5f59fa053d79e3cc71fc5bb8759bec191ff41 192.168.1.109:7001
slots:0-5460 (5461 slots) master
M: 4bda83bae0aeb9988aeb7d34eff2cc75226a0edd 192.168.1.109:7002
slots:10923-16383 (5461 slots) master
S: b782f527ead6f63c8278c1b0713b92a9fd4fe995 192.168.1.109:7003
replicates 86c8cb7975f85a859147fef9672d44b24c5a718b
M: 86c8cb7975f85a859147fef9672d44b24c5a718b 192.168.1.110:7004
slots:3671,5461-10922,11797,15924 (5465 slots) master
S: 2d44accacb1bb4cd1eef5f98f7af64a43f01544d 192.168.1.110:7005
replicates 08d5f59fa053d79e3cc71fc5bb8759bec191ff41
S: 2c42433953bcd3ad47015d9f8ac72e7a7cf447c5 192.168.1.110:7006
replicates 4bda83bae0aeb9988aeb7d34eff2cc75226a0edd
Can I set the above configuration? (type 'yes' to accept):
從運行結果看 主節點就是7001 7004 7002 從節點分別是7005 7003 7006
7001分配到的哈希槽是 0-5460
7004分配到的哈希槽是 5461-10922
7002分配到的哈希槽是 10923-16383
最后問我們是否接受上面的設置,輸入yes 就表示接受,我們輸入yes
然后顯示:
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join...
>>> Performing Cluster Check (using node 192.168.1.109:7001)
M: d06aebf92e99169864893decc371e34cd321640f 192.168.1.109:7001
slots:0-5460 (5461 slots) master
1 additional replica(s)
M: 189dc189dd743f9fc61c439d44cc3379d8cbe535 192.168.1.110:7004
slots:5461-10922 (5462 slots) master
1 additional replica(s)
S: fe2b0038304c02bc807c38fe006338bd5dd85a70 192.168.1.110:7006
slots: (0 slots) slave
replicates 25a6be3b62fa675a3835e6c62641c3784ec62257
M: 25a6be3b62fa675a3835e6c62641c3784ec62257 192.168.1.109:7002
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: df28512d63af4a3e6dc1739e1c267ba916f3a579 192.168.1.110:7005
slots: (0 slots) slave
replicates d06aebf92e99169864893decc371e34cd321640f
S: aaa3310d3c30cc9a7f235db27e3c395a5e180a52 192.168.1.109:7003
slots: (0 slots) slave
replicates 189dc189dd743f9fc61c439d44cc3379d8cbe535
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
顯示配置哈希槽,以及集群創建成功,可以用了;
第七步:集群數據測試
我們先連接任意一個節點,然后添加一個key:
redis-cli是redis默認的客戶端工具,啟動時加上`-c`參數,`-p`指定端口,就可以連接到集群。
這里還得加-h 指定機器IP
連接任意一個節點端口:
[root@localhost ~]# /usr/local/redis/bin/redis-cli -h 192.168.1.109 -c -p 7002
192.168.1.109:7002>
連接到7002節點
192.168.1.109:7002> set xxx 'fadfa'
-> Redirected to slot [4038] located at 192.168.1.109:7001
OK
前面說過Redis Cluster值分配規則,所以分配key的時候,它會使用CRC16(‘my_name’)%16384算法,來計算,將這個key 放到哪個節點,這里分配到了4038slot 就分配到了7001(0-5460)這個節點上。所以有:
Redirected to slot [4038] located at 192.168.1.109:7001
我們從其他集群節點 ,都可以獲取到數據
[root@localhost ~]# /usr/local/redis/bin/redis-cli -h 192.168.1.110 -c -p 7005
192.168.1.110:7005>
192.168.1.110:7005> get xxx
-> Redirected to slot [4038] located at 192.168.1.109:7001
"fadfa"
192.168.1.109:7001>
第八步:集群宕機測試
假如我們干掉一個節點,比如7002 這個主節點
[root@localhost ~]# ps -ef | grep redis
root 9799 1 0 21:33 ? 00:00:02 /usr/local/redis/bin/redis-server 192.168.1.109:7001 [cluster]
root 9803 1 0 21:33 ? 00:00:02 /usr/local/redis/bin/redis-server 192.168.1.109:7002 [cluster]
root 9807 1 0 21:33 ? 00:00:03 /usr/local/redis/bin/redis-server 192.168.1.109:7003 [cluster]
root 9830 2214 0 21:53 pts/0 00:00:00 grep --color=auto redis
[root@localhost ~]# kill -9 9803
[root@localhost ~]# ps -ef | grep redis
root 9799 1 0 21:33 ? 00:00:02 /usr/local/redis/bin/redis-server 192.168.1.109:7001 [cluster]
root 9807 1 0 21:33 ? 00:00:03 /usr/local/redis/bin/redis-server 192.168.1.109:7003 [cluster]
root 9832 2214 0 21:54 pts/0 00:00:00 grep --color=auto redis
[root@localhost ~]#
然后我們查看下集群情況
[root@localhost ~]# redis-trib.rb check 192.168.1.109:7001
>>> Performing Cluster Check (using node 192.168.1.109:7001)
M: d06aebf92e99169864893decc371e34cd321640f 192.168.1.109:7001
slots:0-5460 (5461 slots) master
1 additional replica(s)
M: 189dc189dd743f9fc61c439d44cc3379d8cbe535 192.168.1.110:7004
slots:5461-10922 (5462 slots) master
1 additional replica(s)
M: fe2b0038304c02bc807c38fe006338bd5dd85a70 192.168.1.110:7006
slots:10923-16383 (5461 slots) master
0 additional replica(s)
S: df28512d63af4a3e6dc1739e1c267ba916f3a579 192.168.1.110:7005
slots: (0 slots) slave
replicates d06aebf92e99169864893decc371e34cd321640f
S: aaa3310d3c30cc9a7f235db27e3c395a5e180a52 192.168.1.109:7003
slots: (0 slots) slave
replicates 189dc189dd743f9fc61c439d44cc3379d8cbe535
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
我們發現 7006本來是從節點,由於他對應的主節點掛了,就自動變成主節點master,所有會有最后一個說明
All 16384 slots covered. 所有哈希槽都可覆蓋了; 集群可以正常使用;
假如我們把7006也干掉,試試看
[root@localhost ~]# ps -ef | grep redis
root 2578 1 0 21:33 ? 00:00:03 /usr/local/redis/bin/redis-server 192.168.1.110:7004 [cluster]
root 2582 1 0 21:33 ? 00:00:03 /usr/local/redis/bin/redis-server 192.168.1.110:7005 [cluster]
root 2586 1 0 21:33 ? 00:00:03 /usr/local/redis/bin/redis-server 192.168.1.110:7006 [cluster]
root 2623 2270 0 21:56 pts/0 00:00:00 grep --color=auto redis
[root@localhost ~]# kill -9 2586
[root@localhost ~]# ps -ef | grep redis
root 2578 1 0 21:33 ? 00:00:03 /usr/local/redis/bin/redis-server 192.168.1.110:7004 [cluster]
root 2582 1 0 21:33 ? 00:00:03 /usr/local/redis/bin/redis-server 192.168.1.110:7005 [cluster]
root 2625 2270 0 21:56 pts/0 00:00:00 grep --color=auto redis
[root@localhost ~]#
[root@localhost ~]# redis-trib.rb check 192.168.1.109:7001
這里我們發現 出事了,因為主從節點都掛了 所以有一部分哈希槽沒得分配,最后一句
[ERR] Not all 16384 slots are covered by nodes. 沒有安全覆蓋;
所以不能正常使用集群