centos8平台redis cluster集群搭建(redis5.0.7)

一，規划 redis cluster

1,cluster采用六台redis，3主3從

 

redis1    ： ip: 172.17.0.2

redis2    ： ip: 172.17.0.3

redis3    ： ip: 172.17.0.4

redis4    ： ip: 172.17.0.5

redis5    ： ip: 172.17.0.6

redis6    ： ip: 172.17.0.7

 

2,為什么需要6個節點？

redis主從集群最少需要6個節點
master節點至少要3個，slave節點也是3個，

因為一個redis集群要對外提供可用的服務，那么集群中必須要有過半的master節點正常工作。

所以如果想搭建一個能夠允許 n 個master節點掛掉的集群，那么就要搭建2n+1個master節點的集群。

 

2個節點，一個宕掉，剩下的1個不超過1半(1)，集群停止工作

3個節點，一個宕掉，剩下的兩個超過1半(1.5)，集群繼續工作

4個節點，一個宕掉，剩下的3個超過1半（2個），集群繼續工作

4個節點，兩個宕掉，剩下的2個不超過1半（2個），集群停止工作

3個節點和4個節點，都是只允許一個節點宕掉，

可見3個是最實際的選擇

 

說明：劉宏締的架構森林是一個專注架構的博客，地址：https://www.cnblogs.com/architectforest

         對應的源碼可以訪問這里獲取： https://github.com/liuhongdi/

 說明：作者:劉宏締 郵箱: [e-mail]

 

二，在每台機器上安裝redis,操作相同

1，安裝wget

[root@redis source]# yum install wget

2,安裝gcc

[root@redis source]# yum install gcc

3,安裝tcl

[root@redis source]# yum install tcl

4,安裝make

[root@redis redis-5.0.7]# yum install make

5,下載redis

[root@redis source]# wget http://download.redis.io/releases/redis-5.0.7.tar.gz

6,解壓redis

[root@redis source]# tar -zxvf redis-5.0.7.tar.gz 

7,編譯:

[root@redis source]# cd redis-5.0.7
[root@redis redis-5.0.7]# make MALLOC=libc

8,測試編譯效果:

[root@redis redis-5.0.7]# make test

遇到報錯：

You need tcl 8.5 or newer in order to run the Redis test

解決：我們已安裝了tcl

make test找不到它是因為缺少which命令，

[root@redis redis-5.0.7]# yum install which

 

9，再來一次:

[root@redis redis-5.0.7]# make clean
[root@redis redis-5.0.7]# make MALLOC=libc
[root@redis redis-5.0.7]# make test

最終看到以下提示則表示編譯無問題

\o/ All tests passed without errors!

 

10,安裝

生成目錄

[root@redis soft]# mkdir /usr/local/soft/redis5
[root@redis soft]# mkdir /usr/local/soft/redis5/bin
[root@redis soft]# mkdir /usr/local/soft/redis5/conf

復制文件

[root@redis soft]# cp /usr/local/source/redis-5.0.7/src/redis-cli /usr/local/soft/redis5/bin/
[root@redis soft]# cp /usr/local/source/redis-5.0.7/src/redis-server /usr/local/soft/redis5/bin/
[root@redis soft]# cp /usr/local/source/redis-5.0.7/redis.conf /usr/local/soft/redis5/conf/

 

三，在每台機器上配置redis,注意有區別的地方:

1，生成目錄:

[root@redis soft]# mkdir /data/
[root@redis soft]# mkdir /data/redis
[root@redis soft]# mkdir /data/redis/data
[root@redis soft]# mkdir /data/redis/log
[root@redis soft]# mkdir /data/redis/cluster

說明：data存放rdb或aof

log存放日志

cluster存放cluster的配置文件nodes.conf

2,配置redis.conf

配置以下各項

#數據文件存儲目錄
dir /data/redis/data/

#日志級別
loglevel notice(使用這個默認值，不用變)

#日志文件
logfile "/data/redis/log/redis.log"

#是否以服務方式運行
daemonize yes

#pid文件
pidfile /var/run/redis_6379.pid(使用這個默認值，不用變)

#是否以cluster方式運行
cluster-enabled yes(此行取消注釋即可)

#cluster的配置緩存文件
cluster-config-file /data/redis/cluster/nodes-6379.conf

#連接node的超時時間
cluster-node-timeout 15000(此行取消注釋即可)

#訪問每台redis的密碼
requirepass lhd123

#slave訪問master的密碼,注意與上一個相同
masterauth lhd123

#是否啟用保護模式
protected-mode no

#綁定ip:注意要改成自己當前機器的可訪問的ip
#如果是127.0.0.1,則只能從本機訪問
bind 172.17.0.2

說明：這里要注意的一點：

綁定ip:注意要改成自己當前機器的可訪問的ip

 

四，給每台機器上的redis生成systemd啟動文件,操作相同

1,生成service文件

[root@redis conf]# vi /lib/systemd/system/redis.service

內容:

[Unit]
Description=Redis
After=network.target

[Service]
Type=forking
PIDFile=/var/run/redis_6379.pid
ExecStart=/usr/local/soft/redis5/bin/redis-server /usr/local/soft/redis5/conf/redis.conf
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s QUIT $MAINPID
PrivateTmp=true

[Install]
WantedBy=multi-user.target

重載systemd服務

[root@redis conf]# systemctl daemon-reload

啟動redis服務

[root@redis conf]# systemctl start redis

測試：

[root@redis conf]# /usr/local/soft/redis5/bin/redis-cli 
127.0.0.1:6379> auth lhd123
OK
127.0.0.1:6379> keys *
(empty list or set)
127.0.0.1:6379> set a aaa
(error) CLUSTERDOWN Hash slot not served

系統提示hash slot還沒分配,因為cluster還未創建

 

五，創建cluster,測試寫入,(在任一台機器上操作)

1，創建cluster,在上面創建的任一台機器上操作即可

命令:

[root@redis1 /]# /usr/local/soft/redis5/bin/redis-cli -a lhd123 --cluster create 172.17.0.2:6379 172.17.0.3:6379 
172.17.0.4:6379 172.17.0.5:6379 172.17.0.6:6379 172.17.0.7:6379 --cluster-replicas 1

 

在需要回答下面問題時：輸入yes

Can I set the above configuration? (type 'yes' to accept): yes

 

看一個例子：

#-a: auth 訪問密碼

#--cluster create :用來創建集群

#--cluster-replicas :為集群中的每個主節點創建一個從節點

[root@redis1 conf]# /usr/local/soft/redis5/bin/redis-cli -a lhd123 --cluster create 172.17.0.2:6379 172.17.0.3:6379 
172.17.0.4:6379 172.17.0.5:6379 172.17.0.6:6379 172.17.0.7:6379 --cluster-replicas 1

Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 172.17.0.6:6379 to 172.17.0.2:6379
Adding replica 172.17.0.7:6379 to 172.17.0.3:6379
Adding replica 172.17.0.5:6379 to 172.17.0.4:6379
M: eb701616b26b5a350605ae3ea5f80e4fc79d84c3 172.17.0.2:6379
   slots:[0-5460] (5461 slots) master
M: 5bdaafe57b1c46f61c5910d3822633a516feb4ae 172.17.0.3:6379
   slots:[5461-10922] (5462 slots) master
M: e024e898a21f3f4051abfb0957046dc4a81ef947 172.17.0.4:6379
   slots:[10923-16383] (5461 slots) master
S: 1ca00d6a680fc1b0d617a46996eaaefc3636fd5a 172.17.0.5:6379
   replicates e024e898a21f3f4051abfb0957046dc4a81ef947
S: 9cd94c491211542dbfae96002489c9b63a5a54e7 172.17.0.6:6379
   replicates eb701616b26b5a350605ae3ea5f80e4fc79d84c3
S: 6f90338cef5af2aa9f0580cd660c80d2ea5fab82 172.17.0.7:6379
   replicates 5bdaafe57b1c46f61c5910d3822633a516feb4ae
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
...
>>> Performing Cluster Check (using node 172.17.0.2:6379)
M: eb701616b26b5a350605ae3ea5f80e4fc79d84c3 172.17.0.2:6379
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: 1ca00d6a680fc1b0d617a46996eaaefc3636fd5a 172.17.0.5:6379
   slots: (0 slots) slave
   replicates e024e898a21f3f4051abfb0957046dc4a81ef947
M: 5bdaafe57b1c46f61c5910d3822633a516feb4ae 172.17.0.3:6379
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
M: e024e898a21f3f4051abfb0957046dc4a81ef947 172.17.0.4:6379
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: 9cd94c491211542dbfae96002489c9b63a5a54e7 172.17.0.6:6379
   slots: (0 slots) slave
   replicates eb701616b26b5a350605ae3ea5f80e4fc79d84c3
S: 6f90338cef5af2aa9f0580cd660c80d2ea5fab82 172.17.0.7:6379
   slots: (0 slots) slave
   replicates 5bdaafe57b1c46f61c5910d3822633a516feb4ae
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

 

2,測試寫入:

# -c:  Enable cluster mode,用集群模式訪問

# -h: Server hostname (default: 127.0.0.1),指定要訪問的主機

[root@redis1 conf]# /usr/local/soft/redis5/bin/redis-cli -a lhd123 -c -h 172.17.0.2
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
172.17.0.2:6379> set c bb
-> Redirected to slot [7365] located at 172.17.0.3:6379
OK
172.17.0.3:6379> set d bb
-> Redirected to slot [11298] located at 172.17.0.4:6379
OK
172.17.0.4:6379> get b
-> Redirected to slot [3300] located at 172.17.0.2:6379
"bb"
172.17.0.2:6379> get c
-> Redirected to slot [7365] located at 172.17.0.3:6379
"bb"
172.17.0.3:6379> get d
-> Redirected to slot [11298] located at 172.17.0.4:6379
"bb"

 

六，查看cluster信息(在任一台機器上操作)

1，查看節點列表：

以集群方式登錄redis-cli后查看

172.17.0.3:6379> CLUSTER NODES
9cd94c491211542dbfae96002489c9b63a5a54e7 172.17.0.6:6379@16379 slave eb701616b26b5a350605ae3ea5f80e4fc79d84c3 0 1586861138000 1 connected
5bdaafe57b1c46f61c5910d3822633a516feb4ae 172.17.0.3:6379@16379 myself,master - 0 1586861135000 2 connected 5461-10922
6f90338cef5af2aa9f0580cd660c80d2ea5fab82 172.17.0.7:6379@16379 slave 5bdaafe57b1c46f61c5910d3822633a516feb4ae 0 1586861137000 6 connected
e024e898a21f3f4051abfb0957046dc4a81ef947 172.17.0.4:6379@16379 master - 0 1586861136668 3 connected 10923-16383
1ca00d6a680fc1b0d617a46996eaaefc3636fd5a 172.17.0.5:6379@16379 slave e024e898a21f3f4051abfb0957046dc4a81ef947 0 1586861138674 3 connected
eb701616b26b5a350605ae3ea5f80e4fc79d84c3 172.17.0.2:6379@16379 master - 0 1586861136000 1 connected 0-5460

用 check命令查看

#--cluster check: 列出群集中的機器信息

[root@redis2 /]# /usr/local/soft/redis5/bin/redis-cli --cluster check -a lhd123 172.17.0.3:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
172.17.0.3:6379 (5bdaafe5...) -> 2 keys | 5462 slots | 1 slaves.
172.17.0.4:6379 (e024e898...) -> 3 keys | 5461 slots | 1 slaves.
172.17.0.2:6379 (eb701616...) -> 2 keys | 5461 slots | 1 slaves.
[OK] 7 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 172.17.0.3:6379)
M: 5bdaafe57b1c46f61c5910d3822633a516feb4ae 172.17.0.3:6379
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: 9cd94c491211542dbfae96002489c9b63a5a54e7 172.17.0.6:6379
   slots: (0 slots) slave
   replicates eb701616b26b5a350605ae3ea5f80e4fc79d84c3
S: 6f90338cef5af2aa9f0580cd660c80d2ea5fab82 172.17.0.7:6379
   slots: (0 slots) slave
   replicates 5bdaafe57b1c46f61c5910d3822633a516feb4ae
M: e024e898a21f3f4051abfb0957046dc4a81ef947 172.17.0.4:6379
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: 1ca00d6a680fc1b0d617a46996eaaefc3636fd5a 172.17.0.5:6379
   slots: (0 slots) slave
   replicates e024e898a21f3f4051abfb0957046dc4a81ef947
M: eb701616b26b5a350605ae3ea5f80e4fc79d84c3 172.17.0.2:6379
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

 

2,查看節點的信息:

使用CLUSTER INFO命令

#CLUSTER INFO:cluster的統計信息

172.17.0.3:6379> CLUSTER INFO
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:2
cluster_stats_messages_ping_sent:2071
cluster_stats_messages_pong_sent:1919
cluster_stats_messages_meet_sent:1
cluster_stats_messages_sent:3991
cluster_stats_messages_ping_received:1915
cluster_stats_messages_pong_received:2072
cluster_stats_messages_meet_received:4
cluster_stats_messages_received:3991

 

七，創建redis cluster時遇到的報錯或問題：

1,創建cluster時，停在Waiting for the cluster to join不動了

1，16379這個集群管理端口被防火牆做了攔截

2,   docker環境中,nodes-6379.conf 文件中redis節點id如果相同也會有這個問題

刪除文件后重啟

例:

[root@redis1 conf]# rm /data/redis/cluster/nodes-6379.conf
rm: remove regular file '/data/redis/cluster/nodes-6379.conf'? y

 

2,客戶端寫入時報錯：

(error) MOVED 

例子：

172.17.0.2:6379> set a aaa
(error) MOVED 15495 172.17.0.4:6379

解決：

因為啟動redis-cli時沒有設置集群模式所導致

[root@redis1 conf]# /usr/local/soft/redis5/bin/redis-cli -c -h 172.17.0.2

 

3，客戶端寫入時報錯：

(error) NOAUTH Authentication required.

例子：

[root@redis1 conf]# /usr/local/soft/redis5/bin/redis-cli -c -h 172.17.0.2
172.17.0.2:6379> auth lhd123
OK
172.17.0.2:6379> set a aaaa
-> Redirected to slot [15495] located at 172.17.0.4:6379
(error) NOAUTH Authentication required.
172.17.0.4:6379> auth "lhd123"
OK
172.17.0.4:6379> set a aaaa
OK

解決：

[root@redis1 conf]# /usr/local/soft/redis5/bin/redis-cli -a lhd123 -c -h 172.17.0.2

 

八，查看redis版本

[root@redis1 /]# /usr/local/soft/redis5/bin/redis-server --version
Redis server v=5.0.7 sha=00000000:0 malloc=libc bits=64 build=c52ab39fadfc446c

 

九，查看centos版本

[root@redis1 /]# cat /etc/redhat-release 
CentOS Linux release 8.1.1911 (Core)

 

十，針對redis cluster中node的管理，請移步這一篇:

https://www.cnblogs.com/architectforest/p/12714889.html