分布式锁
-
什么是分布式锁？
1）分布式锁是控制分布式系统中或不同系统之间共同访问共享资源的一种锁实现。 2）如果不同系统或同一系统的不同主机之间共享了某个资源时，往往通过互斥来防止彼此之间的干扰。 3）不会发生死锁，即使一个server在持有锁时间出现问题没能主动解锁，也能保证后续其他server能正常加锁。 -
分布式锁的目的？
可以保证分布式部署的应用集群中，同一个资源在同一时刻只能被一台机器上的一个线程执行。 -
Redis分布式锁可能出现的问题？
在setnx和setex中间发生了服务宕机 解决方案：1）该问题需要setnx和setex连用，可以采用lua脚本 2）Redis从2.6之后支持setnx、setex连用 当server1执行任务时间大于setex设置的过期时间时，可能在解锁时解了server2的锁 解决方案：每个server解锁时需要判断当前要remove掉的锁的value是不是该server的value，再去解锁，可采用lua脚本做解锁流程
主从节点搭建
-
在redis目录下创建redis-replication目录
mkdir redis-replication -
在redis-replication目录下创建目录6380 6381
mkdir 6380 mkdir 6381 -
将src目录下的redis-server拷贝到redis-replication目录下
[root@bogon redis-replication]# cp ../src/redis-server redis-server -
将redis.conf分别拷贝到6380 6381,目录下
[root@bogon redis-replication]# cp ../redis.conf 6380 [root@bogon redis-replication]# cp ../redis.conf 6381 -
修改6380和6381中redis.conf的配置
daemonize yes #守护进程模式开启 port 6380 slaveof 127.0.0.1 6379 #模拟环境三个redis都在同一台机器上daemonize yes #守护进程模式开启 port 6381 slaveof 127.0.0.1 6379 #模拟环境三个redis都在同一台机器上 -
分别启动三个节点服务
[root@bogon redis-replication]# ./redis-server ../redis.conf [root@bogon redis-replication]# ./redis-server 6380/redis.conf [root@bogon redis-replication]# ./redis-server 6381/redis.conf -
在6379节点redis-cli查看info，并set，然后分别在两个从节点get，get到即集群搭建完毕
[root@bogon /]# redis-cli 127.0.0.1:6379> info replication # Replication role:master connected_slaves:2 slave0:ip=127.0.0.1,port=6380,state=online,offset=1639,lag=0 slave1:ip=127.0.0.1,port=6381,state=online,offset=1639,lag=0 master_replid:5f13eaede714511b727b6058ed7299c317c66d8b master_replid2:0000000000000000000000000000000000000000 master_repl_offset:1639 second_repl_offset:-1 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:1 repl_backlog_histlen:1639 127.0.0.1:6379>
主从复制的原理
**全量复制**
实现原理：建立主从关系时，从机会给主机发送sync命令，主机接收命令，后台启动的存盘进程，同时收集所有用于修改命令，传送给从机。
**增量复制**
实现原理：主机会继续将新收集到的修改命令依次传给从机，实现数据的同步效果
主从复制的缺点
Redis的主从复制最大的缺点就是延迟，主机负责写，从机负责备份，这个过程有一定的延迟，当系统很繁忙的时候，延迟问题会更加严重，从机器数量的增加也会使这个问题更加严重
故障转移
-
将redis-5.0.5-1目录下sentinel.conf拷贝到redis-replication
[root@bogon redis-5.0.5-1]# cp sentinel.conf redis-replication/ -
将sentinel.conf重命名为sentinel_1.conf并修改配置
sentinel monitor mymaster 127.0.0.1 6379 1 #指定master节点，1代表1个sentinel认为某节点挂掉就挂掉了 sentinel down-after-milliseconds mymaster 10000 #指定sentinel认为服务器断线所需毫秒数 sentinel failover-timeout mymaster 60000 #执行故障转移确认时间毫秒数 sentinel parallel-syncs mymaster 1 #执行故障转移时，最大可以有多少个从服务器同时对新的主服务器进行同步，这个数字越小，完成故障转移所需时间就越长 -
保存后启动redis-sentinel
[root@bogon redis-5.0.5-1]# ./src/redis-sentinel redis-replication/sentinel_1.conf 19559:X 04 Nov 2019 18:45:56.540 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo 19559:X 04 Nov 2019 18:45:56.540 # Redis version=5.0.5, bits=64, commit=00000000, modified=0, pid=19559, just started 19559:X 04 Nov 2019 18:45:56.540 # Configuration loaded 19559:X 04 Nov 2019 18:45:56.541 * Increased maximum number of open files to 10032 (it was originally set to 1024). _._ _.-``__ ''-._ _.-`` `. `_. ''-._ Redis 5.0.5 (00000000/0) 64 bit .-`` .-```. ```\/ _.,_ ''-._ ( ' , .-` | `, ) Running in sentinel mode |`-._`-...-` __...-.``-._|'` _.-'| Port: 26379 | `-._ `._ / _.-' | PID: 19559 `-._ `-._ `-./ _.-' _.-' |`-._`-._ `-.__.-' _.-'_.-'| | `-._`-._ _.-'_.-' | http://redis.io `-._ `-._`-.__.-'_.-' _.-' |`-._`-._ `-.__.-' _.-'_.-'| | `-._`-._ _.-'_.-' | `-._ `-._`-.__.-'_.-' _.-' `-._ `-.__.-' _.-' `-._ _.-' `-.__.-' 19559:X 04 Nov 2019 18:45:56.542 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128. 19559:X 04 Nov 2019 18:45:56.543 # Sentinel ID is a3bbbce34899d8534d98f7328ba9d61857a94168 19559:X 04 Nov 2019 18:45:56.543 # +monitor master mymaster 127.0.0.1 6379 quorum 1 19559:X 04 Nov 2019 18:45:56.545 * +slave slave 127.0.0.1:6380 127.0.0.1 6380 @ mymaster 127.0.0.1 6379 19559:X 04 Nov 2019 18:45:56.545 * +slave slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6379 -
然后shutdown主节点6379，可以看到下面日志，将6381选举为了新的主节点，当6379再次启动时，会作为6381的从节点
./redis-cli -p 6379 shutdown 19559:X 04 Nov 2019 18:47:24.078 # +sdown master mymaster 127.0.0.1 6379 19559:X 04 Nov 2019 18:47:24.078 # +odown master mymaster 127.0.0.1 6379 #quorum 1/1 19559:X 04 Nov 2019 18:47:24.078 # +new-epoch 1 19559:X 04 Nov 2019 18:47:24.078 # +try-failover master mymaster 127.0.0.1 6379 19559:X 04 Nov 2019 18:47:24.079 # +vote-for-leader a3bbbce34899d8534d98f7328ba9d61857a94168 1 19559:X 04 Nov 2019 18:47:24.079 # +elected-leader master mymaster 127.0.0.1 6379 19559:X 04 Nov 2019 18:47:24.079 # +failover-state-select-slave master mymaster 127.0.0.1 6379 19559:X 04 Nov 2019 18:47:24.133 # +selected-slave slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6379 19559:X 04 Nov 2019 18:47:24.133 * +failover-state-send-slaveof-noone slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6379 19559:X 04 Nov 2019 18:47:24.191 * +failover-state-wait-promotion slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6379 19559:X 04 Nov 2019 18:47:25.120 # +promoted-slave slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6379 19559:X 04 Nov 2019 18:47:25.120 # +failover-state-reconf-slaves master mymaster 127.0.0.1 6379 19559:X 04 Nov 2019 18:47:25.172 * +slave-reconf-sent slave 127.0.0.1:6380 127.0.0.1 6380 @ mymaster 127.0.0.1 6379 19559:X 04 Nov 2019 18:47:26.128 * +slave-reconf-inprog slave 127.0.0.1:6380 127.0.0.1 6380 @ mymaster 127.0.0.1 6379 19559:X 04 Nov 2019 18:47:26.128 * +slave-reconf-done slave 127.0.0.1:6380 127.0.0.1 6380 @ mymaster 127.0.0.1 6379 19559:X 04 Nov 2019 18:47:26.222 # +failover-end master mymaster 127.0.0.1 6379 19559:X 04 Nov 2019 18:47:26.222 # +switch-master mymaster 127.0.0.1 6379 127.0.0.1 6381 19559:X 04 Nov 2019 18:47:26.222 * +slave slave 127.0.0.1:6380 127.0.0.1 6380 @ mymaster 127.0.0.1 6381 19559:X 04 Nov 2019 18:47:26.222 * +slave slave 127.0.0.1:6379 127.0.0.1 6379 @ mymaster 127.0.0.1 6381
高可用Sentinel故障转移原理
-
Sentinel是如何工作的呢？
主观下线：单个Sentinel实例对服务器做出下线判断 主观下线特点：如果一个服务器没有在master-down-after-milliseconds所指时间内，对发送PING命令的Sentinel得到一个有效回复，那么将会标记这个服务器为主观下线。 客观下线：多个Sentinel投票对一个服务做出判断。 客观下线条件只适用于主节点，主观下线适用于从节点。
SpringBoot整合Sentinel
-
直接在yml文件加入配置即可
spring: redis: sentinel: master: mymaster nodes: 192.168.47.129:26379 #多个sentinel以逗号分隔 -
启动后set
redisTemplate.opsForValue().set(key, value); return redisTemplate.opsForValue().get(key); -
可能会遇到问题
io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: no further information: /127.0.0.1:6379 at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:327) at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:340) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:670) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:617) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:534) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496) at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:906) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) -
导致问题的原因是Sentinel.conf中的配置
sentinel monitor mymaster 127.0.0.1 6379 1 改成 sentinel monitor mymaster 192.168.47.129 6379 1 -
问题解决
集群搭建（三主三从）
-
在local目录下创建redis-cluster文件夹，在redis-cluster下建6个节点分别拷入redis.conf
cd /usr/local/ mkdir redis_cluster cd redis_cluster mkdir 7000 7001 7002 7003 7004 7005 cp /usr/local/redis/redis.conf /usr/local/redis/7000 -
修改6个节点的redis.conf
daemonize yes //redis后台运行 port 7000 //端口7000,7001,7002,7003,7004,7005 cluster-enabled yes //开启集群 把注释#去掉 cluster-config-file nodes-xxx.conf //集群的配置 配置文件首次启动自动生成 cluster-node-timeout 5000 //请求超时 设置5秒够了 appendonly yes //aof日志开启 有需要就开启，它会每次写操作都记录一条日志 bind 127.0.0.1 192.168.80.129(此处为自己内网的ip地址，centos7下面采用ip addr来查看，其他系统试一下ifconfig查看) -
启动所有节点
cd /usr/local/redis_cluster/7000 ../redis-server ./redis.conf cd /usr/local/redis-cluster/7001 ../redis-server ./redis.conf cd /usr/local/redis-cluster/7002 ../redis-server ./redis.conf cd /usr/local/redis-cluster/7003 ../redis-server ./redis.conf cd /usr/local/redis-cluster/7004 ../redis-server ./redis.conf cd /usr/local/redis-cluster/7005 ../redis-server ./redis.conf -
注意：Redis 5.X前的版本安装ruby。Redis 5.X版本请看下面第8点
yum -y install ruby ruby-devel rubygems rpm-build gem install redis -
gem install redis可能会遇到如下错误
ERROR: Error installing redis: redis requires Ruby version >= 2.3.0. CentOS7 库中的支持到2.0.0，可ruby安装Redis（我用的redis版本是5.0.5）的需要最低是2.3.0 -
解决办法
# 下载 curl -L get.rvm.io | bash -s stable cd /usr/local/rvm/archives #解压 tar xvzf rvm-1.29.9.tgz cd rvm-1.29.9 ./install source /usr/local/rvm/scripts/rvm #查看所有版本 rvm list known #安装2.3.3 rvm install 2.3.3 rvm use 2.3.3 ruby --version gem install redis -
Redis 5.X前的版本，进入redis的src目录，执行如下命令
./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 -
Redis 5.X版本，进入redis的src目录，执行如下命令
./redis-cli --cluster create 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 --cluster-replicas 1 -
出现下面信息表示启动成功
>>> Performing hash slots allocation on 6 nodes... Master[0] -> Slots 0 - 5460 Master[1] -> Slots 5461 - 10922 Master[2] -> Slots 10923 - 16383 Adding replica 127.0.0.1:7004 to 127.0.0.1:7000 Adding replica 127.0.0.1:7005 to 127.0.0.1:7001 Adding replica 127.0.0.1:7003 to 127.0.0.1:7002 >>> Trying to optimize slaves allocation for anti-affinity [WARNING] Some slaves are in the same host as their master M: ebb3d01bd2578d5b38400f5330ba7c9a5bdaefba 127.0.0.1:7000 slots:[0-5460] (5461 slots) master M: 842bbbcbfd9c7b98067a8cf8cff09cb54817c8cc 127.0.0.1:7001 slots:[5461-10922] (5462 slots) master M: 815fe658394702b8e38498918510aa9ea75ff73d 127.0.0.1:7002 slots:[10923-16383] (5461 slots) master S: 55784ce9df135a51efeba908643018b13e33d11a 127.0.0.1:7003 replicates ebb3d01bd2578d5b38400f5330ba7c9a5bdaefba S: b630b0dae92977f1447623ad34981a1086bfdbfa 127.0.0.1:7004 replicates 842bbbcbfd9c7b98067a8cf8cff09cb54817c8cc S: 7a383e5d8a3b1c22c7b47c6ce9f92f820f44f9d5 127.0.0.1:7005 replicates 815fe658394702b8e38498918510aa9ea75ff73d Can I set the above configuration? (type 'yes' to accept): yes -
Can I set the above configuration? (type 'yes' to accept): yes (是否这样配置？ 是)
>>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join ... >>> Performing Cluster Check (using node 127.0.0.1:7000) M: ebb3d01bd2578d5b38400f5330ba7c9a5bdaefba 127.0.0.1:7000 slots:[0-5460] (5461 slots) master 1 additional replica(s) S: 7a383e5d8a3b1c22c7b47c6ce9f92f820f44f9d5 127.0.0.1:7005 slots: (0 slots) slave replicates 815fe658394702b8e38498918510aa9ea75ff73d M: 815fe658394702b8e38498918510aa9ea75ff73d 127.0.0.1:7002 slots:[10923-16383] (5461 slots) master 1 additional replica(s) S: b630b0dae92977f1447623ad34981a1086bfdbfa 127.0.0.1:7004 slots: (0 slots) slave replicates 842bbbcbfd9c7b98067a8cf8cff09cb54817c8cc M: 842bbbcbfd9c7b98067a8cf8cff09cb54817c8cc 127.0.0.1:7001 slots:[5461-10922] (5462 slots) master 1 additional replica(s) S: 55784ce9df135a51efeba908643018b13e33d11a 127.0.0.1:7003 slots: (0 slots) slave replicates ebb3d01bd2578d5b38400f5330ba7c9a5bdaefba [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. -
测试，redis-cli -c -p 7002，-c 代表连接集群结点，如下结果说明没问题
[root@bogon src]# redis-cli -c -p 7002 127.0.0.1:7002> set w x -> Redirected to slot [3696] located at 127.0.0.1:7000 OK 127.0.0.1:7000> get w "x" [root@bogon src]# redis-cli -c -p 7003 127.0.0.1:7003> get w -> Redirected to slot [3696] located at 127.0.0.1:7000 "x" 127.0.0.1:7000>