前言
前面利用《Docker-Compose搭建Redis高可用哨兵集群》,
我們的思路是將Redis、Sentinel、Redis Client App鏈接到同一個網橋網絡,這個網橋內的Redis Client App就可以使用ContainerIP訪問網橋內任意redis節點。
同一網橋網絡訪問
規避了Docker上發生的NAT,端口映射的復雜性, 但實際上並不是最常規的做法(今天咱們也不說Docker host形式部署Redis-Sentinel了)。
Redis Client App獨立組網遇到的問題
很多時候,Redis-Sentinel與Redis Client App獨立組網,涉及Docker宿主機NAT轉換和 Port映射。
Sentinel,Docker或其他形式的網絡地址轉換或端口映射應謹慎混合。
我實際測試發現,如果將前文Sentinel.conf中Master(ContainerIP,Port) 換成(宿主機IP,映射Port),
確實會引起混亂,無法找到正確的Slaves, 無法正常故障轉移。
為了解決Redis-Sentinel在Docker環境下因為NAT,Forward Port導致的無法正確獲知Slaves和正確故障轉移的問題。
Redis3.2之后可以強制讓Slave聲明自己的(IP,Port);強制讓Sentinel聲明自己的(IP,Port)
# since Redis 3.2.2, to force a replica to announce an arbitrary pair of IP and port to the master. The two configurations directives to use are:
replica-announce-ip <ip>
replica-announce-port <port>
上述配置可以寫在Docker Command參數指定或通過Volume redis.conf 加載進redis容器
# you can use the following two Sentinel configuration directives in order to force Sentinel to announce a specific set of IP and port:
sentinel announce-ip <ip>
sentinel announce-port <port>
sentinel.conf的配置只能通過Config加載進sentinel容器。
通過明牌方式通知 所有交互對象,redis實例就是在這個(IP,Port)上發生了NAT轉化,Port映射,上述搭建Docker搭建Redis-sentinel才是常規實踐。
C#兩大客戶端訪問Redis-Sentinel的方式
歸根到底一張圖:
- Redis Client先詢問Sentinels,Sentinel返回Master (IP,Port)
- Redis Client再與以上Master (IP,Port)建立連接
Redis-Sentinel
這里我們采用Docker-compose在單機上部署了Redis-Sentinel集群,
1Master- 2 Slave- 3Sentinel,
分別占據宿主機6380、6381、6382、 26379、26380、26381端口.
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
484da8d832f1 redis "docker-entrypoint.s…" 2 hours ago Up 2 hours 6379/tcp, 0.0.0.0:26380->26379/tcp redis-sentinel-2
50599c15adba redis "docker-entrypoint.s…" 2 hours ago Up 2 hours 6379/tcp, 0.0.0.0:26379->26379/tcp redis-sentinel-1
51ce90cc52d7 redis "docker-entrypoint.s…" 2 hours ago Up 2 hours 6379/tcp, 0.0.0.0:26381->26379/tcp redis-sentinel-3
d58d6973de28 redis "docker-entrypoint.s…" 2 hours ago Up 2 hours 0.0.0.0:6381->6379/tcp redis-slave-1
b88bd85ac109 redis "docker-entrypoint.s…" 2 hours ago Up 8 seconds 0.0.0.0:6382->6379/tcp redis-slave-2
3dc26c01a90d redis "docker-entrypoint.s…" 2 hours ago Up About an hour 0.0.0.0:6380->6379/tcp redis-master
進入任意Sentinel節點,使用sentinel master mymaster
確認集群信息
添加測試鍵值: testKey:hello Redis-sentinel!
StackExchange.Redis & CSRedisCore連接Redis哨兵
老牌StackExchange.Redis 今年才真正支持Sentinel, Github上有關Sentinel的Issue、PR歷時久遠,PR像便秘一樣最近才關閉。
https://github.com/StackExchange/StackExchange.Redis/pull/692#issuecomment-375298108
https://github.com/StackExchange/StackExchange.Redis/pull/1067
CSRedisCore得到真傳,很早就支持了,而且編程寫法更簡單,清晰。
話不多說:
using StackExchange.Redis;
using System;
namespace ConsoleApp
{
class Program
{
static void Main(string[] args)
{
var sw = new Stopwatch();
sw.Start();
UseStackExchangeRedis();
sw.Stop();
Console.WriteLine("連接+查詢測試key,耗時"+sw.ElapsedMilliseconds);
sw.Reset();
sw.Start();
UseCSRedisCore();
sw.Stop();
Console.WriteLine("連接+查詢測試key,耗時" + sw.ElapsedMilliseconds);
Console.ReadKey();
}
// StackExchange.Reids連接Redis-Sentinel
public static void UseStackExchangeRedis()
{
ConfigurationOptions sentinelOptions = new ConfigurationOptions();
sentinelOptions.EndPoints.Add("180.76.*.*", 26379);
sentinelOptions.EndPoints.Add("180.76.*.*", 26380);
sentinelOptions.EndPoints.Add("180.76.*.*", 26381);
sentinelOptions.TieBreaker = "";
sentinelOptions.CommandMap = CommandMap.Sentinel;
sentinelOptions.AbortOnConnectFail = false;
// Connect!
ConnectionMultiplexer sentinelConnection = ConnectionMultiplexer.Connect(sentinelOptions);
// Get a connection to the master
ConfigurationOptions redisServiceOptions = new ConfigurationOptions();
redisServiceOptions.ServiceName = "mymaster1"; //master名稱
redisServiceOptions.Password = "redis_pwd"; //master訪問密碼
redisServiceOptions.AbortOnConnectFail = true;
ConnectionMultiplexer masterConnection = sentinelConnection.GetSentinelMasterConnection(redisServiceOptions);
var db = masterConnection.GetDatabase();
var value= db.StringGet("testKey");
Console.WriteLine($"[Use StackExchange-Redis] The remote redis-sentinel test key value:{value}");
}
// CSRedisCore連接Redis-Sentinel
public static void UseCSRedisCore()
{
var csredis = new CSRedis.CSRedisClient("mymaster1,password=redis_pwd",
new[] { "180.76.*.*:26379", "180.76.*.*:26380", "180.76.*.*:26381" });
var value = csredis.Get("testKey");
Console.WriteLine($"[Use CSRedisCore] The remote redis-sentinel test key value:{value}");
}
}
}
執行輸出:
本文長話短說,快速介紹兩塊C#常見的Redis客戶端連接Redis哨兵集群的方式,各有千秋。
StackExchange.Redis更能體現連接的實質過程: 先查詢,再連接。
CSRedisCore 小白寫法,無感知。
Github地址:https://github.com/zaozaoniao/Redis-sentinel-with-docker-compose
總結輸入
本文記錄兩個內容:
- Redis-Sentinel在Docker環境因NAT,Forward_Port觸發的問題, 以及Redis官方給出的方案
- C#兩個Redis客戶端如何感知Redis-Sentinel的Master查詢節點