一、實驗背景
canal有一個參數canal.instance.standby.address可以指定源端數據庫的從庫為備選數據庫，當源端master宕機后，canal仍能指向備庫進行同步。
但是發現canal 1.1.4不兼容mariadb的gtid：
當canal instance里指定了gtid(示例：canal.instance.master.gtid=0-1-146)，重啟canal instance會報錯，不識別這種gtid格式：
ERROR com.alibaba.otter.canal.common.alarm.LogAlarmHandler - destination:ff-test[java.lang.RuntimeException: parseUUIDSet failed due to wrong format: 0-1-146
- 當canal instance里指定了binlog名稱及position，canal instance不報錯，但是識別不到gtid：
2021-04-22 17:17:42.168 [destination = ff-test , address = /192.168.144.246:3306 , EventParser] WARN c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - ---> find start position successfully, EntryPosition[included=false,journalName=slave-bin.000009,position=4585,serverId=1,gtid=,timestamp=1619016171000] cost : 238ms , the next step is binlog dump
- 當主庫宕機，發生主從切換后，canal由於識別不到gtid，因此按時間戳找位置:
2021-04-22 17:42:37.002 [destination = ff-test , address = /192.168.144.245:3306 , EventParser] WARN c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - ---> begin to find start position, it will be long time for reset or first position
2021-04-22 17:42:37.094 [destination = ff-test , address = /192.168.144.245:3306 , EventParser] WARN c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - prepare to find start position by switch ::1619083861000
2021-04-22 17:42:41.910 [destination = ff-test , address = /192.168.144.245:3306 , EventParser] WARN c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - ---> find start position successfully, EntryPosition[included=false,journalName=master-bin.000013,position=7080,serverId=1,gtid=,timestamp=1619021230000] cost : 4400ms , the next step is binlog dump
按時間戳找的位置是不准確的，經過多次驗證，發現找的位置靠前，因此會重復執行一些SQL，造成數據混亂。
Canal 1.1.5版本修復了mariadb gtid問題，現記錄下實驗過程。
二、實驗步驟
2.1 配置canal server
## detecing config
canal.instance.detecting.enable = true
#canal.instance.detecting.sql = insert into retl.xdual values(1,now()) on duplicate key update x=now()
canal.instance.detecting.sql = select 1
canal.instance.detecting.interval.time = 30
canal.instance.detecting.retry.threshold = 4
canal.instance.detecting.heartbeatHaEnable = true
canal.instance.tsdb.enable = false
2.2 配置canal instance
canal.instance.mysql.slaveId=5 #不要和集群里現有的server_id重復
canal.instance.gtidon=true
canal.instance.master.address=192.168.144.245:3306 #源端ip:端口
canal.instance.tsdb.enable=false
canal.instance.standby.address =192.168.144.246:3306 #備庫ip:端口
canal.instance.master.gtid=0-1-165 #可在主庫上執行SELECT @@gtid_binlog_pos;查詢
canal.instance.dbUsername=canal
canal.instance.dbPassword=canal
canal.instance.filter.regex=hh.* #同步的庫表
啟動canal instance后，可看到識別到gtid了：
2021-04-22 17:57:46.687 [destination = hh-test , address = /192.168.144.245:3306 , EventParser] WARN c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - ---> begin to find start position, it will be long time for reset or first position
2021-04-22 17:57:46.724 [destination = hh-test , address = /192.168.144.245:3306 , EventParser] WARN c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - ---> find start position successfully, EntryPosition[included=false,journalName=,position=<null>,serverId=<null>,gtid=0-1-165,timestamp=<null>] cost : 1ms , the next step is binlog dump
2.3 驗證mariadb主從切換
關掉主庫，canal instance日志顯示主從切換過程：
2021-04-22 18:03:41.114 [destination = hh-test , address = /192.168.144.245:3306 , HeartBeatTimeTask] WARN c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - try to ha switch, old:192.168.144.245/192.168.144.245:3306, new:/192.168.144.246:3306
2021-04-22 18:03:41.114 [destination = hh-test , address = /192.168.144.245:3306 , HeartBeatTimeTask] ERROR com.alibaba.otter.canal.common.alarm.LogAlarmHandler - destination:hh-test[try to ha switch, old:192.168.144.245/192.168.144.245:3306, new:/192.168.144.246:3306]
2021-04-22 18:03:41.126 [destination = hh-test , address = /192.168.144.245:3306 , HeartBeatTimeTask] WARN c.a.o.canal.parse.inbound.mysql.dbsync.LogEventConvert - --> init table filter : ^hh.*$
2021-04-22 18:03:41.126 [destination = hh-test , address = /192.168.144.245:3306 , HeartBeatTimeTask] WARN c.a.o.canal.parse.inbound.mysql.dbsync.LogEventConvert - --> init table black filter :
2021-04-22 18:03:41.127 [destination = hh-test , address = /192.168.144.245:3306 , HeartBeatTimeTask] WARN c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - connect failed by
java.io.IOException: connect /192.168.144.245:3306 failure
……
2021-04-22 18:03:41.183 [destination = hh-test , address = /192.168.144.246:3306 , EventParser] WARN c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - ---> begin to find start position, it will be long time for reset or first position
2021-04-22 18:03:41.184 [destination = hh-test , address = /192.168.144.246:3306 , EventParser] WARN c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - ---> find start position successfully, EntryPosition[included=false,journalName=master-bin.000015,position=4288,serverId=1,gtid=0-1-171,timestamp=1619085735000] cost : 0ms , the next step is binlog dump
經測試，找的位置是准確的。
但是canal 1.1.5目前發現有一點不足的地方，canal server重啟后，canal client不會自動連canal server，canal client停止往目標庫同步，一直在報錯：
AdapterProcessor - com.alibaba.otter.canal.protocol.exception.CanalClientException: java.io.IOException: Broken pipe Error sync but ACK!
需要重啟canal client才行。
不知是我哪里沒配對，還是canal 1.1.5目前還不完善。
————————————————
版權聲明：本文為CSDN博主「雅冰石」的原創文章，遵循CC 4.0 BY-SA版權協議，轉載請附上原文出處鏈接及本聲明。
原文鏈接：https://blog.csdn.net/yabingshi_tech/article/details/116002271