MAXWELL系列（一）-利用maxwell 解析binlog 到 redis

今天豬腳是maxwell，zendesk公司開源 https://github.com/zendesk/maxwell

先看架構，和他競爭的有 Debezium Connector for MySQL

 

 

廢話不多說，搭建目標任務 mysql的binlog 到redis （192.168.0.1 ~~~~~~~192.168.0.3）

1：下載 https://github.com/zendesk/maxwell/releases/download/v1.22.0/maxwell-1.22.0.tar.gz

2： 安裝java ，配置好java環境變量 ，解壓maxwell-1.22.0.tar.gz（因為是java 寫的）

3：mv  maxwell-1.22.0  /usr/local   && ln -s maxwell  maxwell-1.22.0

4:  配置原庫（192.168.0.1）添加賬號，(后面要用）.

 

mysql> select user,host from mysql.user where user='canal';
+-------+------+
| user  | host |
+-------+------+
| canal | %    |
+-------+------+
1 row in set (0.01 sec)

mysql> show grants for canal@'%';
+---------------------------------------------------------------------------+
| Grants for canal@% |
+---------------------------------------------------------------------------+
| GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'canal'@'%' |
+---------------------------------------------------------------------------+

 

5:配置偽目標mysql庫：（maxwell的主機）為什么叫偽目標庫？？？因為就是模擬的從庫行為。

mysql> select user,host from mysql.user where user='maxwell';
+---------+-----------+
| user    | host      |
+---------+-----------+
| maxwell | localhost |
+---------+-----------+
1 row in set (0.00 sec)

mysql> show grants for maxwell@'localhost';
+--------------------------------------------------------------+
| Grants for maxwell@localhost                                 |
+--------------------------------------------------------------+
| GRANT ALL PRIVILEGES ON *.* TO 'maxwell'@'localhost'         |
+--------------------------------------------------------------+
2 rows in set (0.00 sec)

 

 

 6：配置maxwell：就是他了！

config.properties

 

[root@redis-binlog-reportform maxwell]# pwd;ls  -la config.properties
/usr/local/maxwell
-rw-r--r-- 1 root root 10505 May  8 23:19 config.properties

 

配置redis入口

#           *** redis ***

redis_host=192.168.0.3
redis_port=xxxx
redis_auth=XXXX
redis_database=0
redis_pub_channel=maxwell
redis_list_key=maxwell
redis_type=lpush

 

 

配置源庫入口

replication_host=192.168.0.2
replication_user=canal
replication_password=xxx
replication_port=3306

 

配置偽從庫入口

# mysql login info
host=localhost
user=maxwell
password=xxx
port=3306

7 啟動maxwell

nohup bin/maxwell --config=config.properties &

8 查看進程及日志

MaxwellConfig - disabling bootstrapping; not available when using a separate replication host.
18:23:29,734 WARN  MaxwellMetrics - Metrics will not be exposed: metricsReportingType not configured.
18:23:30,153 INFO  Maxwell - Maxwell v1.22.0 is booting (MaxwellRedisProducer), starting at Position[BinlogPosition[mysql_bin.003563:231462217], lastHeartbeat=0]
18:23:30,334 INFO  MysqlSavedSchema - Restoring schema id 1 (last modified at Position[BinlogPosition[mysql_bin.003555:753826693], lastHeartbeat=0])
18:23:30,537 INFO  BinlogConnectorReplicator - Setting initial binlog pos to: mysql_bin.003563:231462217

9：查看redis 接收情況

[root@redis-binlog-reportform maxwell]# redis-cli -p 9001
127.0.0.1:9001> auth xxxx
OK
127.0.0.1:9001> 
127.0.0.1:9001> 
127.0.0.1:9001> 
127.0.0.1:9001> type maxwell
list
127.0.0.1:9001> 
127.0.0.1:9001> llen maxwell
(integer) 53900946
127.0.0.1:9001> llen maxwell
(integer) 53901342
127.0.0.1:9001> llen maxwell
(integer) 53901028
127.0.0.1:9001> llen maxwell
(integer) 53900745
127.0.0.1:9001>