MySQL MTS復制: hitting slave_pending_jobs_size_max

測試步驟：

從庫停止復制：stop slave;

主庫創建大表400萬條記錄。

開啟從庫復制：start slave;

監測從庫error log持續輸出：

2018-12-06T10:40:52.616289+08:00 4 [Note] Multi-threaded slave: Coordinator has waited 2431 times hitting slave_pending_jobs_size_max; current event size = 8207.
2018-12-06T10:40:52.647618+08:00 4 [Note] Multi-threaded slave: Coordinator has waited 2441 times hitting slave_pending_jobs_size_max; current event size = 8207.
2018-12-06T10:40:52.679589+08:00 4 [Note] Multi-threaded slave: Coordinator has waited 2451 times hitting slave_pending_jobs_size_max; current event size = 8207.
2018-12-06T10:40:52.711510+08:00 4 [Note] Multi-threaded slave: Coordinator has waited 2461 times hitting slave_pending_jobs_size_max; current event size = 8207.
2018-12-06T10:40:52.750250+08:00 4 [Note] Multi-threaded slave: Coordinator has waited 2471 times hitting slave_pending_jobs_size_max; current event size = 8207.
2018-12-06T10:40:52.785731+08:00 4 [Note] Multi-threaded slave: Coordinator has waited 2481 times hitting slave_pending_jobs_size_max; current event size = 8207.

 

搜索發現報錯有以下兩種情況
第一種
Last_Error: Cannot schedule event Rows_query, relay-log name ./db-s18-relay-bin.000448, position 419156572 to Worker thread because its size 18483519 exceeds 16777216 of slave_pending_jobs_size_max.
第二種
[Note] Multi-threaded slave: Coordinator has waited 701 times hitting slave_pending_jobs_size_max; current event size = 8167.

BUG地址：https://bugs.mysql.com/bug.php?id=68462
以上兩種報錯，初步判斷問題可能在 slave_pending_jobs_size_max 的大小上，此值，官方默認是 16M，此值可以動態調整

slave-pending-jobs-size-max參數說明
在多線程復制時，在隊列中Pending的事件所占用的最大內存，默認為16M，如果內存富余，或者延遲較大時，可以適當調大;注意這個值要比主庫的max_allowed_packet大

slave-pending-jobs-size-max有如下幾種情況：
1.- 如果event大小已經超過了等待任務大小的上限（配置slave-pending-jobs-size-max )，就報event太大的錯，然后返回；
2.- 如果event大小+已經在等待的任務大小超過了slave-pending-jobs-size-max，就等待，至到等待隊列變小；
3.- 如果當前的worker的隊列滿的話，也等待。
---------------------

 

檢查slave_pending_jobs_size_max參數值為默認：

+-----------------------------+----------+
| Variable_name               | Value    |
+-----------------------------+----------+
| slave_pending_jobs_size_max | 16777216 |
+-----------------------------+----------+

 

調大該參數：

root@localhost:3306.sock [(none)]>set global slave_pending_jobs_size_max=16777216*8;
Query OK, 0 rows affected (0.02 sec)

root@localhost:3306.sock [(none)]>show variables like '%job%';
+-----------------------------+-----------+
| Variable_name               | Value     |
+-----------------------------+-----------+
| slave_pending_jobs_size_max | 134217728 |
+-----------------------------+-----------+
1 row in set (0.00 sec)

 

重新測試，告警消失。

 

WL#11348: Defaults: Increase Slave's Multi-Threaded Event Applier Buffer

看來MySQL 8.0的下一個版本會將默認值提高了。