MySQL線程處於Waiting for table flush的分析


 

最近遇到一個案例,很多查詢被阻塞沒有返回結果,使用show processlist查看,發現不少MySQL線程處於Waiting for table flush狀態,查詢語句一直被阻塞,只能通過Kill進程來解決。那么我們先來看看Waiting for table flush的官方解釋:https://dev.mysql.com/doc/refman/5.6/en/general-thread-states.html

 

Waiting for table flush

 

The thread is executing FLUSH TABLES and is waiting for all threads to close their tables, or the thread got a notification that the underlying structure for a table has changed and it needs to reopen the table to get the new structure. However, to reopen the table, it must wait until all other threads have closed the table in question.

This notification takes place if another thread has used FLUSH TABLES or one of the following statements on the table in question: FLUSH TABLES tbl_name, ALTER TABLE, RENAME TABLE, REPAIR TABLE, ANALYZE TABLE, or OPTIMIZE TABLE.

 

 

那么我們接下來模擬一下線程處於Waiting for table flush狀態的情況,如所示:

 

在第一個會話連接(connection id=13)中,我們使用lock table 鎖定表test。 

 

mysql> use MyDB;
Database changed
mysql> select connection_id();
+-----------------+
| connection_id() |
+-----------------+
|              13 |
+-----------------+
1 row in set (0.00 sec)
 
mysql> lock table test read;
Query OK, 0 rows affected (0.00 sec)
 
mysql> 

 

 

 

在第二個會話連接(connection id=17)中,我們執行flush table 或 flush table test 皆可。此時你會發現flush table處於阻塞狀態。

 

mysql> use MyDB;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
 
Database changed
mysql> select connection_id();
+-----------------+
| connection_id() |
+-----------------+
|              17 |
+-----------------+
1 row in set (0.00 sec)
 
mysql> flush table test;

 

 

clip_image001

 

 

在第三個會話/連接中,當你切換到MyDB時,就會提示You can turn off this feature to get a quicker startup with -A ,此時處於阻塞狀態。此時你退出會話,使用參數-A登錄數據庫后,你如果查詢test表,就會處於阻塞狀態(當然查詢其它表不會被阻塞)。如下所示:

 

mysql> use MyDB;

Reading table information for completion of table and column names

You can turn off this feature to get a quicker startup with -A

 

 

mysql> use MyDB;

Database changed

mysql> select * from test;

 

clip_image002

 

 

在第四個會話/連接,我們用show processlist查看到當前數據庫所有連接線程狀態,你會看到17、18都處於Waiting for table flush的狀態。如下截圖所示:

 

mysql> show processlist;
+----+------+-----------+------+---------+------+-------------------------+--------------------+
| Id | User | Host      | db   | Command | Time | State                   | Info               |
+----+------+-----------+------+---------+------+-------------------------+--------------------+
| 13 | root | localhost | MyDB | Sleep   |   90 |                         | NULL               |
| 14 | root | localhost | NULL | Query   |    0 | init                    | show processlist   |
| 17 | root | localhost | MyDB | Query   |   52 | Waiting for table flush | flush table test   |
| 18 | root | localhost | MyDB | Query   |    9 | Waiting for table flush | select * from test |
+----+------+-----------+------+---------+------+-------------------------+--------------------+
4 rows in set (0.00 sec)
 
mysql> 

 

clip_image003

 

mysql> show processlist;
+----+------+-----------+------+---------+------+-------------------------+--------------------+
| Id | User | Host      | db   | Command | Time | State                   | Info               |
+----+------+-----------+------+---------+------+-------------------------+--------------------+
| 13 | root | localhost | MyDB | Sleep   |   90 |                         | NULL               |
| 14 | root | localhost | NULL | Query   |    0 | init                    | show processlist   |
| 17 | root | localhost | MyDB | Query   |   52 | Waiting for table flush | flush table test   |
| 18 | root | localhost | MyDB | Query   |    9 | Waiting for table flush | select * from test |
+----+------+-----------+------+---------+------+-------------------------+--------------------+
4 rows in set (0.00 sec)
 
mysql> 
mysql> 
mysql> 
mysql> 
mysql> show open tables where in_use >=1;
+----------+-------+--------+-------------+
| Database | Table | In_use | Name_locked |
+----------+-------+--------+-------------+
| MyDB     | test  |      1 |           0 |
+----------+-------+--------+-------------+
1 row in set (0.00 sec)
 
mysql> kill 17;
Query OK, 0 rows affected (0.00 sec)
 
mysql> show processlist;
+----+------+-----------+------+---------+------+-------------------------+--------------------+
| Id | User | Host      | db   | Command | Time | State                   | Info               |
+----+------+-----------+------+---------+------+-------------------------+--------------------+
| 13 | root | localhost | MyDB | Sleep   |  442 |                         | NULL               |
| 14 | root | localhost | NULL | Query   |    0 | init                    | show processlist   |
| 18 | root | localhost | MyDB | Query   |  361 | Waiting for table flush | select * from test |
+----+------+-----------+------+---------+------+-------------------------+--------------------+
3 rows in set (0.00 sec)
 
mysql> kill 13;
Query OK, 0 rows affected (0.00 sec)
 
mysql> show processlist;
+----+------+-----------+------+---------+------+-------+------------------+
| Id | User | Host      | db   | Command | Time | State | Info             |
+----+------+-----------+------+---------+------+-------+------------------+
| 14 | root | localhost | NULL | Query   |    0 | init  | show processlist |
| 18 | root | localhost | MyDB | Sleep   |  427 |       | NULL             |
+----+------+-----------+------+---------+------+-------+------------------+
2 rows in set (0.00 sec)
 
mysql> 

 

| clip_image004

 

注意:我們需要Kill線程13, Kill掉線程17是解決不了問題的。

 

 

 

生產環境中,很多時候可能不是lock table read引起的阻塞,而是由於慢查詢,導致flush table一直無法關閉該表而一直處於等待狀態,例如下面測試案例中,我使用同一張大表做笛卡爾積模擬一個慢查詢,其它操作相同,如下所示,你會看到同樣產生了Waiting for table flush

 

mysql> SELECT T.* FROM TEST1 T, TEST1 L;

 

clip_image005

 

 

另外,網上有個案例,mysqldump備份時,如果沒有使用參數single-transaction 或由於同時使用了flush-logs與single-transaction兩個參數也可能引起這樣的等待場景,這個兩個參數放在一起,會在開始dump數據之前先執行一個FLUSH TABLES操作。

 

 

 

解決方案:

 

 

出現Waiting for table flush時,我們一般需要找到那些表被lock住或那些慢查詢導致flush table一直在等待而無法關閉該表。然后Kill掉對應的線程即可,但是如何精准定位是一個挑戰,尤其是生產環境,你使用show processlist會看到大量的線程。讓你眼花繚亂的,怎么一下子定位問題呢?

 

對於慢查詢引起的其它線程處於Waiting for table flush狀態的情形:

 

可以查看show processlist中Time值很大的線程。然后甄別確認后Kill掉,如上截圖所示,會話連接14就是引起阻塞的源頭SQL。有種規律就是這個線程的Time列值必定比被阻塞的線程要高。這個就能過濾很多記錄。

 

對於lock table read引起的其它線程處於Waiting for table flush狀態的情形:

 

對於實驗中使用lock table read這種情況,這種會話可能處於Sleep狀態,而且它也不會出現在show engine innodb status \G命令的輸出信息中。 即使show open tables where in_use >=1;能找到是那張表被lock住了,但是無法定位到具體的線程(連接),其實這個是一個頭痛的問題。但是inntop這款利器就可以定位到,如下所示,線程17鎖住了表test,在innotop里面就能定位到是線程17。所謂工欲善其事必先利其器!

 

clip_image006

 

clip_image007

 

 

 

另外,在官方文檔中ALTER TABLE, RENAME TABLE, REPAIR TABLE, ANALYZE TABLE, or OPTIMIZE TABLE都能引起這類等待,下面也做了一些簡單測試,如下所示:

 

 

 

Waiting for table flush的另外一個場景

 

會話連接(connection id=18)執行下面SQL語句,模擬一個慢查詢SQL

 

mysql> select connection_id();
+-----------------+
| connection_id() |
+-----------------+
|              18 |
+-----------------+
1 row in set (0.00 sec)
 
mysql> select name, sleep(64) from test;

 

會話連接(connection id=6)執行下面SQL語句,分析表test

 

mysql> select connection_id();
+-----------------+
| connection_id() |
+-----------------+
|               6 |
+-----------------+
1 row in set (0.00 sec)
mysql> analyze table test;
+-----------+---------+----------+----------+
| Table     | Op      | Msg_type | Msg_text |
+-----------+---------+----------+----------+
| MyDB.test | analyze | status   | OK       |
+-----------+---------+----------+----------+
1 row in set (0.04 sec)
 
mysql> 

 

會話連接(connection id=8)執行下面SQL語句

 

mysql> select connection_id();
+-----------------+
| connection_id() |
+-----------------+
|               8 |
+-----------------+
1 row in set (0.00 sec)
 
mysql> select * from test;

 

查看線程的狀態,你會發現被阻塞的會話處於 Waiting for table flush狀態。 因為當對表做了ANALYZE TABLE后,后台針對該表的查詢需要等待,因為MySQL已經檢測到該表內部變化,需要使用FLUSH TABLE關閉然后重新打開該表,所以當你查詢該表時,就會處於 Waiting for table flush

 

mysql> show processlist;
+----+------+-----------+------+---------+------+-------------------------+----------------------------------+
| Id | User | Host      | db   | Command | Time | State                   | Info                             |
+----+------+-----------+------+---------+------+-------------------------+----------------------------------+
|  6 | root | localhost | MyDB | Sleep   |   22 |                         | NULL                             |
|  8 | root | localhost | MyDB | Query   |   14 | Waiting for table flush | select * from test               |
| 15 | root | localhost | NULL | Sleep   |    3 |                         | NULL                             |
| 16 | root | localhost | NULL | Query   |    0 | init                    | show processlist                 |
| 18 | root | localhost | MyDB | Query   |   46 | User sleep              | select name, sleep(64) from test |
+----+------+-----------+------+---------+------+-------------------------+----------------------------------+
5 rows in set (0.00 sec)
 
mysql> 

 

clip_image008

 

 

 

Waiting for table metadata lock

 

 

會話連接(connection id=17)執行下面SQL語句,模擬一個慢查詢SQL

 

 

mysql> select connection_id();
+-----------------+
| connection_id() |
+-----------------+
|              17 |
+-----------------+
1 row in set (0.00 sec)
 
mysql> select name, sleep(100) from test;

 

 

會話連接(connection id=6)執行下面SQL語句, 修改表結構操作

 

mysql> select connection_id();
+-----------------+
| connection_id() |
+-----------------+
|               6 |
+-----------------+
1 row in set (0.00 sec)
 
mysql> alter table test add tname varchar(10); // rename table test to kkk 同樣會引起Waiting for table metadata lock

 

 

會話連接(connection id=8)執行下面SQL語句,查詢表test

 

mysql> select connection_id();
+-----------------+
| connection_id() |
+-----------------+
|               8 |
+-----------------+
1 row in set (0.00 sec)
 
mysql> select * from test;

 

 

查看線程的狀態,你會發現被阻塞的會話處於 Waiting for table metadata lock狀態。

 

 

mysql> show processlist;
+----+------+-----------+------+---------+------+---------------------------------+----------------------------------------+
| Id | User | Host      | db   | Command | Time | State                           | Info                                   |
+----+------+-----------+------+---------+------+---------------------------------+----------------------------------------+
|  6 | root | localhost | MyDB | Query   |   19 | Waiting for table metadata lock | alter table test add tname varchar(10) |
|  8 | root | localhost | MyDB | Query   |    6 | Waiting for table metadata lock | select * from test                     |
| 15 | root | localhost | NULL | Sleep   |    8 |                                 | NULL                                   |
| 16 | root | localhost | NULL | Query   |    0 | init                            | show processlist                       |
| 17 | root | localhost | MyDB | Query   |   55 | User sleep                      | select name, sleep(100) from test      |
+----+------+-----------+------+---------+------+---------------------------------+----------------------------------------+
5 rows in set (0.00 sec)
 
mysql> 

 

clip_image009

 

 

 

 

參考資料:

 

https://www.percona.com/blog/2013/02/27/mysql-optimizer-analyze-table-and-waiting-for-table-flush/

http://www.cnblogs.com/jackhub/p/3841004.html

http://myrock.github.io/2014/11/20/mysql-waiting-for-table-flush/

http://mysql.taobao.org/monthly/2016/03/10/


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM