MySQL Test Suite使用

MySQL自動測試套件（The MySQL Test Suite）用於對MySQL程序進行測試，包括各種功能與存儲引擎。包含於MySQL與MariaDB版本代碼中，位於mysql-test目錄下。總體測試腳本為mysql-test-run.pl。該perl腳本通過調用各種已有的測試腳本，將測試結果與預置的result文件做對比來判斷測試是否通過。（任何一點差異都會導致測試失敗，包括預期之外的warning）

 

 

一、主要目錄介紹

• • include

包含.inc文件，用於測試開始時判斷是否滿足測試條件，在測試用通過source命令引入

如hava_innodb.inc:

 

1 2 3 4

if (`SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.ENGINES WHERE engine = 'innodb' AND support IN ('YES', 'DEFAULT', 'ENABLED')`) {              --skip Test requires InnoDB. }

如不滿足，則會跳過需要該驗證的測試。

 

• • suite

包含所有測試suite，每一個suite為一個測試用例集，具體用例集在2節介紹

 

 

• • t

1、測試文件

例：1st.test

 

1 2 3 4 5

# # Check that we haven't any strange new tables or databases # show databases; show tables in mysql;

上面內容即為1st這個測試用例的測試腳本

 

 2、配置文件

例：innodb_bug53674-master.opt

–loose-innodb-locks-unsafe-for-binlog –binlog-format=mixed

為該測試需要的特殊配置項

 

3、腳本文件，在測試開始之前執行的腳本

例：rpl_misc_functions-slave.sh

rm -f $MYSQLTEST_VARDIR/master-data/test/rpl_misc_functions.outfile

 

 

• • r

結果文件，判斷是否通過的依據

使用上面的例子：1st.result

 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39

show databases; Database information_schema mtr mysql performance_schema test show tables in mysql; Tables_in_mysql column_stats columns_priv db event func general_log gtid_slave_pos help_category help_keyword help_relation help_topic host index_stats innodb_index_stats innodb_table_stats plugin proc procs_priv proxies_priv roles_mapping servers slow_log table_stats tables_priv time_zone time_zone_leap_second time_zone_name time_zone_transition time_zone_transition_type user

執行完1st用例會嚴格比對該文件。完全一致則測試通過。

 

• • std_date

有些測試需要用到一些標准數據。

 

• • var

測試默認使用該文件夾下的my.cnf文件

 

 

二、測試用例集

./mysql-test-run.pl不帶任何參數表示執行所有測試用例集。使用—suite=suitename來指定單獨執行suitename目錄下的所有測試用例。

測試用例集包括：

 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37

main：主要測試用例 archive binlog csv encryption：加密功能 engines federated funcs_1：額外功能（包括視圖、存儲過程、INFORMATION_SCHEMA等） funcs_2：額外功能（字符集等） galera galera_3nodes handler heap innodb innodb_fts：innodb全文索引 innodb_zip：innodb壓縮 jp：日本字符相關測試 large_tests： maria mtr mtr2 multi_source ndb optimizer_unfixed_bugs parts percona perfschema perfschema_stress plugins roles rpl storage_engine stress sys_vars unit vcol wsrep

 

 

三、主要參數

force：默認情況下，只要遇到一個用例出錯，測試程序就會退出，在指定force的情況下，測試程序會繼續執行下面的測試（但是最多發現10個錯誤還是會退出）

suite：指定使用的測試suite

do-test：會以輸入的字符串進行匹配用例執行

skip-test：會以輸入的字符串進行匹配跳過用例執行

big-test：執行標記為big的測試用例。因為用例較大、耗時較長，標記為big的用例默認不會執行。輸入兩遍big-test則只執行標記為big的測試用例

recored：重新生成結果文件

mysqld：傳遞啟動參數給mysqld，每個參數需要分別指定一個mysqld

其他還有很多選項，可以使用./mtr –help查看

 

如果需要在服務啟動前執行一些腳本，可以寫在 t/testname.sh文件中，由mtr自動執行。

 

 

四、測試舉例

例1：

 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39

[lzk@redhat64-26 mysql-test]$ ./mtr  --force --big-test --suite=archive Logging: ./mtr  --force --big-test --suite=archive vardir: /home/lzk/mysql-test/var Checking leftover processes... Removing old var directory... Creating var directory '/home/lzk/mysql-test/var'... Checking supported features... MariaDB Version 10.1.12-MariaDB  - SSL connections supported Using suites: archive Collecting tests... Installing system database...   ==============================================================================   TEST                                      RESULT   TIME (ms) or COMMENT --------------------------------------------------------------------------   worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019 archive.archive_debug                    [ skipped ]  Requires debug build archive.partition_archive                [ pass ]     74 archive.archive_no_symlink               [ pass ]      5 archive.archive-big                      [ pass ]  298619 archive.archive                          [ pass ]    417 archive.archive_bitfield                 [ pass ]     37 archive.archive_gis                      [ pass ]     66 archive.archive_symlink                  [ pass ]     11 archive.discover                         [ pass ]     47 archive.discover_5438                    [ pass ]      7 archive.mysqlhotcopy_archive             [ pass ]   1035 archive.repair                           [ pass ]      3 archive.archive_plugin                   [ pass ]      6 -------------------------------------------------------------------------- The servers were restarted 3 times Spent 300.327 of 309 seconds executing testcases   Completed: All 12 tests were successful.   1 tests were skipped, 0 by the test itself.

 

可以看出共完成12個測試用例，全部通過。1個用例必須在debug模式下測試，所以跳過。測試過程中服務重啟3次。測試默認使用16000-16019端口。

 

例2：

 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

[lzk@redhat64-26 mysql-test]$  ./mtr binlog_innodb_row Logging: ./mtr  binlog_innodb_row vardir: /home/lzk/mysql-test/var Checking leftover processes...  - found old pid 28610 in 'mysqld.1.pid', killing it...    process did not exist! Removing old var directory... Creating var directory '/home/lzk/mysql-test/var'... Checking supported features... MariaDB Version 10.1.12-MariaDB  - SSL connections supported klist: No credentials cache found (ticket cache FILE:/tmp/krb5cc_550) Collecting tests... Installing system database...   ==============================================================================   TEST                                      RESULT   TIME (ms) or COMMENT --------------------------------------------------------------------------   worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019 binlog.binlog_innodb_row 'innodb_plugin,row' [ pass ]   3361 binlog.binlog_innodb_row 'row,xtradb'    [ pass ]   3349 -------------------------------------------------------------------------- The servers were restarted 1 times Spent 6.710 of 17 seconds executing testcases   Completed: All 2 tests were successful.

每個測試結束前，mtr會檢索error日志，如果發現warning或error，則測試失敗。

 

再來看兩個失敗用例：

（截取部分日志）

 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

innodb.innodb_bug53674 'innodb_plugin'   [ fail ]  Found warnings/errors in server log file!         Test ended at 2016-03-24 20:57:36 line 2016-03-24 20:57:35 7fd0fc6fc7e0 InnoDB: Warning: Using innodb_locks_unsafe_for_binlog is DEPRECATED. This option may be removed in future releases. Please use READ COMMITTED transaction isolation level instead, see http://dev.mysql.com/doc/refman/5.6/en/set-transaction.html. ^ Found warnings in /home/lzk/mysql-test/var/log/mysqld.1.err ok    - saving '/home/lzk/mysql-test/var/log/innodb.innodb_bug53674-innodb_plugin/' to '/home/lzk/mysql-test/var/log/innodb.innodb_bug53674-innodb_plugin/' innodb.innodb_bug53674 'xtradb'          [ fail ]  Found warnings/errors in server log file!         Test ended at 2016-03-24 20:57:37 line 2016-03-24 20:57:36 7f274c58b7e0 InnoDB: Warning: Using innodb_locks_unsafe_for_binlog is DEPRECATED. This option may be removed in future releases. Please use READ COMMITTED transaction isolation level instead, see http://dev.mysql.com/doc/refman/5.6/en/set-transaction.html. ^ Found warnings in /home/lzk/mysql-test/var/log/mysqld.1.err ok    - saving '/home/lzk/mysql-test/var/log/innodb.innodb_bug53674-xtradb/' to '/home/lzk/mysql-test/var/log/innodb.innodb_bug53674-xtradb/'

分析日志可以看出是在測試過程中在錯誤日志中發現了warnings/errors。原因是使用了廢棄的參數innodb_locks_unsafe_for_binlog導致warning。

 

找到並修改配置文件：innodb_bug53674-master.opt

將–loose-innodb-locks-unsafe-for-binlog –binlog-format=mixed

修改為：–transaction-isolation=READ-COMMITTED –binlog-format=mixed

 

重新測試：

 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

[lzk@redhat64-26 mysql-test]$ ./mtr innodb_bug53674 Logging: ./mtr  innodb_bug53674 vardir: /home/lzk/mysql-test/var Checking leftover processes... Removing old var directory... Creating var directory '/home/lzk/mysql-test/var'... Checking supported features... MariaDB Version 10.1.12-MariaDB  - SSL connections supported klist: No credentials cache found (ticket cache FILE:/tmp/krb5cc_550) Collecting tests... Installing system database...   ==============================================================================   TEST                                      RESULT   TIME (ms) or COMMENT --------------------------------------------------------------------------   worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019 innodb.innodb_bug53674 'innodb_plugin'   [ pass ]    211 innodb.innodb_bug53674 'xtradb'          [ pass ]    205 -------------------------------------------------------------------------- The servers were restarted 1 times Spent 0.416 of 10 seconds executing testcases   Completed: All 2 tests were successful.

不再有warning，測試通過。