都說pt-toolkit工具集中的pt-online-schema-change可以在線不鎖表修改表結構,那么這個工具具體是什么原理呢,請見下面娓娓道來:
1.pt-online-schema-change工具的使用限制:
1)、如果修改表有外鍵,除非使用 --alter-foreign-keys-method 指定特定的值,否則工具不予執行
2)、被修改表必須要有主鍵,否則報錯:Cannot chunk the original table `houyi`.`ga`: There is no good index and the table is oversized. at ./pt-online-schema-change line 5353.
3)、被修改表上不能有針對after delete|insert|update三個觸發器,否則修改表結構操作失敗
2.創建執行環境
安裝這里就不說了,直接到這里下載安裝:
源碼:https://www.percona.com/downloads/percona-toolkit/2.2.17/tarball/percona-toolkit-2.2.17.tar.gz
RPM:https://www.percona.com/downloads/percona-toolkit/2.2.17/RPM/percona-toolkit-2.2.17-1.noarch.rpm
創建測試數據:
mysql> create database xiaoboluo;
Query OK, 0 rows affected (0.04 sec)
mysql> create table teset_ptosc(id int unsigned not null primary key auto_increment,test varchar(100));
Query OK, 0 rows affected (0.04 sec)
mysql> insert into teset_ptosc(test) values('test1'),('test2'),('test3');
Query OK, 3 rows affected (0.16 sec)
Records: 3 Duplicates: 0 Warnings: 0
mysql> set global general_log=1;
Query OK, 0 rows affected (0.02 sec)
3.執行pt-online-schema-change命令添加一個字段test2:
shell > pt-online-schema-change --alter 'add column test2 varchar(100)' p='password',u=root,D=xiaoboluo,t=teset_ptosc --no-check-replication-filters --execute
執行輸出過程如下:
Found 1 slaves:
localhost.localdomain
Will check slave lag on:
localhost.localdomain
Operation, tries, wait:
copy_rows, 10, 0.25
create_triggers, 10, 1
drop_triggers, 10, 1
swap_tables, 10, 1
update_foreign_keys, 10, 1
Altering `xiaoboluo`.`teset_ptosc`...
Creating new table...
Created new table xiaoboluo._teset_ptosc_new OK.
Altering new table...
Altered `xiaoboluo`.`_teset_ptosc_new` OK.
2016-03-14T00:55:56 Creating triggers...
2016-03-14T00:55:56 Created triggers OK.
2016-03-14T00:55:56 Copying approximately 1 rows...
2016-03-14T00:55:56 Copied rows OK.
2016-03-14T00:55:56 Swapping tables...
2016-03-14T00:55:56 Swapped original and new tables OK.
2016-03-14T00:55:56 Dropping old table...
2016-03-14T00:55:56 Dropped old table `xiaoboluo`.`_teset_ptosc_old` OK.
2016-03-14T00:55:56 Dropping triggers...
2016-03-14T00:55:56 Dropped triggers OK.
Successfully altered `xiaoboluo`.`teset_ptosc`.
從上面的執行輸出中就可以大概看到執行過程:
創建一個新表,然后alter新表,然后創建觸發器,然后copy數據,然后交換表,然后刪除old表,然后刪除觸發器,最后返回成功alter的提示,那么,具體在數據庫中是如何操作的呢,前面打開了general_log,現在去查看下general_log文件中的內容:
4.結合general_log文件中的輸出,pt-online-schema-change的大致過程如下:
1)、首先使用帳號密碼連接到mysql后,獲取指定表的狀態信息,檢查是否有觸發器,檢查表是否有主鍵。
2)、接着按照修改表的表定義,新建一個名為'_tb_new'不可見的臨時表,對這個表執行alter添加字段,並校驗是否執行成功。
3)、然后針對源表創建三個觸發器,分別如下:
create trigger db_tb_del after delete on db.tb for each row delete ignore from db._tb_new where db._tb_new.id <=> OLD.id #刪掉新表中db._tb_new.id <=> OLD.id的數據,否則忽略操作
create trigger db_tb_del after update on db.tb for each row replace into db._tb_new(id,...) values(new.id,...) #源表執行update的時候,把對應的數據replace into的方式寫入新表
create trigger db_tb_del after insert on db.tb for each row replace into db._tb_new(id,...) values(new.id,...) #源表執行iinsert操作的時候,把對應的數據replace into的方式寫入新表
4)、觸發器創建好之后會執行insert low_priority ignore into db._tb_new(id,..) select id,... from tb lock in share mode語句復制源表數據到新表。
5)、復制完成之后執行語句:rename table db.tb to db._tb_old,db._tb_new to db.tb同時把源表修改為_tb_old格式,把新表_tb_new修改為源表名字的原子修改。
6)、接着,如果沒有加不刪除old表的選項,那么就會刪除Old表,然后刪除三個觸發器。到這里就完成了在線表結構的修改 。整個過程只在rename表的時間會鎖一下表,其他時候不鎖表。
5.下面是general_log輸出原文的相關部分:
642 Query SHOW TABLES FROM `xiaoboluo` LIKE 'teset\_ptosc'
642 Query SHOW TRIGGERS FROM `xiaoboluo` LIKE 'teset\_ptosc'
642 Query /*!40101 SET @OLD_SQL_MODE := @@SQL_MODE, @@SQL_MODE := '', @OLD_QUOTE := @@SQL_QUOTE_SHOW_CREATE, @@SQL_QUOTE_SHOW_CREATE := 1 */
642 Query USE `xiaoboluo`
642 Query SHOW CREATE TABLE `xiaoboluo`.`teset_ptosc`
642 Query /*!40101 SET @@SQL_MODE := @OLD_SQL_MODE, @@SQL_QUOTE_SHOW_CREATE := @OLD_QUOTE */
642 Query EXPLAIN SELECT * FROM `xiaoboluo`.`teset_ptosc` WHERE 1=1
642 Query SELECT table_schema, table_name FROM information_schema.key_column_usage WHERE referenced_table_schema='xiaoboluo' AND referenced_table_name='teset_ptosc'
642 Query SHOW VARIABLES LIKE 'wsrep_on'
642 Query /*!40101 SET @OLD_SQL_MODE := @@SQL_MODE, @@SQL_MODE := '', @OLD_QUOTE := @@SQL_QUOTE_SHOW_CREATE, @@SQL_QUOTE_SHOW_CREATE := 1 */
642 Query USE `xiaoboluo`
642 Query SHOW CREATE TABLE `xiaoboluo`.`teset_ptosc`
642 Query /*!40101 SET @@SQL_MODE := @OLD_SQL_MODE, @@SQL_QUOTE_SHOW_CREATE := @OLD_QUOTE */
642 Query CREATE TABLE `xiaoboluo`.`_teset_ptosc_new` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`test` varchar(100) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
642 Query ALTER TABLE `xiaoboluo`.`_teset_ptosc_new` add column test2 varchar(100)
642 Query /*!40101 SET @OLD_SQL_MODE := @@SQL_MODE, @@SQL_MODE := '', @OLD_QUOTE := @@SQL_QUOTE_SHOW_CREATE, @@SQL_QUOTE_SHOW_CREATE := 1 */
642 Query USE `xiaoboluo`
642 Query SHOW CREATE TABLE `xiaoboluo`.`_teset_ptosc_new`
642 Query /*!40101 SET @@SQL_MODE := @OLD_SQL_MODE, @@SQL_QUOTE_SHOW_CREATE := @OLD_QUOTE */
642 Query CREATE TRIGGER `pt_osc_xiaoboluo_teset_ptosc_del` AFTER DELETE ON `xiaoboluo`.`teset_ptosc` FOR EACH ROW DELETE IGNORE FROM `xiaoboluo`.`_teset_ptosc_new` WHERE `xiaoboluo`.`_teset_ptosc_new`.`id` <=> OLD.`id`
642 Query CREATE TRIGGER `pt_osc_xiaoboluo_teset_ptosc_upd` AFTER UPDATE ON `xiaoboluo`.`teset_ptosc` FOR EACH ROW REPLACE INTO `xiaoboluo`.`_teset_ptosc_new` (`id`, `test`) VALUES (NEW.`id`, NEW.`test`)
642 Query CREATE TRIGGER `pt_osc_xiaoboluo_teset_ptosc_ins` AFTER INSERT ON `xiaoboluo`.`teset_ptosc` FOR EACH ROW REPLACE INTO `xiaoboluo`.`_teset_ptosc_new` (`id`, `test`) VALUES (NEW.`id`, NEW.`test`)
642 Query EXPLAIN SELECT * FROM `xiaoboluo`.`teset_ptosc` WHERE 1=1
642 Query EXPLAIN SELECT `id`, `test` FROM `xiaoboluo`.`teset_ptosc` LOCK IN SHARE MODE /*explain pt-online-schema-change 45383 copy table*/
642 Query INSERT LOW_PRIORITY IGNORE INTO `xiaoboluo`.`_teset_ptosc_new` (`id`, `test`) SELECT `id`, `test` FROM `xiaoboluo`.`teset_ptosc` LOCK IN SHARE MODE /*pt-online-schema-change 45383 copy table*/
642 Query SHOW WARNINGS
642 Query SHOW GLOBAL STATUS LIKE 'Threads_running'
642 Query RENAME TABLE `xiaoboluo`.`teset_ptosc` TO `xiaoboluo`.`_teset_ptosc_old`, `xiaoboluo`.`_teset_ptosc_new` TO `xiaoboluo`.`teset_ptosc`
642 Query DROP TABLE IF EXISTS `xiaoboluo`.`_teset_ptosc_old`
642 Query DROP TRIGGER IF EXISTS `xiaoboluo`.`pt_osc_xiaoboluo_teset_ptosc_del`
642 Query DROP TRIGGER IF EXISTS `xiaoboluo`.`pt_osc_xiaoboluo_teset_ptosc_upd`
642 Query DROP TRIGGER IF EXISTS `xiaoboluo`.`pt_osc_xiaoboluo_teset_ptosc_ins`
642 Query SHOW TABLES FROM `xiaoboluo` LIKE '\_teset\_ptosc\_new'