MySQL回滾到某一時刻數據的方法
Oracle Database 12c RMAN全量+增量備份+歸檔日志恢復詳解
Oracle 12c數據庫定時備份和清理腳本
mysql不同於oracle和db2這種企業級數據庫,它沒有oracle里面的redo日志,也沒有db2里面的循環日志。mysql有類似於oracle和db2歸檔日志的binlog,而這個binlog可以看作是循環日志和歸檔日志的結合。有一定的大小限制。未完成的事務和已完成的事務都會記錄在binlog中,當一個binlog寫滿之后,就會開啟一個新的binlog。binlog還有三種方式,row,statement,mixed。其中row記錄量最大,但是對於各種工具支持最好,所以對於安全要求比較高的數據庫,推薦使用row格式。
同時,binlog也是mysql主從復制的依據,所以使用binlog來恢復數據庫是比較可靠的。不足的就是mysql並沒有內置binlog的清理工具,對於長時間的binlog我們需要去手動清理或者編寫腳本清理。mysql也沒有提供oracle,db2那樣的增量備份方法。所以保證binlog不要丟失就比較重要。雖然手動的操作多了一些,但是這也代表着mysql的恢復更偏向於無狀態的,即異地跨平台恢復會比較方便,不需要像oracle那樣必須找到控制文件。
數據庫恢復的過程於oracle,db2區別不大。基本都是通過先恢復全備份,再逐個恢復增量備份,再根據歸檔日志逐條重做事務,一直重做到你需要恢復到的日期為止。mysql由於沒有增量備份,所以先恢復全備,再手動找到binlog中全備時間的那一行,從那一行往后開始執行重做事務,直到你需要的停止的那一行。
下面來介紹一下如何將mysql數據庫回滾到某一時刻。大概有如下步驟
1、找一個現有的mysql數據庫,先不打開binlog,插入幾條數據
2、打開binlog,重啟數據庫,再插入幾條數據
3、使用mysqldump全備一次數據庫
4、再插入幾條數據,模擬全備之后執行成功的事務,記錄執行完畢的時間。
5、模擬數據庫崩潰或者誤刪操作,然后將全備文件和binlog都拷貝到另一台服務器上進行異地恢復。
這里使用的版本是mysql 5.7.18
首先我們先建庫建表,但是此時沒有開啟binlog,這里我主要像說明,如果沒有binlog,那么就沒有歸檔日志,我們就不知道以前做過了哪些事務,只能使用全備進行恢復,而全備之后發生的操作就都會丟失。my.cnf配置如下
-
# server-id = 1
-
#log_bin = /var/ log/mysql/mysql-bin.log
-
expire_logs_days = 10
-
max_binlog_size = 100M
可以看見log_bin前面被注釋掉了,也就是沒有開啟。下面開始建庫建表
-
mysql> show databases;
-
+--------------------+
-
| Database |
-
+--------------------+
-
| information_schema |
-
| mydb |
-
| mysql |
-
| performance_schema |
-
| sys |
-
+--------------------+
-
5 rows in set (0.00 sec)
-
-
mysql> create database dbtest;
-
Query OK, 1 row affected (0.00 sec)
-
-
mysql> use dbtest;
-
Database changed
-
-
mysql> create table table1 ( id int primary key, name varchar( 40), birthday datetime);
-
Query OK, 0 rows affected (0.44 sec)
-
-
mysql> insert into table1 values ( 1,'befor_binlog1',NOW());
-
Query OK, 1 row affected (0.05 sec)
-
-
mysql> insert into table1 values ( 2,'befor_binlog2',NOW());
-
Query OK, 1 row affected (0.07 sec)
-
-
mysql> select * from table1;
-
+----+---------------+---------------------+
-
| id | name | birthday |
-
+----+---------------+---------------------+
-
| 1 | befor_binlog1 | 2018-06-09 08:36:33 |
-
| 2 | befor_binlog2 | 2018-06-09 08:36:40 |
-
+----+---------------+---------------------+
-
2 rows in set (0.00 sec)
此時我們是看不到當前binlog是哪個文件第幾行的
-
mysql> show master status \G
-
Empty set (0.00 sec)
可以發現,再不開啟binlog的情況下,是可以正常插入數據的,但我還是推薦從開始,也就是建庫建表之前就開啟binlog,那樣即使沒有全備,也可以從頭開始恢復。但是現在這種情況下,如果出現了誤刪操作,我們是無法拯救我們的數據的。
然后我們執行第二步,開啟binlog然后重啟數據庫。首先修改配置文件my.cnf
-
server-id = 1
-
log_bin = /var/ log/mysql/mysql-bin.log
-
expire_logs_days = 10
-
max_binlog_size = 100M
然后我們再來插入兩條數據
-
mysql> use dbtest;
-
Reading table information for completion of table and column names
-
You can turn off this feature to get a quicker startup with -A
-
-
Database changed
-
mysql> insert into table1 values ( 3,'after_binlog1',NOW());
-
Query OK, 1 row affected (0.12 sec)
-
-
mysql> insert into table1 values ( 4,'after_binlog2',NOW());
-
Query OK, 1 row affected (0.20 sec)
-
-
mysql> select * from table1;
-
+----+---------------+---------------------+
-
| id | name | birthday |
-
+----+---------------+---------------------+
-
| 1 | befor_binlog1 | 2018-06-09 08:36:33 |
-
| 2 | befor_binlog2 | 2018-06-09 08:36:40 |
-
| 3 | after_binlog1 | 2018-06-09 08:44:33 |
-
| 4 | after_binlog2 | 2018-06-09 08:44:38 |
-
+----+---------------+---------------------+
-
4 rows in set (0.00 sec)
如果此時發生了誤刪,那我們在進行異地恢復的時候,只能恢復出id為3和4的兩條數據,因為1,2在插入的時候沒有開啟binlog,binlog中沒有這兩條事務的記錄,所以就恢復不了。
下面我們進行第三部,使用mysqldump進行一次全備
-
root@f4d417a2e6ea :/# mysqldump --single-transaction --master-data=2 --triggers --routines --all-databases -uroot -p > /backup/full.sql
-
Enter password:
查看一下備份出來的文件所在時刻歸檔日志的為止
-
--
-
-- Position to start replication or point-in-time recovery from
-
--
-
-
-- CHANGE MASTER TO MASTER_LOG_FILE='mysql-bin.000001', MASTER_LOG_POS=740;
此時查看一下數據庫里歸檔日志的位置
-
mysql> show master status \G
-
*************************** 1. row ***************************
-
File: mysql-bin.000001
-
Position: 740
-
Binlog_Do_DB:
-
Binlog_Ignore_DB:
-
Executed_Gtid_Set:
-
1 row in set (0.00 sec)
可以看到,在備份前和備份后,歸檔日志並沒有發生變化,還是停留在同一行里。這個行數和文件名被記錄在了全備文件中,以后會用到。
現在如果出現了誤刪或者存儲損壞我們是可以高枕無憂的,因為有了全備,我們可以輕松的異地恢復回來
我們進行第四步,插入兩條新數據
-
mysql> insert into table1 values ( 5,'after_backup',NOW());
-
Query OK, 1 row affected (0.13 sec)
-
-
mysql> insert into table1 values ( 6,'after_backup2',NOW());
-
Query OK, 1 row affected (0.11 sec)
-
-
mysql> select * from table1;
-
+----+---------------+---------------------+
-
| id | name | birthday |
-
+----+---------------+---------------------+
-
| 1 | befor_binlog1 | 2018-06-09 08:36:33 |
-
| 2 | befor_binlog2 | 2018-06-09 08:36:40 |
-
| 3 | after_binlog1 | 2018-06-09 08:44:33 |
-
| 4 | after_binlog2 | 2018-06-09 08:44:38 |
-
| 5 | after_backup | 2018-06-09 09:11:18 |
-
| 6 | after_backup2 | 2018-06-09 09:11:25 |
-
+----+---------------+---------------------+
-
6 rows in set (0.00 sec)
然后我們進行第六步,假裝系統崩潰存儲損壞,我們來嘗試恢復到當前數據。在這里我們分兩步來做一個是系統的崩潰,我們需要異地恢復到最新為止,另一個是進行了誤刪操作,我們將6條數據全部恢復回來。
1、系統崩潰情況下,我們先將全備文件和binlog都拷貝到要恢復的新服務器上,這里要注意binlog可能不止一個,但是本例由於數據量少只有一個binlog。binlog的目錄就寫在my.cnf我們剛改過的文件里
按照正常的步驟,我們應該先恢復全備,然后根據binlog重做已經提交的事務,在恢復之前,可以看到是沒有我們原來的庫的
-
mysql> show databases;
-
+--------------------+
-
| Database |
-
+--------------------+
-
| information_schema |
-
| mydb |
-
| mysql |
-
| performance_schema |
-
| sys |
-
+--------------------+
-
5 rows in set (0.00 sec)
-
-
mysql>
全備份恢復,恢復其實就是讓mysql執行一大段sql,而這段sql就是我們用mysqldump導出來的那個。此時可以不開啟新數據庫的binlog。開了反而還會變慢
# mysql -uroot -p < /import/full.sql
然后查看一下導入了哪些數據
-
mysql> show databases;
-
+--------------------+
-
| Database |
-
+--------------------+
-
| information_schema |
-
| dbtest |
-
| mydb |
-
| mysql |
-
| performance_schema |
-
| sys |
-
+--------------------+
-
6 rows in set (0.00 sec)
-
-
mysql> use dbtest;
-
Reading table information for completion of table and column names
-
You can turn off this feature to get a quicker startup with -A
-
-
Database changed
-
mysql> select * from table1;
-
+----+---------------+---------------------+
-
| id | name | birthday |
-
+----+---------------+---------------------+
-
| 1 | befor_binlog1 | 2018-06-09 08:36:33 |
-
| 2 | befor_binlog2 | 2018-06-09 08:36:40 |
-
| 3 | after_binlog1 | 2018-06-09 08:44:33 |
-
| 4 | after_binlog2 | 2018-06-09 08:44:38 |
-
+----+---------------+---------------------+
-
4 rows in set (0.00 sec)
不出所料,由於全備時間點的關系,只有id為1-4的被恢復回來了,5,6由於是全備之后插入的,所以當前沒有,我們需要從binlog中恢復。這里我們使用mysql自帶的mysqlbinlog工具,將binlog解析成sql,然后用mysql執行這段sql,就把后面的事務給執行了,
-
root@5d41cbeb4b4d :/import/mysql# ls
-
error.log mysql-bin. 000001 mysql-bin.index
-
root@5d41cbeb4b4d :/import/mysql# mysqlbinlog --no-defaults /import/mysql/mysql-bin.000001 | mysql -uroot -p
-
Enter password:
-
ERROR 1062 (23000) at line 38: Duplicate entry '3' for key 'PRIMARY'
-
mysql> select * from table1
-
-> ;
-
+----+---------------+---------------------+
-
| id | name | birthday |
-
+----+---------------+---------------------+
-
| 1 | befor_binlog1 | 2018-06-09 08:36:33 |
-
| 2 | befor_binlog2 | 2018-06-09 08:36:40 |
-
| 3 | after_binlog1 | 2018-06-09 08:44:33 |
-
| 4 | after_binlog2 | 2018-06-09 08:44:38 |
-
+----+---------------+---------------------+
-
4 rows in set (0.00 sec)
我們會發現報了一個錯,同時數據也沒有導入成功。這是什么原因呢。
由於我們在id=2之后,開啟了binlog,所以此時binlog中的內容就是insert id為3,4,5,6的語句,所以當我們把整個binlog文件全部重做一遍的話,在insert id=3的時候就會報主鍵沖突的錯誤,這點是顯而易見的。那么如何避免重復執行已備份的事務呢,這就要我們手動指定binlog的重做時間點。前面我們已經知道,從全備文件full.sql中可以看到備份時間點的binlog文件和行數,也就是mysql-bin.000001的第740行,所以我們就從這一行開始恢復
-
# mysqlbinlog --no-defaults --start-position= 740 /import/mysql/mysql-bin.000001 | mysql -
-
uroot -p
-
Enter password:
-
mysql> select * from table1;
-
+----+---------------+---------------------+
-
| id | name | birthday |
-
+----+---------------+---------------------+
-
| 1 | befor_binlog1 | 2018-06-09 08:36:33 |
-
| 2 | befor_binlog2 | 2018-06-09 08:36:40 |
-
| 3 | after_binlog1 | 2018-06-09 08:44:33 |
-
| 4 | after_binlog2 | 2018-06-09 08:44:38 |
-
| 5 | after_backup | 2018-06-09 09:11:18 |
-
| 6 | after_backup2 | 2018-06-09 09:11:25 |
-
+----+---------------+---------------------+
-
6 rows in set (0.00 sec)
2、上面是針對系統崩潰這種情況,進行的全量恢復操作,那么如果是因為操作事務刪除了某些數據或者插入了某些臟數據怎么辦呢。
首先我們去源庫中刪除一些數據,然后插入一些錯誤數據
-
mysql> select * from table1;
-
+----+---------------+---------------------+
-
| id | name | birthday |
-
+----+---------------+---------------------+
-
| 1 | befor_binlog1 | 2018-06-09 08:36:33 |
-
| 2 | befor_binlog2 | 2018-06-09 08:36:40 |
-
| 3 | after_binlog1 | 2018-06-09 08:44:33 |
-
| 4 | after_binlog2 | 2018-06-09 08:44:38 |
-
| 5 | after_backup | 2018-06-09 09:11:18 |
-
| 6 | after_backup2 | 2018-06-09 09:11:25 |
-
+----+---------------+---------------------+
-
6 rows in set (0.00 sec)
-
-
mysql> delete from table1 where id > 3;
-
Query OK, 3 rows affected (0.14 sec)
-
-
mysql> select * from table1;
-
+----+---------------+---------------------+
-
| id | name | birthday |
-
+----+---------------+---------------------+
-
| 1 | befor_binlog1 | 2018-06-09 08:36:33 |
-
| 2 | befor_binlog2 | 2018-06-09 08:36:40 |
-
| 3 | after_binlog1 | 2018-06-09 08:44:33 |
-
+----+---------------+---------------------+
-
3 rows in set (0.00 sec)
-
-
mysql> insert into table1 values ( 4,'xxxxxxx1',NOW());
-
Query OK, 1 row affected (0.12 sec)
-
-
mysql> insert into table1 values ( 5,'xxxxxxx1',NOW());
-
Query OK, 1 row affected (0.14 sec)
-
-
mysql> select * from table1;
-
+----+---------------+---------------------+
-
| id | name | birthday |
-
+----+---------------+---------------------+
-
| 1 | befor_binlog1 | 2018-06-09 08:36:33 |
-
| 2 | befor_binlog2 | 2018-06-09 08:36:40 |
-
| 3 | after_binlog1 | 2018-06-09 08:44:33 |
-
| 4 | xxxxxxx1 | 2018-06-09 09:43:00 |
-
| 5 | xxxxxxx1 | 2018-06-09 09:43:05 |
-
+----+---------------+---------------------+
-
5 rows in set (0.00 sec)
如上所示,先刪掉了一半的數據,然后又插入了兩條數據,占用了原來的4,5.id=6的數據沒了。也就是兩條數據被篡改,1條丟失的情況。
此時再看一下binlog寫到什么位置了。
-
mysql> show master status \G
-
*************************** 1. row ***************************
-
File: mysql-bin.000001
-
Position: 2233
-
Binlog_Do_DB:
-
Binlog_Ignore_DB:
-
Executed_Gtid_Set:
-
1 row in set (0.00 sec)
如果再按照上面的方法對binlog進行恢復,那么我們的誤刪操作也會被恢復,就失去了意義,所以此時我們必須指定恢復的時間點。
從上面可以看出,進行誤刪操作是9:11:25之后,所以我們只需恢復到這個時間點就可以了。
首先還是根據前面的全備進行恢復
-
mysql> show databases;
-
+--------------------+
-
| Database |
-
+--------------------+
-
| information_schema |
-
| mydb |
-
| mysql |
-
| performance_schema |
-
| sys |
-
+--------------------+
-
5 rows in set (0.00 sec)
-
# mysql -uroot -p < / import/full.sql
-
Enter password:
-
mysql> select * from table1;
-
+----+---------------+---------------------+
-
| id | name | birthday |
-
+----+---------------+---------------------+
-
| 1 | befor_binlog1 | 2018-06-09 08:36:33 |
-
| 2 | befor_binlog2 | 2018-06-09 08:36:40 |
-
| 3 | after_binlog1 | 2018-06-09 08:44:33 |
-
| 4 | after_binlog2 | 2018-06-09 08:44:38 |
-
+----+---------------+---------------------+
-
4 rows in set (0.00 sec)
然后關鍵點就是既要指定清楚binlog起始位置,又要指明結束時間,這里時間使用UTC時間,要和binlog里面對應注意,id=6的數據是9:11:25才插入成功的,所以我們恢復最早只能是9:11:26,如果恢復到9:11:25是看不到id=6的數據的
# mysqlbinlog --no-defaults --start-position=740 --stop-datetime="2018-06-09 01:11:26" /import/mysql/mysql-bin.000001 | mysql -uroot -p
然后發現我們的恢復成功了
-
mysql> select * from table1;
-
+----+---------------+---------------------+
-
| id | name | birthday |
-
+----+---------------+---------------------+
-
| 1 | befor_binlog1 | 2018-06-09 08:36:33 |
-
| 2 | befor_binlog2 | 2018-06-09 08:36:40 |
-
| 3 | after_binlog1 | 2018-06-09 08:44:33 |
-
| 4 | after_binlog2 | 2018-06-09 08:44:38 |
-
| 5 | after_backup | 2018-06-09 09:11:18 |
-
| 6 | after_backup2 | 2018-06-09 09:11:25 |
-
+----+---------------+---------------------+
-
6 rows in set (0.00 sec)
題外話:
剛才看到--stop-datetime要用UTC時間,是因為我們查看binlog得來的,下面我們就來看看如何查看binlog文件
比如我們可以看到以下binlog的記錄
-
mysqlbinlog --no-defaults / import/mysql/mysql-bin.000001
-
......
-
-
# at 1179
-
#180609 1:11:25 server id 1 end_log_pos 1235 CRC32 0x6b867c5b Table_map: `dbtest`.`table1` mapped to number 254
-
# at 1235
-
#180609 1:11:25 server id 1 end_log_pos 1294 CRC32 0xc058c50f Write_rows: table id 254 flags: STMT_END_F
-
-
BINLOG '
-
PSkbWxMBAAAAOAAAANMEAAAAAP4AAAAAAAEABmRidGVzdAAGdGFibGUxAAMDDxIDeAAABlt8hms=
-
PSkbWx4BAAAAOwAAAA4FAAAAAP4AAAAAAAEAAgAD//gGAAAADWFmdGVyX2JhY2t1cDKZoBIS2Q/F
-
WMA=
-
'/*!*/;
-
# at 1294
-
#180609 1:11:25 server id 1 end_log_pos 1325 CRC32 0x8113be5c Xid = 992
-
COMMIT /*!*/;
-
# at 1325
-
#180609 1:42:32 server id 1 end_log_pos 1390 CRC32 0x6c1bff71 Anonymous_GTID last_committed=4 sequence_number=5
-
SET @@SESSION.GTID_NEXT= 'ANONYMOUS'/*!*/;
-
# at 1390
-
#180609 1:42:32 server id 1 end_log_pos 1464 CRC32 0xfb668ab1 Query thread_id=11 exec_time=0 error_code=0
-
SET TIMESTAMP= 1528508552/*!*/;
-
BEGIN
-
/*!*/;
-
# at 1464
-
#180609 1:42:32 server id 1 end_log_pos 1520 CRC32 0x89d622f5 Table_map: `dbtest`.`table1` mapped to number 254
-
# at 1520
-
#180609 1:42:32 server id 1 end_log_pos 1626 CRC32 0x4c8f4330 Delete_rows: table id 254 flags: STMT_END_F
-
-
BINLOG '
-
iDAbWxMBAAAAOAAAAPAFAAAAAP4AAAAAAAEABmRidGVzdAAGdGFibGUxAAMDDxIDeAAABvUi1ok=
-
iDAbWyABAAAAagAAAFoGAAAAAP4AAAAAAAEAAgAD//gEAAAADWFmdGVyX2JpbmxvZzKZoBILJvgF
-
AAAADGFmdGVyX2JhY2t1cJmgEhLS+AYAAAANYWZ0ZXJfYmFja3VwMpmgEhLZMEOPTA==
-
'/*!*/;
上面展示了我們最后一次正常的插入操作和第一次誤刪操作的日志。可以看到誤刪操作是1325開始,1520結束,1:42:32秒一秒鍾之內結束。而最后一次正常插入是1235行結束,1:11:25時候。
然后我們再找全備起始時間也就是740行所對應的時間
-
# at 709
-
#180609 0:44:38 server id 1 end_log_pos 740 CRC32 0x1119b274 Xid = 15
-
COMMIT /*!*/;
-
# at 740
-
#180609 1:11:18 server id 1 end_log_pos 805 CRC32 0x7732118b Anonymous_GTID last_committed=2 sequence_number=3
-
SET @@SESSION.GTID_NEXT= 'ANONYMOUS'/*!*/;
-
# at 805
-
#180609 1:11:18 server id 1 end_log_pos 887 CRC32 0x22d66d83 Query thread_id=10 exec_time=0 error_code=0
-
SET TIMESTAMP= 1528506678/*!*/;
-
BEGIN
-
/*!*/;
但是奇怪的事情是,如果我們按照上面的,--start-position和s--stop-datetime相結合,那么binlog並不是從740行1294行,而是從第四行開始的,如下
-
# mysqlbinlog --no-defaults --start-position=740 --stop-datetime="2018-06-09 01:11:26" /import/mysql/mysql-bin.000001
-
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
-
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
-
DELIMITER /*!*/;
-
# at 4
-
#180609 0:43:43 server id 1 end_log_pos 123 CRC32 0x73c53756 Start: binlog v 4, server v 5.7.18-0ubuntu0.16.04.1-log created 180609 0:43:43 at startup
-
# Warning: this binlog is either in use or was not closed properly.
-
ROLLBACK /*!*/;
-
BINLOG '
-
vyIbWw8BAAAAdwAAAHsAAAABAAQANS43LjE4LTB1YnVudHUwLjE2LjA0LjEtbG9nAAAAAAAAAAAA
-
AAAAAAAAAAAAAAAAAAC/IhtbEzgNAAgAEgAEBAQEEgAAXwAEGggAAAAICAgCAAAACgoKKioAEjQA
-
AVY3xXM=
-
'/*!*/;
-
# at 740
-
#180609 1:11:18 server id 1 end_log_pos 805 CRC32 0x7732118b Anonymous_GTID last_committed=2 sequence_number=3
-
SET @@SESSION.GTID_NEXT= 'ANONYMOUS'/*!*/;
-
# at 805
-
#180609 1:11:18 server id 1 end_log_pos 887 CRC32 0x22d66d83 Query thread_id=10 exec_time=0 error_code=0
-
SET TIMESTAMP= 1528506678/*!*/;
-
SET @@session.pseudo_thread_id= 10/*!*/;
-
SET @@session.foreign_key_checks= 1, @@session.sql_auto_is_null=0, @@session.unique_checks=1, @@session.autocommit=1/*!*/;
-
SET @@session.sql_mode= 1436549152/*!*/;
-
SET @@session.auto_increment_increment= 1, @@session.auto_increment_offset=1/*!*/;
-
/*!\C utf8 *//*!*/;
-
SET @@session.character_set_client= 33,@@session.collation_connection=33,@@session.collation_server=33/*!*/;
-
SET @@session.time_zone= 'SYSTEM'/*!*/;
-
SET @@session.lc_time_names= 0/*!*/;
-
SET @@session.collation_database= DEFAULT/*!*/;
-
BEGIN
-
/*!*/;
-
# at 887
-
#180609 1:11:18 server id 1 end_log_pos 943 CRC32 0x5d815b86 Table_map: `dbtest`.`table1` mapped to number 254
-
# at 943
-
#180609 1:11:18 server id 1 end_log_pos 1001 CRC32 0x5f1763c3 Write_rows: table id 254 flags: STMT_END_F
-
-
BINLOG '
-
NikbWxMBAAAAOAAAAK8DAAAAAP4AAAAAAAEABmRidGVzdAAGdGFibGUxAAMDDxIDeAAABoZbgV0=
-
NikbWx4BAAAAOgAAAOkDAAAAAP4AAAAAAAEAAgAD//gFAAAADGFmdGVyX2JhY2t1cJmgEhLSw2MX
-
Xw==
-
'/*!*/;
-
# at 1001
-
#180609 1:11:18 server id 1 end_log_pos 1032 CRC32 0x38b9cbf4 Xid = 991
-
COMMIT /*!*/;
-
# at 1032
-
#180609 1:11:25 server id 1 end_log_pos 1097 CRC32 0xbf1c5f5e Anonymous_GTID last_committed=3 sequence_number=4
-
SET @@SESSION.GTID_NEXT= 'ANONYMOUS'/*!*/;
-
# at 1097
-
#180609 1:11:25 server id 1 end_log_pos 1179 CRC32 0xeebaef40 Query thread_id=10 exec_time=0 error_code=0
-
SET TIMESTAMP= 1528506685/*!*/;
-
BEGIN
-
/*!*/;
-
# at 1179
-
#180609 1:11:25 server id 1 end_log_pos 1235 CRC32 0x6b867c5b Table_map: `dbtest`.`table1` mapped to number 254
-
# at 1235
-
#180609 1:11:25 server id 1 end_log_pos 1294 CRC32 0xc058c50f Write_rows: table id 254 flags: STMT_END_F
-
-
BINLOG '
-
PSkbWxMBAAAAOAAAANMEAAAAAP4AAAAAAAEABmRidGVzdAAGdGFibGUxAAMDDxIDeAAABlt8hms=
-
PSkbWx4BAAAAOwAAAA4FAAAAAP4AAAAAAAEAAgAD//gGAAAADWFmdGVyX2JhY2t1cDKZoBIS2Q/F
-
WMA=
-
'/*!*/;
-
# at 1294
-
#180609 1:11:25 server id 1 end_log_pos 1325 CRC32 0x8113be5c Xid = 992
-
COMMIT /*!*/;
-
SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/;
-
DELIMITER ;
-
# End of log file
-
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
-
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;
剛才我們看到的是只有1個binlog的情況,而我們知道binlog是會不斷增加的,當有兩個以上的binlog的時候該怎么辦。
沃恩首先到源數據庫上,手動建立新的binlog
-
mysql> select * from table1;
-
+----+---------------+---------------------+
-
| id | name | birthday |
-
+----+---------------+---------------------+
-
| 1 | befor_binlog1 | 2018-06-09 08:36:33 |
-
| 2 | befor_binlog2 | 2018-06-09 08:36:40 |
-
| 3 | after_binlog1 | 2018-06-09 08:44:33 |
-
| 4 | xxxxxxx1 | 2018-06-09 09:43:00 |
-
| 5 | xxxxxxx1 | 2018-06-09 09:43:05 |
-
+----+---------------+---------------------+
-
5 rows in set (0.00 sec)
-
mysql> show master status \G
-
*************************** 1. row ***************************
-
File: mysql-bin.000001
-
Position: 2233
-
Binlog_Do_DB:
-
Binlog_Ignore_DB:
-
Executed_Gtid_Set:
-
1 row in set (0.00 sec)
現在手動創建新的binlog文件
-
mysql> flush logs;
-
Query OK, 0 rows affected (0.30 sec)
-
-
mysql> show master status \G
-
*************************** 1. row ***************************
-
File: mysql-bin.000002
-
Position: 154
-
Binlog_Do_DB:
-
Binlog_Ignore_DB:
-
Executed_Gtid_Set:
-
1 row in set (0.00 sec)
這樣一個新的00002的binlog就開始使用了,起始行數154,並且原來的00001並沒有被刪掉,而是永久的保存在那里。
我們在創建新binlog后插入幾條數據進去
-
mysql> insert into table1 values ( 6,'new_binlog1',NOW());
-
Query OK, 1 row affected (0.13 sec)
-
-
mysql> insert into table1 values ( 7,'new_binlog2',NOW());
-
Query OK, 1 row affected (0.15 sec)
-
-
mysql> select * from table1;
-
+----+---------------+---------------------+
-
| id | name | birthday |
-
+----+---------------+---------------------+
-
| 1 | befor_binlog1 | 2018-06-09 08:36:33 |
-
| 2 | befor_binlog2 | 2018-06-09 08:36:40 |
-
| 3 | after_binlog1 | 2018-06-09 08:44:33 |
-
| 4 | xxxxxxx1 | 2018-06-09 09:43:00 |
-
| 5 | xxxxxxx1 | 2018-06-09 09:43:05 |
-
| 6 | new_binlog1 | 2018-06-09 10:47:21 |
-
| 7 | new_binlog2 | 2018-06-09 10:47:28 |
-
+----+---------------+---------------------+
-
7 rows in set (0.00 sec)
-
mysql> show master status \G
-
*************************** 1. row ***************************
-
File: mysql-bin.000002
-
Position: 736
-
Binlog_Do_DB:
-
Binlog_Ignore_DB:
-
Executed_Gtid_Set:
-
1 row in set (0.00 sec)
現在假設系統崩潰,我們要異地恢復這個數據庫,我們會有之前的一個全備,和兩個binlog
-
mysql> show databases;
-
+--------------------+
-
| Database |
-
+--------------------+
-
| information_schema |
-
| mydb |
-
| mysql |
-
| performance_schema |
-
| sys |
-
+--------------------+
-
5 rows in set (0.00 sec)
還是首先恢復全庫備份,顯而易見,恢復之后只有id=1-4的數據,並且id=4的數據還是舊版本的
-
mysql> use dbtest;
-
Reading table information for completion of table and column names
-
You can turn off this feature to get a quicker startup with -A
-
-
Database changed
-
mysql> select * from table1;
-
+----+---------------+---------------------+
-
| id | name | birthday |
-
+----+---------------+---------------------+
-
| 1 | befor_binlog1 | 2018-06-09 08:36:33 |
-
| 2 | befor_binlog2 | 2018-06-09 08:36:40 |
-
| 3 | after_binlog1 | 2018-06-09 08:44:33 |
-
| 4 | after_binlog2 | 2018-06-09 08:44:38 |
-
+----+---------------+---------------------+
-
4 rows in set (0.00 sec)
然后我們需要按照binlog的順序依次恢復,也就是先重做00001上的,再重做00002上的,由於全備是從00001的740行開始的,所以再重做00001的時候要采用--start-position,而重做00002的時候就不需要了。
# mysqlbinlog --no-defaults --start-position=740 /import/mysql/mysql-bin.000001 | mysql -uroot -p
此時會恢復到我們flush log之前的狀態
-
mysql> select * from table1;
-
+----+---------------+---------------------+
-
| id | name | birthday |
-
+----+---------------+---------------------+
-
| 1 | befor_binlog1 | 2018-06-09 08:36:33 |
-
| 2 | befor_binlog2 | 2018-06-09 08:36:40 |
-
| 3 | after_binlog1 | 2018-06-09 00:44:33 |
-
| 4 | xxxxxxx1 | 2018-06-09 01:43:00 |
-
| 5 | xxxxxxx1 | 2018-06-09 01:43:05 |
-
+----+---------------+---------------------+
-
5 rows in set (0.00 sec)
然后再重做00002上的事務,此時不需要指定--start-postion了,因為00001和00002是連續的。
-
-
error. log mysql-bin.000001 mysql-bin.000002 mysql-bin.index
-
發現我們已經把全部數據都恢復回來了
-
mysql> select * from table1;
-
+----+---------------+---------------------+
-
| id | name | birthday |
-
+----+---------------+---------------------+
-
| 1 | befor_binlog1 | 2018-06-09 08:36:33 |
-
| 2 | befor_binlog2 | 2018-06-09 08:36:40 |
-
| 3 | after_binlog1 | 2018-06-09 00:44:33 |
-
| 4 | xxxxxxx1 | 2018-06-09 01:43:00 |
-
| 5 | xxxxxxx1 | 2018-06-09 01:43:05 |
-
| 6 | new_binlog1 | 2018-06-09 10:47:21 |
-
| 7 | new_binlog2 | 2018-06-09 10:47:28 |
-
+----+---------------+---------------------+
-
7 rows in set (0.00 sec)
同樣,你也可以在恢復00001的時候指定--stop-datetime,但是這樣在重做00002的時候可能會碰到主鍵沖突的問題,需要你自己去把握了。
上面都是DML操作,通過binlog可以准確無誤的還原回來,下面再執行一個DDL操作,看看binlog是否還有用
首先我們建立一張新表table2,然后drop掉table1,之后看看能否異地恢復。然后我們再考慮drop table1是一個誤操作的情況下,能否異地恢復。
-
mysql> create table table2 (
-
-> id int primary key,
-
-> hobby varchar(40),
-
-> starttime datetime)
-
-> ;
-
Query OK, 0 rows affected (0.55 sec)
-
-
mysql> insert into table2 values(1,'play',NOW());
-
Query OK, 1 row affected (0.12 sec)
-
-
mysql> select * from table2;
-
+----+-------+---------------------+
-
| id | hobby | starttime |
-
+----+-------+---------------------+
-
| 1 | play | 2018-06-09 11:06:07 |
-
+----+-------+---------------------+
-
1 row in set (0.00 sec)
-
mysql> show master status \G
-
*************************** 1. row ***************************
-
File: mysql-bin.000002
-
Position: 1243
-
Binlog_Do_DB:
-
Binlog_Ignore_DB:
-
Executed_Gtid_Set:
-
1 row in set (0.00 sec)
-
mysql> drop table table1;
-
Query OK, 0 rows affected (0.42 sec)
-
-
mysql> show master status \G
-
*************************** 1. row ***************************
-
File: mysql-bin.000002
-
Position: 1431
-
Binlog_Do_DB:
-
Binlog_Ignore_DB:
-
Executed_Gtid_Set:
-
1 row in set (0.00 sec)
可以看到,drop操作也被寫入到了binlog。下面我們開始異地恢復
-
mysql> show databases;
-
+--------------------+
-
| Database |
-
+--------------------+
-
| information_schema |
-
| mydb |
-
| mysql |
-
| performance_schema |
-
| sys |
-
+--------------------+
-
5 rows in set (0.00 sec)
我們先進行全備份的恢復,然后恢復00001,然后恢復00002,其中先恢復到create table2並插入數據完成的階段,再恢復到drop table1之后的階段,全面檢查DML的可恢復性
-
-
Enter password:
-
-
Enter password:
-
mysql> use dbtest;
-
Reading table information for completion of table and column names
-
You can turn off this feature to get a quicker startup with -A
-
-
Database changed
-
mysql> select * from table1;
-
+----+---------------+---------------------+
-
| id | name | birthday |
-
+----+---------------+---------------------+
-
| 1 | befor_binlog1 | 2018-06-09 08:36:33 |
-
| 2 | befor_binlog2 | 2018-06-09 08:36:40 |
-
| 3 | after_binlog1 | 2018-06-09 00:44:33 |
-
| 4 | after_binlog2 | 2018-06-09 00:44:38 |
-
+----+---------------+---------------------+
-
4 rows in set (0.00 sec)
-
-
mysql> select * from table2;
-
ERROR 1146 (42S02): Table 'dbtest.table2' doesn't exist
然后恢復00001的全部內容和00002中drop table1之前的內容,由於表2的第一條數據是11:06:07插入成功的,所以我們要恢復到11:06:08才能看到着一條數據,恢復到11:06:07是看不到的。
-
# mysqlbinlog --no-defaults --start-position=740 /import/mysql/mysql-bin.000001 | mysql -uroot -p
-
Enter password:
-
-
# mysqlbinlog --no-defaults --stop-datetime='2018-06-09 03:06:08' /import/mysql/mysql-bin.000002 | mysql -uroot -p
-
Enter password:
-
root@5d41cbeb4b4d :/import/mysql# mysql -uroot -p
-
Enter password:
-
-
mysql> use dbtest;
-
Reading table information for completion of table and column names
-
You can turn off this feature to get a quicker startup with -A
-
-
Database changed
-
mysql> select * from table1;
-
+----+---------------+---------------------+
-
| id | name | birthday |
-
+----+---------------+---------------------+
-
| 1 | befor_binlog1 | 2018-06-09 08:36:33 |
-
| 2 | befor_binlog2 | 2018-06-09 08:36:40 |
-
| 3 | after_binlog1 | 2018-06-09 00:44:33 |
-
| 4 | xxxxxxx1 | 2018-06-09 01:43:00 |
-
| 5 | xxxxxxx1 | 2018-06-09 01:43:05 |
-
| 6 | new_binlog1 | 2018-06-09 10:47:21 |
-
| 7 | new_binlog2 | 2018-06-09 10:47:28 |
-
+----+---------------+---------------------+
-
7 rows in set (0.00 sec)
-
-
mysql> select * from table2;
-
+----+-------+---------------------+
-
| id | hobby | starttime |
-
+----+-------+---------------------+
-
| 1 | play | 2018-06-09 11:06:07 |
-
+----+-------+---------------------+
-
1 row in set (0.00 sec)
此時table1和table2的數據都已經異地恢復成功,可見即使源數據庫執行了刪表操作也是不妨礙我們恢復的。下面我們再恢復到和源數據庫一模一樣的狀態,就是把上一句的--stop-datetime換成--start-datetime。
# mysqlbinlog --no-defaults --start-datetime='2018-06-09 03:06:08' /import/mysql/mysql-bin.000002 | mysql -uroot -p
-
mysql> select * from table1;
-
ERROR 1146 (42S02): Table 'dbtest.table1' doesn't exist
-
mysql> select * from table2;
-
+----+-------+---------------------+
-
| id | hobby | starttime |
-
+----+-------+---------------------+
-
| 1 | play | 2018-06-09 11:06:07 |
-
+----+-------+---------------------+
-
1 row in set (0.00 sec)
如何簡化日常備份
還有一種特殊的備份方式,就是再mysqldump語句中加入--flush-logs這樣的話會把當前沒有寫滿的binlog停止,另起一個新的binlog來寫,這樣就不用在重做binlog的時候添加--start-position語句了,如下
-
mysql> show master status \G
-
*************************** 1. row ***************************
-
File: mysql-bin .000002
-
Position: 1431
-
Binlog_Do_DB:
-
Binlog_Ignore_DB:
-
Executed_Gtid_Set:
-
1 row in set (0.00 sec)
-
-
mysql> quit
-
Bye
-
root@f4d417a2e6ea:/# mysqldump --single-transaction --master-data=2 --triggers --routines --flush-logs --flush-privileges --databases dbtest -p > /backup/full2.sql
-
Enter password:
在備份的時候還加入了--flush-privileges,這個是在恢復的時候能夠自動賦予相關用戶相關權限,如果不加這個更適合主從復制遷移數據 --databases dbtest 是只備份 dbtest這個庫,減少備份提及,但是mysql里面的user就會不一致了,這樣即使加了--flush-privileges,用戶權限還是會丟失,所以這樣適合單機恢復,就是在出問題后將當前mysql的庫drop掉,然后再執行導入。
在備份完畢之后,產生了一個新的binlog 00003,並從154行開始
-
mysql> show master status \G
-
*************************** 1. row ***************************
-
File: mysql-bin.000003
-
Position: 154
-
Binlog_Do_DB:
-
Binlog_Ignore_DB:
-
Executed_Gtid_Set:
-
1 row in set (0.01 sec)
此時我們查看我們備份出來的這個文件,看他的起始文件和位置
-
-
......
-
--
-
-- Position to start replication or point-in-time recovery from
-
--
-
-
-- CHANGE MASTER TO MASTER_LOG_FILE='mysql-bin.000003', MASTER_LOG_POS=154;
能看到和備份完之后的master狀態是一樣的。
然后我們在源庫里添加幾行數據
-
mysql> use dbtest;
-
Reading table information for completion of table and column names
-
You can turn off this feature to get a quicker startup with -A
-
-
Database changed
-
mysql> insert into table2 values(2,'hahaha',NOW());
-
Query OK, 1 row affected (0.14 sec)
-
-
mysql> insert into table2 values(3,'enenen',NOW());
-
Query OK, 1 row affected (0.14 sec)
-
-
mysql> select * from table2;
-
+----+--------+---------------------+
-
| id | hobby | starttime |
-
+----+--------+---------------------+
-
| 1 | play | 2018-06-09 11:06:07 |
-
| 2 | hahaha | 2018-06-09 04:09:17 |
-
| 3 | enenen | 2018-06-09 04:09:34 |
-
+----+--------+---------------------+
-
3 rows in set (0.00 sec)
-
-
mysql> show master status \G
-
*************************** 1. row ***************************
-
File: mysql-bin .000003
-
Position: 726
-
Binlog_Do_DB:
-
Binlog_Ignore_DB:
-
Executed_Gtid_Set:
-
1 row in set (0.00 sec)
可以看到00003到了726行了。現在我們假設源庫崩潰,然后我們把full2.sql和binlog 00003都拷貝到另一台服務器上。
開始恢復全備
# mysql -uroot -p < /import/full2.sql
-
mysql> select * from table2;
-
+----+-------+---------------------+
-
| id | hobby | starttime |
-
+----+-------+---------------------+
-
| 1 | play | 2018-06-09 11:06:07 |
-
+----+-------+---------------------+
-
1 row in set (0.00 sec)
不出所料只有一行數據,現在我們要重做00003,由於之前--flush-logs的作用,我們雖然從157行開始,但是無需指定--start-positon了,簡化了數據庫恢復的過程,而且由於重新啟用一個binlog,之前的0001,00002就都可以刪掉或者轉移走保存起來,節省服務器上的寶貴空間,不然的話我們也許還要等00002寫滿之后才能轉移走。
-
# mysqlbinlog --no-defaults / import/mysql/mysql-bin.000003 | mysql -uroot -p
-
Enter password:
-
mysql> select * from table2;
-
+----+--------+---------------------+
-
| id | hobby | starttime |
-
+----+--------+---------------------+
-
| 1 | play | 2018-06-09 11:06:07 |
-
| 2 | hahaha | 2018-06-09 04:09:17 |
-
| 3 | enenen | 2018-06-09 04:09:34 |
-
+----+--------+---------------------+
-
3 rows in set (0.00 sec)