線上數據庫服務器上mysql運行一段時間了,突然出現了異常:啟動mysql后隨即就又關閉了,mysql服務啟動失敗!!
查看mysql錯誤日志如下:
160920 22:41:41 mysqld_safe Starting mysqld daemon with databases from /home/MysqlData/
2016-09-20 22:41:41 0 [Note] /Data/app/mysql5.6.25/bin/mysqld (mysqld 5.6.25-log) starting as process 32372 ...
2016-09-20 22:41:42 32372 [Note] Plugin 'FEDERATED' is disabled.
2016-09-20 22:41:42 32372 [Warning] option 'innodb-write-io-threads': unsigned value 1000 adjusted to 64
2016-09-20 22:41:42 32372 [Warning] option 'innodb-read-io-threads': unsigned value 1000 adjusted to 64
2016-09-20 22:41:42 32372 [Note] InnoDB: Using atomics to ref count buffer pool pages
2016-09-20 22:41:42 32372 [Note] InnoDB: The InnoDB memory heap is disabled
2016-09-20 22:41:42 32372 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2016-09-20 22:41:42 32372 [Note] InnoDB: Memory barrier is not used
2016-09-20 22:41:42 32372 [Note] InnoDB: Compressed tables use zlib 1.2.3
2016-09-20 22:41:42 32372 [Note] InnoDB: Using CPU crc32 instructions
2016-09-20 22:41:42 32372 [Note] InnoDB: Initializing buffer pool, size = 1.0G
2016-09-20 22:41:42 32372 [Note] InnoDB: Completed initialization of buffer pool
2016-09-20 22:41:42 32372 [Note] InnoDB: Highest supported file format is Barracuda.
2016-09-20 22:41:42 32372 [Note] InnoDB: Log scan progressed past the checkpoint lsn 20293587957
2016-09-20 22:41:42 32372 [Note] InnoDB: Database was not shutdown normally!
2016-09-20 22:41:42 32372 [Note] InnoDB: Starting crash recovery.
2016-09-20 22:41:42 32372 [Note] InnoDB: Reading tablespace information from the .ibd files...
2016-09-20 22:41:42 32372 [Note] InnoDB: Restoring possible half-written data pages
2016-09-20 22:41:42 32372 [Note] InnoDB: from the doublewrite buffer...
InnoDB: Doing recovery: scanned up to log sequence number 20293596130
2016-09-20 22:41:42 32372 [Note] InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percent: 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
InnoDB: Apply batch completed
InnoDB: Last MySQL binlog file position 0 136254, file name mysql-bin.000013
2016-09-20 22:41:43 32372 [Note] InnoDB: 128 rollback segment(s) are active.
2016-09-20 22:41:43 32372 [Note] InnoDB: Waiting for purge to start
2016-09-20 22:41:43 7f77a9edd700 InnoDB: Assertion failure in thread 140151928772352 in file trx0purge.cc line 699
InnoDB: Failing assertion: purge_sys->iter.trx_no <= purge_sys->rseg->last_trx_no
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
02:41:43 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
分析日志后發現,數據庫無法重啟的原因是因為ibdata1文件 (即共享表空間) 損壞,重啟后無法正常恢復。
解決辦法:
需要跳過恢復步驟,修改my.cnf文件,在my.cnf中的[mysqld]中添加:
innodb_force_recovery = 6
innodb_purge_threads = 1
修改后,接着重啟Mysql服務,發現可正常啟動。如果還是無法啟動,則就需要刪除mysql數據目錄下的 "ibdata1、ib_logfile*" 等文件 (刪除前,提前做好備份),然后再做Mysql服務啟動操作!!
######################## innodb_force_recovery參數說明 ########################
MySQL數據庫當innodb表空間損壞時(如ibdata1文件損壞),嘗試啟動Mysql服務失敗。這個時候可以使用innodb_force_recovery參數進行強制啟動!!下面說下這個參數的用法:
innodb_force_recovery參數解釋: ------------------------------------------------------------------------------------------------------------- innodb_force_recovery影響整個InnoDB存儲引擎的恢復狀況,默認值為0,表示當需要恢復時執行所有的恢復操作!! 當不能進行有效的恢復操作時,Mysql有可能無法啟動,並記錄下錯誤日志。 innodb_force_recovery可以設置為1-6,大的數字包含前面所有數字的影響。 當該參數的數值設置大於0后,可以對表進行select,create,drop操作,但insert,update或者delete這類操作是不允許的。 innodb_force_recovery=0 表示當需要恢復時執行所有的恢復操作; innodb_force_recovery=1 表示忽略檢查到的corrupt頁; innodb_force_recovery=2 表示阻止主線程的運行,如主線程需要執行full purge操作,會導致crash; innodb_force_recovery=3 表示不執行事務回滾操作; innodb_force_recovery=4 表示不執行插入緩沖的合並操作; innodb_force_recovery=5 表示不查看重做日志,InnoDB存儲引擎會將未提交的事務視為已提交; innodb_force_recovery=6 表示不執行前滾的操作,強制重啟! ------------------------------------------------------------------------------------------------------------- 記一次事故: 線上Mysql環境采用一主兩從模式,突然一天上午發現主從庫的Mysql服務都啟動失敗,最后排查是Mysql共享表空間ibdata1文件損壞了。 恢復記錄: 1) 在主從庫的Mysql配置文件my.cnf中添加 # vim /etc/my.cnf innodb_force_recovery=6 2) 啟動mysql服務 # service mysqld start 3)如果啟動成功后,再將my.cnf文件中的"innodb_force_recovery=6"配置去掉,然后再次嘗試mysql服務的重啟,應該OK。 在主從庫出現這種情況時,如果配置文件里之前就有這個參數,則嘗試將該參數值修改為0或6,依次嘗試重啟。
