mysql 無意重啟 [Note] /usr/sbin/mysqld: Normal shutdown

情況：

今早發現，昨天下午安裝的4台mysql服務器，突然出現，由於在shell窗口

(root@localhost:mysql.sock) [(none)]> 190102 18:12:16 mysqld_safe mysqld from pid file /home/data/mysqldata/3306/data/mysql3.pid ended

什么情況，mysql意外關閉，4個shell窗口都出現了同樣的錯誤，還是有點緊張，於是趕緊找error日志

[mysql@mysql3 scripts]$ tail -f -n 200 /home/data/mysqldata/3306/log/mysql-error.log

2019-01-02 17:24:21 3654 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.6.15-log'  socket: '/home/data/mysqldata/3306/mysql.sock'  port: 3306  MySQL Community Server - (GPL)
2019-01-02 17:24:21 3654 [Note] Event Scheduler: scheduler thread started with id 1
2019-01-02 17:27:46 3654 [Warning] 'proxies_priv' entry '@ root@mysql1' ignored in --skip-name-resolve mode.
2019-01-02 18:12:05 3654 [Note] /usr/sbin/mysqld: Normal shutdown

2019-01-02 18:12:05 3654 [Note] Giving 3 client threads a chance to die gracefully
2019-01-02 18:12:05 3654 [Note] Event Scheduler: Killing the scheduler thread, thread id 1
2019-01-02 18:12:05 3654 [Note] Event Scheduler: Waiting for the scheduler thread to reply
2019-01-02 18:12:05 3654 [Note] Event Scheduler: Stopped
2019-01-02 18:12:05 3654 [Note] Event Scheduler: Purging the queue. 0 events
2019-01-02 18:12:05 3654 [Note] Shutting down slave threads
2019-01-02 18:12:15 3654 [Note] Forcefully disconnecting 1 remaining clients
2019-01-02 18:12:15 3654 [Warning] /usr/sbin/mysqld: Forcing close of thread 6  user: 'root'

2019-01-02 18:12:15 3654 [Note] Binlog end
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'partition'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'BLACKHOLE'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_SYS_DATAFILES'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_SYS_TABLESPACES'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN_COLS'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_SYS_FIELDS'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_SYS_COLUMNS'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_SYS_INDEXES'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_SYS_TABLESTATS'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_SYS_TABLES'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_FT_INDEX_TABLE'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_FT_INDEX_CACHE'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_FT_CONFIG'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_FT_BEING_DELETED'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_FT_DELETED'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_FT_DEFAULT_STOPWORD'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_METRICS'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_BUFFER_POOL_STATS'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE_LRU'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX_RESET'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_CMPMEM_RESET'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_CMPMEM'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_CMP_RESET'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_CMP'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_LOCK_WAITS'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_LOCKS'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_TRX'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'InnoDB'
2019-01-02 18:12:15 3654 [Note] InnoDB: FTS optimize thread exiting.
2019-01-02 18:12:15 3654 [Note] InnoDB: Starting shutdown...
2019-01-02 18:12:16 3654 [Note] InnoDB: Shutdown completed; log sequence number 7120303
2019-01-02 18:12:16 3654 [Note] Shutting down plugin 'ARCHIVE'
2019-01-02 18:12:16 3654 [Note] Shutting down plugin 'PERFORMANCE_SCHEMA'
2019-01-02 18:12:16 3654 [Note] Shutting down plugin 'MRG_MYISAM'
2019-01-02 18:12:16 3654 [Note] Shutting down plugin 'MyISAM'
2019-01-02 18:12:16 3654 [Note] Shutting down plugin 'MEMORY'
2019-01-02 18:12:16 3654 [Note] Shutting down plugin 'CSV'
2019-01-02 18:12:16 3654 [Note] Shutting down plugin 'sha256_password'
2019-01-02 18:12:16 3654 [Note] Shutting down plugin 'mysql_old_password'
2019-01-02 18:12:16 3654 [Note] Shutting down plugin 'mysql_native_password'
2019-01-02 18:12:16 3654 [Note] Shutting down plugin 'binlog'
2019-01-02 18:12:16 3654 [Note] /usr/sbin/mysqld: Shutdown complete

190102 18:12:16 mysqld_safe mysqld from pid file /home/data/mysqldata/3306/data/mysql3.pid ended
190102 18:13:32 mysqld_safe Starting mysqld daemon with databases from /home/data/mysqldata/3306/data 2019-01-02 18:13:34 4770 [Note] Plugin 'FEDERATED' is disabled.

如上面的日志標記所示，4台都是一樣，很奇怪為什么會發現這樣的問題

系統：cpu 4核，內存18g，磁盤hdd 200g

[root@mysql3 ~]# cat /etc/redhat-release
Kylin Linux release 3.3.1707 (Core)

mysql version：

Server version: 5.6.15-log MySQL Community Server - (GPL)

剛安裝好的機器，不應該出現這樣的問題，但是除了我，又沒有誰來搞這個mysql，看了 linux的history

[root@mysql3 ~]# export HISTTIMEFORMAT='%F %T '
  105  2019-01-02 15:55:27 whereis mysql
  106  2019-01-02 15:55:31 which mysql
  107  2019-01-02 15:57:08 vim /home/data/mysqldata/3306/my.cnf
  108  2019-01-02 16:00:39 chown -R mysql:mysql /home/data/mysqldata/
  109  2019-01-02 16:00:43 su - mysql
  110  2019-01-03 09:00:19 history
  111  2019-01-03 09:00:44 su - mysql
  112  2019-01-03 10:06:49 cd /home/so
  113  2019-01-03 10:06:54 cd /home/

沒有相應時間的history

[root@mysql1 3306]# su - mysql
[mysql@mysql1 ~]$ history
   66  2019-01-02 17:41:10 cd ../scripts/
   67  2019-01-02 17:41:11 ll
   68  2019-01-02 17:44:00 mysql -uroot -p -S /home/data/mysqldata/3306/mysql.sock
   69  2019-01-03 08:38:13 ll
   70  2019-01-03 08:38:15 cd ..
   71  2019-01-03 08:38:17 cd 3306/log/

找不到問題，這時候開始網上找找，發現有一些人遇到這個問題

2019-01-02 18:09:56 3549 [Note] /usr/sbin/mysqld: Normal shutdown

但是回答的大部分都是，找找是否有history |grep mysqladmin 或者service mysqld stop等命令，看看是否手動或者其他的關閉了mysql

因為從error上看，是正常的關閉mysql服務

查看slow log也沒有什么發現

還看到有一個帖子

https://dba.stackexchange.com/questions/40581/usr-libexec-mysqld-normal-shutdown-but-my-team-doesnt-do-that

有提到bug的帖子

https://bugs.mysql.com/bug.php?id=51081

由於還是找不到原因，尋思許久，不得解

這是問了一下開發同學 是否有重啟過數據庫，一問，才發現，他們悄悄的修改了my.cnf，然后同時重啟了這4台機器。。。。。

哎，搞了半天，。。被開發的重啟了，

 

這里警示了一個重要的問題，數據庫就讓dba團隊來管控，開發只需要提出需求就好，而不是自己想當然的進行修改，，

按理說我們已經有 開發規范，管理規范，這種還是控制不住，畢竟。。。