轉 mysql 文件系統空間滿了

 

#######################sample

【OIP - 互聯網開放平台】在2019-07-28 21:30:11發生10.194.42.19 - - Linux上的監控項【磁盤空間】【Free disk space on /db/mysql】產生了異常: 【嚴重】: 空閑磁盤空間低於10%, /db/mysql. 當前值: 9.6740 【珠海華潤銀行】

 

 

原因：

 在問題出現時,登錄MySQL,執行show processlist查看是否存在異常SQL.

1.mysql -uroot --socket=/db/mysql/data/mysqltmp/mysql.sock -p

Enter password:

mysql> show processlist

    -> ;

+-----+-------------+-----------+------+---------+----------+-----------------------------------------------------------------------------+------------------+

| Id  | User        | Host      | db   | Command | Time     | State                                                                       | Info             |

+-----+-------------+-----------+------+---------+----------+-----------------------------------------------------------------------------+------------------+

|   2 | system user |           | NULL | Connect | 24025479 | Waiting for master to send event                                            | NULL             |

3 rows in set (0.00 sec)

 

2. mysql慢查詢日志

mysql> show variables like '%slow%';

+---------------------------+------------------------------------------+

| Variable_name             | Value                                    |

+---------------------------+------------------------------------------+

| log_slow_admin_statements | OFF                                      |

| log_slow_slave_statements | OFF                                      |

| slow_launch_time          | 2                                        |

| slow_query_log            | ON                                       |

| slow_query_log_file       | /db/mysql/data/mydata/mysql-slow.log |

+---------------------------+------------------------------------------+

 

mysql> show variables like '%long%';

| long_query_time                                        | 2.000000 |

慢查詢時間為2s，

 

3.配置如下：

mysql> show variables like 'default_storage_engine';

+------------------------+--------+

| Variable_name          | Value  |

+------------------------+--------+

| default_storage_engine | InnoDB |

+------------------------+--------+

1 row in set (0.00 sec)

 

mysql> show variables like 'transaction_isolation';

Empty set (0.00 sec)

 

mysql> show variables like 'binlog_format';

+---------------+-------+

| Variable_name | Value |

+---------------+-------+

| binlog_format | MIXED |

+---------------+-------+

1 row in set (0.00 sec)

 

mysql> show variables like 'binlog_cache_size';

+-------------------+---------+

| Variable_name     | Value   |

+-------------------+---------+

| binlog_cache_size | 2097152 |

+-------------------+---------+

1 row in set (0.00 sec)

 

mysql> show variables like 'max_binlog_cache_size';

+-----------------------+----------------------+

| Variable_name         | Value                |

+-----------------------+----------------------+

| max_binlog_cache_size | 18446744073709547520 |

+-----------------------+----------------------+

1 row in set (0.00 sec)

 

mysql> show variables like 'tmpdir';

+---------------+-------------------------------+

| Variable_name | Value                         |

+---------------+-------------------------------+

| tmpdir        | /db/mysql/data/mydata/tmp |

+---------------+-------------------------------+

1 row in set (0.00 sec)

 

mysql> show variables like 'transaction_isolation';

Empty set (0.00 sec)

 

4.查看數據庫容量

mysql> select

    -> table_schema as 'db',

    -> sum(table_rows) as 'count',

    -> sum(truncate(data_length/1024/1024, 2)) as 'data(MB)',

    -> sum(truncate(index_length/1024/1024, 2)) as 'index(MB)'

    -> from information_schema.tables

    -> group by table_schema

    -> order by sum(data_length) desc, sum(index_length) desc;

+--------------------+----------+----------+-----------+

| db                 | count    | data(MB) | index(MB) |

+--------------------+----------+----------+-----------+

| openapi            | 21915376 | 11710.97 |   1216.19 |

| mysql              |     2697 |     0.76 |      0.06 |

| information_schema |     NULL |     0.00 |      0.00 |

| performance_schema |    74026 |     0.00 |      0.00 |

+--------------------+----------+----------+-----------+

4 rows in set (1.28 sec)

 

mysql> select

    -> table_schema as 'db',

    -> table_name as 'table',

    -> table_rows as 'count',

    -> truncate(data_length/1024/1024, 2) as 'data(MB)',

    -> truncate(index_length/1024/1024, 2) as 'index(MB)'

    -> from information_schema.tables

    -> order by data_length desc, index_length desc;

+--------------------+----------------------------------------------------+---------+----------+-----------+

| db                 | table                                              | count   | data(MB) | index(MB) |

+--------------------+----------------------------------------------------+---------+----------+-----------+

| openapi            | log_r_esb                                          | 5312774 |  1957.00 |      0.00 |

| openapi            | api_log_sys_0                                      | 2886067 |  1727.39 |    218.74 |

| openapi            | api_log_sys_6                                      | 2795942 |  1673.90 |    217.91 |

| openapi            | api_log_sys_5                                      | 2205103 |  1319.96 |    165.51 |

| openapi            | api_log_sys_2                                      | 2114416 |  1264.59 |    163.54 |

| openapi            | api_log_sys_3                                      | 2017041 |  1206.44 |    156.33 |

| openapi            | api_log_sys_4                                      | 1910052 |  1142.62 |    143.61 |

| openapi            | api_log_sys_1                                      | 1825220 |  1091.11 |    141.35 |

 

6.查看文件系統目錄大小

[root@poipredis03 openapi]# du -s * |sort -nr

5971972 log_r_esb.ibd

1768856 api_log_sys_0.MYD

1714084 api_log_sys_6.MYD

1351648 api_log_sys_5.MYD

1294952 api_log_sys_2.MYD

1235412 api_log_sys_3.MYD

1179140 api_log_sys_1.MYD

1170056 api_log_sys_4.MYD

 

7．查看大表的engine

mysql> select table_catalog

    ->      ,table_schema

    ->     ,table_name

    ->    ,engine

    ->    from information_schema.tables

    ->    where table_schema='openapi' and table_name='log_r_esb';

+---------------+--------------+------------+--------+

| table_catalog | table_schema | table_name | engine |

+---------------+--------------+------------+--------+

| def           | openapi      | log_r_esb  | InnoDB |

+---------------+--------------+------------+--------+

1 row in set (0.27 sec)

 

可能是因為表碎片，刪除了數據，OS文件並不會收縮，但是表的大小是會變小的。所以就造成了OS文件里有了碎片化空間。

 

解決思路：

1.建議找操作系統人員擴容該目錄/db/mysql

 

 

 

##########################

MySQL查看數據庫表容量大小

https://cloud.tencent.com/info/429e1f0cadbc960659e7a43232450ae0.html

【https://blog.csdn.net/fdipzone/article/details/80144166】

1.查看所有數據庫容量大小

1 2 3 4 5 6 7 8

select table_schema  as  '數據庫' , sum(table_rows)  as  '記錄數' , sum(truncate(data_length/1024/1024, 2))  as  '數據容量(MB)' , sum(truncate(index_length/1024/1024, 2))  as  '索引容量(MB)' from  information_schema.tables group  by  table_schema order  by  sum(data_length) desc, sum(index_length) desc;

　　

2.查看所有數據庫各表容量大小

1 2 3 4 5 6 7 8

select table_schema  as  '數據庫' , table_name  as  '表名' , table_rows  as  '記錄數' , truncate(data_length/1024/1024, 2)  as  '數據容量(MB)' , truncate(index_length/1024/1024, 2)  as  '索引容量(MB)' from  information_schema.tables order  by  data_length desc, index_length desc;

　　

3.查看指定數據庫容量大小

例：查看mysql庫容量大小

1 2 3 4 5 6 7

select table_schema  as  '數據庫' , sum(table_rows)  as  '記錄數' , sum(truncate(data_length/1024/1024, 2))  as  '數據容量(MB)' , sum(truncate(index_length/1024/1024, 2))  as  '索引容量(MB)' from  information_schema.tables where  table_schema= 'mysql' ;

　　

4.查看指定數據庫各表容量大小

例：查看mysql庫各表容量大小

1 2 3 4 5 6 7 8 9

select table_schema  as  '數據庫' , table_name  as  '表名' , table_rows  as  '記錄數' , truncate(data_length/1024/1024, 2)  as  '數據容量(MB)' , truncate(index_length/1024/1024, 2)  as  '索引容量(MB)' from  information_schema.tables where  table_schema= 'mysql' order  by  data_length desc, index_length desc;

 

查詢整個MySQL實例里面存儲引擎為MyISAM的表

 

select table_catalog

      ,table_schema

      ,table_name

      ,engine

from information_schema.tables

where engine='MyISAM';

 

查詢MyDB數據庫里面存儲引擎為MyISAM的表

 

select table_catalog

      ,table_schema

      ,table_name

      ,engine

from information_schema.tables

where table_schema='MyDB' and engine='MyISAM';

###########################

http://blog.chinaunix.net/uid-16728139-id-3495657.html

案例分享-MySQL服務器/tmp目錄被占滿

 

分類： Mysql/postgreSQL

2013-02-22 14:48:20

原文地址：案例分享-MySQL服務器/tmp目錄被占滿 作者：ning_lianjie

 

案例分享-MySQL服務器/tmp目錄被占滿

描述:

MySQL服務器在每天的22點/tmp目錄磁盤空間被占滿,持續10分鍾左右,然后自動恢復./tmp目錄大小10G,平時可用空間8G左右.MySQL版本 5.5

分析

1.       在問題出現時,進入/tmp目錄,ls –al查看具體文件.

2.       在問題出現時,登錄MySQL,執行show processlist查看是否存在異常SQL.

3.       查看MySQL慢查詢日志.

4.       MySQL配置情況:

a)         default_storage_engine         = InnoDB

b)         transaction_isolation           = READ-COMMITTED

c)         binlog_format                 = mixed

d)         binlog_cache_size              = 32K

e)         max_binlog_cache_size         = 18446744073709547520

f)          tmpdir                       = /tmp

總結

1.       InnoDB存儲引擎,在READ-COMMITTED事務隔離級別的情況下(默認的級別是REPEATABLE-READ),普通的DELETE操作,在記錄binlog的時候,會采用ROW模式.(暫時還不清楚原因,以后分析).

2.       程序在每天的22點,有一個清理的定時任務.自動刪除R表的數據,如下:

delete from R where time < xxx;

將某天之前的數據清除.但是該表比較大,近50G.

3.       MySQL參數

binlog_cache_size

max_binlog_cache_size

參考http://dev.mysql.com/doc/refman/5.5/en/replication-options-binary-log.html

4.       每次執行定時任務的時候,因為binlog記錄的是ROW模式,再加上表的數據量比較大,binlog緩存一定會超過32K,結果就會在/tmp目錄下生成臨時文件(參考: When a thread that handles the transaction starts, it allocates a buffer of binlog_cache_size to buffer statements. If a statement is bigger than this, the thread opens a temporary file to store the transaction. The temporary file is deleted when the thread ends),MySQL默認配置,在64位系統情況下,binlog文件大小最大可以達到16EB.但是系統的/tmp目錄是10G,所以事務執行一半,磁盤空間被占滿,事務回滾.

5.       事后查看R表的數據以及binlog記錄,驗證了第4步的推論.

解決

在沒有新數據寫入的前提下,把確定保留的數據先放到新表里面,然后刪除舊表,再把新表重命名.

create table R_20130220 select * FROM R where  time >= xxx;

DROP TABLE R;

RENAME TABLE R_20130220 TO R;

隔天觀察nagios和cacti監控,故障恢復.

https://www.2cto.com/database/201303/192747.html

 

######### binglog 太大，導致磁盤空間太滿了。

mysql開啟BINGLOG后日志文件增長比較快，很快占滿磁盤空間。

lrm929 關注 0人評論 441人閱讀2013-04-19 14:39:01

 

 mysql開啟BINGLOG后日志文件增長比較快，很快占滿磁盤空間。

 

cd  /mysql/data/mydata

ls -ltr 

-rw-rw---- 1 mysql mysql 1073742040 Jul 21 20:20 mysql-bin.000036
-rw-rw---- 1 mysql mysql 1073741977 Jul 23 21:33 mysql-bin.000037
-rw-rw---- 1 mysql mysql 1073742172 Jul 26 12:45 mysql-bin.000038
-rw-rw---- 1 mysql mysql 1073742485 Jul 28 10:45 mysql-bin.000039

 

通過以下幾種方式刪除日志文件：
一、設置日志保留時長expire_logs_days自動刪除
查看當前日志保存天數：

show variables like '%expire_logs_days%';

這個默認是0，也就是logs不過期，可通過設置全局的參數，使他臨時生效：

set global expire_logs_days=7;

設置了只保留7天BINLOG, 下次重啟mysql這個參數默認會失敗，所以需在my.cnf中設置

expire_logs_days = 7

二、手動刪除BINLOG (purge binary logs)
用於刪除列於在指定的日志或日期之前的日志索引中的所有二進制日志。這些日志也會從記錄在日志索引文件

 PURGE {MASTER | BINARY} LOGS TO ‘log_name’
 PURGE {MASTER | BINARY} LOGS BEFORE ‘date’

例如：

PURGE MASTER LOGS TO ‘mysql-bin.010′;
PURGE MASTER LOGS BEFORE ‘2008-06-22 13:00:00′;
PURGE MASTER LOGS BEFORE DATE_SUB( NOW( ), INTERVAL 3 D

 

生產上有一個比較小的系統，發現mysql占用空間較大，經排查發現是binlog比較多引起的

查看binlog過期時間，設置的時間為90天，這個值默認是0天，也就是說不自動清理，可以根據生產情況修改，本例修改為7天

mysql> show variables like 'expire_logs_days';
+------------------+-------+
| Variable_name | Value |
+------------------+-------+
| expire_logs_days | 90 |
+------------------+-------+
1 row in set (0.00 sec)

mysql> set global expire_logs_days=7;
Query OK, 0 rows affected (0.00 sec)

設置之后不會立即清除，觸發條件是：

 

binlog大小超過max_binlog_size
手動執行flush logs
重新啟動時(MySQL將會new一個新文件用於記錄binlog)

我們執行flush logs;

mysql> flush logs;
Query OK, 0 rows affected, 64 warnings (0.16 sec

如果binlog非常多，不要輕易設置改參數，有可能導致io爭用，這時候可以使用purge命令予以清除:

 

將bin.000055之前的binlog清掉:

mysql>purge binary logs to 'bin.000055';

將指定時間之前的binlog清掉:

 

mysql>purge binary logs before '2017-05-01 13:09:51';

 

mysql 日志自動切換

 

感謝ATCO

 

有三種解決方法：1.關閉mysql主從，關閉binlog；2.開啟mysql主從，設置expire_logs_days；3.手動清除binlog文件，> PURGE MASTER LOGS TO ‘MySQL-bin.010′;

實現：
1.關閉mysql主從，關閉binlog

# vim /etc/my.cnf  //注釋掉log-bin,binlog_format # Replication Master Server (default) # binary logging is required for replication # log-bin=mysql-bin # binary logging format - mixed recommended # binlog_format=mixed

然后重啟數據庫

2.重啟mysql,開啟mysql主從，設置expire_logs_days

# vim /etc/my.cnf  //修改expire_logs_days,x是自動刪除的天數，一般將x設置為短點，如10 expire_logs_days = x //二進制日志自動刪除的天數。默認值為0,表示“沒有自動刪除”

此方法需要重啟mysql，附錄有關於expire_logs_days的英文說明

當然也可以不重啟mysql,開啟mysql主從，直接在mysql里設置expire_logs_days

> show binary logs; > show variables like '%log%'; > set global expire_logs_days = 10;

 
3.手動清除binlog文件

# /usr/local/mysql/bin/mysql -u root -p > PURGE MASTER LOGS BEFORE DATE_SUB(CURRENT_DATE, INTERVAL 10 DAY); //刪除10天前的MySQL binlog日志,附錄2有關於PURGE MASTER LOGS手動刪除用法及示例 > show master logs;

 
也可以重置master，刪除所有binlog文件：

# /usr/local/mysql/bin/mysql -u root -p > reset master; //附錄3有清除binlog時，對從mysql的影響說明

附錄：
1.expire_logs_days英文說明
Where X is the number of days you’d like to keep them around. I would recommend 10, but this depends on how busy your MySQL server is and how fast these log files grow. Just make sure it is longer than the slowest slave takes to replicate the data from your master.
Just a side note: You know that you should do this anyway, but make sure you back up your mysql database. The binary log can be used to recover the database in certain situations; so having a backup ensures that if your database server does crash, you will be able to recover the data.

2.PURGE MASTER LOGS手動刪除用法及示例,MASTER和BINARY是同義詞

> PURGE {MASTER | BINARY} LOGS TO 'log_name' > PURGE {MASTER | BINARY} LOGS BEFORE 'date'

刪除指定的日志或日期之前的日志索引中的所有二進制日志。這些日志也會從記錄在日志索引文件中的清單中被刪除MySQL BIN-LOG 日志，這樣被給定的日志成為第一個。

實例：

> PURGE MASTER LOGS TO 'MySQL-bin.010'; //清除MySQL-bin.010日志 > PURGE MASTER LOGS BEFORE '2008-06-22 13:00:00'; //清除2008-06-22 13:00:00前binlog日志 > PURGE MASTER LOGS BEFORE DATE_SUB( NOW( ), INTERVAL 3 DAY); //清除3天前binlog日志BEFORE，變量的date自變量可以為'YYYY-MM-DD hh:mm:ss'格式。

 
3.清除binlog時，對從mysql的影響
如果您有一個活性的從屬服務器，該服務器當前正在讀取您正在試圖刪除的日志之一，則本語句不會起作用，而是會失敗，並伴隨一個錯誤。不過，如果從屬服務器是休止的，並且您碰巧清理了其想要讀取的日志之一，則從屬服務器啟動后不能復制。當從屬服務器正在復制時，本語句可以安全運行。您不需要停止它們。