一、zabbix 數據庫存儲
zabbix-server將采集到的數據存儲在數據庫(mysql、oracle等),而數據存儲的大小與每秒處理的數量量有關,因此數據存儲取決於以下兩個因數:
(1)Required server performance, new values per second(每秒處理的數據量)
(2)Housekeeper的設置(zabbix_server.conf文件中):刪除數據
zabbix-server將采集到數據主要存儲於Histroy和Trends表中,其表結構中的數據類型如下:
對於存儲超過期限的數據,主要使用DELETE sql語句進行刪除,當數據量太多,這將嚴重影響數據庫的性能。
查詢zabbix數據庫中各種表存儲的大小和行數:
mysql> select table_name, (data_length + index_length)/1024/1024 as total_mb, table_rows from information_schema.tables where table_schema='zabbix';
從上圖可知,我這僅僅只是一個測試環境,histroy表存儲的數據已經幾十萬行,如果是生成環境,估計幾千、上億萬行。
二、數據庫 表分區
1、
數據庫的優化有橫向和縱向擴展,這里使用數據的分布式,而分表可以看做是分布式的一種。。
在zabbix_server.conf文件中,找到如下兩個參數:
(1)HousekeepingFrequency=1 解釋:多久刪除一次數據庫里面的過期數據(間隔時間),默認一小時
(2)MaxHousekeeperDelete=5000 解釋:每次刪除數據量的上線(最大刪除量),默認5000
采用表分區,需要關閉Housekeeping功能,關閉流程如下:Administration ---> General ---> Housekeepin
去掉Histroy和Trends的勾選狀態,如下圖:
2、
(1)進行表分區,這里采用GitHub上一位大神寫的表分區腳本,連接如下:
wget https://github.com/itnihao/zabbix-book/blob/master/03-chapter/partitiontables.sh
該腳本具有如下功能:
> 備份數據庫
> 對表進行分區間
> 添加定時任務
如果數據量過大,可以將表數據刪除(當然這樣數據會全部被刪除)
清空語句如下:
sql> use zabbix; sql> truncate table histroy; sql> optimize table histroy; sql> truncate table histroy_str; sql> optimize table histroy_str; sql> truncate table histroy_uint; sql> optimize table histroy_unit; sql> truncate table trends; sql> optimize table trends; sql> truncate table trends_unit; sql> optimize table trends_unit; sql> truncate table events; sql> optimize table events;
(2)運行表分區腳本
為了防止網絡中斷引起腳本運行中斷而造成數據庫故障,這里選用screen后台執行方法。
# yum install -y screen
# screen -R zabbix 需要退出的話可以按 Ctral + A 以后再執行 Ctral + D
#bash partitiontables.sh
執行過程如下:
Ready to partition tables. Ready to update permissions of Zabbix user to create routines Enter root DB user: zabbix Enter zabbix password: zabbix mysql: [Warning] Using a password on the command line interface can be insecure. ERROR 1044 (42000) at line 1: Access denied for user 'zabbix'@'localhost' to database 'zabbix' # 上述錯誤可以忽略 Do you want to backup the database (recommended) (Y/n): y Enter output file, press return for default of /tmp/zabbix.sql mysql: [Warning] Using a password on the command line interface can be insecure. ERROR 1044 (42000) at line 1: Access denied for user 'zabbix'@'localhost' to database 'zabbix' mysqldump: [Warning] Using a password on the command line interface can be insecure. Mysqldump succeeded!, proceeding with upgrade... Ready to proceed: Starting yearly partioning at: 2019 and ending at: 2019 With 30 days of daily history Ready to proceed (Y/n): y Altering table: history Altering table: history_log Altering table: history_str Altering table: history_text Altering table: history_uint Altering table: trends Altering table: trends_uint Creating monthly partitions for table: trends Creating monthly partitions for table: trends_uint Creating daily partitions for table: history Creating daily partitions for table: history_log Creating daily partitions for table: history_str Creating daily partitions for table: history_text Creating daily partitions for table: history_uint Ready to apply script to database, this may take a while.(Y/n): y mysql: [Warning] Using a password on the command line interface can be insecure. Altering tables history history_log ERROR 1091 (42000) at line 4: Can't DROP 'history_log_2'; check that column/key exists If Zabbix Version = 2.0 Do you want to update the /etc/zabbix/zabbix_server.conf to disable housekeeping (Y/n): n Do you want to update the crontab (Y/n): y The crontab entry can be either in /etc/cron.daily, or added to the crontab for root Do you want to add this to the /etc/cron.daily directory (Y/n): y Enter email of who should get the daily housekeeping reports:
執行完腳本以后,查看計划任務:
[root@manager1 script_sh 19:24:08]#cat /etc/cron.daily/zabbixhousekeeping #!/bin/bash /usr/local/zabbix/cron.d/housekeeping.sh [root@manager1 script_sh 19:26:30]#cat /usr/local/zabbix/cron.d/housekeeping.sh #!/bin/bash MAILTO=root@localhost tmpfile=/tmp/housekeeping$$ date >$tmpfile /usr/bin/mysql --skip-column-names -B -h localhost -uzabbix -pzabbix zabbix -e "CALL create_zabbix_partitions();" >>$tmpfile 2>&1 /usr/bin/mail -s "Zabbix MySql Partition Housekeeping" $MAILTO <$tmpfile rm -f $tmpfile [root@manager1 script_sh 19:26:51]#
ok