zabbix(4)數據庫表分區優化


 

 

 

 

 

一、zabbix 數據庫存儲

  zabbix-server將采集到的數據存儲在數據庫(mysql、oracle等),而數據存儲的大小與每秒處理的數量量有關,因此數據存儲取決於以下兩個因數:

  (1)Required server performance, new values per second(每秒處理的數據量)

  (2)Housekeeper的設置(zabbix_server.conf文件中):刪除數據

  zabbix-server將采集到數據主要存儲於Histroy和Trends表中,其表結構中的數據類型如下:

  對於存儲超過期限的數據,主要使用DELETE sql語句進行刪除,當數據量太多,這將嚴重影響數據庫的性能。

  查詢zabbix數據庫中各種表存儲的大小和行數:

mysql> select table_name, (data_length + index_length)/1024/1024 as total_mb, table_rows from information_schema.tables where table_schema='zabbix';

  從上圖可知,我這僅僅只是一個測試環境,histroy表存儲的數據已經幾十萬行,如果是生成環境,估計幾千、上億萬行。

 

二、數據庫 表分區

1、

  數據庫的優化有橫向和縱向擴展,這里使用數據的分布式,而分表可以看做是分布式的一種。。

  在zabbix_server.conf文件中,找到如下兩個參數:

    (1)HousekeepingFrequency=1 解釋:多久刪除一次數據庫里面的過期數據(間隔時間),默認一小時

    (2)MaxHousekeeperDelete=5000  解釋:每次刪除數據量的上線(最大刪除量),默認5000

  采用表分區,需要關閉Housekeeping功能,關閉流程如下:Administration ---> General ---> Housekeepin

 

  去掉Histroy和Trends的勾選狀態,如下圖:

2、

   (1)進行表分區,這里采用GitHub上一位大神寫的表分區腳本,連接如下:

    wget  https://github.com/itnihao/zabbix-book/blob/master/03-chapter/partitiontables.sh

     該腳本具有如下功能:

      > 備份數據庫

      > 對表進行分區間

      > 添加定時任務

     如果數據量過大,可以將表數據刪除(當然這樣數據會全部被刪除)

     清空語句如下:

sql> use zabbix;
sql> truncate table histroy;
sql> optimize table histroy; 

sql> truncate table histroy_str;
sql> optimize table histroy_str; 

sql> truncate table histroy_uint;
sql> optimize table histroy_unit; 

sql> truncate table trends;
sql> optimize table trends; 

sql> truncate table trends_unit;
sql> optimize table trends_unit; 

sql> truncate table events;
sql> optimize table events; 

  (2)運行表分區腳本

    為了防止網絡中斷引起腳本運行中斷而造成數據庫故障,這里選用screen后台執行方法。

    # yum install -y screen

    # screen -R zabbix    需要退出的話可以按 Ctral + A 以后再執行 Ctral + D

    #bash partitiontables.sh

    執行過程如下:

Ready to partition tables.

Ready to update permissions of Zabbix user to create routines

Enter root DB user: zabbix
Enter zabbix password: zabbix
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 1044 (42000) at line 1: Access denied for user 'zabbix'@'localhost' to database 'zabbix'

#   上述錯誤可以忽略

Do you want to backup the database (recommended) (Y/n): y

Enter output file, press return for default of /tmp/zabbix.sql

mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 1044 (42000) at line 1: Access denied for user 'zabbix'@'localhost' to database 'zabbix'
mysqldump: [Warning] Using a password on the command line interface can be insecure.
Mysqldump succeeded!, proceeding with upgrade...


Ready to proceed:

Starting yearly partioning at: 2019
and ending at: 2019
With 30 days of daily history


Ready to proceed (Y/n): 
y
Altering table: history
Altering table: history_log
Altering table: history_str
Altering table: history_text
Altering table: history_uint
Altering table: trends
Altering table: trends_uint
Creating monthly partitions for table: trends
Creating monthly partitions for table: trends_uint
Creating daily partitions for table: history
Creating daily partitions for table: history_log
Creating daily partitions for table: history_str
Creating daily partitions for table: history_text
Creating daily partitions for table: history_uint


Ready to apply script to database, this may take a while.(Y/n): 
y
mysql: [Warning] Using a password on the command line interface can be insecure.
Altering tables
history
history_log
ERROR 1091 (42000) at line 4: Can't DROP 'history_log_2'; check that column/key exists

If Zabbix Version = 2.0 
Do you want to update the /etc/zabbix/zabbix_server.conf
to disable housekeeping (Y/n): n

Do you want to update the crontab (Y/n): y
The crontab entry can be either in /etc/cron.daily, or added
to the crontab for root

Do you want to add this to the /etc/cron.daily directory (Y/n): y

Enter email of who should get the daily housekeeping reports:

  執行完腳本以后,查看計划任務:

[root@manager1 script_sh 19:24:08]#cat /etc/cron.daily/zabbixhousekeeping 
#!/bin/bash
/usr/local/zabbix/cron.d/housekeeping.sh
[root@manager1 script_sh 19:26:30]#cat /usr/local/zabbix/cron.d/housekeeping.sh
#!/bin/bash
MAILTO=root@localhost
tmpfile=/tmp/housekeeping$$
date >$tmpfile
/usr/bin/mysql --skip-column-names -B -h localhost -uzabbix -pzabbix zabbix -e "CALL create_zabbix_partitions();" >>$tmpfile 2>&1
/usr/bin/mail -s "Zabbix MySql Partition Housekeeping" $MAILTO <$tmpfile
rm -f $tmpfile
[root@manager1 script_sh 19:26:51]#

ok

 


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM