參考：http://blog.sina.com.cn/s/blog_4673e6030102v46l.html
參考：http://hcymysql.blog.51cto.com/5223301/1431698
參考：http://blog.csdn.net/yueliangdao0608/article/details/21230627
參考：http://code.openark.org/blog/mysql/tokudb-configuration-variables-of-interest
參考：http://blog.itpub.net/22418990/viewspace-1251497/
官網說明：http://docs.tokutek.com/tokudb/tokudb-index-variables.html
官網說明：https://mariadb.com/kb/en/mariadb/tokudb-system-variables/#tokudb_pk_insert_mode
備份：http://mysql.taobao.org/monthly/2015/12/06/
因為MariaDB 10版本需要單獨編譯TokuDB引擎，編譯需要GCC >= 4.7和CMake >=2.8.8版本，暫沒有進行單獨編譯，后續再測試。
官網TokuDB源碼下面的說明：
Before you start, make sure you have a C++11-compatible compiler (GCC >= 4.7 is recommended), as well as CMake >=2.8.8, and the libraries and header files for valgrind,zlib, and Berkeley DB. We are using the gcc 4.7 in devtoolset-1.1.
測試版本：Percona-Server-5.6.24-rel72.2-Linux.x86_64.ssl101.tar.gz
測試版本：Percona-Server-5.6.24-rel72.2-TokuDB.Linux.x86_64.ssl101.tar.gz
OS:Centos 6.2
# - path of TokuDB file - # > tokudb_data_dir > tokudb_log_dir > tokudb_tmp_dir # - variables of TokuDB engine - # > tokudb_cache_size (Dynamic: NO,Default Value: Half of the total system memory) This variable configures the size in bytes of the TokuDB cache table. The default cache table size is 1/2 of physical memory. Tokutek highly recommends using the default setting if using buffered IO, if using direct IO then consider setting this parameter to 80% of available memory. > tokudb_directio (Dynamic: NO,Default Value: OFF) When set to ON, TokuDB employs Direct IO rather than Buffered IO for writes. When using Direct IO consider increasing tokudb cache size from its default of 1/2 physical memory. The default values is OFF. > tokudb_commit_sync (Dynamic: Yes,Default Value: ON) Session variable tokudb_commit_sync controls whether or not the transaction log is flushed when a transaction commits. The default behavior is that the transaction log is flushed by the commit. Setting this parameter to off may make the system run faster. However, transactions committed since the last checkpoint are not guaranteed to survive a crash. > tokudb_fsync_log_period (Dynamic: Yes,Default Value: 0) Controls the frequency, in milliseconds, for fsync() operations. If set to 0 then the fsync() behavior is only controlled by the tokudb commit sync, which is on or off. The default values is 0. > tokudb_row_format (Dynamic: Yes,Default Value: tokudb_zlib) tokudb_default, tokudb_zlib: Use the zlib library, tokudb_fast, tokudb_quicklz: Use the quicklz library, the lightest compression with low CPU usage, tokudb_small, tokudb_lzma: Use the lzma library. the highest compression and highest CPU usage tokudb_uncompressed: No compression is used. http://docs.tokutek.com/tokudb/tokudb-index-using-tokudb.html#tokudb-compression-details > tokudb_read_buf_size > tokudb_loader_memory_size
上面兩個官網的說明沒有讀懂...求高人指點
二進制包解壓后把toku包中的文件放到server包中的路徑下
注意配置要添加
[mysqld_safe] malloc_lib = /usr/local/Percona-Mysql/lib/mysql/libjemalloc.so [mysqld] plugin_dir = /usr/local/Percona-Mysql/lib/mysql/plugin/ plugin_load=ha_tokudb.so
修改系統Huge pages配置
[root@test percona]# echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled [root@test percona]# echo never > /sys/kernel/mm/redhat_transparent_hugepage/defarg
否則會報錯
#### 系統Huge pages配置需要修改，Centos我的是redhat_transparent_hugepage/enabled和defrag [ERROR] TokuDB: Huge pages are enabled, disable them before continuing [ERROR] ************************************************************ [ERROR] [ERROR] @@@@@@@@@@@ [ERROR] @@' '@@ [ERROR] @@ _ _ @@ [ERROR] | (.) (.) | [ERROR] | ` | [ERROR] | > ' | [ERROR] | .----. | [ERROR] .. |.----.| .. [ERROR] .. ' ' .. [ERROR] .._______,. [ERROR] [ERROR] TokuDB will not run with transparent huge pages enabled. [ERROR] Please disable them to continue. [ERROR] (echo never > /sys/kernel/mm/transparent_hugepage/enabled) [ERROR] [ERROR] ************************************************************ [ERROR] Plugin 'TokuDB' init function returned error. [ERROR] Plugin 'TokuDB' registration as a STORAGE ENGINE failed. #### 配置malloc_lib放錯位置，應該放在[mysqld_safe]下 [ERROR] /usr/local/Percona-Server-5.6.24-rel72.2-Linux.x86_64.ssl101/bin/mysqld: unknown variable 'malloc_lib=/usr/local/Percona-Server-5.6.24-rel72.2-Linux.x86_64.ssl101/lib/mysql/libjemalloc.so'
#### 配置malloc_lib缺項 [ERROR] TokuDB is not initialized because jemalloc is not loaded [ERROR] Plugin 'TokuDB' init function returned error. [ERROR] Plugin 'TokuDB' registration as a STORAGE ENGINE failed.
#### 啟動失敗后刪除TokuDB遺留文件 [ERROR] TokuDB: Persistent environment information is missing (but log exists) [ERROR] TokuDB unknown error 2 [ERROR] Plugin 'TokuDB' init function returned error. [ERROR] Plugin 'TokuDB' registration as a STORAGE ENGINE failed.
然后進入MySQL確認TokuDB可用
mysql> show engines; +--------------------+---------+----------------------------------------------------------------------------+--------------+------+------------+ | Engine | Support | Comment | Transactions | XA | Savepoints | +--------------------+---------+----------------------------------------------------------------------------+--------------+------+------------+ | InnoDB | DEFAULT | Percona-XtraDB, Supports transactions, row-level locking, and foreign keys | YES | YES | YES | | CSV | YES | CSV storage engine | NO | NO | NO | | MRG_MYISAM | YES | Collection of identical MyISAM tables | NO | NO | NO | | BLACKHOLE | YES | /dev/null storage engine (anything you write to it disappears) | NO | NO | NO | | MEMORY | YES | Hash based, stored in memory, useful for temporary tables | NO | NO | NO | | TokuDB | YES | Tokutek TokuDB Storage Engine with Fractal Tree(tm) Technology | YES | YES | YES | | PERFORMANCE_SCHEMA | YES | Performance Schema | NO | NO | NO | | ARCHIVE | YES | Archive storage engine | NO | NO | NO | | MyISAM | YES | MyISAM storage engine | NO | NO | NO | | FEDERATED | NO | Federated MySQL storage engine | NULL | NULL | NULL | +--------------------+---------+----------------------------------------------------------------------------+--------------+------+------------+ 10 rows in set (0.00 sec)
【性能測試】
由於使用的是Percona 5.6版本，默認performance_schema是開啟的，所以關掉后再試。
建兩張測試表，表字段類型長度一樣引擎不一樣
CREATE TABLE `test1_innodb` \ `test1_tokudb` ( `a` bigint(20) NOT NULL AUTO_INCREMENT, `b` varchar(255) DEFAULT NULL, `c` varchar(255) DEFAULT NULL, `d` text, `e` varchar(255) DEFAULT NULL, `f` varchar(255) DEFAULT NULL, `g` varchar(255) DEFAULT NULL, `h` varchar(255) DEFAULT NULL, `i` text, `j` varchar(255) DEFAULT NULL, `k` varchar(255) DEFAULT NULL, `l` varchar(255) DEFAULT NULL, `m` varchar(255) DEFAULT NULL, `n` varchar(255) DEFAULT NULL, PRIMARY KEY (`a`) ) ENGINE=InnoDB \ TokuDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8
##### 測試1
binlog_format = MIX sync_binlog = 0 innodb_flush_log_at_trx_commit = 1 innodb_buffer_pool_size = 1G tokudb_cache_size = 1G tokudb_row_format = tokudb_zlib tokudb_commit_sync = ON tokudb_fsync_log_period = 0 mysql> load data local infile 'test.csv' into table test_innodb ...; Query OK, 65536 rows affected (51.46 sec) Records: 65536 Deleted: 0 Skipped: 0 Warnings: 0 mysql> load data local infile 'test.csv' into table test_tokudb ...; Query OK, 65536 rows affected (31.86 sec) Records: 65536 Deleted: 0 Skipped: 0 Warnings: 0
##### 統計信息為information_schema.tables的table_length,index_length,total(table_length+index_length)，換算完單位為G mysql> test_innodb; +---------+---------+---------+ | DATA | idx | total | +---------+---------+---------+ | 2.66791 | 0.00000 | 2.66791 | +---------+---------+---------+ mysql> test_tokudb; +---------+---------+---------+ | DATA | idx | total | +---------+---------+---------+ | 1.44171 | 0.00000 | 1.44171 | +---------+---------+---------+ [root@test percona]# ll -h */*test_* -rw-rw---- 1 mysql mysql 17K Jun 30 11:11 test/test_innodb.frm -rw-rw---- 1 mysql mysql 2.8G Jun 30 11:24 test/test_innodb.ibd -rw-rw---- 1 mysql mysql 17K Jun 30 11:10 test/test_tokudb.frm -rw-rw---- 1 mysql mysql 16M Jun 30 11:26 toku_data/_test_test_tokudb_main_90_2_1b.tokudb -rw-rw---- 1 mysql mysql 64K Jun 30 11:27 toku_data/_test_test_tokudb_status_90_1_1b.tokudb
##### 插入數據后，發現系統中tokudb數據文件大小與實例中統計的表大小相差很多，暫不分析原因，后面測試實例中統計信息的依據性。
##### 測試2
tokudb_row_format = tokudb_small 其余配置不變 mysql> load data local infile 'test.csv' into table test_tokudb_small ...; Query OK, 65536 rows affected (33.91 sec) Records: 65536 Deleted: 0 Skipped: 0 Warnings: 0 mysql> test_tokudb_small; +---------+---------+---------+ | DATA | idx | total | +---------+---------+---------+ | 1.44171 | 0.00000 | 1.44171 | +---------+---------+---------+ 1 row in set (0.00 sec) [root@test percona]# ll -h */*test_toku* -rw-rw---- 1 mysql mysql 17K Jun 30 11:10 test/test_tokudb.frm -rw-rw---- 1 mysql mysql 17K Jun 30 11:36 test/test_tokudb_small.frm -rw-rw---- 1 mysql mysql 16M Jun 30 11:26 toku_data/_test_test_tokudb_main_90_2_1b.tokudb -rw-rw---- 1 mysql mysql 16M Jun 30 11:38 toku_data/_test_test_tokudb_small_main_a1_2_1b.tokudb -rw-rw---- 1 mysql mysql 64K Jun 30 11:38 toku_data/_test_test_tokudb_small_status_a1_1_1b.tokudb -rw-rw---- 1 mysql mysql 64K Jun 30 11:27 toku_data/_test_test_tokudb_status_90_1_1b.tokudb
##### 使用tokudb_small的表發現數據文件大小與tokudb_zlib相差並不多
##### 使用tokudb_small\tokudb_lzma CPU負載會比較高
##### tokudb_small, tokudb_lzma: Use the lzma library. the highest compression and highest CPU usage
##### 試了一下更換tokudb_row_format不影響表數據文件
##### 刪除完數據使用optimize table測試磁盤回收，發現innodb表數據文件空間釋放掉，而且速度很快，但是tokudb表數據文件空間沒有釋放，重啟也未釋放
#####rename table更換表名但是tokudb表數據文件名字不會更改，表結構.frm會變更
【MySQL 5.5.41官方帶TokuDB版本】
版本：mysql-5.5.41-tokudb-7.5.5-linux-x86_64
加載TokuDB與MariaDB版本語法不一樣
root@mysql_5541.sock::[tmp] ->INSTALL PLUGIN TOKUDB SONAME 'ha_tokudb.so';
測試數據
CREATE TABLE `test2` ( `id` int(10) unsigned NOT NULL AUTO_INCREMENT, `inta` bigint(20) DEFAULT NULL, `char1` varchar(255) DEFAULT NULL, `char2` varchar(255) DEFAULT NULL, `time1` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, `char3` varchar(255) DEFAULT NULL, `char4` varchar(255) DEFAULT NULL, `intb` bigint(20) DEFAULT NULL, PRIMARY KEY (`id`) ) ENGINE=TokuDB; begin;insert into test2(inta,char1,char2,time1,char3,char4,intb) values (12345678901,'1234567890112345678901','1234567890112345678901',now(),'1234567890112345678901','1234567890112345678901',12345678901);commit;
單線程寫測試結果
tokudb_directio=OFF real 13m1.932s tokudb_directio=ON real 11m46.517s tokudb_directio=ON tokudb_commit_sync = OFF tokudb_fsync_log_period = 1000 real 9m18.774s tokudb_directio=OFF tokudb_commit_sync = OFF tokudb_fsync_log_period = 1000 real 9m3.667s
並發5線程和10線程寫速度測試，單線程數據50W
分別測試tokudb_directio開和關對性能的影響
tokudb_commit_sync=OFF
tokudb_fsync_log_period=1000
5線程 tokudb_directio=OFF
real 16m1.031s 10線程 tokudb_directio=OFF real 22m0.711s 10線程 tokudb_directio=ON real 22m37.935s
10線程更新數據速度測試，單線程數據10W
CREATE TABLE `test2` ( `id` int(10) unsigned NOT NULL AUTO_INCREMENT, `inta` bigint(20) DEFAULT NULL, `char1` varchar(255) DEFAULT NULL, `char2` varchar(255) DEFAULT NULL, `time1` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, `char3` varchar(255) DEFAULT NULL, `char4` varchar(255) DEFAULT NULL, `intb` bigint(20) DEFAULT NULL, PRIMARY KEY (`id`) ) ENGINE=TokuDB; ... begin;update test2 set inta=99998 ,char1='char1' where id=023567;commit; begin;update test2 set inta=99999 ,char1='char1' where id=760214;commit; begin;update test2 set inta=100000 ,char1='char1' where id=560488;commit;
用時:
real 3m51.970s
【MariaDB 10編譯TokuDB】
參考：http://blog.sina.com.cn/s/blog_4673e6030102v46l.html
參考：http://blog.sina.com.cn/s/blog_4673e6030102v46k.html
參考：http://www.ithov.com/linux/130504.shtml
按照上面的參考按照高版本的gcc和cmake。(C++11-compatible compiler (GCC >= 4.7 is recommended), as well as CMake >=2.8.8)
如果出現報錯/usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.15' not found參考上面鏈接的方法解決
如果報下面的按照jemalloc
CMake Warning at storage/tokudb/CMakeLists.txt:17 (MESSAGE):
TokuDB is enabled, but jemalloc is not. This configuration is not
supported
編譯完的ha_tokudb.so放在官方同版本分發包中就可以使用了
MariaDB [(none)]> INSTALL SONAME 'ha_tokudb';
使用注意：
1.MariaDB分發包解壓復制到/usr/local/下
2.MariaDB配置文件初始化時不要帶TokuDB引擎參數
3.初始化實例系統表
4.啟動實例執行“install soname 'ha_tokudb;'”
5.停止實例並將配置文件添加TokuDB引擎參數
如果使用默認參數可以不停實例
如果要修改tokudb_data_dir/tokudb_log_dir/tokudb_tmp_dir需要刪除實例數據目錄下面TokuDB加載文件，-BASH "rm -rf *tokudb*"
不建議tokudb使用自定義數據文件、臨時文件、日志文件路徑，自定義路徑會導致tokudb引擎文件字典中使用絕對路徑，如果實例數據庫目錄要在服務器系統移動位置時出現找不到數據文件的錯誤
6.第一次加載成功TokuDB，之后啟動自動加載，但是tokudb_*配置錯誤會導致引擎注冊啟動失敗，TokuDB引擎表無法使用
【MySLQ 5.5.21實例升級Percona 5.6.24】
tokudb最新5.5下載地址：http://www.tokutek.com/download.php
官方安裝都需要停止實例安裝tokudb引擎
參考：http://docs.tokutek.com/tokudb/tokudb-index-installation.html
測試從MySQL-5.5.21升級到Percona-Server-5.6.24
1、停止實例
2、使用Percona-Server-5.6.24啟動實例
3、使用Percona-Server-5.6.24的mysql_upgrade -u -p -S 升級系統表結構
4、啟動實例
[root@test Percona]# /usr/local/Percona-Server-5.6.24-rel72.2-Linux.x86_64.ssl101/bin/mysql_upgrade -uroot -p -S /tmp/mysql_3306.sock Enter password: Looking for 'mysql' as: /usr/local/Percona-Server-5.6.24-rel72.2-Linux.x86_64.ssl101/bin/mysql Looking for 'mysqlcheck' as: /usr/local/Percona-Server-5.6.24-rel72.2-Linux.x86_64.ssl101/bin/mysqlcheck Running 'mysqlcheck' with connection arguments: '--socket=/tmp/mysql_3306.sock' Warning: Using a password on the command line interface can be insecure. Running 'mysqlcheck' with connection arguments: '--socket=/tmp/mysql_3306.sock' Warning: Using a password on the command line interface can be insecure. mysql.columns_priv OK mysql.db OK mysql.event OK mysql.func OK mysql.general_log OK mysql.help_category OK mysql.help_keyword OK mysql.help_relation OK mysql.help_topic OK mysql.host OK mysql.ndb_binlog_index OK mysql.plugin OK mysql.proc OK mysql.procs_priv OK mysql.proxies_priv OK mysql.servers OK mysql.slow_log OK mysql.tables_priv OK mysql.time_zone OK mysql.time_zone_leap_second OK mysql.time_zone_name OK mysql.time_zone_transition OK mysql.time_zone_transition_type OK mysql.user OK Running 'mysql_fix_privilege_tables'... Warning: Using a password on the command line interface can be insecure. Running 'mysqlcheck' with connection arguments: '--socket=/tmp/mysql_3306.sock' Warning: Using a password on the command line interface can be insecure. Running 'mysqlcheck' with connection arguments: '--socket=/tmp/mysql_3306.sock' Warning: Using a password on the command line interface can be insecure. test.t1 OK tmp.tb OK tmp.tb_merge_goldeye_main_role OK OK
【MySQL 5.5.37實例升級MariaDB 10.0.20】
官網對於mysql_upgrade的說明：https://mariadb.com/kb/en/mariadb/mysql_upgrade/
1、MariaDB 10.0.20官網分發包，下載解壓
2、MySQL 5.5.37挺實例
3、MariaDB 10.0.20啟動實例，如果實例啟動正常，errlog報錯忽略
4、MariaDB 10.0.20分發包中的mysql_upgrade升級實例系統表，查看升級狀態
5、實例停止再啟動，errlog無報錯
[root@test mariadb]# ./bin/mysql_upgrade -uroot -p -S /tmp/mysql_3306.sock Enter password: MySQL upgrade detected Phase 1/6: Checking and upgrading mysql database Processing databases mysql mysql.columns_priv OK mysql.db OK mysql.event OK mysql.func OK mysql.help_category OK mysql.help_keyword OK mysql.help_relation OK mysql.help_topic OK mysql.host OK mysql.ndb_binlog_index OK mysql.plugin OK mysql.proc OK mysql.procs_priv OK mysql.proxies_priv OK mysql.servers OK mysql.tables_priv OK mysql.time_zone OK mysql.time_zone_leap_second OK mysql.time_zone_name OK mysql.time_zone_transition OK mysql.time_zone_transition_type OK mysql.user OK Phase 2/6: Fixing views from mysql Phase 3/6: Running 'mysql_fix_privilege_tables' Phase 4/6: Fixing table and database names Phase 5/6: Checking and upgrading tables Processing databases information_schema jiangxu jiangxu.jiangxu1 OK jiangxu.jiangxu2 OK jiangxu.jiangxu3 OK performance_schema test tmp Phase 6/6: Running 'FLUSH PRIVILEGES' OK
【功能特性說明】：支持輔助聚集索引、在線添加索引、在線修改列屬性等
輔助聚集索引 --> 使得通過該索引的查詢都為覆蓋索引，速度更快
在線添加索引 --> tokudb對於alter table add index 和create index處理方式不同。
Using the ALTER TABLE command for creating an index will create the index offline (with the table unavailable for inserts or queries), regardless of the value of tokudb_create_index_online. The only way to hot create an index is to use the CREATE INDEX command.
Hot creating an index will be slower than creating the index offline, and progress depends how busy the mysqld server is with other tasks.
在線修改列屬性 --> 有一些修改並不支持。
Hot column expansion operations are only supported to char, varchar, varbinary, and integer data types. Hot column expansion is not supported if the given column is part of the primary key or any secondary keys.
Hot column rename does not support the following data types: TIME, ENUM, BLOB, TINYBLOB, MEDIUMBLOB, LONGBLOB. Renaming columns of these types will revert to the standard MySQL blocking behavior.
上面只是官網的一小部分，詳細看下面鏈接原文
參考：http://docs.tokutek.com/tokudb/tokudb-index-using-tokudb.html
【鎖的說明】
實例主從復制建議使用binlog_format=ROW
參考：http://docs.tokutek.com/tokudb/tokudb-index-troubleshooting.html