碼上快樂

相關內容    簡體    繁體

Mycat 分片策略

本文轉載自   查看原文   2018-08-13 19:33   2013    MySQL

 

  • @(學習)[mycat, mysql]

 

 

1. 分片規則配置文件

分片規則配置文件rule.xml位於$MYCAT_HOME/conf目錄，它定義了所有拆分表的規則。在使用過程中可以靈活使用不同的分片算法，或者對同一個分片算法使用不同的參數，它讓分片過程可配置化，只需要簡單的幾步就可以讓運維人員及數據庫管理員輕松將數據拆分到不同的物理庫中。該文件包含兩個重要的標簽，分別是Funcation和tableRule。

1.1 Funcation標簽

<function name="rang-long" class="io.mycat.route.function.AutoPartitionByLong"> <property name="mapFile">autopartition-long.txt</property> </function>

 <function name="rang-long" class="io.mycat.route.function.AutoPartitionByLong"> <property name="mapFile">autopartition-long.txt</property> </function>

  • name屬性指定算法的名稱，在該文件中唯一。
  • class屬性對應具體的分片算法，需要指定算法的具體類。
  • property屬性根據算法的要求指定。

1.2 tableRule標簽

<tableRule name="auto-sharding-long"> <rule> <columns>id</columns> <algorithm>rang-long</algorithm> </rule> </tableRule>
  • name屬性指定分片規則的名稱，在該文件中唯一。
  • rule屬性指定分片算法的具體內容，包含columns和algorithm兩個屬性。
  • columns屬性指定對應的表中用於分片的列名。
  • algorithm屬性對應function中指定的算法的名稱。

2. 連續分片和離散分片

Alt text
Alt text

分片優缺點

分片 連續分片 離散分片
優點 擴容無需遷移數據
范圍條件查詢消耗資源少		 並發訪問能力增強
范圍條件查詢性能提升
缺點 存在數據熱點的可能性
並發訪問能力受限於單一或少量DataNode		 數據擴容比較困難，涉及到數據遷移問題
數據庫連接消耗比較多

Alt text

特殊分片 
Alt text

2.1 連續分片

連續分片規則 
* 自定義數字范圍分片 
* 按日期（天）分片 
* 按單月小時分片 
* 自然月分片

2.1.1 自定義數字范圍分片

適用於明確知道分片字段的某個范圍屬於哪個分片，其字段為數字類型。

vim conf/schema.xml

<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="auto_sharding_long" primaryKey="id" dataNode="dn0,dn1,dn2" rule="auto-sharding-long" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>

vim conf/rule.xml

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="auto-sharding-long"> <rule> <columns>age</columns> <algorithm>rang-long</algorithm> </rule> </tableRule> <function name="rang-long" class="io.mycat.route.function.AutoPartitionByLong"> <property name="mapFile">autopartition-long.txt</property> <property name="defaultNode">0</property> </function> </mycat:rule>

配置說明： 
* columns指定分片的表列名。 
* algorithm指定分片函數與function對應。 
* rang-long函數中的mapFile代表規則配置文件的路徑，可在$MYCAT_HOME/conf下新建目錄作好規范，比如將autopartition-long.txt放在$MYCAT_HOME/conf/func中，則用<property name="mapFile">func/autopartition-long.txt</property>來引用。 
* defaultNode為超過范圍后的默認節點。

vim conf/autopartition-long.txt

# range start-end ,data node index # K=1000,M=10000. 0-2M=0 2M-3M=1 3M-6M=2

測試sql：

CREATE TABLE auto_sharding_long (`age` int NOT NULL ,`db_nm` varchar(20) NULL); INSERT INTO auto_sharding_long (age,db_nm) VALUES (20000, database()); INSERT INTO auto_sharding_long (age,db_nm) VALUES (25000, database()); INSERT INTO auto_sharding_long (age,db_nm) VALUES (35000, database());
  • 1
  • 2
  • 3
  • 4

測試結果：

[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE auto_sharding_long (`age` int NOT NULL ,`db_nm` varchar(20) NULL); Query OK, 0 rows affected (0.10 sec) mysql> INSERT INTO auto_sharding_long (age,db_nm) VALUES (20000, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO auto_sharding_long (age,db_nm) VALUES (25000, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO auto_sharding_long (age,db_nm) VALUES (35000, database()); Query OK, 1 row affected (0.00 sec) mysql> select * from auto_sharding_long; +-------+-------+ | age | db_nm | +-------+-------+ | 20000 | db0 | | 35000 | db2 | | 25000 | db1 | +-------+-------+ 3 rows in set (0.00 sec) mysql>

 

注意 
schema里的table的dataNode節點個數必須 大於等於 autopartition-long.txt里的配置個數 
dataNode節點從0開始

2.1.2 按日期（天）分片

根據時間類型字段，按天分片。

vim conf/schema.xml

<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="sharding_by_day" primaryKey="id" dataNode="dn$0-2" rule="sharding-by-day" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>

vim conf/rule.xml

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="sharding-by-day"> <rule> <columns>create_time</columns> <algorithm>part-by-day</algorithm> </rule> </tableRule> <function name="part-by-day" class="io.mycat.route.function.PartitionByDate"> <property name="dateFormat">yyyy-MM-dd</property> <property name="sBeginDate">2017-10-01</property> <property name="sPartionDay">10</property> </function> </mycat:rule>

配置說明： 
* columns標識將要分片的表字段。 
* algorithm為分片函數。 
* dateFormat為日期格式。 
* sBeginDate為開始日期。 
* sEndDate為結束日期，如果配置了這個屬性，則代表數據達到了這個日期的分片后會重復從開始分片插入。 
* sPartionDay為分區天數，默認從開始日期算起，每隔10天一個分區。

測試SQL：

CREATE TABLE sharding_by_day (create_time timestamp NULL ON UPDATE CURRENT_TIMESTAMP ,`db_nm` varchar(20) NULL); INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-01', database()); INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-10', database()); INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-11', database()); INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-21', database()); INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-31', database()); select * from sharding_by_day;

測試結果：

[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE sharding_by_day (create_time timestamp NULL ON UPDATE CURRENT_TIMESTAMP ,`db_nm` varchar(20) NULL); INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-01', database()); INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-10', database()); INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-11', database()); INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-21', database()); INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-31', database()); select * from sharding_by_day; Query OK, 0 rows affected (0.24 sec) mysql> INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-01', database()); Query OK, 1 row affected (0.10 sec) mysql> INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-10', database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-11', database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-21', database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-31', database()); ERROR 1064 (HY000): Can't find a valid data node for specified node index :SHARDING_BY_DAY -> CREATE_TIME -> 2017-10-31 -> Index : 3 mysql> select * from sharding_by_day; +---------------------+-------+ | create_time | db_nm | +---------------------+-------+ | 2017-10-01 00:00:00 | db0 | | 2017-10-10 00:00:00 | db0 | | 2017-10-11 00:00:00 | db1 | | 2017-10-21 00:00:00 | db2 | +---------------------+-------+ 4 rows in set (0.05 sec) mysql>

 

 

注意 
以上測試結果中有個錯誤，因為配置的dataNode節點數為3，而2017-10-31為2017-10-01后的第四個10天中的第一天，因此需要至少4個dataNode，節點不夠就報如上錯誤了。

2.1.3 按單月小時分片

單月內按照小時拆分，最小粒度是小時，一天最多可以有24個分片，最少1個分片，下個月從頭開始循環，每個月末需要手工清理數據。

字段為字符串類型，yyyymmddHH 10位。

vim conf/schema.xml

<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="sharding_by_hour" primaryKey="id" dataNode="dn$0-2" rule="sharding-by-hour" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>

 

 

vim conf/rule.xml

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="sharding-by-hour"> <rule> <columns>sharding_col</columns> <algorithm>sharding-by-hour</algorithm> </rule> </tableRule> <function name="sharding-by-hour" class="io.mycat.route.function.LatestMonthPartion"> <property name="splitOneDay">3</property> </function> </mycat:rule>

 

 

配置說明： 
* columns標識將要分片的表字段，字符串類型（yyyymmddHH），格式需要符合Java標准。 
* algorithm為分片函數。 
* splitOneDay為一天切分的分片數。

測試SQL：

CREATE TABLE sharding_by_hour (create_time timestamp NULL ON UPDATE CURRENT_TIMESTAMP ,`db_nm` varchar(20) NULL,sharding_col varchar(10) null); INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100101','2017-10-01', database()); INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100108','2017-10-01', database()); INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100109','2017-10-01', database()); INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100116','2017-10-01', database()); INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100117','2017-10-01', database()); INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100123','2017-10-01', database()); select * from sharding_by_hour;

 

 

測試結果：

[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE sharding_by_hour (create_time timestamp NULL ON UPDATE CURRENT_TIMESTAMP ,`db_nm` varchar(20) NULL,sharding_col varchar(10) null); INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100101','2017-10-01', database()); INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100108','2017-10-01', database()); INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100109','2017-10-01', database()); INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100116','2017-10-01', database()); INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100117','2017-10-01', database()); INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100123','2017-10-01', database()); select * from sharding_by_hour; Query OK, 0 rows affected (0.21 sec) mysql> INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100101','2017-10-01', database()); Query OK, 1 row affected (0.11 sec) mysql> INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100108','2017-10-01', database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100109','2017-10-01', database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100116','2017-10-01', database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100117','2017-10-01', database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100123','2017-10-01', database()); Query OK, 1 row affected (0.00 sec) mysql> select * from sharding_by_hour; +---------------------+-------+--------------+ | create_time | db_nm | sharding_col | +---------------------+-------+--------------+ | 2017-10-01 00:00:00 | db0 | 2017100101 | | 2017-10-01 00:00:00 | db1 | 2017100108 | | 2017-10-01 00:00:00 | db1 | 2017100109 | | 2017-10-01 00:00:00 | db2 | 2017100116 | | 2017-10-01 00:00:00 | db2 | 2017100117 | | 2017-10-01 00:00:00 | db2 | 2017100123 | +---------------------+-------+--------------+ 6 rows in set (0.06 sec)

 

 

注意 
* 分片數必須跟dataNode節點數一致 
* 最大分24個分片 
* 分片字段的值必須是字符串，如：2017103112.(即為2017年10月31日12點)

2.1.4 按自然月分片

使用場景為按月份列分區，每個自然月一個分片，查詢條件時使用between and。 
字段為時間類型。

vim conf/schema.xml

<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="sharding_by_month" primaryKey="id" dataNode="dn$0-2" rule="sharding-by-month" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>

 

 

vim conf/rule.xml

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="sharding-by-month"> <rule> <columns>create_time</columns> <algorithm>sharding-by-month</algorithm> </rule> </tableRule> <function name="sharding-by-month" class="io.mycat.route.function.PartitionByMonth"> <property name="dateFormat">yyyy-MM-dd</property> <property name="sBeginDate">2017-10-01</property> <property name="sEndDate">2017-12-30</property> </function> </mycat:rule>

 

 

配置說明： 
* columns標識將要分片的表字段，字符串類型，與dateFormat格式一致。 
* algorithm為分片函數。 
* dateFormat為日期字符串格式。 
* sBeginDate為開始日期。 
* sEndDate為結束日期

測試SQL：

CREATE TABLE sharding_by_month (create_time timestamp NULL ON UPDATE CURRENT_TIMESTAMP ,`db_nm` varchar(20) NULL); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-10-01', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-10-30', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-11-11', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-11-21', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-12-01', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-12-31', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2018-01-01', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2018-01-31', database()); select * from sharding_by_month

 

 

測試結果：

[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE sharding_by_month (create_time timestamp NULL ON UPDATE CURRENT_TIMESTAMP ,`db_nm` varchar(20) NULL); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-10-01', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-10-30', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-11-11', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-11-21', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-12-01', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-12-31', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2018-01-01', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2018-01-31', database()); Query OK, 0 rows affected (0.15 sec) mysql> INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-10-01', database()); select * from sharding_by_month; Query OK, 1 row affected (0.10 sec) mysql> INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-10-30', database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-11-11', database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-11-21', database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-12-01', database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-12-31', database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2018-01-01', database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2018-01-31', database()); Query OK, 1 row affected (0.01 sec) mysql> select * from sharding_by_month; +---------------------+-------+ | create_time | db_nm | +---------------------+-------+ | 2017-10-01 00:00:00 | db0 | | 2017-10-30 00:00:00 | db0 | | 2018-01-01 00:00:00 | db0 | | 2018-01-31 00:00:00 | db0 | | 2017-11-11 00:00:00 | db1 | | 2017-11-21 00:00:00 | db1 | | 2017-12-01 00:00:00 | db2 | | 2017-12-31 00:00:00 | db2 | +---------------------+-------+ 8 rows in set (0.06 sec)

 

注意 
* schema里的table的dataNode節點數必須：大於rule的開始時間按照分片數計算到現在的個數 
* 按照自然月計算（無論是28、30、31天都是一個月的） 
* 分片節點個數可以后增加，但是必須符合第一點說明。

2.2 離散分片

離散分片規則 
* 枚舉分片 
* 程序指定分區的分片 
* 十進制求模分片 
* 字符串hash解析分片 
* 一致性哈希分片

2.2.1 枚舉分片

通過在配置文件中配置可能的枚舉id，指定數據分布到不同的物理節點上，本規則適用於按照省份或區縣來拆分數據類業務。

vim conf/schema.xml

<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="sharding_by_intfile" primaryKey="id" dataNode="dn$0-2" rule="sharding-by-intfile" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>

vim conf/rule.xml

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="sharding-by-intfile"> <rule> <columns>age</columns> <algorithm>hash-int</algorithm> </rule> </tableRule> <function name="hash-int" class="io.mycat.route.function.PartitionByFileMap"> <property name="mapFile">partition-hash-int.txt</property> <property name="type">0</property> <property name="defaultNode">0</property> </function> </mycat:rule>

配置說明： 
* columns標識將要分片的表字段，字符串類型，與dateFormat格式一致。 
* algorithm為分片函數。 
* dateFormat為日期字符串格式。 
* sBeginDate為開始日期。 
* sEndDate為結束日期

vim conf/partition-hash-int.txt

# 枚舉項=節點標識，枚舉項可以是數字和字符（非漢字） 11=1 12=2

測試SQL：

CREATE TABLE sharding_by_intfile (`age` int NOT NULL ,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_intfile` (age,db_nm) VALUES (10, database()); INSERT INTO `sharding_by_intfile` (age,db_nm) VALUES (11, database()); INSERT INTO `sharding_by_intfile` (age,db_nm) VALUES (12, database()); select * from `sharding_by_intfile`;

 

 

測試結果：

[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE sharding_by_intfile (`age` int NOT NULL ,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_intfile` (age,db_nm) VALUES (10, database()); INSERT INTO `sharding_by_intfile` (age,db_nm) VALUES (11, database()); INSERT INTO `sharding_by_intfile` (age,db_nm) VALUES (12, database()); select * from `sharding_by_intfile`; Query OK, 0 rows affected (0.21 sec) mysql> INSERT INTO `sharding_by_intfile` (age,db_nm) VALUES (10, database()); Query OK, 1 row affected (0.06 sec) mysql> INSERT INTO `sharding_by_intfile` (age,db_nm) VALUES (11, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_intfile` (age,db_nm) VALUES (12, database()); Query OK, 1 row affected (0.01 sec) mysql> select * from `sharding_by_intfile`; +-----+-------+ | age | db_nm | +-----+-------+ | 10 | db0 | | 12 | db2 | | 11 | db1 | +-----+-------+ 3 rows in set (0.21 sec)

 

 

注意 
* schema里的table的dataNode節點個數必須：大於等於 partition-hash-int.txt里的配置個數

2.2.2 程序指定分區分片

在程序運行階段，由程序自主決定路由到哪個分片。

vim conf/schema.xml

<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="sharding_by_substring" primaryKey="id" dataNode="dn$0-2" rule="sharding-by-substring" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>

 

 

vim conf/rule.xml

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="sharding-by-substring"> <rule> <columns>user_id</columns> <algorithm>sharding-by-substring</algorithm> </rule> </tableRule> <function name="sharding-by-substring" class="io.mycat.route.function.PartitionDirectBySubString"> <property name="startIndex">0</property> <property name="size">2</property> <property name="partitionCount">3</property> <property name="defaultPartition">0</property> </function> </mycat:rule>

 

配置說明： 
* columns標識將要分片的表字段。 
* algorithm為分片函數。 
* startIndex為字符串開始截取位，從0開始。 
* size為截取的位數。 
* partitionCount為分片個數 
* defaultPartition為默認節點

測試SQL：

CREATE TABLE sharding_by_substring (`user_id` varchar(20) NOT NULL ,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_substring` (user_id,db_nm) VALUES ('05-10000', database()); INSERT INTO `sharding_by_substring` (user_id,db_nm) VALUES ('02-10001', database()); INSERT INTO `sharding_by_substring` (user_id,db_nm) VALUES ('03-10002', database()); select * from `sharding_by_substring`;

 

測試結果：

[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB)

Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE sharding_by_substring (`user_id` varchar(20) NOT NULL ,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_substring` (user_id,db_nm) VALUES ('05-10000', database()); INSERT INTO `sharding_by_substring` (user_id,db_nm) VALUES ('02-10001', database()); INSERT INTO `sharding_by_substring` (user_id,db_nm) VALUES ('03-10002', database()); select * from `sharding_by_substring`; Query OK, 0 rows affected (0.35 sec) mysql> INSERT INTO `sharding_by_substring` (user_id,db_nm) VALUES ('05-10000', database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_substring` (user_id,db_nm) VALUES ('02-10001', database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_substring` (user_id,db_nm) VALUES ('03-10002', database()); Query OK, 1 row affected (0.01 sec) mysql> select * from `sharding_by_substring`; +----------+-------+ | user_id | db_nm | +----------+-------+ | 05-10000 | db0 | | 03-10002 | db0 | | 02-10001 | db2 | +----------+-------+ 3 rows in set (0.00 sec)

 

注意 
* 直接根據字符子串（必須是數字）計算分區號（由應用傳遞參數，指定分區號）。 
例如user_id=05-10000，其中user_id是從startIndex=0開始的，截取長度為兩位數字，即05，05就是獲取的分區，默認分配到defaultPartition。

2.2.3 十進制求模分片

對分片字段十進制取模運算，其數據分布最均勻。

vim conf/schema.xml

<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="sharding_by_substring" primaryKey="id" dataNode="dn$0-2" rule="sharding-by-substring" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>

 

vim conf/rule.xml

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="sharding-by-substring"> <rule> <columns>user_id</columns> <algorithm>sharding-by-substring</algorithm> </rule> </tableRule> <function name="sharding-by-substring" class="io.mycat.route.function.PartitionDirectBySubString"> <property name="startIndex">0</property> <property name="size">2</property> <property name="partitionCount">3</property> <property name="defaultPartition">0</property> </function> </mycat:rule>

 

配置說明： 
* columns標識將要分片的表字段。 
* algorithm為分片函數。 
* startIndex為字符串開始截取位，從0開始。 
* size為截取的位數。 
* partitionCount為分片個數 
* defaultPartition為默認節點

測試SQL：

CREATE TABLE `sharding_by_mod` (id int(10) null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_mod` (id,db_nm) VALUES (1, database()); INSERT INTO `sharding_by_mod` (id,db_nm) VALUES (2, database()); INSERT INTO `sharding_by_mod` (id,db_nm) VALUES (3, database()); INSERT INTO `sharding_by_mod` (id,db_nm) VALUES (10, database()); select * from sharding_by_mod;

 

測試結果：

[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE `sharding_by_mod` (id int(10) null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_mod` (id,db_nm) VALUES (1, database()); INSERT INTO `sharding_by_mod` (id,db_nm) VALUES (2, database()); INSERT INTO `sharding_by_mod` (id,db_nm) VALUES (3, database()); INSERT INTO `sharding_by_mod` (id,db_nm) VALUES (10, database()); select * from sharding_by_mod; Query OK, 0 rows affected (0.22 sec) mysql> INSERT INTO `sharding_by_mod` (id,db_nm) VALUES (1, database()); Query OK, 1 row affected (0.04 sec) mysql> INSERT INTO `sharding_by_mod` (id,db_nm) VALUES (2, database()); Query OK, 1 row affected (0.02 sec) mysql> INSERT INTO `sharding_by_mod` (id,db_nm) VALUES (3, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_mod` (id,db_nm) VALUES (10, database()); Query OK, 1 row affected (0.01 sec) mysql> select * from sharding_by_mod; +------+-------+ | id | db_nm | +------+-------+ | 3 | db0 | | 1 | db1 | | 10 | db1 | | 2 | db2 | +------+-------+ 4 rows in set (0.10 sec)

 

注意 
* 對分片字段十進制取模運算，dataNode=id%count 
* 數據分布最均勻。 
* 擴容需要重新計算分片，數據遷移繁瑣。 
* 盡量的提前計算出來增量，創建足夠多的分片數。但是也不宜太多，給運維造成負擔。平衡點自己掌控。

2.2.4 字符串hash解析分片

此規則是截取字符串中的int數值的hash分片。

vim conf/schema.xml

<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="sharding_by_stringhash" primaryKey="ord_no" dataNode="dn$0-2" rule="sharding-by-stringhash" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>

 

vim conf/rule.xml

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="sharding-by-stringhash"> <rule> <columns>ord_no</columns> <algorithm>sharding-by-stringhash</algorithm> </rule> </tableRule> <function name="sharding-by-stringhash" class="io.mycat.route.function.PartitionByString"> <property name="partitionLength">512</property> <property name="partitionCount">2</property> <property name="hashSlice">-6:0</property> </function> </mycat:rule>

 

配置說明： 
* columns標識將要分片的表字段。 
* algorithm為分片函數。 
* partitionLength為字符串hash的求模基數。 
* partitionCount為分區數。其中partitionLength*partitionCount=1024 
* hashSlice為預算位，即根據子字符串中的int值進行hash運算。 
0 代表 str.length(), -1 代表 str.length()-1，大於0只代表數字自身 
可以理解為substring（start，end），start為0則只表示0 
例1：值“45abc”，hash預算位0:2 ，取其中45進行計算 
例2：值“aaaabbb2345”，hash預算位-4:0 ，取其中2345進行計算

測試SQL：

CREATE TABLE sharding_by_stringhash (ord_no varchar(20) NULL,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_stringhash` (ord_no,db_nm) VALUES (171022237582, database()); INSERT INTO `sharding_by_stringhash` (ord_no,db_nm) VALUES (171022553756, database()); select * from sharding_by_stringhash;

 

測試結果：

[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE sharding_by_stringhash (ord_no varchar(20) NULL,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_stringhash` (ord_no,db_nm) VALUES (171022237582, database()); INSERT INTO `sharding_by_stringhash` (ord_no,db_nm) VALUES (171022553756, database()); select * from sharding_by_stringhash; Query OK, 0 rows affected (0.18 sec) mysql> INSERT INTO `sharding_by_stringhash` (ord_no,db_nm) VALUES (171022237582, database()); Query OK, 1 row affected (0.07 sec) mysql> INSERT INTO `sharding_by_stringhash` (ord_no,db_nm) VALUES (171022553756, database()); Query OK, 1 row affected (0.00 sec) mysql> select * from sharding_by_stringhash; +--------------+-------+ | ord_no | db_nm | +--------------+-------+ | 171022237582 | db0 | | 171022553756 | db1 | +--------------+-------+ 2 rows in set (0.09 sec)

 

注意 
* 分片數量必須**小於等於**dataNode數

2.2.5 一致性hash分片

一致性hash算法有效解決了分布式數據的擴容問題。因為此規則優點在於擴容時遷移數據量比較少，前提是分片節點比較多，虛擬節點分配多些。虛擬節點分配的少就會造成數據分布不夠均勻。但如果實際分片數據比較少，遷移量也會比較多。

vim conf/schema.xml

<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="sharding_by_murmurhash" primaryKey="id" dataNode="dn$0-2" rule="sharding-by-murmurhash" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>

 

vim conf/rule.xml

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="sharding-by-murmurhash"> <rule> <columns>id</columns> <algorithm>sharding-by-murmurhash</algorithm> </rule> </tableRule> <function name="sharding-by-murmurhash" class="io.mycat.route.function.PartitionByMurmurHash"> <property name="seed">0</property><!-- 默認是0 --> <property name="count">2</property><!-- 要分片的數據庫節點數量，必須指定，否則沒法分片 --> <property name="virtualBucketTimes">160</property><!-- 一個實際的數據庫節點被映射為這么多虛擬節點，默認是160倍，也就是虛擬節點數是物理節點數的160倍 --> <!-- <property name="weightMapFile">weightMapFile</property> 節點的權重，沒有指定權重的節點默認是1。以properties文件的格式填寫，以從0開始到count-1的 整數值也就是節點索引為key，以節點權重值為值。所有權重值必須是正整數，否則以1代替 --> <!-- <property name="bucketMapPath">/etc/mycat/bucketMapPath</property> 用於測試時觀察各物理節點與虛擬節點的分布情況，如果指定了這個屬性，會把虛擬節點的murmur hash值與物理節點的映射按行輸出到 這個文件，沒有默認值> ，如果不指定，就不會輸出任何東西 --> </function> </mycat:rule>

 

 

測試SQL：

CREATE TABLE sharding_by_murmurhash (id int(10) null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (1, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (2, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (3, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (4, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (5, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (6, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (7, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (8, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (9, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (10, database()); select * from sharding_by_murmurhash;

 

 

測試結果：

[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE sharding_by_murmurhash (id int(10) null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (1, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (2, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (3, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (4, database()); Query OK, 0 rows affected (0.11 sec) mysql> INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (1, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (5, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (2, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (6, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (7, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (8, database()); Query OK, 1 row affected (0.05 sec) mysql> INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (3, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (4, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (9, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (5, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (6, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (7, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (8, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (10, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (9, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (10, database()); Query OK, 1 row affected (0.01 sec) mysql> select * from sharding_by_murmurhash; +------+-------+ | id | db_nm | +------+-------+ | 1 | db1 | | 2 | db1 | | 3 | db1 | | 4 | db1 | | 7 | db1 | | 9 | db1 | | 10 | db1 | | 5 | db0 | | 6 | db0 | | 8 | db0 | +------+-------+ 10 rows in set (0.08 sec)

 

注意 
* 分片數量必須**小於等於**dataNode數

3. 綜合類分片規則

  • 范圍求模分片
  • 日期范圍hash分片
  • 取模范圍約束分片
  • ASCII碼求模范圍約束（字符串）
  • 固定分片hash（二進制）

3.1 范圍求模分片

該算法先進行范圍分片，計算出分片組，組內再求模，綜合了范圍分片和求模分片的優點。分片組內使用求模可以保證組內的數據分布比較均勻，分片組之間采用范圍分片可以兼顧范圍分片的特點。事先規定好分片的數量，數據擴容時按分片組擴容，則原有分片組的數據不需要遷移。由於分片組內的數據分布比較均勻，所以分片組內可以避免熱點數據問題。

vim conf/schema.xml

<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="sharding_by_rang_mod" primaryKey="id" dataNode="dn$0-10" rule="sharding-by-rang-mod" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataNode name="dn3" dataHost="dh-1" database="db3"/> <dataNode name="dn4" dataHost="dh-1" database="db4"/> <dataNode name="dn5" dataHost="dh-1" database="db5"/> <dataNode name="dn6" dataHost="dh-1" database="db6"/> <dataNode name="dn7" dataHost="dh-1" database="db7"/> <dataNode name="dn8" dataHost="dh-1" database="db8"/> <dataNode name="dn9" dataHost="dh-1" database="db9"/> <dataNode name="dn10" dataHost="dh-1" database="db10"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>

 

vim conf/rule.xml

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="sharding-by-rang-mod"> <rule> <columns>id</columns> <algorithm>sharding-by-rang-mod</algorithm> </rule> </tableRule> <function name="sharding-by-rang-mod" class="io.mycat.route.function.PartitionByRangeMod"> <property name="mapFile">partition-range-mod.txt</property> <property name="defaultNode">0</property> </function> </mycat:rule>

 

vim conf/partition-range-mod.txt

# range start-end ,data node group size 0-200M=3 200M1-400M=1 400M1-600M=4 600M1-800M=2

 

配置說明： 
* columns標識將要分片的表字段。 
* algorithm為分片函數。 
* mapFile為指定分片規則的配置文件。 
* defaultNode為未包含以上規則的數據存儲節點，節點從0開始。

測試SQL：

CREATE TABLE sharding_by_rang_mod (id bigint null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (1000, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (1002, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (30000, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (30004, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (40000, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (40005, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60005, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60006, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (80006, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60008, database()); select * from sharding_by_rang_mod;

 

測試結果：

[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE sharding_by_rang_mod (id bigint null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (1000, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (1002, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (30000, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (30004, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (40000, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (40005, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60005, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60006, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (80006, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60008, database()); select * from sharding_by_rang_mod; Query OK, 0 rows affected (0.58 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (1000, database()); Query OK, 1 row affected (0.08 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (1002, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (30000, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (30004, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (40000, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (40005, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60005, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60006, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (80006, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60008, database()); Query OK, 1 row affected (0.00 sec) mysql> select * from sharding_by_rang_mod; +-------+-------+ | id | db_nm | +-------+-------+ | 1002 | db0 | | 80006 | db0 | | 30000 | db3 | | 30004 | db3 | | 40000 | db3 | | 1000 | db1 | | 40005 | db5 | | 60005 | db9 | | 60006 | db8 | | 60008 | db8 | +-------+-------+ 10 rows in set (0.12 sec) mysql>

注意 
* 分片數量必須**小於等於**dataNode數

3.2 日期范圍hash分片

其思想與范圍求模一致，由於日期取模方法會出現數據熱點問題，所以先根據日期分組，再根據時間hash使得短期內數據分布得更均勻。其優點是可以避免擴容時的數據遷移，又可以在一定程度上避免范圍分片的熱點問題，要求日期格式盡量精確，不然達不到局部均勻的目的。

vim conf/schema.xml

<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="sharding_by_rang_mod" primaryKey="id" dataNode="dn$0-10" rule="sharding-by-rang-mod" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataNode name="dn3" dataHost="dh-1" database="db3"/> <dataNode name="dn4" dataHost="dh-1" database="db4"/> <dataNode name="dn5" dataHost="dh-1" database="db5"/> <dataNode name="dn6" dataHost="dh-1" database="db6"/> <dataNode name="dn7" dataHost="dh-1" database="db7"/> <dataNode name="dn8" dataHost="dh-1" database="db8"/> <dataNode name="dn9" dataHost="dh-1" database="db9"/> <dataNode name="dn10" dataHost="dh-1" database="db10"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>

vim conf/rule.xml

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="sharding-by-rang-mod"> <rule> <columns>id</columns> <algorithm>sharding-by-rang-mod</algorithm> </rule> </tableRule> <function name="sharding-by-rang-mod" class="io.mycat.route.function.PartitionByRangeMod"> <property name="mapFile">partition-range-mod.txt</property> <property name="defaultNode">0</property> </function> </mycat:rule>

 

配置說明： 
* columns標識將要分片的表字段。 
* algorithm為分片函數。 
* mapFile為指定分片規則的配置文件。 
* defaultNode為未包含以上規則的數據存儲節點，節點從0開始。

vim conf/partition-range-mod.txt

# range start-end ,data node group size 0-200M=3 200M1-400M=1 400M1-600M=4 600M1-800M=2

 

測試SQL：

CREATE TABLE sharding_by_rang_mod (id bigint null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (1000, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (1002, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (30000, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (30004, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (40000, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (40005, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60005, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60006, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (80006, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60008, database()); select * from sharding_by_rang_mod;

 

測試結果：

[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE sharding_by_rang_mod (id bigint null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (1000, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (1002, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (30000, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (30004, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (40000, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (40005, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60005, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60006, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (80006, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60008, database()); select * from sharding_by_rang_mod; Query OK, 0 rows affected (0.58 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (1000, database()); Query OK, 1 row affected (0.08 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (1002, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (30000, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (30004, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (40000, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (40005, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60005, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60006, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (80006, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60008, database()); Query OK, 1 row affected (0.00 sec) mysql> select * from sharding_by_rang_mod; +-------+-------+ | id | db_nm | +-------+-------+ | 1002 | db0 | | 80006 | db0 | | 30000 | db3 | | 30004 | db3 | | 40000 | db3 | | 1000 | db1 | | 40005 | db5 | | 60005 | db9 | | 60006 | db8 | | 60008 | db8 | +-------+-------+ 10 rows in set (0.12 sec) mysql>

 

注意 
* 分片數量必須**小於等於**dataNode數

3.3 取模范圍約束分片

取模運算與范圍約束的結合，主要為了后續數據遷移做准備，即可以自主決定取模后數據的節點分布。

vim conf/schema.xml

<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="sharding_by_pattern" primaryKey="id" dataNode="dn$0-10" rule="sharding-by-pattern" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataNode name="dn3" dataHost="dh-1" database="db3"/> <dataNode name="dn4" dataHost="dh-1" database="db4"/> <dataNode name="dn5" dataHost="dh-1" database="db5"/> <dataNode name="dn6" dataHost="dh-1" database="db6"/> <dataNode name="dn7" dataHost="dh-1" database="db7"/> <dataNode name="dn8" dataHost="dh-1" database="db8"/> <dataNode name="dn9" dataHost="dh-1" database="db9"/> <dataNode name="dn10" dataHost="dh-1" database="db10"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>

 

vim conf/rule.xml

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="sharding-by-pattern"> <rule> <columns>id</columns> <algorithm>sharding-by-pattern</algorithm> </rule> </tableRule> <function name="sharding-by-pattern" class="io.mycat.route.function.PartitionByPattern"> <property name="mapFile">partition-pattern.txt</property> <property name="defaultNode">0</property> <property name="patternValue">256</property> </function> </mycat:rule>

 

配置說明： 
* columns標識將要分片的表字段。 
* algorithm為分片函數。 
* mapFile為指定分片規則的配置文件。 
* defaultNode為未包含以上規則的數據存儲節點，節點從0開始。 
* patternValue為求模基數。

vim conf/partition-pattern.txt

# id patition range start-end, data node index ## first host configuration 1-32=0 33-64=1 65-96=2 97-128=3 ## second host configuration 129-160=4 161-192=5 193-224=6 225-256=7 0-0=7

 

配置說明： 
* 1-32即代表id%256后分布的范圍，如果在1-32則分區在0，其他類推。 
* 如果id非數字，則會分配在defaultNode默認節點。

測試SQL：

CREATE TABLE sharding_by_pattern (id varchar(20) null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (1000, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (1002, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (30000, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (30004, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (40000, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (40005, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (60005, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (60006, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (80006, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES ("abcd0", database()); select * from sharding_by_pattern;

 

測試結果：

[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE sharding_by_pattern (id varchar(20) null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (1000, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (1002, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (30000, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (30004, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (40000, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (40005, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (60005, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (60006, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (80006, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES ("abcd0", database()); select * from sharding_by_pattern; Query OK, 0 rows affected (0.72 sec) mysql> INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (1000, database()); Query OK, 1 row affected (0.08 sec) mysql> INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (1002, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (30000, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (30004, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (40000, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (40005, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (60005, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (60006, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (80006, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES ("abcd0", database()); Query OK, 1 row affected (0.07 sec) mysql> select * from sharding_by_pattern; +-------+-------+ | id | db_nm | +-------+-------+ | 80006 | db4 | | 30000 | db1 | | 30004 | db1 | | 40000 | db1 | | 1000 | db7 | | 1002 | db7 | | 60005 | db3 | | 60006 | db3 | | 40005 | db2 | | abcd0 | db0 | +-------+-------+ 10 rows in set (0.14 sec) mysql>

 

注意 
* 分片數量必須**小於等於**dataNode數

3.4 ASCII碼求模范圍約束分片（字符串）

此種規則類似於取模范圍約束，此規則支持數據符號字母取模。

vim conf/schema.xml

<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="sharding_by_pattern" primaryKey="id" dataNode="dn$0-10" rule="sharding-by-pattern" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataNode name="dn3" dataHost="dh-1" database="db3"/> <dataNode name="dn4" dataHost="dh-1" database="db4"/> <dataNode name="dn5" dataHost="dh-1" database="db5"/> <dataNode name="dn6" dataHost="dh-1" database="db6"/> <dataNode name="dn7" dataHost="dh-1" database="db7"/> <dataNode name="dn8" dataHost="dh-1" database="db8"/> <dataNode name="dn9" dataHost="dh-1" database="db9"/> <dataNode name="dn10" dataHost="dh-1" database="db10"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>

 

vim conf/rule.xml

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="sharding-by-pattern"> <rule> <columns>id</columns> <algorithm>sharding-by-pattern</algorithm> </rule> </tableRule> <function name="sharding-by-pattern" class="io.mycat.route.function.PartitionByPattern"> <property name="mapFile">partition-pattern.txt</property> <property name="defaultNode">0</property> <property name="patternValue">256</property> </function> </mycat:rule>

 

配置說明： 
* columns標識將要分片的表字段。 
* algorithm為分片函數。 
* mapFile為指定分片規則的配置文件。 
* defaultNode為未包含以上規則的數據存儲節點，節點從0開始。 
* patternValue為求模基數。

vim conf/partition-pattern.txt

# id patition range start-end, data node index ## first host configuration 1-32=0 33-64=1 65-96=2 97-128=3 ## second host configuration 129-160=4 161-192=5 193-224=6 225-256=7 0-0=7

 

配置說明： 
* 1-32即代表id%256后分布的范圍，如果在1-32則分區在0，其他類推。 
* 如果id非數字，則會分配在defaultNode默認節點。

測試SQL：

CREATE TABLE sharding_by_prefix_pattern (id varchar(20) null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("1000a", database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("1002A", database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (30000, database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (30004, database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("4000B", database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("4000b", database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (60007, database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (60006, database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (80006, database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("abcd0", database()); select * from sharding_by_prefix_pattern;

 

測試結果：

[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 4 Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE sharding_by_prefix_pattern (id varchar(20) null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("1000a", database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("1002A", database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (30000, database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (30004, database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("4000B", database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("4000b", database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (60007, database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (60006, database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (80006, database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("abcd0", database()); select * from sharding_by_prefix_pattern; Query OK, 0 rows affected (0.27 sec) mysql> INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("1000a", database()); Query OK, 1 row affected (0.03 sec) mysql> INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("1002A", database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (30000, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (30004, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("4000B", database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("4000b", database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (60007, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (60006, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (80006, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("abcd0", database()); Query OK, 1 row affected (0.01 sec) mysql> select * from sharding_by_prefix_pattern; +-------+-------+ | id | db_nm | +-------+-------+ | 1000a | db3 | | 1002A | db3 | | 30000 | db3 | | 30004 | db3 | | 4000B | db3 | | 4000b | db3 | | 60007 | db3 | | 60006 | db3 | | 80006 | db3 | | abcd0 | db6 | +-------+-------+ 10 rows in set (0.01 sec) mysql>

注意 
* 分片數量必須**小於等於**dataNode數

3.5 固定分片hash算法分片（二進制）

本條規則類似於十進制的求模運算，區別在於是二進制的操作，是取id的二進制低10位，即id二進制&1111111111。此算法的優點在於如果按照十進制取模運算，則在連續插入1~10時，1~10會被分到1~10個分片，增大了插入事務的控制難度。而此算法根據二進制則可能會分到連接的分片，降低了插入事務的控制難度。

vim conf/schema.xml

<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="sharding_by_long" primaryKey="id" dataNode="dn$0-10" rule="sharding-by-long" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataNode name="dn3" dataHost="dh-1" database="db3"/> <dataNode name="dn4" dataHost="dh-1" database="db4"/> <dataNode name="dn5" dataHost="dh-1" database="db5"/> <dataNode name="dn6" dataHost="dh-1" database="db6"/> <dataNode name="dn7" dataHost="dh-1" database="db7"/> <dataNode name="dn8" dataHost="dh-1" database="db8"/> <dataNode name="dn9" dataHost="dh-1" database="db9"/> <dataNode name="dn10" dataHost="dh-1" database="db10"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>

 

vim conf/rule.xml

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="sharding-by-long"> <rule> <columns>id</columns> <algorithm>sharding-by-long</algorithm> </rule> </tableRule> <function name="sharding-by-long" class="io.mycat.route.function.PartitionByLong"> <property name="partitionCount">2,1</property> <property name="partitionLength">256,512</property> </function> </mycat:rule>

 

配置說明： 
* columns標識將要分片的表字段。 
* algorithm為分片函數。 
* partitionCount為指定分片個數列表。 
* partitionLength為分片范圍列表。 
分區長度:默認為最大2^n=1024 ,即最大支持1024分區 
約束 : 
count,length兩個數組的長度必須是一致的。 
1024 = sum((count[i]*length[i])). count和length兩個向量的點積恆等於1024 
本例的分區策略：希望將數據水平分成3份，前兩份各占25%，第三份占50%。（故本例非均勻分區） 
// |<———————1024———————————>| 
// |<—-256—>|<—-256—>|<———-512————->| 
// | partition0 | partition1 | partition2 | 
// | 共2份,故count[0]=2 | 共1份，故count[1]=1 | 
int[] count = new int[] { 2, 1 }; 
int[] length = new int[] { 256, 512 };

vim conf/partition-pattern.txt

測試SQL：

CREATE TABLE sharding_by_long (id int(10) null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (1000, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (1002, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (30000, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (30004, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (4000, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (4000, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (60007, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (60006, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (80006, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (0, database()); select * from sharding_by_long;

 

測試結果：

[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 4 Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE sharding_by_long (id int(10) null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (1000, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (1002, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (30000, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (30004, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (4000, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (4000, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (60007, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (60006, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (80006, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (0, database()); select * from sharding_by_long; Query OK, 0 rows affected (0.22 sec) mysql> INSERT INTO `sharding_by_long` (id,db_nm) VALUES (1000, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_long` (id,db_nm) VALUES (1002, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_long` (id,db_nm) VALUES (30000, database()); Query OK, 1 row affected (0.04 sec) mysql> INSERT INTO `sharding_by_long` (id,db_nm) VALUES (30004, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_long` (id,db_nm) VALUES (4000, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_long` (id,db_nm) VALUES (4000, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_long` (id,db_nm) VALUES (60007, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_long` (id,db_nm) VALUES (60006, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_long` (id,db_nm) VALUES (80006, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_long` (id,db_nm) VALUES (0, database()); Query OK, 1 row affected (0.00 sec) mysql> select * from sharding_by_long; +-------+-------+ | id | db_nm | +-------+-------+ | 80006 | db0 | | 0 | db0 | | 30000 | db1 | | 30004 | db1 | | 1000 | db2 | | 1002 | db2 | | 4000 | db2 | | 4000 | db2 | | 60007 | db2 | | 60006 | db2 | +-------+-------+ 10 rows in set (0.01 sec) mysql>

 

注意 
* 分片數量必須**小於等於**dataNode數

4. Mycat分片場景和策略

  • 根據表數據量判斷是否需要切分，確保切分后單分片表數據量為1000W左右
  • 根據業務的情況選擇合適的分片字段： 最頻繁的或者最重要的查詢條件
  • 需要考慮擴容數據遷移問題（范圍類，范圍取模類不需要遷移，哈希類需要遷移）
  • 有關聯關系的表配置相同分片規則（ER思想，為了應用join等復雜sql），一對多對應關系一般按多的那一方切分
  • 如果配置類數據， 更新頻率比較少，考慮全局表

Alt text
Alt text


免責聲明！

本站轉載的文章為個人學習借鑒使用，本站對版權不負任何法律責任。如果侵犯了您的隱私權益，請聯系本站郵箱yoyou2525@163.com刪除。



猜您在找 Mycat分片規則詳解 Mycat跨分片Join mycat分片規則 mycat分片及主從（二） MyCAT常用分片規則之分片枚舉 mycat 中的dataHost和dataNode 以及如何按月分片 Mycat探索之旅（5）----常用的分片規則 Mycat 分片規則詳解--數據遷移及節點擴容 MyCat + PostgreSQL不支持二級子表分片 linux中mycat的配置，分片，以及主從復制
 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM