Windows環境Mycat數據庫分庫分表中間件部署

下載地址MYCAT官方網站

jdk安裝配置

1. 首先去oracle官網下載並安裝jdk8,添加環境變量，JAVA_HOME設置為D:\Worksoftware\Java\jdk1.8
2. CLASSPATH設置為.;%JAVA_HOME%\lib;%JAVA_HOME%\lib\tools.jar
3. path系統變量追加%JAVA_HOME%\bin;

---

Mycat安裝配置

1. 首先添加Windows環境變量，MYCAT_HOME設置為安裝目錄E:\WorkSoftWare\MycatServer1.5

2. 為了降低資源占用，mycat的jvm設置在startup_nowrap.bat，打開后可以清楚看到如下配置：
   “%JAVA_CMD%” -server -Xms1G -Xmx2G -XX:MaxPermSize=64M -XX:+AggressiveOpts -XX:MaxDirectMemorySize=1G -DMYCAT_HOME=%MYCAT_HOME% -cp “..\conf;..\lib*” io.mycat.MycatStartup
   這里將-Xms1G改成-Xms512M，-Xmx2G改成-Xmx1024M，保存后重新啟動即可.

3. wrapper.conf文件里的改成wrapper.java.command=E:\WorkSoftWare\jdk1.8\bin\java.exe

---

Mycat啟動的配置

schema.xml配置文件，如果分庫在不同的服務器，可以配置兩個datahost；如果在一個datahost中配置多個writeHost，則為主從配置。type=”global”時，為全局表

<?xml version="1.0"?>
<!DOCTYPE mycat:schema SYSTEM "schema.dtd">
<mycat:schema xmlns:mycat="http://org.opencloudb/" >

<!--在這一行參數里面,schema name定義了可以在MyCAT前端顯示的邏輯數據庫的名字,checkSQLschema這個參數為False的時候,表明MyCAT會自動忽略掉表名前的數據庫名,比如說mydatabase1.test1,會被當做test1;sqlMaxLimit指定了SQL語句返回的行數限制-->
    <schema name="TESTDB" checkSQLschema="false" sqlMaxLimit="100">
        <!-- 主鍵范圍規則 -->
        <!-- 這一行代表在MyCAT前端會顯示哪些表名,類似幾行都代表一樣的意思,這里強調的是表,而MyCAT並不會在配置文件里面定義表結構，如果在前端使用show create table ,MyCAT會顯示正常的表結構信息,觀察Debug日志,可以看到,MyCAT把命令分發給了dn1代表的數據庫,然后把dn1的查詢結果返回給了前端 可以判斷,類似的數據庫級別的一些查詢指令,有可能是單獨分發給某個節點,然后再把某個節點的信息返回給前端。
        dataNode的意義很簡單,這個邏輯表的數據存儲在后端的哪幾個數據庫里面rule代表的是這個邏輯表students的具體切分策略,目前MyCAT只支持按照某一個特殊列,遵循一些特殊的規則來切分,如取模,枚舉等,具體的留給之后細說                  
            --> 
        <table name="travelrecord" dataNode="dn1,dn2,dn3" rule="auto-sharding-long" />

        <table name="company" primaryKey="ID" dataNode="dn3,dn2,dn1" rule="mod-long"/>
        <table name="goods" primaryKey="ID" type="global" dataNode="dn1,dn2" />
        <!--求模分片隨機規則 -->
        <table name="hotnews" primaryKey="ID" dataNode="dn1,dn2,dn3"
            rule="mod-long" />

        <table name="employee" primaryKey="ID" dataNode="dn1,dn2"
            rule="sharding-by-intfile" />
        <table name="customer" primaryKey="ID" dataNode="dn1,dn2"
            rule="sharding-by-intfile">
             <!-- 
                childtable我在測試中並沒有實際用起來不過在MyCAT的設計文檔里面有提到，childtable是一種依賴於父表的結構，
                這意味着，childtable的joinkey會按照父表的parentKey的策略一起切分，當父表與子表進行連接，
                且連接條件是childtable.joinKey=parenttable.parentKey時,不會進行跨庫的連接.
                --> 
            <childTable name="orders" primaryKey="ID" joinKey="customer_id"
                parentKey="id">
                <childTable name="order_items" joinKey="order_id"
                    parentKey="id" />
            </childTable>
            <childTable name="customer_addr" primaryKey="ID" joinKey="customer_id"
                parentKey="id" />
        </table>

        <!-- 全局表是自動克隆到所有定義的數據節點，這樣可以與拆分節點的任何表連接查詢，是在同一個數據節點-->
        <table name="news_table" primaryKey="ID" type="global" dataNode="dn1,dn2,dn3" />
    </schema>

    <dataNode name="dn1" dataHost="localhost1" database="mycatDB1" />
    <dataNode name="dn2" dataHost="localhost1" database="mycatDB2" />
    <dataNode name="dn3" dataHost="localhost1" database="mycatDB3" />

    <!--
          dataHost配置的是實際的后端數據庫集群,大部分參數簡單易懂,這里就不一個個介紹了,只介紹比較重要的兩個參數,writeType和balance.
         --> 
        <!-- writeType和balance是用來控制后端集群的讀寫分離的關鍵參數，這里我用了雙主雙從的集群配置
            這里的測試過程比較麻煩，所以直接貼結論：
           1.balance=0時,讀操作都在localhost上(localhost失敗時,后端直接失敗)
           2.balance=1時,讀操作會隨機分散在localhost1和兩個readhost上面(localhost失敗時,寫操作會在localhost1,如果localhost1再失敗,則無法進行寫操作)
           3.balance=2時,寫操作會在localhost上，讀操作會隨機分散在localhost1,localhost1和兩個readhost上面(同上)
           4.writeType=0時,寫操作會在localhost上,如果localhost失敗,會自動切換到localhost1,localhost恢復以后並不會切換回localhost進行寫操作
           5.writeType=1時,寫操作會隨機分布在localhost和localhost1上,單點失敗並不會影響集群的寫操作,但是后端的從庫會無法從掛掉的主庫獲取更新,會在讀數據的時候出現數據不一致
           舉例:localhost失敗了,寫操作會在localhost1上面進行,localhost1的主從正常運行,但是localhost的從庫無法從localhost獲取更新,localhost的從庫於其他庫出現數據不一致
         --> 
    <dataHost name="localhost1" maxCon="1000" minCon="10" balance="0"
        writeType="0" dbType="mysql" dbDriver="native" switchType="1"  slaveThreshold="100">
        <heartbeat>select user()</heartbeat>
        <!-- can have multi write hosts -->
        <writeHost host="hostM1" url="localhost:3306" user="root"
            password="123456">
            <!-- can have multi read hosts -->
            <!--<readHost host="hostS2" url="192.168.1.200:3306" user="root" password="xxx" />-->
        </writeHost>
    </dataHost>

</mycat:schema>

 

---

server.xml配置文件

<?xml version="1.0" encoding="UTF-8"?>
<!-- - - Licensed under the Apache License, Version 2.0 (the "License"); 
    - you may not use this file except in compliance with the License. - You 
    may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 
    - - Unless required by applicable law or agreed to in writing, software - 
    distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT 
    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the 
    License for the specific language governing permissions and - limitations 
    under the License. -->
<!DOCTYPE mycat:server SYSTEM "server.dtd">
<mycat:server xmlns:mycat="http://org.opencloudb/">
    <system>
    <property name="defaultSqlParser">druidparser</property>
      <!--  <property name="useCompression">1</property>--> <!--1為開啟mysql壓縮協議-->
    <!-- <property name="processorBufferChunk">40960</property> -->
    <!-- 
    <property name="processors">1</property> 
    <property name="processorExecutor">32</property> 
     -->
        <!--默認是65535 64K 用於sql解析時最大文本長度 -->
        <!--<property name="maxStringLiteralLength">65535</property>-->
        <!--<property name="sequnceHandlerType">0</property>-->
        <!--<property name="backSocketNoDelay">1</property>-->
        <!--<property name="frontSocketNoDelay">1</property>-->
        <!--<property name="processorExecutor">16</property>-->
        <!-- 
            <property name="mutiNodeLimitType">1</property> 0：開啟小數量級（默認） ；1：開啟億級數據排序
            <property name="mutiNodePatchSize">100</property> 億級數量排序批量
            <property name="processors">32</property> <property name="processorExecutor">32</property> 
            <property name="serverPort">8066</property> <property name="managerPort">9066</property> 
            <property name="idleTimeout">300000</property> <property name="bindIp">0.0.0.0</property> 
            <property name="frontWriteQueueSize">4096</property> <property name="processors">32</property> -->
    </system>
    <user name="test"><!--用戶名-->
        <property name="password">test</property><!--密碼-->
         <!--實例名，和schema.xml定義的schema對應，這里的實例名是虛擬名，也就是對mycat服務的一種別名，是應用程序以及客戶端連接的入口。-->
        <property name="schemas">TESTDB</property>
    </user>

    <user name="user">
        <property name="password">user</property>
        <property name="schemas">TESTDB</property>
        <property name="readOnly">true</property>
    </user>
    <!-- 
    <quarantine> 
       <whitehost>
          <host host="127.0.0.1" user="mycat"/>
          <host host="127.0.0.2" user="mycat"/>
       </whitehost>
       <blacklist check="false"></blacklist>
    </quarantine>
    -->

</mycat:server>

 

---

rule.xml配置文件

<?xml version="1.0" encoding="UTF-8"?>
<!-- - - Licensed under the Apache License, Version 2.0 (the "License"); 
    - you may not use this file except in compliance with the License. - You 
    may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 
    - - Unless required by applicable law or agreed to in writing, software - 
    distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT 
    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the 
    License for the specific language governing permissions and - limitations 
    under the License. -->
<!DOCTYPE mycat:rule SYSTEM "rule.dtd">
<mycat:rule xmlns:mycat="http://org.opencloudb/">
    <tableRule name="rule1">
        <rule>
            <columns>id</columns>
            <algorithm>func1</algorithm>
        </rule>
    </tableRule>

    <tableRule name="rule2">
        <rule>
            <columns>user_id</columns>
            <algorithm>func1</algorithm>
        </rule>
    </tableRule>

    <tableRule name="sharding-by-intfile">
        <rule>
            <columns>sharding_id</columns>
            <algorithm>hash-int</algorithm>
        </rule>
    </tableRule>
    <tableRule name="auto-sharding-long">
        <rule>
            <columns>id</columns>
            <algorithm>rang-long</algorithm>
        </rule>
    </tableRule>
    <tableRule name="mod-long">
        <rule>
            <columns>id</columns>
            <algorithm>mod-long</algorithm>
        </rule>
    </tableRule>
    <tableRule name="sharding-by-murmur">
        <rule>
            <columns>id</columns>
            <algorithm>murmur</algorithm>
        </rule>
    </tableRule>
    <tableRule name="sharding-by-month">
        <rule>
            <columns>create_date</columns>
            <algorithm>partbymonth</algorithm>
        </rule>
    </tableRule>
    <tableRule name="latest-month-calldate">
        <rule>
            <columns>calldate</columns>
            <algorithm>latestMonth</algorithm>
        </rule>
    </tableRule>

    <tableRule name="auto-sharding-rang-mod">
        <rule>
            <columns>id</columns>
            <algorithm>rang-mod</algorithm>
        </rule>
    </tableRule>

    <tableRule name="jch">
        <rule>
            <columns>id</columns>
            <algorithm>jump-consistent-hash</algorithm>
        </rule>
    </tableRule>

    <function name="murmur"
        class="org.opencloudb.route.function.PartitionByMurmurHash">
        <property name="seed">0</property><!-- 默認是0 -->
        <property name="count">2</property><!-- 要分片的數據庫節點數量，必須指定，否則沒法分片 -->
        <property name="virtualBucketTimes">160</property><!-- 一個實際的數據庫節點被映射為這么多虛擬節點，默認是160倍，也就是虛擬節點數是物理節點數的160倍 -->
        <!-- <property name="weightMapFile">weightMapFile</property> 節點的權重，沒有指定權重的節點默認是1。以properties文件的格式填寫，以從0開始到count-1的整數值也就是節點索引為key，以節點權重值為值。所有權重值必須是正整數，否則以1代替 -->
        <!-- <property name="bucketMapPath">/etc/mycat/bucketMapPath</property> 
            用於測試時觀察各物理節點與虛擬節點的分布情況，如果指定了這個屬性，會把虛擬節點的murmur hash值與物理節點的映射按行輸出到這個文件，沒有默認值，如果不指定，就不會輸出任何東西 -->
    </function>
    <function name="hash-int"
        class="org.opencloudb.route.function.PartitionByFileMap">
        <property name="mapFile">partition-hash-int.txt</property>
    </function>
    <function name="rang-long"
        class="org.opencloudb.route.function.AutoPartitionByLong">
        <property name="mapFile">autopartition-long.txt</property>
    </function>
    <function name="mod-long" class="org.opencloudb.route.function.PartitionByMod">
        <!-- how many data nodes -->
        <property name="count">3</property>
    </function>

    <function name="func1" class="org.opencloudb.route.function.PartitionByLong">
        <property name="partitionCount">8</property>
        <property name="partitionLength">128</property>
    </function>
    <function name="latestMonth"
        class="org.opencloudb.route.function.LatestMonthPartion">
        <property name="splitOneDay">24</property>
    </function>
    <function name="partbymonth"
        class="org.opencloudb.route.function.PartitionByMonth">
        <property name="dateFormat">yyyy-MM-dd</property>
        <property name="sBeginDate">2015-01-01</property>
    </function>

    <function name="rang-mod" class="org.opencloudb.route.function.PartitionByRangeMod">
            <property name="mapFile">partition-range-mod.txt</property>
    </function>

    <function name="jump-consistent-hash" class="org.opencloudb.route.function.PartitionByJumpConsistentHash">
        <property name="totalBuckets">3</property>
    </function>
</mycat:rule>

 

Mycat連接mysql示例

cmd命令運行:

結果:

Mycat連接mysql結果

上圖上面的mycatdb1,mycatdb2,mycatdb3數據庫是真實物理數據庫，TESTDB是中間件邏輯數據庫，此時已經所有工作都准備好。

閑的蛋疼，隨便研究一些好東西，.net環境太封閉，每個.net程序員都要擴展視野，技術交流，本人QQ827937686