(一)、SpringBoot2.0.3.RELEASE+sharding-jdbc4.1.0+mybatis-plus3.4.1+druid1.1.22 快速搭建分庫分表


文末有demo的git地址

Apache ShardingSphere 是一套開源的分布式數據庫解決方案組成的生態圈,它由 JDBC、Proxy 和 Sidecar(規划中)這 3 款既能夠獨立部署,又支持混合部署配合使用的產品組成。

目前,數據分片、讀寫分離、數據加密、影子庫壓測等功能,以及對 MySQL、PostgreSQL、SQLServer、Oracle 等 SQL 與協議的支持,均通過插件的方式織入項目。

ShardingSphere-JDBC

定位為輕量級 Java 框架,在 Java 的 JDBC 層提供的額外服務。 它使用客戶端直連數據庫,以 jar 包形式提供服務,無需額外部署和依賴,可理解為增強版的 JDBC 驅動,完全兼容 JDBC 和各種 ORM 框架。

  • 適用於任何基於 JDBC 的 ORM 框架,如:JPA, Hibernate, Mybatis, Spring JDBC Template 或直接使用 JDBC。
  • 支持任何第三方的數據庫連接池,如:DBCP, C3P0, BoneCP, Druid, HikariCP 等。
  • 支持任意實現 JDBC 規范的數據庫,目前支持 MySQL,Oracle,SQLServer,PostgreSQL 以及任何遵循 SQL92 標准的數據庫。

ShardingSphere-Proxy

定位為透明化的數據庫代理端,提供封裝了數據庫二進制協議的服務端版本,用於完成對異構語言的支持。 目前提供 MySQL 和 PostgreSQL 版本,它可以使用任何兼容 MySQL/PostgreSQL 協議的訪問客戶端(如:MySQL Command Client, MySQL Workbench, Navicat 等)操作數據,對 DBA 更加友好。

  • 向應用程序完全透明,可直接當做 MySQL/PostgreSQL 使用。
  • 適用於任何兼容 MySQL/PostgreSQL 協議的的客戶端。

本次我們主要使用SHardingJDBC進行分庫分表

首先創建數據庫

    

其中t_order是訂單表,t_order_item訂單商品信息表,t_config廣播表,ds_user.t_user專庫專表。也就是ShardingJDBC不僅可以配置分庫分表,還可以配置單獨的數據庫使用

腳本已經放入git項目中

 導入依賴

<dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <!-- mysql -->
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <scope>runtime</scope>
        </dependency>
        <!-- https://mvnrepository.com/artifact/com.alibaba/druid-spring-boot-starter -->
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>druid-spring-boot-starter</artifactId>
            <version>1.1.22</version>
        </dependency>
        <!-- lombok -->
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <!-- mybatis plus 代碼生成器 -->
        <dependency>
            <groupId>org.mybatis.spring.boot</groupId>
            <artifactId>mybatis-spring-boot-starter</artifactId>
            <version>2.1.3</version>
        </dependency>
        <dependency>
            <groupId>com.baomidou</groupId>
            <artifactId>mybatis-plus-boot-starter</artifactId>
            <version>3.4.1</version>
        </dependency>
        <dependency>
            <groupId>com.baomidou</groupId>
            <artifactId>mybatis-plus-generator</artifactId>
            <version>3.4.1</version>
        </dependency>
        <dependency>
            <groupId>org.freemarker</groupId>
            <artifactId>freemarker</artifactId>
            <version>2.3.28</version>
        </dependency>
        <!-- fastjson -->
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>fastjson</artifactId>
            <version>1.2.47</version>
        </dependency>
        <!-- ShardingSphere -->
        <dependency>
            <groupId>org.apache.shardingsphere</groupId>
            <artifactId>sharding-jdbc-spring-boot-starter</artifactId>
            <version>4.1.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.shardingsphere</groupId>
            <artifactId>sharding-core-common</artifactId>
            <version>4.1.0</version>
        </dependency>
        <!-- commons-lang3 -->
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-lang3</artifactId>
            <version>3.10</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/cn.hutool/hutool-all -->
        <dependency>
            <groupId>cn.hutool</groupId>
            <artifactId>hutool-all</artifactId>
            <version>5.5.8</version>
        </dependency>

然后通過代碼自動生成controller,service,repository代碼

package com.qjc.config;

import com.baomidou.mybatisplus.core.exceptions.MybatisPlusException;
import com.baomidou.mybatisplus.core.toolkit.StringUtils;
import com.baomidou.mybatisplus.generator.AutoGenerator;
import com.baomidou.mybatisplus.generator.config.*;
import com.baomidou.mybatisplus.generator.config.rules.NamingStrategy;
import com.baomidou.mybatisplus.generator.engine.FreemarkerTemplateEngine;

import java.util.Scanner;

/**
 * @ClassName: GeneratorCodeConfig
 * @Description: 自動生成mybatisplus的相關代碼
 * @Author: qiaojiacheng
 * @Date: 2021/3/23 6:32 下午
 */
public class GeneratorCodeConfig {

    public static String scanner(String tip) {
        Scanner scanner = new Scanner(System.in);
        StringBuilder help = new StringBuilder();
        help.append("請輸入" + tip + ":");
        System.out.println(help.toString());
        if (scanner.hasNext()) {
            String ipt = scanner.next();
            if (!StringUtils.isBlank(ipt)) {
                return ipt;
            }
        }
        throw new MybatisPlusException("請輸入正確的" + tip + "!");
    }

    public static void main(String[] args) {
        // 代碼生成器
        AutoGenerator mpg = new AutoGenerator();

        // 全局配置
        GlobalConfig gc = new GlobalConfig();
        String projectPath = System.getProperty("user.dir");
        gc.setOutputDir(projectPath + "/src/main/java");
        gc.setAuthor("astupidcoder");
        gc.setOpen(false);
        //實體屬性 Swagger2 注解
        gc.setSwagger2(false);
        mpg.setGlobalConfig(gc);

        // 數據源配置
        DataSourceConfig dsc = new DataSourceConfig();
        dsc.setUrl("jdbc:mysql://127.0.0.1:3306/test?serverTimezone=UTC&useUnicode=true&characterEncoding=utf-8&zeroDateTimeBehavior=convertToNull&useSSL=false&allowPublicKeyRetrieval=true");
        dsc.setDriverName("com.mysql.jdbc.Driver");
        dsc.setUsername("root");
        dsc.setPassword("123456");
        mpg.setDataSource(dsc);

        // 包配置
        PackageConfig pc = new PackageConfig();
//        pc.setModuleName(scanner("模塊名"));
        pc.setParent("com.qjc");
        pc.setEntity("entity");
        pc.setMapper("mapper");
        pc.setService("service");
        pc.setServiceImpl("service.impl");
        mpg.setPackageInfo(pc);// 配置模板
        TemplateConfig templateConfig = new TemplateConfig();

        // 配置自定義輸出模板
        //指定自定義模板路徑,注意不要帶上.ftl/.vm, 會根據使用的模板引擎自動識別
        // templateConfig.setEntity("templates/entity2.java");
        // templateConfig.setService();
        // templateConfig.setController();

        templateConfig.setXml(null);
        mpg.setTemplate(templateConfig);

        // 策略配置
        StrategyConfig strategy = new StrategyConfig();
        strategy.setNaming(NamingStrategy.underline_to_camel);
        strategy.setColumnNaming(NamingStrategy.underline_to_camel);
        strategy.setSuperEntityClass("com.baomidou.mybatisplus.extension.activerecord.Model");
        strategy.setEntityLombokModel(true);
        strategy.setRestControllerStyle(true);

        strategy.setEntityLombokModel(true);
        // 公共父類
//        strategy.setSuperControllerClass("com.baomidou.ant.common.BaseController");
        // 寫於父類中的公共字段
//        strategy.setSuperEntityColumns("id");
        strategy.setInclude(scanner("表名,多個英文逗號分割").split(","));
        strategy.setControllerMappingHyphenStyle(true);
        strategy.setTablePrefix(pc.getModuleName() + "_");
        mpg.setStrategy(strategy);
        mpg.setTemplateEngine(new FreemarkerTemplateEngine());
        mpg.execute();
    }
}

最后就是最主要的配置了,ShardingSphere-JDBC 可以通過 JavaYAMLSpring 命名空間和 Spring Boot Starter 這 4 種方式進行配置,我們使用Spring Boot Starter,也就是配置文件properties進行配置

server.port=8081
server.servlet.context-path=/

spring.jackson.date-format=yyyy-MM-dd HH:mm:ss
spring.jackson.time-zone=GMT+8
spring.jackson.serialization.write-dates-as-timestamps=false

spring.main.allow-bean-definition-overriding=true
# 命名數據源  這個是自定義的
spring.shardingsphere.datasource.names=ds-0,ds-1,ds-2
# 配置數據源ds-0
spring.shardingsphere.datasource.ds-0.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.ds-0.driverClassName=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds-0.url=jdbc:mysql://127.0.0.1:3306/ds-0?serverTimezone=Asia/Shanghai&useUnicode=true&characterEncoding=utf-8&zeroDateTimeBehavior=convertToNull&useSSL=false&allowPublicKeyRetrieval=true
spring.shardingsphere.datasource.ds-0.username=root
spring.shardingsphere.datasource.ds-0.password=123456
# 配置數據源ds-1
spring.shardingsphere.datasource.ds-1.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.ds-1.driverClassName=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds-1.url=jdbc:mysql://127.0.0.1:3306/ds-1?serverTimezone=Asia/Shanghai&useUnicode=true&characterEncoding=utf-8&zeroDateTimeBehavior=convertToNull&useSSL=false&allowPublicKeyRetrieval=true
spring.shardingsphere.datasource.ds-1.username=root
spring.shardingsphere.datasource.ds-1.password=123456
# 配置數據源ds-2
spring.shardingsphere.datasource.ds-2.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.ds-2.driverClassName=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds-2.url=jdbc:mysql://127.0.0.1:3306/ds-user?serverTimezone=Asia/Shanghai&useUnicode=true&characterEncoding=utf-8&zeroDateTimeBehavior=convertToNull&useSSL=false&allowPublicKeyRetrieval=true
spring.shardingsphere.datasource.ds-2.username=root
spring.shardingsphere.datasource.ds-2.password=123456
# 配置默認數據源ds-0
spring.shardingsphere.sharding.default-data-source-name=ds-0

# 配置分片表 t_order
# 配置真實數據節點  庫:ds-0,ds-1;表:t_order_0,t_order_1,t_order_2
spring.shardingsphere.sharding.tables.t_order.actual-data-nodes=ds-$->{0..1}.t_order_$->{0..2}
# 分庫分片建
spring.shardingsphere.sharding.tables.t_order.database-strategy.inline.sharding-column=order_id
# 分庫分片算法: 對order_id字段進行取模分庫,2代表分片庫的個數
spring.shardingsphere.sharding.tables.t_order.database-strategy.inline.algorithm-expression=ds-$->{order_id % 2}
# 分表分片健
spring.shardingsphere.sharding.tables.t_order.table-strategy.inline.sharding-column=order_id
# 分表算法:對order_id字段進行取模分庫,3代表分片表的個數
spring.shardingsphere.sharding.tables.t_order.table-strategy.inline.algorithm-expression=t_order_$->{order_id % 3}
# 配置自增主鍵字段
spring.shardingsphere.sharding.tables.t_order.key-generator.column=order_id
# 自增主鍵ID 生成方案:雪花算法
spring.shardingsphere.sharding.tables.t_order.key-generator.type=SNOWFLAKE

# 配置分片表 t_order_item
spring.shardingsphere.sharding.tables.t_order_item.actual-data-nodes=ds-$->{0..1}.t_order_item_$->{0..2}
spring.shardingsphere.sharding.tables.t_order_item.database-strategy.inline.sharding-column=order_id
spring.shardingsphere.sharding.tables.t_order_item.database-strategy.inline.algorithm-expression=ds-$->{order_id % 2}
spring.shardingsphere.sharding.tables.t_order_item.table-strategy.inline.sharding-column=order_id
spring.shardingsphere.sharding.tables.t_order_item.table-strategy.inline.algorithm-expression=t_order_item_$->{order_id % 3}

#配置專庫專表
spring.shardingsphere.sharding.tables.t_user.actual-data-nodes=ds-2.t_user
spring.shardingsphere.sharding.tables.t_user.database-strategy.inline.sharding-column=id
spring.shardingsphere.sharding.tables.t_user.database-strategy.inline.algorithm-expression=ds-2
spring.shardingsphere.sharding.tables.t_user.table-strategy.inline.sharding-column=id
spring.shardingsphere.sharding.tables.t_user.table-strategy.inline.algorithm-expression=t_user
spring.shardingsphere.sharding.tables.t_user.key-generator.column=id
spring.shardingsphere.sharding.tables.t_user.key-generator.type=SNOWFLAKE

# 綁定表關系  不加[0]查詢時會導致笛卡爾積
spring.shardingsphere.sharding.binding-tables[0]=t_order,t_order_item
# 配置廣播表=公共表(PS:所有數據庫中都必須有這張表)
spring.shardingsphere.sharding.broadcast-tables=t_config
# 開啟SQL解析日志
spring.shardingsphere.props.sql.show=true

# 配置mybatis-plus
mybatis-plus.configuration.map-underscore-to-camel-case=true
mybatis-plus.configuration.auto-mapping-behavior=full
mybatis-plus.configuration.log-impl=org.apache.ibatis.logging.stdout.StdOutImpl
mybatis-plus.mapper-locations=classpath*:mapper/**/*Mapper.xml
mybatis-plus.global-config.db-config.logic-not-delete-value=0
mybatis-plus.global-config.db-config.logic-delete-value=1
mybatis-plus.global-config.db-config.logic-delete-field=deleted

進行測試

package com.qjc;

import com.alibaba.fastjson.JSON;
import com.qjc.entity.TConfig;
import com.qjc.entity.TOrder;
import com.qjc.entity.TOrderItem;
import com.qjc.entity.TUser;
import com.qjc.mapper.TConfigMapper;
import com.qjc.mapper.TOrderItemMapper;
import com.qjc.mapper.TOrderMapper;
import com.qjc.mapper.TUserMapper;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

import java.math.BigDecimal;
import java.util.List;

@RunWith(SpringRunner.class)
@SpringBootTest
public class ShardingSphereDemoTest {

    @Autowired
    private TOrderMapper orderMapper;
    @Autowired
    private TOrderItemMapper orderItemMapper;
    @Autowired
    private TUserMapper userMapper;
    @Autowired
    private TConfigMapper configMapper;

    private static final Integer num = 1000;
    private static final String orderNoPre = "NO-";

    /**
     * 測試分庫分表-插入
     */
    @Test
    public void testOrderSave() {
        for (int i = 1; i <= num; i++) {
            String orderNo = orderNoPre + String.format("%04d", i);
            TOrder order = new TOrder();
            order.setOrderNo(orderNo);
            order.setCreateName("訂單名稱" + i);
            order.setPrice(new BigDecimal("" + i));
            orderMapper.insert(order);

            TOrderItem orderItem = new TOrderItem();
            Long orderId = order.getOrderId();
            orderItem.setOrderId(orderId);
            orderItem.setOrderNo(orderNo);
            orderItem.setItemName("商品名稱" + i);
            orderItem.setItemDesc("商品描述" + i);
            orderItemMapper.insert(orderItem);
        }
    }

    /**
     * 測試分庫分表-查詢
     */
    @Test
    public void testOrderQuery() {
        List<TOrder> orderListByPage = orderMapper.findOrderListByPage();
        System.err.println(JSON.toJSONString(orderListByPage));
    }

    /**
     * 測試專庫專表
     */
    @Test
    public void test() {
//        TUser user = new TUser();
//        user.setName("用戶");
//        user.setAge(18);
//        user.setAddress("地址");
//        userMapper.insert(user);
        TUser user1 = userMapper.selectById(1376363351721992194L);
        System.err.println(user1);
    }

    /**
     * 測試公共表
     */
    @Test
    public void testConfig() {
        TConfig config = new TConfig();
        config.setRemark("bbbb");
        configMapper.insert(config);
    }

}

通過數據庫中的數據就可以查看是否成功,首先插入1000條數據,t_order和t_order_item的三張表加起來肯定是1000,而且由於t_order和t_order_item是綁定關系,所以他們后綴相同的表,里面的數據條數以及分庫分片鍵都是對應的

代碼地址:https://gitee.com/xiaorenwu_dashije/sharding-sphere-demo.git

 


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM