背景
项目中用到了 Shardingsphere4.0.1，用于对大数据量表进行分表、读写分离，对部分敏感数据进行数据脱敏。由于项目版本升级，把 Shardingsphere 从 4.0.1 升级为 5.0.0-alpha ，发现很多版本差异问题。本文主要对 springboot2.x 集成 Shardingsphere 5.0.0-alpha进行实践
版本差异
Shardingsphere 5.0.0-alpha 主要配置变为
1. spring.shardingsphere.rules : 规则配置
2. spring.shardingsphere.rules.sharding.sharding-algorithms : 分库分表规则配置
3. spring.shardingsphere.rules.sharding.tables : 分库分表对象配置
4. spring.shardingsphere.rules.sharding.key-generators : 分布式序列配置
5. spring.shardingsphere.rules.replica-query : 读写分离配置
6. spring.shardingsphere.rules.encrypt : 数据脱敏配置
功能实战
一、准备工作
1. 创建两个数据库 testdb1 、testdb2 分别作为主从库(本次测试没有主从环境)
2. 创建表 order(用于测试制作数据脱敏情况) 、order_0、 order_1 ， （本例子中表中的 sku 字段为分库分表字段，理论上分库有分库的字段，分表有分表的字段 ； mobile 为数据脱敏字段）
二、分库分表配置
#数据分片 #YamlShardingRuleSpringBootConfiguration #数据源 spring.shardingsphere.datasource.names=test1,test2 spring.shardingsphere.datasource.common.type=com.alibaba.druid.pool.DruidDataSource spring.shardingsphere.datasource.common.driver-class-name=com.mysql.jdbc.Driver spring.shardingsphere.datasource.test1.url=jdbc:mysql://localhost:3306/testdb1?useUnicode=true&characterEncoding=utf-8&useSSL=false spring.shardingsphere.datasource.test1.username=root spring.shardingsphere.datasource.test1.password= spring.shardingsphere.datasource.test2.url=jdbc:mysql://localhost:3306/testdb2?useUnicode=true&characterEncoding=utf-8&useSSL=false spring.shardingsphere.datasource.test2.username=root spring.shardingsphere.datasource.test2.password= #分片配置 spring.shardingsphere.rules.sharding.tables.order.actual-data-nodes=testdb$->{1..2}.order_$->{0..1} ##分布式序列配置（该项为必需，不是很友好） spring.shardingsphere.rules.sharding.key-generators.snowflake.type=SNOWFLAKE spring.shardingsphere.rules.sharding.key-generators.snowflake.props.max-vibration-offset=2048 spring.shardingsphere.rules.sharding.key-generators.snowflake.props.worker-id=1 ##分库规则 spring.shardingsphere.rules.sharding.sharding-algorithms.dbstrategy.type=INLINE spring.shardingsphere.rules.sharding.sharding-algorithms.dbstrategy.props.algorithm-expression=testdb$->{ Long.parseLong(sku.replaceAll('[a-z]','').replaceAll('[A-Z]','')) % 2 + 1} ##分表规则 spring.shardingsphere.rules.sharding.sharding-algorithms.tbstrategy.type=INLINE spring.shardingsphere.rules.sharding.sharding-algorithms.tbstrategy.props.algorithm-expression=order_$->{ Long.parseLong(sku.replaceAll('[a-z]','').replaceAll('[A-Z]','')) % 2} ##库配置 spring.shardingsphere.rules.sharding.tables.order.database-strategy.standard.sharding-column=sku spring.shardingsphere.rules.sharding.tables.order.database-strategy.standard.sharding-algorithm-name=dbstrategy ##分表配置 spring.shardingsphere.rules.sharding.tables.order.table-strategy.standard.sharding-column=sku spring.shardingsphere.rules.sharding.tables.order.table-strategy.standard.sharding-algorithm-name=tbstrategy ##分布式ID spring.shardingsphere.rules.sharding.tables.order.key-generate-strategy.column=order_no spring.shardingsphere.rules.sharding.tables.order.key-generate-strategy.key-generator-name=snowflake
三、读写分离配置
#读写分离 # YamlReplicaQueryRuleSpringBootConfiguration #数据源 spring.shardingsphere.datasource.names=master,slave spring.shardingsphere.datasource.common.type=com.alibaba.druid.pool.DruidDataSource spring.shardingsphere.datasource.common.driver-class-name=com.mysql.jdbc.Driver spring.shardingsphere.datasource.master.url=jdbc:mysql://localhost:3306/testdb1?useUnicode=true&characterEncoding=utf-8&useSSL=false spring.shardingsphere.datasource.master.username=root spring.shardingsphere.datasource.master.password= spring.shardingsphere.datasource.slave.url=jdbc:mysql://localhost:3306/testdb2?useUnicode=true&characterEncoding=utf-8&useSSL=false spring.shardingsphere.datasource.slave.username=root spring.shardingsphere.datasource.slave.password= #读写分离 spring.shardingsphere.rules.replica-query.data-sources.readwrite.name=readwrite spring.shardingsphere.rules.replica-query.data-sources.readwrite.primary-data-source-name=master spring.shardingsphere.rules.replica-query.data-sources.readwrite.replica-data-source-names=slave spring.shardingsphere.rules.replica-query.load-balancers.balancers.type=ROUND_ROBIN spring.shardingsphere.rules.replica-query.load-balancers.balancers.props.value=value #无用切必须要配置，不太友好
四、数据脱敏
#数据加密 #YamlEncryptRuleSpringBootConfiguration #数据源 spring.shardingsphere.datasource.names=testdb1 spring.shardingsphere.datasource.common.type=com.alibaba.druid.pool.DruidDataSource spring.shardingsphere.datasource.common.driver-class-name=com.mysql.jdbc.Driver spring.shardingsphere.datasource.test1.url=jdbc:mysql://localhost:3306/testdb1?useUnicode=true&characterEncoding=utf-8&useSSL=false spring.shardingsphere.datasource.test1.username=root spring.shardingsphere.datasource.test1.password= #数据加密 ##加密算法 spring.shardingsphere.rules.encrypt.encryptors.aesencrypt.type=AES spring.shardingsphere.rules.encrypt.encryptors.aesencrypt.props.aes-key-value=123456 spring.shardingsphere.rules.encrypt.tables.order.name=orderEncryptMobile spring.shardingsphere.rules.encrypt.tables.order.columns.mobile.cipher-column=mobile #spring.shardingsphere.rules.encrypt.tables.order.columns.mobile.logic-column=mobile #spring.shardingsphere.rules.encrypt.tables.order.columns.mobile.plain-column=mobile #spring.shardingsphere.rules.encrypt.tables.order.columns.mobile.assisted-query-column=mobile spring.shardingsphere.rules.encrypt.tables.order.columns.mobile.encryptor-name=aesencrypt
五、综合实例
################### #数据源 spring.shardingsphere.datasource.names=test1,test2,slave1,slave2 spring.shardingsphere.datasource.common.type=com.alibaba.druid.pool.DruidDataSource spring.shardingsphere.datasource.common.driver-class-name=com.mysql.jdbc.Driver spring.shardingsphere.datasource.test1.url=jdbc:mysql://localhost:3306/testdb1?useUnicode=true&characterEncoding=utf-8&useSSL=false spring.shardingsphere.datasource.test1.username=root spring.shardingsphere.datasource.test1.password= spring.shardingsphere.datasource.test2.url=jdbc:mysql://localhost:3306/testdb2?useUnicode=true&characterEncoding=utf-8&useSSL=false spring.shardingsphere.datasource.test2.username=root spring.shardingsphere.datasource.test2.password= spring.shardingsphere.datasource.slave1.url=jdbc:mysql://localhost:3306/testdb1?useUnicode=true&characterEncoding=utf-8&useSSL=false spring.shardingsphere.datasource.slave1.username=root spring.shardingsphere.datasource.slave1.password= spring.shardingsphere.datasource.slave2.url=jdbc:mysql://localhost:3306/testdb2?useUnicode=true&characterEncoding=utf-8&useSSL=false spring.shardingsphere.datasource.slave2.username=root spring.shardingsphere.datasource.slave2.password= #分片配置 spring.shardingsphere.rules.sharding.tables.order.actual-data-nodes=testdb$->{1..2}.order_$->{0..1} ##分布式序列配置 spring.shardingsphere.rules.sharding.key-generators.snowflake.type=SNOWFLAKE spring.shardingsphere.rules.sharding.key-generators.snowflake.props.max-vibration-offset=0 spring.shardingsphere.rules.sharding.key-generators.snowflake.props.worker-id=2 ##分库规则 spring.shardingsphere.rules.sharding.sharding-algorithms.dbstrategy.type=INLINE spring.shardingsphere.rules.sharding.sharding-algorithms.dbstrategy.props.algorithm-expression=testdb$->{ Long.parseLong(sku.replaceAll('[a-z]','').replaceAll('[A-Z]','')) % 2 + 1} ##分表规则 spring.shardingsphere.rules.sharding.sharding-algorithms.tbstrategy.type=INLINE spring.shardingsphere.rules.sharding.sharding-algorithms.tbstrategy.props.algorithm-expression=order_$->{ Long.parseLong(sku.replaceAll('[a-z]','').replaceAll('[A-Z]','')) % 2} ##库配置 spring.shardingsphere.rules.sharding.tables.order.database-strategy.standard.sharding-column=sku spring.shardingsphere.rules.sharding.tables.order.database-strategy.standard.sharding-algorithm-name=dbstrategy ##分表配置 spring.shardingsphere.rules.sharding.tables.order.table-strategy.standard.sharding-column=sku spring.shardingsphere.rules.sharding.tables.order.table-strategy.standard.sharding-algorithm-name=tbstrategy ##分布式ID spring.shardingsphere.rules.sharding.tables.order.key-generate-strategy.column=order_no spring.shardingsphere.rules.sharding.tables.order.key-generate-strategy.key-generator-name=snowflake ## 默认分片配置 spring.shardingsphere.rules.sharding.default-database-strategy.standard.sharding-column=sku spring.shardingsphere.rules.sharding.default-database-strategy.standard.sharding-algorithm-name=dbstrategy #读写分离 spring.shardingsphere.rules.replica-query.data-sources.readwrite.name=readwrite spring.shardingsphere.rules.replica-query.data-sources.readwrite.primary-data-source-name=test1 spring.shardingsphere.rules.replica-query.data-sources.readwrite.replica-data-source-names=slave1 spring.shardingsphere.rules.replica-query.data-sources.readwrite1.name=readwrite1 spring.shardingsphere.rules.replica-query.data-sources.readwrite1.primary-data-source-name=test2 spring.shardingsphere.rules.replica-query.data-sources.readwrite1.replica-data-source-names=slave2 #负载均衡 spring.shardingsphere.rules.replica-query.load-balancers.balancers.type=ROUND_ROBIN spring.shardingsphere.rules.replica-query.load-balancers.balancers.props.value=value #数据加密 ##加密算法 spring.shardingsphere.rules.encrypt.encryptors.aesencrypt.type=AES spring.shardingsphere.rules.encrypt.encryptors.aesencrypt.props.haes-key-value=123456 spring.shardingsphere.rules.encrypt.tables.order.name=orderEncryptMobile spring.shardingsphere.rules.encrypt.tables.order.columns.mobile.cipher-column=mobile spring.shardingsphere.rules.encrypt.tables.order.columns.mobile.encryptor-name=aesencrypt
六、自定义分布式序列
resources 文件夹下增加 META-INF.services(SPI加载方式) org.apache.shardingsphere.sharding.spi.KeyGenerateAlgorithm
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
org.apache.shardingsphere.sharding.algorithm.keygen.SnowflakeKeyGenerateAlgorithm
org.apache.shardingsphere.sharding.algorithm.keygen.UUIDKeyGenerateAlgorithm
com.bruce.common.algorithm.MyKeyGenerateAlgorithm
MyKeyGenerateAlgorithm实现为：
import lombok.Getter; import lombok.Setter; import org.apache.shardingsphere.sharding.spi.KeyGenerateAlgorithm; import java.util.Properties; import java.util.UUID; @Getter @Setter public final class MyKeyGenerateAlgorithm implements KeyGenerateAlgorithm { private Properties props = new Properties(); @Override public void init() { } @Override public synchronized Comparable<?> generateKey() { return 自定义分布式ID; } @Override public String getType() { return "自定义分布式ID算法名称"; } }
在配置中修改
spring.shardingsphere.rules.sharding.key-generators.snowflake.type=自定义分布式ID算法名称
spring.shardingsphere.rules.sharding.key-generators.snowflake.props.自定义属性=
七、自定义数据脱敏方式
resources 文件夹下增加 META-INF.services(SPI加载方式) org.apache.shardingsphere.encrypt.spi.EncryptAlgorithm
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
org.apache.shardingsphere.encrypt.algorithm.MD5EncryptAlgorithm
org.apache.shardingsphere.encrypt.algorithm.AESEncryptAlgorithm
org.apache.shardingsphere.encrypt.algorithm.RC4EncryptAlgorithm
com.bruce.common.encrypt.MyEncryptAlgorithm
MyEncryptAlgorithm实现为实现为：
import lombok.Getter;
import lombok.Setter;
import org.apache.commons.codec.binary.Base64;
import org.apache.commons.codec.binary.StringUtils;
import org.apache.shardingsphere.encrypt.spi.EncryptAlgorithm;
import java.nio.charset.StandardCharsets;
import java.util.Properties;
@Getter
@Setter
public final class MyEncryptAlgorithm implements EncryptAlgorithm {
private static final String HAES_KEY = "自定义属性";
private Properties props = new Properties();
private byte[] secretKey;
@Override
public void init() {
secretKey = createSecretKey();
}
private byte[] createSecretKey() {
return HAES_KEY.getBytes();
}
@Override
public String encrypt(final Object plaintext) {
if (null == plaintext) {
return null;
}
return 自定义加密算法;
}
@Override
public Object decrypt(final String ciphertext) {
if (null == ciphertext) {
return null;
}
return 自定义解密算法;
}
@Override
public String getType() {
return "自定义脱敏算法名称";
}
}
在配置中修改
spring.shardingsphere.rules.encrypt.encryptors.aesencrypt.type=自定义脱敏算法名称
spring.shardingsphere.rules.encrypt.encryptors.aesencrypt.props.自定义属性=