背景
項目中用到了 Shardingsphere4.0.1，用於對大數據量表進行分表、讀寫分離，對部分敏感數據進行數據脫敏。由於項目版本升級，把 Shardingsphere 從 4.0.1 升級為 5.0.0-alpha ，發現很多版本差異問題。本文主要對 springboot2.x 集成 Shardingsphere 5.0.0-alpha進行實踐
版本差異
Shardingsphere 5.0.0-alpha 主要配置變為
1. spring.shardingsphere.rules : 規則配置
2. spring.shardingsphere.rules.sharding.sharding-algorithms : 分庫分表規則配置
3. spring.shardingsphere.rules.sharding.tables : 分庫分表對象配置
4. spring.shardingsphere.rules.sharding.key-generators : 分布式序列配置
5. spring.shardingsphere.rules.replica-query : 讀寫分離配置
6. spring.shardingsphere.rules.encrypt : 數據脫敏配置
功能實戰
一、准備工作
1. 創建兩個數據庫 testdb1 、testdb2 分別作為主從庫(本次測試沒有主從環境)
2. 創建表 order(用於測試制作數據脫敏情況) 、order_0、 order_1 ， （本例子中表中的 sku 字段為分庫分表字段，理論上分庫有分庫的字段，分表有分表的字段 ； mobile 為數據脫敏字段）
二、分庫分表配置
#數據分片 #YamlShardingRuleSpringBootConfiguration #數據源 spring.shardingsphere.datasource.names=test1,test2 spring.shardingsphere.datasource.common.type=com.alibaba.druid.pool.DruidDataSource spring.shardingsphere.datasource.common.driver-class-name=com.mysql.jdbc.Driver spring.shardingsphere.datasource.test1.url=jdbc:mysql://localhost:3306/testdb1?useUnicode=true&characterEncoding=utf-8&useSSL=false spring.shardingsphere.datasource.test1.username=root spring.shardingsphere.datasource.test1.password= spring.shardingsphere.datasource.test2.url=jdbc:mysql://localhost:3306/testdb2?useUnicode=true&characterEncoding=utf-8&useSSL=false spring.shardingsphere.datasource.test2.username=root spring.shardingsphere.datasource.test2.password= #分片配置 spring.shardingsphere.rules.sharding.tables.order.actual-data-nodes=testdb$->{1..2}.order_$->{0..1} ##分布式序列配置（該項為必需，不是很友好） spring.shardingsphere.rules.sharding.key-generators.snowflake.type=SNOWFLAKE spring.shardingsphere.rules.sharding.key-generators.snowflake.props.max-vibration-offset=2048 spring.shardingsphere.rules.sharding.key-generators.snowflake.props.worker-id=1 ##分庫規則 spring.shardingsphere.rules.sharding.sharding-algorithms.dbstrategy.type=INLINE spring.shardingsphere.rules.sharding.sharding-algorithms.dbstrategy.props.algorithm-expression=testdb$->{ Long.parseLong(sku.replaceAll('[a-z]','').replaceAll('[A-Z]','')) % 2 + 1} ##分表規則 spring.shardingsphere.rules.sharding.sharding-algorithms.tbstrategy.type=INLINE spring.shardingsphere.rules.sharding.sharding-algorithms.tbstrategy.props.algorithm-expression=order_$->{ Long.parseLong(sku.replaceAll('[a-z]','').replaceAll('[A-Z]','')) % 2} ##庫配置 spring.shardingsphere.rules.sharding.tables.order.database-strategy.standard.sharding-column=sku spring.shardingsphere.rules.sharding.tables.order.database-strategy.standard.sharding-algorithm-name=dbstrategy ##分表配置 spring.shardingsphere.rules.sharding.tables.order.table-strategy.standard.sharding-column=sku spring.shardingsphere.rules.sharding.tables.order.table-strategy.standard.sharding-algorithm-name=tbstrategy ##分布式ID spring.shardingsphere.rules.sharding.tables.order.key-generate-strategy.column=order_no spring.shardingsphere.rules.sharding.tables.order.key-generate-strategy.key-generator-name=snowflake
三、讀寫分離配置
#讀寫分離 # YamlReplicaQueryRuleSpringBootConfiguration #數據源 spring.shardingsphere.datasource.names=master,slave spring.shardingsphere.datasource.common.type=com.alibaba.druid.pool.DruidDataSource spring.shardingsphere.datasource.common.driver-class-name=com.mysql.jdbc.Driver spring.shardingsphere.datasource.master.url=jdbc:mysql://localhost:3306/testdb1?useUnicode=true&characterEncoding=utf-8&useSSL=false spring.shardingsphere.datasource.master.username=root spring.shardingsphere.datasource.master.password= spring.shardingsphere.datasource.slave.url=jdbc:mysql://localhost:3306/testdb2?useUnicode=true&characterEncoding=utf-8&useSSL=false spring.shardingsphere.datasource.slave.username=root spring.shardingsphere.datasource.slave.password= #讀寫分離 spring.shardingsphere.rules.replica-query.data-sources.readwrite.name=readwrite spring.shardingsphere.rules.replica-query.data-sources.readwrite.primary-data-source-name=master spring.shardingsphere.rules.replica-query.data-sources.readwrite.replica-data-source-names=slave spring.shardingsphere.rules.replica-query.load-balancers.balancers.type=ROUND_ROBIN spring.shardingsphere.rules.replica-query.load-balancers.balancers.props.value=value #無用切必須要配置，不太友好
四、數據脫敏
#數據加密 #YamlEncryptRuleSpringBootConfiguration #數據源 spring.shardingsphere.datasource.names=testdb1 spring.shardingsphere.datasource.common.type=com.alibaba.druid.pool.DruidDataSource spring.shardingsphere.datasource.common.driver-class-name=com.mysql.jdbc.Driver spring.shardingsphere.datasource.test1.url=jdbc:mysql://localhost:3306/testdb1?useUnicode=true&characterEncoding=utf-8&useSSL=false spring.shardingsphere.datasource.test1.username=root spring.shardingsphere.datasource.test1.password= #數據加密 ##加密算法 spring.shardingsphere.rules.encrypt.encryptors.aesencrypt.type=AES spring.shardingsphere.rules.encrypt.encryptors.aesencrypt.props.aes-key-value=123456 spring.shardingsphere.rules.encrypt.tables.order.name=orderEncryptMobile spring.shardingsphere.rules.encrypt.tables.order.columns.mobile.cipher-column=mobile #spring.shardingsphere.rules.encrypt.tables.order.columns.mobile.logic-column=mobile #spring.shardingsphere.rules.encrypt.tables.order.columns.mobile.plain-column=mobile #spring.shardingsphere.rules.encrypt.tables.order.columns.mobile.assisted-query-column=mobile spring.shardingsphere.rules.encrypt.tables.order.columns.mobile.encryptor-name=aesencrypt
五、綜合實例
################### #數據源 spring.shardingsphere.datasource.names=test1,test2,slave1,slave2 spring.shardingsphere.datasource.common.type=com.alibaba.druid.pool.DruidDataSource spring.shardingsphere.datasource.common.driver-class-name=com.mysql.jdbc.Driver spring.shardingsphere.datasource.test1.url=jdbc:mysql://localhost:3306/testdb1?useUnicode=true&characterEncoding=utf-8&useSSL=false spring.shardingsphere.datasource.test1.username=root spring.shardingsphere.datasource.test1.password= spring.shardingsphere.datasource.test2.url=jdbc:mysql://localhost:3306/testdb2?useUnicode=true&characterEncoding=utf-8&useSSL=false spring.shardingsphere.datasource.test2.username=root spring.shardingsphere.datasource.test2.password= spring.shardingsphere.datasource.slave1.url=jdbc:mysql://localhost:3306/testdb1?useUnicode=true&characterEncoding=utf-8&useSSL=false spring.shardingsphere.datasource.slave1.username=root spring.shardingsphere.datasource.slave1.password= spring.shardingsphere.datasource.slave2.url=jdbc:mysql://localhost:3306/testdb2?useUnicode=true&characterEncoding=utf-8&useSSL=false spring.shardingsphere.datasource.slave2.username=root spring.shardingsphere.datasource.slave2.password= #分片配置 spring.shardingsphere.rules.sharding.tables.order.actual-data-nodes=testdb$->{1..2}.order_$->{0..1} ##分布式序列配置 spring.shardingsphere.rules.sharding.key-generators.snowflake.type=SNOWFLAKE spring.shardingsphere.rules.sharding.key-generators.snowflake.props.max-vibration-offset=0 spring.shardingsphere.rules.sharding.key-generators.snowflake.props.worker-id=2 ##分庫規則 spring.shardingsphere.rules.sharding.sharding-algorithms.dbstrategy.type=INLINE spring.shardingsphere.rules.sharding.sharding-algorithms.dbstrategy.props.algorithm-expression=testdb$->{ Long.parseLong(sku.replaceAll('[a-z]','').replaceAll('[A-Z]','')) % 2 + 1} ##分表規則 spring.shardingsphere.rules.sharding.sharding-algorithms.tbstrategy.type=INLINE spring.shardingsphere.rules.sharding.sharding-algorithms.tbstrategy.props.algorithm-expression=order_$->{ Long.parseLong(sku.replaceAll('[a-z]','').replaceAll('[A-Z]','')) % 2} ##庫配置 spring.shardingsphere.rules.sharding.tables.order.database-strategy.standard.sharding-column=sku spring.shardingsphere.rules.sharding.tables.order.database-strategy.standard.sharding-algorithm-name=dbstrategy ##分表配置 spring.shardingsphere.rules.sharding.tables.order.table-strategy.standard.sharding-column=sku spring.shardingsphere.rules.sharding.tables.order.table-strategy.standard.sharding-algorithm-name=tbstrategy ##分布式ID spring.shardingsphere.rules.sharding.tables.order.key-generate-strategy.column=order_no spring.shardingsphere.rules.sharding.tables.order.key-generate-strategy.key-generator-name=snowflake ## 默認分片配置 spring.shardingsphere.rules.sharding.default-database-strategy.standard.sharding-column=sku spring.shardingsphere.rules.sharding.default-database-strategy.standard.sharding-algorithm-name=dbstrategy #讀寫分離 spring.shardingsphere.rules.replica-query.data-sources.readwrite.name=readwrite spring.shardingsphere.rules.replica-query.data-sources.readwrite.primary-data-source-name=test1 spring.shardingsphere.rules.replica-query.data-sources.readwrite.replica-data-source-names=slave1 spring.shardingsphere.rules.replica-query.data-sources.readwrite1.name=readwrite1 spring.shardingsphere.rules.replica-query.data-sources.readwrite1.primary-data-source-name=test2 spring.shardingsphere.rules.replica-query.data-sources.readwrite1.replica-data-source-names=slave2 #負載均衡 spring.shardingsphere.rules.replica-query.load-balancers.balancers.type=ROUND_ROBIN spring.shardingsphere.rules.replica-query.load-balancers.balancers.props.value=value #數據加密 ##加密算法 spring.shardingsphere.rules.encrypt.encryptors.aesencrypt.type=AES spring.shardingsphere.rules.encrypt.encryptors.aesencrypt.props.haes-key-value=123456 spring.shardingsphere.rules.encrypt.tables.order.name=orderEncryptMobile spring.shardingsphere.rules.encrypt.tables.order.columns.mobile.cipher-column=mobile spring.shardingsphere.rules.encrypt.tables.order.columns.mobile.encryptor-name=aesencrypt
六、自定義分布式序列
resources 文件夾下增加 META-INF.services(SPI加載方式) org.apache.shardingsphere.sharding.spi.KeyGenerateAlgorithm
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
org.apache.shardingsphere.sharding.algorithm.keygen.SnowflakeKeyGenerateAlgorithm
org.apache.shardingsphere.sharding.algorithm.keygen.UUIDKeyGenerateAlgorithm
com.bruce.common.algorithm.MyKeyGenerateAlgorithm
MyKeyGenerateAlgorithm實現為：
import lombok.Getter; import lombok.Setter; import org.apache.shardingsphere.sharding.spi.KeyGenerateAlgorithm; import java.util.Properties; import java.util.UUID; @Getter @Setter public final class MyKeyGenerateAlgorithm implements KeyGenerateAlgorithm { private Properties props = new Properties(); @Override public void init() { } @Override public synchronized Comparable<?> generateKey() { return 自定義分布式ID; } @Override public String getType() { return "自定義分布式ID算法名稱"; } }
在配置中修改
spring.shardingsphere.rules.sharding.key-generators.snowflake.type=自定義分布式ID算法名稱
spring.shardingsphere.rules.sharding.key-generators.snowflake.props.自定義屬性=
七、自定義數據脫敏方式
resources 文件夾下增加 META-INF.services(SPI加載方式) org.apache.shardingsphere.encrypt.spi.EncryptAlgorithm
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
org.apache.shardingsphere.encrypt.algorithm.MD5EncryptAlgorithm
org.apache.shardingsphere.encrypt.algorithm.AESEncryptAlgorithm
org.apache.shardingsphere.encrypt.algorithm.RC4EncryptAlgorithm
com.bruce.common.encrypt.MyEncryptAlgorithm
MyEncryptAlgorithm實現為實現為：
import lombok.Getter;
import lombok.Setter;
import org.apache.commons.codec.binary.Base64;
import org.apache.commons.codec.binary.StringUtils;
import org.apache.shardingsphere.encrypt.spi.EncryptAlgorithm;
import java.nio.charset.StandardCharsets;
import java.util.Properties;
@Getter
@Setter
public final class MyEncryptAlgorithm implements EncryptAlgorithm {
private static final String HAES_KEY = "自定義屬性";
private Properties props = new Properties();
private byte[] secretKey;
@Override
public void init() {
secretKey = createSecretKey();
}
private byte[] createSecretKey() {
return HAES_KEY.getBytes();
}
@Override
public String encrypt(final Object plaintext) {
if (null == plaintext) {
return null;
}
return 自定義加密算法;
}
@Override
public Object decrypt(final String ciphertext) {
if (null == ciphertext) {
return null;
}
return 自定義解密算法;
}
@Override
public String getType() {
return "自定義脫敏算法名稱";
}
}
在配置中修改
spring.shardingsphere.rules.encrypt.encryptors.aesencrypt.type=自定義脫敏算法名稱
spring.shardingsphere.rules.encrypt.encryptors.aesencrypt.props.自定義屬性=