Protocol Buffer序列化Java框架-Protostuff

了解Protocol Buffer

首先要知道什么是Protocol Buffer，在編程過程中，當涉及數據交換時，我們往往需要將對象進行序列化然后再傳輸。常見的序列化的格式有JSON，XML等，這些格式雖然可讀性較好，但占用的空間大小並不是最優的。基於此，Google創建了一種名叫Protocol Buffer的序列化格式，它與JSON，XML相比可讀性較差，但占用的空間也會更小，在一些對於速度要求比較高的場景中較為常用。

Java序列化Protocol Buffer框架—ProtoStuff

Google對於Protocol Buffer提供了多種語言的實現方法：Java，C++，go和python。但我們在使用時仍然需要去編寫可讀性不高的.proto文件，然后使用Google提供的實現方法編譯成對應的語言，這就提高了我們使用Protocol Buffer的門檻。因此ProtoStuff就誕生了，通過ProtoStuff這個框架，我們能直接將對象通過Protocol Buffer這種序列化的方式轉成對應的字節，極大地降低了我們使用Protocol Buffer的使用成本。

實例

首先我們新建一個maven項目，然后添加ProtoStuff的依賴，其中Objenesis是一個用來實例化一個特定類的新對象的Java庫。通過該庫，我們能在不調用構造函數的情況下實例化一個類的對象。

<dependency>
	<groupId>com.dyuproject.protostuff</groupId>
    <artifactId>protostuff-core</artifactId>
    <version>${protostuff.version}</version>
</dependency>

<dependency>
	<groupId>com.dyuproject.protostuff</groupId>
    <artifactId>protostuff-runtime</artifactId>
    <version>${protostuff.version}</version>
</dependency>

<!-- Objenesis -->
<dependency>
	<groupId>org.objenesis</groupId>
    <artifactId>objenesis</artifactId>
    <version>${objenesis.version}</version>
</dependency>
<!-- Lombok -->
<dependency>
	<groupId>org.projectlombok</groupId>
    <artifactId>lombok</artifactId>
    <version>${lombok.version}</version>
</dependency>

然后我們創建兩個POJO來進行序列化的測試

@Data
@Builder
public Class Goods {
    
    private Integer num;
    private String name;
    private Double price;
    
}

@Data
@Builder
public Class Repository {
    
    private String name;
    private String location;
    private List<Goods> goodsList;
    
}

再之后編寫Protocol Buffer序列化的工具類

public Class SerializationUtil {
    
    private static Map<Class<?>, Schema<?>> cacheSchema = new ConcurrentHashMap();
    private static Objenesis objenesis = new ObjenesisStd(true);
    
    /**
    * 序列化（對象 -> 字節數組）
    *
    */
    public static <T> byte[] serialize(T obj) {
        Class<T> cls = (Class<T>) obj.getClass();
        LinkedBuffer buffer = LinkedBuffer.allocate(LinkedBuffer.DEFAULT_BUFFER_SIZE);
        try {
            Schema<T> schema = getSchema(cls);
            return ProtobufIOUtil.toByteArray(obj, schema, buffer);
        } catch (Exception e) {
            throw new IllegalStateException(e.getMessage(), e);
        } finally {
            buffer.clear();
        }
    }
    
    /**
    * 反序列化（字節數組 -> 對象）
    *
    */
    public static <T> T deserilize(byte[] data, Class<T> cls) {
        try {
            T message = objenesis.newInstance(cls);
            Schema<T> schema = getSchema(cls);
            ProtobufIOUtil.mergeFrom(data, message, schema);
            return message;
        } catch (Exception e) {
            throw new IllegalStateException(e.getMessage(), e);
        }
    }
    
    @SuppressWarnings("unchecked")
    private static <T> Schema<T> getSchema(Class<T> cls) {
        Schema<T> schema = (Schema<T>) cacheSchema.get(cls);
        if (schema == null) {
            schema = RuntimeSchema.createFrom(cls);
            cacheSchema.put(cls, schema);
        }
        return schema;
    }
}

最后編寫測試類來對序列化工具類進行測試

public Class Test {
    public static void main(String[] args) {
        Goods phone = Goods.builder().num(10).name("phone").price(1999.99).build();
        Goods water = Goods.builder().num(100).name("water").price(1.00).build();
        Repository repository = Repository.builder().name("Taobao").location("china").goodsList(Arrays.asList(phone, water)).build();
        byte[] data = SerializationUtil.serialize(repository);
        System.out.println("序列化結果：" + Arrays.toString(data));
        Repository result = SerializationUtil.deserilize(data, Repository.class);
        System.out.println("反序列化結果：" + result);
    }
}

輸出結果：

序列化結果：[10, 6, 84, 97, 111, 98, 97, 111, 18, 5, 99, 104, 105, 110, 97, 26, 18, 8, 10, 18, 5, 112, 104, 111, 110, 101, 25, 41, 92, -113, -62, -11, 63, -97, 64, 26, 18, 8, 100, 18, 5, 119, 97, 116, 101, 114, 25, 0, 0, 0, 0, 0, 0, -16, 63]
反序列化結果：Repository(name=Taobao, location=china, goodsList=[Goods(num=10, name=phone, price=1999.99), Goods(num=100, name=water, price=1.0)])

與JSON的對比

首先導入JSON處理的依賴，這里我們使用jackson來對JSON進行處理

<dependency>
	<groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>${jackson.version}</version>
</dependency>

之后修改測試類

public class Test {
    public static void main(String[] args) throws IOException {
        Goods phone = Goods.builder().num(10).name("phone").price(1999.99).build();
        Goods water = Goods.builder().num(100).name("water").price(1.00).build();
        Repository repository = Repository.builder().name("Taobao").location("china").goodsList(Arrays.asList(phone, water)).build();

        byte[] protobufData = SerializationUtil.serialize(repository);
        System.out.println("ProtoBuf序列化結果：" + Arrays.toString(protobufData));
        Repository protobufResult = SerializationUtil.deserilize(protobufData, Repository.class);
        System.out.println("ProtoBuf反序列化結果：" + protobufResult);

        ObjectMapper mapper = new ObjectMapper();
        byte[] jsonData = mapper.writeValueAsBytes(repository);
        System.out.println("JSON序列化結果：" + Arrays.toString(jsonData));
        Repository jsonResult = mapper.readValue(jsonData, Repository.class);
        System.out.println("JSON序列化結果：" + jsonResult);

        System.out.println();
        System.out.println("ProtoBuf序列化后字符串結果：" + new String(protobufData, StandardCharsets.UTF_8));
        System.out.println("JSON序列化后字符串結果：" + new String(jsonData, StandardCharsets.UTF_8));

        System.out.println();
        System.out.println("ProtoBuf序列化長度：" + protobufData.length);
        System.out.println("JSON序列化長度：" + jsonData.length);
    }
}

輸出結果：

ProtoBuf序列化結果：[10, 6, 84, 97, 111, 98, 97, 111, 18, 5, 99, 104, 105, 110, 97, 26, 18, 8, 10, 18, 5, 112, 104, 111, 110, 101, 25, 41, 92, -113, -62, -11, 63, -97, 64, 26, 18, 8, 100, 18, 5, 119, 97, 116, 101, 114, 25, 0, 0, 0, 0, 0, 0, -16, 63]
ProtoBuf反序列化結果：Repository(name=Taobao, location=china, goodsList=[Goods(num=10, name=phone, price=1999.99), Goods(num=100, name=water, price=1.0)])
JSON序列化結果：[123, 34, 110, 97, 109, 101, 34, 58, 34, 84, 97, 111, 98, 97, 111, 34, 44, 34, 108, 111, 99, 97, 116, 105, 111, 110, 34, 58, 34, 99, 104, 105, 110, 97, 34, 44, 34, 103, 111, 111, 100, 115, 76, 105, 115, 116, 34, 58, 91, 123, 34, 110, 117, 109, 34, 58, 49, 48, 44, 34, 110, 97, 109, 101, 34, 58, 34, 112, 104, 111, 110, 101, 34, 44, 34, 112, 114, 105, 99, 101, 34, 58, 49, 57, 57, 57, 46, 57, 57, 125, 44, 123, 34, 110, 117, 109, 34, 58, 49, 48, 48, 44, 34, 110, 97, 109, 101, 34, 58, 34, 119, 97, 116, 101, 114, 34, 44, 34, 112, 114, 105, 99, 101, 34, 58, 49, 46, 48, 125, 93, 125]
JSON序列化結果：Repository(name=Taobao, location=china, goodsList=[Goods(num=10, name=phone, price=1999.99), Goods(num=100, name=water, price=1.0)])

ProtoBuf序列化后字符串結果：
Taobaochina
phone)\���?�@dwater �?
JSON序列化后字符串結果：{"name":"Taobao","location":"china","goodsList":[{"num":10,"name":"phone","price":1999.99},{"num":100,"name":"water","price":1.0}]}

ProtoBuf序列化長度：55
JSON序列化長度：131

從結果來看在可讀性上顯然JSON更加易讀，ProtoBuf序列化后再轉為字符串甚至會亂碼，但在長度上則顯然ProtoBuf更占優勢，JSON的長度比ProtoBuf多了一倍多。

⚠️：在使用Jackson進行JSON反序列化時我們需要對我們的POJO類添加有參和無參構造，即添加@NoArgsConstructor 和@AllArgsConstructor 這兩個注解，否則會拋出如下異常：

Exception in thread "main" com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of com.xxx.xxx.Repository (no Creators, like default constructor, exist): cannot deserialize from Object value (no delegate- or property-based Creator)
at [Source: (byte[])"{"name":"Taobao","location":"china","goodsList":[{"num":10,"name":"phone","price":1999.99},{"num":100,"name":"water","price":1.0}]}"; line: 1, column: 2]
at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:67)
at com.fasterxml.jackson.databind.DeserializationContext.reportBadDefinition(DeserializationContext.java:1764)
at com.fasterxml.jackson.databind.DatabindContext.reportBadDefinition(DatabindContext.java:400)
at com.fasterxml.jackson.databind.DeserializationContext.handleMissingInstantiator(DeserializationContext.java:1209)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1400)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:362)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:195)
at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:322)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4593)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3609)
at com.silence.rpc.test.Test.main(Test.java:31)

原因是因為@Builder並不會添加無參構造，而Jackson的反序列化需要無參構造，因為在反序列化的時候，會先初始化對象，此時默認調用的是無參函數，然后再進行賦值，故此我們需要添加@NoArgsConstructor ，如果只添加這個注解，又會導致缺少有參構造，因此我們還需要添加@AllArgsConstructor 。