flatbuffer介紹和用法

介紹

flatbuffer是google發布的一個跨平台序列化框架具有如下特點

1、對序列化的數據不需要打包和拆包

2、內存和效率速度高，擴展靈活

3、代碼依賴較少

4、強類型設計，編譯期即可完成類型檢查

5、使用簡單、可跨平台使用

安裝

git clone [e-mail]:google/flatbuffers.git
cd flatbuffers
brew install cmake
cmake -G "Unix Makefiles"
make
make install
flatc --version

編寫flatbuffer文件

// Example IDL file for our monster's schema.
namespace com.frank.learning;
enum Color:byte { Red = 0, Green, Blue = 2 } 
union Equipment { Weapon } // Optionally add more tables.
struct Vec3 {
  x:float;
  y:float;
  z:float;
}
table Monster {
  pos:Vec3; // Struct.
  mana:short = 150;
  hp:short = 100;
  name:string;
  friendly:bool = false (deprecated);
  inventory:[ubyte];  // Vector of scalars.
  color:Color = Blue; // Enum.
  weapons:[Weapon];   // Vector of tables.
  equipped:Equipment; // Union.
}
table Weapon {
  name:string;
  damage:short;
}
root_type Monster;

將文件保存為monster.fbs，下面進行編譯

flatc --java monster.fbs

執行完后會在當前目錄下生成Java文件

IntelliJ測試flatbuffer

將生成的Java代碼拷到項目中，新建SampleBinary類

package com.frank.learning;

import com.google.flatbuffers.FlatBufferBuilder;

import java.nio.ByteBuffer;

public class SampleBinary {

    public static void  main(String[] args){

        //使用FlatBufferBuilder 完成對象序列化
        FlatBufferBuilder builder = new FlatBufferBuilder(1024);

        //返回該String的偏移地址
        int weaponOneName = builder.createString("Sword");
        short weaponOneDamage = 3;
        int weaponTwoName = builder.createString("Axe");
        short weaponTwoDamage = 5;

        // 使用createWeapon創建Weapon對象，並返回該對象的偏移地址
        int sword = Weapon.createWeapon(builder, weaponOneName, weaponOneDamage);
        int axe = Weapon.createWeapon(builder, weaponTwoName, weaponTwoDamage);

        // Serialize a name for our monster, called "Orc".
        int name = builder.createString("Orc");

        // 創建一個Vector對象，並且返回它的偏移地址
        byte[] treasure = {0, 1, 13, 12, 4, 5, 6, 7, 8, 9};
        int inv = Monster.createInventoryVector(builder, treasure);

        // Place the two weapons into an array, and pass it to the `createWeaponsVector()` method to
        // create a FlatBuffer vector.
        int[] weaps = new int[2];
        weaps[0] = sword;
        weaps[1] = axe;
        // Pass the `weaps` array into the `createWeaponsVector()` method to create a FlatBuffer vector.
        int weapons = Monster.createWeaponsVector(builder, weaps);

        // startMonster聲明開始創建Monster對象，使用endMonster聲明完成Monster對象
        Monster.startMonster(builder);
        Monster.addPos(builder, Vec3.createVec3(builder, 1.0f, 2.0f, 3.0f));
        Monster.addName(builder, name);
        Monster.addColor(builder, Color.Red);
        Monster.addHp(builder, (short)300);
        Monster.addInventory(builder, inv);
        Monster.addWeapons(builder, weapons);
        Monster.addEquippedType(builder, Equipment.Weapon);
        Monster.addEquipped(builder, axe);
        int orc = Monster.endMonster(builder);

        // 調用finish方法完成Monster對象
        builder.finish(orc); // You could also call `Monster.finishMonsterBuffer(builder, orc);`.

        // 生成二進制文件
        byte[] buf = builder.sizedByteArray();

        // 至此完成對象數據序列化


        //模擬從獲取到二進制數據 進行反序列化對象
        ByteBuffer buffer = ByteBuffer.wrap(buf);

        //根據該二進制數據列生成Monster對象
        Monster monster = Monster.getRootAsMonster(buffer);

        short hp = monster.hp();
        System.out.println(hp);

        short mana = monster.mana();
        System.out.println(mana);
        String resultName = monster.name();
        System.out.println(resultName);

        Vec3 pos = monster.pos();
        float x = pos.x();
        float y = pos.y();
        float z = pos.z();
        System.out.println("X: "+x+"  Y: "+y+"  Z: "+z);

        int invLength = monster.inventoryLength();
        int thirdItem = monster.inventory(2);
        System.out.println(thirdItem);

        int weaponsLength = monster.weaponsLength();
        String secondWeaponName = monster.weapons(1).name();
        short secondWeaponDamage = monster.weapons(1).damage();
        System.out.println("weaponsLength: "+weaponsLength+"  secondWeaponName: "+secondWeaponName+"  secondWeaponDamage: "+secondWeaponDamage);
        int unionType = monster.equippedType();
        if (unionType == Equipment.Weapon) {
            Weapon weapon = (Weapon)monster.equipped(new Weapon()); // Requires explicit cast
            // to `Weapon`.
            String weaponName = weapon.name();    // "Axe"
            short weaponDamage = weapon.damage(); // 5
            System.out.println("weaponName: "+weaponName+"  weaponDamage: "+weaponDamage);
        }
    }
}

pom文件加入flatbuffer相關jar包

<dependency>
    <groupId>com.google.flatbuffers</groupId>
    <artifactId>flatbuffers-java</artifactId>
    <version>1.12.0</version>
</dependency>

輸出結果如下

300
150
Orc
X: 1.0  Y: 2.0  Z: 3.0
13
weaponsLength: 2  secondWeaponName: Axe  secondWeaponDamage: 5
weaponName: Axe  weaponDamage: 5

flatbuffer原理

flatbuffer將數據存在一個一維數組當中，緩存在一個bytebuffer當中。一個flatbuffer對象在數組中被分為兩部分，元數據部分和數據部分。元數據負責存放相對於中間部分的索引，數據部分存放真實的value。分割的節點為（pivot point）。它的將數據以及對應的數據位置都保存在一個線性的數組中。使用的時候只需要把byte流發送出去，解析的時候只需要根據保存的位置，截取對應的數值即可。

例子：
假設我們創建了一個Person對象，它的name是John，friendshipStatus是2.那么對應圖中元數據部分第一個byte是1--->從中心點處數一個位置，開始的字符就是name的值即john。第二個byte是6，從中心點數6個位置值是2.

class Person {
    String name;//john
    int friendshipStatus;//2
    Person spouse;
    List<Person>friends;
}

 

 從圖中可以看出來，flatbuffer將索引和數據文件存在一個一位數組中通過查找，還原對象，所以不需要打包和拆包的過程相對高效。

參考學習鏈接

1、https://www.jianshu.com/p/8df23cd182ec

2、https://www.jianshu.com/p/fa999434776a

3、https://google.github.io/flatbuffers/flatbuffers_guide_tutorial.html