thrift的全名叫做Apache thrift，是一款軟件開發RPC框架，可以很高效地實現跨語言的RPC服務。
本文簡要介紹了thrift的背景、相關概念以及安裝流程。並給出了C++以及python版本的入門例子。其中背景概念部分翻譯自[1]。
1 Krzysztof Rakowski的Apache Thrift介紹
1.1 部分歷史與背景
Thrift由facebook在2007年提出。facebook的公司文化就是不限制你選擇任何開發語言，用最佳方案解決實際問題。毫無疑問，公司內部許多應用程序都是用不同語言編寫的。這也是為什么facebook需要一種工具，使這些應用程序能夠相互通信。然后他們花了一些精力尋找這樣的工具，可惜並沒找到合適的。facebook發布的白皮書也提到了他們曾經考察的工具，以及Thrift框架的設計思路。
當然啦，牛逼如facebook的程序員克服了一些挑戰，自己開發了該需求的解決方案——Thrift就是這樣誕生的。不久之后，他們開源了Thrift，並且托管到Apache基金會，后者開始負責Thrift的開發。現在Thrift不僅僅被用在facebook（公司內部應用間通信的主要工具），而且被其他許多公司使用。（包括Evernote、Twitter和Netflix等知名公司）。facebook的工程師仍然會在fork版本FBThrift上繼續開發。並且有希望與Apache Thrift整合。
1.2 Apache Thrift究竟是什么
我們想象一下這種情形：你有許多用不同語言開發的應用程序，比如一種很常見的情況就是公司內部獨立的開發小組各自開發了一些應用程序，用來執行一些不同的內部任務。那怎么讓這些應用程序相互通信呢？當然啦，你可以添加一些REST API。但是在許多情況下——尤其是你要傳輸二進制數據——這種方案並不能滿足性能和可靠性要求。
1.3 Thrift如何工作
首先，讓我們從Apache Thrift開發者的角度來看看。
Thrift框架的主要概念是服務，服務和面向對象編程語言中的類很相似。每個服務中都包含方法，也是OOP中的類似概念。Thrift還實現了需要數據類型。這些數據類型在每種編程語言中都有具體對應的類型，最簡單的例子比如Thrift中的int，在所有語言中都會映射成整型。但是復雜點的比如set，在php中會映射成array，在java中會映射成HashSet。編寫服務的文本文件也被稱為Thrift文檔（后綴是.thrift），包括服務在內，文檔中的所有代碼都是接口定義語言(Interface Description Language ,IDL)，如果要了解詳細語法，請參考官方文檔[2]。（注：其實IDL語法很簡單，源碼中有一份自解釋的Thrift文件：tutorial.thrift）
# Thrift Tutorial
# Mark Slee (mcslee@facebook.com)
#
# This file aims to teach you how to use Thrift, in a .thrift file. Neato. The
# first thing to notice is that .thrift files support standard shell comments.
# This lets you make your thrift file executable and include your Thrift build
# step on the top line. And you can place comments like this anywhere you like.
#
# Before running this file, you will need to have installed the thrift compiler
# into /usr/local/bin.
/**
* The first thing to know about are types. The available types in Thrift are:
*
* bool Boolean, one byte
* i8 (byte) Signed 8-bit integer
* i16 Signed 16-bit integer
* i32 Signed 32-bit integer
* i64 Signed 64-bit integer
* double 64-bit floating point value
* string String
* binary Blob (byte array)
* map<t1,t2> Map from one type to another
* list<t1> Ordered list of one type
* set<t1> Set of unique elements of one type
*
* Did you also notice that Thrift supports C style comments?
*/
// Just in case you were wondering... yes. We support simple C comments too.
/**
* Thrift files can reference other Thrift files to include common struct
* and service definitions. These are found using the current path, or by
* searching relative to any paths specified with the -I compiler flag.
*
* Included objects are accessed using the name of the .thrift file as a
* prefix. i.e. shared.SharedObject
*/
include "shared.thrift"
/**
* Thrift files can namespace, package, or prefix their output in various
* target languages.
*/
namespace cl tutorial
namespace cpp tutorial
namespace d tutorial
namespace dart tutorial
namespace java tutorial
namespace php tutorial
namespace perl tutorial
namespace haxe tutorial
namespace netstd tutorial
/**
* Thrift lets you do typedefs to get pretty names for your types. Standard
* C style here.
*/
typedef i32 MyInteger
/**
* Thrift also lets you define constants for use across languages. Complex
* types and structs are specified using JSON notation.
*/
const i32 INT32CONSTANT = 9853
const map<string,string> MAPCONSTANT = {'hello':'world', 'goodnight':'moon'}
/**
* You can define enums, which are just 32 bit integers. Values are optional
* and start at 1 if not supplied, C style again.
*/
enum Operation {
ADD = 1,
SUBTRACT = 2,
MULTIPLY = 3,
DIVIDE = 4
}
/**
* Structs are the basic complex data structures. They are comprised of fields
* which each have an integer identifier, a type, a symbolic name, and an
* optional default value.
*
* Fields can be declared "optional", which ensures they will not be included
* in the serialized output if they aren't set. Note that this requires some
* manual management in some languages.
*/
struct Work {
1: i32 num1 = 0,
2: i32 num2,
3: Operation op,
4: optional string comment,
}
/**
* Structs can also be exceptions, if they are nasty.
*/
exception InvalidOperation {
1: i32 whatOp,
2: string why
}
/**
* Ahh, now onto the cool part, defining a service. Services just need a name
* and can optionally inherit from another service using the extends keyword.
*/
service Calculator extends shared.SharedService {
/**
* A method definition looks like C code. It has a return type, arguments,
* and optionally a list of exceptions that it may throw. Note that argument
* lists and exception lists are specified using the exact same syntax as
* field lists in struct or exception definitions.
*/
void ping(),
i32 add(1:i32 num1, 2:i32 num2),
i32 calculate(1:i32 logid, 2:Work w) throws (1:InvalidOperation ouch),
/**
* This method has a oneway modifier. That means the client only makes
* a request and does not listen for any response at all. Oneway methods
* must be void.
*/
oneway void zip()
}
/**
* That just about covers the basics. Take a look in the test/ folder for more
* detailed examples. After you run this file, your generated code shows up
* in folders with names gen-<language>. The generated code isn't too scary
* to look at. It even has pretty indentation.
*/
接着，從這個Thrift文件——通過Apache Thrift編譯器——你可以生成server和client的stub。這些自動生成的代碼被稱為Apache Thrift庫，然后你借助庫，按照你的需求，實現指定語言的server和client——過程很像代碼填空，填好自己相關部分即可。（比如對象創建、方法調用等語句），這樣就實現了不同應用程序相互通信。你生成的server和client代碼嵌入到你的應用程序中。過程如下所示：
看實例之前，讓我們大致看下Apache Thrift的架構。如下圖所示：
傳輸層的功能是從你使用的介質（一般是socket）中讀寫負載。協議層一般和傳輸層解耦，負責數據的編碼和解析，方便數據傳輸。常用協議包括：binary，compact（Thrift獨有）以及JSON。至於處理器層，這里說的很模糊，主要是把用戶指定的輸入輸出協議作為參數，然后從輸入讀取數據，按照用戶規則進行處理，最后向輸出寫入數據[3]，如下圖所示：
在server和client代碼中，這三層組合在一起。如果你想兩個應用程序相互通信，那么server和client要使用相同的傳輸層和協議層集合，這樣才能正確編碼和解析數據。第三節有C++以及python版本的入門例子。
2 安裝
1、安裝依賴項，官方給出了不同系統下的依賴項安裝步驟。
2、編譯源代碼，安裝
# github mirror: https://github.com/apache/thrift
git clone https://git-wip-us.apache.org/repos/asf/thrift.git
cd thrift
./bootstrap.sh
# 禁用lua語言，查看更多參數：./configure --help
./configure --with-lua=no
# 編譯
make
# 安裝
sudo make install
configure這一步結束會在terminal輸出對不同語言的支持信息，檢查一下是否滿足自己要求。
Building Plugin Support ...... : yes
Building C++ Library ......... : yes
Building C (GLib) Library .... : no
Building Java Library ........ : no
Building C# Library .......... : no
Building .NET Core Library ... : no
Building Python Library ...... : yes
Building Ruby Library ........ : no
Building Haxe Library ........ : no
Building Haskell Library ..... : no
Building Perl Library ........ : no
Building PHP Library ......... : yes
Building Dart Library ........ : no
Building Erlang Library ...... : no
Building Go Library .......... : no
Building D Library ........... : no
Building NodeJS Library ...... : no
Building Lua Library ......... : no
C++ Library:
Build TZlibTransport ...... : yes
Build TNonblockingServer .. : yes
Build TQTcpServer (Qt4) .... : no
Build TQTcpServer (Qt5) .... : no
Python Library:
Using Python .............. : /usr/bin/python
Using Python3 ............. : /usr/local/bin/python3
如果make這一步出現缺少.a文件的問題：
g++: error: /usr/lib64/libboost_unit_test_framework.a: No such file or directory
首先檢查上一步的依賴項有沒有全部安裝，沒問題的話可以看看/usr/local/lib/下有沒有該文件，再拷貝到make過程中所尋找的路徑下。
$ ls /usr/local/lib/libboost_unit_test_framework.*
/usr/local/lib/libboost_unit_test_framework.a
/usr/local/lib/libboost_unit_test_framework.so
/usr/local/lib/libboost_unit_test_framework.so.1.53.0
$ ls /usr/lib64/libboost_unit_test_framework*
/usr/lib64/libboost_unit_test_framework-mt.so /usr/lib64/libboost_unit_test_framework.so
/usr/lib64/libboost_unit_test_framework-mt.so.1.53.0 /usr/lib64/libboost_unit_test_framework.so.1.53.0
$ make -n | grep libboost_unit_test_framework
/bin/sh ../../../libtool --tag=CXX --mode=link g++ -Wall -Wextra -pedantic -g -O2 -std=c++11 -L/usr/lib64 -o processor_test processor/ProcessorTest.o processor/EventLog.o processor/ServerThread.o libprocessortest.la ../../../lib/cpp/libthrift.la ../../../lib/cpp/libthriftnb.la /usr/lib64/libboost_unit_test_framework.a -L/usr/lib64 -levent -lrt -lpthread
$ sudo cp /usr/local/lib/libboost_unit_test_framework.a /usr/lib64/
$ make
no error.
3、驗證
$ thrift -version
Thrift version 1.0.0-dev
$ which thrift
/usr/local/bin/thrift
3 入門例子
編寫一個簡單的thrift文件，定義相乘函數
$ ls
multiplication.thrift
$ cat multiplication.thrift
namespace py tutorial
service MultiplicationService
{
i32 multiply(1:i32 n1, 2:i32 n2),
}
3.1 C++ server和client
編譯生成c++代碼
$ thrift --gen cpp multiplication.thrift
$ ls -R
.:
gen-cpp multiplication.thrift
./gen-cpp:
multiplication_constants.cpp MultiplicationService.h multiplication_types.h
multiplication_constants.h MultiplicationService_server.skeleton.cpp
MultiplicationService.cpp multiplication_types.cpp
接下來修改官方模版skeleton.cpp[4]
$ mv MultiplicationService_server.skeleton.cpp server.cpp
只需要修改server.cpp中int32_t multiply(const int32_t n1, const int32_t n2)函數
$ cat server.cpp
...(中間省略)...
int32_t multiply(const int32_t n1, const int32_t n2) {
// Your implementation goes here
return n1*n2;
}
...(中間省略)...
然后編寫client.cpp
#include "MultiplicationService.h"
#include <thrift/transport/TSocket.h>
#include <thrift/transport/TBufferTransports.h>
#include <thrift/protocol/TBinaryProtocol.h>
#include <iostream>
using namespace std;
using namespace apache::thrift;
using namespace apache::thrift::protocol;
using namespace apache::thrift::transport;
int main(int argc, char *argv[]) {
boost::shared_ptr<TSocket> socket(new TSocket("localhost", 9090));
boost::shared_ptr<TTransport> transport(new TBufferedTransport(socket));
boost::shared_ptr<TProtocol> protocol(new TBinaryProtocol(transport));
int res=0;
MultiplicationServiceClient client(protocol);
transport->open();
res = client.multiply(10,10);
cout << "res "<<res<<endl;
transport->close();
return 0;
}
編譯源碼
$ g++ -c MultiplicationService.cpp
$ g++ -c multiplication_constants.cpp multiplication_types.cpp server.cpp
$ g++ -lthrift *.o -o server.exe
$ g++ -c MultiplicationService.cpp multiplication_constants.cpp multiplication_types.cpp client.cpp
$ g++ -lthrift *.o -o client.exe
$ ls -R
.:
gen-cpp multiplication.thrift
./gen-cpp:
client.cpp multiplication_constants.cpp MultiplicationService.cpp multiplication_types.cpp server.cpp
client.exe multiplication_constants.h MultiplicationService.h multiplication_types.h server.exe
client.o multiplication_constants.o MultiplicationService.o multiplication_types.o server.o
運行
shell1> ./server.exe
shell2> ./client.exe
res 100
如果運行server.exe出現找不到共享庫的錯誤，解決辦法如下：
$ ./server
./server: error while loading shared libraries: libthrift-1.0.0-dev.so: cannot open shared object file: No such file or directory
$ sudo find / -name "libthrift*so" #where is your libthrift
/usr/local/lib/libthrift-1.0.0-dev.so
/usr/local/lib/libthrift.so
...
$ echo "export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib/:/usr/lib/" >> ~/.bashrc #set env var
$ . ~/.bashrc #reload
3.2 python server和client
生成python代碼
$ thrift --gen py multiplication.thrift
$ ls -R gen-py
./gen-py:
__init__.py tutorial
./gen-py/tutorial:
constants.py __init__.pyc MultiplicationService.pyc ttypes.py
__init__.py MultiplicationService.py MultiplicationService-remote ttypes.pyc
創建server.py和client.py
server.py
import glob
import sys
sys.path.append('gen-py')
sys.path.insert(0, glob.glob('../../lib/py/build/lib*')[0])
from tutorial import MultiplicationService
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol
from thrift.server import TServer
class MultiplicationServiceHandler(MultiplicationService.Iface):
def __init__(self):
self.log = {}
def multiply(self, n1, n2):
print('multiply(%d,%d)' % (n1,n2))
return n1*n2
if __name__ == '__main__':
handler = MultiplicationServiceHandler()
processor = MultiplicationService.Processor(handler)
transport = TSocket.TServerSocket(port=9090)
tfactory = TTransport.TBufferedTransportFactory()
pfactory = TBinaryProtocol.TBinaryProtocolFactory()
server = TServer.TSimpleServer(processor, transport, tfactory, pfactory)
# You could do one of these for a multithreaded server
# server = TServer.TThreadedServer(
# processor, transport, tfactory, pfactory)
# server = TServer.TThreadPoolServer(
# processor, transport, tfactory, pfactory)
print('Starting the server...')
server.serve()
client.py
import sys
import glob
sys.path.append('gen-py')
sys.path.insert(0, glob.glob('../../lib/py/build/lib*')[0])
from tutorial import MultiplicationService
from thrift import Thrift
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol
def main():
transport = TSocket.TSocket('localhost', 9090)
# Buffering is critical. Raw sockets are very slow
transport = TTransport.TBufferedTransport(transport)
# Wrap in a protocol
protocol = TBinaryProtocol.TBinaryProtocol(transport)
# Create a client to use the protocol encoder
client = MultiplicationService.Client(protocol)
# Connect!
transport.open()
print(client.multiply(10,10))
# Close!
transport.close()
if __name__ == '__main__':
try:
main()
except Thrift.TException as tx:
print('%s' % tx.message)
運行
$ chmod a+x server.py
$ chmod a+x client.py
shell1> ./server.py
Starting the server...
multiply(10,10)
shell2> ./client.py
100
python client使用C++ server提供的服務
shell1> ./server.exe
shell2> ./client.py
100