目錄：
一、Thrift1和Thrift2的簡要介紹
1) 寫在前面
2) Thrift1和Thrift2的區別
二、Thrift0.9.2的安裝
1) 安裝依賴插件
2) Thrift0.9.2的編譯
3) Thrift0.9.2編譯后，配置成可執行文件
三、Hbase0.96.2的安裝
1) Hbase的介紹
2) Hbase的安裝
3) Hbase基於源碼的Thrift生成接口文件
四、PHP和Golang使用Thrift1和Thrift2訪問Hbase
1) PHP基於Thrift1訪問Hbase
2) Golang基於Thrift2訪問Hbase
五、常見問題
六、擴展閱讀
目錄：
一、Thrift1和Thrift2的簡要介紹
1) 寫在前面
從Hbase0.94.11開始有兩套thrift接口（可以叫Thrift1和Thrift2），根據官方文檔，Thrift1很可能被拋棄，但網上的文章基本是介紹Thrift1的，本文則兼容介紹Thrift1和Thrift2，目前在網上是比較詳細的介紹Thrift2文章了。
Thrift2相比較Thrift1做了簡化和合成，但不提供查詢所有Table、創建Table、刪除Table的功能了。日常在工作中應該也很少會在Hbase中使用Thrift來創建Table。
從Thrift0.9.1開始支持golang語言，相關使用和demo可以參考這里《
Golang、Php、Python、Java基於Thrift0.9.1實現跨語言調用》，不過由於寫本文時Thrift0.9.1中對Golang訪問Hbase支持的並不友好，在0.9.2中的Thrift2已經進行了改進可以很好的支持Golang訪問Hbase。所以后面在使用Golang訪問Hbase時，使用的是Thrift0.9.2。
網上關於Thrift1的文章一搜索有很多，也就不在這里多描述，在后面的演示中Thrift2中90%的方法都會有，不過在實際使用中可能還需要根據你的系統要求進行二次封裝 。
2) Thrift1和Thrift2的區別
現在官網也沒有太詳細的介紹，下面的對比也是從IDL描述中找到的。這是一位網友整理的，后面的擴展閱讀中鏈接了他的文章，更多的介紹可以去那里看。
|
|
Thrift |
Thrift2 |
| 結構 |
struct TCell struct ColumnDescriptor struct TRegionInfo struct Mutation struct BatchMutation struct TIncrement struct TColumn struct TRowResult struct TScan |
struct TTimeRange struct TColumn struct TColumnValue struct TColumnIncrement struct TResult struct TGet struct TPut struct TDelete struct TIncrement struct TScan struct TRowMutations |
| 異常 |
exception IOError exception IllegalArgument exception AlreadyExists |
exception TIOError exception TIllegalArgument |
| 其他 |
|
union TMutation enum TDeleteType enum TDurability |
| 服務 |
名稱為：Hbase void enableTable() void disableTable() bool isTableEnabled() void compact() void majorCompact() list<Text> getTableNames() map<Text,ColumnDescriptor> getColumnDescriptors() list<TRegionInfo> getTableRegions() void createTable() void deleteTable() list<TCell> get() list<TCell> getVer() list<TCell> getVerTs() list<TRowResult> getRow() list<TRowResult> getRowWithColumns() list<TRowResult> getRowTs() list<TRowResult> getRowWithColumnsTs() list<TRowResult> getRows() list<TRowResult> getRowsWithColumns() list<TRowResult> getRowsTs() list<TRowResult> getRowsWithColumnsTs() void mutateRow() void mutateRowTs() void mutateRows() void mutateRowsTs() i64 atomicIncrement() void deleteAll() void deleteAllTs() void deleteAllRow() void increment() void incrementRows() void deleteAllRowTs() ScannerID scannerOpenWithScan() ScannerID scannerOpen() ScannerID scannerOpenWithStop() ScannerID scannerOpenWithPrefix() ScannerID scannerOpenTs() ScannerID scannerOpenWithStopTs() list<TRowResult> scannerGet() list<TRowResult> scannerGetList() void scannerClose() list<TCell> getRowOrBefore() TRegionInfo getRegionInfo() |
名稱為：THBaseService bool exists(...) TResult get(...) list<TResult> getMultiple(...) void put(...) bool checkAndPut(...) void putMultiple(...) void deleteSingle(...) list<TDelete> deleteMultiple(...) bool checkAndDelete(...) TResult increment(...) i32 openScanner(...) list<TResult> getScannerRows(...) void closeScanner(...) void mutateRow(...) list<TResult> getScannerResults(...) |
二、Thrift0.9.2的安裝
1) 安裝依賴插件
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
|
root@m1:
/home/hadoop
#sudo apt-get install libboost-dev libboost-test-dev libboost-program-options-dev libevent-dev automake libtool flex bison pkg-config g++ libssl-dev
#下載thirft0.9.2版本
root@m1:
/home/hadoop
# git clone https://github.com/apache/thrift.git thrift-git0.9.2
Cloning into
'thrift-git0.9.2'
...
remote: Counting objects: 37221,
done
.
remote: Compressing objects: 100% (195
/195
),
done
.
remote: Total 37221 (delta 312), reused 407 (delta 272)
Receiving objects: 100% (37221
/37221
), 9.61 MiB | 74 KiB
/s
,
done
.
Resolving deltas: 100% (25830
/25830
),
done
.
root@m1:
/home/hadoop
#cd thrift-git0.9.2
# 切換成相應所要用的分支，如果你編譯通過，就不用像我現在這樣切換了
root@m2:
/home/hadoop/thrift-git0
.9.2
# git checkout --track origin/0.9.2
Branch 0.9.2
set
up to track remote branch 0.9.2 from origin.
Switched to a new branch ‘0.9.2’
|
2) Thrift0.9.2的編譯
#編譯Thrift0.9.2版本，注意這個版本和0.9.1編譯方法不一樣，詳情可以參考/home/hadoop/thrift-git0.9.2/compiler/cpp/README.md中的說明
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
|
root@m1:
/home/hadoop/thrift-git0
.9.2
# mkdir -p compiler/cpp/build
root@m1:
/home/hadoop/thrift-git0
.9.2
# cd compiler/cpp/build
root@m1:
/home/hadoop/thrift-git0
.9.2
/compiler/cpp/build
# cmake ..
-- The C compiler identification is GNU
-- The CXX compiler identification is GNU
-- Check
for
working C compiler:
/usr/bin/gcc
-- Check
for
working C compiler:
/usr/bin/gcc
-- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info -
done
-- Check
for
working CXX compiler:
/usr/bin/c
++
-- Check
for
working CXX compiler:
/usr/bin/c
++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info -
done
-- Thrift package version: 0.9.2
-- Thrift version: 0.9.2 (0.9.2)
-- Found FLEX:
/usr/bin/flex
(found version
"2.5.35"
)
-- Found BISON:
/usr/bin/bison
(found version
"2.5"
)
-- Configuring
done
-- Generating
done
-- Build files have been written to:
/home/hadoop/thrift-git0
.9.2
/compiler/cpp/build
root@m1:
/home/hadoop/thrift-git0
.9.2
/compiler/cpp/build
# make
[ 3%] [BISON][thrifty] Building parser with bison 2.5
[ 6%] Generating thrifty.h
[ 9%] [FLEX][thriftl] Building scanner with flex 2.5.35
Scanning dependencies of target libparse
[ 12%] Building CXX object CMakeFiles
/libparse
.
dir
/thrifty
.cc.o
[ 15%] Building CXX object CMakeFiles
/libparse
.
dir
/thriftl
.cc.o
Linking CXX static library liblibparse.a
[ 15%] Built target libparse
Scanning dependencies of target thrift
[ 18%] Building CXX object CMakeFiles
/thrift
.
dir
/src/main
.cc.o
[ 21%] Building C object CMakeFiles
/thrift
.
dir
/src/md5
.c.o
[ 25%] Building CXX object CMakeFiles
/thrift
.
dir
/src/generate/t_generator
.cc.o
[ 28%] Building CXX object CMakeFiles
/thrift
.
dir
/src/parse/t_typedef
.cc.o
[ 31%] Building CXX object CMakeFiles
/thrift
.
dir
/src/parse/parse
.cc.o
[ 34%] Building CXX object CMakeFiles
/thrift
.
dir
/src/generate/t_c_glib_generator
.cc.o
[ 37%] Building CXX object CMakeFiles
/thrift
.
dir
/src/generate/t_cpp_generator
.cc.o
[ 40%] Building CXX object CMakeFiles
/thrift
.
dir
/src/generate/t_java_generator
.cc.o
[ 43%] Building CXX object CMakeFiles
/thrift
.
dir
/src/generate/t_as3_generator
.cc.o
[ 46%] Building CXX object CMakeFiles
/thrift
.
dir
/src/generate/t_csharp_generator
.cc.o
[ 50%] Building CXX object CMakeFiles
/thrift
.
dir
/src/generate/t_py_generator
.cc.o
[ 53%] Building CXX object CMakeFiles
/thrift
.
dir
/src/generate/t_rb_generator
.cc.o
[ 56%] Building CXX object CMakeFiles
/thrift
.
dir
/src/generate/t_perl_generator
.cc.o
[ 59%] Building CXX object CMakeFiles
/thrift
.
dir
/src/generate/t_php_generator
.cc.o
[ 62%] Building CXX object CMakeFiles
/thrift
.
dir
/src/generate/t_erl_generator
.cc.o
[ 65%] Building CXX object CMakeFiles
/thrift
.
dir
/src/generate/t_cocoa_generator
.cc.o
[ 68%] Building CXX object CMakeFiles
/thrift
.
dir
/src/generate/t_st_generator
.cc.o
[ 71%] Building CXX object CMakeFiles
/thrift
.
dir
/src/generate/t_ocaml_generator
.cc.o
[ 75%] Building CXX object CMakeFiles
/thrift
.
dir
/src/generate/t_hs_generator
.cc.o
[ 78%] Building CXX object CMakeFiles
/thrift
.
dir
/src/generate/t_xsd_generator
.cc.o
[ 81%] Building CXX object CMakeFiles
/thrift
.
dir
/src/generate/t_html_generator
.cc.o
[ 84%] Building CXX object CMakeFiles
/thrift
.
dir
/src/generate/t_js_generator
.cc.o
[ 87%] Building CXX object CMakeFiles
/thrift
.
dir
/src/generate/t_javame_generator
.cc.o
[ 90%] Building CXX object CMakeFiles
/thrift
.
dir
/src/generate/t_delphi_generator
.cc.o
[ 93%] Building CXX object CMakeFiles
/thrift
.
dir
/src/generate/t_go_generator
.cc.o
[ 96%] Building CXX object CMakeFiles
/thrift
.
dir
/src/generate/t_d_generator
.cc.o
[100%] Building CXX object CMakeFiles
/thrift
.
dir
/src/generate/t_lua_generator
.cc.o
Linking CXX executable thrift
[100%] Built target thrift
root@m1:
/home/hadoop/thrift-git0
.9.2
/compiler/cpp/build
#
root@m1:
/home/hadoop/thrift-git0
.9.2
/compiler/cpp/build
# ./thrift -version
Thrift version 0.9.2
|
3) Thrift0.9.2編譯后，配置成可執行文件
#將編譯后的Thrift0.9.2放到可執行程序目錄，並命名為thrift092
|
1
2
3
4
5
6
|
root@m1:
/home/hadoop/thrift-git0
.9.2
/compiler/cpp/build
# whereis thrift
thrift:
/usr/local/bin/thrift
root@m1:
/home/hadoop/thrift-git0
.9.2
/compiler/cpp/build
# cp thrift /usr/local/bin/thrift092
root@m1:
/home/hadoop/thrift-git0
.9.2
/compiler/cpp/build
# thrift092 -version
Thrift version 0.9.2
root@m1:
/home/hadoop/thrift-git0
.9.2
/compiler/cpp/build
#
|
三、Hbase0.96.2的安裝
1) Hbase的介紹
hbase的介紹，參考這里：http://baike.baidu.com/view/1993870.htm
2) Hbase的安裝
后面的測試，也是基於上面這篇文章的環境測試的。
3) Hbase基於源碼的Thrift生成接口文件
#下載Hbase的源碼包（后面會使用里面的Thrift1和Thrift2）
|
1
2
|
root@m1:
/home/hadoop
# wget http://mirrors.hust.edu.cn/apache/hbase/hbase-0.96.2/hbase-0.96.2-src.tar.gz
root@m1:
/home/hadoop
# tar xzvf hbase-0.96.2-src.tar.gz
|
#創建測試目錄
|
1
2
|
root@m1:
/home/hadoop
# mkdir thrift_hbase
root@m1:
/home/hadoop
# cd thrift_hbase/
|
#生成Php基於Hbase的Thrift1的接口文件，由於Thrift0.9.1版本對Golang訪問Hbase支持不夠后，后面Golang訪問Hbase用Thrift0.9.2來測試
|
1
2
3
4
5
6
7
|
root@m1:
/home/hadoop/thrift_hbase
# thrift --gen php /home/hadoop/hbase-0.96.2/hbase-thrift/src/main/resources/org/apache/hadoop/hbase/thrift/Hbase.thrift
root@m1:
/home/hadoop/thrift_hbase
# ll
總用量 16
drwxr-xr-x 4 root root 4096 Aug 19 21:19 ./
drwxr-xr-x 47 hadoop hadoop 4096 Aug 19 21:15 ../
drwxr-xr-x 3 root root 4096 Aug 19 21:19 gen-php/
root@m1:
/home/hadoop/thrift_hbase
#
|
四、PHP和Golang使用Thrift1和Thrift2訪問Hbase
1) PHP基於Thrift1訪問Hbase
#將Php用到的Thrift1包復制到thrift_hbase里面
|
1
2
3
4
5
|
root@m1:
/home/hadoop/thrift_hbase
# mkdir libphp
root@m1:
/home/hadoop/thrift_hbase
# cp -r /home/hadoop/thrift-git/lib/php/src/* libphp/
root@m1:
/home/hadoop/thrift_hbase
# cp -r /home/hadoop/thrift-git/lib/php/lib/Thrift libphp/
root@m1:
/home/hadoop/thrift_hbase
# cp -r gen-php/Hbase libphp/
root@m1:
/home/hadoop/thrift_hbase
# rm -rf gen-php/
|
#編寫Hbase.php文件
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
|
<?php
/**
* PHP基於Hbase0.96.2的Thrift1訪問Hbase
*
* @author 迦壹
* @copyright Copyright (c) 2014 (http://idoall.org)
* @license GNU General Public License 2.0
* @version 1
*/
$startTime
= getMillisecond();
$GLOBALS
[
'THRIFT_ROOT'
] =
'./libphp'
; # 指定庫目錄，可以是絕對路徑或是相對路徑
require_once
$GLOBALS
[
'THRIFT_ROOT'
].
'/Thrift/ClassLoader/ThriftClassLoader.php'
;
use
Thrift\ClassLoader\ThriftClassLoader;
use
Thrift\Type\TMessageType;
use
Thrift\Type\TType;
use
Thrift\Exception\TException;
use
Thrift\Factory\TStringFuncFactory;
use
Thrift\StringFunc\TStringFunc;
use
Thrift\StringFunc\Core;
use
Thrift\Protocol\TBinaryProtocol;
use
Thrift\Transport\TSocket;
use
Thrift\Transport\TSocketPool;
use
Thrift\Transport\TBufferedTransport;
use
Hbase\HbaseClient;
use
Hbase\ColumnDescriptor;
use
Hbase\Mutation;
$loader
=
new
ThriftClassLoader();
$loader
->registerNamespace(
'Thrift'
,
$GLOBALS
[
'THRIFT_ROOT'
]); # 加載thrift
$loader
->register();
require_once
(
$GLOBALS
[
'THRIFT_ROOT'
].
'/Hbase/Hbase.php'
);
require_once
(
$GLOBALS
[
'THRIFT_ROOT'
].
'/Hbase/Types.php'
);
$socket
=
new
TSocket(
'm1'
, 9090 );
$socket
->setSendTimeout( 10000 );
// Ten seconds (too long for production, but this is just a demo ;)
$socket
->setRecvTimeout( 20000 );
// Twenty seconds
$transport
=
new
TBufferedTransport(
$socket
);
$protocol
=
new
TBinaryProtocol(
$transport
);
$client
=
new
HbaseClient(
$protocol
);
$transport
->open();
$tmpendTime
= getMillisecond();
echo
"----建立連接用時:"
.
$tmpendTime
.
"-"
.
$startTime
.
"="
.(
$tmpendTime
-
$startTime
).
"毫秒\n"
;
//測試表的名稱
$tempTableName
=
"idoallorg_student"
;
$tmpstartTime
= getMillisecond();
$tables
=
$client
->getTableNames();
sort(
$tables
);
echo
"----列出所有的Table：----\n"
;
foreach
(
$tables
as
$name
) {
echo
(
"\tfound: {$name}\n"
);
if
(
$name
==
$tempTableName
) {
if
(
$client
->isTableEnabled(
$tempTableName
)) {
echo
(
"\t表已經存在，即將關閉: {$tempTableName}\n"
);
$client
->disableTable(
$tempTableName
);
}
echo
(
"\t刪除表: {$tempTableName}\n"
);
$client
->deleteTable(
$tempTableName
);
}
}
$tmpendTime
= getMillisecond();
echo
"----列出所有Table用時:"
.
$tmpendTime
.
"-"
.
$tmpstartTime
.
"="
.(
$tmpendTime
-
$tmpstartTime
).
"毫秒\n"
;
echo
"\r\n"
;
echo
(
"----創建表: {$tempTableName}----\n"
);
$tmpstartTime
= getMillisecond();
$columns
=
array
(
new
ColumnDescriptor(
array
(
'name'
=>
'id'
,
'maxVersions'
=> 10
) ),
new
ColumnDescriptor(
array
(
'name'
=>
'name:'
) ),
new
ColumnDescriptor(
array
(
'name'
=>
'score:'
) )
);
try
{
$client
->createTable(
$tempTableName
,
$columns
);
}
catch
( AlreadyExists
$ae
) {
echo
(
"WARN: {$ae->message}\n"
);
}
$tmpendTime
= getMillisecond();
echo
"----創建表: {$tempTableName}用時:"
.
$tmpendTime
.
"-"
.
$tmpstartTime
.
"="
.(
$tmpendTime
-
$tmpstartTime
).
"毫秒\n"
;
//讀取創建后的表結構
//$tmpstartTime = getMillisecond();
echo
(
"----查看表結構： {$tempTableName}:----\n"
);
$tmpstartTime
= getMillisecond();
$descriptors
=
$client
->getColumnDescriptors(
$tempTableName
);
asort(
$descriptors
);
foreach
(
$descriptors
as
$col
) {
echo
(
"\tcolumn: {$col->name}, maxVer: {$col->maxVersions}\n"
);
}
$tmpendTime
= getMillisecond();
echo
"----查看表結構： {$tempTableName}用時:"
.
$tmpendTime
.
"-"
.
$tmpstartTime
.
"="
.(
$tmpendTime
-
$tmpstartTime
).
"毫秒\n"
;
echo
"\r\n"
;
//
echo
"----向表{$tempTableName} 中寫入1條數據----\n"
;
$tmpstartTime
= getMillisecond();
$row
=
'1'
;
$valid
=
"foobar-000000"
;
$valid1
=
"foobar-11111111"
;
$mutations
=
array
(
new
Mutation(
array
(
'column'
=>
'id:'
,
'value'
=> 1,
)),
new
Mutation(
array
(
'column'
=>
'score:a'
,
'value'
=>
$valid
,
)),
new
Mutation(
array
(
'column'
=>
'name:a'
,
'value'
=>
"idoall.org"
,
)),
);
$client
->mutateRow(
$tempTableName
,
$row
,
$mutations
, null);
$tmpendTime
= getMillisecond();
echo
"----向表{$tempTableName} 中寫入1條數據用時:"
.
$tmpendTime
.
"-"
.
$tmpstartTime
.
"="
.(
$tmpendTime
-
$tmpstartTime
).
"毫秒\n"
;
echo
"\r\n"
;
//
echo
"----獲取剛寫入的score列的數據->get----\n"
;
$tmpstartTime
= getMillisecond();
$arr
=
$client
->get(
$tempTableName
,
$row
,
"score:a"
, null);
foreach
(
$arr
as
$k
=>
$v
) {
// $k = TCell
echo
"\t ------ get one : value = {$v->value}\n"
;
echo
"\t ------ get one : timestamp = {$v->timestamp}\n"
;
}
$tmpendTime
= getMillisecond();
echo
"----獲取剛寫入的 score列的數據->get用時:"
.
$tmpendTime
.
"-"
.
$tmpstartTime
.
"="
.(
$tmpendTime
-
$tmpstartTime
).
"毫秒\n"
;
echo
"\r\n"
;
//
echo
"----獲取剛寫入的數據->getRow----\n"
;
$tmpstartTime
= getMillisecond();
$result
=
$client
->getRow(
$tempTableName
,
$row
, null);
foreach
(
$result
as
$key
=>
$TRowResult
) {
echo
"\trowkey:$TRowResult->row\tcolumns(array):\r\n"
;
foreach
(
$TRowResult
->columns
as
$key
=>
$value
) {
echo
"\t\t\tkey:$key\tvalue:$value->value\ttimestamp:$value->timestamp\n"
;
}
}
$tmpendTime
= getMillisecond();
echo
"----獲取剛寫入的數據->getRow用時:"
.
$tmpendTime
.
"-"
.
$tmpstartTime
.
"="
.(
$tmpendTime
-
$tmpstartTime
).
"毫秒\n"
;
echo
"\r\n"
;
//
echo
"----測試更新數據----\n"
;
$tmpstartTime
= getMillisecond();
try
{
$mutations
=
array
(
new
Mutation(
array
(
'column'
=>
'score:a'
,
'value'
=>
$valid1
) ),
);
$client
->mutateRow(
$tempTableName
,
$row
,
$mutations
, null );
}
catch
( IOError
$e
) {
echo
(
"expected error: {$e->message}\n"
);
}
$tmpendTime
= getMillisecond();
echo
"----測試更新數據用時:"
.
$tmpendTime
.
"-"
.
$tmpstartTime
.
"="
.(
$tmpendTime
-
$tmpstartTime
).
"毫秒\n"
;
echo
"\r\n"
;
echo
"----批量更新10條數據----\n"
;
$tmpstartTime
= getMillisecond();
$mutationsbatch
=
array
();
for
(
$i
=0;
$i
<10;
$i
++)
{
$rowkey
=
$i
+10;
$mutations
=
array
(
new
Mutation(
array
(
'column'
=>
'id:int'
,
'value'
=>
$i
+10,
)),
new
Mutation(
array
(
'column'
=>
'score:txt'
,
'value'
=>
"score"
.
$i
,
)),
new
Mutation(
array
(
'column'
=>
'name:txt'
,
'value'
=>
"idoall.org"
.
$i
,
)),
);
array_push
(
$mutationsbatch
,
new
\Hbase\BatchMutation(
array
(
'row'
=>
$rowkey
,
'mutations'
=>
$mutations
)));
}
$client
->mutateRows(
$tempTableName
,
$mutationsbatch
,null);
$tmpendTime
= getMillisecond();
echo
"----批量更新10條數據用時:"
.
$tmpendTime
.
"-"
.
$tmpstartTime
.
"="
.(
$tmpendTime
-
$tmpstartTime
).
"毫秒\n"
;
echo
"\r\n"
;
echo
"----scanner get scannerOpen-----\r\n"
;
$tmpstartTime
= getMillisecond();
$startRow
= 0;
$scan
=
$client
->scannerOpen(
$tempTableName
,
$startRow
,
array
(
'column'
=>
'score'
,
'column'
=>
'name'
), null);
$nbRows
= 20;
$arr
=
$client
->scannerGetList(
$scan
,
$nbRows
);
echo
'count of result :'
.
count
(
$arr
).
"\n"
;
foreach
(
$arr
as
$k
=>
$TRowResult
) {
echo
"\trow:$TRowResult->row\tcolumns(array):"
;
foreach
(
$TRowResult
->columns
as
$key
=>
$value
) {
echo
"key:$key\tvalue:$value->value\ttimestamp:$value->timestamp\n"
;
}
}
$client
->scannerClose(
$scan
);
$tmpendTime
= getMillisecond();
echo
"----scanner get scannerOpen用時:"
.
$tmpendTime
.
"-"
.
$tmpstartTime
.
"="
.(
$tmpendTime
-
$tmpstartTime
).
"毫秒\n"
;
echo
"\r\n"
;
echo
"----通過RowKey的范圍獲取數據 scannerOpenWithStop-----\r\n"
;
$tmpstartTime
= getMillisecond();
$result
=
$client
->scannerOpenWithStop(
$tempTableName
,
$startRow
,
$nbRows
,
array
(
'column'
=>
'score'
, ),null);
$recordArray
=
array
();
while
(true) {
$record
=
$client
->scannerGet(
$result
);
if
(
$record
== NULL) {
break
;
}
foreach
(
$record
as
$TRowResult
) {
echo
"\trow:$TRowResult->row\tcolumns(array):"
;
foreach
(
$TRowResult
->columns
as
$key
=>
$value
) {
echo
"key:$key\tvalue:$value->value\ttimestamp:$value->timestamp\n"
;
}
}
}
$tmpendTime
= getMillisecond();
echo
"----通過RowKey的范圍獲取數據 scannerOpenWithStop用時:"
.
$tmpendTime
.
"-"
.
$tmpstartTime
.
"="
.(
$tmpendTime
-
$tmpstartTime
).
"毫秒\n"
;
echo
"\r\n"
;
//刪除表
echo
"----刪除表 {$tempTableName}----\n"
;
$tmpstartTime
= getMillisecond();
try
{
$tmpstartTime_disableTable
= getMillisecond();
$client
->disableTable(
$tempTableName
);
$tmpendTime_disableTable
= getMillisecond();
echo
"\t其中關閉表 {$tempTableName}用時:"
.
$tmpendTime_disableTable
.
"-"
.
$tmpstartTime_disableTable
.
"="
.(
$tmpendTime_disableTable
-
$tmpstartTime_disableTable
).
"毫秒\n"
;
$tmpstartTime_disableTable
= getMillisecond();
$client
->deleteTable(
$tempTableName
);
$tmpendTime_disableTable
= getMillisecond();
echo
"\t其中刪除表 {$tempTableName}用時:"
.
$tmpendTime_disableTable
.
"-"
.
$tmpstartTime_disableTable
.
"="
.(
$tmpendTime_disableTable
-
$tmpstartTime_disableTable
).
"毫秒\n"
;
}
catch
( AlreadyExists
$ae
) {
echo
(
"WARN: {$ae->message}\n"
);
}
$tmpendTime
= getMillisecond();
echo
"----刪除表 {$tempTableName}用時:"
.
$tmpendTime
.
"-"
.
$tmpstartTime
.
"="
.(
$tmpendTime
-
$tmpstartTime
).
"毫秒\n"
;
$endTime
= getMillisecond();
echo
"\r\n"
;
echo
"PHP調用 總計 用時:"
.
$endTime
.
"-"
.
$startTime
.
"="
.(
$endTime
-
$startTime
).
"毫秒\n"
;
function
getMillisecond() {
list(
$t1
,
$t2
) =
explode
(
' '
, microtime());
return
(float)sprintf(
'%.0f'
, (
floatval
(
$t1
) +
floatval
(
$t2
)) * 1000);
}
$transport
->close();
?>
|
#啟動Hbase的Thrift，如果不了解，看下這篇文章吧，本文的環境都是基於這篇文章的《
ubuntu12.04+hadoop2.2.0+zookeeper3.4.5+hbase0.96.2+hive0.13.1分布式環境部署》
|
1
|
root@m1:
/home/hadoop/thrift_hbase
# /home/hadoop/hbase-0.96.2-hadoop2/bin/hbase-daemon.sh start thrift
|
#運行Hbase.php，
如果你也可以看到下面和我一樣的提示，那么恭喜你也成功地使用PHP通過Thrift1訪問到Hbase
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
|
root@m1:
/home/hadoop/thrift_hbase
# php hbase.php
----建立連接用時:1408545184136-1408545184102=34毫秒
----列出所有的Table：----
found: demo_table
found: hbase2hive_idoall
found: hive2hbase_idoall
found: test_idoall_org
----列出所有Table用時:1408545184229-1408545184136=93毫秒
----創建表: idoallorg_student----
----創建表: idoallorg_student用時:1408545185026-1408545184229=797毫秒
----查看表結構： idoallorg_student:----
column:
id
:, maxVer: 10
column: name:, maxVer: 3
column: score:, maxVer: 3
----查看表結構： idoallorg_student用時:1408545185047-1408545185026=21毫秒
----向表idoallorg_student 中寫入1條數據----
----向表idoallorg_student 中寫入1條數據用時:1408545185288-1408545185047=241毫秒
----獲取剛寫入的score列的數據->get----
------ get one : value = foobar-000000
------ get one : timestamp = 1408545184901
----獲取剛寫入的 score列的數據->get用時:1408545185373-1408545185289=84毫秒
----獲取剛寫入的數據->getRow----
rowkey:1 columns(array):
key:
id
: value:1 timestamp:1408545184901
key:name:a value:idoall.org timestamp:1408545184901
key:score:a value:foobar-000000 timestamp:1408545184901
----獲取剛寫入的數據->getRow用時:1408545185383-1408545185373=10毫秒
----測試更新數據----
----測試更新數據用時:1408545185389-1408545185384=5毫秒
----批量更新10條數據----
----批量更新10條數據用時:1408545185442-1408545185389=53毫秒
----scanner get scannerOpen-----
count of result :11
row:1 columns(array):key:name:a value:idoall.org timestamp:1408545184901
row:10 columns(array):key:name:txt value:idoall.org0 timestamp:1408545185053
row:11 columns(array):key:name:txt value:idoall.org1 timestamp:1408545185053
row:12 columns(array):key:name:txt value:idoall.org2 timestamp:1408545185053
row:13 columns(array):key:name:txt value:idoall.org3 timestamp:1408545185053
row:14 columns(array):key:name:txt value:idoall.org4 timestamp:1408545185053
row:15 columns(array):key:name:txt value:idoall.org5 timestamp:1408545185053
row:16 columns(array):key:name:txt value:idoall.org6 timestamp:1408545185053
row:17 columns(array):key:name:txt value:idoall.org7 timestamp:1408545185053
row:18 columns(array):key:name:txt value:idoall.org8 timestamp:1408545185053
row:19 columns(array):key:name:txt value:idoall.org9 timestamp:1408545185053
----scanner get scannerOpen用時:1408545185484-1408545185443=41毫秒
----通過RowKey的范圍獲取數據 scannerOpenWithStop-----
row:1 columns(array):key:score:a value:foobar-11111111 timestamp:1408545185003
row:10 columns(array):key:score:txt value:score0 timestamp:1408545185053
row:11 columns(array):key:score:txt value:score1 timestamp:1408545185053
row:12 columns(array):key:score:txt value:score2 timestamp:1408545185053
row:13 columns(array):key:score:txt value:score3 timestamp:1408545185053
row:14 columns(array):key:score:txt value:score4 timestamp:1408545185053
row:15 columns(array):key:score:txt value:score5 timestamp:1408545185053
row:16 columns(array):key:score:txt value:score6 timestamp:1408545185053
row:17 columns(array):key:score:txt value:score7 timestamp:1408545185053
row:18 columns(array):key:score:txt value:score8 timestamp:1408545185053
row:19 columns(array):key:score:txt value:score9 timestamp:1408545185053
----通過RowKey的范圍獲取數據 scannerOpenWithStop用時:1408545185536-1408545185484=52毫秒
----刪除表 idoallorg_student----
其中關閉表 idoallorg_student用時:1408545186819-1408545185536=1283毫秒
其中刪除表 idoallorg_student用時:1408545186974-1408545186819=155毫秒
----刪除表 idoallorg_student用時:1408545186974-1408545185536=1438毫秒
本次調用 總計 用時:1408545186974-1408545184102=2872毫秒
|
2) Golang基於Thrift2訪問Hbase
#生成Golang基於Hbase的Thrift2的接口文件
|
1
|
root@m1:
/home/hadoop/thrift_hbase
# thrift092 -r --gen go /home/hadoop/hbase-0.96.2/hbase-thrift/src/main/resources/org/apache/hadoop/hbase/thrift2/hbase.thrift
|
#將Thrift2生成的開發庫復制到GOPATH的src中
#刪除掉0.9.1的文件，可能作者比較匆忙，在我使用0.9.2生成Thrift2的時候，還存在Thrift1的接口，這會對我們的測試帶來干擾（后來我下載thrift1.0.0 dev時發現已經刪除了這個文件，如果你生成IDL后的接口文件中，不包含這個文件，可以不用刪除）
|
1
2
|
root@m1:
/home/hadoop/thrift_hbase
# rm gen-go/hbase/t_h_base_service.go
root@m1:
/home/hadoop/thrift_hbase
# cp -r gen-go/hbase $GOPATH/src
|
#編寫hbase.go文件
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
|
/
/
/
/
Golang基於Hbase0.
96.2
的Thrift2訪問Hbase
/
/
/
/
@author 迦壹
/
/
@copyright Copyright (c)
2014
(http:
/
/
idoall.org)
/
/
@license GNU General Public License
2.0
/
/
@version
1
package main
import
(
"encoding/binary"
"fmt"
"git.apache.org/thrift.git/lib/go/thrift"
"hbase"
"net"
"os"
"reflect"
"strconv"
"time"
)
const (
HOST
=
"m1"
PORT
=
"9090"
TESTRECORD
=
10
)
func main() {
startTime :
=
currentTimeMillis()
logformatstr_ :
=
"----%s\n"
logformatstr :
=
"----%s 用時:%d-%d=%d毫秒\n\n"
logformattitle :
=
"建立連接"
rowkey :
=
"1"
temptable :
=
"test_idoall_org"
protocolFactory :
=
thrift.NewTBinaryProtocolFactoryDefault()
transport, err :
=
thrift.NewTSocket(net.JoinHostPort(HOST, PORT))
if
err !
=
nil {
fmt.Fprintln(os.Stderr,
"error resolving address:"
, err)
os.Exit(
1
)
}
client :
=
hbase.NewTHBaseServiceClientFactory(transport, protocolFactory)
if
err :
=
transport.
Open
(); err !
=
nil {
fmt.Fprintln(os.Stderr,
"Error opening socket to "
+
HOST
+
":"
+
PORT,
" "
, err)
os.Exit(
1
)
}
tmpendTime :
=
currentTimeMillis()
fmt.Printf(logformatstr, logformattitle, tmpendTime, startTime, (tmpendTime
-
startTime))
defer transport.Close()
/
/
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Exists
logformattitle
=
"調用Exists方法"
fmt.Printf(logformatstr_, logformattitle)
tmpstartTime :
=
currentTimeMillis()
/
/
isexists, err :
=
(client.Exists([]byte(temptable), &hbase.TGet{Row: []byte(rowkey)}))
fmt.Printf(
"rowkey{%s} in table{%s} Exists:%t\t"
, rowkey, temptable, isexists)
if
err !
=
nil {
fmt.Printf(
"Exists err:%s\n"
, err)
}
fmt.Println("")
tmpendTime
=
currentTimeMillis()
fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime
-
tmpstartTime))
/
/
-
-
-
-
-
-
-
-
-
-
-
-
-
-
put
logformattitle
=
"調用Put方法寫數據"
fmt.Printf(logformatstr_, logformattitle)
tmpstartTime
=
currentTimeMillis()
cvarr :
=
[]
*
hbase.TColumnValue{
&hbase.TColumnValue{
Family: []byte(
"name"
),
Qualifier: []byte(
"idoall.org"
),
Value: []byte(
"welcome idoall.org"
)}}
temptput :
=
hbase.TPut{Row: []byte(rowkey), ColumnValues: cvarr}
err
=
client.Put([]byte(temptable), &temptput)
if
err !
=
nil {
fmt.Printf(
"Put err:%s\n"
, err)
}
else
{
fmt.Println(
"Put done"
)
}
tmpendTime
=
currentTimeMillis()
fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime
-
tmpstartTime))
/
/
-
-
-
-
-
-
-
-
-
-
-
-
Get
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
logformattitle
=
"調用Get方法獲取新增加的數據"
fmt.Printf(logformatstr_, logformattitle)
tmpstartTime
=
currentTimeMillis()
/
/
result, err :
=
(client.Get([]byte(temptable), &hbase.TGet{Row: []byte(rowkey)}))
if
err !
=
nil {
fmt.Printf(
"Get err:%s\n"
, err)
}
else
{
fmt.Println(
"Rowkey:"
+
string(result.Row))
for
_, cv :
=
range
result.ColumnValues {
printscruct(cv)
}
}
tmpendTime
=
currentTimeMillis()
fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime
-
tmpstartTime))
/
/
-
-
-
-
-
-
-
-
-
-
-
-
-
-
put update
logformattitle
=
"調用Put update方法'修改'數據"
fmt.Printf(logformatstr_, logformattitle)
tmpstartTime
=
currentTimeMillis()
cvarr
=
[]
*
hbase.TColumnValue{
&hbase.TColumnValue{
Family: []byte(
"name"
),
Qualifier: []byte(
"idoall.org"
),
Value: []byte(
"welcome idoall.org---update"
)}}
temptput
=
hbase.TPut{Row: []byte(rowkey), ColumnValues: cvarr}
err
=
client.Put([]byte(temptable), &temptput)
if
err !
=
nil {
fmt.Printf(
"Put update err:%s\n"
, err)
}
else
{
fmt.Println(
"Put update done"
)
}
tmpendTime
=
currentTimeMillis()
fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime
-
tmpstartTime))
/
/
-
-
-
-
-
-
-
-
-
-
-
-
Get update
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
logformattitle
=
"調用Get方法獲取'修改'后的數據"
fmt.Printf(logformatstr_, logformattitle)
tmpstartTime
=
currentTimeMillis()
/
/
result, err
=
(client.Get([]byte(temptable), &hbase.TGet{Row: []byte(rowkey)}))
if
err !
=
nil {
fmt.Printf(
"Get update err:%s\n"
, err)
}
else
{
fmt.Println(
"update Rowkey:"
+
string(result.Row))
for
_, cv :
=
range
result.ColumnValues {
printscruct(cv)
}
}
tmpendTime
=
currentTimeMillis()
fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime
-
tmpstartTime))
/
/
-
-
-
-
-
-
-
-
-
-
-
-
DeleteSingle
-
-
-
-
-
-
-
-
-
-
-
-
logformattitle
=
"調用DeleteSingle方法刪除一條數據"
fmt.Printf(logformatstr_, logformattitle)
tmpstartTime
=
currentTimeMillis()
tdelete :
=
hbase.TDelete{Row: []byte(rowkey)}
err
=
client.DeleteSingle([]byte(temptable), &tdelete)
if
err !
=
nil {
fmt.Printf(
"DeleteSingle err:%s\n"
, err)
}
else
{
fmt.Printf(
"DeleteSingel done\n"
)
}
tmpendTime
=
currentTimeMillis()
fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime
-
tmpstartTime))
/
/
-
-
-
-
-
-
-
-
-
-
-
-
-
PutMultiple
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
logformattitle
=
"調用PutMultiple方法添加"
+
strconv.Itoa(TESTRECORD)
+
"條數據"
fmt.Printf(logformatstr_, logformattitle)
tmpstartTime
=
currentTimeMillis()
var tputArr []
*
hbase.TPut
for
i :
=
0
; i < TESTRECORD; i
+
+
{
putrowkey :
=
strconv.Itoa(i)
tputArr
=
append(tputArr, &hbase.TPut{
Row: []byte(putrowkey),
ColumnValues: []
*
hbase.TColumnValue{
&hbase.TColumnValue{
Family: []byte(
"name"
),
Qualifier: []byte(
"idoall.org"
),
Value: []byte(time.Now().String())}}})
}
err
=
client.PutMultiple([]byte(temptable), tputArr)
if
err !
=
nil {
fmt.Printf(
"PutMultiple err:%s\n"
, err)
}
else
{
fmt.Printf(
"PutMultiple done\n"
)
}
tmpendTime
=
currentTimeMillis()
fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime
-
tmpstartTime))
/
/
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
GetMultiple
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
logformattitle
=
"調用GetMultiple方法獲取"
+
strconv.Itoa(TESTRECORD)
+
"數據"
fmt.Printf(logformatstr_, logformattitle)
tmpstartTime
=
currentTimeMillis()
/
/
var tgets []
*
hbase.TGet
for
i :
=
0
; i < TESTRECORD; i
+
+
{
putrowkey :
=
strconv.Itoa(i)
tgets
=
append(tgets, &hbase.TGet{
Row: []byte(putrowkey)})
}
results, err :
=
client.GetMultiple([]byte(temptable), tgets)
if
err !
=
nil {
fmt.Printf(
"GetMultiple err:%s"
, err)
}
else
{
fmt.Printf(
"GetMultiple Count:%d\n"
,
len
(results))
for
_, k :
=
range
results {
fmt.Println(
"Rowkey:"
+
string(k.Row))
for
_, cv :
=
range
k.ColumnValues {
printscruct(cv)
}
}
}
tmpendTime
=
currentTimeMillis()
fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime
-
tmpstartTime))
/
/
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
TMutation
/
/
TMutation包含一個TGet一個TPut，就不做測試了
/
/
可以和MutateRow結合使用
/
/
/
/
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
OpenScanner
logformattitle
=
"調用OpenScanner方法"
fmt.Printf(logformatstr_, logformattitle)
tmpstartTime
=
currentTimeMillis()
startrow :
=
make([]byte,
4
)
binary.LittleEndian.PutUint32(startrow,
1
)
stoprow :
=
make([]byte,
4
)
binary.LittleEndian.PutUint32(stoprow,
10
)
scanresultnum, err :
=
client.OpenScanner([]byte(temptable), &hbase.TScan{
StartRow: startrow,
StopRow: stoprow,
/
/
FilterString: []byte(
"RowFilter(=, 'regexstring:00[1-3]00')"
),
/
/
FilterString: []byte(
"PrefixFilter('1407658495588-')"
),
Columns: []
*
hbase.TColumn{
&hbase.TColumn{
Family: []byte(
"name"
),
Qualifier: []byte(
"idoall.org"
)}}})
if
err !
=
nil {
fmt.Printf(
"OpenScanner err:%s\n"
, err)
}
else
{
fmt.Printf(
"OpenScanner %d done\n"
, scanresultnum)
scanresult, err :
=
client.GetScannerRows(scanresultnum,
100
)
if
err !
=
nil {
fmt.Printf(
"GetScannerRows err:%s\n"
, err)
}
else
{
fmt.Printf(
"GetScannerRows %d done\n"
,
len
(scanresult))
for
_, k :
=
range
scanresult {
fmt.Println(
"scan Rowkey:"
+
string(k.Row))
for
_, cv :
=
range
k.ColumnValues {
printscruct(cv)
}
}
}
}
tmpendTime
=
currentTimeMillis()
fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime
-
tmpstartTime))
/
/
-
-
closescanner
logformattitle
=
"調用CloseScanner方法"
fmt.Printf(logformatstr_, logformattitle)
tmpstartTime
=
currentTimeMillis()
err
=
client.CloseScanner(scanresultnum)
if
err !
=
nil {
fmt.Printf(
"CloseScanner err:%s\n"
, err)
}
tmpendTime
=
currentTimeMillis()
fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime
-
tmpstartTime))
/
/
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
GetScannerResults
logformattitle
=
"調用GetScannerResults方法"
fmt.Printf(logformatstr_, logformattitle)
tmpstartTime
=
currentTimeMillis()
/
/
gsr, err :
=
client.GetScannerResults([]byte(temptable), &hbase.TScan{
StartRow: startrow,
StopRow: stoprow,
/
/
FilterString: []byte(
"RowFilter(=, 'regexstring:00[1-3]00')"
),
/
/
FilterString: []byte(
"PrefixFilter('1407658495588-')"
),
Columns: []
*
hbase.TColumn{
&hbase.TColumn{
Family: []byte(
"name"
),
Qualifier: []byte(
"idoall.org"
)}}},
100
)
if
err !
=
nil {
fmt.Printf(
"GetScannerResults err:%s\n"
, err)
}
else
{
fmt.Printf(
"GetScannerResults %d done\n"
,
len
(gsr))
for
_, k :
=
range
gsr {
fmt.Println(
"scan Rowkey:"
+
string(k.Row))
for
_, cv :
=
range
k.ColumnValues {
printscruct(cv)
}
}
}
tmpendTime
=
currentTimeMillis()
fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime
-
tmpstartTime))
/
/
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
DeleteMultiple
-
-
-
-
-
-
-
-
-
-
-
-
-
-
logformattitle
=
"調用DeleteMultiple方法刪除"
+
strconv.Itoa(TESTRECORD)
+
"數據"
fmt.Printf(logformatstr_, logformattitle)
tmpstartTime
=
currentTimeMillis()
var tdelArr []
*
hbase.TDelete
for
i :
=
0
; i < TESTRECORD; i
+
+
{
putrowkey :
=
strconv.Itoa(i)
tdelArr
=
append(tdelArr, &hbase.TDelete{
Row: []byte(putrowkey)})
}
r, err :
=
client.DeleteMultiple([]byte(temptable), tdelArr)
if
err !
=
nil {
fmt.Printf(
"DeleteMultiple err:%s\n"
, err)
}
else
{
fmt.Printf(
"DeleteMultiple %d done\n"
, TESTRECORD)
fmt.Println(r)
}
tmpendTime
=
currentTimeMillis()
fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime
-
tmpstartTime))
endTime :
=
currentTimeMillis()
fmt.Printf(
"\nGolang調用總計用時:%d-%d=%d毫秒\n"
, endTime, startTime, (endTime
-
startTime))
}
func printscruct(cv interface{}) {
switch reflect.ValueOf(cv).Interface().(
type
) {
case
*
hbase.TColumnValue:
s :
=
reflect.ValueOf(cv).Elem()
typeOfT :
=
s.
Type
()
/
/
獲取Thrift2中struct的field
for
i :
=
0
; i < s.NumField(); i
+
+
{
f :
=
s.Field(i)
fileldformatstr :
=
"\t%d: %s(%s)= %v\n"
switch f.Interface().(
type
) {
case []uint8:
fmt.Printf(fileldformatstr, i, typeOfT.Field(i).Name, f.
Type
(), string(f.Interface().([]uint8)))
case
*
int64:
var tempint64 int64
if
f.Interface().(
*
int64)
=
=
nil {
tempint64
=
0
}
else
{
tempint64
=
*
f.Interface().(
*
int64)
}
fmt.Printf(fileldformatstr, i, typeOfT.Field(i).Name, f.
Type
(), tempint64)
default:
fmt.Printf(
"I don't know"
)
}
}
default:
fmt.Printf(
"I don't know"
)
fmt.
Print
(reflect.ValueOf(cv))
}
}
func currentTimeMillis() int64 {
return
time.Now().UnixNano()
/
1000000
}
|
#在hbase創建一個測試表“test_idoall_org”,用於使用Golang基於Thrift2的客戶端來測試，因為Thrift2並沒有提供創建和刪除表的方法
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
|
root@m1:/home/hadoop# /home/hadoop/hbase-0.96.2-hadoop2/bin/hbase shell
2014-07-27 09:31:07,601 INFO [main] Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 0.96.2-hadoop2, r1581096, Mon Mar 24 16:03:18 PDT 2014
hbase(main):001:0> list
TABLE
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hadoop/hbase-0.96.2-hadoop2/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hadoop/hadoop-2.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
0 row(s) in 2.8030 seconds
=> []
hbase(main):002:0> version
0.96.2-hadoop2, r1581096, Mon Mar 24 16:03:18 PDT 2014
hbase(main):003:0> status
2 servers, 0 dead, 1.0000 average load
hbase(main):004:0> create 'test_idoall_org','uid','name'
0 row(s) in 0.5800 seconds
=> Hbase::Table - test_idoall_org
hbase(main):005:0> list
TABLE
test_idoall_org
1 row(s) in 0.0320 seconds
=> ["test_idoall_org"]
hbase(main):006:0> put 'test_idoall_org','10086','name:idoall','idoallvalue'
0 row(s) in 0.1090 seconds ^
hbase(main):009:0> get 'test_idoall_org','10086'
COLUMN CELL
name:idoall timestamp=1406424831473, value=idoallvalue
1 row(s) in 0.0450 seconds
hbase(main):010:0> scan 'test_idoall_org'
ROW COLUMN+CELL
10086 column=name:idoall, timestamp=1406424831473, value=idoallvalue
1 row(s) in 0.0620 seconds
hbase(main):011:0>
|
#啟動Hbase的Thrift2
|
1
2
3
4
5
6
7
8
9
10
11
12
13
|
root@m1:
/home/hadoop/thrift_hbase
# /home/hadoop/hbase-0.96.2-hadoop2/bin/hbase-daemon.sh start thrift2
#在這里可以看到thrift2的日志記錄到下面的路徑中
starting thrift2, logging to
/home/hadoop/hbase-0
.96.2-hadoop2
/bin/
..
/logs/hbase-root-thrift2-m1
.out
root@m1:
/home/hadoop/thrift_hbase
# jps
5705 Main
3614 ResourceManager
9426 HMaster
9944 Jps
3844 DFSZKFailoverController
9679 ThriftServer
2773 QuorumPeerMain
2944 JournalNode
root@m1:
/home/hadoop/thrift_hbase
#
|
#運行基於Thrift2的Golang客戶端，在之前要先確保hbase已經打開，在瀏覽器輸入：http://m1:60010/master-status,可以看到下圖效果，如果看不到，去調試下hbase吧，還沒有啟動~~~：
#運行hbase.go，
如果你看到和我一樣的提示信息，再次恭喜你我們的Golang也可以正常訪問到Hbase了。
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
|
root@m1:
/home/hadoop/thrift_hbase
# go run hbase.go
----建立連接 用時:1408639700869-1408639700867=2毫秒
----調用Exists方法
rowkey{1}
in
table{test_idoall_org} Exists:
false
----調用Exists方法 用時:1408639700894-1408639700869=25毫秒
----調用Put方法寫數據
Put
done
----調用Put方法寫數據 用時:1408639700905-1408639700894=11毫秒
----調用Get方法獲取新增加的數據
Rowkey:1
0: Family([]uint8)= name
1: Qualifier([]uint8)= idoall.org
2: Value([]uint8)= welcome idoall.org
3: Timestamp(*int64)= 1408639700843
----調用Get方法獲取新增加的數據 用時:1408639700909-1408639700905=4毫秒
----調用Put update方法
'修改'
數據
Put update
done
----調用Put update方法
'修改'
數據 用時:1408639700916-1408639700909=7毫秒
----調用Get方法獲取
'修改'
后的數據
update Rowkey:1
0: Family([]uint8)= name
1: Qualifier([]uint8)= idoall.org
2: Value([]uint8)= welcome idoall.org---update
3: Timestamp(*int64)= 1408639700858
----調用Get方法獲取
'修改'
后的數據 用時:1408639700918-1408639700916=2毫秒
----調用DeleteSingle方法刪除一條數據
DeleteSingel
done
----調用DeleteSingle方法刪除一條數據 用時:1408639700931-1408639700918=13毫秒
----調用PutMultiple方法添加10條數據
PutMultiple
done
----調用PutMultiple方法添加10條數據 用時:1408639700941-1408639700931=10毫秒
----調用GetMultiple方法獲取10數據
GetMultiple Count:10
Rowkey:0
0: Family([]uint8)= name
1: Qualifier([]uint8)= idoall.org
2: Value([]uint8)= 2014-08-22 00:48:20.931138625 +0800 CST
3: Timestamp(*int64)= 1408639700881
Rowkey:1
0: Family([]uint8)= name
1: Qualifier([]uint8)= idoall.org
2: Value([]uint8)= 2014-08-22 00:48:20.931689108 +0800 CST
3: Timestamp(*int64)= 1408639700881
Rowkey:2
0: Family([]uint8)= name
1: Qualifier([]uint8)= idoall.org
2: Value([]uint8)= 2014-08-22 00:48:20.931691688 +0800 CST
3: Timestamp(*int64)= 1408639700881
Rowkey:3
0: Family([]uint8)= name
1: Qualifier([]uint8)= idoall.org
2: Value([]uint8)= 2014-08-22 00:48:20.931693392 +0800 CST
3: Timestamp(*int64)= 1408639700881
Rowkey:4
0: Family([]uint8)= name
1: Qualifier([]uint8)= idoall.org
2: Value([]uint8)= 2014-08-22 00:48:20.931694819 +0800 CST
3: Timestamp(*int64)= 1408639700881
Rowkey:5
0: Family([]uint8)= name
1: Qualifier([]uint8)= idoall.org
2: Value([]uint8)= 2014-08-22 00:48:20.931696588 +0800 CST
3: Timestamp(*int64)= 1408639700881
Rowkey:6
0: Family([]uint8)= name
1: Qualifier([]uint8)= idoall.org
2: Value([]uint8)= 2014-08-22 00:48:20.931698745 +0800 CST
3: Timestamp(*int64)= 1408639700881
Rowkey:7
0: Family([]uint8)= name
1: Qualifier([]uint8)= idoall.org
2: Value([]uint8)= 2014-08-22 00:48:20.931700279 +0800 CST
3: Timestamp(*int64)= 1408639700881
Rowkey:8
0: Family([]uint8)= name
1: Qualifier([]uint8)= idoall.org
2: Value([]uint8)= 2014-08-22 00:48:20.931702153 +0800 CST
3: Timestamp(*int64)= 1408639700881
Rowkey:9
0: Family([]uint8)= name
1: Qualifier([]uint8)= idoall.org
2: Value([]uint8)= 2014-08-22 00:48:20.931704203 +0800 CST
3: Timestamp(*int64)= 1408639700881
----調用GetMultiple方法獲取10數據 用時:1408639700950-1408639700941=9毫秒
----調用OpenScanner方法
OpenScanner 23
done
GetScannerRows 0
done
----調用OpenScanner方法 用時:1408639700956-1408639700950=6毫秒
----調用CloseScanner方法
----調用CloseScanner方法 用時:1408639700957-1408639700956=1毫秒
----調用GetScannerResults方法
GetScannerResults 0
done
----調用GetScannerResults方法 用時:1408639700961-1408639700958=3毫秒
----調用DeleteMultiple方法刪除10數據
DeleteMultiple 10
done
[]
----調用DeleteMultiple方法刪除10數據 用時:1408639700967-1408639700961=6毫秒
root@m1:
/home/hadoop/thrift_hbase
#
|
五、常見問題
1) 並發訪問線程數
首先，為了盡可能減少由於網絡傳輸帶來的時間開銷，HBase的Thrift Server最好和應用客戶端部署在同一台機器上。Thrift Server啟動時可以通過參數配置並發線程數，否則很容易導致Thrift Server線程滿了不響應客戶端的讀寫請求，具體命令：/home/hadoop/hbase-0.96.2-hadoop2/bin/hbase- daemon.sh start thrift –threadpool -m 200 -w 500（更多參數參考這里：/home/hadoop/hbase-0.96.2-hadoop2/bin/hbase-daemon.sh start thrift -h）。
2) 最大堆內存配置
如果客戶端與Thrift Server進行scan操作順序讀取數據，而且設置了一定的cache記錄條數（通過TScan的int32_t caching變量設置），那么這些被caching的記錄數可能會占用Thrift Server相當部分的堆內存，尤其在多客戶端並發訪問時更明顯。
因此，在Thrift Server啟動前，可以調大最大堆內存，否則可 能由於java.lang.OutOfMemoryError異常而導致進程被殺掉，尤其是當Scan時設置了較大的caching記錄條數的情況（默認為expor t HBASE_HEAPSIZE=1000MB，可以在
/home/hadoop/hbase-0.96.2-hadoop2/hbase- env.sh中設置）。
六、擴展閱讀
-------- -------------------------------
博文作者：迦壹
轉載聲明：可以轉載, 但必須以超鏈接形式標 明文章原始出處和作者信息及版權聲明，謝謝合作！
---------------------------------------