1.安裝tensorflow serving
1.1確保當前環境已經安裝並可運行tensorflow
從github上下載源碼
git clone --recurse-submodules https:
//github.com/tensorflow/serving
|
進入到serving目錄下的tensorflow運行./configure,並安裝步驟完成（需將 2問題解決的的步驟全操作完后執行安裝步驟）
1.2.編譯example代碼
bazel build tensorflow_serving/example/...
|
1.3.運行mnist例子導出model到/tmp/mnist_export目錄下，目錄下會根據export_version創建一個目錄名為 /tmp/mnist_export/00000001
rm -rf /tmp/mnist_export/(第一次執行不存在，不必操作)
bazel-bin/tensorflow_serving/example/mnist_export --training_iteration=
10000
--export_version=
1
/tmp/mnist_export
Training model...
(
'Extracting'
,
'/tmp/train-images-idx3-ubyte.gz'
)
(
'Extracting'
,
'/tmp/train-labels-idx1-ubyte.gz'
)
(
'Extracting'
,
'/tmp/t10k-images-idx3-ubyte.gz'
)
(
'Extracting'
,
'/tmp/t10k-labels-idx1-ubyte.gz'
)
training accuracy
0.9219
Done training!
Exporting trained model to /tmp/mnist_export
Done exporting!
|
1.4 執行inference開啟服務，端口9000，目錄指向之前導出的目錄
bazel-bin/tensorflow_serving/example/mnist_inference --port=
9000
/tmp/mnist_export/
00000001
I tensorflow_serving/session_bundle/session_bundle.cc:
130
] Attempting to load a SessionBundle from: /tmp/mnist_export/
00000001
I tensorflow_serving/session_bundle/session_bundle.cc:
107
] Running restore op
for
SessionBundle
I tensorflow_serving/session_bundle/session_bundle.cc:
178
] Done loading SessionBundle
I tensorflow_serving/example/mnist_inference.cc:
163
] Running...
|
1.6 執行測試client
bazel-bin/tensorflow_serving/example/mnist_client --num_tests=
1000
--server=localhost:
9000
(
'Extracting'
,
'/tmp/train-images-idx3-ubyte.gz'
)
(
'Extracting'
,
'/tmp/train-labels-idx1-ubyte.gz'
)
(
'Extracting'
,
'/tmp/t10k-images-idx3-ubyte.gz'
)
(
'Extracting'
,
'/tmp/t10k-labels-idx1-ubyte.gz'
)
........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
Inference error rate:
9.2
%
|
2.問題解決
no such
package
'@boringssl_git//'
: Error cloning repository:https://boringssl.googlesource.com/boringssl: cannot open git-upload-pack and referenced by
'//external:libssl'
.
|
由於GFW把google的很多地址給牆了，所以無法下載相關的內容，修改seving目錄下的tensorflow/tensorflow/workspace.bzl 文件相關的repository
git_repository(
name =
"boringssl_git"
,
#commit =
"436432d849b83ab90f18773e4ae1c7a8f148f48d"
,
commit =
"db0729054d5964feab9e60089ba2d06a181e78b1"
,
init_submodules = True,
)
|
https://github.com/tensorflow/serving/issues/6
運行mnist_client時報錯
Traceback (most recent call last):
File
"/root/tensorflow-serving/bazel-bin/tensorflow_serving/example/mnist_client.runfiles/__main__/tensorflow_serving/example/mnist_client.py"
, line
34
, in <module>
from grpc.beta
import
implementations
ImportError: No module named grpc.beta
|
使用pip安裝grpcio模塊
pip install grpcio
|
https://github.com/grpc/grpc/tree/master/src/python/grpcio
export過程報錯缺少 manifest_pb2.py 的解決方法：
首先編譯serving下的example目錄得到
bazel-bin/tensorflow_serving/example/mnist_export.runfiles/org_tensorflow/tensorflow/contrib/session_bundle/manifest_pb2.py
隨后copy到python主目錄下的lib目錄 例如：/usr/lib/python2.7/site-packages/
如果還報相同錯誤
修改 /usr/lib/python2.7/site-packages/tensorflow/contrib/session_bundle/目錄下的exporter.py文件
刪除 from tensorflow.contrib.session_bundle import manifest_pb2
增加 import manifest_pb2
解決思路：主要就是PYTHONPATH中缺少manifest_pb2.py，所以需要設置，設置路徑可以找到manifest_pb2.py即可
manifest_pb2.py為tensorflow/contrib/session_bundle/manifest.proto 的protobuf生成文件，若有條件可手動生成
3.參考地址
tensorflow serving github : https://github.com/tensorflow/serving
https://github.com/tensorflow/serving/blob/master/tensorflow_serving/g3doc/setup.md
bazel http://www.bazel.io/ (需翻牆)
4.Serving Framework
4.1.Train
訓練模型的過程
4.2.exporter
負責將訓練好的模型導出
4.3.Sever
負責存儲操作，例如將對象存儲到磁盤
4.4.Server
提供grpc server,組織request調用Module，將結果response client
4.5.ModuleManager
負責加載訓練好的模型
4.6.Scheduler
負責請求的調度，例如BatchScheduler(buffer 某一批數據才發給Service)
4.7.client
負責發送Request請求接收Response
5.如何編寫Serving
5.1 export model
模型訓練完成后，需要export model
1) 需要確定 signature : (classification_signature,regression_signature,generic_signature)
classification_signature: input , classes , scores
regression_signature: input , output
generic_signature: map<string,tensor_name>
signature規定了輸入和輸出的tensor_name, 這個tensor_name應該對應到graph里的tensor
例如 mnist 為classification模型 訓練輸入了 x 訓練出 y 則在export的時候使用：
signature
=
exporter.classification_signature(input_tensor
=
x, scores_tensor
=
y)
|
5.2 確定輸出的路徑
導出model需要一個可存儲的路徑，這個路徑會在inference程序讀取時使用
5.3 編寫inference
inference主要流程：
1）獲得 SessionBundle
std::unique_ptr<SessionBundleFactory> bundle_factory;
TF_QCHECK_OK(
SessionBundleFactory::Create(session_bundle_config, &bundle_factory));
std::unique_ptr<SessionBundle> bundle(
new
SessionBundle);
TF_QCHECK_OK(bundle_factory->CreateSessionBundle(bundle_path, &bundle));
|
2) 提供輸入與輸出的tensor
Tensor input(tensorflow::DT_FLOAT, {1, kImageDataSize});
std::copy_n(request->image_data().begin(), kImageDataSize,
input.flat<
float
>().data());
std::vector<Tensor> outputs;
|
3) 通過signatrue傳入input,output到session_bundle並執行
const
tensorflow::Status status = bundle_->session->Run(
{{signature_.input().tensor_name(), input}},
{signature_.scores().tensor_name()}, {}, &outputs);
|