--- 如何安裝
https://docs.nvidia.com/deeplearning/sdk/tensorrt-install-guide/index.html#installing
tensorrt安裝后的so如下圖所示.所以一台機器只能有一個版本的tensorrt.
TensorRT是什么
建議先看看這篇https://zhuanlan.zhihu.com/p/35657027
深度學習
- 訓練
- 部署
平常自學深度學習的時候關注的更多是訓練的部分,即得到一個模型.而實際工作很大一塊的工作內容集中於如何將模型部署到具體的芯片上.你自己寫的模型效果是很難優於成熟的知名的模型的.
以無人駕駛為例,拍攝到圖片后,芯片上的加載的模型要能夠識別出圖片里是什么.對自動駕駛這種場景而言,對實時性地要求是非常高的.試想,從圖片輸入到模型,到模型識別出圖片中前方有個人花了1分鍾,你正以100km/h行駛,后果自然是災難性的.
這就引出了推理引擎.model里的信息其實就是一些權重矩陣信息而已.輸入圖片數據后,進行一大堆的矩陣運算,得到最終圖片的分類.推理引擎干的事情就是優化矩陣運算,縮短運算時間.
CUDA cuDNN
https://blog.csdn.net/u014380165/article/details/77340765
- CUDA是NVIDIA推出的用於自家GPU的並行計算框架，也就是說CUDA只能在NVIDIA的GPU上運行，而且只有當要解決的計算問題是可以大量並行計算的時候才能發揮CUDA的作用
- cuDNN（CUDA Deep Neural Network library）：是NVIDIA打造的針對深度神經網絡的加速庫，是一個用於深層神經網絡的GPU加速庫。如果你要用GPU訓練模型，cuDNN不是必須的，但是一般會采用這個加速庫
查看TensorRT版本
dpkg -l | grep TensorRT
convert_to_uff安裝在哪里?
使用TensorRT推導SSD網絡
以下是TensorRT中自帶的一個例子
The sampleUffSSD example is based on the following paper, SSD: Single Shot MultiBox
Detector (https://arxiv.org/abs/1512.02325). The SSD network performs the
task of object detection and localization in a single forward pass of the network.
The tensorflow SSD network was trained on the InceptionV2 architecture using
the MSCOCO dataset.
The sample makes use of TensorRT plugins to run the SSD network. To use these
plugins the TensorFlow graph needs to be preprocessed.
Steps to generate UFF file:
0. Make sure you have the UFF converter installed. For installation instructions, see:
https://docs.nvidia.com/deeplearning/sdk/tensorrt-api/#python and click on the 'TensorRT Python API' link.
1. Get the pre-trained Tensorflow model (ssd_inception_v2_coco) from:
http://download.tensorflow.org/models/object_detection/ssd_inception_v2_coco_2017_11_17.tar.gz
2. Call the UFF converter with the preprocessing flag set (-p [config_file]).
The config.py script specifies the preprocessing operations necessary for SSD TF graph.
It must be copied to the working directory for the file to be imported properly.
The plugin nodes and plugin parameters used in config.py should match the registered plugins
in TensorRT. Please read the plugins documentation for more details.
'convert-to-uff --input-file frozen_inference_graph.pb -O NMS -p config.py'
This script saves the converted .uff file in the same directory as the input with
the name frozen_inference_graph.pb.uff. Copy this converted .uff file to the
data directory as sample_ssd_relu6.uff <TensorRT Install>/data/ssd/sample_ssd_relu6.uff
The sample also requires a labels .txt file with a list of all labels used to
train the model. Current example for this network is <TensorRT Install>/data/ssd/ssd_coco_labels.txt
Steps to run the network:
1. To run the network in FP32 mode, ./sample_uff_ssd
2. To run the network in INT8 mode, ./sample_uff_ssd --int8
To run the network in INT8 mode, refer to BatchStreamPPM.h for details on how
calibration can be performed. Currently we require a file (list.txt) with
a list of all PPM images for calibration in the <TensorRT Install>/data/ssd/ folder.
The PPM images to be used for calibration can also reside in the same folder.
NOTE - There might be some precision loss when running the network in INT8
mode causing some objects to go undetected. Our general observation is that
\>500 images is a good number for calibration purposes.
Python API
- Graph Surgeon API
- UFF API
Included within the Python API is the UFF API; a package that contains a set of utilities to convert trained models from various frameworks to a common format.
The UFF API is located in uff/uff.html and contains two conversion type tool classes called Tensorflow Modelstream to UFF and Tensorflow Frozen Protobuf Model to UFF.
TensorRT的python_api地址:https://docs.nvidia.com/deeplearning/sdk/tensorrt-api/python_api/index.html
官方的readme是有問題的.
'convert-to-uff --input-file frozen_inference_graph.pb -O NMS -p config.py'
**開始排坑之旅! **
convert-to-uff並不是一個二進制文件,而是一個.py腳本. 位置在/usr/lib/python2.7/dist-packages/uff/bin/convert_to_uff.py,
/usr/lib/python3.6/dist-packages/uff/bin
具體參見這個https://devtalk.nvidia.com/default/topic/1025246/jetson-tx2/where-is-convert-to-uff/2
以及下面link中'3.2.4. Importing From TensorFlow Using Python'
https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#python_samples_section
運行腳本,提示
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/uff/bin/convert_to_uff.py", line 15, in <module>
import uff
File "/usr/lib/python2.7/dist-packages/uff/__init__.py", line 1, in <module>
from uff import converters, model # noqa
File "/usr/lib/python2.7/dist-packages/uff/model/__init__.py", line 1, in <module>
from . import uff_pb2 as uff_pb # noqa
File "/usr/lib/python2.7/dist-packages/uff/model/uff_pb2.py", line 6, in <module>
from google.protobuf.internal import enum_type_wrapper
ImportError: No module named google.protobuf.internal
安裝protobuf.ubuntu18.04默認是沒安裝pip的.要先裝pip.
- sudo apt update
- sudo apt install python-pip
- pip install protobuf
再次執行腳本,報錯
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/uff/bin/convert_to_uff.py", line 15, in <module>
import uff
File "/usr/lib/python2.7/dist-packages/uff/__init__.py", line 2, in <module>
from uff.converters.tensorflow.conversion_helpers import from_tensorflow # noqa
File "/usr/lib/python2.7/dist-packages/uff/converters/tensorflow/conversion_helpers.py", line 12, in <module>
from .converter_functions import * # noqa
File "/usr/lib/python2.7/dist-packages/uff/converters/tensorflow/converter_functions.py", line 12, in <module>
from uff.converters.tensorflow.converter import TensorFlowToUFFConverter as tf2uff
File "/usr/lib/python2.7/dist-packages/uff/converters/tensorflow/converter.py", line 24, in <module>
https://www.tensorflow.org/install/""".format(err))
ImportError: ERROR: Failed to import module (No module named tensorflow)
Please make sure you have TensorFlow installed.
安裝tensorflow
pip install tensorflow-gpu
Collecting tensorflow-gpu
Could not find a version that satisfies the requirement tensorflow-gpu (from versions: )
No matching distribution found for tensorflow-gpu
https://www.tensorflow.org/install/pip?lang=python2
嘗試用以下方式安裝
pip install --upgrade https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.12.0-cp27-none-linux_x86_64.whl
再次報錯!
tensorflow_gpu-1.12.0-cp27-none-linux_x86_64.whl is not a supported wheel on this platform.
nvidia官方提供了官方說明,如何install TensorFlow for Jetson Platform
https://docs.nvidia.com/deeplearning/dgx/install-tf-xavier/index.html
- sudo apt-get install libhdf5-serial-dev hdf5-tools
- sudo apt-get install python3-pip
- pip3 install -U pip
- sudo apt-get install zlib1g-dev zip libjpeg8-dev libhdf5-dev
- sudo pip3 install -U numpy grpcio absl-py py-cpuinfo psutil portpicker grpcio six mock requests gast h5py astor termcolor
又報錯!!!
nano@nano-desktop:/usr/src/tensorrt/samples/sampleUffSSD$ sudo pip3 install -U numpy grpcio absl-py py-cpuinfo psutil portpicker grpcio six mock requests gast h5py astor termcolor
Traceback (most recent call last):
File "/usr/bin/pip3", line 9, in <module>
from pip import main
ImportError: cannot import name 'main'
搜索了一大圈,說啥的都有.應該是pip本身的bug.
https://stackoverflow.com/questions/54420766/python3-6-importerror-cannot-import-name-main-after-upgrading-pip-from-8-1-1
將pip3 install替換成python3 -m pip install.
或者按照有的文章里說的比如這篇https://blog.csdn.net/zong596568821xp/article/details/80410416去修改試一下/usr/bin/pip3.
- sudo python3 -m pip install -U numpy grpcio absl-py py-cpuinfo psutil portpicker grpcio six mock requests gast h5py astor termcolor
這一步要耐心的多嘗試幾次,動不動就http連接失敗,即便我已經用代理翻牆的情況下也是如此.
查看包安裝到了什么位置:pip show命令
這里自己打開那個http地址看一下.官方文檔里的版本1.12.0不存在.
- pip3 install --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v42 tensorflow-gpu==1.13.1+nv19.3
等待一段時間.會自動卸載掉之前安裝的版本過低的protobuf,重新安裝符合要求的protobuf.
Successfully installed keras-applications-1.0.7 keras-preprocessing-1.0.9 markdown-3.1 protobuf-3.7.1 tensorboard-1.13.1 tensorflow-estimator-1.13.0 tensorflow-gpu-1.13.1+nv19.3 werkzeug-0.15.2
把.pb格式的模型轉換為uff格式(TensorRT可以識別的格式)
python3 /usr/lib/python3.6/dist-packages/uff/bin/convert_to_uff.py --input-file ./ssd_inception_v2_coco_2017_11_17/frozen_inference_graph.pb -O NMS -p config.py
運行ssd模型
#拷貝uff模型文件到data/ssd下
cp ./ssd_inception_v2_coco_2017_11_17/frozen_inference_graph.uff ~/tensorrt/data/ssd/sample_ssd_relu6.uff
#去tensort/bin下執行sample_uff_ssd程序
~/tensorrt/bin$ ./sample_uff_ssd
坑又來啦!!!
nano@nano-desktop:~/tensorrt/bin$ ./sample_uff_ssd
../data/ssd/sample_ssd_relu6.uff
Begin parsing model...
End parsing model...
Begin building engine...
Killed
一番搜索,https://devtalk.nvidia.com/default/topic/1046206/tensorrt5-sample_uff_ssd-can-t-run/ 有說官方readme里的link下載的model不對.
https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md
tensorflow官方提供的pretrained-model.
換個模型重新做一次轉換.
這里嘗試了2個不同的模型,均來自tensorflow官方github.一個在轉uff格式時失敗,一個轉換后解析模型時失敗 to do.
換回了最初的模型,嘗試成功了,原因不明.........
--- update 20200306
https://s0docs0nvidia0com.icopy.site/deeplearning/sdk/tensorrt-install-guide/index.html
