Flow-Developers

Raspberry Pi, Raspbian, python, ROS, IoT, 電子工作, 機械学習, Deep learning, ライフハック

Build and Install onnx-tensorrt on JetsonXavierNX(Jetpack 4.4)

build and install onnx-tensorrt

f:id:flow-dev:20210213134706p:plain

environment

  • jetson xavier nx
  • jetpack4.4
uname -a
Linux jetson xaveir-nx 4.9.140-tegra #1 SMP PREEMPT Tue Oct 27 21:02:46 PDT 2020 aarch64 aarch64 aarch64 GNU/Linux

update cmake (3.13 Later for onnx-tensorrt)

  • Install what you need to build cmake.
sudo apt install libssl-dev libprotobuf-dev protobuf-compiler

# Was neccessary for my environment for $ ./bootstrap
  • Build and install the latest version of cmake

qiita.com

install onnx-tensorrt on jetosn xavier nx

  • Build onnx-tensorrt using the following steps
git clone --recursive -b 7.1  https://github.com/onnx/onnx-tensorrt.git  onnx_tensorrt
cd onnx-tensorrt
mkdir build && cd build
cmake .. -DTENSORRT_ROOT=/usr/src/tensorrt -DCMAKE_INSTALL_PREFIX=/usr/  
make -j8
sudo make install

www.codenong.com

※If you want to Build with Docker, please refer to here.

https://github.com/onnx/onnx-tensorrt/issues/385

convert model onnx to tensorrt

  • How to execute
onnx2trt my_model.onnx -o my_engine.trt

install onnxruntime 1.4.0

wget https://nvidia.box.com/shared/static/8sc6j25orjcpl6vhq3a4ir8v219fglng.whl -O onnxruntime_gpu-1.4.0-cp36-cp36m-linux_aarch64.whl
pip3 install onnxruntime_gpu-1.4.0-cp36-cp36m-linux_aarch64.whl

developer.nvidia.com

install onnx_tensorrt.backend

  • PATH to CUDA.
#Add a blank, then these 2 lines
# (letting your Nano know where CUDA is) to the bottom of the file.

sudo nano ~/.bashrc

export PATH=${PATH}:/usr/local/cuda/bin
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/cuda/lib64

source ~/.bashrc

# test nvcc path done
nvcc --version
  • In order to install "onnx_tensorrt.backend" on Jetson XavierNX, you need to change the following two files in onnx-tensorrt. The differences are as follows.
--- a/NvOnnxParser.h
+++ b/NvOnnxParser.h
@@ -31,6 +31,10 @@
 #define NV_ONNX_PARSER_MINOR 1
 #define NV_ONNX_PARSER_PATCH 0
 
+#ifndef TENSORRTAPI
+#define TENSORRTAPI
+#endif // TENSORRTAPI
+
 static const int NV_ONNX_PARSER_VERSION = ((NV_ONNX_PARSER_MAJOR * 10000) + (NV_ONNX_PARSER_MINOR * 100) + NV_ONNX_PARSER_PATCH);
 
 //! \typedef SubGraph_t
diff --git a/setup.py b/setup.py
index 8ffa543..d6244a3 100644
--- a/setup.py
+++ b/setup.py
@@ -59,10 +59,11 @@ EXTRA_COMPILE_ARGS =  [
     '-std=c++11',
     '-DUNIX',
     '-D__UNIX',
-    '-m64',
     '-fPIC',
     '-O2',
     '-w',
+    '-march=armv8-a+crypto',
+    '-mcpu=cortex-a57+crypto',
     '-fmessage-length=0',
     '-fno-strict-aliasing',
     '-D_FORTIFY_SOURCE=2',
  • After the above preparation, I was able to install it with the following command.
cd onnx-tensorrt
sudo apt install swig
pip3 install pycuda
sudo python3 setup.py install
  • In python3, if import onnx_tensorrt.backend as backend succeeds, install succeeds.
python3
>>import onnx_tensorrt.backend as backend

onnx->TensorRT変換するときに便利なサイト

  • TopK layer(2次元)や,Dobule型は2021/1現在,TensorRT変換できないので対処する記述が必要.

PyTorch documentation — PyTorch 1.10 documentation

  • onnx->TensorRT変換のErrがどこで起きてるか探しやすい.Ctrl+Fでlayer番号で検索できる.

Netron

  • Nvidiaのフォーラムに行けば大体似たようなErrに遭遇しているので,きっかけにはなる.

NVIDIA Developer Forums - NVIDIA Developer Forums

  • onnx-tensorrt/issuesも活発でよい質問多い.

github.com

  • onnxモデルをきれいにしてくれるので使ってもよいが,今回の場合は,綺麗にしすぎてTensorRT変換が出来なくなった.
  • --input-shapeで動的サイズのinputを,静的サイズに変更できるのは便利な時があるかもしれない.
  • Jetson XavierNXでonnx-tensorrtの環境を作った後だとうまくinstallできなかったので,別のPCで環境作った.

github.com