TensorRTx aims to implement popular deep learning networks with tensorrt network definition APIs. As we know, tensorrt has builtin parsers, including caffeparser, uffparser, onnxparser, etc. But when we use these parsers, we often run into some "unsupported operations or layers" problems, especially some state-of-the-art models are using new type of layers.
So why don't we just skip all parsers? We just use TensorRT network definition APIs to build the whole network, it's not so complicated.
I wrote this project to get familiar with tensorrt API, and also to share and learn from the community.
All the models are implemented in pytorch/mxnet/tensorflown first, and export a weights file xxx.wts, and then use tensorrt to load weights, define network and do inference. Some pytorch implementations can be found in my repo Pytorchx, the remaining are from polular open-source implementations.
23 May 2021. SsisyphusTao: CenterNet DLA-34 with DCNv2 plugin.17 May 2021. ybw108: arcface LResNet100E-IR and MobileFaceNet.6 May 2021. makaveli10: scaled-yolov4 yolov4-csp.29 Apr 2021. upczww: hrnet segmentation w18/w32/w48, ocr branch also.28 Apr 2021. aditya-dl: mobilenetv2, alexnet, densenet121, mobilenetv3 with python API.26 Apr 2021. makaveli10 add Inceptionv4.25 Apr 2021. YOLOv5 updated to v5.0, supporting s/m/l/x/s6/m6/l6/x6.23 Apr 2021. irvingzhang0512 add TSM: Temporal Shift Module for Efficient Video Understanding, ICCV2019.23 Apr 2021. freedenS implement MaskRCNN, till now the MOST complicated model in this repo.16 Apr 2021. irvingzhang0512 implement lenet and resnet50 with Python API, freedenS implement FasterRCNN with five plugins, cheers!2 Apr 2021. mingyu6yang added a python wrapper for retinaface, makaveli10 added DenseNet-121.17 Mar 2021. wuzuowuyou added refinedet, which utilized libtorch to do postprocessing.5 Mar 2021. chgit0214 added the LPRNet.31 Jan 2021. RepVGG added by upczww.29 Jan 2021. U-Net added by YuzhouPeng.
- Install the dependencies.
- A guide for quickly getting started, taking lenet5 as a demo.
- The .wts file content format
- Frequently Asked Questions (FAQ)
- Migrating from TensorRT 4 to 7
- How to implement multi-GPU processing, taking YOLOv4 as example
- Check if Your GPU support FP16/INT8
- How to Compile and Run on Windows
- Deploy YOLOv4 with Triton Inference Server
- From pytorch to trt step by step, hrnet as example(Chinese)
- GTX1080 / Ubuntu16.04 / cuda10.0 / cudnn7.6.5 / tensorrt7.0.0 / nvinfer7.0.0 / opencv3.3
Each folder has a readme inside, which explains how to run the models inside.