8000 TensorRT 8.6 change onnx to trt error · Issue #1 · zhudongwork/SAM_TensorRT · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

TensorRT 8.6 change onnx to trt error #1

New issue

8000 Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
avBuffer opened this issue Nov 3, 2023 · 5 comments
Open

TensorRT 8.6 change onnx to trt error #1

avBuffer opened this issue Nov 3, 2023 · 5 comments

Comments

@avBuffer
Copy link
avBuffer commented Nov 3, 2023

  1. /usr/local/TensorRT-8.6.1.6//bin/trtexec -- --saveEngine=./models/mobile_sam_encoder.engine --memPoolSize=workspace:10240

  2. TensorRT 8.6 change onnx to trt error log:
    &&&& RUNNING TensorRT.trtexec [TensorRT v8601] # /usr/local/TensorRT-8.6.1.6//bin/trtexec -- --saveEngine=./models/mobile_sam_encoder.engine --memPoolSize=workspace:10240
    [11/03/2023-16:35:20] [I] === Model Options ===
    [11/03/2023-16:35:20] [I] Format: ONNX
    [11/03/2023-16:35:20] [I] Model: ./models/mobile_sam_encoder.onnx
    [11/03/2023-16:35:20] [I] Output:
    [11/03/2023-16:35:20] [I] === Build Options ===
    [11/03/2023-16:35:20] [I] Max batch: explicit batch
    [11/03/2023-16:35:20] [I] Memory Pools: workspace: 10240 MiB, dlaSRAM: default, dlaLocalDRAM: default, dlaGlobalDRAM: default
    [11/03/2023-16:35:20] [I] minTiming: 1
    [11/03/2023-16:35:20] [I] avgTiming: 8
    [11/03/2023-16:35:20] [I] Precision: FP32
    [11/03/2023-16:35:20] [I] LayerPrecisions:
    [11/03/2023-16:35:20] [I] Layer Device Types:
    [11/03/2023-16:35:20] [I] Calibration:
    [11/03/2023-16:35:20] [I] Refit: Disabled
    [11/03/2023-16:35:20] [I] Version Compatible: Disabled
    [11/03/2023-16:35:20] [I] TensorRT runtime: full
    [11/03/2023-16:35:20] [I] Lean DLL Path:
    [11/03/2023-16:35:20] [I] Tempfile Controls: { in_memory: allow, temporary: allow }
    [11/03/2023-16:35:20] [I] Exclude Lean Runtime: Disabled
    [11/03/2023-16:35:20] [I] Sparsity: Disabled
    [11/03/2023-16:35:20] [I] Safe mode: Disabled
    [11/03/2023-16:35:20] [I] Build DLA standalone loadable: Disabled
    [11/03/2023-16:35:20] [I] Allow GPU fallback for DLA: Disabled
    [11/03/2023-16:35:20] [I] DirectIO mode: Disabled
    [11/03/2023-16:35:20] [I] Restricted mode: Disabled
    [11/03/2023-16:35:20] [I] Skip inference: Disabled
    [11/03/2023-16:35:20] [I] Save engine: ./models/mobile_sam_encoder.engine
    [11/03/2023-16:35:20] [I] Load engine:
    [11/03/2023-16:35:20] [I] Profiling verbosity: 0
    [11/03/2023-16:35:20] [I] Tactic sources: Using default tactic sources
    [11/03/2023-16:35:20] [I] timingCacheMode: local
    [11/03/2023-16:35:20] [I] timingCacheFile:
    [11/03/2023-16:35:20] [I] Heuristic: Disabled
    [11/03/2023-16:35:20] [I] Preview Features: Use default preview flags.
    [11/03/2023-16:35:20] [I] MaxAuxStreams: -1
    [11/03/2023-16:35:20] [I] BuilderOptimizationLevel: -1
    [11/03/2023-16:35:20] [I] Input(s)s format: fp32:CHW
    [11/03/2023-16:35:20] [I] Output(s)s format: fp32:CHW
    [11/03/2023-16:35:20] [I] Input build shapes: model
    [11/03/2023-16:35:20] [I] Input calibration shapes: model
    [11/03/2023-16:35:20] [I] === System Options ===
    [11/03/2023-16:35:20] [I] Device: 0
    [11/03/2023-16:35:20] [I] DLACore:
    [11/03/2023-16:35:20] [I] Plugins:
    [11/03/2023-16:35:20] [I] setPluginsToSerialize:
    [11/03/2023-16:35:20] [I] dynamicPlugins:
    [11/03/2023-16:35:20] [I] ignoreParsedPluginLibs: 0
    [11/03/2023-16:35:20] [I]
    [11/03/2023-16:35:20] [I] === Inference Options ===
    [11/03/2023-16:35:20] [I] Batch: Explicit
    [11/03/2023-16:35:20] [I] Input inference shapes: model
    [11/03/2023-16:35:20] [I] Iterations: 10
    [11/03/2023-16:35:20] [I] Duration: 3s (+ 200ms warm up)
    [11/03/2023-16:35:20] [I] Sleep time: 0ms
    [11/03/2023-16:35:20] [I] Idle time: 0ms
    [11/03/2023-16:35:20] [I] Inference Streams: 1
    [11/03/2023-16:35:20] [I] ExposeDMA: Disabled
    [11/03/2023-16:35:20] [I] Data transfers: Enabled
    [11/03/2023-16:35:20] [I] Spin-wait: Disabled
    [11/03/2023-16:35:20] [I] Multithreading: Disabled
    [11/03/2023-16:35:20] [I] CUDA Graph: Disabled
    [11/03/2023-16:35:20] [I] Separate profiling: Disabled
    [11/03/2023-16:35:20] [I] Time Deserialize: Disabled
    [11/03/2023-16:35:20] [I] Time Refit: Disabled
    [11/03/2023-16:35:20] [I] NVTX verbosity: 0
    [11/03/2023-16:35:20] [I] Persistent Cache Ratio: 0
    [11/03/2023-16:35:20] [I] Inputs:
    [11/03/2023-16:35:20] [I] === Reporting Options ===
    [11/03/2023-16:35:20] [I] Verbose: Disabled
    [11/03/2023-16:35:20] [I] Averages: 10 inferences
    [11/03/2023-16:35:20] [I] Percentiles: 90,95,99
    [11/03/2023-16:35:20] [I] Dump refittable layers:Disabled
    [11/03/2023-16:35:20] [I] Dump output: Disabled
    [11/03/2023-16:35:20] [I] Profile: Disabled
    [11/03/2023-16:35:20] [I] Export timing to JSON file:
    [11/03/2023-16:35:20] [I] Export output to JSON file:
    [11/03/2023-16:35:20] [I] Export profile to JSON file:
    [11/03/2023-16:35:20] [I]
    [11/03/2023-16:35:20] [I] === Device Information ===
    [11/03/2023-16:35:20] [I] Selected Device: NVIDIA GeForce GTX 1080
    [11/03/2023-16:35:20] [I] Compute Capability: 6.1
    [11/03/2023-16:35:20] [I] SMs: 20
    [11/03/2023-16:35:20] [I] Device Global Memory: 8112 MiB
    [11/03/2023-16:35:20] [I] Shared Memory per SM: 96 KiB
    [11/03/2023-16:35:20] [I] Memory Bus Width: 256 bits (ECC disabled)
    [11/03/2023-16:35:20] [I] Application Compute Clock Rate: 1.7715 GHz
    [11/03/2023-16:35:20] [I] Application Memory Clock Rate: 5.005 GHz
    [11/03/2023-16:35:20] [I]
    [11/03/2023-16:35:20] [I] Note: The application clock rates do not reflect the actual clock rates that the GPU is currently running at.
    [11/03/2023-16:35:20] [I]
    [11/03/2023-16:35:20] [I] TensorRT version: 8.6.1
    [11/03/2023-16:35:20] [I] Loading standard plugins
    [11/03/2023-16:35:21] [I] [TRT] [MemUsageChange] Init CUDA: CPU +196, GPU +0, now: CPU 204, GPU 5740 (MiB)
    [11/03/2023-16:35:22] [I] [TRT] [MemUsageChange] Init builder kernel library: CPU +7, GPU +0, now: CPU 230, GPU 5740 (MiB)
    [11/03/2023-16:35:22] [I] Start parsing network model.
    [11/03/2023-16:35:23] [I] [TRT] ----------------------------------------------------------------
    [11/03/2023-16:35:23] [I] [TRT] Input filename: ./models/mobile_sam_encoder.onnx
    [11/03/2023-16:35:23] [I] [TRT] ONNX IR version: 0.0.7
    [11/03/2023-16:35:23] [I] [TRT] Opset version: 12
    [11/03/2023-16:35:23] [I] [TRT] Producer name: pytorch
    [11/03/2023-16:35:23] [I] [TRT] Producer version: 1.11.0
    [11/03/2023-16:35:23] [I] [TRT] Domain:
    [11/03/2023-16:35:23] [I] [TRT] Model version: 0
    [11/03/2023-16:35:23] [I] [TRT] Doc string:
    [11/03/2023-16:35:23] [I] [TRT] ----------------------------------------------------------------
    [11/03/2023-16:35:23] [W] [TRT] onnx2trt_utils.cpp:369: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
    [11/03/2023-16:35:23] [W] [TRT] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
    [11/03/2023-16:35:23] [E] Error[4]: [shuffleNode.cpp::symbolicExecute::392] Error Code 4: Internal Error (Reshape_98: IShuffleLayer applied to shape tensor must have 0 or 1 reshape dimensions: dimensions were [-1,2])
    [11/03/2023-16:35:23] [E] [TRT] ModelImporter.cpp:773: While parsing node number 109 [Pad -> "onnx::Reshape_442"]:
    [11/03/2023-16:35:23] [E] [TRT] ModelImporter.cpp:774: --- Begin node ---
    [11/03/2023-16:35:23] [E] [TRT] ModelImporter.cpp:775: input: "x"
    input: "onnx::Pad_440"
    input: "onnx::Pad_441"
    output: "onnx::Reshape_442"
    name: "Pad_109"
    op_type: "Pad"
    attribute {
    name: "mode"
    s: "constant"
    type: STRING
    }

[11/03/2023-16:35:23] [E] [TRT] ModelImporter.cpp:776: --- End node ---
[11/03/2023-16:35:23] [E] [TRT] ModelImporter.cpp:779: ERROR: ModelImporter.cpp:180 In function parseGraph:
[6] Invalid Node - Pad_109
[shuffleNode.cpp::symbolicExecute::392] Error Code 4: Internal Error (Reshape_98: IShuffleLayer applied to shape tensor must have 0 or 1 reshape dimensions: dimensions were [-1,2])
[11/03/2023-16:35:23] [E] Failed to parse onnx file
[11/03/2023-16:35:23] [I] Finished parsing network model. Parse time: 0.133705
[11/03/2023-16:35:23] [E] Parsing model failed
[11/03/2023-16:35:23] [E] Failed to create engine from model or file.
[11/03/2023-16:35:23] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8601] # /usr/local/TensorRT-8.6.1.6//bin/trtexec -- --saveEngine=./models/mobile_sam_encoder.engine --memPoolSize=workspace:10240
@zhudongwork
Copy link
Owner

Please try TensorRT-8.5

@avBuffer
Copy link
Author
avBuffer commented Nov 6, 2023

1> Changing Ten 8000 sorRT-8.5 is also same error!
/usr/local/TensorRT-8.5.3.1/bin/trtexec -- --saveEngine=./models/mobile_sam_encoder.engine --memPoolSize=workspace:10240

2> error log:
&&&& RUNNING TensorRT.trtexec [TensorRT v8503] # /usr/local/TensorRT-8.5.3.1/bin/trtexec -- --saveEngine=./models/mobile_sam_encoder.engine --memPoolSize=workspace:10240
[11/06/2023-16:33:17] [I] === Model Options ===
[11/06/2023-16:33:17] [I] Format: ONNX
[11/06/2023-16:33:17] [I] Model: ./models/mobile_sam_encoder.onnx
[11/06/2023-16:33:17] [I] Output:
[11/06/2023-16:33:17] [I] === Build Options ===
[11/06/2023-16:33:17] [I] Max batch: explicit batch
[11/06/2023-16:33:17] [I] Memory Pools: workspace: 10240 MiB, dlaSRAM: default, dlaLocalDRAM: default, dlaGlobalDRAM: default
[11/06/2023-16:33:17] [I] minTiming: 1
[11/06/2023-16:33:17] [I] avgTiming: 8
[11/06/2023-16:33:17] [I] Precision: FP32
[11/06/2023-16:33:17] [I] LayerPrecisions:
[11/06/2023-16:33:17] [I] Calibration:
[11/06/2023-16:33:17] [I] Refit: Disabled
[11/06/2023-16:33:17] [I] Sparsity: Disabled
[11/06/2023-16:33:17] [I] Safe mode: Disabled
[11/06/2023-16:33:17] [I] DirectIO mode: Disabled
[11/06/2023-16:33:17] [I] Restricted mode: Disabled
[11/06/2023-16:33:17] [I] Build only: Disabled
[11/06/2023-16:33:17] [I] Save engine: ./models/mobile_sam_encoder.engine
[11/06/2023-16:33:17] [I] Load engine:
[11/06/2023-16:33:17] [I] Profiling verbosity: 0
[11/06/2023-16:33:17] [I] Tactic sources: Using default tactic sources
[11/06/2023-16:33:17] [I] timingCacheMode: local
[11/06/2023-16:33:17] [I] timingCacheFile:
[11/06/2023-16:33:17] [I] Heuristic: Disabled
[11/06/2023-16:33:17] [I] Preview Features: Use default preview flags.
[11/06/2023-16:33:17] [I] Input(s)s format: fp32:CHW
[11/06/2023-16:33:17] [I] Output(s)s format: fp32:CHW
[11/06/2023-16:33:17] [I] Input build shapes: model
[11/06/2023-16:33:17] [I] Input calibration shapes: model
[11/06/2023-16:33:17] [I] === System Options ===
[11/06/2023-16:33:17] [I] Device: 0
[11/06/2023-16:33:17] [I] DLACore:
[11/06/2023-16:33:17] [I] Plugins:
[11/06/2023-16:33:17] [I] === Inference Options ===
[11/06/2023-16:33:17] [I] Batch: Explicit
[11/06/2023-16:33:17] [I] Input inference shapes: model
[11/06/2023-16:33:17] [I] Iterations: 10
[11/06/2023-16:33:17] [I] Duration: 3s (+ 200ms warm up)
[11/06/2023-16:33:17] [I] Sleep time: 0ms
[11/06/2023-16:33:17] [I] Idle time: 0ms
[11/06/2023-16:33:17] [I] Streams: 1
[11/06/2023-16:33:17] [I] ExposeDMA: Disabled
[11/06/2023-16:33:17] [I] Data transfers: Enabled
[11/06/2023-16:33:17] [I] Spin-wait: Disabled
[11/06/2023-16:33:17] [I] Multithreading: Disabled
[11/06/2023-16:33:17] [I] CUDA Graph: Disabled
[11/06/2023-16:33:17] [I] Separate profiling: Disabled
[11/06/2023-16:33:17] [I] Time Deserialize: Disabled
[11/06/2023-16:33:17] [I] Time Refit: Disabled
[11/06/2023-16:33:17] [I] NVTX verbosity: 0
[11/06/2023-16:33:17] [I] Persistent Cache Ratio: 0
[11/06/2023-16:33:17] [I] Inputs:
[11/06/2023-16:33:17] [I] === Reporting Options ===
[11/06/2023-16:33:17] [I] Verbose: Disabled
[11/06/2023-16:33:17] [I] Averages: 10 inferences
[11/06/2023-16:33:17] [I] Percentiles: 90,95,99
[11/06/2023-16:33:17] [I] Dump refittable layers:Disabled
[11/06/2023-16:33:17] [I] Dump output: Disabled
[11/06/2023-16:33:17] [I] Profile: Disabled
[11/06/2023-16:33:17] [I] Export timing to JSON file:
[11/06/2023-16:33:17] [I] Export output to JSON file:
[11/06/2023-16:33:17] [I] Export profile to JSON file:
[11/06/2023-16:33:17] [I]
[11/06/2023-16:33:17] [I] === Device Information ===
[11/06/2023-16:33:17] [I] Selected Device: NVIDIA GeForce GTX 1080
[11/06/2023-16:33:17] [I] Compute Capability: 6.1
[11/06/2023-16:33:17] [I] SMs: 20
[11/06/2023-16:33:17] [I] Compute Clock Rate: 1.7715 GHz
[11/06/2023-16:33:17] [I] Device Global Memory: 8112 MiB
[11/06/2023-16:33:17] [I] Shared Memory per SM: 96 KiB
[11/06/2023-16:33:17] [I] Memory Bus Width: 256 bits (ECC disabled)
[11/06/2023-16:33:17] [I] Memory Clock Rate: 5.005 GHz
[11/06/2023-16:33:17] [I]
[11/06/2023-16:33:17] [I] TensorRT version: 8.5.3
[11/06/2023-16:33:18] [I] [TRT] [MemUsageChange] Init CUDA: CPU +196, GPU +0, now: CPU 204, GPU 904 (MiB)
[11/06/2023-16:33:19] [I] [TRT] [MemUsageChange] Init builder kernel library: CPU +7, GPU +0, now: CPU 230, GPU 895 (MiB)
[11/06/2023-16:33:19] [I] Start parsing network model
[11/06/2023-16:33:19] [I] [TRT] ----------------------------------------------------------------
[11/06/2023-16:33:19] [I] [TRT] Input filename: ./models/mobile_sam_encoder.onnx
[11/06/2023-16:33:19] [I] [TRT] ONNX IR version: 0.0.7
[11/06/2023-16:33:19] [I] [TRT] Opset version: 12
[11/06/2023-16:33:19] [I] [TRT] Producer name: pytorch
[11/06/2023-16:33:19] [I] [TRT] Producer version: 1.11.0
[11/06/2023-16:33:19] [I] [TRT] Domain:
[11/06/2023-16:33:19] [I] [TRT] Model version: 0
[11/06/2023-16:33:19] [I] [TRT] Doc string:
[11/06/2023-16:33:19] [I] [TRT] ----------------------------------------------------------------
[11/06/2023-16:33:19] [W] [TRT] onnx2trt_utils.cpp:369: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[11/06/2023-16:33:19] [W] [TRT] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[11/06/2023-16:33:19] [E] Error[4]: [shuffleNode.cpp::symbolicExecute::392] Error Code 4: Internal Error (Reshape_98: IShuffleLayer applied to shape tensor must have 0 or 1 reshape dimensions: dimensions were [-1,2])
[11/06/2023-16:33:19] [E] [TRT] ModelImporter.cpp:773: While parsing node number 109 [Pad -> "onnx::Reshape_442"]:
[11/06/2023-16:33:19] [E] [TRT] ModelImporter.cpp:774: --- Begin node ---
[11/06/2023-16:33:19] [E] [TRT] ModelImporter.cpp:775: input: "x"
input: "onnx::Pad_440"
input: "onnx::Pad_441"
output: "onnx::Reshape_442"
name: "Pad_109"
op_type: "Pad"
attribute {
name: "mode"
s: "constant"
type: STRING
}

[11/06/2023-16:33:19] [E] [TRT] ModelImporter.cpp:776: --- End node ---
[11/06/2023-16:33:19] [E] [TRT] ModelImporter.cpp:779: ERROR: ModelImporter.cpp:180 In function parseGraph:
[6] Invalid Node - Pad_109
[shuffleNode.cpp::symbolicExecute::392] Error Code 4: Internal Error (Reshape_98: IShuffleLayer applied to shape tensor must have 0 or 1 reshape dimensions: dimensions were [-1,2])
[11/06/2023-16:33:19] [E] Failed to parse onnx file
[11/06/2023-16:33:19] [I] Finish parsing network model
[11/06/2023-16:33:19] [E] Parsing model failed
[11/06/2023-16:33:19] [E] Failed to create engine from model or file.
[11/06/2023-16:33:19] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8503] # /usr/local/TensorRT-8.5.3.1/bin/trtexec -- --saveEngine=./models/mobile_sam_encoder.engine --memPoolSize=workspace:10240

@zhudongwork
Copy link
Owner

This is confusing, I tried it on both TensorRT-8.6.1.6 and 8.5.3.1 and it works fine. This may be an environmental problem or hardware issue with your local tensorrt.

@dijiupianhai9
Copy link

Hello, have you solved this problem？

@qingwang-usc
Copy link

Maybe you can try const-folding your ONNX file: https://github.com/NVIDIA/TensorRT/tree/main/tools/Polygraphy/examples/cli/surgeon/02_folding_constants ?
@dijiupianhai9 @avBuffer

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@avBuffer @zhudongwork @qingwang-usc @dijiupianhai9
0