How to convert warp network

[GrigorySamokhin]

Hello! i have encountered following issue:

==================== warp_network start ====================
[01/16/2025-16:38:26] [TRT] [I] [MemUsageChange] Init CUDA: CPU +14, GPU +0, now: CPU 110, GPU 934 (MiB)
[01/16/2025-16:38:29] [TRT] [I] [MemUsageChange] Init builder kernel library: CPU +2101, GPU +384, now: CPU 2367, GPU 1318 (MiB)
[01/16/2025-16:38:29] [TRT] [E] IPluginRegistry::loadLibrary: Error Code 3: API Usage Error (SymbolAddress for getCreators could not be loaded, check function name against library symbol)
[01/16/2025-16:38:29] [TRT] [I] ----------------------------------------------------------------
[01/16/2025-16:38:29] [TRT] [I] Input filename:   ./checkpoints/ditto_onnx/warp_network.onnx
[01/16/2025-16:38:29] [TRT] [I] ONNX IR version:  0.0.10
[01/16/2025-16:38:29] [TRT] [I] Opset version:    17
[01/16/2025-16:38:29] [TRT] [I] Producer name:    pytorch
[01/16/2025-16:38:29] [TRT] [I] Producer version: 2.4.0
[01/16/2025-16:38:29] [TRT] [I] Domain:           
[01/16/2025-16:38:29] [TRT] [I] Model version:    0
[01/16/2025-16:38:29] [TRT] [I] Doc string:       
[01/16/2025-16:38:29] [TRT] [I] ----------------------------------------------------------------
[01/16/2025-16:38:29] [TRT] [I] No checker registered for op: GridSample3D. Attempting to check as plugin.
[01/16/2025-16:38:29] [TRT] [E] IPluginRegistry::getCreator: Error Code 4: API Usage Error (Cannot find plugin: GridSample3D, version: 1, namespace:.)
[01/16/2025-16:38:29] [TRT] [I] No checker registered for op: GridSample3D. Attempting to check as plugin.
[01/16/2025-16:38:29] [TRT] [E] IPluginRegistry::getCreator: Error Code 4: API Usage Error (Cannot find plugin: GridSample3D, version: 1, namespace:.)
[01/16/2025-16:38:29] [TRT] [E] ModelImporter.cpp:949: While parsing node number 11 [GridSample3D -> "/dense_motion_network/GridSample_output_0"]:
[01/16/2025-16:38:29] [TRT] [E] ModelImporter.cpp:950: --- Begin node ---
input: "/dense_motion_network/Reshape_3_output_0"
input: "/dense_motion_network/Reshape_4_output_0"
output: "/dense_motion_network/GridSample_output_0"
name: "/dense_motion_network/GridSample"
op_type: "GridSample3D"
attribute {
  name: "name"
  s: "GridSample3D"
  type: STRING
}
attribute {
  name: "version"
  i: 1
  type: INT
}
attribute {
  name: "namespace"
  s: ""
  type: STRING
}

[01/16/2025-16:38:29] [TRT] [E] ModelImporter.cpp:951: --- End node ---
[01/16/2025-16:38:29] [TRT] [E] ModelImporter.cpp:954: ERROR: onnxOpCheckers.cpp:781 In function checkFallbackPluginImporter:
[6] creator && "Plugin not found, are the plugin name, version, and namespace correct?"
[01/16/2025-16:38:29] [TRT] [E] ModelImporter.cpp:949: While parsing node number 77 [GridSample3D -> "/GridSample_output_0"]:
[01/16/2025-16:38:29] [TRT] [E] ModelImporter.cpp:950: --- Begin node ---
input: "feature_3d"
input: "/dense_motion_network/Transpose_1_output_0"
output: "/GridSample_output_0"
name: "/GridSample"
op_type: "GridSample3D"
attribute {
  name: "name"
  s: "GridSample3D"
  type: STRING
}
attribute {
  name: "version"
  i: 1
  type: INT
}
attribute {
  name: "namespace"
  s: ""
  type: STRING
}

[01/16/2025-16:38:29] [TRT] [E] ModelImporter.cpp:951: --- End node ---
[01/16/2025-16:38:29] [TRT] [E] ModelImporter.cpp:954: ERROR: onnxOpCheckers.cpp:781 In function checkFallbackPluginImporter:
[6] creator && "Plugin not found, are the plugin name, version, and namespace correct?"
Fail parsing ./checkpoints/ditto_onnx/warp_network.onnx
In node 11 with name: /dense_motion_network/GridSample and operator: GridSample3D (checkFallbackPluginImporter): INVALID_NODE: creator && "Plugin not found, are the plugin name, version, and namespace correct?"
error.code() = <ErrorCode.INVALID_NODE: 6>
error.file() = 'onnxOpCheckers.cpp'
error.func() = 'checkFallbackPluginImporter'
error.line() = 781
error.local_function_stack_size() = 0
error.local_function_stack() = []
error.node_name() = '/dense_motion_network/GridSample'
error.node_operator() = 'GridSample3D'
error.node() = 11
In node 77 with name: /GridSample and operator: GridSample3D (checkFallbackPluginImporter): INVALID_NODE: creator && "Plugin not found, are the plugin name, version, and namespace correct?"
error.code() = <ErrorCode.INVALID_NODE: 6>
error.file() = 'onnxOpCheckers.cpp'
error.func() = 'checkFallbackPluginImporter'
error.line() = 781
error.local_function_stack_size() = 0
error.local_function_stack() = []
error.node_name() = '/GridSample'
error.node_operator() = 'GridSample3D'
error.node() = 77
[01/16/2025-16:38:29] [TRT] [W] profileSharing0806 is on by default in TensorRT 10.0. This flag is deprecated and has no effect.
[01/16/2025-16:38:29] [TRT] [I] Lowering maximum device shared memory from 102400 to 49152 because of enabling hardware forward compatibility
[01/16/2025-16:38:29] [TRT] [E] IBuilder::buildSerializedNetwork: Error Code 4: API Usage Error (Network must have at least one output.)
==================== warp_network done ====================

[BlueSkyyyyyy]

I encountered the same problem

[BlueSkyyyyyy]

it seems to be the invalid plugin GridSample3D

[digital-avatar]

@GrigorySamokhin and @BlueSkyyyyyy
It looks like the libgrid_sample_3d_plugin.so plugin cannot be loaded properly. Can you show the running environment?

You can also refer to this repository to compile your own plugin and provide it to the conversion script.

[BlueSkyyyyyy]

@GrigorySamokhin and @BlueSkyyyyyy It looks like the libgrid_sample_3d_plugin.so plugin cannot be loaded properly. Can you show the running environment?

You can also refer to this repository to compile your own plugin and provide it to the conversion script.

Thanks for your reply! I run on the base images tensorrt_llm/release. OS is Ubuntu 22.04.4. A100 GPU.
I follow https://github.com/SeanWangJS/grid-sample3d-trt-plugin build libgrid_sample_3d_plugin.so successfully. But I encounter the problem "Segmentation fault (core dumped)" when step in res = parser.parse_from_file

ditto-talkinghead/scripts/cvt_onnx_to_trt.py

Line 46 in 8041450

res = parser.parse_from_file(onnx_path) # parse from file

.

Could you provide the docker images, that is maybe a easier way for users. I try "conda env create -f environment.yaml" but fail.

Thanks again for your patient!

[digital-avatar]

@BlueSkyyyyyy
Sorry for the late reply, saw here that you have solved the issue.

[BlueSkyyyyyy]

@digital-avatar yes，tensorrt environment is not easy to install， maybe because of its poor version compatibility