[Apache TVM Discuss] [Questions] [TensorRT integration error]tvm._ffi.base.TVMError: TVMError: Driver error:

[Lulin via Apache TVM Discuss]

Hello,

I try to use relay TensorRT integration to accelerate the tensorflow inference, 
reference these [relay TensorRT 
integration](https://tvm.apache.org/docs/deploy/tensorrt.html) tutorials. When 
I run the program, the following error occurs:

```
Traceback (most recent call last):
  File "test_tvm.py", line 220, in <module>
    test(arg.checkpoint_dir, arg.style_name, arg.test_dir, 
arg.if_adjust_brightness)
  File "test_tvm.py", line 127, in test
    gen_module.run(data=tvm.nd.array(x.astype(dtype)))
  File "/home/lulin/work/tvm/python/tvm/contrib/graph_runtime.py", line 206, in 
run
    self._run()
  File "/home/lulin/work/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 237, 
in __call__
    raise get_last_ffi_error()
tvm._ffi.base.TVMError: TVMError: Driver error:
```

The error program as followes:

```
    x = np.asarray(load_test_data(test_img, img_size))

    shape_dict = {"generator_input": x.shape}
    mod, params = relay.frontend.from_tensorflow(graph_def, layout=layout, 
shape=shape_dict)

    with tvm.transform.PassContext(opt_level=3, 
config={'relay.ext.tensorrt.options': config}):
        lib = relay.build(mod, target=target, params=params)
    lib.export_library('compiled.so')

    dtype = "float32"
    ctx = tvm.gpu(0)
    loaded_lib = tvm.runtime.load_module('compiled.so')
    gen_module = 
tvm.contrib.graph_runtime.GraphModule(loaded_lib['default'](ctx))

    # gen_module.set_input("generator_input", tvm.nd.array(x.astype(dtype)))

    gen_module.run(data=tvm.nd.array(x.astype(dtype)))
    tvm_output = gen_module.get_output(0, tvm.nd.empty(x.shape, "float32"))
```
I tried not use TensorRT integration to compile the tensorflow models, refer to 
this links [compile tensorflow 
model](https://tvm.apache.org/docs/tutorials/frontend/from_tensorflow.html#sphx-glr-tutorials-frontend-from-tensorflow-py),
 and it works fine. here is the program without using TensorRT integration:
```
    x = np.asarray(load_test_data(test_img, img_size))

    shape_dict = {"generator_input": x.shape}
    mod, params = relay.frontend.from_tensorflow(graph_def, layout=layout, 
shape=shape_dict)

    with tvm.transform.PassContext(opt_level=3):
        lib = relay.build(mod, target=target, target_host=target_host, 
params=params)

    dtype = "float32"
    m = graph_runtime.GraphModule(lib["default"](ctx))

    # set inputs
    m.set_input("generator_input", tvm.nd.array(x.astype(dtype)))

    m.run()

    # get outputs
    tvm_output = m.get_output(0, tvm.nd.empty(x.shape, "float32"))

    predictions = tvm_output.asnumpy()
```
My environment:

1. Ubuntu 16.04
2. Python 3.7.10
3. TensorFlow-gpu 1.15
4. TensorRT-7.2.3.4
5. CUDA 11.0 with cudnn 8.1.0

I follow the NVIDIA official doc to install TensorRT by .tar file, and also add 
the path to the `~/.bashrc`.

The way build TVM with TensorRT is following the official documentation, which 
is modify the config.cmake file, and then build.

```
set(USE_TENSORRT_CODEGEN ON)
set(USE_TENSORRT_RUNTIME /home/XXX/TensorRT-7.2.3.4)
```


Could this error caused by the TensorRT version issue? Anyone have some ideas 
about this error?

Deepest thanks for your reply!





---
[Visit 
Topic](https://discuss.tvm.apache.org/t/tensorrt-integration-error-tvm-ffi-base-tvmerror-tvmerror-driver-error/9593/1)
 to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click 
here](https://discuss.tvm.apache.org/email/unsubscribe/34dfb14666eed8f5629cb42039c386d9536a2036d1a582774761c30a59962110).