[Apache TVM Discuss] [Questions] [TensorRT] Seems ctx.sync() does not work while using TensorRT on Jetson Xavier NX

[Beom Woo Kang via Apache TVM Discuss]

Thanks for your replies!
I checked the result with the code like below and it seems the results are the 
same:

    #tvm_trt_compare.py

    ...

    mod, params = relay.frontend.from_pytorch(scripted_model, shape_list)

    tgt = tvm.target.cuda()
    ctx = tvm.gpu(0)

    ###Same Input
    data = np.random.uniform(-1, 1, size=input_shape).astype(dtype)


    ###GPU with TVM
    with tvm.transform.PassContext(opt_level=args.opt_level):
        g1, m1, p1 = relay.build(mod, tgt, params=params)

    module1 = graph_runtime.create(g1, m1, ctx)
    module1.set_input(**p1)
    module1.set_input("data", data)
    module1.run()
    out_tvm = module1.get_output(0).asnumpy()


    ###GPU with TensorRT
    from tvm.relay.op.contrib.tensorrt import partition_for_tensorrt
    mod, config = partition_for_tensorrt(mod, params)

    with tvm.transform.PassContext(opt_level=args.opt_level, 
config={'relay.ext.tensorrt.options': config}):
        g2, m2, p2 = relay.build(mod, tgt, params=params)

    module2 = graph_runtime.create(g2, m2, ctx)
    module2.set_input(**p2)
    module2.set_input("data",data)
    module2.run()
    out_trt = module2.get_output(0).asnumpy()


    if np.all( np.abs( out_tvm - out_trt ) < 1e-5):
        print("CLEAR")
    else:
        print("FAIL")



Also Ccing @trevor-m,

But what I don't understand is that applying opt_level=0~2 gives me "FAIL", 
while opt_level=3 gives me "CLEAR".

    $ python3 tvm_trt_compare.py --opt_level 0
    FAIL
    $ python3 tvm_trt_compare.py --opt_level 3
    CLEAR

Does TensorRT on TVM support only with opt_level=3?

Besides, I measured the time taken with the same code base:
    
    $ python3 tvm_trt_compare.py --opt_level 3
    GPU_TVM: 6.222900390625
    GPU_TRT: 6.257080078125
    CLEAR

Maybe it is because these two different runtime modules (module1 and module2) 
share the same context?





---
[Visit 
Topic](https://discuss.tvm.apache.org/t/tensorrt-seems-ctx-sync-does-not-work-while-using-tensorrt-on-jetson-xavier-nx/9579/5)
 to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click 
here](https://discuss.tvm.apache.org/email/unsubscribe/9571318365e8521b499caaf3a9fe22d9ee766dfdbd3732a80af71b1c159fb38a).