Hi @masahi :
I also test tvm-cutlass-eval /resnet50 /run_int8.py on my RTX 3070. However, I
get similar error:
Traceback (most recent call last):
File "run_int8.py", line 94, in
rt_mod, dev, num_partition = profile_and_build(mod, params, sm,
tmp_dir="../maskrcnn/tmp", lib_path="compil
I understand their purpose through your explanation
---
[Visit
Topic](https://discuss.tvm.apache.org/t/tvm-device-and-tvm-target-what-are-they-effects/12396/3)
to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click
here](https:
@Lyken17 I think your code is passing the function (mod['main']) instead of the
module (mod) to `relay.optimize`, though the arguments will still be binded if
you use module
(https://github.com/apache/tvm/blob/main/python/tvm/relay/build_module.py#L188).
Using:
```python
import tvm
import tv
@Ganler Just re-ran above code example and notice that the args passed in
`tvm.IRModule` rather than `Function`, thus the `bind_params_by_name` will not
be called when building the model
https://github.com/apache/tvm/blob/main/python/tvm/relay/build_module.py#L432-L434
Seems the folding is pe
I think generally **device** is **hardware (primitives)**. **(codegen) target**
is **software** that drives the **deivice**.
examples of dev v.s. target:
- NVIDIA GPU ~ CUDA (=> NVPTX)
- AMD GPU ~ ROCm
- CPU ~ OPENCL/C/C++ (compiler)
- PhD students ~ advisors
But if you look at
https://tvm.ap
@Ganler Thanks for the pointer. I see. I actually notice this function but
thought `bind_params_by_name` was only for binding without constant folding.
---
[Visit
Topic](https://discuss.tvm.apache.org/t/how-params-are-converted-and-stored-in-lib-get-params/12406/3)
to respond.
You are re
Looks like `SimplifyExpr` doesn't support folding `bias_add` and `bias`, see
https://github.com/apache/tvm/blob/6942b3660df3551a3a9a86c2faba834d366a2a7e/src/relay/transforms/simplify_expr.cc#L651-L652.
So both of cases don't work unless you modify that pass. But I recommend not
depending on `b
This is because `relay.build` optimizes the constants through constant folding.
That said, those constants you marked have been folded and simplified.
Actually, it is simplified in the `bind_params_by_name` function in
`relay.build`. And if you print the output module of that you see:
```
de
Interesting point. I agree that having a hierachy would make the IR more
readable. Perhaps the nested structure can be achieved in A-norm form and can
be flatten to graph-norm when we need to tune the model?
---
[Visit Topic](https://discuss.tvm.apache.org/t/hierarchy-in-tvm/12306/4) to
r
Either case should be possible with tvm transformation pass.
If you need to preserve `conv2d` and `biad_add`, an alternative way to achieve
this is to perform such transform in original models before exporting to TVM.
For example, I used to use this
[script](https://gist.github.com/Lyken17/de
Assume we have an expression constructed as follows
```python
w_bit = 32
x = relay.var("input", shape=[7], dtype=f"int{w_bit}")
w = relay.var("x2", shape=[2,], dtype=f"int{w_bit}")
zx = relay.var("x3", shape=[3, ], dtype=f"int{w_bit}")
zy = relay.var("x4", shape=[4, ], dtype=f"int{w_bit}")
e_sca
Hi @masahi , I am not quite clear regarding this bias_add and add op folding
that you mentioned.
So, what I intend to achieve and what I assume you are also implying above is
as follows:
case 1:
>**before:** conv2d -> bias_add -> add (shift from batchnorm) is transformed to:
>
> **after tr
Does TVM support multithreaded inference? That is to have each thread load a
precompiled .so into a module. Given the per thread modules, inference by the
set_input, run, and get_output pattern. Thanks!
---
[Visit Topic](https://discuss.tvm.apache.org/t/multithreaded-inference/12400/1)
to
13 matches
Mail list logo
