[Apache TVM Discuss] [Questions] [MultiThread][ThreadPool] Performance degradation when running relay module in multiple threads

2021-07-02 Thread Zhao Wu via Apache TVM Discuss
Do you mean if we use Python's multi process, you could get ideal result but use Python's multi thread, you get bad result? Or what else things you mean? --- [Visit Topic](https://discuss.tvm.apache.org/t/multithread-threadpool-performance-degradation-when-running-relay-module-in-multiple-

[Apache TVM Discuss] [Questions] [MultiThread][ThreadPool] Performance degradation when running relay module in multiple threads

2021-07-02 Thread Zhao Wu via Apache TVM Discuss
if we change the thread pool to OpenMP, do we have the same problem? --- [Visit Topic](https://discuss.tvm.apache.org/t/multithread-threadpool-performance-degradation-when-running-relay-module-in-multiple-threads/10374/6) to respond. You are receiving this because you enabled mailing list

[Apache TVM Discuss] [Questions] [MultiThread][ThreadPool] Performance degradation when running relay module in multiple threads

2021-07-02 Thread Zhao Wu via Apache TVM Discuss
[quote="MinminSun, post:1, topic:10374"] TVM_NUM_THREADS [/quote] How about setting one environment `TVM_THREAD_POOL_SPIN_COUNT = 0` , does it improve your case? --- [Visit Topic](https://discuss.tvm.apache.org/t/multithread-threadpool-performance-degradation-when-running-relay-module-in

[Apache TVM Discuss] [Questions] How to manually controll CPU affinity in multithreading scenario?

2021-07-02 Thread Zhao Wu via Apache TVM Discuss
TVM doesn't provide it currently. --- [Visit Topic](https://discuss.tvm.apache.org/t/how-to-manually-controll-cpu-affinity-in-multithreading-scenario/10348/4) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https:/

[Apache TVM Discuss] [Questions] Add new backend to TVM

2021-07-02 Thread Zhao Wu via Apache TVM Discuss
I assume you don't have your own NPU accelerate library, so you couldn't go TVM BYOC. Firstly, you should implement your own quantization algorithm based on your NPU (not all operation / data type could be provided on your NPU, like int64) Secondly, you should consider provide your own relay

[Apache TVM Discuss] [Questions] TFLite model with dynamic shapes

2021-05-17 Thread Zhao Wu via Apache TVM Discuss
I think we don't support it. --- [Visit Topic](https://discuss.tvm.apache.org/t/tflite-model-with-dynamic-shapes/10012/2) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubsc

[Apache TVM Discuss] [Questions] Do we have any way to process codegen with more fine grade control?

2021-05-06 Thread Zhao Wu via Apache TVM Discuss
Yeah, it is unfriendly for Ansor. However, I think it is not contradict. We could not expect we could generate asm like ACL, but we could expect we could achieve the same optimization. For example, your example is we can not do `register blocking` optimization easily, but we could expect we ha

[Apache TVM Discuss] [Questions] Do we have any way to process codegen with more fine grade control?

2021-05-06 Thread Zhao Wu via Apache TVM Discuss
When we want to do some advanced optimization like `register blocking` the goal you want to achieve , TVM codegen can not handle it very well. My experience is 1. write micro gemm like `4x4` or `8x8` and then tensorize 2. try, try and try different schedule and find one combination to match yo

[Apache TVM Discuss] [Application] Auto-scheduler seems slower on int8

2021-04-01 Thread Zhao Wu via Apache TVM Discuss
For the int8, there are intrinsic like DP4A to accelerate. Currently, auto scheduler doesn't support it. --- [Visit Topic](https://discuss.tvm.apache.org/t/auto-scheduler-seems-slower-on-int8/9585/2) to respond. You are receiving this because you enabled mailing list mode. To unsubscrib

[Apache TVM Discuss] [Questions] How can I get output name from graph runtime

2020-11-18 Thread Zhao Wu via Apache TVM Discuss
Good question. I think we can not get it directly currently. But the mechanism should have support it. For example, when you pass the output index, we could use `nodes_[outputs_[index].node_id].name` to get the name. Something a bit more, I think the name you want more is the model's name, the

[Apache TVM Discuss] [Questions] About document version

2020-11-15 Thread Zhao Wu via Apache TVM Discuss
I want this feature too and we should have this. One workaround is to pull the version of tvm code you want and generate the doc by yourself. @tqchen @comaniac --- [Visit Topic](https://discuss.tvm.apache.org/t/about-document-version/8459/2) to respond. You are receiving this because yo

[Apache TVM Discuss] [Questions] Do layer names get imported into Relay?

2020-09-25 Thread Zhao Wu via Apache TVM Discuss
I don't know why relay drop it, but i think @tqchen @jroesch maybe know more information and background. --- [Visit Topic](https://discuss.tvm.apache.org/t/do-layer-names-get-imported-into-relay/7986/4) to respond. You are receiving this because you enabled mailing list mode. To unsubsc

[Apache TVM Discuss] [Questions] Do layer names get imported into Relay?

2020-09-25 Thread Zhao Wu via Apache TVM Discuss
Previous IR (NNVM) has one name attribute but relay miss it. --- [Visit Topic](https://discuss.tvm.apache.org/t/do-layer-names-get-imported-into-relay/7986/2) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https:/

[Apache TVM Discuss] [Questions] How to run module with hexagon simulator "sim_dev"

2020-09-18 Thread Zhao Wu via Apache TVM Discuss
If i remember correctly, we still not support relay e2e on hexagon, right? @kparzysz --- [Visit Topic](https://discuss.tvm.apache.org/t/how-to-run-module-with-hexagon-simulator-sim-dev/7946/2) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from

[Apache TVM Discuss] [Questions] TVM opencl context how to choose device type as accelerator

2020-09-11 Thread Zhao Wu via Apache TVM Discuss
The problem is in ```cpp void Init(const std::string& type_key, const std::string& device_type, const std::string& platform_name = ""); virtual void Init() { Init("opencl", "gpu"); } // gpu is the device type ``` Inside `Init`, we will only try to match the `gpu` device type, if