[Apache TVM Discuss] [Development/RFC] [RFC][Tensorcore] INT4 end-to-end inference

2021-04-13 Thread zachzzc via Apache TVM Discuss
No problem. I think the GPU you ran on before doesn't have Tensor Core so the TVM doesn't find the corresponding schedule to use. --- [Visit Topic](https://discuss.tvm.apache.org/t/rfc-tensorcore-int4-end-to-end-inference/7534/22) to respond. You are receiving this because you enabled ma

[Apache TVM Discuss] [Development/RFC] [RFC][Tensorcore] INT4 end-to-end inference

2021-05-02 Thread zachzzc via Apache TVM Discuss
So you want to run on CPU instead of GPU? What's the error you are seeing after changing the target? The inference script in my repo won't work because I think some of the convolution layouts are not supported yet in CPU x86 computation. If you want to run the quantized NNs on CPU, you may nee