> I guess the saturation is exactly what activations (ReLU family) mean, > semantically. :) In the case of the tflite quantized models I've looked at, the batch normalization and relu6 operations in training are fused into the conv operations used during inference. You probably need to fuse the relu6 to match their results.
This paper removes the relu6 and batch norm associated with the depthwise convs in a mobilenet modification. You would still need the min and max values for those depthwise conv operations even though there is no fused activation. So, that is all I was trying to say ... those min and max values are really to saturate the quantization range, rather than representing an activation operation. ` https://arxiv.org/pdf/1803.08607.pdf ` -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/issues/2351#issuecomment-502521939
