naveentatikonda commented on issue #13519: URL: https://github.com/apache/lucene/issues/13519#issuecomment-2207802401
@benwtrent The new numbers looks good. But, I have few questions based on your [code changes ](https://github.com/apache/lucene/compare/main...benwtrent:fix-8-bit?expand=1): 1. Why are we setting `SIGNED_CORRECTION ` as 127 instead of 128 ? 2. Also, [here](https://github.com/apache/lucene/compare/main...benwtrent:fix-8-bit?expand=1#diff-f4b5ebf64c154fb2edaec936b36771bb15f9953717e868c7c2cbda2ca6335fbeR140) the min should be -128 but you are using -127 3. In this new code we are not adding this [rounding error correction](https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/util/quantization/ScalarQuantizer.java#L147)(`(dx - dxq) * dxq`) like you are doing in the regular implementation. Is this not required ? Last time when I tried to run a test without this rounding error corrective offset using 7 bits there is a huge drop in recall. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
