shbhar commented on PR #15903: URL: https://github.com/apache/lucene/pull/15903#issuecomment-4239393141
>You can't/shouldn't remove the code in OSQ that corrects the integer dot product using the 16 byte footer, it doesn't make any more sense than returning the int8 dot product directly for TQ. You are right - I guess I can only disable optimizeInterval() and see if the per vector footer still provides benefit or not on already rotated vectors. So on centered+rotated data, if OSQ recall without optimizeInterval() is also same as TQ on centered+unrotated vectors (avoiding double rotation), then maybe that would be an argument for the remaining TQ approach over just adding rotation as an option in OSQ? Does that make sense? But I guess the footer is negligible storage cost and optimizeIntervals is cheap anyway (right?) so might not be worth optimizing for and you are making the argument that it is better to just add rotation & datablind options to OSQ (to be able to drop fp32). Let me look into that. One thing I've ignored completely so far is the power of 2 limitation of the current FWHT implementation, so with padding/block-diagonal etc approaches I am not sure what happens to recall/performance on something like 1536d vectors -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
