benwtrent commented on issue #13519: URL: https://github.com/apache/lucene/issues/13519#issuecomment-2343689270
@mikemccand that makes sense to me. All the numerics we are messing with here shows that we are hitting some weird edge cases where int8 just isn't worth it if it remains signed & we attempt to accurate scale the linear transformation of the scores. I also don't have cycles right now to dig further. Though I welcome others attempts. My gut reaction is that the only way to handle this for `int8` is to go "full unsigned" and have some custom scoring logic that transforms the bytes as unsigned, etc. though that adds significant code to our vector util's, etc. (see my very old and probably now defunct draft: https://github.com/apache/lucene/pull/12694) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
