jmazanec15 commented on issue #13350: URL: https://github.com/apache/lucene/issues/13350#issuecomment-2123110105
I am trying to understand one thing: Does the corrective offset for dot product rectify issues with sign shift that is caused by going from signed domain: [-x, +y] to unsigned domain: [0, 127]? Or is this handled elsewhere? For an overly simplified example, for a data set with `min = -6` and `max = 6` (and for simplicity assume min max are the respective upper and lower quantiles) ``` python # Query vectors query_vector = [-5] quantized_query_vector = [11] # Index vectors ## Full precision 1: [-6] 2: [-2] 3: [3] 4: [6] ## Quantized 1: [0] 2: [42] 3: [95] 4: [127] ## Full precision Ordering of query_vector . index_vectors: 1,2,3,4 ## Quantized Ordering of quantized_query_vector . quantized_index_vectors 4,3,2,1 ``` I think I might be missing something, but how is this accounted for? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
