dungba88 commented on PR #14009: URL: https://github.com/apache/lucene/pull/14009#issuecomment-3006697973
> result computation for k-NN is done upfront, but queries can also be used to re-score after the initial phase via the [QueryRescorer](https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/search/QueryRescorer.java) (as @mikemccand mentioned awhile ago), so I dont think a separate rescorer is necessary. Rescorer can be used, but IIUC Rescorer works only in the collection phase. After the first pass collection we will rescore the final results. This would not work if we combine semantic and lexical matching into a single Query, in that case we can only rescore the combined matches. Like @benwtrent mentioned, rescoring should be done in the rewrite phase. This will work for both the cases where semantic and lexical are combined or where semantic matching alone. > "or atleast provide a scorer that iterates the kNN query results calculating the higher fidelity scores" This is also handled by this PR, technically not a Scorer but a DoubleValueSources. User can use it to either use full-precision vectors, or even use another field for rescoring (amongst other use cases, as DoubleValueSources is extensible). A potential idea is to have 1-bit vector field for matching and another 4-bit or 7-bit vector field for rescoring. However the quantization cost of query vector even for scalar 7-bit is a bit high. We will tackle it as future optimization to this new Query. Also thanks @vigyasharma for approving the PR! Can you help to merge it if there is no objection? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
