sgup432 commented on issue #15768:
URL: https://github.com/apache/lucene/issues/15768#issuecomment-4035368883
Interesting
@jpountz
>duplicate terms in query strings see their scores boosted by their number
of occurrences
I guess you meant here that if a user passes "hello hello hello world" -> we
convert this into something like `BooleanQuery(
SHOULD(BoostQuery(TermQuery("hello"), 3.0)),
SHOULD(TermQuery("word"))
)` where `hello` score gets multiplied by 3? I see the logic
[here](https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/search/BooleanQuery.java#L438)
where the boost is calculated for BooleanQuery.
>It would likely be better to saturate the contribution of such repeated
terms, as suggested in the original BM25 paper through the k3 tuning parameter.
Makes sense. I guess we can add a logic before we form BooleanQuery in query
parser(QueryParser), identify duplicates, rewrite the query with the updated
boost(via k3 tuning param), so boolean query never sees the duplicates in the
first place.
If this all makes sense, I can work on raising a PR for this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]