msfroh commented on code in PR #15818:
URL: https://github.com/apache/lucene/pull/15818#discussion_r2984756884
##########
lucene/core/src/java/org/apache/lucene/search/BooleanQuery.java:
##########
@@ -425,24 +426,39 @@ public Query rewrite(IndexSearcher indexSearcher) throws
IOException {
}
}
- // Deduplicate SHOULD clauses by summing up their boosts
+ // Deduplicate SHOULD clauses. Tracks [boostSum, count] per query.
+ // Uses Similarity.computeQueryTermWeight for unweighted duplicates (e.g.
"a a a"),
+ // falls back to linear boost sum when explicit boosts are present (e.g.
"a^2 a^3").
if (clauseSets.get(Occur.SHOULD).size() > 0 && minimumNumberShouldMatch <=
1) {
- Map<Query, Double> shouldClauses = new HashMap<>();
+ Map<Query, double[]> shouldClauses = new HashMap<>();
for (Query query : clauseSets.get(Occur.SHOULD)) {
double boost = 1;
while (query instanceof BoostQuery) {
BoostQuery bq = (BoostQuery) query;
boost *= bq.getBoost();
query = bq.getQuery();
}
- shouldClauses.put(query, shouldClauses.getOrDefault(query, 0d) +
boost);
+ double[] entry = shouldClauses.get(query);
+ if (entry == null) {
+ shouldClauses.put(query, new double[] {boost, 1});
+ } else {
+ entry[0] += boost;
+ entry[1] += 1;
+ }
}
if (shouldClauses.size() != clauseSets.get(Occur.SHOULD).size()) {
+ Similarity similarity = indexSearcher.getSimilarity();
BooleanQuery.Builder builder =
new
BooleanQuery.Builder().setMinimumNumberShouldMatch(minimumNumberShouldMatch);
- for (Map.Entry<Query, Double> entry : shouldClauses.entrySet()) {
+ for (Map.Entry<Query, double[]> entry : shouldClauses.entrySet()) {
Query query = entry.getKey();
- float boost = entry.getValue().floatValue();
+ double boostSum = entry.getValue()[0];
+ int qtf = (int) entry.getValue()[1];
+ // Only apply similarity-based query term weighting when all
occurrences have the
+ // default boost of 1.0 (the common case from query parsers). When
explicit boosts
+ // are present (e.g. programmatic BoostQuery), preserve the original
linear sum.
+ float boost =
+ (boostSum == qtf) ? similarity.computeQueryTermWeight(qtf) :
(float) boostSum;
Review Comment:
I'm a little scared of the equality check between an `int` and a `double`,
since I always expect some rounding errors (though I guess it's unlikely before
we hit the clause limit).
Regardless, I think this could be simplified by adding a `Set<Query>` that
holds all the clauses that have a boost. Then you could go back to tracking the
total boost / qtf in a `Map<Query, Double>`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]