[ 
https://issues.apache.org/jira/browse/LUCENE-10207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17441817#comment-17441817
 ] 

Greg Miller commented on LUCENE-10207:
--------------------------------------

OK, I buy that. I hadn't spent enough time looking at how [disjunction cost is 
estimated|https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/search/ScorerUtil.java#L25],
 but after looking into that logic, I agree this should be consistent with a 
disjunction where minShouldMatch is effectively 1.

So I think that just looks like the following (with the proposal that MTQ can 
provide its number of unique terms, or -1 if it doesn't know up-front):


{code:java}
        final long cost;
        final int queryTermsCount = query.getTermsCount();
        if (queryTermsCount == -1) {
          cost = indexTerms.getSumDocFreq();
        } else {
          cost = queryTermsCount + (indexTerms.getSumDocFreq() - 
indexTerms.size());
        }
{code}


> Make TermInSetQuery usable with IndexOrDocValuesQuery
> -----------------------------------------------------
>
>                 Key: LUCENE-10207
>                 URL: https://issues.apache.org/jira/browse/LUCENE-10207
>             Project: Lucene - Core
>          Issue Type: Improvement
>            Reporter: Adrien Grand
>            Priority: Minor
>         Attachments: LUCENE-10207_multitermquery.patch
>
>
> IndexOrDocValuesQuery is very useful to pick the right execution mode for a 
> query depending on other bits of the query tree.
> We would like to be able to use it to optimize execution of TermInSetQuery. 
> However IndexOrDocValuesQuery only works well if the "index" query can give 
> an estimation of the cost of the query without doing anything expensive (like 
> looking up all terms of the TermInSetQuery in the terms dict). Maybe we could 
> implement it for primary keys (terms.size() == sumDocFreq) by returning the 
> number of terms of the query? Another idea is to multiply the number of terms 
> by the average postings length, though this could be dangerous if the field 
> has a zipfian distribution and some terms have a much higher doc frequency 
> than the average.
> [~romseygeek] and I were discussing this a few weeks ago, and more recently 
> [~mikemccand] and [~gsmiller] again independently. So it looks like there is 
> interest in this. Here is an email thread where this was recently discussed: 
> https://lists.apache.org/thread.html/re3b20a486c9a4e66b2ca4a2646e2d3be48535a90cdd95911a8445183%40%3Cdev.lucene.apache.org%3E.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

Reply via email to