[ 
https://issues.apache.org/jira/browse/LUCENE-10425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17501790#comment-17501790
 ] 

Adrien Grand commented on LUCENE-10425:
---------------------------------------

This would require a new API on PostingsEnum so I'll write what I think the 
applications of this change are to make sure I get the benefits correctly: If 
the field is sorted by a numeric field, Lucene could efficiently compute range 
facets (and special forms of range facets like histograms) for this numeric 
field as long as there are no deletions and the query has a single term. For 
instance, an index containing logs and sorted by timestamp could very 
efficiently compute an histogram of the timestamp field given any term query. 
To use an example from a different use-case, an index of an e-commerce catalog 
sorted by price could compute a histogram of prices very efficiently for any 
term query.

This feels quite powerful. The main thing that annoys me a bit is that it only 
works on the primary sort field, so we'd be adding an API for PostingsEnum for 
something that requires a very careful setup of the index as their can be a 
single primary sort field. I wonder if LUCENE-10396 could help this 
optimization more often applicable, e.g. to logs indices sorted by host then 
timestamp, or to e-commerce indices sorted by category then price. Having this 
optimization more generally applicable would make me feel better about 
increasing the surface area of PostingsEnum. At first sight, it feels like this 
should work? Maybe this use-case would also help figure out what the API should 
be on LUCENE-10396.

> count aggregation optimization inside one segment in log scenario
> -----------------------------------------------------------------
>
>                 Key: LUCENE-10425
>                 URL: https://issues.apache.org/jira/browse/LUCENE-10425
>             Project: Lucene - Core
>          Issue Type: New Feature
>          Components: core/search
>            Reporter: jianping weng
>            Priority: Major
>          Time Spent: 4.5h
>  Remaining Estimate: 0h
>
> In log scenario, we usually want to know the doc count of documents between 
> every time intervals. One possible optimized method is to sort the docuemt in 
> ascend order according to @timestamp field in one segment. then we can use    
> this pr [https://github.com/apache/lucene/pull/687] to find out the min/max 
> docId in on time interval.
> If there is no other filter query, the doc count of one time interval is (max 
> docId- min docId +1)
> if there is only one another term filter query, we can use this pr 
> [https://github.com/apache/lucene/pull/688 
> |https://github.com/apache/lucene/pull/688]to get the diff value of index, 
> when we call advance(minId) and advance(maxId), the diff value is also the 
> doc count of one time interval
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

Reply via email to