Re: Keeping the index naturally ordered by some field

2017-10-02 Thread Erick Erickson
Have you looked at Streaming and Streaming Expressions? This is pretty much what they were built for. Since you're talking a billion documents, you're probably sharding anyway, in which case I'd guess you're using SolrCloud. That's what I'd be using first if at all possible. Best, Erick On Mon

Re: Keeping the index naturally ordered by some field

2017-10-02 Thread alexpusch
The reason I'm interested in this is kind of unique. I'm writing a custom query parser and search component. These components go over the search results and perform some calculation over it. This calculation depends on input sorted by a certain value. In this scenario, regular solr sorting is insuf

Re: Keeping the index naturally ordered by some field

2017-10-02 Thread alessandro.benedetti
Hi Alex, just to explore a bit your question, why do you need that ? Do you need to reduce query time ? Have you tried enabling docValues for the fields of interest ? Doc Values seem to me a pretty useful data structure when sorting is a requirement. I am curious to understand why that was not an o

Re: Keeping the index naturally ordered by some field

2017-10-01 Thread Erick Erickson
I think you're looking for SortingMergePolicyFactory, see: https://issues.apache.org/jira/browse/SOLR-5730 The JIRA has some extensive discussion and the reference guide has an example. It might take a little digging Best, Erick On Sun, Oct 1, 2017 at 4:36 AM, Ahmet Arslan wrote: > > > Hi A

Re: Keeping the index naturally ordered by some field

2017-10-01 Thread Ahmet Arslan
Hi Alex, Lucene has this capability (borrowed from Nutch) underĀ  org.apache.lucene.index.sorter package.I think it has been integrated into Solr, but could not find the Jira issue. Ahmet On Sunday, October 1, 2017, 10:22:45 AM GMT+3, alexpusch wrote: Hello, We've got a pretty big