Re: range queries on string field with millions of values

2008-11-30 Thread Yonik Seeley
On Sun, Nov 30, 2008 at 2:04 AM, Naomi Dushay <[EMAIL PROTECTED]> wrote: > The terms component approach, if i understand it correctly, will be > problematic. I need to present not only the next X call numbers in > sequence, but other fields in those documents (e.g. title, author). You can still u

Re: range queries on string field with millions of values

2008-11-29 Thread Naomi Dushay
Hi Hoss, Thanks for this. The terms component approach, if i understand it correctly, will be problematic. I need to present not only the next X call numbers in sequence, but other fields in those documents (e.g. title, author). I assume the Terms Component approach will only give me the

Re: range queries on string field with millions of values

2008-11-29 Thread Chris Hostetter
: The results are correct. But the response time sucks. : : Reading the docs about caches, I thought I could populate the query result : cache with an autowarming query and the response time would be okay. But that : hasn't worked. (See excerpts from my solrConfig file below.) : : A repeated

Re: range queries on string field with millions of values

2008-11-28 Thread Yonik Seeley
On Wed, Nov 26, 2008 at 5:43 PM, Naomi Dushay <[EMAIL PROTECTED]> wrote: > sortCallNum["A123 B34 1970" TO *]&rows=10. If you really just want to get call numbers X through X+10, then you are in luck: https://issues.apache.org/jira/browse/SOLR-877 http://wiki.apache.org/solr/TermsComponent But loo

Re: range queries on string field with millions of values

2008-11-28 Thread Walter Underwood
Do you want to page through all items or through the result of a query (like all hits for "civil war" in call number order). If you want the former, then a text search engine is really the wrong tool. This problem only requires indexed sequential file formats (like B-trees), something that worked

Re: range queries on string field with millions of values

2008-11-28 Thread Naomi Dushay
Gosh, I'm sorry to be so unclear. Hmm. Trying to clarify below: On Nov 28, 2008, at 3:52 PM, Chris Hostetter wrote: Having read through this thread, i'm not sure i understand what exactly the problem is. my naive understanding is... 1) you want to sort by a field 2) you want to be able t

Re: range queries on string field with millions of values

2008-11-28 Thread Chris Hostetter
Having read through this thread, i'm not sure i understand what exactly the problem is. my naive understanding is... 1) you want to sort by a field 2) you want to be able to "paginate" through all docs in order of this field. 3) you want to be able to start your pagination at any arbitrary val

Re: range queries on string field with millions of values

2008-11-28 Thread Glen Newton
Hi Naomi, Try fixing your data. :-) No, really: 1 - Sort all of your call numbers using whatever sort makes sense to you. 2 - Assign them - in your sort order - sort keys that are floats, starting: 0.01 0.02 ... 1.01 1.02 ... 79,999.98 79,999.99 This should ap

Re: range queries on string field with millions of values

2008-11-28 Thread Naomi Dushay
The point isn't really how the exact sort works - it's the performance issues, coupled with an unpredictable distribution along the entire possible sort space. the sort works the range queries work the performance sucks and I haven't thought of a clever work around. - Naomi On Nov 27, 2008

Re: range queries on string field with millions of values

2008-11-27 Thread Alexander Ramos Jardim
I did not even understand what you are considering to be the order on your call numbers. 2008/11/26 Naomi Dushay <[EMAIL PROTECTED]> > I have a performance problem and I haven't thought of a clever way around > it. > > I work at the Stanford University Libraries. We have a collection of over > 8

range queries on string field with millions of values

2008-11-26 Thread Naomi Dushay
I have a performance problem and I haven't thought of a clever way around it. I work at the Stanford University Libraries. We have a collection of over 8 million items. Each item has a call number. I have been asked to provide a way to browse forward and backward from an arbitrary call