Re: Range queries with Grouping is slow?

2014-01-09 Thread Kranti Parisa
Thank you, will take a look at it. Thanks, Kranti K. Parisa http://www.linkedin.com/in/krantiparisa On Thu, Jan 9, 2014 at 10:25 AM, Mikhail Khludnev < mkhlud...@griddynamics.com> wrote: > Hello, > > Here is workaround for caching separate clauses in OR filters. > http://blog.griddynamics.com/

Re: Range queries with Grouping is slow?

2014-01-09 Thread Mikhail Khludnev
Hello, Here is workaround for caching separate clauses in OR filters. http://blog.griddynamics.com/2014/01/segmented-filter-cache-in-solr.html No coding is required, just try to experiment with request parameters. On Wed, Jan 8, 2014 at 9:11 PM, Erick Erickson wrote: > Well, actually you can us

Re: Range queries with Grouping is slow?

2014-01-09 Thread Smiley, David W.
It won¹t hit the filter cache if you set {! cache=false} local-param. On 1/8/14, 12:18 PM, "Kranti Parisa" wrote: >yes thats the key, these time ranges change frequently and hitting >filtercache then is a problem. I will try few more samples and probably >debug thru it. thanks. > > >Thanks, >Kra

Re: Range queries with Grouping is slow?

2014-01-08 Thread Erick Erickson
Yeah, time fqs in particular can be tricky to re-use if you specify NOW, see: http://searchhub.org/2012/02/23/date-math-now-and-filter-queries/ Best, Erick On Wed, Jan 8, 2014 at 12:18 PM, Kranti Parisa wrote: > yes thats the key, these time ranges change frequently and hitting > filtercache t

Re: Range queries with Grouping is slow?

2014-01-08 Thread Kranti Parisa
yes thats the key, these time ranges change frequently and hitting filtercache then is a problem. I will try few more samples and probably debug thru it. thanks. Thanks, Kranti K. Parisa http://www.linkedin.com/in/krantiparisa On Wed, Jan 8, 2014 at 12:11 PM, Erick Erickson wrote: > Well, act

Re: Range queries with Grouping is slow?

2014-01-08 Thread Erick Erickson
Well, actually you can use fqs, it's just that re-using them becomes a bit more tricky. Specifically, fq=field1:blah OR field2:blort is perfectly reasonable. However, it doesn't break things down into sub-clauses, so fq=field1:blah will create a new entry in the filtercache. And fq=field2:blort OR

Re: Range queries with Grouping is slow?

2014-01-08 Thread Kranti Parisa
I was trying with the [* TO *] as an example, the real use case is OR query between 2/more range queries of timestamp fields (saved in milliseconds). So I can't use FQs as they are ANDed by definition. Am I missing something here? Thanks, Kranti K. Parisa http://www.linkedin.com/in/krantipari

Re: Range queries with Grouping is slow?

2014-01-08 Thread Joel Bernstein
Kranti, The range query also looks like a good candidate to be moved to a filter query so it can be cached. Joel Bernstein Search Engineer at Heliosearch On Tue, Jan 7, 2014 at 11:34 PM, Smiley, David W. wrote: > Kranti, > > I can't speak to the specific slow-down while grouping, but if you e

RE: Range queries with Grouping is slow?

2014-01-07 Thread Smiley, David W.
Kranti, I can't speak to the specific slow-down while grouping, but if you expect to run [* TO *] queries with any frequency then you should index a boolean flag and query for that instead. You might also reduce the precisionStep value for the field you are using to 6 or even 4. But wow that'

Re: Range Queries performing differently on SortableIntField vs TrieField of type integer

2012-12-04 Thread Jack Krupansky
Could you show us some input data, both WITH a i_yearStopSort value and WITHOUT the the value? I tried a quick test using the stock Solr 3.6.1 example schema and a dynamic integer field and the filter query did in fact filter out all documents that did not have a value in that field: http://

Re: Range Queries performing differently on SortableIntField vs TrieField of type integer

2012-12-04 Thread Chris Hostetter
: q=*:*&fq=+i_yearStartSort:{* TO 1995}&fq=+i_yearStopSort:{* TO *} ... : Unfortunately, under 3.6.1 with class="solr.TrieField" type="integer", this : query is returning docs that have neither an i_yearStopSort nor a : i_yearStartSort value. H... I can't seem to reproduce this. Here

Re: Range Queries performing differently on SortableIntField vs TrieField of type integer

2012-12-04 Thread Aaron Daubman
I forgot a possibly important piece... Given the different Solr versions, the schema version (and it's related different defaults) is also a change: Solr 1.4.1 Has: Solr 3.6.1 Has: > Solr 1.4.1 Relevant Schema Parts - Working as desired: > > >

Re: Range Queries performing differently on SortableIntField vs TrieField of type integer

2012-12-04 Thread Aaron Daubman
Hi Upayavira, One small question - did you re-index in-between? The index structure > will be different for each. > Yes, the Solr 1.4.1 (working) instance was built using the original schema and that solr version. The Solr 3.6.1 (not working) instance was re-built using the new schema and Solr 3.

Re: Range Queries performing differently on SortableIntField vs TrieField of type integer

2012-12-04 Thread Upayavira
One small question - did you re-index in-between? The index structure will be different for each. Upayavira On Tue, Dec 4, 2012, at 02:30 PM, Aaron Daubman wrote: > Greetings, > > I'm finally updating an old instance and in testing, discovered that > using > the recommended TrieField instead of

Re: Range Queries -sfloat

2012-04-10 Thread Chris Hostetter
: query is price: [ 1 TO 20 ] is returning values out of this range ,like : 23.00 AND 55.00 .The field type of the price field is sfloat . can you provide more details about the documents matching out of the range? are you sure this isn't a multivalued field? : When I check this form admin Deb

Re: Range queries

2009-06-17 Thread gwk
Yes, this works perfectly, guess the "Never use equality comparison for floating point numbers"-rule was so strong in my mind I didn't even think to consider this possibility. Thanks, gwk Avlesh Singh wrote: Really sorry, this is what I meant: x:{5 TO 8} OR x:5 Cheers Avlesh On Wed, Jun 17

Re: Range queries

2009-06-16 Thread Avlesh Singh
Really sorry, this is what I meant: x:{5 TO 8} OR x:5 Cheers Avlesh On Wed, Jun 17, 2009 at 9:36 AM, Avlesh Singh wrote: > And how about this - x:{5 TO 8} AND x:5 > > Cheers > Avlesh > > > On Wed, Jun 17, 2009 at 1:57 AM, Peter Keegan wrote: > >> How about this: x:[5 TO 8] AND x:{0 TO 8} >> >>

Re: Range queries

2009-06-16 Thread Avlesh Singh
And how about this - x:{5 TO 8} AND x:5 Cheers Avlesh On Wed, Jun 17, 2009 at 1:57 AM, Peter Keegan wrote: > How about this: x:[5 TO 8] AND x:{0 TO 8} > > On Tue, Jun 16, 2009 at 1:16 PM, Otis Gospodnetic < > otis_gospodne...@yahoo.com> wrote: > > > > > Hi, > > > > I think the square brackets/cu

Re: Range queries

2009-06-16 Thread Peter Keegan
How about this: x:[5 TO 8] AND x:{0 TO 8} On Tue, Jun 16, 2009 at 1:16 PM, Otis Gospodnetic < otis_gospodne...@yahoo.com> wrote: > > Hi, > > I think the square brackets/curly braces need to be balanced, so this is > currently not doable with existing query parsers. > > Otis > -- > Sematext -- ht

Re: Range queries

2009-06-16 Thread Otis Gospodnetic
Hi, I think the square brackets/curly braces need to be balanced, so this is currently not doable with existing query parsers. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: gwk > To: solr-user@lucene.apache.org > Sent: Tuesday, June 1

Re: range queries on string field with millions of values

2008-11-30 Thread Yonik Seeley
On Sun, Nov 30, 2008 at 2:04 AM, Naomi Dushay <[EMAIL PROTECTED]> wrote: > The terms component approach, if i understand it correctly, will be > problematic. I need to present not only the next X call numbers in > sequence, but other fields in those documents (e.g. title, author). You can still u

Re: range queries on string field with millions of values

2008-11-29 Thread Naomi Dushay
Hi Hoss, Thanks for this. The terms component approach, if i understand it correctly, will be problematic. I need to present not only the next X call numbers in sequence, but other fields in those documents (e.g. title, author). I assume the Terms Component approach will only give me the

Re: range queries on string field with millions of values

2008-11-29 Thread Chris Hostetter
: The results are correct. But the response time sucks. : : Reading the docs about caches, I thought I could populate the query result : cache with an autowarming query and the response time would be okay. But that : hasn't worked. (See excerpts from my solrConfig file below.) : : A repeated

Re: range queries on string field with millions of values

2008-11-28 Thread Yonik Seeley
On Wed, Nov 26, 2008 at 5:43 PM, Naomi Dushay <[EMAIL PROTECTED]> wrote: > sortCallNum["A123 B34 1970" TO *]&rows=10. If you really just want to get call numbers X through X+10, then you are in luck: https://issues.apache.org/jira/browse/SOLR-877 http://wiki.apache.org/solr/TermsComponent But loo

Re: range queries on string field with millions of values

2008-11-28 Thread Walter Underwood
Do you want to page through all items or through the result of a query (like all hits for "civil war" in call number order). If you want the former, then a text search engine is really the wrong tool. This problem only requires indexed sequential file formats (like B-trees), something that worked

Re: range queries on string field with millions of values

2008-11-28 Thread Naomi Dushay
Gosh, I'm sorry to be so unclear. Hmm. Trying to clarify below: On Nov 28, 2008, at 3:52 PM, Chris Hostetter wrote: Having read through this thread, i'm not sure i understand what exactly the problem is. my naive understanding is... 1) you want to sort by a field 2) you want to be able t

Re: range queries on string field with millions of values

2008-11-28 Thread Chris Hostetter
Having read through this thread, i'm not sure i understand what exactly the problem is. my naive understanding is... 1) you want to sort by a field 2) you want to be able to "paginate" through all docs in order of this field. 3) you want to be able to start your pagination at any arbitrary val

Re: range queries on string field with millions of values

2008-11-28 Thread Glen Newton
Hi Naomi, Try fixing your data. :-) No, really: 1 - Sort all of your call numbers using whatever sort makes sense to you. 2 - Assign them - in your sort order - sort keys that are floats, starting: 0.01 0.02 ... 1.01 1.02 ... 79,999.98 79,999.99 This should ap

Re: range queries on string field with millions of values

2008-11-28 Thread Naomi Dushay
The point isn't really how the exact sort works - it's the performance issues, coupled with an unpredictable distribution along the entire possible sort space. the sort works the range queries work the performance sucks and I haven't thought of a clever work around. - Naomi On Nov 27, 2008

Re: range queries on string field with millions of values

2008-11-27 Thread Alexander Ramos Jardim
I did not even understand what you are considering to be the order on your call numbers. 2008/11/26 Naomi Dushay <[EMAIL PROTECTED]> > I have a performance problem and I haven't thought of a clever way around > it. > > I work at the Stanford University Libraries. We have a collection of over > 8