Re: query with wild card with AND taking lot of time

2017-09-04 Thread Shawn Heisey
On 9/1/2017 5:24 PM, Walter Underwood wrote: > Hmm. Solr really should convert an fq of “a AND b” to separate “a” and “b” fq > filters. That should be a simple special-case rewrite. It might take less > time to implement than explaining it to everyone. > > Well, I guess then we’d have to explain

Re: query with wild card with AND taking lot of time

2017-09-03 Thread Dave
ginal Message- > From: Erick Erickson [mailto:erickerick...@gmail.com] > Sent: Saturday, 2 September 2017 12:01 p.m. > To: solr-user > Subject: Re: query with wild card with AND taking lot of time > > How far would you take that? Say you had 100 terms joined by AND (ridiculou

RE: query with wild card with AND taking lot of time

2017-09-03 Thread Phil Scadden
5 seems a reasonable limit to me. After that revert to slow. -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Saturday, 2 September 2017 12:01 p.m. To: solr-user Subject: Re: query with wild card with AND taking lot of time How far would you take that? Say

Re: query with wild card with AND taking lot of time

2017-09-01 Thread Erick Erickson
How far would you take that? Say you had 100 terms joined by AND (ridiculous I know, just sayin' ). Then you'd chew up 100 entries in the filterCache. On Fri, Sep 1, 2017 at 4:24 PM, Walter Underwood wrote: > Hmm. Solr really should convert an fq of “a AND b” to separate “a” and “b” fq > filters

Re: query with wild card with AND taking lot of time

2017-09-01 Thread Walter Underwood
Hmm. Solr really should convert an fq of “a AND b” to separate “a” and “b” fq filters. That should be a simple special-case rewrite. It might take less time to implement than explaining it to everyone. Well, I guess then we’d have to explain how it wasn’t really necessary to send separate fq pa

Re: query with wild card with AND taking lot of time

2017-09-01 Thread Erick Erickson
Shawn: See: https://issues.apache.org/jira/browse/SOLR-7219 Try fq=filter(foo) filter(bar) filter(baz) Patches to docs welcome ;) On Fri, Sep 1, 2017 at 1:50 PM, Shawn Heisey wrote: > On 9/1/2017 8:13 AM, Alexandre Rafalovitch wrote: >> You can OR cachable filter queries in the latest Solr

Re: query with wild card with AND taking lot of time

2017-09-01 Thread Mikhail Khludnev
Shawn, you are welcome: http://lucene.apache.org/solr/guide/6_6/the-standard-query-parser.html Support for a special filter(…​) syntax to... https://issues.apache.org/jira/browse/SOLR-7219 On Fri, Sep 1, 2017 at 11:50 PM, Shawn Heisey wrote: > On 9/1/2017 8:13 AM, Alexandre Rafalovitch wrote:

Re: query with wild card with AND taking lot of time

2017-09-01 Thread Shawn Heisey
On 9/1/2017 8:13 AM, Alexandre Rafalovitch wrote: > You can OR cachable filter queries in the latest Solr. There is a special > (filter) syntax for that. This is actually possible?  If so, I didn't see anything come across the dev list about it. I opened an issue for it, didn't know anything had

Re: query with wild card with AND taking lot of time

2017-09-01 Thread Alexandre Rafalovitch
You can OR cachable filter queries in the latest Solr. There is a special (filter) syntax for that. Regards, Alex On 31 Aug. 2017 2:11 pm, "Josh Lincoln" wrote: As I understand it, using a different fq for each clause makes the resultant caches more likely to be used in future requests. Fo

Re: query with wild card with AND taking lot of time

2017-08-31 Thread David Hastings
a field:* query always takes a long time, and should be avoided if at all possible. solr/lucene is still going to try to rank the documents based on that, even thought theres nothing to really rank. every single document where that field is not empty will have the same score for that part of the

Re: query with wild card with AND taking lot of time

2017-08-31 Thread Josh Lincoln
The closest thing to an execution plan that I know of is debug=true.That'll show timings of some of the components I also find it useful to add echoParams=all when troubleshooting. That'll show every param solr is using for the request, including params set in solrconfig.xml and not passed in the r

Re: query with wild card with AND taking lot of time

2017-08-31 Thread suresh pendap
Thanks Lincoln for your suggestions. It was very helpful. I am still curious as to why is the original query taking long time. It is something that Lucene should have ideally optimized. Is there any way to see the execution plan used by Lucene? Thanks Suresh On Thu, Aug 31, 2017 at 11:11 AM, J

Re: query with wild card with AND taking lot of time

2017-08-31 Thread Josh Lincoln
As I understand it, using a different fq for each clause makes the resultant caches more likely to be used in future requests. For the query fq=first:bob AND last:smith a subsequent query for fq=first:tim AND last:smith won't be able to use the fq cache from the first query. However, if the first

Re: query with wild card with AND taking lot of time

2017-08-31 Thread David Hastings
> > 2) Because all your clauses are more like filters and are ANDed together, > you'll likely get better performance by putting them _each_ in an fq > E.g. > fq=product_identifier_type:DOTCOM_OFFER > fq=abstract_or_primary_product_id:[* TO *] why is this the case? is it just better to have no lo

Re: query with wild card with AND taking lot of time

2017-08-31 Thread Josh Lincoln
Suresh, Two things I noticed. 1) If your intent is to only match records where there's something, anything, in abstract_or_primary_product_id, you should use fieldname:[* TO *] but that will exclude records where that field is empty/missing. If you want to match records even if that field is empty

query with wild card with AND taking lot of time

2017-08-31 Thread suresh pendap
Hello everybody, We are seeing that the below query is running very slow and taking almost 4 seconds to finish [] webapp=/solr path=/select params={df=_text_&distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=modified_dtm+desc&shard.url=http://:8983/solr/flat_product_index_shard7_replica