Re: Search with very large boolean filter

2015-11-20 Thread jichi
Hi Shawn, We have already switched the request method to POST. I am going to try the term query parser soon. I will post the performance difference against the IN syntax here later. Thanks! 2015-11-20 15:23 GMT-08:00 Shawn Heisey : > On 11/20/2015 4:09 PM, jichi wrote: > > Thanks for the quick

Re: Search with very large boolean filter

2015-11-20 Thread Shawn Heisey
On 11/20/2015 4:09 PM, jichi wrote: > Thanks for the quick replies, Alex and Jack! > >> definitely can improve on the ORing the ids with > Going to try that! But I guess it would still hit the maxBooleanClauses=1024 > threshold. The terms query parser does not have a limit like boolean queries do.

Re: Search with very large boolean filter

2015-11-20 Thread jichi
Thanks for the quick replies, Alex and Jack! > definitely can improve on the ORing the ids with Going to try that! But I guess it would still hit the maxBooleanClauses=1024 threshold. > 1. Are you trying to retrieve a large number of documents, or simply perform queries against a subset of the in

Re: Search with very large boolean filter

2015-11-20 Thread Jack Krupansky
1. Are you trying to retrieve a large number of documents, or simply perform queries against a subset of the index? 2. How many unique queries are you expecting to perform against each specific filter set of IDs? 3. How often does the set of IDs change? 4. Is there more than one filter set of ID

Re: Search with very large boolean filter

2015-11-20 Thread Alexandre Rafalovitch
I don't know what to do about 30K ids, but you definitely can improve on the ORing the ids with https://cwiki.apache.org/confluence/display/solr/Other+Parsers#OtherParsers-TermsQueryParser Regards, Alex. Newsletter and resources for Solr beginners and intermediates: http://www.solr-start.c

Search with very large boolean filter

2015-11-20 Thread jichi
Hi, I am using Solr 4.7.0 to search text with an id filter, like this: id:(100 OR 2 OR 5 OR 81 OR 10 ...) The number of IDs in the boolean filter are usually less than 100, but could sometimes be very large (around 30k IDs). We currently set maxBooleanClauses to 1024, partitioned the IDs