Re: SOLR Boolean clause impact on memory/Performance

2014-10-14 Thread Yonik Seeley
A terms query will be better than a boolean query here (assuming you don't care about scoring those terms): http://heliosearch.org/solr-terms-query/ But you need a recent version of Solr or Heliosearch. -Yonik http://heliosearch.org - native code faceting, facet functions, sub-facets, off-heap da

Re: SOLR Boolean clause impact on memory/Performance

2014-10-14 Thread Erick Erickson
Then I predict they will continue to grow and whatever limit you put on maxBooleanClauses will be exceeded later. And so on, so I really think you need to re-think your model. One approach: 1> change your model so your users are assigned to a fixed number of groups. Then index group tokens with ea

Re: SOLR Boolean clause impact on memory/Performance

2014-10-14 Thread ankit gupta
Thanks Erick for responding. We have assigned 4GB memory for SOLR server and at high load where queries are having more than 10K boolean clauses, combination of cache and high boolean clauses are causing system to break. The system was working fine for last 8 months but ofcourse the boolean clause

Re: SOLR Boolean clause impact on memory/Performance

2014-10-13 Thread Erick Erickson
Of course there will be performance and memory changes. The only real question is whether your situation can tolerate them. The whole point of maxBooleanClauses is exactly that going above that limit should be a conscious decision because it has implications for both memory and performance That sa

SOLR Boolean clause impact on memory/Performance

2014-10-13 Thread ankit gupta
hi, Can we quantify the impact on SOLR memory usage/performance if we increase the boolean clause. I am currently using lot of OR clauses in the query (close to 10K) and can see heap size growing. Thanks, Ankit