Whenever I see one of this "big" query filters, my first thought is that
there is something wrong with the application data model.
Where do the long list of IDs come from? Somebody must be generating and/or
storing them, right? Why not store them in Solr, right in the data model?
Maybe store them as a separate collection and do a join operation.
How unique are they? I mean, a large number of large filters is not going to
be very efficient.
In any case, start by looking at your data model.
-- Jack Krupansky
-----Original Message-----
From: Igor Kustov
Sent: Saturday, June 15, 2013 6:49 PM
To: solr-user@lucene.apache.org
Subject: Solr large boolean filter
I know i'm not the first one with this problem.
I'm currently using solr 4.2.1 with approximately 10 mln documents in the
index.
The index is updated frequently.
The filter_query is just a one big boolean or query by id.
fq=id:(1 2 3 4 ... 50950)
ids list is always different and not sequential.
The problem is that query performance not so well, as you can imagine.
In some particular cases i'm able to do filtering based on different fields,
but in some cases (like 30-40% of all queries) i'm still end up with this
large id filter.
I'm looking for the ways to improve this query performance.
It doesn't seem like solr join could be applied there.
Another option that I found is to somehow use Lucene FieldCacheTermsFilter.
Does it worth a try?
Maybe i've missed some other options?
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-large-boolean-filter-tp4070747.html
Sent from the Solr - User mailing list archive at Nabble.com.
