There was a major Lucene change in filter handling from Solr 1.3 to Solr 1.4. They are much much faster in 1.4. Really Lucene 2.4.1 to Lucene 2.9.2. The filter is now consulted much earlier in the search process, thus weeding out many more documents early.
It sounds like in Solr 1.3, you should only use filter queries for queries with large document sets. On Wed, Aug 11, 2010 at 12:21 PM, Bargar, Matthew B <matthew.bar...@verizonwireless.com> wrote: > The search with the filter takes longer than a search for the same term > but no filter after repeated searches, after the cache should have come > into play. To be more specific, this happens on filters that exclude > very few results from the overall set. > > For instance, type:video returns few results and as one would expect, > returns much quicker than a search without that filter. > > -type:video, on the other hand returns a lot of results and excludes > very few, and actually takes longer than a search without any filter at > all. > > Is this what one might expect when using a filter that excludes few > results, or does it still seem like something strange might be > happening? > > Thanks, > Matt > > -----Original Message----- > From: Geert-Jan Brits [mailto:gbr...@gmail.com] > Sent: Wednesday, August 11, 2010 2:55 PM > To: solr-user@lucene.apache.org > Subject: Re: Filter Performance in Solr 1.3 > > fq's are the preferred way to use for filtering when the same filter is > often used. (since the filter-set can be cached seperately) . > > as to your direct question: >> My question is whether there is anything that can be done in 1.3 to > help alleviate the problem, before upgrading to 1.4? > > I don't think so (perhaps some patches that I'm not aware of) . > > When are you seeing increased search time? > > is it the first time the filter is used? If that's the case: that's > logical since the filter needs to be build. > (fq)-filters only show their strength (as said above) when you use them > repeatedly. > > If on the other hand you're seeing slower repsonse times with a > fq-filter applied all the time, then the same queries without the > fq-filter, there must be something strange going on since this really > shouldn't happen in normal situations. > > Geert-Jan > > > > > > 2010/8/11 Bargar, Matthew B <matthew.bar...@verizonwireless.com> > >> Hi there, I have a question about filter (fq) performance in Solr 1.3. >> After doing some testing it seems as though adding a filter increases >> search time. From what I've read here >> http://www.derivante.com/2009/06/23/solr-filtering-performance-increas >> e/ >> >> and here >> http://www.lucidimagination.com/blog/2009/05/27/filtered-query-perform >> an >> ce-increases-for-solr-14/ >> >> it seems as though upgrading to 1.4 would solve this problem. My >> question is whether there is anything that can be done in 1.3 to help >> alleviate the problem, before upgrading to 1.4? It becomes an issue >> because the majority of searches that are done on our site need some >> content type excluded or filtered for. Does it make sense to use the >> fq parameter in this way, or is there some better approach since >> filters are almost always used? >> >> Thank you! >> > -- Lance Norskog goks...@gmail.com
