Disclaimer: I am not actually familiar with the solr code, all of the below is extrapolation from being pretty familiar with Solr's behavior.
Yeah, it would be nice, but it would be a lot harder to code for solr. Right now, the thing putting and retrieving entries into/from the filter cache doesn't really need to parse the query at all. It just takes the whole query and uses it (effectively) for a cache key. Keep in mind that Solr has pluggable query parsers, and the fq can (quite usefully!) be used with any query parser, not just lucene. lucene, dismax, field, raw, a few others out of the box, others not officially part of solr but that users might write and use with their solr. Query parsers can be in use (and work with filter cache) that didn't even exist when the filter caching logic was written. This is actually a very useful feature -- if there's behavior that's possible with lucene but not supported in a convenient way (or at all) by Solr API, you can write a query parser to do it yourself if you need to -- and your query parser will plug right in, and all other Solr features (such as filter caching!) will still work fine with it. So to get the filter caching to somehow go inside the query and cache and retrieve parts of it -- it would probably really need to be something each query parser were responsible for -- storing and retrieving elements from the filter cache as part of it's ordinary query parsing behavior -- but only when it was inside an fq, not a q, which I'm not sure the query parser even knows right now. Right now I think the query parser doesn't even have to know about the filter cache -- if an fq is retrieved from cache, then it doesn't even make it to the query parser. So yeah, it would be useful if seperate components of an fq query could be cached seperately -- but it would also be a lot more complicated. But I'm sure nobody would mind seeing a patch if you want to figure it out. :) ________________________________________ From: Em [mailformailingli...@yahoo.de] Sent: Thursday, January 06, 2011 2:36 AM To: solr-user@lucene.apache.org Subject: Re: (FQ) Filter Query Caching Differences with OR and AND? Thank you Jonathan. "fq=foo:bar&fq=foo:baz" seems to be the better alternative for "fq=foo:bar AND foo:baz" if "foo:bar" and "foo:baz" were often used in different combinations (not always together). However, in most of the usecases I can think of, an "fq=foo:bar OR foo:baz"-behaviour is expected and it would be nice if this fq would benefit from a cached "fq=foo:bar". I can imagine why this is not the case, if only one of two fq-clauses were cached. However, when "foo:bar" and "foo:baz" were cached seperately, why not benefiting from them when a "fq=foo:bar OR foo:baz" or "fq=foo:bar AND foo:baz" is requested? Who is responsible for putting fq's in the filterCache? I think one has to modify the logic of that class do benefit from already cached but recombined filterCaches. This would have a little bit less performance than caching the entire "foo:bar AND foo:baz" BitVector, since you need to reproduce one for that special use-case, but I think the usage of the cache is far more efficient, if "foo:bar" and "foo:baz" occur very frequently but "foo:bar AND foo:baz" do not. What do you think? Regards Jonathan Rochkind wrote: > > Each 'fq' clause is it's own cache key. > > 1. fq=foo:bar OR foo:baz > => one entry in filter cache > > 2. fq=foo:bar&fq=foo:baz > => two entries in filter cache, will not use cached entry from #1 > > 3. fq=foo:bar > => One entry, will use cached entry from #2 > > 4. fq=foo:bar > => One entry, will use cached entry from #2. > > So if you do queries in succession using each of those four fq's in > order, you will wind up with 3 entries in the cache. > > Note that "fq=foo:bar OR foo:baz" is not semantically identical to > "fq=foo&fq=bar". Rather that latter is semantically identical to > "fq=foo:bar AND foo:baz". But "fq=foo&fq=bar" will be two cache > entries, and "fq=foo:bar AND foo:baz" will be one cache entry, and the > two won't share any cache entries. > > > On 1/5/2011 3:17 PM, Em wrote: >> Hi, >> >> while reading through some information on the list and in the wiki, i >> found >> out that something is missing: >> >> When I specify a filter queries like this >> >> fq=foo:bar OR foo:baz >> or >> fq=foo:bar&fq=foo:baz >> or >> fq=foo:bar >> or >> fq=foo:baz >> >> How many filter query entries will be cached? >> Two, since there are two filters (foo:bar, foo:baz) or 3, since there are >> three different combinations (foo:bar OR foo:baz, foo:bar, foo:baz)? >> >> Thank you! > > Jonathan Rochkind wrote: > > Each 'fq' clause is it's own cache key. > > 1. fq=foo:bar OR foo:baz > => one entry in filter cache > > 2. fq=foo:bar&fq=foo:baz > => two entries in filter cache, will not use cached entry from #1 > > 3. fq=foo:bar > => One entry, will use cached entry from #2 > > 4. fq=foo:bar > => One entry, will use cached entry from #2. > > So if you do queries in succession using each of those four fq's in > order, you will wind up with 3 entries in the cache. > > Note that "fq=foo:bar OR foo:baz" is not semantically identical to > "fq=foo&fq=bar". Rather that latter is semantically identical to > "fq=foo:bar AND foo:baz". But "fq=foo&fq=bar" will be two cache > entries, and "fq=foo:bar AND foo:baz" will be one cache entry, and the > two won't share any cache entries. > > > On 1/5/2011 3:17 PM, Em wrote: >> Hi, >> >> while reading through some information on the list and in the wiki, i >> found >> out that something is missing: >> >> When I specify a filter queries like this >> >> fq=foo:bar OR foo:baz >> or >> fq=foo:bar&fq=foo:baz >> or >> fq=foo:bar >> or >> fq=foo:baz >> >> How many filter query entries will be cached? >> Two, since there are two filters (foo:bar, foo:baz) or 3, since there are >> three different combinations (foo:bar OR foo:baz, foo:bar, foo:baz)? >> >> Thank you! > > -- View this message in context: http://lucene.472066.n3.nabble.com/FQ-Filter-Query-Caching-Differences-with-OR-and-AND-tp2201004p2204235.html Sent from the Solr - User mailing list archive at Nabble.com.
