If you're committing that rapidly then you're correct, filter caching
may not be a good fit. The entire _point_ of
filter caching is to increase performance of subsequent executions of
the exact same fq clause. But if you're
throwing them away every second there's little/no benefit.
You really have two choices here
1> lengthen out the commit interval. Frankly, 1 second commit
intervals are rarely necessary despite what
your product manager says. Really, check this requirement out.
2> disable caches.
Autowarming is potentially useful here, but if your filter queries are
taking on the order of a second and
you're committing every second then autowarming takes too long to help.
Best,
Erick
On Wed, Aug 19, 2015 at 12:26 AM, Mikhail Khludnev
<mkhlud...@griddynamics.com> wrote:
> Maulin,
> Did you check performance with segmented filters which I advised recently?
>
> On Wed, Aug 19, 2015 at 10:24 AM, Maulin Rathod <mrat...@asite.com> wrote:
>
>> As per my understanding caches are flushed every time when add new
>> document to collection (we do soft commit at every 1 sec to make newly
>> added document available for search). Due to which it is not effectively
>> uses cache and hence it slow every time in our case.
>>
>> -----Original Message-----
>> From: Toke Eskildsen [mailto:t...@statsbiblioteket.dk]
>> Sent: 19 August 2015 12:16
>> To: solr-user@lucene.apache.org
>> Subject: Re: Performance issue with FILTER QUERY
>>
>> On Wed, 2015-08-19 at 05:55 +0000, Maulin Rathod wrote:
>> > SLOW WITH FILTER QUERY (takes more than 1 second)
>> > ============================================
>> >
>> > q=+recipient_id:(4042) AND project_id:(332) AND resource_id:(13332247
>> > 13332245 13332243 13332241 13332239) AND entity_type:(2) AND
>> > -action_id:(20 32) ==> This returns 5 records
>> > fq=+action_status:(0) AND is_active:(true) ==> This Filter Query
>> > returns 9432252 records
>>
>> The fq is evaluated independently of the q: For the fq a bitset is
>> allocated, filled and stored in cache. Then the q is evaluated and the two
>> bitsets are merged.
>>
>> Next time you use the same fq, it should be cached (if you have caching
>> enabled) and be a lot faster.
>>
>>
>> Also, if you ran your two tests right after each other, the second one
>> benefits from disk caching. If you had executed them in reverse order, the
>> q+fq might have been the fastest one.
>>
>> - Toke Eskildsen, State and University Library, Denmark
>>
>>
>>
>
>
> --
> Sincerely yours
> Mikhail Khludnev
> Principal Engineer,
> Grid Dynamics
>
> <http://www.griddynamics.com>
> <mkhlud...@griddynamics.com>
