On 6/3/2014 2:44 PM, Brett Hoerner wrote:
> If I run a query like this,
>
> fq=text:lol
> fq=created_at_tdid:[1400544000 TO 1400630400]
>
> It takes about 6 seconds. Following queries take only 50ms or less, as
> expected because my fqs are cached.
>
> However, if I change the query to not cache my big range query:
>
> fq=text:lol
> fq={!cache=false}created_at_tdid:[1400544000 TO 1400630400]
>
> It takes 2 seconds every time, which is a much better experience for my
> "first query for that range."
>
> What's odd to me is that I would expect both of these (first) queries to
> have to do the same amount of work, expect the first one stuffs the
> resulting bitset into a map at the end... which seems to have a 4 second
> overhead?
>
> Here's my filterCache from solrconfig:
>
> <filterCache class="solr.FastLRUCache"
> size="64"
> initialSize="64"
> autowarmCount="32"/>
I think that probably depends on how many documents you have in the
single index/shard. If you have one hundred million documents stored in
the Lucene index, then each filter entry is 12500000 bytes (11.92MB) in
size - it is a bitset representing every document and whether it is
included or excluded. That data would need to be gathered and copied
into the cache. I suspect that it's the gathering that takes the most
time ... several megabytes of memory is not very much for a modern
processor to copy.
As for how long this takes, I actually have no idea. You have two
filters here, so it would need to do everything twice.
Thanks,
Shawn
