Thanks Erick. The links you provided are invaluable.
Here are our commit settings. Since we have NRT search, softCommit is set
to 1000s which explains why cache is constantly invalidated.
<autoCommit>
<maxTime>600000</maxTime>
<openSearcher>false</openSearcher>
</autoCommit>
<autoSoftCommit>
<maxTime>1000</maxTime>
</autoSoftCommit>
With constant cache invalidation it becomes almost impossible to get better
response times. Is the only to solve this is to fine tune softCommit
settings?
On Fri, Mar 7, 2014 at 6:17 PM, Erick Erickson <erickerick...@gmail.com>wrote:
> OK, something is not right here. What are
> your autocommit settings? What you pasted
> above looks like you're looking at a searcher that
> has _just_ opened, which would mean either
> 1> you just had a hard commit with openSearcher=false happen
> or
> 2> you just had a soft commit happen
>
> In either case, the cache is thrown out. That said, if you
> have autowarming for the cache set up you should be
> seeing some hits eventually.
>
> The top part is the _current_ searcher. The cumulative_*
> is all the cache results since the application started.
>
> A couple of blogs:
>
>
> http://searchhub.org/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
>
> http://searchhub.org/2012/02/23/date-math-now-and-filter-queries/
>
> I'm going to guess that you have soft commits or hard commits
> with openSearcher=true set to a very short interval and are
> having your filter caches invalidated very frequently, and that is
> misleading you, but that's just a guess.
>
> Best,
> Erick
>
>
>
> On Thu, Mar 6, 2014 at 9:32 PM, Vijay Kokatnur <kokatnur.vi...@gmail.com>
> wrote:
> > My initial approach was to use filter cache static fields. However when
> > filter query is used, every query after the first has the same response
> > time as the first. For instance, when cache is enabled in the query
> under
> > review, response time shoots up to 4-5secs and stays there.
> >
> > We are using default filter cache settings provided with 4.5.0
> > distribution.
> >
> > Current Filter Cache stats :
> >
> > lookups:0
> > hits:0
> > hitratio:0
> > inserts:0
> > evictions:0
> > size:0
> > warmupTime:0
> > cumulative_lookups:17135
> > cumulative_hits:2465
> > cumulative_hitratio:0.14
> > cumulative_inserts:14670
> > cumulative_evictions:0
> >
> > I did not find what cumulative_* fields mean
> > here<http://wiki.apache.org/solr/SolrAdminStats> ,
> > but it looks like nothing is being cached with fq as hit ratio is 0.
> >
> > Any idea whats happening?
> >
> >
> >
> > On Thu, Mar 6, 2014 at 2:41 PM, Ahmet Arslan <iori...@yahoo.com> wrote:
> >
> >> Hoss,
> >>
> >> Thanks for the correction. I missed the /DAY part and thought as it was
> >> StartDate:[NOW TO NOW+1YEAR]
> >>
> >> Ahmet
> >>
> >>
> >> On Friday, March 7, 2014 12:33 AM, Chris Hostetter <
> >> hossman_luc...@fucit.org> wrote:
> >>
> >> : That did the trick Ahmet. The first response was around 200ms, but
> the
> >> : subsequent queries were around 2-5ms.
> >>
> >> Are you really sure you want "cache=false" on all of those filters?
> >>
> >> While the "ClientID:4" query may by something that cahnges significantly
> >> enough in every query to not be useful to cache, i suspect you'd find a
> >> lot of value in going ahead and caching those Status:Booked and
> >> StartDate:[NOW/DAY TO NOW/DAY+1YEAR] clauses ... the first query to hit
> >> them might be "slower" but ever query after that should be fairly fast
> --
> >> and if you really need them to *always* be fast, configure them as
> static
> >> newSeracher warming queries (or make sure you have autowarming on.
> >>
> >> It also look like you forgot the "StartDate:" part of your range query
> in
> >> your last test...
> >>
> >> : &fq={!cache=false cost=50}[NOW/DAY TO NOW/DAY+1YEAR]
> >>
> >> And one finally comment just to make sure it doesn't slip throug hthe
> >> cracks....
> >>
> >>
> >> : > > Since your range query has NOW in it, it won't be cached
> >> meaningfully.
> >>
> >> this is not applicable. the use of "NOW" in a range query doesn't mean
> >> that it can't be cached -- the problem is anytime you use really precise
> >> dates (or numeric values) that *change* in every query.
> >>
> >> if your range query uses "NOW" as a lower/upper end point, then it calls
> >> in that "really precise dates" situation -- but for this user, who is
> >> specifically rounding his dates to hte nearest day, that advice isn't
> >> really applicable -- the date range queries can be cached & reused for
> an
> >> entire day.
> >>
> >>
> >>
> >> -Hoss
> >> http://www.lucidworks.com/
> >>
> >>
>
