Re: DocValue field & commit

2020-03-31 Thread Revas
Hi Erick, Thanks. We do have NRT requirement in our application that updates be immediately visible. We do have constant updates. The push is for even faster visibility but we are holding off at 2 secs soft-commit for now. What I am not able to understand is that as per query debugging, the facet

Re: DocValue field & commit

2020-03-30 Thread Erick Erickson
Oh dear. Your autowarming is almost, but not quite totally, useless given your 2 second soft commit interval. See: https://lucidworks.com/post/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ So autowarming is probably not a cure, when you originally said “commit” I was assumin

Re: DocValue field & commit

2020-03-30 Thread Revas
Thanks, Erick, The process time execution based on debugQuery between the query and facets is as follows query 10ms facets 4900ms since max time is spent on facet processing (docValues enabled), query and filter cache do no apply to this, correct? - Autowarm count is at 32 for both and aut

Re: DocValue field & commit

2020-03-30 Thread Revas
Correcting some typos ... Thanks, Eric. 1) We are using dynamic string field for faceting where indexing =false and stored=false . By default docValues are enabled for primitive fields (solr 6.6.), so not explicitly defined in schema. Do you think its wrong assumption? Also I do not see this fiel

Re: DocValue field & commit

2020-03-30 Thread Erick Erickson
OK, sounds like docValues is set. Sure, in solrconfig.xml, there are two sections “firstSearcher” and “newSearcher”. These are queries (or lists of queries) that are fired as part of autowarming when Solr is first started (firstSearcher) or when a commit happens that opens a new searcher (newSear

Re: DocValue field & commit

2020-03-30 Thread Revas
Thanks, Eric. 1) We are using dynamic string field for faceting where indexing =false and stored=false . By default docValues are enabled for primitive fields (solr 6.6.), so not explicitly defined in schema. Do you think its wrong assumption? Also I do not this field listed in feild cache, but do

Re: DocValue field & commit

2020-03-30 Thread Erick Erickson
Response spikes after commits are almost always something to do with autowarming or docValues being set to false. So here’s what I’d look at, in order. 1> are the fields used defined with docValues=true? They should be. With this much variance it sounds like you don’t have that value set. You’ll h

DocValue field & commit

2020-03-30 Thread sujatha arun
A facet heavy query which uses docValue fields for faceting returns about 5k results executes between 10ms to 5 secs and the 5 secs time seems to coincide with after a hard commit. Does that have any relation? Why the fluctuation in execution time? Thanks, Revas