Re: SolrCloud admin UI core/stats showing commit count even without no explicit commit

2012-08-29 Thread Erick Erickson
Been busy the last couple of days, sorry it took so long to get back You have basically 2 questions: About the 80% rate. It's not quite clear. What I meant was say you have 20M docs on a server. You push it until you max out the QPS rate, say that's 100 queries/second. Now, configure your loa

Re: SolrCloud admin UI core/stats showing commit count even without no explicit commit

2012-08-27 Thread Srikanth S
Thanks for your response Erick. Your explanation seems to make sense for the commit count. But I guess the UI needs to be fixed. Regarding the performance, I went through your blog (nicely written btw (and good links to other interesting blogs too)). I didn't realize that everything that is index

Re: SolrCloud admin UI core/stats showing commit count even without no explicit commit

2012-08-27 Thread Erick Erickson
The autocommits are about what I'd expect. 17 hours == 102 ten minute blocks, which is roughly your 115 autocommits. I'm _guessing_ that the total commits are a combination of soft and hard. You'll have 20,400 soft commits in that time frame, so this works as a rough estimate And SolrJ doesn't

Re: SolrCloud admin UI core/stats showing commit count even without no explicit commit

2012-08-27 Thread Srikanth S
As explained below, the servers are started with a minimum heap size of 3G and max heap size of 4G, though I've never seen the heap grow more than 3G. On Mon, Aug 27, 2012 at 3:02 AM, Lance Norskog wrote: > How much memory is allocated? There is a feature in modern Unix > systems called 'Large P

Re: SolrCloud admin UI core/stats showing commit count even without no explicit commit

2012-08-26 Thread Lance Norskog
How much memory is allocated? There is a feature in modern Unix systems called 'Large Pages' or 'Huge Pages'. This is an operating system feature to run very large processes with better virtual memory tracking strategies inside the CPU RAM subsystem. Search for 'Large Pages' and 'Translation Lookas