Re: Solr caching clarifications

2013-07-15 Thread Manuel Le Normand
Great explanation and article. Yes, this buffer for merges seems very small, and still optimized. Thats impressive.

Re: Solr caching clarifications

2013-07-15 Thread Erick Erickson
Manuel: First off, anything that Mike McCandless says about low-level details should override anything I say. The memory savings he's talking about there are actually something he tutored me in once on a chat. The savings there, as I understand it, aren't huge. For large sets I think it's a 25% s

Re: Solr caching clarifications

2013-07-14 Thread Manuel Le Normand
Alright, thanks Erick. For the question about memory usage of merges, taken from Mike McCandless Blog The big thing that stays in RAM is a logical int[] mapping old docIDs to new docIDs, but in more recent versions of Lucene (4.x) we use a much more efficient structure than a simple int[] ... see

Re: Solr caching clarifications

2013-07-12 Thread Erick Erickson
Inline On Thu, Jul 11, 2013 at 8:36 AM, Manuel Le Normand wrote: > Hello, > As a result of frequent java OOM exceptions, I try to investigate more into > the solr jvm memory heap usage. > Please correct me if I am mistaking, this is my understanding of usages for > the heap (per replica on a solr

Solr caching clarifications

2013-07-11 Thread Manuel Le Normand
Hello, As a result of frequent java OOM exceptions, I try to investigate more into the solr jvm memory heap usage. Please correct me if I am mistaking, this is my understanding of usages for the heap (per replica on a solr instance): 1. Buffers for indexing - bounded by ramBufferSize 2. Solr caches