Re: Understanding solr commit

2016-01-25 Thread Rahul Ramesh
Thank you Emir, Allesandro for the inputs. We use sematext for monitoring. We understand that Solr needs more memory but unfortunately we have to move towards an altogether new range of servers. As you say eventually, we will have to upgrade our servers. Thanks, Rahul On Mon, Jan 25, 2016 at 6:3

Re: Understanding solr commit

2016-01-25 Thread Emir Arnautovic
Hi Rahul, It is hard to tell without seeing metrics, but 8GB heap seems small for such setup - e.g. with indexing buffer of 32MB and 30 collections, it will eat almost 1GB memory. About commits, you can set auto commit to be more frequent (keep openSearcher=false) and add soft commits every 3 m

Re: Understanding solr commit

2016-01-25 Thread Rahul Ramesh
Can you give us bit more details about Solr heap parameters. Each node has 32Gb of RAM and we are using 8Gb for heap. Index size in each node is around 80Gb #of collections 30 Also can you give us info about auto commit (both hard and soft) you used when experienced OOM. 15000 15000 false soft

Re: Understanding solr commit

2016-01-25 Thread Emir Arnautovic
Hi Rahul, It is good that you commit only once, but not sure how external commits can do something auto commit cannot. Can you give us bit more details about Solr heap parameters. Running Solr on the edge of OOM is always risk of starting snowball effect and crashing entire cluster. Also can yo

Re: Understanding solr commit

2016-01-25 Thread Rahul Ramesh
Thanks for your replies. A bit more detail about our setup. The index size is close to 80Gb spread across 30 collections. The main memory available is around 32Gb. We are always in short of memory! Unfortunately we could not expand the memory as the server motherboard doesnt support it. We tried

Re: Understanding solr commit

2016-01-25 Thread Alessandro Benedetti
Let me answer in line : On 25 January 2016 at 11:02, Rahul Ramesh wrote: > We are facing some issue and we are finding it difficult to debug the > problem. We wanted to understand how solr commit works. > A background on our setup: > We have 3 Node Solr Cluster running in version 5.3.1. Its a i

Re: Understanding solr commit

2016-01-25 Thread Emir Arnautovic
Hi Rahul, If I got your mail right there is misconception of SolrCloud - nodes are infrastructure of cloud and collection is something that is "unit". So when you commit, you are committing changes you did on collection and SolrCloud will handle nodes. When you commit to three 3 nodes it is ac