RE: Solr Cloud with 5 servers cluster failed due to Leader out of memory

2016-08-09 Thread Tim Chen
:apa...@elyograg.org] Sent: Monday, 8 August 2016 11:44 PM To: solr-user@lucene.apache.org Subject: Re: Solr Cloud with 5 servers cluster failed due to Leader out of memory On 8/7/2016 6:53 PM, Tim Chen wrote: > Exception in thread "http-bio-8983-exec-6571" java.lang.OutOfMemoryEr

Re: Solr Cloud with 5 servers cluster failed due to Leader out of memory

2016-08-09 Thread Shawn Heisey
On 8/8/2016 11:09 AM, Ritesh Kumar (Avanade) wrote: > This is great but where can I do this change in SOLR 6 as I have > implemented CDCR. In Solr 6, the chance of using Tomcat will be near zero, and the maxThreads setting in Solr's Jetty config should already be set to 1. If you're seeing th

RE: Solr Cloud with 5 servers cluster failed due to Leader out of memory

2016-08-08 Thread Ritesh Kumar (Avanade)
.com] Sent: 08 August 2016 21:30 To: solr-user Subject: Re: Solr Cloud with 5 servers cluster failed due to Leader out of memory Yeah, Shawn, but you, like, know something about Tomcat and actually provide useful advice ;) On Mon, Aug 8, 2016 at 6:44 AM, Shawn Heisey wrote: > On 8/7/2016 6:53

Re: Solr Cloud with 5 servers cluster failed due to Leader out of memory

2016-08-08 Thread Erick Erickson
Yeah, Shawn, but you, like, know something about Tomcat and actually provide useful advice ;) On Mon, Aug 8, 2016 at 6:44 AM, Shawn Heisey wrote: > On 8/7/2016 6:53 PM, Tim Chen wrote: >> Exception in thread "http-bio-8983-exec-6571" java.lang.OutOfMemoryError: >> unable to create new native thr

Re: Solr Cloud with 5 servers cluster failed due to Leader out of memory

2016-08-08 Thread Shawn Heisey
On 8/7/2016 6:53 PM, Tim Chen wrote: > Exception in thread "http-bio-8983-exec-6571" java.lang.OutOfMemoryError: > unable to create new native thread > at java.lang.Thread.start0(Native Method) > at java.lang.Thread.start(Thread.java:714) > at > java.util.concurrent.Thread

Re: Solr Cloud with 5 servers cluster failed due to Leader out of memory

2016-08-07 Thread Erick Erickson
bably look at consolidate 4 Solr servers into 2 bigger/better >> server - it gives more memory, and it cut down the replica the Leader needs >> to manage. >> >> Also, I may look into write a script to monitor the tomcat log and if there >> is OOM, kill tomcat, then restar

RE: Solr Cloud with 5 servers cluster failed due to Leader out of memory

2016-08-07 Thread Tim Chen
age- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Saturday, 6 August 2016 2:31 AM To: solr-user Subject: Re: Solr Cloud with 5 servers cluster failed due to Leader out of memory You don't really have to worry that much about memory consumed during indexing. The ramBufferSizeMB

RE: Solr Cloud with 5 servers cluster failed due to Leader out of memory

2016-08-07 Thread Tim Chen
.@gmail.com] Sent: Saturday, 6 August 2016 2:31 AM To: solr-user Subject: Re: Solr Cloud with 5 servers cluster failed due to Leader out of memory You don't really have to worry that much about memory consumed during indexing. The ramBufferSizeMB setting in solrconfig.xml pretty much limits

Re: Solr Cloud with 5 servers cluster failed due to Leader out of memory

2016-08-05 Thread Erick Erickson
s as well. > > Many Thanks guys. > > Cheers, > Tim > > -Original Message- > From: Shawn Heisey [mailto:apa...@elyograg.org] > Sent: Friday, 5 August 2016 4:55 PM > To: solr-user@lucene.apache.org > Subject: Re: Solr Cloud with 5 servers cluster failed due to Leader o

RE: Solr Cloud with 5 servers cluster failed due to Leader out of memory

2016-08-05 Thread Tim Chen
o:apa...@elyograg.org] Sent: Friday, 5 August 2016 4:55 PM To: solr-user@lucene.apache.org Subject: Re: Solr Cloud with 5 servers cluster failed due to Leader out of memory On 8/4/2016 8:14 PM, Tim Chen wrote: > Couple of thoughts: 1, If Leader goes down, it should just go down, > like dead down, so other

Re: Solr Cloud with 5 servers cluster failed due to Leader out of memory

2016-08-04 Thread Shawn Heisey
On 8/4/2016 8:14 PM, Tim Chen wrote: > Couple of thoughts: 1, If Leader goes down, it should just go down, > like dead down, so other servers can do the election and choose the > new leader. This at least avoids bringing down the whole cluster. Am I > right? Supplementing what Erick told you: Wh

Re: Solr Cloud with 5 servers cluster failed due to Leader out of memory

2016-08-04 Thread Erick Erickson
The fact that all the shards have the same leader is somewhat of a red herring. Until you get hundreds of shards (perhaps across a _lot_ of collections), the additional load on the leaders is hard to measure. If you really see this as a problem, consider the BALANCESHARDUNIQUE and REBALANCELEADERS

Solr Cloud with 5 servers cluster failed due to Leader out of memory

2016-08-04 Thread Tim Chen
Hi Guys, Me again. :) We have 5 Solr servers: 01 -04 running Solr version 4.10 and ZooKeeper service 05 running ZooKeeper only. JVM Max Memory set to 10G. We have around 20 collections, and for each collection, there are 4 shards, for each shard, there are 4 replica sitting across on 4 Solr se