SolrCloud running away with resources

2013-05-30 Thread ltenny
I've set up a simple 10 node, 5 shard SolrCloud 4.3. I'm pushing just a few thousand documents into it. What I'm doing is rather write intensive 100x...more writes than reads. I've noticed that there seems to be an unbounded use of resources. I'm seeing a steadily increasing number of network co

Re: SolrCloud running away with resources

2013-05-31 Thread ltenny
Thanks! I've found the cause for this problem was the hardware load balancer (F5 LTM) was creating thousands of connections. So it turns out that it had nothing to do with SolrCloud. However, now I have another problem for which I'll create another post. -- View this message in context: ht

Getting tons of EofException with jetty/SolrCloud

2013-05-31 Thread ltenny
I have a 10 node, 4.3 SolrCloud with 5 shards. It's a heavy write (100x more writes than reads) environment. After less than 10k docs in 20 minutes or so I get tons of EofExceptions from what appears to be the synchronization traffic between the nodes. When I do queries during this exception sto

Re: Getting tons of EofException with jetty/SolrCloud

2013-05-31 Thread ltenny
15000 false 1000 I think these are close to the default values...not sure if I changed them. These mean a hard commit every 15 seconds...right? Seems sort of reasonable since we get a few hundred doc inserts in 15 seconds. Not sure...any advice is very welcome. -- View this message i

Re: Getting tons of EofException with jetty/SolrCloud

2013-05-31 Thread ltenny
Wonderful discussion, but it seems that the exact values here should only affect the freshness of the search results and the growth of the logs. What I have going on in a very simple, 10 node SolrCloud with quite low insert rates (10K docs/20 minutes) absolutely kills the cloud in the first 20 min