Yeah, I second Mark's suggestion on reducing the stack size. The default on
modern 64-bit boxes is usually 1024KB which adds up to a lot when you're
running 5000 cores (5000 * 2 = 1MB). I think the zk register thread can
be pooled together but the search threads can't be because we'd run into
d
>
> so you might still end up with these out of threads issue again.
You can also generally drop the stack size (Xss) quite a bit to to handle
more threads.
Beyond that, there are some thread pools you can configure. However, until
we fix the distrib deadlock issue, you don't want to drop the co
apache.org
Subject: Re: Scaling to large Number of Collections
What is your access pattern? By that I mean do all the cores need to be
searched at the same time or is it reasonable for them to be loaded on
demand? This latter would impose the penalty of the first time a collection
was accessed there
What is your access pattern? By that I mean do all the cores need to be
searched at the same time or is it reasonable for them to be loaded on
demand? This latter would impose the penalty of the first time a collection
was accessed there would be a delay while the core loaded. I suppose I'm
asking
On 31 Aug 2014 13:24, "Mark Miller" wrote:
>
>
> > On Aug 31, 2014, at 4:04 AM, Christoph Schmidt <
christoph.schm...@moresophy.de> wrote:
> >
> > we see at least two problems when scaling to large number of
collections. I would like to ask the community, if they are known and maybe
already addres
On 8/31/2014 8:58 AM, Joseph Obernberger wrote:
> Could you add another field(s) to your application and use that instead of
> creating collections/cores? When you execute a search, instead of picking
> a core, just search a single large core but add in a field which contains
> some core ID.
This
Could you add another field(s) to your application and use that instead of
creating collections/cores? When you execute a search, instead of picking
a core, just search a single large core but add in a field which contains
some core ID.
-Joe
http://www.lovehorsepower.com
On Sun, Aug 31, 2014 at
> On Aug 31, 2014, at 4:04 AM, Christoph Schmidt
> wrote:
>
> we see at least two problems when scaling to large number of collections. I
> would like to ask the community, if they are known and maybe already
> addressed in development:
> We have a SolrCloud running with the following numbers
How are the 5 servers arranged in terms of shards and replicas? 5 shards
with 1 replica each, 1 shard with 5 replicas, 2 shards with 2 and 3
replicas, or... what?
How big is each collection? The key strength of SolrCloud is scaling large
collections via shards, NOT scaling large numbers of col
