Re: a core for every user, lots of users... are there issues

2013-12-04 Thread Erick Erickson
I don't know of anyone who's tried and failed to combine transient cores and SolrCloud. I also don't know of anyone who's tried and succeeded. I'm saying that the transient core stuff has been thoroughly tested in non-cloud mode. And people have been working with it for a couple of releases now. I

Re: a core for every user, lots of users... are there issues

2013-12-04 Thread hank williams
Oh my... when you say "I don't know anyone who's combined the two." do you mean that those that have tried have failed or that no one has gotten around to trying? It sounds like you are saying you have some specific knowledge that right now these wont work, otherwise you wouldnt say "committers wil

Re: a core for every user, lots of users... are there issues

2013-12-04 Thread Erick Erickson
Hank: I should add that lots of cores and SolrCloud aren't guaranteed to play nice together. I think some of the committers will be addressing this sometime soon. I'm not saying that this will certainly fail, OTOH I don't know anyone who's combined the two. Erick On Wed, Dec 4, 2013 at 3:18 PM

Re: a core for every user, lots of users... are there issues

2013-12-04 Thread hank williams
Super helpful. Thanks. On Wed, Dec 4, 2013 at 2:53 PM, Shawn Heisey wrote: > On 12/4/2013 12:34 PM, hank williams wrote: > >> Ok one more simple question. We just upgraded to 4.6 from 4.2. In 4.2 we >> were *trying* to use the rest API function "create" to create cores >> without >> having to m

Re: a core for every user, lots of users... are there issues

2013-12-04 Thread Shawn Heisey
On 12/4/2013 12:34 PM, hank williams wrote: Ok one more simple question. We just upgraded to 4.6 from 4.2. In 4.2 we were *trying* to use the rest API function "create" to create cores without having to manually mess with files on the server. Is this what "create" was supposed to do? If so it was

Re: a core for every user, lots of users... are there issues

2013-12-04 Thread hank williams
Ok one more simple question. We just upgraded to 4.6 from 4.2. In 4.2 we were *trying* to use the rest API function "create" to create cores without having to manually mess with files on the server. Is this what "create" was supposed to do? If so it was borken or we werent using it right. In any ca

Re: a core for every user, lots of users... are there issues

2013-12-03 Thread Erick Erickson
bq: Do you have any sense of what a good upper limit might be, or how we might figure that out? As always, "it depends" (tm). And the biggest thing it depends upon is the number of simultaneous users you have and the size of their indexes. And we've arrived at the black box of estimating size agai

Re: a core for every user, lots of users... are there issues

2013-12-03 Thread hank williams
Sorry, I see that we are up to solr 4.6. I missed that. On Tue, Dec 3, 2013 at 3:53 PM, hank williams wrote: > Also, I see that the "lotsofcores" stuff is for solr 4.4 and above. What > is the state of the 4.4 codebase? Could we start using it now? Is it safe? > > > On Tue, Dec 3, 2013 at 3:33

Re: a core for every user, lots of users... are there issues

2013-12-03 Thread hank williams
Also, I see that the "lotsofcores" stuff is for solr 4.4 and above. What is the state of the 4.4 codebase? Could we start using it now? Is it safe? On Tue, Dec 3, 2013 at 3:33 PM, hank williams wrote: > > > > On Tue, Dec 3, 2013 at 3:20 PM, Erick Erickson wrote: > >> You probably want to look a

Re: a core for every user, lots of users... are there issues

2013-12-03 Thread hank williams
On Tue, Dec 3, 2013 at 3:20 PM, Erick Erickson wrote: > You probably want to look at "transient cores", see: > http://wiki.apache.org/solr/LotsOfCores > > But millions will be "interesting" for a single node, you must have some > kind of partitioning in mind? > > Wow. Thanks for that great link. Y

Re: a core for every user, lots of users... are there issues

2013-12-03 Thread Erick Erickson
You probably want to look at "transient cores", see: http://wiki.apache.org/solr/LotsOfCores But millions will be "interesting" for a single node, you must have some kind of partitioning in mind? Best, Erick On Tue, Dec 3, 2013 at 2:38 PM, hank williams wrote: > We are building a system wher

a core for every user, lots of users... are there issues

2013-12-03 Thread hank williams
We are building a system where there is a core for every user. There will be many tens or perhaps ultimately hundreds of thousands or millions of users. We do not need each of those users to have “warm” data in memory. In fact doing so would consume lots of memory unnecessarily, for users that mig