Re: Solr Cluster management having too many cores

2014-08-08 Thread Anshum Gupta
You're correct Mukesh, that's the JIRA with pretty much all of that discussion. On Fri, Aug 8, 2014 at 8:44 PM, Mukesh Jha wrote: > Looks like https://issues.apache.org/jira/browse/SOLR-5473 is the story :) > > > On Fri, Aug 8, 2014 at 9:30 PM, Mukesh Jha wrote: > >> Hey *Shawn*, *Erik*, >> >> I

Re: Solr Cluster management having too many cores

2014-08-08 Thread Mukesh Jha
Looks like https://issues.apache.org/jira/browse/SOLR-5473 is the story :) On Fri, Aug 8, 2014 at 9:30 PM, Mukesh Jha wrote: > Hey *Shawn*, *Erik*, > > I's wondering if there is a JIRA story for splitting the current > clusterstate.json to collection specific clusterstate config that I can > tr

Re: Solr Cluster management having too many cores

2014-08-08 Thread Mukesh Jha
Hey *Shawn*, *Erik*, I's wondering if there is a JIRA story for splitting the current clusterstate.json to collection specific clusterstate config that I can track. I looked around a bit but couldn't get my hands on anything useful on that. On Mon, Apr 28, 2014 at 7:43 AM, Shawn Heisey wrote:

Re: Solr Cluster management having too many cores

2014-04-28 Thread Shawn Heisey
On 4/28/2014 5:05 AM, Mukesh Jha wrote: > Thanks Erik, > > Sounds about right. > > BTW how long can I keep adding collections i.e. can I keep 5/10 years data > like this? > > Also what do you think of bullet 2) of having collection specific > configurations in zookeeper? Regarding bullet 2, the

Re: Solr Cluster management having too many cores

2014-04-28 Thread Mukesh Jha
Thanks Erik, Sounds about right. BTW how long can I keep adding collections i.e. can I keep 5/10 years data like this? Also what do you think of bullet 2) of having collection specific configurations in zookeeper? On Fri, Apr 25, 2014 at 11:44 PM, Erick Erickson wrote: > So you're talking abo

Re: Solr Cluster management having too many cores

2014-04-25 Thread Erick Erickson
So you're talking about 700 or so collections. That should be do-able, especially as Solr is rapidly evolving to handle more and more collections and there's two years for that to happen. The aging out bit is manual (well, you'd script it I suppose). So every day there'd be a script that ran and "

Re: Solr Cluster management having too many cores

2014-04-25 Thread Mukesh Jha
Thanks for quick reply Erik, I want to keep my collections till I run out of hardware, which is at least a couple of years worth data. I'd like to know more on ageing out aliases, did a quick search but didn't find much. On Fri, Apr 25, 2014 at 9:45 PM, Erick Erickson wrote: > Hmmm, tell us a l

Re: Solr Cluster management having too many cores

2014-04-25 Thread Erick Erickson
Hmmm, tell us a little more about your use-case. In particular, how long do you need to keep the data around? Days? Months? Years? Because if you only need to keep the data for a specified period, you can use the collection aliasing process to age-out collections and keep the number of cores from

Solr Cluster management having too many cores

2014-04-25 Thread Mukesh Jha
Hi Experts, I need to divide my indexes based on hour/day with each index having ~50-80 GB data & ~50-80 mill docs, so I'm planning to create daily collection with names e.g. *sample_colledction__mm_dd_hh.* I'll also create an alias *sample_collection* and update it whenever I will create a ne