So you're talking about 700 or so collections. That should be do-able, especially as Solr is rapidly evolving to handle more and more collections and there's two years for that to happen.
The aging out bit is manual (well, you'd script it I suppose). So every day there'd be a script that ran and "just knew" the right collection to change the alias on, there's nothing automatic yet. Best, Erick On Fri, Apr 25, 2014 at 9:37 AM, Mukesh Jha <me.mukesh....@gmail.com> wrote: > Thanks for quick reply Erik, > > I want to keep my collections till I run out of hardware, which is at least > a couple of years worth data. > I'd like to know more on ageing out aliases, did a quick search but didn't > find much. > > > On Fri, Apr 25, 2014 at 9:45 PM, Erick Erickson > <erickerick...@gmail.com>wrote: > >> Hmmm, tell us a little more about your use-case. In particular, how >> long do you need to keep the data around? Days? Months? Years? >> >> Because if you only need to keep the data for a specified period, you >> can use the collection aliasing process to age-out collections and >> keep the number of cores from growing too large. >> >> Best, >> Erick >> >> On Fri, Apr 25, 2014 at 6:49 AM, Mukesh Jha <me.mukesh....@gmail.com> >> wrote: >> > Hi Experts, >> > >> > I need to divide my indexes based on hour/day with each index having >> ~50-80 >> > GB data & ~50-80 mill docs, so I'm planning to create daily collection >> with >> > names e.g. *sample_colledction_yyyy_mm_dd_hh.* >> > I'll also create an alias *sample_collection* and update it whenever I >> will >> > create a new collection so that the entire data set is searchable. >> > >> > I've a couple of question on the above design >> > 1) How far can it scale? As my collections will increase (so will the >> > shards & replicas) do we have a breaking point when adding more/searching >> > will become an issue? >> > 2) As my cluster will grow because of huge number of collections the >> > clusterstate.json file present in zookeeper will grow too, won't this be >> a >> > limiting factor? If so instead of storing all this info in one >> > clusterstate.json file shouldn't Solr save cluster specific details in >> this >> > file & have collection specific config files present on zookeeper? >> > 3) How can I easily manage all these collections? Do we have Java >> Coreadmin >> > API's available. I cannot find much documented on it. >> > >> > -- >> > Txz, >> > >> > *Mukesh Jha <me.mukesh....@gmail.com>* >> > > > > -- > > > Thanks & Regards, > > *Mukesh Jha <me.mukesh....@gmail.com>*
