100K is really not the limit, it's just hard to imagine 100K cores on a single machine unless some were really rarely used. And it's per node, not cluster-wide.
The current state is that everything is in place, including transient cores, auto-discovery, etc. So you should be able to go ahead and try it out. The next bit that will help with efficiency is sharing named config sets. The intent here is that <solrhome>/configs will contain sub-dirs like "conf1", "conf2" etc. Then your cores can reference configName=conf1 and only one copy of the configuration data will be used rather than re-loading one for each core as it comes up and down. Do note that the _first_ query in to one of the not-yet-loaded cores will be slow. The model here is that you can tolerate some queries taking more time at first than you might like in exchange for the hardware savings. This pre-supposes that you simply cannot fit all the cores into memory at once. The "won't fix" bits are there because, as we got farther into this process, the approach changed and the functionality of the won't fix JIRAs was subsumed by other changes by and large. I've got to update that documentation sometime, but just haven't had time yet. If you go down this route, we'll be happy to add your name to the authorized editors of the wiki list if you'd like. Best Erick On Thu, Jun 6, 2013 at 3:08 PM, Aleksey <bitterc...@gmail.com> wrote: > I was looking at this wiki and linked issues: > http://wiki.apache.org/solr/LotsOfCores > > they talk about a limit being 100K cores. Is that per server or per > entire fleet because zookeeper needs to manage that? > > I was considering a use case where I have tens of millions of indices > but less that a million needs to be active at any time, so they need > to be loaded on demand and evicted when not used for a while. > Also since number one requirement is efficient loading of course I > assume I will store a prebuilt index somewhere so Solr will just > download it and strap it in, right? > > The root issue is marked as "won;t fix" but some other important > subissues are marked as resolved. What's the overall status of the > effort? > > Thank you in advance, > > Aleksey
