If we were to make this work, and support productionized embedded Zookeeper, then it would absolutely be something that we want to support by default in the Solr Operator.
I don't think we'd be able to cut the Zookeeper Operator dependency really quickly, because this is going in at the earliest in Solr 9 and more likely Solr 10 (probably). The Solr Operator still needs to support older versions, especially Solr 8 for a fair amount of time. So once the minimum supported Solr version is one that has this feature, then we can get rid of the Zookeeper Operator for good. This is probably my favorite thing about the SIP. The Zookeeper Operator is fine, but removing that dependency would lift a huge burden off of the Solr Operator's shoulders. I also think it's a good idea to be able to start solr in a ZK-Only mode. Also you should be able to tell Solr whether you want it to start as a ZK member or observer, or not run ZK on that node at all. I'm not extremely in touch with the ZK community at this point, but what cluster sizes are people scaling up to nowadays? If people are running 100 or 1000 node clusters and use each node as a ZK server, by default, what kind of impact would that have? On Tue, Sep 14, 2021 at 8:20 PM Mike Drob <md...@apache.org> wrote: > I like the idea of starting nodes in a ZK-only mode, probably we would > call it something like coordination mode. It ties in to ideas that I've had > while discussing with other folks about other Solr node specializations, > like "edge" nodes that are part of a cluster but do not host collections > and exist solely for routing http queries to the appropriate places. > > I think it could be useful in a k8s deployment as well, but I'd have to > think about how we want to do all the port magic there. I know that I've > had conversations with Houston about wanting to move away from > ZookeeperOperator, but those haven't quite taken hold yet. > > On Tue, Sep 14, 2021 at 6:02 PM Jan Høydahl <jan....@cominvent.com> wrote: > >> Thanks for kicking this off Mike. I added a few "rejected alternatives" >> and put a few questions for thought in a comment. You may want to keep all >> discussion in this email thread, so here are the questions copied: >> >> >> *This is promising! Question: Would this mode be valuable also for >> Kubernetes deployments, i.e. we could get rid of the ZookeeperOperator and >> instead let the SolrOperator keep track of which Solr pods that also act as >> ZK nodes?Would we allow a Solr node to start in a ZK-only mode, i.e. not >> eligible for collections/cores/overseer? This would also support those huge >> clusters where you want dedicated ZKs.* >> >> This also ties in with SIP-6 Solr should own the bootstrap process >> <https://cwiki.apache.org/confluence/display/SOLR/SIP-6+Solr+should+own+the+bootstrap+process>, >> as we'd want to control startup/shutdown behavior wrt the zk, e.g. start >> embedded zk before solr and stop solr before stopping zk. Perhaps also >> gracefully exiting the quorum on planned shutdown? >> >> Jan >> >> 14. sep. 2021 kl. 22:09 skrev Mike Drob <md...@apache.org>: >> >> Devs, >> >> We've previously discussed maintaining ZK as being an operational hurdle >> for some groups getting started or migrating to SolrCloud from non-ZK cloud >> mode. I'd like to discuss the idea of embedding ZK in our own process >> control. >> >> Please see the SIP at >> https://cwiki.apache.org/confluence/display/SOLR/SIP-14+Embedded+Zookeeper >> >> Thank you, >> Mike >> >> >>
