Re: Discuss SIP-14 Embedded Zookeeper

2023-09-11 Thread Eric Pugh
Agreed (though SIP-14 helps towards that end)… Thoughts on what the commands might look like? What do you think will be the road blocks? Do we have any JIRA tickets that layout the path forward? In the SIP there is some discussion of starting with a ZookeeperServerEmbedded, and even a PR by

Re: Discuss SIP-14 Embedded Zookeeper

2023-09-11 Thread Ilan Ginzburg
Getting rid of standalone Solr mode is not part of SIP-14, I suggest we discuss it separately from the effort to make SolrCloud easier to deploy or the default. On Sat, Sep 9, 2023 at 4:11 PM Ishan Chattopadhyaya < ichattopadhy...@gmail.com> wrote: > +1 fully support this, also support the move f

Re: Discuss SIP-14 Embedded Zookeeper

2023-09-09 Thread Ishan Chattopadhyaya
+1 fully support this, also support the move for us to get rid of the standalone Solr mode. On Sat, 9 Sept, 2023, 5:12 pm Eric Pugh, wrote: > The baby step that Jan is proposing for a ‘zookeeper’ node-role makes > sense to me, for those who are only deploying a very small Solr setup. > > What wo

Re: Discuss SIP-14 Embedded Zookeeper

2023-09-09 Thread Eric Pugh
The baby step that Jan is proposing for a ‘zookeeper’ node-role makes sense to me, for those who are only deploying a very small Solr setup. What would that look like?Would you start your solr like this?I looked a bit at the Ref Guide page for Nodes, and I’m gathering it might look li

Re: Discuss SIP-14 Embedded Zookeeper

2023-09-06 Thread Jan Høydahl
Hi, Eric Pugh and I discussed this SIP the other day, as a stepping stone for making cloud mode the default. Perhaps there is new energy for this two years down the road? We don't need to tackle the full dynamic scaling of ZK on day one. Just adding a 'zookeeper' node-role so we could have tre

Re: Discuss SIP-14 Embedded Zookeeper

2022-01-16 Thread Mark Miller
Yeah, there two reasons we didn’t push embedded Zookeeper out of the gate and even went so far as to call it a non production “demo” feature. Dynamic reconfiguration as a cluster changed over time, and a Zookeeper instance per Solr node being prohibitive. At least the latter was theoretical externa

Re: Discuss SIP-14 Embedded Zookeeper

2022-01-12 Thread Mike Drob
I put together a draft PR for the first stage of the SIP which is to cut over unit tests to using the embedded ZK. There's a few things missing, like setting limiters and our injected use of ZKDatabase, but we've also had discussions about how those are not particularly useful as is. The biggest c

Re: Discuss SIP-14 Embedded Zookeeper

2021-09-15 Thread Mike Drob
> If people are running 100 or 1000 node clusters and use each node as a ZK server, by default, what kind of impact would that have? Bad. Very very bad. The largest ZK quorum I've personally seen is 9, and I've heard rumors of somebody running 15. I think the recommended approach for distributing

Re: Discuss SIP-14 Embedded Zookeeper

2021-09-15 Thread Houston Putman
If we were to make this work, and support productionized embedded Zookeeper, then it would absolutely be something that we want to support by default in the Solr Operator. I don't think we'd be able to cut the Zookeeper Operator dependency really quickly, because this is going in at the earliest i

Re: Discuss SIP-14 Embedded Zookeeper

2021-09-14 Thread Mike Drob
I like the idea of starting nodes in a ZK-only mode, probably we would call it something like coordination mode. It ties in to ideas that I've had while discussing with other folks about other Solr node specializations, like "edge" nodes that are part of a cluster but do not host collections and ex

Re: Discuss SIP-14 Embedded Zookeeper

2021-09-14 Thread Jan Høydahl
Thanks for kicking this off Mike. I added a few "rejected alternatives" and put a few questions for thought in a comment. You may want to keep all discussion in this email thread, so here are the questions copied: This is promising! Question: Would this mode be valuable also for Kubernetes depl

Discuss SIP-14 Embedded Zookeeper

2021-09-14 Thread Mike Drob
Devs, We've previously discussed maintaining ZK as being an operational hurdle for some groups getting started or migrating to SolrCloud from non-ZK cloud mode. I'd like to discuss the idea of embedding ZK in our own process control. Please see the SIP at https://cwiki.apache.org/confluence/displ