[
https://issues.apache.org/jira/browse/SOLR-14306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17052458#comment-17052458
]
Tomas Eduardo Fernandez Lobbe commented on SOLR-14306:
------------------------------------------------------
Yes, I don't like merging the two, but I felt moving the Solr part alone could
have been more difficult while at the same time, maybe not the best long term
if we are talking about moving to Curator eventually.
bq. I also am concerned that if we do both changes at the same time we end up
with a bad abstraction that looks ok but is actually very Curator specific
That's a very good point. I did a POC and it's easy to fall into this. It may
or may not be a problem, if we like the interfaces to be curator-oriended, we'd
have to make whatever replacement we have look like it later.
bq. are the overseer and shard leader election code paths using common tools
right now
They are in part, yes. One thing I noticed also while looking at Curator is
that those two could actually fall into different "recipes". Overseer is
essentially "do some work while you are the leader. Stop doing it when you are
no longer the leader" (LeaderElector in Curator), while shard leader is "act
differently while you are the leader" (Leader latch in Curator). Of course they
can both use the same implementation if we want (i.e. we can keep asking in the
Overseer "amILeader" and then have listeners to interrupt), but I like that
differentiation that Curator makes.
> Refactor coordination code into separate module and evaluate using Curator
> --------------------------------------------------------------------------
>
> Key: SOLR-14306
> URL: https://issues.apache.org/jira/browse/SOLR-14306
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
> Reporter: Tomas Eduardo Fernandez Lobbe
> Priority: Major
>
> This Jira issue is to discuss two changes that unfortunately are difficult to
> address separately
> # Separate all ZooKeeper coordination logic into it’s own module, that can
> be tested in isolation
> # Evaluate using Apache Curator for coordination instead of our own logic.
> I drafted a
> [SIP|https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=148640472],
> but this is very much WIP, I’d like to hear opinions before I spend too much
> time on something people hates.
> From the initial draft of the SIP:
> {quote}The main goal of this change is to allow better testing of the
> different ZooKeeper interactions related to coordination (leader election,
> queues, etc).
There are already some abstractions in place for lower level
> operations (set-data, get-data, etc, see DistribStateManager), so the idea is
> to have a new, related abstraction named CoordinationManager, where we could
> have some higher level coordination-related classes, like LeaderRunner
> (Overseer), LeaderLatch (for shard leaders), etc.
Curator comes into place
> because, in order to refactor the existing code into these new abstractions,
> we’d have to rework much of it, so we could instead consider using Curator, a
> library that was mentioned in the past many times. While I don’t think this
> is required, It would make this transition and our code simpler (from what I
> could see, however, input from people with more Curator experience would be
> greatly appreciated).
> While it would be out of the scope of this change, If the
> abstractions/interfaces are correctly designed, this could lead to, in the
> future, be able to use something other than ZooKeeper for coordination,
> either etcd or maybe even some in-memory replacement for tests.
> {quote}
> There are still many open questions, and many questions I still don’t know
> we’ll have, but please, let me know if you have any early feedback, specially
> if you’ve worked with Curator in the past.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
