Maybe. Yeah the song is just meant to be funny.
The long / extremely short, limited value, overview of my view is that pretty much every classical way you can share clusterstate with Zookeeper is an Overseer design, at least how I defined or thought of an Overseer before it had any implementation . The fundamental concept of the Overseer has always been some kind of cluster lock for some duration. If you simplify a bit, I see how you implement an Overseer as a spectrum, where on one end you have something like a Hadoop NameNode, or a permanent cluster lock. Maybe you have a hot backup in the wings. As you move toward the other end of the spectrum in this simplified model, you have cluster locks that are held for less and less time, and maybe even more cluster locks than one for some tasks. At the very far end, you have something like an optimistic concurrency cluster lock, and effectively, ridiculously short lock holds. Personally, I would have aimed for the optimistic concurrency end of the spectrum in implementation, but short of it, that is actually where I started as a foothold, with no intention of staying there. But anyway, I would have moved up the spectrum a short ways. I don’t think anyone really aimed for where the current Overseer did land, it kind of gets there through sheer weight and silliness as opposed to any grand design, but it’s obviously spent most of its life closer to the NameNode end, while trying to pretend it’s more in the middle. It very much could be where I initially would have imagined it ended up with extremely minimal functional changes - I mean, 100% code change, but functional extremely close. Which isn’t to say that is the ideal target. But one should understand that reality if they want to understand what that monster actually is. Going back to the Overseer looking much closer to a NameNode but kind of pretending it’s more in the middle, Fundamentally, that’s indicative of everything that’s wrong with the Overseer. It doesn’t know what it is. It leans on Zookeeper like an unsure underseer. And in the end, that helps ensure it would never be good even if you wanted to target its location on the spectrum as your desired location. And of course that’s compounded by the ridiculous cluster communication cadence it allows for, the more than ridiculous communication cost it requires. And the hairy, complexity that fairly weak mitigation attempts have added over time to the already hairy, way overly complex code. And already, I can’t wrap that up in any meaningful way here, I’m already way past my attempted word limit.
