On Fri, Sep 7, 2018 at 10:00 PM Blake Eggleston <beggles...@apple.com> wrote: > > Right, I understand the arguments for starting a new project. I’m not saying > reaper is, technically speaking, the best place to start. The point I’m > trying to make is that the non-technical advantages of using an existing > project as a starting point may outweigh the technical benefits of a clean > slate. Whether that’s the case or not, it’s not a strictly technical > decision, and the non-technical advantages of starting with reaper need to be > weighed. >
Technical debt doesn't just refer to the technical solutions, having an existing user base means that a project has made promises in the past (in the case of Priam 5+ years ago) that the new project would have to keep if we make keeping users of those sidecars a goal (which for the record I don't think should be a goal, I think the goal is to make Cassandra the database work out of the box in the 4.x series). For example, Priam has to continue supporting the following as users actively use them (including Netflix): * Parallel token assignment and creation allowing parallel bootstrap and parallel doubling of hundred node clusters at once (as long as you use single tokens and run in AWS with asgs). * 3+ backup solutions, as well as assumptions about where in e.g. S3 backups are present and what format they're present in. * Numerous configuration options and UI elements (mostly 5 year old JSON APIs) * Support for Cassandra 2.0, 2.1, 2.2, 3.0, 3.11 and soon 4.0 Reaper has to continue supporting the following as users actively use them: * Postgres and h2 storage backends. These were the original storage engines and users may not have (probably haven't?) migrated to the relatively recently added Cassandra backend (which is probably the only one an official sidecar should support imo). * The three historical modes of running Reaper [1] all of which involve remote JMX (disallowed by many companies security policies including Netflix's) and none of which are a sidecar design (although Mick says we can add that back in, at which point it has to support four). * Numerous configuration options and UI elements (e.g. a UI around a single Reaper instance controlling many Cassandra clusters instead of each cluster having a separate UI more consistent with how Cassandra architecture works) * Support for Cassandra 2.2, 3.0, 3.11, and soon 4.0 [1] http://cassandra-reaper.io/docs/usage/multi_dc/ [2] https://github.com/hashbrowncipher/cassandra-mirror We can't "get the community" of these sidecars and drop support for 90+% of the supported configurations and features at the same time ... These projects have made promises to users, as per the name technical debt these choices made over years have explicit costs that we have to take into account if we accept a project as is with the goal of taking the community with us. If we don't have the goal of taking the existing community with us and are instead aiming to simply make Cassandra work out of the box without external tools, then we don't have to continue fulfilling these promises. Instead I think the organizations that made those promises (TLP and Netflix in these specific cases) should continue keeping them, and the Cassandra management process should be incrementally built by the Cassandra community with decisions made as a community and we only GA it when the PMC/community is happy with making a promise of support for the features that we've merged (and since we're starting from a fresh start if people have strong opinions about fundamental architecture we can try to take those into account like we did with the months of feedback on repair scheduling runtime/architecture/design). If we can't prove value over other community tools for running 4.x, which is definitely a possibility, then we don't mark the management process GA, we re-focus on individual community tools, and imo failure is a reasonable result of attempted innovation. -Joey --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
