Pluggable storage engine discussion
Hi, We are having discussions about the pluggable storage engine plan on the jira: https://issues.apache.org/jira/browse/CASSANDRA-13475. We are trying to figure out a plan for the pluggable storage engine effort. Right now, the discussion is mainly happening between couple C* committers, like Blake and me. But I want to increase the visibility, and I'm very welcome more developers to be involved in the discussion. It will help us on moving forward on this effort. Also, I have a quip as a (very high level) design doc for this project. https://quip.com/bhw5ABUCi3co Thanks Dikang.
Re: Pluggable storage engine discussion
Hi Dikang Have you been able to continue evaluating RocksDB? I'm afraid we might be a bit too much ahead in the discussion by already talking about a pluggable architecture, while we haven't fully evaluated yet if we can and want to support an alternative RocksDB engine implementation at all. Because if we don't, we also don't need a pluggable architecture at this point, do we? There's little to be gained from a major refactoring, just to find out that alternative engines we thought of didn't turn out to be a good fit for production for whatever reasons. On the other hand, if RocksDB is (by whatever standards) a better storage implementation, why not completely switch, instead of just making it an option? But if it's not, is a major refactoring still worth it? On 03.11.17 19:22, Dikang Gu wrote: > Hi, > > We are having discussions about the pluggable storage engine plan on the > jira: https://issues.apache.org/jira/browse/CASSANDRA-13475. > > We are trying to figure out a plan for the pluggable storage engine effort. > Right now, the discussion is mainly happening between couple C* committers, > like Blake and me. But I want to increase the visibility, and I'm very > welcome more developers to be involved in the discussion. It will help us > on moving forward on this effort. > > Also, I have a quip as a (very high level) design doc for this project. > https://quip.com/bhw5ABUCi3co > > Thanks > Dikang. > - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
Re: Pluggable storage engine discussion
Hi Stefan, Yeah, my team has been working for 6+ months on the development and testing of the RocksDB based storage engine, and we are in the process of rolling it out to our production deployment inside Instagram! We see huge gains in reducing the latency and footprint of our Cassandra clusters. I will publish those when I have more metrics after the rollout. I do hope someone in community can try it out as well, in your shadow cluster of course. We have pushed the code here: https://github.com/Instagram/cassandra/commits/rocks_3.0 . On Fri, Nov 3, 2017 at 1:48 PM, Stefan Podkowinski wrote: > Hi Dikang > > Have you been able to continue evaluating RocksDB? I'm afraid we might > be a bit too much ahead in the discussion by already talking about a > pluggable architecture, while we haven't fully evaluated yet if we can > and want to support an alternative RocksDB engine implementation at all. > Because if we don't, we also don't need a pluggable architecture at this > point, do we? There's little to be gained from a major refactoring, just > to find out that alternative engines we thought of didn't turn out to be > a good fit for production for whatever reasons. > > On the other hand, if RocksDB is (by whatever standards) a better > storage implementation, why not completely switch, instead of just > making it an option? But if it's not, is a major refactoring still worth > it? > > > On 03.11.17 19:22, Dikang Gu wrote: > > Hi, > > > > We are having discussions about the pluggable storage engine plan on the > > jira: https://issues.apache.org/jira/browse/CASSANDRA-13475. > > > > We are trying to figure out a plan for the pluggable storage engine > effort. > > Right now, the discussion is mainly happening between couple C* > committers, > > like Blake and me. But I want to increase the visibility, and I'm very > > welcome more developers to be involved in the discussion. It will help us > > on moving forward on this effort. > > > > Also, I have a quip as a (very high level) design doc for this project. > > https://quip.com/bhw5ABUCi3co > > > > Thanks > > Dikang. > > > > > - > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > > -- Dikang
