Hi, I submitted patches for the pluggable CommitLog. Appreciate any comments. Thanks! https://issues.apache.org/jira/browse/CASSANDRA-14062
Rei Odaira 2017-11-06 17:14 GMT-06:00 大平怜 <rei.oda...@gmail.com>: > Thanks for the feedback, Ariel, > > Based on your comments, we are revisiting our code changes, > and then we will submit a patch for review. > I hope this effort will help further modularize Cassandra > for better maintainability. > > > Thanks, > Rei Odaira > > > 2017-11-06 8:20 GMT-06:00 Ariel Weisberg <ar...@weisberg.ws>: > >> Hi, >> >> OK sorry I am very late to the discussion. I think the existing >> consensus around doing it is fine I just think you will find that making >> the commit log pluggable might be a little trickier than making a cache >> which is a glorified K/V store pluggable. >> >> The commit log reaches into a bunch of other internal API during replay >> and even a few at runtime. I think it's a refactor away from >> abstracting out those concerns from the concerns of making log records >> durable, providing notifications of durability, and then releasing them >> though. >> >> If I'm the only person unhappy about breaking the plugins in bug fix >> releases and the resulting problems that creates for anyone who wants to >> operate this hardware in production then I am willing to look past it. >> We could also address it via documentation the plugin link page as well >> as at landing sites for individual plugins. >> >> Otherwise I think this is a bigger commitment from us to a larger API >> then originally scoped and one that restricts what changes can be made >> in bug fix releases to the existing C* commit log. Unless we have >> current version and version next of the plugin API so that we can move >> ahead in bug fix releases without breaking existing plugins. >> >> Thanks, >> Ariel >> >> On Thu, Nov 2, 2017, at 04:46 PM, 大平怜 wrote: >> > Hell Ariel, >> > >> > About the pluggability, we have discussed this topic in the dev list >> last >> > May: >> > https://www.mail-archive.com/dev@cassandra.apache.org/msg11102.html >> > >> > I don't think the whole community has reached a consensus, but >> > the result of the discussion at that time was that >> > 1) We were going to release our extensions as plugins. >> > 2) The project would support the plugin ecosystem by creating a plugin >> > link >> > page on the Web site. >> > >> > Appreciate it if you could shed new light on the discussion. >> > >> > >> > Thanks, >> > Rei Odaira >> > >> > >> > 2017-11-01 17:53 GMT-05:00 Ariel Weisberg <ar...@weisberg.ws>: >> > >> > > Hi, >> > > >> > > Just so I don't seem too negative, what I would really like to see is >> an >> > > in tree implementation. The real challenge there is that the hardware >> is >> > > not widely available. If it were something you could get in GCE or AWS >> > > or at least get via an emulator that would be a different story. >> > > >> > > Ariel >> > > >> > > On Wed, Nov 1, 2017, at 06:46 PM, Ariel Weisberg wrote: >> > > > Hi, >> > > > >> > > > OK. It makes sense that most of the existing plumbing is not >> applicable >> > > > since it operates on a filesystem. >> > > > >> > > > How does replay work? Presumably you will need to refactor >> > > > CommitLogReplayer as well? >> > > > >> > > > I think the best way for us to decide whether it's something we >> want in >> > > > tree is to see a patch. You would need to do this even if it doesn't >> > > > make it in tree and you end up having to deploy a patched build. >> > > > >> > > > Pluggability is a little bit of a touchy subject because we don't >> want >> > > > to directly or indirectly become responsible for interfaces to out >> of >> > > > tree implementations. I don't know if there is consensus around >> this, >> > > > but I think even if we made the commit log pluggable it would be >> with >> > > > the understanding that we may change the API even in bug fix >> releases. >> > > > >> > > > Down the line where this becomes tricky is unmaintained out of tree >> > > > implementations that people depend on being broken due to interface >> > > > changes and then no one being around to fix them. People who depend >> on >> > > > the out of tree implementation have no one to complain to but us. >> This >> > > > becomes even more likely when the maintainers aren't using the >> latest >> > > > version of C* and are busy with other things. >> > > > >> > > > You are characterizing the API as being just a few methods on >> CommitLog >> > > > but that isn't true. >> > > > >> > > > These are the imports for CommitLogReplayer >> > > > >> > > > import org.apache.cassandra.concurrent.Stage; >> > > > import org.apache.cassandra.concurrent.StageManager; >> > > > import org.apache.cassandra.config.CFMetaData; >> > > > import org.apache.cassandra.config.Schema; >> > > > import org.apache.cassandra.db.*; >> > > > import org.apache.cassandra.io.util.FastByteArrayInputStream; >> > > > import org.apache.cassandra.io.util.FileUtils; >> > > > import org.apache.cassandra.io.util.RandomAccessReader; >> > > > import org.apache.cassandra.utils.*; >> > > > >> > > > And these are the imports for CommitLog >> > > > >> > > > import org.apache.cassandra.config.Config; >> > > > import org.apache.cassandra.config.DatabaseDescriptor; >> > > > import org.apache.cassandra.db.*; >> > > > import org.apache.cassandra.io.FSWriteError; >> > > > import org.apache.cassandra.io.sstable.SSTableDeletingTask; >> > > > import org.apache.cassandra.io.util.DataOutputByteBuffer; >> > > > import org.apache.cassandra.metrics.CommitLogMetrics; >> > > > import org.apache.cassandra.net.MessagingService; >> > > > import org.apache.cassandra.service.StorageService; >> > > > import org.apache.cassandra.utils.JVMStabilityInspector; >> > > > >> > > > If we change any code that changes a line in CommitLog or >> > > > CommitLogReplayer in a bug fix release it's probably going to break >> your >> > > > plugin JAR. Anyone running it in production will now have to fix it >> and >> > > > recompile or be unable to get bug fixes. >> > > > >> > > > Regards, >> > > > Ariel >> > > > On Wed, Nov 1, 2017, at 05:25 PM, 大平怜 wrote: >> > > > > Hi Ariel, >> > > > > >> > > > > CommitLogSegment assumes commit log files stored on a regular file >> > > > > system. >> > > > > Our CAPI Flash system bypasses OS and directly accesses flash, >> > > > > so we cannot use the current framework of CommitLogSegment as it >> is. >> > > > > Intel's SPDK also bypasses a file system, so we think this kind of >> > > > > requirement >> > > > > is not uncommon. >> > > > > >> > > > > It would not be easy to reuse AbstractCommitLogSegmentManager, >> either, >> > > > > because the archiving and synchronization logics have to be >> decoupled. >> > > > > It would require major rework, and we don't think we should affect >> > > > > the existing implementation so much. >> > > > > >> > > > > We do not change any existing format of CommitLog. Our plugin >> will use >> > > > > its own format, as it must manage commit logs on the >> 4KB-block-oriented >> > > > > address spaces of flash devices. >> > > > > >> > > > > >> > > > > Regards, >> > > > > Rei Odaira >> > > > > >> > > > > >> > > > > 2017-10-31 15:38 GMT-05:00 Ariel Weisberg <ar...@weisberg.ws>: >> > > > > >> > > > > > Hi, >> > > > > > >> > > > > > There are pluggable elements to the commit log such as those >> used to >> > > > > > support mmap or compressed. >> > > > > > >> > > > > > Can you describe at a high level what a new implementation >> would look >> > > > > > like and why it can't be a mode of the existing implementation? >> > > > > > >> > > > > > You are not proposing changing the format correct? >> > > > > > >> > > > > > Regards, >> > > > > > Ariel >> > > > > > >> > > > > > On Tue, Oct 31, 2017, at 04:09 PM, 大平怜 wrote: >> > > > > > > Hello, >> > > > > > > >> > > > > > > We are developing a Cassandra plugin to store CommitLog on our >> > > > > > > low-latency >> > > > > > > Flash device (CAPI-Flash). To do that, the original CommitLog >> > > interface >> > > > > > > must be changed to allow plugins. Anyone has any thoughts >> about >> > > it? We >> > > > > > > have our codebase ready, but we think we should start with >> > > high-level >> > > > > > > discussion. >> > > > > > > >> > > > > > > The runtime overhead will be minimal. The only overhead will >> be >> > > changing >> > > > > > > method invocations to CommitLog#add(), >> > > CommitLog#getCurrentPosition(), >> > > > > > > etc. >> > > > > > > into interface invocations. >> > > > > > > >> > > > > > > Synching to CommitLog is one of the performance bottlenecks in >> > > Cassandra >> > > > > > > especially with batch commit. I think the pluggable CommitLog >> > > will allow >> > > > > > > other interesting alternatives, such as one using SPDK. >> > > Appreciate any >> > > > > > > comments. >> > > > > > > >> > > > > > > >> > > > > > > Regards, >> > > > > > > Rei Odaira >> > > > > > >> > > > > > ------------------------------------------------------------ >> > > --------- >> > > > > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org >> > > > > > For additional commands, e-mail: dev-h...@cassandra.apache.org >> > > > > > >> > > > > > >> > > > >> > > > ------------------------------------------------------------ >> --------- >> > > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org >> > > > For additional commands, e-mail: dev-h...@cassandra.apache.org >> > > > >> > > >> > > --------------------------------------------------------------------- >> > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org >> > > For additional commands, e-mail: dev-h...@cassandra.apache.org >> > > >> > > >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org >> For additional commands, e-mail: dev-h...@cassandra.apache.org >> >> >
