Re: Making CommitLog pluggable

2017-11-01 Thread Ariel Weisberg
Hi, Just so I don't seem too negative, what I would really like to see is an in tree implementation. The real challenge there is that the hardware is not widely available. If it were something you could get in GCE or AWS or at least get via an emulator that would be a different story. Ariel On W

Re: Making CommitLog pluggable

2017-11-01 Thread Ariel Weisberg
Hi, OK. It makes sense that most of the existing plumbing is not applicable since it operates on a filesystem. How does replay work? Presumably you will need to refactor CommitLogReplayer as well? I think the best way for us to decide whether it's something we want in tree is to see a patch. You

Re: Making CommitLog pluggable

2017-11-01 Thread Michael Kjellman
Awesome!! You're two steps ahead ;) Not sure if you're allowed to share, but can you highlight any details on endurance and performance? Are the pages 4kb or 16kb? How many writes do you expect to handle over a 1 year window of the device? I assume because you're directly accessing the hardware

Re: Making CommitLog pluggable

2017-11-01 Thread 大平怜
Hi Michael, Yes, testing is always a problem, and that is exactly why we would like to release our code as a plugin, outside of the main source tree, so that the project won't need to test the hardware-dependent code. The pluggable CommitLog will allow this approach. Actually, we have already rel

Re: Making CommitLog pluggable

2017-11-01 Thread Michael Kjellman
Rei: One thing that comes up when these type of conversations occur is how the project can test hardware dependent code. In the case of the PPC64 stuff, hardware actually got donated to the ASF so Jenkins runs could be done to check that things work. Any thoughts on this aspect? Might be a bit

Re: Making CommitLog pluggable

2017-11-01 Thread 大平怜
Hi Ariel, CommitLogSegment assumes commit log files stored on a regular file system. Our CAPI Flash system bypasses OS and directly accesses flash, so we cannot use the current framework of CommitLogSegment as it is. Intel's SPDK also bypasses a file system, so we think this kind of requirement is

Re: Somewhat Weekly Cassandra Dev Wrapup

2017-11-01 Thread Jeff Jirsa
Good questions. Right now we're not actively using it (at least not publicly, as far as I know, individual contributors may be using it or sonar or something else). For the specific warning (index out of bounds) you point out below, if map.length was odd, then the consumer.consume(map[i],map[i+1])

Re: Somewhat Weekly Cassandra Dev Wrapup

2017-11-01 Thread Salih Gedik
Hi, As an undergrad student I actually question the output of static analysis tools. Are you guys actively using it or do you find projects like Sonar efficient in such open source projects? Last time I heard that FindBugs are no longer maintained because the code was hard to maintain. For inst

Re: Somewhat Weekly Cassandra Dev Wrapup

2017-11-01 Thread Jeff Jirsa
Ah, I remember that now. Blocked by a guava bug? 4.0 seems like a good time to upgrade guava. -- Jeff Jirsa > On Nov 1, 2017, at 2:49 AM, Stefan Podkowinski wrote: > > >> 2) Static Analysis stuff: > > I think it's worth mentioning that I also tried to integrate the Error > Prone analyzer (

Re: Somewhat Weekly Cassandra Dev Wrapup

2017-11-01 Thread Stefan Podkowinski
> 2) Static Analysis stuff: I think it's worth mentioning that I also tried to integrate the Error Prone analyzer (http://errorprone.info/) a while ago as part of CASSANDRA-13175. Eventually I dropped the ball there due to some classpath issues, but maybe that can be fix or worked around. Having

Re: Somewhat Weekly Cassandra Dev Wrapup

2017-11-01 Thread Malcolm Taylor
Glad to hear you are finding lgtm.com useful. I work for Semmle, the company behind lgtm.com. I see you are interested in checking regularly for new and fixed alerts on lgtm.com. This can be achieved through our Github integration described in https://lgtm.com/docs/lgtm/using-lgtm-analysis-contin