[QUESTION] Should JVM dtests be resistant to System.exit called in the node thread?
I'm testing startup checks and expect them to fail in a certain scenario but the test just exists because when a startup check fails, Cassandra calls System.exit. It is possible to create CassandraDaemon with "runManaged" constructor arg which prevents System.exit and make the code throw RuntimeException instead (this is already implemented but the parameter needs to be explicitly passed to the constructor of CassandraDaemon, thus it is not the option without some refactoring). It is also possible to install security manager which prevents System.exit. If so, should this be implemented globally for all the JVM dtests or individually for JVM dtests which need that? I'm asking because when System.exit is called unexpectedly, the test is not reported as failed - it is not reported at all (at least I saw that in IntelliJ). Thanks, Jacek - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
Re: [QUESTION] Should JVM dtests be resistant to System.exit called in the node thread?
We can also ByteWeave System.exit away, or migrate it to some other method invocation that e.g. throws an exception. The simulator makes extensive use of byte weaving on top of the dtest api. It should be landing soon, and we could repurpose some of its code for regular dtests. Or we may want to consider porting some dtests to the Simulator. From: Jacek Lewandowski Date: Wednesday, 27 October 2021 at 11:01 To: dev@cassandra.apache.org Subject: [QUESTION] Should JVM dtests be resistant to System.exit called in the node thread? I'm testing startup checks and expect them to fail in a certain scenario but the test just exists because when a startup check fails, Cassandra calls System.exit. It is possible to create CassandraDaemon with "runManaged" constructor arg which prevents System.exit and make the code throw RuntimeException instead (this is already implemented but the parameter needs to be explicitly passed to the constructor of CassandraDaemon, thus it is not the option without some refactoring). It is also possible to install security manager which prevents System.exit. If so, should this be implemented globally for all the JVM dtests or individually for JVM dtests which need that? I'm asking because when System.exit is called unexpectedly, the test is not reported as failed - it is not reported at all (at least I saw that in IntelliJ). Thanks, Jacek - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
Re: [QUESTION] Should JVM dtests be resistant to System.exit called in the node thread?
> It is also possible to install security manager which prevents System.exit. > If so, should this be implemented globally for all the JVM dtests or > individually for JVM dtests which need that? I'm asking because when > System.exit is called unexpectedly, the test is not reported as failed - it > is not reported at all (at least I saw that in IntelliJ). Many tests do this already, so adding a security manager is the simplest thing IMO. Another option (think we still need security manager, this comment is mostly for consistency) is to change all call sites to go through the JVMStability class as this has a shim to swap out how to shutdown a JVM. > On Oct 27, 2021, at 3:08 AM, bened...@apache.org wrote: > > We can also ByteWeave System.exit away, or migrate it to some other method > invocation that e.g. throws an exception. > > The simulator makes extensive use of byte weaving on top of the dtest api. It > should be landing soon, and we could repurpose some of its code for regular > dtests. Or we may want to consider porting some dtests to the Simulator. > > > From: Jacek Lewandowski > Date: Wednesday, 27 October 2021 at 11:01 > To: dev@cassandra.apache.org > Subject: [QUESTION] Should JVM dtests be resistant to System.exit called in > the node thread? > I'm testing startup checks and expect them to fail in a certain scenario but > the test just exists because when a startup check fails, Cassandra calls > System.exit. > > It is possible to create CassandraDaemon with "runManaged" constructor arg > which prevents System.exit and make the code throw RuntimeException instead > (this is already implemented but the parameter needs to be explicitly passed > to the constructor of CassandraDaemon, thus it is not the option without some > refactoring). > > It is also possible to install security manager which prevents System.exit. > If so, should this be implemented globally for all the JVM dtests or > individually for JVM dtests which need that? I'm asking because when > System.exit is called unexpectedly, the test is not reported as failed - it > is not reported at all (at least I saw that in IntelliJ). > > Thanks, > Jacek > > > - > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
Re: [DISCUSS] How to implement backward compatibility (CASSANDRA-17048)
> should we let users enable the new uuid ids later when they are sure they > will not downgrade in the future? I strongly believe new features should be off by default and give a “good enough” grace period before enabling by default (while still offering support to disable). As long as this is disabled by default I am cool with it. > On Oct 26, 2021, at 1:25 PM, Jacek Lewandowski > wrote: > > Yes, those explanations sound very reasonable to me as well and I'll push the > implementation soon. > > Thank you guys > > On 2021/10/26 18:21:44, Joshua McKenzie wrote: >> +1 to Benedict's perspective here. Supporting both sstable ID paradigms >> should be relatively trivial and low cost to maintain going forward. >> >> On Tue, Oct 26, 2021 at 7:54 AM bened...@apache.org >> wrote: >> >>> I think it is probably acceptable to prevent downgrades once a new feature >>> is enabled, as the exposure risk is limited to that one feature. The user >>> can test the new version to ensure everything else works satisfactorily >>> before committing to this one feature. >>> >>> A downgrade tool would also be possible to produce, but probably the >>> additional utility is limited. >>> >>> I think this particular feature is probably easy enough to maintain as >>> permanently optional, simply maintaining two system tables: one for the old >>> generation format, one for the new. So long as the user doesn’t use the new >>> format, it remains forever downgradeable. Though perhaps one day we may >>> want to force users to migrate, I don’t think there’s any rush, and the >>> important thing to avoid is providing users no version buffer to escape new >>> bugs – if a major version later we force upgrade, then they have a whole >>> range of major versions to downgrade to that still support this feature >>> (but perhaps avoid some other new issue) >>> >>> >>> >>> From: Jacek Lewandowski >>> Date: Tuesday, 26 October 2021 at 12:01 >>> To: dev@cassandra.apache.org >>> Subject: Re: [DISCUSS] How to implement backward compatibility >>> (CASSANDRA-17048) >>> Though, the user is unable to test the new feature without enabling it. And >>> when it is enabled, the user is unable to revert it. >>> >>> - - -- --- - - >>> Jacek Lewandowski >>> >>> >>> On Tue, Oct 26, 2021 at 12:54 PM Bowen Song wrote: >>> Personally, I would prefer a transition period in which the new feature is not enabled by default. This not only makes version upgrading easier, it also allows the user to stay on the old behaviour if they experience any issue with the new feature (e.g.: bugs in the new feature, or edge use cases / 3rd party tools depending on the old behaviour) until the issue is resolved. On 26/10/2021 10:21, Jacek Lewandowski wrote: > Hi, > > In short, we are discussing UUID based sstable generation identifiers >>> in https://issues.apache.org/jira/browse/CASSANDRA-17048. > > The question which somehow hold us is support for downgrading. Long story short, when we generate new sstables with uuid based ids, they are not readable by older C* versions. > > 1. should we implement a downgrade tool? (it may be quite complex) > 2. should we let users enable the new uuid ids later when they are sure they will not downgrade in the future? > > Thanks, > Jacek > > > > - > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org >>> >> > > - > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
Re: [DISCUSS] How to implement backward compatibility (CASSANDRA-17048)
Yes, I implemented that as mentioned above - cassandra.yaml option, disabled by default, description explaining the reason and consequences. - - -- --- - - Jacek Lewandowski On Wed, Oct 27, 2021 at 3:58 PM David Capwell wrote: > > should we let users enable the new uuid ids later when they are sure > they will not downgrade in the future? > > I strongly believe new features should be off by default and give a “good > enough” grace period before enabling by default (while still offering > support to disable). As long as this is disabled by default I am cool with > it. > > > On Oct 26, 2021, at 1:25 PM, Jacek Lewandowski < > lewandowski.ja...@gmail.com> wrote: > > > > Yes, those explanations sound very reasonable to me as well and I'll > push the implementation soon. > > > > Thank you guys > > > > On 2021/10/26 18:21:44, Joshua McKenzie wrote: > >> +1 to Benedict's perspective here. Supporting both sstable ID paradigms > >> should be relatively trivial and low cost to maintain going forward. > >> > >> On Tue, Oct 26, 2021 at 7:54 AM bened...@apache.org < > bened...@apache.org> > >> wrote: > >> > >>> I think it is probably acceptable to prevent downgrades once a new > feature > >>> is enabled, as the exposure risk is limited to that one feature. The > user > >>> can test the new version to ensure everything else works satisfactorily > >>> before committing to this one feature. > >>> > >>> A downgrade tool would also be possible to produce, but probably the > >>> additional utility is limited. > >>> > >>> I think this particular feature is probably easy enough to maintain as > >>> permanently optional, simply maintaining two system tables: one for > the old > >>> generation format, one for the new. So long as the user doesn’t use > the new > >>> format, it remains forever downgradeable. Though perhaps one day we may > >>> want to force users to migrate, I don’t think there’s any rush, and the > >>> important thing to avoid is providing users no version buffer to > escape new > >>> bugs – if a major version later we force upgrade, then they have a > whole > >>> range of major versions to downgrade to that still support this feature > >>> (but perhaps avoid some other new issue) > >>> > >>> > >>> > >>> From: Jacek Lewandowski > >>> Date: Tuesday, 26 October 2021 at 12:01 > >>> To: dev@cassandra.apache.org > >>> Subject: Re: [DISCUSS] How to implement backward compatibility > >>> (CASSANDRA-17048) > >>> Though, the user is unable to test the new feature without enabling > it. And > >>> when it is enabled, the user is unable to revert it. > >>> > >>> - - -- --- - - > >>> Jacek Lewandowski > >>> > >>> > >>> On Tue, Oct 26, 2021 at 12:54 PM Bowen Song > wrote: > >>> > Personally, I would prefer a transition period in which the new > feature > is not enabled by default. This not only makes version upgrading > easier, > it also allows the user to stay on the old behaviour if they > experience > any issue with the new feature (e.g.: bugs in the new feature, or edge > use cases / 3rd party tools depending on the old behaviour) until the > issue is resolved. > > On 26/10/2021 10:21, Jacek Lewandowski wrote: > > Hi, > > > > In short, we are discussing UUID based sstable generation identifiers > >>> in > https://issues.apache.org/jira/browse/CASSANDRA-17048. > > > > The question which somehow hold us is support for downgrading. Long > story short, when we generate new sstables with uuid based ids, they > are > not readable by older C* versions. > > > > 1. should we implement a downgrade tool? (it may be quite complex) > > 2. should we let users enable the new uuid ids later when they are > sure > they will not downgrade in the future? > > > > Thanks, > > Jacek > > > > > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > For additional commands, e-mail: dev-h...@cassandra.apache.org > > > > - > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > > > >>> > >> > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > For additional commands, e-mail: dev-h...@cassandra.apache.org > > > > > - > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > >
Re: [DISCUSS] CEP-17: SSTable format API (CASSANDRA-17056)
Reading the CEP I don’t see any mention to the systems which access SSTables; such as streaming (small callout to zero-copy-streaming with ZeroCopyBigTableWriter) and repair. If you are abstracting out BigTableReader then you are not dealing with the implementation assumptions that users of SSTables have (such as direct mutation to auxiliary files outside of -Data.db). > Audience > • Cassandra developers who wish to see SSTableReader and SSTableWriter > more modular than they are today, This statement relates to the above comment, many parts of the code do not use Reader/Writer but instead use direct format knowledge to apply changes to the file format (normally outside of -Data.db); to me the interfaces has to be at the SSTable level, which then expose readers/writers, but also has to expose the other things we do outside of those paths. > • move the metrics related to sstable format out from TableMetrics > class and make them tied to certain sstable implementation I am curious about this comment, are you removing exposing this information? > • have a single factory for creating both readers and writers for > particular implementation of sstable and use it consistently - no direct > creation of any reader / writer I am -1 here, for the reasons listed above; the problem (in my eye) is not reader/writer but higher level at the actual SSTable. If we plug out read/write but still allow direct file access, then these abstractions fail to provide the goals of the CEP. I am +1 to the intent of the CEP. And last comment, which I have also done in the other modularity thread… backwards compatibility and maintenance. It is not clear right now what java interfaces may not break and how we can maintain and extend such interfaces in the future. If the goal is to allow 3rd parties to plugin and offer new SSTable formats, are we as a project ok with having a minor release do a binary or source non-compatible change? If not how do we detect this? Until this problem is solved, I do not think we should add any such interfaces. > On Oct 22, 2021, at 7:23 AM, Jeremiah Jordan > wrote: > > Hi Stefan, > That idea is not related to this CEP which is about the file formats of the > sstables, not file system access. But you should take a look at the work > recently committed in https://issues.apache.org/jira/browse/CASSANDRA-16926 > to switch to using java.nio.file.Path for file access. This should allow > the use of a file system provider to access files which could be the basis > for work to load the files from S3. > > -Jeremiah > > On Fri, Oct 22, 2021 at 4:07 AM Stefan Miklosovic < > stefan.mikloso...@instaclustr.com> wrote: > >> One point I would like to add to this; I was already looking into how >> to extend this but what I saw in SSTableReader was that it is very >> much "file system oriented". There was not any possibility to actually >> hook something like that there. I think what importing does is that it >> will use SSTableReader / Writer stuff so I think that the modification >> of these classes to accommodate this idea would be necessary. >> >> On Fri, 22 Oct 2021 at 11:02, Stefan Miklosovic >> wrote: >>> >>> Hi Jacek, >>> >>> Thanks for taking the lead on this. >>> >>> There was importing of SSTables introduced in 4.0 via >>> StorageService#importNewSSTables. The "problem" with this is that >>> SSTables need to be physically located at disk so Cassandra can read >>> them. If a backup is taken and SSTables are uploaded to, for example, >>> S3 bucket, then upon restore, all these SSTables need to be downloaded >>> first and then imported. What about downloading them / importing them >>> directly from S3? Or any custom source for that matter? Importing of >>> SSTables is a very nice feature in 4.0, we do not need to copy / hard >>> link / refresh, it is all handled internally. >>> >>> I am not sure if your work is related to this idea but I would >>> appreciate it if this is pluggable as well for the sake of simplicity >>> and effectiveness as we would not have to download all sstables before >>> importing them. >>> >>> If it is not related, feel free to skip that completely and I guess I >>> would have to try to push that forward myself. >>> >>> Regards >>> >>> >>> On Fri, 22 Oct 2021 at 10:24, Jacek Lewandowski >>> wrote: I'd like to start a discussion about SSTable format API proposal >> (CEP-17) Jira: https://issues.apache.org/jira/browse/CASSANDRA-17056 CEP: >> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-17%3A+SSTable+format+API Thanks, Jacek - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org >> >> - >> To unsubscribe, e-mail: dev-unsubscr...@cassandr
