Re: Failed Dtest will block cutting releases

2016-12-04 Thread Benjamin Roth
Hi Michael, Thanks for this update. As a newbie it helped me to understand the organization and processes a little bit better. I don't know how many CS-devs know this but I love this rule (actually the whole book): http://programmer.97things.oreilly.com/wiki/index.php/The_Boy_Scout_Rule I person

Re: Failed Dtest will block cutting releases

2016-12-04 Thread Michael Shuler
Thanks for your thoughts on testing Apache Cassandra, I share them. I just wanted to note that the known_failure() annotations were recently removed from cassandra-dtest [0], due to lack of annotation removal when bugs fixed, and the internal webapp that we were using to parse has been broken for

Re: Failed Dtest will block cutting releases

2016-12-04 Thread Benjamin Roth
Sorry for jumping in so boldly before. TL;DR: - I didn't mean to delete every flaky test just like that - To improve quality, each failing test has to be analyzed individually for release More thoughts on that: I had a closer look on some of the tests tagged as flaky and realized that

Re: Failed Dtest will block cutting releases

2016-12-03 Thread Edward Capriolo
I think it is fair to run a flakey test again. If it is determine it flaked out due to a conflict with another test or something ephemeral in a long process it is not worth blocking a release. Just deleting it is probably not a good path. I actually enjoy writing fixing, tweeking, tests so pinge

Re: Failed Dtest will block cutting releases

2016-12-02 Thread Benjamin Roth
Excuse me if I jump into an old thread, but from my experience, I have a very clear opinion about situations like that as I encountered them before: Tests are there to give *certainty*. *Would you like to pass a crossing with a green light if you cannot be sure if green really means green?* Do you

Re: Failed Dtest will block cutting releases

2016-12-02 Thread sankalp kohli
Hi, I dont see any any update on this thread. We will go ahead and make Dtest a blocker for cutting releasing for anything after 3.10. Please respond if anyone has an objection to this. Thanks, Sankalp On Mon, Nov 21, 2016 at 11:57 AM, Josh McKenzie wrote: > Caveat: I'm strongly in favor

Re: Failed Dtest will block cutting releases

2016-11-21 Thread Josh McKenzie
Caveat: I'm strongly in favor of us blocking a release on a non-green test board of either utest or dtest. > put something in prod which is known to be broken in obvious ways In my experience the majority of fixes are actually shoring up low-quality / flaky tests or fixing tests that have been i

Re: Failed Dtest will block cutting releases

2016-11-21 Thread Jonathan Haddad
+1. Kind of silly to put advise people to put something in prod which is known to be broken in obvious ways On Mon, Nov 21, 2016 at 11:31 AM sankalp kohli wrote: > Hi, > We should not cut a releases if Dtest are not passing. I won't block > 3.10 on this since we are just discussing this. >

Failed Dtest will block cutting releases

2016-11-21 Thread sankalp kohli
Hi, We should not cut a releases if Dtest are not passing. I won't block 3.10 on this since we are just discussing this. Please provide feedback on this. Thanks, Sankalp