Thanks for catching this, Dan! Not sure what happened there. Let's see if this works better:
Why are rolling upgrade tests slow? ================================== Six versions of servers have to to be 1. Started with old version 2. Do some work 3. Shut down 4. Updated 5. Restarted 6. Do more work and make assertions The above happens for every test method individually. This takes ~1 minute per method and per version. This is done serially right now for all methods within a class. Our other DUnit tests can reuse JVMs. These tests are starting a fresh JVM. How can we make them faster? ============================ Option 1: Move every test method into its own class. This will allow us to run them in parallel This would allow us to run ~45 tests in parallel Might increase ease of exploring test suite. Option 2: Move rolling upgrade tests into their own Concourse job. This can be done in addition to other options. Option 3: Restricting what version jumps are supported. Can we decide to not support certain version jumps? Will we support jumping from Geode 1.x to arbitrary Geode 2.x version? Option 4: Change test runner to run for different version transitions in parallel. We could generate classes that contain the desired source and destination versions and then runs those in parallel. Next Actions =========== Options 1 & 2 seem to be fairly simple changes without much downside and should get us far. Implementing those two solutions and seeing how long that leaves our test suite seem like the best first step. Jacob Barrett will take the lead on turning these two options into action. On Wed, Jul 18, 2018 at 5:04 PM, Dan Smith <dsm...@pivotal.io> wrote: > Can you fix the formatting of your message? Seems to have gotten mangled. > > On Wed, Jul 18, 2018 at 4:20 PM, Alexander Murmann <amurm...@pivotal.io> > wrote: > > > Hi all, > > > > We just had a conversation among some members of the community about the > > increasingly slow DUnit tests that are related to rolling upgrades. > > > > Please chime in with questions and concerns! > > > > Here are my notes from the discussion: > > > > > > > > > > > > > > > > *Why are rolling upgrade tests slow?Six versions of servers have to to be > > 1. Started with old version2. Do some work3. Shut down4. Updated5. > > Restarted6. Do more work and make assertionsThe above happens for every > > test method individually. This takes ~1 minute per method and per > version. > > This is done serially right now for all methods within a class.Our other > > DUnit tests can reuse JVMs. These tests are starting a fresh JVM.How can > we > > make them faster?Option 1: Move every test method into its own class. > This > > will allow us to run them in parallelThis would allow us to run ~45 tests > > in parallelMight increase ease of exploring test suite.Option 2: Move > > rolling upgrade tests into their own Concourse job.This can be done in > > addition to other options.Option 3: Restricting what version jumps are > > supported.Can we decide to not support certain version jumps?Will we > > support jumping from Geode 1.x to arbitrary Geode 2.x version?Option 4: > > Change test runner to run for different version transitions in > parallel.We > > could generate classes that contain the desired source and destination > > versions and then runs those in parallel.Next ActionsOptions 1 & 2 seem > to > > be fairly simple changes without much downside and should get us far. > > Implementing those two solutions and seeing how long that leaves our test > > suite seem like the best first step. Jacob Barrett will take the lead on > > turning these two options into action.* > > >
