Re: request to reopen ticket #24522: --random test option

2021-03-12 Thread Florian Apolloner
Thank you for diving into how unittest discovery tests! On Friday, March 12, 2021 at 12:14:05 PM UTC+1 chris.j...@gmail.com wrote: > But, regardless of the default, to get the full benefit I think you'd want > at least one of the test runs in CI to be a random one. > This is were I somewhat dis

Re: request to reopen ticket #24522: --random test option

2021-03-12 Thread Chris Jerdonek
On Friday, March 12, 2021 at 10:47:56 AM UTC+1 Florian Apolloner wrote: > Is the current order defined in any way -- if not one should consider it > randoms anyways and as such actually randomizing should not make much of a > difference. FWIW I am fine with making it the default and outputting the

Re: request to reopen ticket #24522: --random test option

2021-03-12 Thread Florian Apolloner
Btw while I am fine with making it default (I really do not care much either way), we certainly need a way to disable it and keep the current algorithm (even though it might not be guaranteed stable). When I develop I often run one suite and fix the errors as they occur. Having a stable order t

Re: request to reopen ticket #24522: --random test option

2021-03-12 Thread Florian Apolloner
Is the current order defined in any way -- if not one should consider it randoms anyways and as such actually randomizing should not make much of a difference. FWIW I am fine with making it the default and outputting the seed at the end of the run so it can reproduced. I am curious if it finds