This is an automated email from the ASF dual-hosted git repository.
aherbert pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/commons-rng.git
commit 741d6a62f903c6a3f7b9830c8c68d2c7f1206830
Author: aherbert <aherb...@apache.org>
AuthorDate: Mon Oct 10 13:15:07 2022 +0100
Add note about 'flaky' tests to developer guide
---
src/site/xdoc/developers.xml | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/src/site/xdoc/developers.xml b/src/site/xdoc/developers.xml
index 8e93bf1f..84ab9604 100644
--- a/src/site/xdoc/developers.xml
+++ b/src/site/xdoc/developers.xml
@@ -245,6 +245,15 @@
expected.
</li>
</ul>
+ <p>
+ It must be noted that, due to the nature of random number
generation, some unit tests
+ are bound to fail with some probability. This is applicable for
tests that compare
+ random output to an expected distribution. These tests <i>should</i>
<b>not</b>
+ use a fixed seed to ensure consistent output. The
'maven-surefire-plugin' is configured to
+ re-run tests that fail, and pass the build if they succeed within
the allotted number of
+ reruns (the test will be marked as 'flaky' in the report). Any test
that is consistently
+ 'flaky' may require an update to the test assumptions and assertions.
+ </p>
</subsection>
<subsection name='Licensing and copyright'>
<ul>
