mukund-thakur commented on code in PR #5081: URL: https://github.com/apache/hadoop/pull/5081#discussion_r1608853482
########## hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md: ########## @@ -539,12 +539,51 @@ Otherwise, set a large timeout in `fs.s3a.scale.test.timeout` The tests are executed in an order to only clean up created files after the end of all the tests. If the tests are interrupted, the test data will remain. +## <a name="CI"/> Testing through continuous integration + +### Parallel CI builds. +For CI testing of the module, including the integration tests, +it is generally necessary to support testing multiple PRs simultaneously. + +To do this +1. A job ID must be supplied in the `job.id` property, so each job works on an isolated directory + tree. This should be a number or unique string, which will be used within a path element, so + must only contain characters valid in an S3/hadoop path element. +2. Root directory tests need to be disabled by setting `fs.s3a.root.tests.enabled` to + `false`, either in the command line to maven or in the XML configurations. + +``` +mvn verify -T 1C -Dparallel-tests -DtestsThreadCount=14 -Dscale -Dfs.s3a.root.tests.enabled=false -Djob.id=001 +``` + +This parallel execution feature is only for isolated builds sharing a single S3 bucket; it does +not support parallel builds and tests from the same local source tree. Review Comment: ah..already mentioned here so we are good. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
