On 2020-04-04 12:28, Joel Sherrill wrote:
On Fri, Apr 3, 2020 at 6:23 PM Joel Sherrill <j...@rtems.org <mailto:j...@rtems.org>> wrote:On Fri, Apr 3, 2020 at 5:26 PM Chris Johns <chr...@rtems.org <mailto:chr...@rtems.org>> wrote: On 2020-04-04 07:57, Joel Sherrill wrote: > How does rtems-bsp-builder calculate the amount of parallelism to use?The option is documented here ... https://docs.rtems.org/branches/master/user/tools/bsp-builder.html#cmdoption-jobs and is ... --jobs The jobs options where the format is build-jobs/make-jobs. The default is 1/num-cores where num-cores is the operating system reported number of cores. What's the definition of build job vs make job? Is make jobs equivalent to N in make -jN? > I ask because I have realized that I selected jobs=6/12 on a very beefy > machine with 12 cores and 32 GB RAM. The main test machine I use is 8 > cores/32 GB RAM. But the slowest ones are old with 4 cores/4 GB RAM. > They are hitting the swap space. All have SSDs. > > Should I drop the --jobs parameter? Will the script select a decent load > automatically? The machines are essentially dedicated to testing. You will need a --jobs option or the machine will not be fully loaded. You need to find a balance between the number of parallel BSPs that are built at once and the number of parallel build jobs each BSP makes runs. The critical factor is the amount of bootstrap and configure time. This is a single core task and if you use the default a lot of the build time the machine is basically idle. But if you go too high, the number of compiles will swamp a machine without a huge amount of memory. So the 4 core/4 GB machines probably would be something like --jobs=2/4. if that's two bsp builds in parallel with a -j4. That generally gives you a 2x factor on processes which is close to the old school rule of the -jN number should be between 1.5 and 2 times the number of cores as long as the RAM and disk subsystem could handle it. That tends to overlap IO and computation. Maybe the default should be (roundup(cores)/2) instead of 1. Temporarily I may have to add a jobs option to my script and hand-tune it to the machines I am testing on. There is no way I know of to automatically find this balance for all the different types of host machines. Thinking a bit more, I wonder if there could be some named build/job ratios. 1/cores, (cores/2)/cores, etc.
As long and we keep the fact one is the number of jobs given to a build or `make` or whatever for a package. I am happy to review a patch.
Chris _______________________________________________ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
