On Fri, Apr 3, 2020 at 6:23 PM Joel Sherrill <j...@rtems.org> wrote: > > > On Fri, Apr 3, 2020 at 5:26 PM Chris Johns <chr...@rtems.org> wrote: > >> On 2020-04-04 07:57, Joel Sherrill wrote: >> > How does rtems-bsp-builder calculate the amount of parallelism to use? >> >> The option is documented here ... >> >> >> https://docs.rtems.org/branches/master/user/tools/bsp-builder.html#cmdoption-jobs >> >> and is ... >> >> --jobs >> >> The jobs options where the format is build-jobs/make-jobs. The default >> is 1/num-cores where num-cores is the operating system reported number >> of cores. >> > > What's the definition of build job vs make job? > > Is make jobs equivalent to N in make -jN? > > > > > >> >> > I ask because I have realized that I selected jobs=6/12 on a very beefy >> > machine with 12 cores and 32 GB RAM. The main test machine I use is 8 >> > cores/32 GB RAM. But the slowest ones are old with 4 cores/4 GB RAM. >> > They are hitting the swap space. All have SSDs. >> > >> > Should I drop the --jobs parameter? Will the script select a decent >> load >> > automatically? The machines are essentially dedicated to testing. >> >> You will need a --jobs option or the machine will not be fully loaded. >> You need to find a balance between the number of parallel BSPs that are >> built at once and the number of parallel build jobs each BSP makes runs. >> >> The critical factor is the amount of bootstrap and configure time. This >> is a single core task and if you use the default a lot of the build time >> the machine is basically idle. >> > > But if you go too high, the number of compiles will swamp a machine > without a huge amount of memory. So the 4 core/4 GB machines > probably would be something like --jobs=2/4. if that's two bsp builds > in parallel with a -j4. > > That generally gives you a 2x factor on processes which is close > to the old school rule of the -jN number should be between 1.5 and > 2 times the number of cores as long as the RAM and disk subsystem > could handle it. That tends to overlap IO and computation. > > Maybe the default should be (roundup(cores)/2) instead of 1. > > Temporarily I may have to add a jobs option to my script and hand-tune > it to the machines I am testing on. > > >> There is no way I know of to automatically find this balance for all the >> different types of host machines. >> > Thinking a bit more, I wonder if there could be some named build/job ratios. 1/cores, (cores/2)/cores, etc.
> >> Chris >> >
_______________________________________________ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
