On Thu, Apr 27, 2023 at 02:25:16PM -0300, Fabiano Rosas wrote: > Save a bit of build time by passing the number of jobs option to > sphinx. > > We cannot use the -j option from make because meson does not support > setting build time parameters for custom targets. Use nproc instead or > the equivalent sphinx option "-j auto", if that is available. > > Also make sure our plugins support parallelism and report it properly > to sphinx. Particularly, implement the merge_domaindata method in > DBusDomain that is used to merge in data from other subprocesses. > > before: > $ time make man html > ... > [1/2] Generating docs/QEMU manual with a custom command > [2/2] Generating docs/QEMU man pages with a custom command > > real 0m43.157s > user 0m42.642s > sys 0m0.576s > > after: > $ time make man html > ... > [1/2] Generating docs/QEMU manual with a custom command > [2/2] Generating docs/QEMU man pages with a custom command > > real 0m25.014s > user 0m51.288s > sys 0m2.085s
On my 12 CPU laptop I see a similar magnitude benefit - about 20 seconds is cut from the docs build time - 50 down to 30 secs. Watching the CPU usage I see sphinx is not very good at keeping all CPUs busy. For perhaps 2 seconds I'll see 8 sphinx processes burning CPUs, but the majority of the time it'll only be 1 or 2 sphinx processes. IOW, we do get a benefit, but it is not nearly as good as one might hope for given the number of CPUs potentially available. > Signed-off-by: Fabiano Rosas <[email protected]> > --- > docs/meson.build | 12 ++++++++++++ > docs/sphinx/dbusdomain.py | 4 ++++ > docs/sphinx/fakedbusdoc.py | 5 +++++ > docs/sphinx/qmp_lexer.py | 5 +++++ > 4 files changed, 26 insertions(+) Tested-by: Daniel P. Berrangé <[email protected]> > > diff --git a/docs/meson.build b/docs/meson.build > index f220800e3e..9e4bed6fa0 100644 > --- a/docs/meson.build > +++ b/docs/meson.build > @@ -10,6 +10,18 @@ if sphinx_build.found() > SPHINX_ARGS += [ '-W', '-Dkerneldoc_werror=1' ] > endif > > + sphinx_version = run_command(SPHINX_ARGS + ['--version'], > + check: false).stdout().split()[1] > + if sphinx_version.version_compare('>=5.1.2') > + SPHINX_ARGS += ['-j', 'auto'] > + else > + nproc = find_program('nproc') > + if nproc.found() > + jobs = run_command(nproc, check:false).stdout() > + SPHINX_ARGS += ['-j', jobs] > + endif > + endif ANy reason for check: false in these 2 run_command calls ? They'll both return 0 on success, so I would have though 'check: true' was more robust at error reporting ? With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
