The gnulib sources are large but more importantly have lots of changes. So initial checkout of the repository can take a long time when network or cpu resources are limited. The later is especially acute in a non-KVM QEMU virtual machine (which can take 40+ minutes compared to <30 seconds with this change[1]). The problem is specific to bootstrap configurations using a specific gnulib revision specified by commit hash. In this case, git can not do a shallow clone by using the --depth option because git does not know ahead of time how deep the revision is from the tip. So git must clone the whole repository.
However, there is an alternate method that requires support from the git server[2], namely by asking for a specific commit on fetch. Refactor to use fetch and fallback to fetching the entire repository if fetching by commit hash fails. Currently the git server hosting the official gnulib git repository does not support fetch by commit hash[3]. However, there are mirrors which do support this[4], and can be specified by setting the $GNULIB_URL. [1] https://savannah.nongnu.org/support/index.php?110553#comment1 [2] https://stackoverflow.com/a/3489576/2108011 [3] https://savannah.nongnu.org/support/index.php?110553 [4] https://github.com/coreutils/gnulib Signed-off-by: Glenn Washburn <developm...@efficientek.com> --- Hi list, This patch can have a dramatic performance improvement in checking out gnulib sources via bootstrap. These changes use the git -C which was introduced in 2013 in v1.8.5, so I think its been around long enough to rely on it. Also, I'm not subscribed to the list, so please ensure to CC me on any replies to this patch. Thanks, Glenn --- build-aux/bootstrap | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/build-aux/bootstrap b/build-aux/bootstrap index 733659850..9f8b918a2 100755 --- a/build-aux/bootstrap +++ b/build-aux/bootstrap @@ -694,9 +694,21 @@ if $use_gnulib; then shallow= if test -z "$GNULIB_REVISION"; then git clone -h 2>&1 | grep -- --depth > /dev/null && shallow='--depth 2' + git clone $shallow ${GNULIB_URL:-$default_gnulib_url} "$gnulib_path" \ + || cleanup_gnulib + else + git fetch -h 2>&1 | grep -- --depth > /dev/null && shallow='--depth 2' + mkdir -p "$gnulib_path" + git -C "$gnulib_path" init + git -C "$gnulib_path" remote add origin ${GNULIB_URL:-$default_gnulib_url} + # Can not do a shallow fetch if fetch by commit hash fails because we + # do not know how deep to go to get to $GNULIB_REVISION, so we must get + # all commits. + git -C "$gnulib_path" fetch $shallow origin "$GNULIB_REVISION" \ + || git -C "$gnulib_path" fetch origin \ + || cleanup_gnulib + git -C "$gnulib_path" reset --hard FETCH_HEAD fi - git clone $shallow ${GNULIB_URL:-$default_gnulib_url} "$gnulib_path" \ - || cleanup_gnulib trap - 1 2 13 15 fi -- 2.27.0
