ftpmirror.gnu.org mirrors failing

2020-12-30 Thread Insurgo via Gcc
Hello there,

Since a couple of days, it seems that http://ftpmirror.gnu.org/ randomly
points to non-functional mirrors to download coreboot relative packages
in CI builds.

I would say that CircleCI fails 2/3 of the time in current builds, at
different places.

As you may know, building coreboot requires to download different
tarballs prior of building its own musl-cross toolchain prior of
building coreboot from it.

In the case of Heads project, which builds boards ROMs for different
coreboot versions, it means that each time that a Heads' module
definition changes, the original coreboot musl-cross related tarballs
need to be downloaded again prior of being in CircleCI's cache for next
builds only related to scripts changes.

Consequently, in the past (3?) days, CircleCI attempted to download
packages from different ftpmirror.gnu.org mirrors, sometimes being
successful, and sometimes failing after 2h of building time. I'm trying
to build for more then 24 hours now with really low success rate from
that situation, and i'm seeking at least acknowledgement and/or some
advices, since pinning coreboot to selected mirrors would be a hack and
not interesting in the long term.


The output exerpt of CircleCI looks like this, which doesn't help much:

make -C util/crossgcc build-i386 build_iasl SKIP_GDB=1
bash ./buildgcc -p i386-elf -j 4   \
         -d /root/project/build/coreboot-4.8.1/util/crossgcc/xgcc
Welcome to the coreboot cross toolchain builder v1.52 (June 11th, 2018)

Building toolchain using 4 thread(s).

Target architecture is i386-elf

Found compatible Ada compiler, enabling Ada support by default.

Downloading and verifing tarballs ...
 * gmp-6.1.2.tar.xz (downloading from
https://ftpmirror.gnu.org/gmp/gmp-6.1.2.tar.xz)...   0%  2% 
5%  7% 10% 13% 15% 18% 21% 23% 26%
28% 31% 34% 36% 39% 42% 44% 47% 49%
52% 55% 57% 60% 63% 65% 68% 71% 73%
76% 78% 81% 84% 86% 89% 92% 94% 97%
99%100%... hash verified ("9dc6981197a7d92f339192eea974f5eca48fcffe")
 * mpfr-3.1.5.tar.xz (downloading from
https://ftpmirror.gnu.org/mpfr/mpfr-3.1.5.tar.xz)...   0%... Failed to
download mpfr-3.1.5.tar.xz.
make[3]: *** [Makefile:26: build_gcc] Error 1
make[2]: *** [Makefile:48: build-i386] Error 2
make[1]: *** [util/crossgcc/Makefile.inc:46: crossgcc-i386] Error 2
make[1]: Leaving directory '/root/project/build/coreboot-4.8.1'
make: *** [modules/coreboot:75:
"/root/project/build/coreboot-4.8.1/.xcompile"] Error 2

CircleCI received exit code 0

And where local tests, happening throuhg QubesOS and whonix, tries
different mirrors at each attempt which normally succeed.

To I guess it comes to where CircleCI's instance is located which
selects failing mirrors.


If you can do anything about this, that would be helpful. Currently, i'm
sure everyone using CircleCI to build stuff depending on your mirrors
are having big migraines.


Thanks!

Thierry Laurion

https://insurgo.ca





Re: ftpmirror.gnu.org mirrors failing

2020-12-30 Thread Insurgo via Gcc


On 12/30/20 4:18 PM, Insurgo wrote:
> Hello there,
>
> Since a couple of days, it seems that http://ftpmirror.gnu.org/ randomly
> points to non-functional mirrors to download coreboot relative packages
> in CI builds.
>
> I would say that CircleCI fails 2/3 of the time in current builds, at
> different places.
>
> As you may know, building coreboot requires to download different
> tarballs prior of building its own musl-cross toolchain prior of
> building coreboot from it.
>
> In the case of Heads project, which builds boards ROMs for different
> coreboot versions, it means that each time that a Heads' module
> definition changes, the original coreboot musl-cross related tarballs
> need to be downloaded again prior of being in CircleCI's cache for next
> builds only related to scripts changes.
>
> Consequently, in the past (3?) days, CircleCI attempted to download
> packages from different ftpmirror.gnu.org mirrors, sometimes being
> successful, and sometimes failing after 2h of building time. I'm trying
> to build for more then 24 hours now with really low success rate from
> that situation, and i'm seeking at least acknowledgement and/or some
> advices, since pinning coreboot to selected mirrors would be a hack and
> not interesting in the long term.
>
>
> The output exerpt of CircleCI looks like this, which doesn't help much:
>
> make -C util/crossgcc build-i386 build_iasl SKIP_GDB=1
> bash ./buildgcc -p i386-elf -j 4   \
>          -d /root/project/build/coreboot-4.8.1/util/crossgcc/xgcc
> Welcome to the coreboot cross toolchain builder v1.52 (June 11th, 2018)
>
> Building toolchain using 4 thread(s).
>
> Target architecture is i386-elf
>
> Found compatible Ada compiler, enabling Ada support by default.
>
> Downloading and verifing tarballs ...
>  * gmp-6.1.2.tar.xz (downloading from
> https://ftpmirror.gnu.org/gmp/gmp-6.1.2.tar.xz)...   0%  2% 
> 5%  7% 10% 13% 15% 18% 21% 23% 26%
> 28% 31% 34% 36% 39% 42% 44% 47% 49%
> 52% 55% 57% 60% 63% 65% 68% 71% 73%
> 76% 78% 81% 84% 86% 89% 92% 94% 97%
> 99%100%... hash verified ("9dc6981197a7d92f339192eea974f5eca48fcffe")
>  * mpfr-3.1.5.tar.xz (downloading from
> https://ftpmirror.gnu.org/mpfr/mpfr-3.1.5.tar.xz)...   0%... Failed to
> download mpfr-3.1.5.tar.xz.
> make[3]: *** [Makefile:26: build_gcc] Error 1
> make[2]: *** [Makefile:48: build-i386] Error 2
> make[1]: *** [util/crossgcc/Makefile.inc:46: crossgcc-i386] Error 2
> make[1]: Leaving directory '/root/project/build/coreboot-4.8.1'
> make: *** [modules/coreboot:75:
> "/root/project/build/coreboot-4.8.1/.xcompile"] Error 2
>
> CircleCI received exit code 0
>
> And where local tests, happening throuhg QubesOS and whonix, tries
> different mirrors at each attempt which normally succeed.
>
> To I guess it comes to where CircleCI's instance is located which
> selects failing mirrors.
>
>
> If you can do anything about this, that would be helpful. Currently, i'm
> sure everyone using CircleCI to build stuff depending on your mirrors
> are having big migraines.
>
>
> Thanks!
>
> Thierry Laurion
>
> https://insurgo.ca
>
All failing builds here are link to this problem (if you want to take a
look, would just give proof, but not much more information, since
coreboot doesn't output on screen (console) the actual server being
used: https://app.circleci.com/pipelines/github/osresearch/heads

Thanks!

>