Hi, I'm writing to ask whether continuous integration capabilities have been considered as a factor in the evaluation of new "forge" services for the FSF. I read through <https://libreplanet.org/wiki/FSF_2020_forge_evaluation> and did not see anything about it.
I've been a contributor to GNU projects for many years, notably both GCC and GNU libc, and recently I led the effort to make the first release of Autoconf since 2012. Autoconf is hosted on Savannah at present, GCC and glibc on sourceware.org; none of these services have built-in continuous integration. Several corporate contributors to GCC and glibc run their own CI and post test results to the mailing lists, but there's no dashboard and no automatic testing of submitted patches. Autoconf presently has no CI at all. The lack of CI was a major hindrance during the run-up to the new Autoconf release--I found dozens of regressions from the previous release, some of which had gone undetected for *years*, because of inadequate testing. The incomplete Autoconf test suite is partially to blame here, but even so, it was easier for these bugs to slip in because there was no requirement for the test suite to pass before merging a patch, and no automated way for me to check for regressions on the many platforms and architectures Autoconf supports. CI is also a major selling point for the proprietary forge services that the FSF's forge would be competing with. For example, I am also a co-maintainer of libxcrypt <https://github.com/besser82/libxcrypt> which has replaced part of GNU libc on most Linux distributions. As you can see from the URL, it's currently hosted on Github, and it relies on TravisCI and coverage.io for continuous integration and related functions. Moving to a Free forge would be a much easier argument for me to make if equivalent CI services were available from the new host. Of the three software packages currently on the short list, I know that sourcehut does support CI to some extent. I intend to experiment with its CI for Autoconf in the near future, using a mirror repository I created on sourcehut's self-hosted instance, <https://git.sr.ht/~zackw/mirror-autoconf>. I don't know about either of the others. I realize CI requires significantly more resources to support than most of the other aspects of a forge service, but it is also significantly more valuable to maintainers than some of the other practical concerns that are listed in the evaluation (e.g. the exact protocol for patch submission). It might be possible to share resources with the GCC Compile Farm <https://gcc.gnu.org/wiki/CompileFarm> or with other such existing clusters. Thanks, zw
