Hi all, As you may have been aware, over the past 24 hours we've had a number of issues with the processing of our hooks on our Gitlab instance at invent.kde.org as a result of the upgrade to Gitlab 13.1.
These have now all been resolved. One of the projects the Gitlab developers have been working on over the past few months has been shifting Gitaly (the backend component that manages repositories within Gitlab) to a pure Go based codebase. Prior to these changes, it made use of a Ruby based component known as gitlab-shell for certain operations, including most notably running hooks when pushes were made to repositories. In the Gitlab 13.1 release, they shifted to a pure Go based codebase and away from using gitlab-shell to run hooks. Unfortunately this came with it a number of behavioural changes, including most notably for us the withdrawal of an environment variable, GITALY_REPO, which was only meant for internal use. Our hooks had previously relied upon this variable to determine where a repository is located on Gitlab (frameworks/extra-cmake-modules for instance) as the paths on disk are a hash of the unique repository number (@hashed/de/83/de83c8c1ab866148371b00e8e562bfb7b6de32104a2979788eba4982b9fe340e.git for instance). As a consequence of it's removal, our hooks were no longer able to determine the path for a repository and acted as if all repositories were personal repositories. This meant that notifications were not triggered, meaning mails were not sent to kde-comm...@kde.org and broadcasts were not made on IRC. It also meant that Bugzilla entries would not be changed if "BUG: " was used in a commit message. As an interim measure to correct this issue, our hooks were switched to retrieve the path to a repository from the on disk repository configuration (.git/config) while we waited for a proper fix. This value is usually written by Gitaly when a repository is either created or moved. Unfortunately due to a bug in Gitaly, this value is not written when forking a repository, which meant that our hooks would be unable to determine their location for any newly created fork. The Gitaly developers have now added a new environment variable, GL_PROJECT_PATH, which provides this necessary information, allowing for our hooks to be restored to full working order again (regardless of whether this is a new fork or a normal mainline repository). Additionally, there was also a change in behaviour surrounding the execution of post-acceptance hooks on the server side. This change resulted in any background process launched by a post-acceptance hook being killed by Gitaly when the post-acceptance hook itself exited. As a consequence of this, our mirroring to GitHub, along with the triggering of Jenkins (build.kde.org and binary-factory.kde.org) did not take place as expected during this time. This has also now been corrected, by having the post-acceptance hook wait for them to complete before exiting. This does unfortunately mean however that pushes may take slightly longer to complete while the mirroring to GitHub and triggering of Jenkins take place. Our apologies for the disruption caused by this upgrade. Hopefully going forward with the transition being complete now for Gitaly/gitlab-shell there won't be too much more in the way of breaking changes. On the upside of this however has been a number of new features included in the Gitlab 13.1 release, a full list of which can be found at https://about.gitlab.com/releases/2020/06/22/gitlab-13-1-released/ Most notably for us, this includes Merge Request Reviews, which brings a more Reviewboard type experience to performing reviews process, allowing for just a single email to be sent for a whole review, rather than a single email for each comment on a line of code made. Please stay tuned for updates concerning mailing list subscriptions to Gitlab activity, which will be made in the coming week. Please let us know if you have any questions regarding the above. Thanks, Ben Cooksley KDE Sysadmin
