TL;DR can we please fix salsa-ci, gbp and dgit?
Good morning,
This is certainly not the most pleasant subject, but as several critical
tools and services are currently taking a direction that is, in my
opinion, misguided, a discussion here may help to steer that ship in a
way that will prevent further damages.
For a bit of context and history, gitattributes were introduced [1] with
git 1.5.2 released on 2007-05-20 [2], almost 18 years ago.
As with many things with git, proper use of various gitattributes
features is not always intuitive and requires carefully reading the
documentation and eventually experimenting to make sure they won't bite
you later. As such, it is expected that some upstream projects committed
some problematic settings at some point in time and eventually fixed the
settings or the repository later. In the meantime, it is possible that
these problematic settings caused or still cause build issues in Debian
packages. If you happen to know some examples of such past or present
breakage, please kindly share the references here so we can discuss the
failure modes and possible appropriate mitigations, especially if you
were using a gbp or dgit-based workflow at that time.
The current mitigations are implemented as a feature of gbp
setup-gitattributes named "defuse-gitattributes" and released with
version 0.9.23 on 2021-06-08 [3] (a bit less than 4 years ago), inspired
after a feature of dgit named setup-gitattributes that appeared with
dgit 3.3 [4], released on 2017-01-16 (8 years ago).
These mitigations are fundamentally flawed because they don't — and
can't — cover all legitimate use cases, resulting in builds breaking in
non-obvious ways, as is documented in salsa issues, even when using the
"right" tools (e.g. gbp) as documented. They are also based on wrong
assumptions about git's design.
The author of that dgit feature states his beliefs pretty explicitly in
[5] among other places:
This isn't compatible with dgit's core invariant, which is that the
git tree object is precisely the same as the content of the source
package. (The alternative invariant would be that source package is
identical to content of the working tree *as transformed by
gitattributes* - but the gitattributes are typically
context-sensitive, lossy, and very complex, so that isn't workable.)
I agree that this whole situation is not optimal. To be honest,
I think the whole gitattributes system in git is a mistake.
Unfortunately the only correct git invariant, as also very explicitly
stated by the designers of git (see [1] and the discussions that
follow), is the alternative one that was rejected by dgit's author.
My opinion is that the author above and subsequent adopters of that
approach underestimate the issues caused by using git in a way that is
basically incompatible with all other git tools in existence, including
upstream repository hosting, git itself and all current IDEs, and that
this approach qualifies as yet another wicked way to misuse
gitattributes configuration (and thus contributing to the set of
problems they may cause) rather than as a "workable" way to avoid them.
It actually only works in very specific workflows.
A codesearch.d.n query [6] finds 804 unique source packages that include
.gitattribute files with settings possibly influencing text file
conversion on checkout, so this is not quite a niche issue either.
So far 4 packages [7] had to explicitly set
SALSA_CI_DISABLE_GBP_SETUP_GITATTRIBUTES to 1 as the default
configuration causes the build to fail in a non obvious way. I had to
ask on #salsa-ci what could cause an affected build [8] to fail, as it
never occured to me that simply using gbp clone could mess with git
configuration in a way that would cause an otherwise perfectly fine
repository to systematically fail the builds.
Reporting issues against salsa-ci [9] [10] and gbp [11] [12] was so far
reminiscent of the famous "ABC game" of bureaucracies [42]: A — we do it
this way because B and C do it this way; B — we do it this way because A
and C do it this way — and then I didn't bother opening an issue with C
as I don't use it and don't plan to in the near future.
Is the Debian project already committed to make a widespread use of
"dgit" repositories that look like git repositories but are not designed
to be compatible with the git repositories that are used everywhere else
— sometimes they are until they aren't — and document that fact and the
very limited set of workflows in which these dgit repositories are
guaranteed to work so contributors can avoid some of the traps; or are
there still other possible outcomes?
Cheers,
[1]:
https://lore.kernel.org/git/pine.lnx.4.64.0702120839490.8...@woody.linux-foundation.org/
[2]:
https://lore.kernel.org/git/7vsl9rq2u2....@assigned-by-dhcp.cox.net/
[3]:
https://salsa.debian.org/agx/git-buildpackage/-/blob/debian/0.9.23/debian/changelog?ref_type=tags#L25
[4]:
https://browse.dgit.debian.org/dgit.git/commit/?h=debian/3.3&id=2a43d50b3d1f62400dfc404d378b0cd9fcb27c85
[5]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1079434#10
[6]:
https://codesearch.debian.net/search?q=%5E%5B%5E%23%5D.*%5B%5Ea-z%5D%28crlf%7Ceol%7Ctext%29%28%5B%5Ea-z%5D%7C%24%29+path%3A%5C.gitattributes&literal=0
[7]:
https://codesearch.debian.net/search?q=SALSA_CI_DISABLE_GBP_SETUP_GITATTRIBUTES&literal=1
[8]: https://salsa.debian.org/jpd/kotlin/-/jobs/7093478#L3479
[9]: https://salsa.debian.org/salsa-ci-team/pipeline/-/issues/409
[10]:
https://salsa.debian.org/salsa-ci-team/pipeline/-/merge_requests/581
[11]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1092800
[12]: https://salsa.debian.org/agx/git-buildpackage/-/merge_requests/38
[42]: https://archive.org/details/msdos_Bureaucracy_1987
--
Julien Plissonneau Duquène
