On Fri, 2 Aug 2024 at 21:29, Helmut Grohne <hel...@subdivi.de> wrote: > > Hi Luca, > > On Fri, Aug 02, 2024 at 01:43:04AM +0100, Luca Boccassi wrote: > > 1) The os-release specification must be adhered to, and it must be > > possible to tell the difference between testing vs unstable, and each > > must be correctly identified by the respective metadata > > Given the state of discussion, I think it is fairly evident that we do > not have consensus for the need to distinguish testing and unstable. We > have people arguing in favour and we have people arguing against that. > This is a point of disagreement.
I'm not sure "consensus for the need" is the right term here? People obviously _do_ what I am speaking of. There might not be consensus that's it's a good idea to do it, but it still happens, and it will continue to happen, regardless of what is decided here. People do need to distinguish it, whether the TC rules that it should be easy to do so like in any other distro, or whether it rules that it will continue to require annoying Debian-specific kludges, won't change this fact. > > 2) Testing and unstable can continue to remain indistinguishable, and > > both be erroneously identified as trixie > > Isn't there the third option of adhering to the os-release specification > without making testing and unstable distinguishable? I did not see this > ranked in your preference. Do you see it as even worse than the status > quo? There isn't such option. Adhering to the specification means identifying them separately, given they can be built separately, ran separately, managed separately. So the option you are referring to is for the opposite: _not_ adhering to the specification, and yes, that is an option. Well, I guess there could be yet another option: stop making it possible to create either testing or unstable images separately in the first place, just like you cannot create a stable-proposed image. > > Sorry but I do not think that is an accurate representation. First of > > all, the implementation of the spec is bugged, period - it's not about > > being pedantic about it, it's about being completely incompatible: sid > > is identified as trixie, and that is just plain wrong, and there's no > > room for interpretation, no matter how one might try to bend it. You > > might say that you don't _care_ that the implementation is buggy, you > > might even say that it is worth, nay _good_ it to leave it buggy - but > > buggy it is, if I create a sid image, it will never in a million years > > be trixie, and yet it says it's trixie. > > I think it has become abundantly clear that your view on this does not > represent consensus of discussion participants. It is not as black and > white as you paint it. Simon compared unstable to Ubuntu's proposed > pocket. It also happens that testing and unstable share the same pool > hierarchy and the vast majority of packages. On the other hand, Devuan > operates as an overlay repository to be added on top of a Debian > repository. I don't think it matters that Ubuntu's proposed is > implemented as a partial distribution overlaid onto another as that's > what other derivatives (that do update their os-release file) do as > well. Sure, there's no consensus on the solution, that's obvious otherwise I would not have needed to open this in the first place, right? There are some things that require opinions to converge - should this bug be fixed? - and there are some things that just are - people create and manage and run sid and trixie images independently, the implementation of the spec in sid is buggy. These last two statements do not need consensus, they are facts. Deciding what to do based on those facts requires consensus on the solution to adopt, if any. The example of Ubuntu's proposed pocket is wrong as already mentioned, from the point of view of the os-release specification, which is what matters here. You cannot create an ubuntu-proposed image without the full release, so os-release is not concerned with it, and doesn't mandate anything in particular. os-release concerns images that have a lifecycle and their identification. Trixie has a lifecycle: it started as a development release, it will be declared stable and start receiving security support, it will move to ELTS, it will go EOL. sid will not do any of these things, its lifecycle is entirely and completely separate, it will always be sid, it will never be security supported, it will never go EOL, and it is a rolling release. Once more for those at the back: the _content_ is irrelevant, it can be fixed, it can change a lot, it can be the same as different releases, it can be partially the same, it can be completely different, it doesn't matter one bit. The lifecycle is what matters for os-release. This is an extremely important distinction and it is crucial here, because this is what os-release is about, and this bug is about os-release and its implementation, not about generic ideas. As already mentioned many times, this bug is not about a philosophical discussion of what unstable is and what testing is, or what they should be, or how they should be used. They already exist whether we like it or not, they are already used as they are used whether we like it or not, and nothing here will change any of that. > It would help if you could stop dismissing other people's views and > respectfully disagree with their views instead. Refer to Simon's reply > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1077764#118 for more > details on this aspect. I am not dismissing any view. Pointing out that there are facts and there are opinions is not dismissing, it's clarifying. Saying "when you wrote 2+2=5 it was incorrect" is not dismissing you, it's helping you, and nobody should feel attacked because of it. It's impossible to debate opinions and options without an underlying baseline of truths and facts, so pointing out incorrect statements is necessary. Because of what I do I am in the best position within this thread for providing facts about an extremely narrow and tiny topic, namely the nature of os-release. Others are best placed at providing facts about other topics, such as implications of the use of t-p-u as discussed in the RT subthreads, just to name two examples. Please see my reply at: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1077764#128 > > Secondly, I am not even sure what these conflicting requirements > > actually are? Could you please spell them out? If trixie was > > identified as trixie, and sid was identified as unstable, what > > compromise would be, er, compromised, precisely? Because the only > > practical objections I could find were based on the idea that > > implementations would be ugly or hard to maintain or so - as already > > mentioned, I am happy to relieve anybody else and take that hard and > > ugly maintenance burden for myself. What other actual problem would > > suddenly appear? What feature or advantage do we leave behind? How are > > things made worse for our users? > > The devil is in the detail. Debian currently does not provide a simple > way to distinguish testing and unstable. As a result, some users have > opted to not rely on such differentiation and instead inspect particular > package versions for the features they are interested in. Others have > implemented heuristics to somehow tell testing and unstable apart as > they saw a need for doing so (that I still fail to see at this time). > Whilst you and others have presented code for telling testing and > unstable apart in multiple places, I am still having a hard time > figuring out why we had to tell them apart and in what way code would > behave differently for one or the other. If Debian were to differentiate > testing and unstable in a simple way, people would start relying on > that. That in turn (as laid out by Russ and Simon) would degrade our > distribution testing as code that was working in unstable may stop > working once it migrates to testing for the sole reason of inspecting > /etc/os-release. To put this another way, enabling users to > differentiate between testing and unstable would encourage developers to > write code that some of us see as broken by design. Users are already doing this, including myself, for reasons that are their own and are irrelevant here. You've never been in a position to do such things, and so it's not been a problem for you, so you don't understand why it would happen - that is entirely, 100% ok. But it's not important. The only question is whether they do that and then say "it's nice that we have a common, standard, agnostic way of figuring this out and it just works (TM) on Debian too", or, "man this Debian thing sure is a gigantic pile of rubbish, it's so painful to deal with it as opposed to literally everything else, why do I even bother with it". Currently it's the latter. You can say it's fine that it's the latter. I say it's not and I want to fix it. > > in the weirdest places. Here's one I am well familiar with as the git > > history will show: > > > > https://github.com/openSUSE/obs-build/blob/master/build-recipe-livebuild#L138 > > This one vastly predates the introduction of os-release and the actual > problem being solved there is to not distinguish testing from unstable > and rather treat unstable in the same way as testing even though > lsb_release would differentiate them. Wrong. It is exactly there because of the impossibility of distinguishing testing vs unstable. It's correct that it predates os-release - it's quite common that very old hacks that have been there for a long time hack around debian_version, as you said it's much older. But they still deal with the same fundamental problem. If you check my bug report against base-files, I originally proposed to fix debian_version too, but I have changed my mind since then, so it's not mentioned here. > > build live-build based images in the open build service, because of > > this base-files bug. Fortunately at the time we didn't care about > > that, for external reasons, so it didn't cause troubles, besides > > needing to hack it. > > > > Here's a random one, first result found searching on Github: > > > > https://github.com/lpereira/hardinfo/blob/4c97625c5666fa5fc353e7cab322b09159e54ed4/modules/computer/os.c#L486 > > > > /* HACK: Some Debian systems doesn't include the distribuition > > * name in /etc/debian_release, so add them here. */ > > Again, this is not concerned with os-release, but with lsb_release > --codename attempting to differentiate between testing and unstable. The > actual result here is that DIST would contain the codename of testing > when run in an unstable environment. > > Both of these are likely no longer needed. No, they are needed. As above, there's older stuff around, but the core issue being worked around is still the same, and still what this bug is about. > > Note how it starts with HACK, all caps? This is what we subject our > > users to. This is what we are known for. What do we even gain in > > exchange? > > I hear you and agree that we should get rid of this code once we no > longer need to support old releases. On the flip side, my reading of the > examples you presented strengthens the position that testing and > unstable should indeed be indistinguishable, which likely is not what > you intended. So I show you a bunch of Debian-specific hacks and your response is, "yes let's keep making sure those horrible hacks are necessary and everyone has to reimplement them, forever"? I mean, that's a legitimate point of view of course, I just find it very strange, that's all. > > "Ah it's Debian Trix..." > > > > $ sudo build/systemd-dissect --copy-from image.raw > > /etc/apt/sources.list /tmp/sources > > $ grep Suites /tmp/sources > > Suites: unstable > > Suites: unstable-debug > > > > "Oh wait no, it's Debian Si..." > > > > $ sudo build/systemd-dissect --copy-from image.raw /etc/debian_version > > /tmp/debian_version > > $ cat /tmp/debian_version > > trixie/sid > > > > "Oh no, wait, it's... both?!" > > You successfully detail how testing and unstable are difficult to > distinguish. What I still fail to see is what negative consequence there > is as a result. I just showed you - I have two images, with radically different lifecycles, and I cannot tell them apart. I can tell apart any other version of any other distribution, not this one. That's a negative consequence for me. If you meant to say it's ok for you that there is such a negative consequence for me, well, ok, that's not great, but fine I guess. But it should be pretty obvious that it is negative for me: I am telling you it is. > > > Consider adding a new Debian binary package containing /etc/os-release > > > with contents suitable for unstable. This package would use the > > > dpkg-divert mechanism to avoid a conflict with base-files which would > > > continue to install /etc/os-release with contents suitable for testing > > > (even in unstable). We would mark this new package "Essential: yes", > > > but have no other package depend on it. We would also file a permanent > > > rc-bug to prevent it from migrating to testing. As a result, the set of > > > essential packages as discovered by bootstrapping tools would differ > > > between testing and unstable and the /etc/os-release would be diverted > > > for unstable only. A significant downside of this approach is that all > > > existing unstable systems would never install this package (and thus > > > continue to be seen as testing installations) and all systems that were > > > bootstrapped as unstable would be keep this package even if they were to > > > be downgraded to testing at a later time (unless the administrator > > > actively removes the new package). > > > > The downsides you have mentioned are significant already, and also as > > you know I very strongly dislike dpkg-diverts, but that's my problem. > > However, having complex maintainer script machinery in the essential > > set is everyone's problem, as it can and will break, and as above, I > > am sure it is best avoided. This still requires blocking the migration > > of a package, so it does not seem to me it provides any advantages > > over simply having an os-release binary package pinned to unstable, > > with no maintainer scripts, and just the content appropriate for > > unstable inside it. It still requires overruling the base-files > > maintainer. What would be the difference/advantage over the other > > approach? > > Unless base-files gains a dependency on the proposed os-release package, > your t-p-u approach shares the downside of existing unstable systems > never getting classified as unstable. Your approach also shares the > downside of systems that are being bootstrapped as unstable keep being > classified as unstable even if they become testing. If you see these as > significant, you should go back to the drawing board. As mentioned in a separate subthread, it doesn't have to be base-files. I could for example add a dependency on the new package to init-system-helpers, for which I am a maintainer, and which is prio essential. This already depends on usrmerge for example. Would this solve your concerns regarding base-files's dependency graph? > > > We have a disagreement about whether the information you intend to > > > convey actually exists beyond the point in time where you deboostrap. If > > > I debootstrap unstable and look at it a week later, it more closely > > > represents testing than unstable. > > > > I don't think so. It is unstable, one week out of date. It's not any > > different than running "debootstrap bookworm" and checking back X > > weeks later, after a point release is available - it will be outdated, > > but still bookworm, and still one apt update; apt upgrade away from > > being up to date again. Same for unstable: if it's a week old, after > > you apt upgrade it, it will be on par again. It will not magically > > become testing by itself. If you debootstrap unstable today and check > > back again in a year, it will not be stable trixie, it will be > > outdated unstable, and an apt upgrade will make it an up to date > > unstable. > > At that point when you have an outdated unstable and a current testing, > differentiating them becomes meaningless as your current testing will > have a lot of more recent packages. What do you do with this knowledge > that one is unstable and the other is testing? It is not meaningless. Again, os-release's VERSION_CODENAME, VERSION_ID and friends do not care about freshness. There's BUILD_ID for timestamping an image on creation, and there's SUPPORT_END to give an image an expiry date. The one true difference is still the same whether they are just built or one year old: a sid image is a rolling image that will never become stable, never receive security support, never move to the LTS team, never go EOL. Its lifespan is unlimited. It must never have SUPPORT_END set. A trixie image is not a rolling release, it has a predecessor, bookworm, it will have a successor, forky, it will become stable, start receiving security support, move to the LTS team, go EOL. It has a finite lifespan and then it has to be thrown out or upgraded to a different release. If we could predict the future we could set SUPPORT_END right now. I think we are a bit more fluid on this, other distributions are quite precise and set an expiration date long before they release. It's ok to not set this field. These differences are crucial for os-release and make all the difference. I appreciate they don't make a difference for you, or for some workflows that you care about, but it does for others. > > > Thank you for referencing a practical effect. It is not clear how mkosi > > > uses the Release field and why it is important to differentiate testing > > > from unstable though. > > > > It's an image builder tool - it builds images, so it needs to know > > _which_ image to build and in this case, I need it to build a guest > > that matches the host - do I pull from testing or unstable? Which one > > is this? Ah it says trixie, so I can use the trixie repository and... > > oh wait now everything is broken because it was actually sid and a > > transition was not finished so I got a broken image. Try again next > > week? > > Both Johannes and me have fallen into this trap. You have create a > flaky autopkgtest that is not actually testing what it is supposed to > test. Rather than inspecting the release name, your test should copy the > hosts' sources and apt pins into the guest. Your typical autopkgtest > runs with both testing and unstable enabled with some packages pinned > from unstable and you need to reproduce this package set inside your > guest. Lest I get accused again of attacking or whatever, I'll start with: thanks for taking the time to look at that, and to think and suggest an alternative. Then, sorry, but this just shows that the amount of time you have spent working on that testsuite is 0. I have spent +infinite on it, and I can say that no, it should not do what you are suggesting. The only improvement I'd like to have for it, is an os-release that works to spec. > This now is an example for why distinguishing testing and unstable bears > a risk of abuse. Thus far, every single use of the testing vs unstable > difference that I've seen in this discussion is better solved in another > way. That is your opinion, and you are free to hold it. In my opinion that is wrong. However, it won't make any difference either way. This will still happen, whether you like it or not, whether you think it's right or not. The only thing you can do as a TC member is decide whether users should continue to curse Debian while they do have to open-code bad workarounds exclusively for it, or whether it should be like any other distribution where it works nicely out of the box and you don't have to even think about it. If you decide for the former, we will just continue to do what we do today, and we will still curse Debian for making us do this. There is nothing you can do, write or say that will change this fact. It's important to be clear about what can change and what cannot, when making a decision. > > Because it's not just standalone images, but images based on other > > images (ie, sysext: > > https://uapi-group.org/specifications/specs/extension_image/ ). If you > > cannot identify an image, what the heck did you just build? And > > against what? And what do you base this other one on? Nobody knows. > > And then the hacks begin. > > If you knew your image was unstable, you'd gain nothing in terms > figuring out what is inside without knowing when it was produced. You > gain something once you add a BUILD_ID. As you rewrite /etc/os-release > anyway, you may update VERSION_CODENAME as well at little extra effort. I would gain a lot. I want to pair bases and extensions according to their lifecycles, otherwise they will diverge and eventually break. If I try to run a trixie extension on a sid base today it will work. If I boot and update them in a year time, it will likely break, as the sid base won't be ABI compatible with the trixie extension anymore at some point, inevitably. That's because in the meanwhile trixie became stable and moved to a different phase in its lifecycle, while sid did not - it's a rolling release. > > Yes, and that solves the "wrong information given" problem (which is > > new, VERSION_CODENAME was introduced somewhat recently in base-files), > > but does not solve the "cannot identify image without horrendous > > hacks" problem, so the implementation is still buggy - less so, but > > still buggy. The point of the spec is to uniquely identify an image, > > and if you are Arch and don't do releases then of course you don't > > specify the release codename (there ain't one), so the field is > > optional to cater to those use cases. It's not optional to cater to > > the "distribution is held together with duct tape and prayers" cases, > > I'm afraid. And if you don't implement the spec to enable its purpose > > of identifying vendor trees, why implement it at all? > > More and more I am reaching the conclusion that not being able to > distinguish testing from unstable is a feature, not a bug. I'm using the > information from /etc/os-release (e.g. via ansible or puppet). For that > use case, Simon gave a recipe that works well with the VERSION_ID as it > is today. I'm trying to better understand other uses of the information > contained. That's not status quo though. They are distinguishable, it's just incredibly painful, error-prone and unreliable to do so, and it's not possible to do it _using exclusively os-release as it's meant to according to spec_ which is the point of this bug. If you decide against this proposal, this will not change. One will still be able to create independently and separately trixie and sid images, and they'll be different, with different lifecycles etc etc already made this point a million times so I won't repeat it here. Unless of course you don't just want to rule "status quo", but you are instead saying that you want to go further than that, and explicitly and categorically say: sid and testing shall be one and the same. Which then means, as the maintainer of several image build tools, such as deboostrap, following such a ruling I would have to apply it and make it impossible for users to create testing images. Only unstable images shall be buildable, because after all, they are the same, so the fact that it is possible to 'deboostrap testing' and get something different than 'deboostrap unstable' is a bug and goes against this hypothetical ruling, so I'll have to solve it, I guess by silently aliasing testing -> unstable and trixie -> sid until release date, and automatically adding apt sources for both testing and unstable to all testing/unstable deboostrap runs. I don't like it, but if the TC decides that way, I'll abide by it.
