Re: [gentoo-dev] [RFC] Splitting dev-lang/python into per-slot packages, starting with 3.14
On Mon, 2024-10-14 at 01:43 +0100, Sam James wrote: > > > However, I think the cleanest way forward would be to stop slotting > > CPython like this, and instead have a separate package for each version, > > just like the vast majority of distributions do, i.e.: > > > > dev-lang/python3_N > > > > As others have noted, such a proposal needs specific arguments as to why > SLOTs aren't a good fit. I agree with you that they're not always a good > fit -- SQLite and libxml2 are good examples you gave downthread, but > the onus is on the one making the proposal. > > Now, for Python, there's a few disadvantages: > * losing the ordering on PV for e.g. has_version (we could add a helper > in python-utils-r1 for this); I don't get this. You can't use has_version directly without specifying the slot, because it's going to match different versions. And there's no real difference between specifying a slot and a different package name. Well, unless you mean doing a meaningless has_version match for the sake of it. Then, yes, unslotting fixes that -- i.e. removes that ability. > * losing the ability to consistently set package.use/package.env for all > Pythons, like always enabling PGO or tests; We aren't losing it, you just need to repeat it. Just like right now you can apply these per-slot or restrict version ranges, so there's no guarantee of consistency either. > * disruption to scripts which have reasonably assumed we'd always have a > dev-lang/python (we'd need to do something like we have planned for > pkgmoves, I think -- make Portage know about it and suggest alternatives > intelligently/rewrite it transparently when given as an argument). Yes, this is a fair point, and the logic in pkgcheck is pretty horibble, so I guess going for slotting just to avoid having to fix that and deploy the fix makes sense. > > This naturally means that only the specific version requested (e.g. via > > targets) would be installed, and no cross-slot autoupgrades would > > happen. Ideally, I'd like to start doing that with Python 3.14 whose > > first alpha is expected next week. Depending on how they handle > > freethreading, we'd end up having the first or both of: > > > > dev-lang/python3_14 > > dev-lang/python3_14t > > > > It's worth noting that we *do* this for pypy, but we retain > dev-python/pypy3. I'm not a huge fan of it there but I know why we have > it -- so that one can test new versions of pypy in parallel even when > they supply the same implementation/version of the Python language. Technically, we could merge PyPy into a single package, as long as we use verisons such as 2.7.7.3.17:2.7, 3.10.7.3.17:3.10, etc. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] [RFC] Splitting dev-lang/python into per-slot packages, starting with 3.14
Michał Górny writes: > On Mon, 2024-10-14 at 01:43 +0100, Sam James wrote: >> >> > However, I think the cleanest way forward would be to stop slotting >> > CPython like this, and instead have a separate package for each version, >> > just like the vast majority of distributions do, i.e.: >> > >> > dev-lang/python3_N >> > >> >> As others have noted, such a proposal needs specific arguments as to why >> SLOTs aren't a good fit. I agree with you that they're not always a good >> fit -- SQLite and libxml2 are good examples you gave downthread, but >> the onus is on the one making the proposal. >> >> Now, for Python, there's a few disadvantages: >> * losing the ordering on PV for e.g. has_version (we could add a helper >> in python-utils-r1 for this); > > I don't get this. You can't use has_version directly without specifying > the slot, because it's going to match different versions. And there's > no real difference between specifying a slot and a different package > name. > > Well, unless you mean doing a meaningless has_version match for the sake > of it. Then, yes, unslotting fixes that -- i.e. removes that ability. Not for the sake of it, I was thinking of: if has_version >=dev-lang/python-3.10 ; then # Skip bogus tests relying on legacy behaviour fi but of course, that doesn't work for pypy, which I forgot about ;) > >> * losing the ability to consistently set package.use/package.env for all >> Pythons, like always enabling PGO or tests; > > We aren't losing it, you just need to repeat it. Just like right now > you can apply these per-slot or restrict version ranges, so there's no > guarantee of consistency either. You can't wildcard on it, though, so you have to explicitly list it for all Pythons. I'm not sure what you mean by the consistency point. You can restrict it right now if you want to via slot or version ranges, but we have no way of doing a wildcard on package name? (I actually think we could do with wildcard matching on version at least in /etc/portage but finding some syntax which is free for it isn't easy. It would be useful for e.g. masking .0 kernels.) > >> * disruption to scripts which have reasonably assumed we'd always have a >> dev-lang/python (we'd need to do something like we have planned for >> pkgmoves, I think -- make Portage know about it and suggest alternatives >> intelligently/rewrite it transparently when given as an argument). > > Yes, this is a fair point, and the logic in pkgcheck is pretty horibble, > so I guess going for slotting just to avoid having to fix that > and deploy the fix makes sense. > >> > This naturally means that only the specific version requested (e.g. via >> > targets) would be installed, and no cross-slot autoupgrades would >> > happen. Ideally, I'd like to start doing that with Python 3.14 whose >> > first alpha is expected next week. Depending on how they handle >> > freethreading, we'd end up having the first or both of: >> > >> > dev-lang/python3_14 >> > dev-lang/python3_14t >> > >> >> It's worth noting that we *do* this for pypy, but we retain >> dev-python/pypy3. I'm not a huge fan of it there but I know why we have >> it -- so that one can test new versions of pypy in parallel even when >> they supply the same implementation/version of the Python language. > > Technically, we could merge PyPy into a single package, as long as we > use verisons such as 2.7.7.3.17:2.7, 3.10.7.3.17:3.10, etc. Ah, good point.
Re: [gentoo-dev] [RFC] Splitting dev-lang/python into per-slot packages, starting with 3.14
Michał Górny writes: > Hello, > > Historically, all versions of CPython were slotted in a single package, > i.e.: > > dev-lang/python:3.N > I feel like this whole thing happened so fast I didn't have a chance to comment properly. I understand you've retracted it but I'd like to add some context and background and so on for future reference anyway. > This approach has been causing a major annoyance for users -- due to > Portage "greedy" upgrade behavior, any time a new Python version was > keyworded, Portage insisted on installing it, even though user's > selected targets did not request the specific version. The potentially > worst consequence of that would be random user scripts stopping to work, > as they suddenly start using new Python, while all their dependencies > are still installed per PYTHON_TARGETS. > This is bug #702806 which happens with python-any-r1 which has an any-of dependency on dev-lang/python. That's why we don't see it with e.g. Qt. It's a bit annoying but not terrible. > Upstream has recently added freethreading support to CPython. Since > this support is not ABI compatible with the regular build, we need to > introduce a separate target for it, and to package it separately. > In the planned patchset, I've already put it as a separate package (dev- > lang/python-freethreading), because otherwise Portage would insist > on upgrading to it! > It wouldn't! See above. It would, however, if we made it eligible for python-any-r1, but to be honest, I think we should exclude the freethreaded build from that. It's all risk (and/or downsides) with no real gain, as I don't expect a whole-freethreaded system is going to be possible any time soon anyway. > However, I think the cleanest way forward would be to stop slotting > CPython like this, and instead have a separate package for each version, > just like the vast majority of distributions do, i.e.: > > dev-lang/python3_N > As others have noted, such a proposal needs specific arguments as to why SLOTs aren't a good fit. I agree with you that they're not always a good fit -- SQLite and libxml2 are good examples you gave downthread, but the onus is on the one making the proposal. Now, for Python, there's a few disadvantages: * losing the ordering on PV for e.g. has_version (we could add a helper in python-utils-r1 for this); * losing the ability to consistently set package.use/package.env for all Pythons, like always enabling PGO or tests; * disruption to scripts which have reasonably assumed we'd always have a dev-lang/python (we'd need to do something like we have planned for pkgmoves, I think -- make Portage know about it and suggest alternatives intelligently/rewrite it transparently when given as an argument). > This naturally means that only the specific version requested (e.g. via > targets) would be installed, and no cross-slot autoupgrades would > happen. Ideally, I'd like to start doing that with Python 3.14 whose > first alpha is expected next week. Depending on how they handle > freethreading, we'd end up having the first or both of: > > dev-lang/python3_14 > dev-lang/python3_14t > It's worth noting that we *do* this for pypy, but we retain dev-python/pypy3. I'm not a huge fan of it there but I know why we have it -- so that one can test new versions of pypy in parallel even when they supply the same implementation/version of the Python language. > (Alternatives: python-3_14, python-freethreading-3_14? Though I think > following PYTHON_TARGETS is cleaner here.) > > As a side notice, the existing versions would probably remain as-is > until removal, since there's really no gain in splitting them, given > we'd have to retain compatibility with existing depstrings. > > Comments? thanks, sam
