John Snow <js...@redhat.com> writes:
> On Tue, Jun 24, 2025 at 2:45 AM Markus Armbruster <arm...@redhat.com> wrote:
>
>> Paolo Bonzini <pbonz...@redhat.com> writes:
>>
>> > On 6/5/25 21:35, John Snow wrote:
>> >> However, if we take as iron-clad our commitment to the build platform
>> promise -- *and* guarantee offline/tarball builds as well -- then Debian 12
>> (as an example) only offers Sphinx 5.3.0 and not newer unless we allow
>> internet access to fetch Sphinx 6.2.1. This is not a problem for developer
>> workstations at all, but I am unclear on what problems this may cause for
>> tarball releases and downstream offline/isolated/ reproducible builds, if
>> any.
>> >> In this case, we can (probably) "fix" the issue by continuing to allow
>> older Sphinx while preferring a newer Sphinx version when it is missing,
>> but then we lose the ability to make code cleanups and drop a lot of
>> back-compat crud. If memory serves, there were other issues recently where
>> older versions of Sphinx behaved differently from newer versions, causing
>> intermittent failures that were hard to track down.
>> >
>> > The *ideal* solution would be to:
>> >
>> > - accept: 4.3.2 or newer, which is what Ubuntu 22.04 has
>> >
>> > - install: 6.2.1, which is what supports Python 3.13
>>
>> I guess this relates to pythondeps.toml line
>>
>> sphinx = { accepted = ">=3.4.3", installed = "5.3.0", canary =
>> "sphinx-build" }
>>
>> I further guess "accepted" means "reject anything older", and
>> "installed" means "preferred version".
>
> Yes. Accepted is what we'll tolerate if it's already installed, "installed"
> is what we will prefer to install.
Thanks!
>> > This lets all supported distros build documentation if they use the
>> > default Python runtime. It would still require a couple hacks in
>> > compat.py: SOURCE_LOCATION_FIX and nested_parse_with_titles().
>> >
>> > I am not sure however whether to count the latter, for two reasons.
>> > First, it has this:
>> >
>> > # necessary so that the child nodes get the right source/line set
>> > content_node.document = directive.state.document
>> >
>> > so it is not a pure compatibility hack. Second, and opposite, currently
>> > none of the uses of nested_parse_with_titles() go through compat.py's
>> > version, therefore it probably can be removed altogether.
>> >
>> > That leaves only SOURCE_LOCATION_FIX.
>> >
>> > As an aside, if the compat.py hacks survive, I would add comments to
>> > document which distros need the hacks.
>> >
>> >> What I'd like to know is: what precisely are our options in this
>> >> scenario? Do we consider it acceptable for some platforms to be unable to
>> >> build docs offline?
>> >
>> > Certainly for platforms not using the default Python runtime, which
>> > right now is only SLES. For others...
>> >
>> >> How highly do we value the ability to locally build docs for any given
>> >> release?
>>
>> Purely offline, or not?
>
>
> Purely offline is my concern as it is the requirement I least understand.
>
> For developer builds from the git tree, it doesn't matter: you have PyPI,
> and we have mkvenv.py to set up an environment we are confident will work
> for the purposes of building QEMU and running tests.
Yes, things should work regardless of what we choose for "accepted" and
"installed": if what the host got is "accepted", we use it, else we put
"installed" in the environment.
> For package builds in foreign lands, I grow less certain ...
>
> I know Fedora and RHEL et al build in a strictly isolated environment where
> we *cannot* pull PyPI packages.
This makes sense.
> For Fedora this is generally less of a
> problem as the distro repository usually has bleeding edge packages for us
> to use. mkvenv.py still runs in RPM builds, it just happens to not need
> anything it doesn't already have, so it succeeds.
>
> For RHEL it's slightly less of a concern as the version of QEMU packaged
> tends to also be old, so the dependencies needed by that version should be
> matched well by what's available in that distro repo.
>
> Where I am less certain is for various "streams" or "modules" or whatever
> for e.g. RHEL that may package newer versions of QEMU but may not have
> other bleeding edge requirements. I am also entirely unfamiliar with the
> release cadence of Debian, Ubuntu, OpenSUSE, etc. I am concerned
> (hypothetically) about cases where Python3.9 is available as an optional
> package, but newer versions of e.g. Sphinx are not.
>
> Is that a problem? Do we care?
If you want to package $software, you get to package its dependencies.
Simple as that.
Widening the version range of our dependencies can make this a bit
easier.
For a distribution's .0, current stable versions get packaged. We
better make this as easy as we can. So the version range needs to
include "current stable". But why wouldn't it include that ever?
Later on, a wider version range can enable "uneven" upgrades: upgrading
to a new QEMU without also upgrading certain dependencies.
How valuable is this for us? The answer should inform our choice of
version range. Extending the range further back can be costly, and it
better be worth it.
I'm rather skeptical here. Feels like a lot of angst about what
distributions might possibly want to do, with precious little evidence
for what they actually do. Show me distribution package upgrades
(uneven or not) we enabled by keeping our version range wide, and I'll
reconsider.
> In theory, it could affect not just documentation building but unit tests
> as well, depending on where and how we "break" our promise.
>
> I suspect I won't really be able to figure out if it's an issue or not
> until I just "yeehaw!" and do it and see who chirps, but I don't actually
> like operating in such a cavalier manner as a maintainer ...
>
>
>>
>> > ... I think I value this a bit higher than Markus, but not really
>> > because of offline builds. Rather, keeping the "accepted" key lower (i.e.
>> > supporting the packaged sphinx on a wide range of distros) makes it easier
>> > to bump the "installed" key when needed, as in this failure to run 5.3.0
>> > under Python 3.13.
>>
>> Showing my ignorance again... I don't understand how keeping "accepted"
>> lower helps.
>>
>> > This time there was a version that works on both the oldest and newest
>> > Python that we support, but there may not always be one because sphinx is
>> > all too happy at dropping support for EOL'd versions of Python.
>>
>> Pretty strong hint we shouldn't try to support EOL'd versions of Python
>> either.
>>
>
> This is the problem I keep running into, generally. Our build platform
> promise generally includes platforms that use EOL versions of Python by
> default - which is not a problem in and of itself, however when combined
> with our support for bleeding edge platforms, it becomes difficult to
> support both an EOL'd version of Python *and* the latest releases, as they
> are quite aggressive in dropping support for deprecated features and
> libraries on the newer end.
>
> Add to that the difficulty of keeping the linters and type checkers happy
> across such a wide matrix, and it becomes burdensome to keep things
> humming. In general my impression is that linters, doc systems, type
> systems etc are designed to be run under strictly pinned versions, not
> "whatever the user happens to have available", which is at stark odds with
> how we manage our build, doc and test system environment today.
For development aids like linters and type checkers, graceful
degradation can make sense. They must work for developers (who we can
expect to use a reasonably current development environment). I can't
see why they wouldn't work in a similarly current packaging environment
(i.e. offline), and that's nice. But I'd loathe to invest much in
keeping development aids work in a non-current environment, especially
not offline.
> I do not know what the "solution" to that contrast is, but I do get the "we
> are doing something wrong" feeling a lot when managing our stack. I am
> proud it works as well as it does and across such a diverse host of
> platforms, but I get the sense we're one of the few or only projects out
> there doing it like this.
Bob is driving on the highway listening to the radio. Suddenly the
music stops for an announcement: there's a wrong-way driver on the
highway! "One?", Bob exclaims, "*hundreds* of them!"
When nobody else does what we try to do, we might be pioneers, or we
might be fools.
> I would be as content as anyone else if Python didn't have such an
> aggressive deprecation window and didn't *frequently* break the packaging
> environment.
We got to play the hand we're dealt.
>> > Paolo
>> >
>> >> Before I throw my weight behind any given option, I just want to know
>> >> what we consider our non-negotiable obligations to be.
>> >>
>> >> Thanks,
>> >> --js
