Re: [Python-Dev] release cadence (was: Request for CPython 3.5.3 release)
As an observer and user— It may be worth asking the Rust team what the main pain points are in coordinating and managing their releases. Some context for those unfamiliar: Rust uses a Chrome- or Firefox-like release train approach, with stable and beta releases every six weeks. Each release cycle includes both the compiler and the standard library. They use feature flags on "nightly" (the master branch) and cut release branches for actually gets shipped in each release. This has the advantage of letting new features and functionality ship whenever they're ready, rather than waiting for Big Bang releases. Because of strong commitments to stability and backwards compatibility as part of that, it hasn't led to any substantial breakage along the way, either. There is also some early discussion of how they might add LTS releases into that mix. The Rust standard library is currently bundled into the same repository as the compiler. Although the stdlib is currently being modularized and somewhat decoupled from the compiler, I don't believe they intend to separate it from the compiler repository or release in that process (not least because there's no need to further speed up their release cadence!). None of that is meant to suggest Python adopt that specific cadence (though I have found it quite nice), but simply to observe that the Rust team might have useful info on upsides, downsides, and particular gotchas as Python considers changing its own release process. Regards, Chris Krycho > On Jul 3, 2016, at 16:22, Brett Cannon wrote: > > [forking the conversation since the subject has shifted] > >> On Sun, 3 Jul 2016 at 09:50 Steve Dower wrote: >> Many of our users prefer stability (the sort who plan operating system >> updates years in advance), but generally I'm in favour of more frequent >> releases. > > So there's our 18 month cadence for feature/minor releases, and then there's > the 6 month cadence for bug-fix/micro releases. At the language summit there > was the discussion kicked off by Ned about our release schedule and a group > of us had a discussion afterward where a more strict release cadence of 12 > months with the release date tied to a consistent month -- e.g. September of > every year -- instead of our hand-wavy "about 18 months after the last > feature release"; people in the discussion seemed to like the 12 months > consistency idea. I think making releases on a regular, annual schedule > requires simply a decision by us to do it since the time scale we are talking > about is still so large it shouldn't impact the workload of RMs & friends > that much (I think). > > As for upping the bug-fix release cadence, if we can automate that then > perhaps we can up the frequency (maybe once every quarter), but I'm not sure > what kind of overhead that would add and thus how much would need to be > automated to make that release cadence work. Doing this kind of shrunken > cadence for bug-fix releases would require the RM & friends to decide what > would need to be automated to shrink the release schedule to make it viable > (e.g. "if we automated steps N & M of the release process then I would be > okay releasing every 3 months instead of 6"). > > For me, I say we shift to an annual feature release in a specific month every > year, and switch to a quarterly bug-fix releases only if we can add zero > extra work to RMs & friends. > >> It will likely require more complex branching though, presumably based on >> the LTS model everyone else uses. > > Why is that? You can almost view our feature releases as LTS releases, at > which point our current branching structure is no different. > >> >> One thing we've discussed before is separating core and stdlib releases. I'd >> be really interested to see a release where most of the stdlib is just >> preinstalled (and upgradeable) PyPI packages. We can pin versions/bundle >> wheels for stable releases and provide a fast track via pip to update >> individual packages. >> >> Probably no better opportunity to make such a fundamental change as we move >> to a new VCS... > > > > Topic 1 > === > If we separate out the stdlib, we first need to answer why we are doing this? > The arguments supporting this idea is (1) it might simplify more frequent > releases of Python (but that's a guess), (2) it would make the stdlib less > CPython-dependent (if purely by the fact of perception and ease of testing > using CI against other interpreters when they have matching version support), > and (3) it might make it easier for us to get more contributors who are > comfortable he
Re: [Python-Dev] Automating the maintenance release pipeline (was Re: Request for CPython 3.5.3 release)
The bots Mozilla runs around both Rust and Servo should make a lot of this much lower overhead if they can be repurposed (as I believe several other communities have already done). Homu, the build manager tool, runs CI (including buildbots, Travis, etc.), is integrated with GitHub PRs so maintainers can trigger it with a comment there, and can also roll up a bunch of changes into one (handy to pull together e.g. a bunch of small documentation changes like typo fixes): https://github.com/barosl/homu That seems to keep the pain level of having an always-building-and-passing-tests nightly version much lower. Aside: I don't want to flood these discussions with "yay Rust!" stuff, so this will probably be my last such response unless something else really jumps out. ;-) Thanks for the work you're all doing here. Regards, Chris Krycho > On Jul 3, 2016, at 7:34 PM, Nick Coghlan wrote: > >> On 4 July 2016 at 00:39, Guido van Rossum wrote: >> Another thought recently occurred to me. Do releases really have to be >> such big productions? A recent ACM article by Tom Limoncelli[1] >> reminded me that we're doing releases the old-fashioned way -- >> infrequently, and with lots of manual labor. Maybe we could >> (eventually) try to strive for a lighter-weight, more automated >> release process? It would be less work, and it would reduce stress for >> authors of stdlib modules and packages -- there's always the next >> release. I would think this wouldn't obviate the need for carefully >> planned and timed "big deal" feature releases, but it could make the >> bug fix releases *less* of a deal, for everyone. > > Yes, getting the maintenance releases to the point of being largely > automated would be beneficial. However, I don't think the problem is > lack of desire for that outcome, it's that maintaining the release > toolchain pretty much becomes a job at that point, as you really want > to be producing nightly builds (since the creation of those nightlies > in effect becomes the regression test suite for the release > toolchain), and you also need to more strictly guard against even > temporary regressions in the maintenance branches. > > There are some variants we could pursue around that model (e.g. > automating Python-only updates without automating updates that require > rebuilding the core interpreter binaries for Windows and Mac OS X), > but none of it is the kind of thing likely to make anyone say "I want > to work on improving this in my free time". Even for commercial > redistributors, it isn't easy for us to make the business case for > assigning someone to work on it, since we're generally working from > the source trees rather than the upstream binary releases. > > I do think it's worth putting this into our bucket of "ongoing > activities we could potentially propose to the PSF for funding", > though. I know Ewa (Jodlowska, the PSF's Director of Operations) is > interested in better supporting the Python development community > directly (hence https://donate.pypi.io/ ) in addition to the more > indirect community building efforts like PyCon US and the grants > program, so I've been trying to build up a mental list of CPython > development pain points where funded activities could potentially > improve the contributor experience for volunteers. So far I have: > > - issue triage (including better acknowledging folks that help out > with triage efforts) > - patch review (currently "wait and see" pending the impact of the > GitHub migration) > - nightly pre-release builds (for ease of contribution without first > becoming a de facto C developer and to help make life easier for > release managers) > > That last one is a new addition to my list based on this thread, and I > think it's particularly interesting in that it would involve a much > smaller set of target users than the first two (with the primary > stakeholders being the release managers and the folks preparing the > binary installers), but also a far more concrete set of deliverables > (i.e. nightly binary builds being available for active development and > maintenance branches for at least Windows and Mac OS X, and > potentially for the manylinux1 baseline API defined in PEP 513) > > Cheers, > Nick. > > -- > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/chris%40chriskrycho.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
