[Python-Dev] Re: Request for comments on final version of PEP 653 (Precise Semantics for Pattern Matching)

[Paul Moore]

On Sun, 4 Apr 2021 at 01:37, Brandt Bucher  wrote:
>
> Mark Shannon said:
> > I was relying on the "reference" implementation, which is also in the PEP.
>
> Can you please stop putting scare quotes around "reference implementation"?

Agreed - apart from the implication Brandt noted, it's also
misleading. The code is in Python 3.10, so the correct term is "the
implementation" (or if you want to be picky, "the CPython
implementation"). To me, the term "reference implementation" implies
"for reference, not yet released".

At this point, we're discussing fixes to an implemented Python 3.10
feature, not tidying up a PEP.

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3VFVVRSTVIV4A7MKW65NZRFZQ4CEKUX5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Request for comments on final version of PEP 653 (Precise Semantics for Pattern Matching)

[Antoine Pitrou]

On Sun, 04 Apr 2021 00:34:18 -
"Brandt Bucher"  wrote:
> Mark Shannon said:
> > I was relying on the "reference" implementation, which is also in the PEP.  
> 
> Can you please stop putting scare quotes around "reference implementation"? 
> You've done it twice now, and it's been a weekend-ruiner for me each time.
> 
> I've put months of work into writing and improving CPython's current pattern 
> matching implementation, mostly on nights and weekends. I don't know whether 
> it's intentional or not, but when you say things like that it instantly 
> devalues all of my hard work in front of everyone on the list.

I'm not a native English speaker, but I don't understand how putting
quotes around a reasonably polite expression devalues anyone's work.

I'm probably missing something...

Antoine.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PT5UEMBF34IJQ7IPAXKHKVZXFPZP33F4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Request for comments on final version of PEP 653 (Precise Semantics for Pattern Matching)

[Chris Angelico]

On Sun, Apr 4, 2021 at 6:20 PM Paul Moore  wrote:
>
> On Sun, 4 Apr 2021 at 01:37, Brandt Bucher  wrote:
> >
> > Mark Shannon said:
> > > I was relying on the "reference" implementation, which is also in the PEP.
> >
> > Can you please stop putting scare quotes around "reference implementation"?
>
> Agreed - apart from the implication Brandt noted, it's also
> misleading. The code is in Python 3.10, so the correct term is "the
> implementation" (or if you want to be picky, "the CPython
> implementation"). To me, the term "reference implementation" implies
> "for reference, not yet released".
>

Normally, the term "reference implementation" means "the basis
implementation that everything else is compared against". For
instance, a compression algorithm might be published as a mathematical
document, with a reference implementation in some language. It's then
possible to create a new implementation in some other language, or
more optimized, or whatever else; but to know whether it's giving the
correct results, you compare its output to the output of the reference
implementation.

CPython is the reference implementation for the Python language. It's
possible to have a discrepancy between the standard and the
implementation, but it's still the reference implementation (just
occasionally a buggy one).

In this case, I believe that the term "reference implementation" is
strictly accurate, and concur with Brandt's request to not discredit
it by implying that it's only purporting to be one.

ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/C5ZWYROKGA4FJWOTCSBYLFDRUWA5FAZO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Request for comments on final version of PEP 653 (Precise Semantics for Pattern Matching)

[Mark Shannon]

Hi Brandt,

On 04/04/2021 1:34 am, Brandt Bucher wrote:

Mark Shannon said:

I was relying on the "reference" implementation, which is also in the PEP.


Can you please stop putting scare quotes around "reference implementation"? 
You've done it twice now, and it's been a weekend-ruiner for me each time.


I'm sorry for ruining your weekends.

My intention, and I apologize for not making this clearer, was not to 
denigrate your work, but to question the implications of the term 
"reference".


Calling something a "reference" implementation suggests that it is 
something that people can refer to, that is near perfectly correct and 
fills in the gaps in the specification.


That is a high standard, and one that is very difficult to attain.
It is why I use the term "implementation", and not "reference 
implementation" in my PEPs.




I've put months of work into writing and improving CPython's current pattern 
matching implementation, mostly on nights and weekends. I don't know whether 
it's intentional or not, but when you say things like that it instantly 
devalues all of my hard work in front of everyone on the list.

It definitely wasn't my intention.



For such a huge feature, I'm honestly quite amazed that this is the only issue 
we've found since it was merged over a month ago (and both authors have agreed 
that it needs to be fixed in the PEP, not the implementation). The PR 
introducing this behavior was reviewed by at least a half-dozen people, 
including you.


Indeed, I reviewed the implementation.
I thought it was good enough to merge.
I still think that.



The last time you said something like this, I just muted the thread. Let's 
please keep this respectful; we're all obviously committing a lot of our own 
time and energy to this, and we need to work well together for it to be 
successful in the long term.
Please don't take my criticisms of PEP 634 as criticisms of you or your 
efforts. I know it can often sound like that, but that really isn't my 
intent.


Pattern matching is a *big* new feature, and to get it right takes a lot 
of discussion.

Having your ideas continually battered is no fun, I know.
So, I'd like to apologize again for any hurt caused.

Cheers,
Mark.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TAINF7X3JMX74FBORYIZSYPT2ZOVV6C6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: NOTE: Python 3.9.3 contains an unintentional ABI incompatibility leading to crashes on 32-bit systems

[Matthias Klose]

On 4/4/21 4:44 AM, Terry Reedy wrote:
> On 4/3/2021 7:15 PM, Miro Hrončok wrote:
>> On 03. 04. 21 21:44, Łukasz Langa wrote:
>>> The memory layout of PyThreadState was unintentionally changed in the recent
>>> 3.9.3 bugfix release. This leads to crashes on 32-bit systems when importing
>>> binary extensions compiled for Python 3.9.0 - 3.9.2. This is a regression.
>>>
>>> We will be releasing a hotfix 3.9.4 around 24 hours from now to address this
>>> issue and restore ABI compatibility with C extensions built for Python 3.9.0
>>> - 3.9.2.
>>
>> Thanks for the hotifx.
>>
>> However, I need to ask: Would this also happen if there was a rc version of
>> 3.9.3?
> 
> Unless the mistake was just introduced, the mistake would have happened.  One
> this severe would likely have been caught within the week or two before a
> final.  But as Łukasz noted when announcing the change, .rcs are generally
> ignored.

Well, that's not true.  I think for at least the past 3.8 and current 3.9 cycle
I always tested the release candidates, and built them for various Linux
architectures.  And I'm filing issues marked as 'released-blocker' when I see
regressions introduced, it's up to the release managers to determine if such
changes are intended or not.

Looking at the failing CI tests triggered by these builds, yes I see that 32bit
archs have the ABI change.

So maybe it's worth to re-introduce these RC builds, or at least provide source
RC tarballs which can be tested.  If it's a matter of resources, maybe fall back
to quarterly subminor releases again (at least for the most current X.Y 
releases).

Matthias
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4TSX2OCZPOSDBB2VG26HJMUZPYZ56IL3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Request for comments on final version of PEP 653 (Precise Semantics for Pattern Matching)

[Stephen J. Turnbull]

Antoine Pitrou writes:

 > On Sun, 04 Apr 2021 00:34:18 -
 > "Brandt Bucher"  wrote:

 > > Can you please stop putting scare quotes

 > I'm not a native English speaker, but I don't understand how
 > putting quotes around a reasonably polite expression devalues
 > anyone's work.

"Scare quotes" refers to an idiom English writers use to deprecate
something.

In what I wrote just above, the quotation marks indicate a focus on
the *string* "scare quotes".  In this case, the fact that they are the
exact words of Brandt, and also that I'm defining those words, not
using their meaning.

In Mark's phrase '"reference" implementation', neither of those usages
apply.  It's possible that they are the deprecated "random quote
emphasis" usage.  Random quote emphasis is implausible here, however.
I can see no reason why Mark would emphasize the modifier "reference"
in this context.

One of the most important remaining usages, and one that I find
plausible in context, is scare quotes.  These are quotation marks used
to focus on the phrase in quotes, and indicate that it is somehow
suspicious: inaccurate, imprecise, false, even the opposite of its
dictionary meaning.

In other words, if you don't have a reason to emphasize focus on the
words themselves rather than their meaning, by adding (scare) quotes
most likely you are turning a "reasonably polite expression" into an
insult.

 > I'm probably missing something...

Probably so did a lot of native speakers; there are English dialects
where scare quotes are rare and random quote emphasis is common.

However, I assure you, many native speakers (along with a fair number
of non-natives) did not.

I neither know nor care what Mark's *intent* is.  I'm explaining what
(some) idiomatic speakers of English will read into what he writes,
because it is a *common* idiom (common enough to have a name, and be
mentioned in standard manuals of English style).

Regards,
Steve


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KZ3U2JXRHYRXRUDAGNRKPBI6UHUKAJ7X/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Request for comments on final version of PEP 653 (Precise Semantics for Pattern Matching)

[Stephen J. Turnbull]

Mark Shannon writes:

 > Calling something a "reference" implementation suggests that it is 
 > something that people can refer to, that is near perfectly correct and 
 > fills in the gaps in the specification.

Shoe fits, doesn't it?  Both Guido and Brandt to my recall have
specifically considered the possibility that the implementation is the
better design, and therefore that the PEP should be changed.

 > That is a high standard, and one that is very difficult to attain.

That depends on context, doesn't it?  In the case of a public release,
yes, it's a very high standard.  In the context of a feature in
development, it *cannot* be that high, because even when the spec and
the implementation are in *perfect* agreement, both may be changed in
the light of experience or a change in requirements.

Furthermore, in this instance, the implementation achieves *your*
standard (Brandt, again):

 > > both authors have agreed that it needs to be fixed in the PEP,
 > > not the implementation

You added:

 > It is why I use the term "implementation", and not "reference 
 > implementation" in my PEPs.

A reasonable usage.

I think my more flexible, context-dependent definition is more useful.
Unmodified, the word "implementation" covers everything from
unrunnable pseudo-code to the high standard of a public release that
is officially denoted "reference implementation".

On the other hand, when Brandt says that a merge request is a
"reference implementation", I interpret that to be a claim that, to
his knowledge the MR is a perfect implementation of the specification,
and an invitation to criticize the specification by referring to that
implementation.  That's a strong claim, even in my interpretation.
However, I think that if the developer dares to make it, it's very
useful to reviewers.

As it was in this case.

Final note: once this is merged and publicly released, it will lose
its status as reference implementation in the above, strong sense.
Any deviations from documented spec (the Language Reference) will be
presumed to have to be fixed in the implementation (with due
consideration for backward compatibility).  "Although practicality
beats purity," of course, but treating the Language Reference as
authoritative is strongly preferred to keeping the implementation and
modifying the Reference (at least as I understand it).

Regards,
Steve

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AOM3ZPO4U263T2ZYGDOLFWIDOO2QSKZB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: NOTE: Python 3.9.3 contains an unintentional ABI incompatibility leading to crashes on 32-bit systems

[Stephen J. Turnbull]

Matthias Klose writes:

 > Looking at the failing CI tests triggered by these builds, yes I
 > see that 32bit archs have the ABI change.

I'm not sure precisely what you mean by that, but if you mean that CI
has caught the bug, then

 > So maybe it's worth to re-introduce these RC builds,

seems to be just makework.  It makes more sense to delay the schedule
somewhat, but only so that the release engineer, or one among the many
eyes, catch the CI's warning.

Steve


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/A642IMRBVHBNMWMTE4VOZLSOCQGAZIBC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Request for comments on final version of PEP 653 (Precise Semantics for Pattern Matching)

[Paul Moore]

On Sun, 4 Apr 2021 at 13:49, Stephen J. Turnbull
 wrote:

> Final note: once this is merged and publicly released, it will lose
> its status as reference implementation in the above, strong sense.

It *is* merged and publicly released - it's in the latest 3.10 alpha.
That's really the point I was trying to make with my comment (I'm
steering clear of the "scare quotes" discussion). The fact that the
implementation kept getting referred to as the "reference
implementation" confused me into thinking it hadn't been released yet,
and that simply isn't true. Calling it "the implementation" avoids
that confusion, IMO.

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JOFZDHPW5NLI4GGBNBEW5FL36OIAIK3Q/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: NOTE: Python 3.9.3 contains an unintentional ABI incompatibility leading to crashes on 32-bit systems

[Matthias Klose]

On 4/4/21 2:49 PM, Stephen J. Turnbull wrote:
> Matthias Klose writes:
> 
>  > Looking at the failing CI tests triggered by these builds, yes I
>  > see that 32bit archs have the ABI change.
> 
> I'm not sure precisely what you mean by that, but if you mean that CI
> has caught the bug, then
> 
>  > So maybe it's worth to re-introduce these RC builds,
> 
> seems to be just makework.  It makes more sense to delay the schedule
> somewhat, but only so that the release engineer, or one among the many
> eyes, catch the CI's warning.

No, you can't see that with CPython's CI alone.  The Debian and Ubuntu build
machines trigger CI tests per architecture for around 3000 packages depending on
python3.9, using the just built python3.9, and without rebuilding these
packages.  That's where I see the 32bit failures.

How would delaying the release schedule have helped with the issue that we just 
saw?

Matthias
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/U6C6LLSQS26RC2XC2B5GWNSZQJ5PPYMF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: NOTE: Python 3.9.3 contains an unintentional ABI incompatibility leading to crashes on 32-bit systems

[Łukasz Langa]

> On 4 Apr 2021, at 11:34, Matthias Klose  wrote:
> 
> On 4/4/21 4:44 AM, Terry Reedy wrote:
>> 
>> Unless the mistake was just introduced, the mistake would have happened.  One
>> this severe would likely have been caught within the week or two before a
>> final.  But as Łukasz noted when announcing the change, .rcs are generally
>> ignored.
> 
> Well, that's not true.  I think for at least the past 3.8 and current 3.9 
> cycle
> I always tested the release candidates, and built them for various Linux
> architectures.  And I'm filing issues marked as 'released-blocker' when I see
> regressions introduced, it's up to the release managers to determine if such
> changes are intended or not.

Can you show us an example of a release-blocker issue you reported?

- Ł___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UOSUV632CJXZE3GCUPC6CWVBGS4KNEZY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: NOTE: Python 3.9.3 contains an unintentional ABI incompatibility leading to crashes on 32-bit systems

[Michał Górny]

On Sat, 2021-04-03 at 21:44 +0200, Łukasz Langa wrote:
> The memory layout of PyThreadState was unintentionally changed in the recent 
> 3.9.3 bugfix release. This leads to crashes on 32-bit systems when importing 
> binary extensions compiled for Python 3.9.0 - 3.9.2. This is a regression.
> 
> We will be releasing a hotfix 3.9.4 around 24 hours from now to address this 
> issue and restore ABI compatibility with C extensions built for Python 3.9.0 
> - 3.9.2.
> 
> Details:
> https://bugs.python.org/issue43710

This is precisely what I meant when I said I don't like the idea of
combining security fixes with irrelevant changes.  Good that I've chosen
to backport the secfixes instead of pushing the new version to Gentoo
stable.

-- 
Best regards,
Michał Górny


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5LSUYFLGTMCDEWRGJIDTOPPUAT4ZH7JE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: NOTE: Python 3.9.3 contains an unintentional ABI incompatibility leading to crashes on 32-bit systems

[Łukasz Langa]

> On 4 Apr 2021, at 01:15, Miro Hrončok  wrote:
> 
> However, I need to ask: Would this also happen if there was a rc version of 
> 3.9.3?

Good question. The RC would not help. Most importantly, 3.9.3 was itself an 
expedited release due to its security content. When I did use an RC phase for 
3.9.2, which also contained security fixes, it met with considerable backlash 
and urges to release the update faster. And I ultimately did, two days after 
the RC was out. Informed by this experience, I would have likely skipped the RC 
for 3.9.3 anyway.

More generally, RCs historically provided little value. Since Python 3.4 we've 
provided 55 bugfix releases. Five of those included an RC2, suggesting testing 
caught a regression. Let's look closer:
- none of those happened for 3.8 and 3.9 releases;
- two of those are a single issue in 3.7.1rc1 and 3.6.7rc1: 
https://bugs.python.org/issue34927 , indeed 
caught by a user downloading an rc1 installer from python.org 
;
- one was found by a third-party during "preparation for Python 3.8" and it 
just happened to be a regression also present in 3.7.4rc1 
(https://bugs.python.org/issue24214 );
- one was found by a third-party using nightly Python builds in CI 
(https://bugs.python.org/issue38216 ) and 
it just happened to be a regression also present in 3.5.8rc1;
- one was found by a core developer running regression tests on what 
coincidentally happened to be 3.6.2rc1 on Windows 
(https://bugs.python.org/issue30716 ). The 
bug was in the tests themselves.

So, we're looking at a single instance of a bug found an RC1 installer being 
out there. Python 3.0 through 3.3 had limited user penetration so looking at 
those isn't informative. But we can look at Python 2.7, and that one had a 
single rc2 in its 10 years of bugfix releases. That was 2.7.3rc2, in 2012. It 
was in the Windows help file, discovered by a core developer looking through it.

In the time of 3.8 and 3.9 so far, there was a single hotfix release which was 
due to a regression not caught by a published release candidate 
(https://bugs.python.org/issue41304 ).

Given the information above, I stand by my decision (confirmed with other 
release managers) to skip RCs for bugfix releases.

- Ł___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ON57KRYDKN57ZUPYW3Y7VRMKDSDIUJ7W/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: NOTE: Python 3.9.3 contains an unintentional ABI incompatibility leading to crashes on 32-bit systems

[Miro Hrončok]

On 04. 04. 21 16:34, Łukasz Langa wrote:


On 4 Apr 2021, at 01:15, Miro Hrončok > wrote:


However, I need to ask: Would this also happen if there was a rc version of 
3.9.3?


Good question. The RC would not help. Most importantly, 3.9.3 was itself an 
expedited release due to its security content. When I *did* use an RC phase for 
3.9.2, which also contained security fixes, it met with considerable backlash 
and urges to release the update faster. And I ultimately did, two days after the 
RC was out. Informed by this experience, I would have likely skipped the RC for 
3.9.3 anyway.


More generally, RCs historically provided little value. Since Python 3.4 we've 
provided 55 bugfix releases. Five of those included an RC2, suggesting testing 
caught a regression. Let's look closer:

- *none* of those happened for 3.8 and 3.9 releases;
- two of those are a single issue in 3.7.1rc1 and 3.6.7rc1: 
https://bugs.python.org/issue34927 , indeed 
caught by a user downloading an rc1 installer from python.org ;
- one was found by a third-party during "*preparation for Python 3.8*" and it 
just happened to be a regression also present in 3.7.4rc1 
(https://bugs.python.org/issue24214 );
- one was found by a third-party using *nightly* Python builds in CI 
(https://bugs.python.org/issue38216 ) and it 
just happened to be a regression also present in 3.5.8rc1;
- one was found by a core developer *running regression tests* on what 
coincidentally happened to be 3.6.2rc1 on Windows 
(https://bugs.python.org/issue30716 ). The 
bug was in the tests themselves.


So, we're looking at a single instance of a bug found an RC1 installer being out 
there. Python 3.0 through 3.3 had limited user penetration so looking at those 
isn't informative. But we can look at Python 2.7, and that one had a *single* 
rc2 in its 10 years of bugfix releases. That was 2.7.3rc2, in *2012*. It was in 
the Windows help file, discovered by a core developer looking through it.


In the time of 3.8 and 3.9 so far, there was a single hotfix release which was 
due to a regression not caught by a published release candidate 
(https://bugs.python.org/issue41304 ).


Given the information above, I stand by my decision (confirmed with other 
release managers) to skip RCs for bugfix releases.


Thanks for the thoughtful analysis, Łukasz!

--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZGD4CWEDYANSYQG6QONF7KKASO4VN3BP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] [RELEASE] Python 3.9.4 hotfix is now available

[Łukasz Langa]

Python 3.9.3 was released two days ago on Friday, April 2nd. It contains 
important security content listed below for reference. Unfortunately, it also 
introduced an unintentional ABI incompatibility, making some C extensions built 
with Python 3.9.0 - 3.9.2 crash with  Python 3.9.3 on 32-bit systems. To 
minimize disruption, I decided to recall 3.9.3 and introduce this hotfix 
release: 3.9.4.

We highly recommend upgrading your Python 3.9 installations to 3.9.4 at your 
earliest convenience.

Get it here:

https://www.python.org/downloads/release/python-394/ 

What is “ABI compatibility”?

Python guarantees that within a given language series (like the current 3.9) 
binary extensions written in C or C++ and compiled against headers of one 
release (like 3.9.0) will be importable from other versions in the same series 
(like 3.9.3). If this weren’t the case, library authors would have to ship 
separate binary wheels on PyPI for every single bugfix release of Python. That 
would be very inconvenient.

 
What
 broke in Python 3.9.3?

In a fix for a corner-case crash around recursion limits and exceptions, the 
PyThreadState struct needed to change. While PyThreadState’s only documented 
public member is the *interp field 
, it’s not 
uncommon for C extensions to access other fields in this struct as well.

When I approved the backport of this fix, I missed the fact that the variable 
size change would change the memory layout of said struct on 32-bit systems (on 
64-bit systems alignment rules made the size change backwards compatible). 
Merging the backport was a mistake, and so 3.9.4 reverts it to restore 
compatibility with binary extensions built against Python 3.9.0 - 3.9.2. 
Details in bpo-43710 .

 
Security
 Content in Python 3.9.3

bpo-43631 : high-severity CVE-2021-3449 and 
CVE-2021-3450 were published for OpenSSL, it’s been upgraded to 1.1.1k in CI, 
and macOS and Windows installers.
bpo-42988 : CVE-2021-3426: Remove the 
getfile feature of the pydoc module which could be abused to read arbitrary 
files on the disk (directory traversal vulnerability). Moreover, even source 
code of Python modules can contain sensitive data like passwords. Vulnerability 
reported by David Schwörer.
bpo-43285 : ftplib no longer trusts the IP 
address value returned from the server in response to the PASV command by 
default. This prevents a malicious FTP server from using the response to probe 
IPv4 address and port combinations on the client network. Code that requires 
the former vulnerable behavior may set a trust_server_pasv_ipv4_address 
attribute on their ftplib.FTP instances to True to re-enable it.
bpo-43439 : Add audit hooks for 
gc.get_objects(), gc.get_referrers() and gc.get_referents(). Patch by Pablo 
Galindo.
 
Release
 Calendar

Maintenance releases for the 3.9 series will continue at regular bi-monthly 
intervals, with 3.9.5 planned for May 3rd 2021 as well.

 
What’s
 new?

The Python 3.9 series contains many new features and optimizations over 3.8. 
See the “What’s New in Python 3.9  
” document for more information 
about features included in the 3.9 series. We also have a detailed change log 
for 3.9.3  
specifically.

Detailed information about all changes made in version 3.8.9 can be found in 
its respective changelog 
.

 
We
 hope you enjoy those new releases!

Thanks to all of the many volunteers who help make Python Development and these 
releases possible! Please consider supporting our efforts by volunteering 
yourself or through organization contributions to the Python Software 
Foundation.

Your friendly release team,
Łukasz Langa @ambv 
Ned Deily @nad 
Steve Dower @steve.dower ___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/

[Python-Dev] Re: NOTE: Python 3.9.3 contains an unintentional ABI incompatibility leading to crashes on 32-bit systems

[Terry Reedy]

On 4/4/2021 9:57 AM, Łukasz Langa wrote:


On 4 Apr 2021, at 11:34, Matthias Klose > wrote:



I always tested the release candidates, and built them for various Linux
architectures.  And I'm filing issues marked as 'released-blocker' 
when I see regressions introduced, it's up to the release manager >> to determine  if such changes are intended or not.



Can you show us an example of a release-blocker issue you reported?


Searching creator: doko; priority: release blocker returns 19 issues, 
from 3 to 20 years ago.  That about 1/years but none for 3.8 or 3.9, for 
which Łukasz is RM.


32232 	36 months ago 	building extensions as builtins is broken in 3.7 
has patch 	has PR 	closed
32233 	40 months ago 	[3.7 Regression] build with --with-system-libmpdec 
is broken 	has patch 	has PR 	closed
31016 	43 months ago 	[Regression] sphinx shows an EOF error when using 
python2.7 from the trunk 			closed
23968 	55 months ago 	rename the platform directory from plat-$(MACHDEP) 
to plat-$(PLATFORM_TRIPLET) 	has patch 		closed
26839 	58 months ago 	Python 3.5 running on Linux kernel 3.17+ can block 
at startup or on importing the random module on getrandom() 	has 
patch 		closed
24226 	71 months ago 	[3.5 Regression] unable to byte-compile the 
attached IN.py 			closed
24162 	71 months ago 	[2.7 regression] test_asynchat test failure on 
i586-linux-gnu 			closed
23842 	72 months ago 	SystemError: ../Objects/longobject.c:998: bad 
argument to internal function 	has patch 		closed
22523 	79 months ago 	[regression] Lib/ssl.py still references 
_ssl.sslwrap 	has patch 		closed

17192   96 months ago   libffi-3.0.13 importhas patch   closed
17579 	97 months ago 	socket module in 2.7.4 raises error instead of 
gaierror in 2.7.3 			closed
14330 	105 months ago 	don't use host python, use host search paths for 
host compiler 	has patch 		closed
4303 	149 months ago 	[2.5 regression] ctypes fails to build on 
arm-linux-gnu 			closed

4469149 months ago  CVE-2008-5031 multiple integer overflows
closed
4552 	150 months ago 	Doc/tools/sphinxext not included in the 2.6.1 
tarball 			closed
4519 	150 months ago 	.pyc files included in 2.6 and 3.0 release 
tarballs 			closed
2601 	157 months ago 	[regression] reading from a urllib2 file 
descriptor happens byte-at-a-time 			closed
984880 	203 months ago 	fix regression on 2.3 branch: Lib/threading 	has 
patch 		closed
598996 	226 months ago 	Failure building the documentation 	has patch 	 
closed


The tradeoffs are the cost of bugfix release candidates (paid most by a 
few people) versus the cost of the relative rare hotfix releases versus 
the frequency of bugfix releases.  I personally prefer a bugfix every 2 
months without candidate to a bugfix every 4 months with one (these 
being alternatives with the same work by the release crew).


A possible compromise would be to cut the release a couple of days ahead 
and then pause while Matthias or anyone else runs tests.  But I would 
understand if Lukasz considered pausing to be worse than finishing the 
release then and there.  Or announce the upcoming release a week ahead 
and ask committers to only merge PRs that could possibly break anything 
to only do so with fresh CI tests.  (Stale tests were one recent reason 
a test-breaking PR got merged.)



--
Terry Jan Reedy


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GEXTJNKD7IZL3J4M2ODESLJLLVAEQVKW/
Code of Conduct: http://python.org/psf/codeofconduct/