[Python-Dev] Re: Dusting off PEP 533?

[Stestagg]

For what it's worth, this recent issue: https://bugs.python.org/issue42762
is somewhat related to the non-async aspect of 533. (generator func getting
closed during __del__ and causing odd side-effects at seemingly random
points in the execution flow)

The example code provided in that issue is contrived, and involves catching
unconditional exceptions and recursing, but following the code flow in that
situation feels unreasonably hard, and deciding if it's a python bug or not
is even harder.

I'm not sure how easy an implementation of 533 would be that doesn't break
a lot of existing code, but if one could be made, it might help make
generator lifecycles more predictable.




On Tue, Jan 5, 2021 at 1:08 PM Senthil Kumaran  wrote:

> Hi Paul,
>
> > Per PEP 525, I can call aclose coroutine method to cleanup the
> generator, but
> > it requires the code iterating to be aware that that closing the
> generator is
> > necessary.
>
> How about treating this as a bug for the specific use case that you
> mentioned,
> rather than a complete addition of " PEP 533 -- Deterministic cleanup for
> iterators". I am not certain why PEP 533 was deffered.
>
> I'd also suggest a documentation update if the details of aclose
> requirement is
> mentioned only in a PEP and not in the documentation.
>
> Thank you,
> Senthil
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/EPKMYH7DH5G6YR3AQHZTQKHBQF46YXLC/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VB64QJAYU2CCLOHRCBUIQOHMS2DIQQCC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: 3.10 change (?) for __bool__

[Stestagg]

I've raised: https://bugs.python.org/issue42899.

The triggering commit was:  c71581c7a4192e6ba9a79eccc583aaadab300efa
bpo-42615: Delete redundant jump instructions that only bypass empty blocks
(GH-23733)




On Mon, Jan 11, 2021 at 8:04 PM Guido van Rossum  wrote:

> All that said (I agree it's surprising that 3.10 seems backwards
> incompatible here) I would personally not raise AttributeError but
> TypeError in the `__bool__()` method.
>
> On Mon, Jan 11, 2021 at 11:51 AM Pablo Galindo Salgado <
> pablog...@gmail.com> wrote:
>
>> This may be related to the changes in https://bugs.python.org/issue42246.
>> Could you open a new issue and add Mark Shannon to it if that turns to be
>> the case?
>>
>> Pablo
>>
>> On Mon, 11 Jan 2021 at 19:36, Chris Angelico  wrote:
>>
>>> On Tue, Jan 12, 2021 at 6:00 AM Mats Wichmann  wrote:
>>> >
>>> >
>>> > On 1/8/21 4:31 PM, Mats Wichmann wrote:
>>> > >
>>> > >
>>> > > Someone reported a testsuite break on stuff I work on (scons) with
>>> > > 3.10a4, and it looks similar to this which appears in the changelog
>>> at
>>> > >
>>> > > https://docs.python.org/3.10/whatsnew/changelog.html#changelog
>>> > >
>>> > > bpo-23898: Fix inspect.classify_class_attrs() to support attributes
>>> with
>>> > > overloaded __eq__ and __bool__. Patch by Mike Bayer.
>>> > >
>>> > > Except when I go look at that BPO issue, it's old - closed 2015.  Is
>>> > > this appearing in the 3.10 changelog in error?  Sorry - confused !!!
>>> >
>>> > okay, that was silly, I didn't realize the changelog was cumulative
>>> over
>>> > many versions, so that entry was not for 3.10 at all (teach me to do
>>> > searching in browser window, where it just flies right past any section
>>> > headings so I miss it was for a different version :) ).
>>> >
>>> > > The test in question does indeed touch a class which overrides
>>> __bool_
>>> > > in order to raise an exception (to say "don't do that"), and in the
>>> test
>>> > > run the (expected) exception is not raised.
>>> >
>>> > So updated information: the test in question is checking if a class (A)
>>> > has an attribute using a truth test, where the attribute's value is an
>>> > instance of another class (B) and expecting that that will cause the
>>> > __bool__ method to be called. [aside: this test is done to validate
>>> that
>>> > a class which really doesn't want this kind of test indeed rejects it]
>>> > That apparently no longer happens, if it's wrapped in a try block ???
>>> > Distilled down to simple case:
>>> >
>>> > class A:
>>> >  pass
>>> >
>>> > class B:
>>> >  def __bool__(self):
>>> >  raise AttributeError("don't do that!")
>>> >
>>> > a = A()
>>> > b = B()
>>> > a.b = b
>>> > # expect this to cause b.__bool__ to be called
>>> > if a.b:
>>> >  print("Found it!")
>>> >
>>> > and it raises the exception.  But when protected:
>>> >
>>> > try:
>>> >  if a.b:
>>> >  pass
>>> > except AttributeError:
>>> >  print("Got expected exception")
>>> > else:
>>> >  print("Missed expected exception")
>>> >
>>> > it won't trigger. But if I add a "real" statement in the block
>>> following
>>> > the "if", then it's back to the pre-3.10 behavior of calling __bool__:
>>> >
>>> >   try:
>>> >  if a.b:
>>> >  dummy = True
>>> > except AttributeError:
>>> >  print("Got expected exception")
>>> > else:
>>> >  print("Missed expected exception")
>>> >
>>> >
>>> > Any thoughts on this?
>>>
>>> Oooh interesting. I tried on a build of 3.10 from October and:
>>> 1) The unguarded version bombed out with an exception
>>> 2) The "if... pass" version reported that it got the exception
>>> 3) The "if... dummy" version reported that it got the exception
>>>
>>> ie every one of them did indeed raise. But on a fresh build from the
>>> master branch, I got the same results you did. That means the change
>>> happened some time between commit 497126f7ea and commit ace008c531, an
>>> 800ish commit span.
>>>
>>> I'll start bisecting to try to track this down. It looks like "if a.b:
>>> pass" is getting partially optimized out; the disassembly shows a
>>> being loaded, its attribute b being looked up, and then it just jumps
>>> to the else - there's no POP_JUMP_IF_FALSE as there is when there's a
>>> bit of actual code in there.
>>>
>>> ChrisA
>>> ___
>>> Python-Dev mailing list -- python-dev@python.org
>>> To unsubscribe send an email to python-dev-le...@python.org
>>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>>> Message archived at
>>> https://mail.python.org/archives/list/python-dev@python.org/message/K2LD2L5RF2ZFUYEXQ3Z5U4TY5QBRFPCQ/
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.pyth

[Python-Dev] Re: seg fault in 3.10a4

[Stestagg]

Hi Robin

It would be ideal if you could please create a new issue here:
https://bugs.python.org/

Thanks

Steve

On Sun, Jan 17, 2021 at 12:53 PM Robin Becker  wrote:

> Not sure if this is where I should report, but I have a segfault in
> reportlab userguide creation with 3.10a4. I am
> getting this info from coredumpctl running archlinux with
>
> Linux hostname 5.10.6-arch1-1 #1 SMP PREEMPT Sat, 09 Jan 2021 18:22:35
> + x86_64 GNU/Linux
>
>
> the error is
> Message: Process 1530711 (python) of user 1000 dumped core.
>
>  Stack trace of thread 1530711:
>  #0  0x7f7c9bf72615 raise (libc.so.6 + 0x3d615)
>  #1  0x7f7c9bf5b862 abort (libc.so.6 + 0x26862)
>  #2  0x7f7c9bfb45e8 __libc_message (libc.so.6 +
> 0x7f5e8)
>  #3  0x7f7c9bfbc27a malloc_printerr (libc.so.6 +
> 0x8727a)
>  #4  0x7f7c9bfbd64c _int_free (libc.so.6 + 0x8864c)
>  #5  0x5631260a01dd n/a
> (/home/robin/devel/reportlab/.py310/bin/python310 + 0x1401dd)
>
> 1) I compile this from source using the standard LTO method my script
> looks like
>
> ###
> CFLAGS="-march=x86-64 -mtune=generic -O3 -pipe -fno-plt
> -fno-semantic-interposition"
> LDFLAGS="-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now
> -rpath=$HOME/LOCAL/3.10.0a4/lib -fno-semantic-interposition"
>
>
> ./configure \
> --prefix=$HOME/LOCAL/3.10.0a4 \
> --with-computed-gotos \
> --enable-optimizations \
> --with-lto \
> --enable-ipv6 \
> --with-system-expat \
> --with-dbmliborder=gdbm:ndbm \
> --with-system-ffi \
> --with-system-libmpdec \
> --enable-loadable-sqlite-extensions && make
> EXTRA_CFLAGS="$CFLAGS" && make install
> ###
>
> 1) can I get more information using a different compile setup
>
> 2) or can I get more information with the python run. I just used
> > $ python genuserguide.py
> > Built story contains 1843 flowables...
> > Saved
> "/home/robin/devel/reportlab/REPOS/reportlab/docs/reportlab-userguide.pdf"
> > free(): invalid pointer
> > Aborted (core dumped)
>
> 3) I am running in an old style virtual environment generated using a
> modified virtualenv 16.2.0 (to fix wrong usage of
> sys.version[:3] in path names)
>
> 4) All other tests seem fine including tests of image generation and the
> accelerated strinGWidths etc etc so I am a bit
> mystified.
> --
> Robin Becker
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/5DCR2R6POCOHTPFUOY4F5G7GTBYX6OPO/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UZSQ6QDZEYSOCLYZVGLP2MC5KG2JM3PH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Security releases of CPython

[Stestagg]

The thing that stood out from this conversation, for me, is: Releases are
too hard, and there’s a risk of not having enough volunteers as a result.

How hard is it to fix that?

Steve

On Fri, 19 Feb 2021 at 22:05, Terry Reedy  wrote:

> On 2/19/2021 5:11 AM, Michał Górny wrote:
> > On Thu, 2021-02-11 at 23:24 -0500, Terry Reedy wrote:
>
> >> Releases are not just a push of a button.  Make the release
> >> job too onerous, and there might not be any more volunteers.
> >
> > While I understand your concerns and sympathize with them,
>
> Your accusations of unfairness in response to my polite, volunteered
> response, suggest otherwise.
>
> > I don't think it's fair to play the volunteer card here.
> ...
> > it just feels unfair for you to be dumping the security work on us.
>
> Which I did not do.  Bye.
>
> --
> Terry Jan Reedy
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/UOWWSWZ3O4NV6PIQBMF2KKBVSTK6HXL3/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2OXDTSZENCMKBOPWXW5E77LM7ENC6PBU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Security releases of CPython

[Stestagg]

On Fri, 19 Feb 2021 at 22:50, Christian Heimes  wrote:

> On 19/02/2021 23.22, Stestagg wrote:
> > The thing that stood out from this conversation, for me, is: Releases
> > are too hard, and there’s a risk of not having enough volunteers as a
> > result.
> >
> > How hard is it to fix that?
>
> Actually it's easy to fix!
>
> The PSF needs needs sufficient money to hire a couple of people, so the
> PSF can turn release management and security maintenance from unpaid
> volunteers into paid fulltime jobs.
>
That’s certainly one option. I’d be super surprised if that were the only
one.

What were the budget requirements for this? How far from the target is the
PSF?  Is there a plan to hit it?

Are there no technical solutions that might help reduce the cost?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/55GFSMWJOG7P6NWU4KDHMKJN2OGX5YSG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

[Stestagg]

This pep sounded kinda scary to me, so I wanted to try it out.

The reference implementation appears to have some bugs in it (around
reraise star) , so it's not entirely clear what the expected behavior is
supposed to be, but by checking out ffc493b5 I got some OK results.

I had a look at the tempfile example given under the Motivations section,
and wanted to see how this would work (as it's the area of code I'm most
familiar with).

Would the proposed tempfile change look something like this? (functionally,
if not stylistically):

--- a/Lib/tempfile.py
+++ b/Lib/tempfile.py
@@ -819,8 +819,14 @@ def __repr__(self):
 def __enter__(self):
 return self.name

-def __exit__(self, exc, value, tb):
-self.cleanup()
+def __exit__(self, exc_cls, exc_value, tb):
+try:
+self.cleanup()
+except Exception as clean_exc:
+if exc_value is not None:
+raise ExceptionGroup('Exception occurred during cleanup',
[exc_value, clean_exc])
+else:
+raise

 def cleanup(self):
 if self._finalizer.detach():

If so, then the following code fails to catch the ZeroDivisionError (there
is an uncaught exception raised):

import tempfile, os, pathlib

def do_some_stuff():
with tempfile.TemporaryDirectory() as td:
os.rmdir(td)
pathlib.Path(td).write_text("Surprise!")
1/0

if __name__ == '__main__':
try:
do_some_stuff()
except Exception:
print("Something went wrong")
else:
print("No error")

The output I get:
"""
Traceback (most recent call last):
  File "/home/sstagg/tmp/fuzztest/cpython/td.py", line 7, in do_some_stuff
1/0
ZeroDivisionError: division by zero

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/sstagg/tmp/fuzztest/cpython/Lib/tempfile.py", line 824, in
__exit__
self.cleanup()
  File "/home/sstagg/tmp/fuzztest/cpython/Lib/tempfile.py", line 833, in
cleanup
self._rmtree(self.name)
  File "/home/sstagg/tmp/fuzztest/cpython/Lib/tempfile.py", line 809, in
_rmtree
_shutil.rmtree(name, onerror=onerror)
  File "/home/sstagg/tmp/fuzztest/cpython/Lib/shutil.py", line 718, in
rmtree
_rmtree_safe_fd(fd, path, onerror)
  File "/home/sstagg/tmp/fuzztest/cpython/Lib/shutil.py", line 631, in
_rmtree_safe_fd
onerror(os.scandir, path, sys.exc_info())
  File "/home/sstagg/tmp/fuzztest/cpython/Lib/shutil.py", line 627, in
_rmtree_safe_fd
with os.scandir(topfd) as scandir_it:
NotADirectoryError: [Errno 20] Not a directory: '/tmp/tmpeedn6r0n'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/sstagg/tmp/fuzztest/cpython/td.py", line 12, in 
do_some_stuff()
  File "/home/sstagg/tmp/fuzztest/cpython/td.py", line 7, in do_some_stuff
1/0
  File "/home/sstagg/tmp/fuzztest/cpython/Lib/tempfile.py", line 827, in
__exit__
raise ExceptionGroup('Exception occurred during cleanup', [exc_value,
clean_exc])
ExceptionGroup: Exception occurred during cleanup
"""

===

So, to catch and handle errors raised from TemporaryDirectory safely, the
try-except has to be wrapped in a try-*except block?:


if __name__ == '__main__':
try:
try:
do_some_stuff()
except Exception:
print("Fail Site 1")
except *NotADirectoryError:
print("Fail Site 2")
except *Exception:
print("Fail Site 3")

In this situation, Sites 2 and 3 are called, but if there is no problem
during cleanup then Site 1 is called?

If, instead, the ExceptionGroup is explicitly handled:

if __name__ == '__main__':
try:
do_some_stuff()
except (Exception, ExceptionGroup):
print("Fail Site 1")

Then this actually works quite nicely for the 'except Exception' scenario,
but is much more complex if you want to catch and handle specific types of
exceptions:

if __name__ == '__main__':
try:
do_some_stuff()
except ExceptionGroup as exc_group:
zd_errors, others = exc_group(lambda e: isinstance(e,
ZeroDivisionError))
if zd_errors:
print("Fail Site 1")
if others:
raise others
except ZeroDivisionError:
print("Fail Site 2")

If the idea is that tempfile.TemporaryDirectory *always* raises an
ExceptionGroup (even if there was no cleanup exception) then any code that
calls anything that might eventually call TemporaryDirectory will have to
be aware that a different type of exception could appear that normal
handling doesn't catch (or for /every/ usage of TemporaryDirectory be
immediately wrapped in try-*except handling code).

It feels like ever letting an ExceptionGroup unwind more than 1 or two
stack frames is super dangerous, as it directly requires all 

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

[Stestagg]

On Tue, Feb 23, 2021 at 11:00 PM Irit Katriel 
wrote:

>
> Hi Steve,
>
> Thank you for trying out the implementation. Please do let me know about
> bugs you find.
>
>
Thanks you for your response!  It seems I had missed the (now) obvious
naked exception wrapping logic.  Apologies for adding noise to the
conversation.

I've added a comment to your fork's PR with the error detail.

Best of luck with the PEP

Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6R6CKUL3QDROZFIFVK7WJRD75DMZQ43X/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: [Release management] schedule for renaming the default branch

[Stestagg]

Hi

I would be great to read the impact analysis for this change, and
understand who the identified impacted parties are, and what the plan is to
notify them and help them update within this timescale.

Has this analysis been published anywhere? I know there are lots of places
where discussions/documentation happens

Thanks

Steve



On Wed, Mar 10, 2021 at 2:10 PM Pablo Galindo Salgado 
wrote:

> Hi,
>
> I am writing on behalf of the Python Release Management team. The Steering
> Council has requested the RM team to schedule
> and perform the necessary changes to rename the default branch from master
> to main.
>
> # The changes
>
> What follows is the technical description of the changes and the timeline.
> In order to keep this thread focused on this particular
> aspect, if you wish to discuss anything related to the change itself,
> please, open a new email thread or reuse an existing one.
>
>- The change will be executed ***when beta 1 is released***, as the
>beta 1 release requires some branching engineering already
>to create the 3.10 branch and point 3.11 to the new one as well as
>changing CI, buildbots...etc **
> *This is scheduled for Monday, 2021-05-03**.*
>- The CI will be adapted to work with the new "main" branch.
>- The buildbots will be adapted to work with the new "main" branch.
>- Branch protection rules will be adapted.
>- The different bots will be adapted by the respective bot maintainer
>teams.
>- All the URLs that point to master in the README and other places
>will be adapted to point to main instead (notice this is
>not strictly necessary because GitHub redirects automatically).
>
> Notice that the renaming will automatically:
>
>
>- Re-target any open pull requests
>- Update any draft releases based on the branch
>- Move any branch protection rules that explicitly reference the old
>name
>- Show a notice to repository contributors, maintainers, and admins on
>the repository homepage with instructions to update local copies of the
>repository
>- Show a notice to contributors who git push to the old branch
>- Redirect web requests for the old branch name to the new branch name
>- Return a "Moved Permanently" response in API requests for the old
>branch name
>
> Check this  for more information.
>
> # What you need to do?
>
> You just need to update your local clone after the branch name changes.
> From the local clone of the repository on a computer,
> run the following commands to update the name of the default branch.
>
> $ git branch -m master main
> $ git fetch origin
> $ git branch -u origin/main main
>
> Apart from that, you should update any local script or command that uses
> the name "master" to use the name "main".
>
> Regards from windy London,
> Pablo Galindo Salgado
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/QWW7KGBW5UH2N5FOZOFXQBQPYELWQM3O/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/66RS2S6ENU55RPL6AIGJ4UH6LFXNVZWV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: [Release management] schedule for renaming the default branch

[Stestagg]

Thanks for the response!

No. It has been discussed by the Steering Council as you can see in the
> February update:
> https://github.com/python/steering-council/blob/main/updates/2021-02-steering-council-update.md
>
>
Ok, great, I assume the missing bits will be coming in the March update

Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VVLZXD7S36H6GXISRGETKQGAGUNLCZPM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Revive PEP 396 -- Module Version Numbers ?

[Stestagg]

On Mon, Apr 26, 2021 at 10:31 PM Brett Cannon  wrote:

>
>
> On Mon, Apr 26, 2021 at 9:37 AM Baptiste Carvello <
> devel2...@baptiste-carvello.net> wrote:
>
>> Hi,
>>
>> sorry for being late to the party, but I may not be the only one
>> wondering…
>>
>> Le 14/04/2021 à 20:56, Barry Warsaw a écrit :
>> >
>> > I’d forgotten that this PEP was in Deferred state.  I think it should
>> be rejected and I plan on making that change.  importlib.metadata is a much
>> better approach to programmatically determining package versions.
>> >
>> >
>> https://docs.python.org/3/library/importlib.metadata.html#distribution-versions
>>
>> This is indeed the correct approach, thanks for letting me learn this.
>>
>> However, I unsuccessfully searched for the canonical way to look up the
>> distribution name based on either a module name or an imported module
>> object. Is there one?
>>
>
> If you mean how to tie a module back to its name on PyPI, you should be
> able to look up the "Name" in the project's metadata:
> https://docs.python.org/3/library/importlib.metadata.html#distribution-metadata
> .
>
>
The missing bit here, for me, is how do you map a module back to it's
project (distribution)?

For example:

```
>>> import bs4
>>> bs4.__version__
'4.9.1'
>>> importlib.metadata.metadata('bs4')
PackageNotFoundError: bs4
```

This is because the distribution calls itself 'beautifulsoup4' instead.

The same goes for another package: `umap`, for which the distribution is
called `umap-learn`


This is the best I could come up with from reading the docs:

import bs4  #<- This is the module we want the version of

import importlib
import sys
from itertools import chain
from pathlib import Path

loaders = sys.meta_path

target_path = Path(bs4.__file__)

distros = list(chain(*(finder.find_distributions() for finder in
loaders if hasattr(finder, 'find_distributions'
distros_files = chain(*(f for f in (d.files for d in distros)))
distro_files = [(d, d.locate_file(f)) for d in distros if d.files for f
in d.files]
matching = [d for d, f in distro_files if f == target_path]

for match in matching:
print("Found Version:", match.version)

Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NMTZZMVSZ5XIU2BTIFMPU7W734ET5VUA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Preventing Unicode-related gotchas (Was: pre-PEP: Unicode Security Considerations for Python)

[Stestagg]

This is my favourite version of the issue:

е = lambda е, e: е if е > e else e
print(е(2, 1), е(1, 2)) # python 3 outputs: 2 2

https://twitter.com/stestagg/status/685239650064162820?s=21

Steve

On Sat, 13 Nov 2021 at 22:05,  wrote:

> I’ve not been following the thread, but Steve Holden forwarded me the
> email from Petr Viktorin, that I might share some of the info I found while
> recently diving into this topic.
>
>
>
> As part of working on the next edition of “Python in a Nutshell” with
> Steve, Alex Martelli, and Anna Ravencroft, Alex suggested that I add a
> cautionary section on homoglyphs, specifically citing “A” (LATIN CAPITAL
> LETTER A) and “Α” (GREEK CAPITAL LETTER ALPHA) as an example problem pair.
> I wanted to look a little further at the use of characters in identifiers
> beyond the standard 7-bit ASCII, and so I found some of these same issues
> dealing with Unicode NFKC normalization. The first discovery was the
> overlapping normalization of “ªº” with “ao”. This was quite a shock to me,
> since I assumed that the inclusion of Unicode for identifier characters
> would preserve the uniqueness of the different code points. Even ligatures
> can be used, and will overlap with their multi-character ASCII forms. So we
> have added a second note in the upcoming edition on the risks of using
> these “homonorms” (which is a word I just made up for the occasion).
>
>
>
> To explore the extreme case, I wrote a pyparsing transformer to convert
> identifiers in a body of Python source to mixed font, equivalent to the
> original source after NFKC normalization. Here are hello.py, and a snippet
> from unittest/utils.py:
>
>
>
> def 𝚑𝓮𝖑𝒍𝑜():
>
> try:
>
> 𝔥e𝗅𝕝𝚘︴ = "Hello"
>
> 𝕨𝔬r𝓵ᵈ﹎ = "World"
>
> ᵖ𝖗𝐢𝘯𝓽(f"{𝗵ｅ𝓵𝔩º_}, {𝖜ₒ𝒓lⅆ︴}!")
>
> except 𝓣𝕪ᵖｅ𝖤𝗿ᵣ𝖔𝚛 as ⅇ𝗑c:
>
> 𝒑rℹₙₜ("failed: {}".𝕗𝗼ʳᵐªｔ(ᵉ𝐱𝓬))
>
>
>
> if _︴ⁿ𝓪𝑚𝕖__ == "__main__":
>
> 𝒉eℓˡ𝗈()
>
>
>
>
>
> # snippet from unittest/util.py
>
> _𝓟Ⅼ𝖠𝙲𝗘ℋ𝒪Lᴰ𝑬𝕽﹏𝕷𝔼𝗡 = 12
>
> def _𝔰ʰ𝓸ʳ𝕥𝙚𝑛(𝔰, p𝑟𝔢fi𝖝𝕝𝚎𝑛, ｓᵤ𝑓𝗳𝗂𝑥𝗹ₑ𝚗):
>
> ˢ𝗸ｉ𝗽 = 𝐥ｅ𝘯(𝖘) - ｐr𝚎𝖋𝐢x𝗅ᵉ𝓷 - 𝒔𝙪ffｉ𝘅𝗹𝙚ₙ
>
> if sｋi𝘱 > _𝐏𝗟𝖠𝘊𝙴H𝕺Ｌ𝕯𝙀𝘙﹏L𝔈𝒩:
>
> 𝘴 = '%s[%d chars]%s' % (𝙨[:𝘱𝐫𝕖𝑓𝕚ｘℓ𝒆𝕟], ₛ𝚔𝒊p, 𝓼[𝓁𝒆𝖓(
> 𝚜) - 𝙨𝚞𝒇fix𝙡ᵉ𝘯:])
>
> return ₛ
>
>
>
>
>
> You should able to paste these into your local UTF-8-aware editor or IDE
> and execute them as-is.
>
>
>
> (If this doesn’t come through, you can also see this as a GitHub gist at 
> Hello,
> World rendered in a variety of Unicode characters (github.com)
> <https://gist.github.com/ptmcg/bf35d5ada416080d481d789988b6b466>. I have
> a second gist containing the transformer, but it is still a private gist
> atm.)
>
>
>
>
>
> Some other discoveries:
>
> “·” (ASCII 183) is a valid identifier body character, making “_···” a
> valid Python identifier. This could actually be another security attack
> point, in which “s·join(‘x’)” could be easily misread as “s.join(‘x’)”, but
> would actually be a call to potentially malicious method “s·join”.
>
> “_” seems to be a special case for normalization. Only the ASCII “_”
> character is valid as a leading identifier character; the Unicode
> characters that normalize to “_” (any of the characters in “︳︴﹍﹎﹏＿”) can
> only be used as identifier body characters. “︳” especially could be
> misread as “|” followed by a space, when it actually normalizes to “_”.
>
>
>
>
>
> Potential beneficial uses:
>
> I am considering taking my transformer code and experimenting with an
> orthogonal approach to syntax highlighting, using Unicode groups instead of
> colors. Module names using characters from one group, builtins from
> another, program variables from another, maybe distinguish local from
> global variables. Colorizing has always been an obvious syntax highlight
> feature, but is an accessibility issue for those with difficulty
> distinguishing colors. Unlike the “ransom note” code above, code
> highlighted in this way might even be quite pleasing to the eye.
>
>
>
>
>
> -- Paul McGuire
>
>
>
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/GBLXJ2ZTIMLBD2MJQ4VDNUKFFTPPIIMO/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/664TQW7KCLKQIMELI4VSP6LRDUWBOVRJ/
Code of Conduct: http://python.org/psf/codeofconduct/