[issue38657] String format for hexadecimal notation breaks padding with alternative form
New submission from Pete Wicken :
When formatting an integer as a hexadecimal value, the '#' alternate form
modifier inserts a preceding '0x'.
If this is used in combination with padding modifiers, the '0x' is counted as
part of the overall width, which does not feel like the natural behaviour as
extra calculation is required to get the correct post '0x' precision.
Example:
In [7]: f'{num:04x}'
Out[7]: '0800'
In [8]: f'{num:#04x}'
Out[8]: '0x800'
To get the hexadecimal representation padded to 4 digits, you have to account
for the preceding 0x:
In [10]: f'{num:#06x}'
Out[10]: '0x0800'
--
messages: 355767
nosy: Wicken
priority: normal
severity: normal
status: open
title: String format for hexadecimal notation breaks padding with alternative
form
type: behavior
versions: Python 3.7, Python 3.8
___
Python tracker
<https://bugs.python.org/issue38657>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38657] String format for hexadecimal notation breaks padding with alternative form
Pete Wicken added the comment:
Given the comments above I appreciate that this is actually due to the padding
being the total field width rather than the padding of the digits themselves.
Having revised the documentation again, I believe this following line is
explaining it:
"When no explicit alignment is given, preceding the width field by a zero ('0')
character enables sign-aware zero-padding for numeric types. This is equivalent
to a fill character of '0' with an alignment type of '='."
(https://docs.python.org/3.8/library/string.html)
I initially read "sign-aware zero-padding for numeric types" to mean the
padding would not blindly prepend, and would take into account any signs and
pad after (hence initially making this a bug). So maybe as suggested above we
should explicitly mention the padding is the total number of characters in the
field, rather than just the numbers.
I can look into adding this soon and see what you all think.
--
___
Python tracker
<https://bugs.python.org/issue38657>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38657] String format for hexadecimal notation breaks padding with alternative form
Change by Pete Wicken : -- keywords: +patch pull_requests: +16548 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/17036 ___ Python tracker <https://bugs.python.org/issue38657> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38655] ipaddress.ip_network('0.0.0.0/0').is_private == True
Pete Wicken added the comment: Looks like this happens because the is_private method that gets called is from _BaseNetwork, which checks if the network address '0.0.0.0' and the broadcast address '255.255.255.255' are both private, which they are as 0.0.0.0 falls into 0.0.0.0/8. I think for this to get it right, you would have to change the is_private check for networks to iterate over each possible subnet and check if that is in the private networks list. This takes an unfeasibly long time. So, we would probably have to add special cases for these networks, unless people have better ideas. -- nosy: +Wicken ___ Python tracker <https://bugs.python.org/issue38655> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28577] ipaddress.ip_network(...).hosts() returns nothing for an IPv4 /32
Change by Pete Wicken : -- pull_requests: +24251 pull_request: https://github.com/python/cpython/pull/25532 ___ Python tracker <https://bugs.python.org/issue28577> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28577] ipaddress.ip_network(...).hosts() returns nothing for an IPv4 /32
Change by Pete Wicken : -- pull_requests: +24252 pull_request: https://github.com/python/cpython/pull/25533 ___ Python tracker <https://bugs.python.org/issue28577> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28577] ipaddress.ip_network(...).hosts() returns nothing for an IPv4 /32
Change by Pete Wicken : -- pull_requests: +24255 pull_request: https://github.com/python/cpython/pull/25536 ___ Python tracker <https://bugs.python.org/issue28577> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33433] ipaddress is_private misleading for IPv4 mapped IPv6 addresses
Change by Pete Wicken : -- keywords: +patch pull_requests: +24799 stage: -> patch review pull_request: https://github.com/python/cpython/pull/26172 ___ Python tracker <https://bugs.python.org/issue33433> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33433] ipaddress is_private misleading for IPv4 mapped IPv6 addresses
Pete Wicken added the comment: I've opened a PR that should hopefully address this issue. -- ___ Python tracker <https://bugs.python.org/issue33433> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38750] Solve IPv4 categorisation issues with the ipaddress module
New submission from Pete Wicken : As alluded to in issue bpo-38655, the behaviour for getting categorising IPv4 networks is inconsistent with the IANA guideline, which the docs say it follows. I'm proposing we either change the documentation so that it describes the behaviour that should actually be expected, or we rewrite some parts of the ipaddress module so that they follow the guidelines. For the latter, my thoughts would be to actually implement the check table on the IANA page (https://www.iana.org/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml). i.e for a given address, you can ask of it "is_forwardable", "is_globally_reachable", etc. -- messages: 356265 nosy: Wicken priority: normal severity: normal status: open title: Solve IPv4 categorisation issues with the ipaddress module ___ Python tracker <https://bugs.python.org/issue38750> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38750] Solve IPv4 categorisation issues with the ipaddress module
Change by Pete Wicken : -- components: +Library (Lib) type: -> behavior versions: +Python 3.9 ___ Python tracker <https://bugs.python.org/issue38750> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38750] Solve IPv4 categorisation issues with the ipaddress module
Pete Wicken added the comment: In addition to my previous comment, I think a more generic "is_special" could cover everything in the IANA special purpose address table for ease of checking anything that isn't publicly available IP. -- ___ Python tracker <https://bugs.python.org/issue38750> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39710] "will be returned as unicode" reminiscent from Python 2
Change by Pete Wicken : -- nosy: +Wicken nosy_count: 3.0 -> 4.0 pull_requests: +18050 pull_request: https://github.com/python/cpython/pull/18691 ___ Python tracker <https://bugs.python.org/issue39710> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39710] "will be returned as unicode" reminiscent from Python 2
Change by Pete Wicken : -- pull_requests: -18050 ___ Python tracker <https://bugs.python.org/issue39710> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39710] "will be returned as unicode" reminiscent from Python 2
Change by Pete Wicken : -- pull_requests: +18053 pull_request: https://github.com/python/cpython/pull/18691 ___ Python tracker <https://bugs.python.org/issue39710> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28577] ipaddress.ip_network(...).hosts() returns nothing for an IPv4 /32
Change by Pete Wicken : -- keywords: +patch nosy: +Wicken nosy_count: 5.0 -> 6.0 pull_requests: +18112 stage: -> patch review pull_request: https://github.com/python/cpython/pull/18757 ___ Python tracker <https://bugs.python.org/issue28577> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28577] ipaddress.ip_network(...).hosts() returns nothing for an IPv4 /32
Pete Wicken added the comment: I've had a go at implementing this. I did not implement for IPv6 as this was not mentioned here, but it seems like it would make sense for it as well. I can add that in too if you like. -- ___ Python tracker <https://bugs.python.org/issue28577> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28577] ipaddress.ip_network(...).hosts() returns nothing for an IPv4 /32
Pete Wicken added the comment: Ok it was bugging me that they were different, so I also added the same logic for IPv6Networks. -- ___ Python tracker <https://bugs.python.org/issue28577> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33433] ipaddress is_private misleading for IPv4 mapped IPv6 addresses
Change by Pete Wicken : -- nosy: +Wicken ___ Python tracker <https://bugs.python.org/issue33433> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28577] ipaddress.ip_network(...).hosts() returns nothing for an IPv4 /32
Pete Wicken added the comment: The patch for this has been merged - I guess this can be closed now? -- ___ Python tracker <https://bugs.python.org/issue28577> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42935] Pickle can't import builtins at exit
New submission from Pete Wicken :
Originally found as an issue in Lib/shelve.py; if we attempt to pickle a
builtin as the program is exiting then Modules/_pickle.c will fail at the point
of the PyImport_Import in save_global. In CPython3.8 this causes a segfault, in
CPython3.9 a PicklingError is raised.
This is especially problematic in shelve.py as object pickling is attempted by
the __del__ method's call stack when writeback=True. Therefore if the program
exits before an explicit sync is called; in 3.8 the data will not be written to
disk and a segfault occurs; in 3.9 the data is written to disk, but with an
uncaught exception being raised.
Exception demonstrated via shelve on 3.9.1 on MacOS with Clang:
Python 3.9.1 (default, Dec 10 2020, 11:11:14)
[Clang 12.0.0 (clang-1200.0.32.27)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import shelve
>>> s = shelve.open('testing', writeback=True)
>>> s['a'] = Exception
>>> exit()
Exception ignored in:
Traceback (most recent call last):
File ".../3.9/lib/python3.9/shelve.py", line 162, in __del__
File ".../3.9/lib/python3.9/shelve.py", line 144, in close
File ".../3.9/lib/python3.9/shelve.py", line 168, in sync
File ".../3.9/lib/python3.9/shelve.py", line 124, in __setitem__
_pickle.PicklingError: Can't pickle : import of module
'builtins' failed
Segfault demonstrated via shelve on 3.8.5 on MacOS with Clang (different system
from above):
Python 3.8.5 (v3.8.5:580fbb018f, Jul 20 2020, 12:11:27)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import shelve
>>> s = shelve.open('testing', writeback=True)
>>> s['a'] = Exception
>>> exit()
[1]10040 segmentation fault python3.8
Exception demonstrated via shelve on 3.9.1 on RHEL with GCC:
Python 3.9.1 (default, Dec 8 2020, 00:00:00)
[GCC 10.2.1 20201125 (Red Hat 10.2.1-9)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import shelve
>>> s = shelve.open("thing", writeback=True)
>>> s['a'] = Exception
>>>
Exception ignored in:
Traceback (most recent call last):
File "/usr/lib64/python3.9/shelve.py", line 162, in __del__
File "/usr/lib64/python3.9/shelve.py", line 144, in close
File "/usr/lib64/python3.9/shelve.py", line 168, in sync
File "/usr/lib64/python3.9/shelve.py", line 124, in __setitem__
_pickle.PicklingError: Can't pickle : import of module
'builtins' failed
Code example to reproduce using Pickle in the class __del__, demonstrated on a
RHEL system:
Python 3.9.1 (default, Dec 8 2020, 00:00:00)
[GCC 10.2.1 20201125 (Red Hat 10.2.1-9)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pickle import DEFAULT_PROTOCOL, Pickler
>>> from io import BytesIO
>>> class T:
... def __del__(self):
... f = BytesIO()
... p = Pickler(f, DEFAULT_PROTOCOL)
... p.dump(sum)
...
>>> t = T()
>>> exit()
Exception ignored in:
Traceback (most recent call last):
File "", line 5, in __del__
_pickle.PicklingError: Can't pickle : import of module
'builtins' failed
Have not tested on 3.6, 3.7 or 3.10.
--
components: Library (Lib)
messages: 385110
nosy: Wicken
priority: normal
severity: normal
status: open
title: Pickle can't import builtins at exit
type: crash
versions: Python 3.8, Python 3.9
___
Python tracker
<https://bugs.python.org/issue42935>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42935] Pickle can't import builtins at exit
Pete Wicken added the comment: Out of curiosity, why is there not much we can do? I'm not familiar enough with Python's C code to appreciate the difficulty of making the builtins available at the point where the pickle save is run when exiting. The fact that this segfaults in the 3.8 version tested was rather concerning. Shelve's implementation without the context manager was how we found the original bug; but it feels the way pickle handles this should be a lot safer. Some clarification might help me craft a warning for the documentation if there is nothing that can be done in the code base. -- ___ Python tracker <https://bugs.python.org/issue42935> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
