[issue1363] python 2.4.4 fails on solaris (sun4u sparc SUNW, Sun-Fire-880)
Paul added the comment: Trying to compile Plone (3.0.2) on a Sun V880 (SunOS genome 5.10 Generic_125100-05 sun4u sparc SUNW,Sun-Fire-880). Plone is a web application which runs on the Zope framework. What fails is the python (2.4.4) which is distributed with Zope. During the configure step of the build, I receive the following warning: *BEGIN Snippet* configure: WARNING: sys/wait.h: present but cannot be compiled configure: WARNING: sys/wait.h: check for missing prerequisite headers? configure: WARNING: sys/wait.h: see the Autoconf documentation configure: WARNING: sys/wait.h: section "Present But Cannot Be Compiled" configure: WARNING: sys/wait.h: proceeding with the preprocessor's result configure: WARNING: sys/wait.h: in the future, the compiler will take precedence configure: WARNING: ## ## configure: WARNING: ## Report this to http://www.python.org/python-bugs ## configure: WARNING: ## ## *END Snippet* Then this nasty error breaks the entire config: **BEGIN Snippet checking size of int... configure: error: cannot compute sizeof (int), 77 See `config.log' for more details. **END Snippet -- severity: normal -> critical type: -> compile error __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1363> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1363] python 2.4.4 fails on solaris (sun4u sparc SUNW, Sun-Fire-880)
New submission from Paul: Trying to compile Plone (3.0.2) on a Sun V880 (SunOS genome 5.10 Generic_125100-05 sun4u sparc SUNW,Sun-Fire-880). Plone is a web application which runs on the Zope framework. What fails is the python (2.4.4) which is distributed with Zope. During the configure step of the build, I receive the following warning: *BEGIN Snippet* configure: WARNING: sys/wait.h: present but cannot be compiled configure: WARNING: sys/wait.h: check for missing prerequisite headers? configure: WARNING: sys/wait.h: see the Autoconf documentation configure: WARNING: sys/wait.h: section "Present But Cannot Be Compiled" configure: WARNING: sys/wait.h: proceeding with the preprocessor's result configure: WARNING: sys/wait.h: in the future, the compiler will take precedence configure: WARNING: ## ## configure: WARNING: ## Report this to http://www.python.org/python-bugs ## configure: WARNING: ## ## *END Snippet* Then this nasty error breaks the entire config: **BEGIN Snippet checking size of int... configure: error: cannot compute sizeof (int), 77 See `config.log' for more details. **END Snippet -- components: Build messages: 56972 nosy: theoryno3 severity: normal status: open title: python 2.4.4 fails on solaris (sun4u sparc SUNW,Sun-Fire-880) versions: Python 2.4 __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1363> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6434] buffer overflow in Zipfile when wrinting more than 2gig file
Paul added the comment:
This is a problem with python2.7 as well. A change in struct between python2.6
and 2.7 raises an exception on overflow instead of silently allowing it. This
prevents zipping any file larger than 4.5G. This exception concurs when
writing the 32-bit headers (which are not used on large files anyway)
The patch should be simple. Just wrap line 1100:
...struct.pack("
<http://bugs.python.org/issue6434>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6434] buffer overflow in Zipfile when wrinting more than 2gig file
Paul added the comment: I attempted to "re-allow overflow" in the struct(...) call by replacing `zinfo.file_size` with `ZIP64_LIMIT % zinfo.file_size` in zipfile.py, and successfully produced a compressed file from a 10G file, but the resulting compressed file could not be uncompressed and was deemed "invalid" by any unzip util I tried. -- ___ Python tracker <http://bugs.python.org/issue6434> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5396] os.read not handling O_DIRECT flag
Paul added the comment: Michael, I ran into the same issue as you. I got it to work by changing the mmap size to 8K. d = os.open(disk_file_path, os.O_RDWR | os.O_DIRECT | os.O_SYNC | os.O_DSYNC) readbuf = mmap.mmap(-1, 8192) os.lseek(d, 0, os.SEEK_SET) fo = os.fdopen(d, 'rb') fo.readinto(readbuf) Should work. What's strange is that further multiples of 4K seem to work OK. readbuf = mmap.mmap(-1, 4096 * 3) Also works... So what's going on with 4K? -- nosy: +yoyoyopcp ___ Python tracker <https://bugs.python.org/issue5396> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5396] os.read not handling O_DIRECT flag
Paul added the comment:
I've dug into stracing this python program in 2.7 vs. 3.7.
directread.py
import mmap
import os
fd = os.open('/dev/dm-2', os.O_DIRECT | os.O_RDWR) # mapped block device
fo = os.fdopen(fd, 'rb+')
m = mmap.mmap(-1, 4096)
fo.readinto(m)
Python 2.7 result:
...
open("/dev/dm-2", O_RDWR|O_DIRECT) = 3
...
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, -1, 0) =
0x7f743db31000
...
read(0x3, 0x7f743db31000, 0x1000) = 0x1000
...
Python 3.7 result:
...
open("/dev/dm-2", O_RDWR|O_DIRECT|O_CLOEXEC) = 3
...
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, -1, 0) =
0x7f5e087ee000
...
read(0x3, 0x256c8a0, 0x1000)= -1 (errno 22)
Notice that Python 3 isn't using the mmap buffer for the read. Why is it using
a stack buffer?
--
___
Python tracker
<https://bugs.python.org/issue5396>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38167] O_DIRECT read fails with 4K mmap buffer
New submission from Paul : The following Python3 script fails. import mmap import os fd = os.open(path_to_file, os.O_DIRECT | os.O_RDWR) fo = os.fdopen(fd, 'rb+') m = mmap.mmap(-1, 4096) fo.readinto(m) But it worked for Python2. It also works for any other multiple of 4K. For example: m = mmap.mmap(-1, 8192) fo.readinto(m) Is fine! -- components: IO messages: 352397 nosy: yoyoyopcp priority: normal severity: normal status: open title: O_DIRECT read fails with 4K mmap buffer type: behavior versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue38167> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38167] O_DIRECT read fails with 4K mmap buffer
Change by Paul : -- keywords: +patch pull_requests: +15741 stage: -> patch review pull_request: https://github.com/python/cpython/pull/16130 ___ Python tracker <https://bugs.python.org/issue38167> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38167] O_DIRECT read fails with 4K mmap buffer
Change by Paul : -- pull_requests: +15742 pull_request: https://github.com/python/cpython/pull/16131 ___ Python tracker <https://bugs.python.org/issue38167> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38167] O_DIRECT read fails with 4K mmap buffer
Paul added the comment: This is the platform that I'm working on as well as the failure. I have a review out for a fix. # uname -a Linux init129-13 3.10.0-957.el7.x86_64 x86_64 x86_64 x86_64 GNU/Linux # python3.7 directread.py Traceback (most recent call last): File "small.py", line 7, in fo.readinto(m) OSError: [Errno 22] Invalid argument -- ___ Python tracker <https://bugs.python.org/issue38167> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38167] O_DIRECT read fails with 4K mmap buffer
Paul added the comment: > I agree with Josh. If you want to use O_DIRECT, use an unbuffered file object > and be sure to issue reads of the right size. I do not believe an unbuffered file uses O_DIRECT. This is why I use os.open(fpath, os.O_DIRECT). > Also I'm curious: why are you using O_DIRECT, and furthermore, why are you > using it to read into mmap'ed memory? I am testing a storage device and must use O_DIRECT to avoid the kernel's cache. I am using mmap because it was the simplest way to get a page-aligned memory buffer, which is required for direct IO. I believe that this is a bug regardless of the use of mmap, especially considering that this worked in Python 2. I believe the fix I have sent out for review addresses it adequately. -- ___ Python tracker <https://bugs.python.org/issue38167> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38167] O_DIRECT read fails with 4K mmap buffer
Paul added the comment: > Problem is you follow it with: > > fo = os.fdopen(fd, 'rb+') > which introduces a Python level of buffering around the kernel unbuffered > file descriptor. You'd need to pass buffering=0 to make os.fdopen avoid > returning a buffered file object, making it: > fo = os.fdopen(fd, 'rb+', buffering=0) You are absolutely right! This fixed the issue. So... is this not a bug, then? Should I discard my patch? -- ___ Python tracker <https://bugs.python.org/issue38167> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43512] Bug in isinstance(instance, cls) with cls being a protocol? (PEP 544)
New submission from Paul : The section "Subtyping relationships with other types" of PEP 544 states: "A concrete type X is a subtype of protocol P if and only if X implements all protocol members of P with compatible types. In other words, subtyping with respect to a protocol is always structural." This requirement is violated by the current implementation of CPython (version 3.9.2): ``` from typing import Protocol class P(Protocol): pm: str # no default value, but still a protocol member class C(P): # inherits P but does NOT implement pm, since P did not provide a default value pass assert isinstance(C(), P) # violates the PEP 544 requirement cited above C().pm # raises: AttributeError: 'C' object has no attribute 'pm' ``` -- components: Library (Lib) messages: 388827 nosy: paul-dest priority: normal severity: normal status: open title: Bug in isinstance(instance, cls) with cls being a protocol? (PEP 544) type: behavior versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue43512> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43512] Bug in isinstance(instance, cls) with cls being a protocol? (PEP 544)
Paul added the comment: That's the very first issue I've reported in bugs.python.org and I'm completely new to the Python dev process: I have some further remarks at the issue (especially about consistency with the current treatment of Protocols vs. ABCs). Will they be read if placed here after the issue has been closed? Or should I (a) open a new issue or (b) change the status of this issue to "open" first? -- ___ Python tracker <https://bugs.python.org/issue43512> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43512] Bug in isinstance(instance, cls) with cls being a protocol? (PEP 544)
Paul added the comment: Regarding "At runtime, protocol classes will be simple ABCs." (PEP 544): Unfortunately, this is currently not the case. Actually, there is an extra metaclass for protocols, solely to provide an __instancecheck__. https://github.com/python/cpython/blob/3.9/Lib/typing.py#L1096 ``` class _ProtocolMeta(ABCMeta): # This metaclass is really unfortunate and exists only because of # the lack of __instancehook__. def __instancecheck__(cls, instance): # We need this method for situations where attributes are # assigned in __init__. if ((not getattr(cls, '_is_protocol', False) or _is_callable_members_only(cls)) and issubclass(instance.__class__, cls)): return True if cls._is_protocol: if all(hasattr(instance, attr) and # All *methods* can be blocked by setting them to None. (not callable(getattr(cls, attr, None)) or getattr(instance, attr) is not None) for attr in _get_protocol_attrs(cls)): return True return super().__instancecheck__(instance) ``` Regarding "There is no intent to provide sophisticated runtime instance and class checks against protocol classes." (PEP 544): I fully understand that. But a runtime instance check that simply checks, if a protocol member is there, is not sophisticated. And as you can see in the code above, these checks are already implemented, but unfortunately they don't cover the case reported by me in the initial message. I could provide a patch for the _ProtocolMeta to cover the case reported by me. It's just a matter of a couple of lines. Even if the runtime isinstance() checking is not required to give the right answer, I think the right answer would be nice - at least for the most basic checks as "Are the protocol members there?" Regarding "if you inherit from a protocol you are deemed to implement it": I couldn't find a rule with this meaning in any of the typing PEPs. But in my point of view, the problem is a different one: If the instance to check is of a class implemented by another developer (maybe the class is from a third-party library - Bob's library), then such a rule does not help the first developer (Alice). Alice doesn't know anything about such-a-rule-compliance of Bob's classes. She just wants to check if the instance returned by one of Bob's functions complies to the protocol. - The bottom line is: I'd like to provide a patch if you want me to. If you think the current implementation must not be touched, then I would appreciate if the reported case could be documented. I could deliver a draft for this, as well. Currently, the last examples in the sections "Protocol members" and "Explicitly declaring implementation" in PEP 544 contain protocol members with no default implementation in the protocol, but do not suggest the behavior reported above. -- ___ Python tracker <https://bugs.python.org/issue43512> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43512] Bug in isinstance(instance, cls) with cls being a protocol? (PEP 544)
Paul added the comment: The authors of PEP 544 are Ivan Levkivskyi, Jukka Lehtosalo, and Łukasz Langa. I think their opinion should count. I can see "levkivskyi" in the noisy list, but not the other two. And don't see any possibility to add them. Who can add them? And if added: will they read a notification of an issue in state "closed"? -- ___ Python tracker <https://bugs.python.org/issue43512> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43512] Bug in isinstance(instance, cls) with cls being a protocol? (PEP 544)
Paul added the comment: @kj Thank you, Ken! I'll try it on the list as advised by you! -- ___ Python tracker <https://bugs.python.org/issue43512> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43512] Bug in isinstance(instance, cls) with cls being a protocol? (PEP 544)
Change by Paul : -- nosy: +Jukka Lehtosalo, lukasz.langa ___ Python tracker <https://bugs.python.org/issue43512> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44046] When writing to the Registry using winreg, it currently allows you to write ONLY to HKEY_CURRENT_USERS.
New submission from Paul : DETAILS: "[WinError 5] Access is denied" error is thrown when user attempts to use a different Registry hive other than HKEY_CURRENT_USER. The first example below will demonstrate that the code snippet works just fine and is implemented correctly. However, when you try to run the exact same snippet, but with it pointed to HKEY_LOCAL_MACHINE, it will throw "[WinError 5] Access is denied" error. I have seen plenty of variant examples on how you are supposed to write to the Registry, but when you attempt to do so to other hives, it simply does NOT work. Complete demonstration below of it working, and not working... # WORKING example: registry = winreg.ConnectRegistry(None, winreg.HKEY_CURRENT_USER) wholeKey = winreg.OpenKey(registry, 'SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\Winlogon', 0, winreg.KEY_ALL_ACCESS | winreg.KEY_WOW64_64KEY) winreg.SetValue(wholeKey, 'AutoAdminLogon', winreg.REG_SZ, '1') winreg.CloseKey(wholeKey) # NON-WORKING example: registry = winreg.ConnectRegistry(None, winreg.HKEY_LOCAL_MACHINE) wholeKey = winreg.OpenKey(registry, 'SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\Winlogon', 0, winreg.KEY_ALL_ACCESS | winreg.KEY_WOW64_64KEY) winreg.SetValue(wholeKey, 'AutoAdminLogon', winreg.REG_SZ, '1') winreg.CloseKey(wholeKey) -- messages: 392982 nosy: paulenet priority: normal severity: normal status: open title: When writing to the Registry using winreg, it currently allows you to write ONLY to HKEY_CURRENT_USERS. type: behavior versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue44046> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44046] When writing to the Registry using winreg, it currently allows you to write ONLY to HKEY_CURRENT_USERS.
Paul added the comment: Clarification: User is a local admin on the machine, and UAC is disabled as well. I can also add, modify, and delete Registry entries in HKLM (or any other hive) by hand with no problem, so it is definitely not a permissions issue. I can also write, update, modify, etc. anything in HKLM using C# just fine, but NOT with winreg in Python. Note: Keep in mind that the exact same Registry write methods work as long as you are pointed to HKCU hive. However, if you point to HKLM and do not change anything else, winreg will throw a permissions error every time, no matter what combination of permission flags you specify. There is no reason why a user should be able to write to HKCU but not HKLM. Try it and you will experience the same. This is clearly a major bug and should be fixed. It severely limits developers from writing to other Registry hives. While HKCU is quite common to use, HKLM is the most common that developers use, especially when you don't want specific Registry keys to be accessible only to a specific user. You will be able to reproduce and see the problem if you try the two examples I provided. Have you tried the two code snippets I provided yet? Please do so, and confirm the outcome so that you are on the same page. -- ___ Python tracker <https://bugs.python.org/issue44046> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44046] When writing to the Registry using winreg, it currently allows you to write ONLY to HKEY_CURRENT_USERS.
Paul added the comment: *** Again, I am using a LOCAL ADMINISTRATIVE account. *** "Actually behind the scenes, winreg uses win32api which doesn't allow setting HKEY_LOCAL_MACHINE keys for unprivileged users. Running the application in admin mode may work because at that point your application getting admin privileges but it also may not work because winreg module may not gain admin privileges even though your application does." That is not correct. What special permissions do I need to leverage winreg to write to HKEY_LOCAL_MACHINE that a normal local administrator does not have? By default, any user that has adequate permissions to write / modify directly to their HKCU hive, can also do so in HKEY_LOCAL_MACHINE. The only isolated exceptions are where there are specialized keys in which custom permissions were modified outside the default inherited permissions. To be clear, there is no "admin mode", but I think what you mean is being logged in either with local Administrator account, or logged in with an account that has local Administrator permissions. (You can either be logged in as Administrator, or have an account that has local admin privileges, both of which I am doing when I am executing winreg methods that fail with permission errors.) "When you run regedit you get an admin pop up, right? Like that also run python in admin mode." No, there is no pop up. The reason there is no pop up is because as explained previously, my account has local administrative permissions, and secondly, I have UAC turned off. I can freely make changes directly to the Registry in any hive, including HKEY_LOCAL_MACHINE. This is NOT a permissions issue in the Registry and / or with the user account I am using. If there is a permission issue do to a lack of proper winreg security flag, then that is different, and I need to understand which winreg security flag(s) I SHOULD be using so I can correct it. You don't want popups, especially for processes you want to run automatically and unattended, right? If there was a pop up during an attempted change to the Registry, then any methods used would otherwise not have a chance to execute correctly, and any application doing this would obviously break. "I've never used C# but as far as I know it doesn't use win32api while Python does. C# is specifically designed to work with these while C is not therefore it uses win32api. Python uses the C interface of win32api." Actually, C++ and C# uses Win32 API through .NET framework, while also allows even more intimate and direct connection to Win32 (both in managed and unmanaged interfaces). Obviously, due to the differences of outcomes, while the interfaces winreg is using to access the Registry are similar, apparently they are quite different, as I can run equivalent Registry functions in C# with C++ Win32 API, and it runs just fine. Again, no permission issues at all. "Please provide an example that does not involve setting of values (I actually don't want to mess with my registry). If this is occurring while setting of values then I am on right track. If it's not then this requires quite a investigation and debugging. " No, you are not on track yet. How can you test a defect in winreg of writing to HKEY_LOCAL_MACHINE without actually trying to write to it? You can't, and you are not even investigating it yet. If you are not going to leverage the examples provided, then how are you going to learn about this winreg defect? If you don't like the Registry values in the examples, then simply change them to something you are more comfortable with that is equally as innocuous as my examples, as long as you use HKCU on one, and run the exact same one pointed to HKLM, so that you can very quickly and easily see for yourself, and reproduce this defect. If your account permissions are set up correctly and mine are not, then BOTH examples I gave you should execute and write Registry settings to both HKCU and HKLM, without any errors. After running into this issue and reviewing documentation all over again, I also see a pattern that every example out there only uses HKCU hive, while there are zero working examples that demonstrate writing to the Registry in the HKEY_LOCAL_MACHINE. This no longer appears to be just a coincidence, but instead a limitation. While HKCU is quite common for user-specific Registry settings, HKLM is very heavily used for nearly everything that is not user-specific settings. For winreg not being able to write to HKLM hive successfully, regardless if the user is a local admin and has inherited admin privileges to HKLM even by default, then this is a major bug and a huge limitation to winreg that someone should investigate and implement a solution. Also, if you are not going to run the examples I provided (even with your own Registry value
[issue44046] When writing to the Registry using winreg, it currently allows you to write ONLY to HKEY_CURRENT_USERS.
Paul added the comment: Eryk: The whoami process check output shows that my account is in BUILTIN\Administrators, which proves that the account I am logged in as local Administrator permissions. As for the OpenKey method, it fails with [WinError 5] Access denied, exactly the same way my example also failed, and the reason why these consistently fail is because they are pointed to HKLM, essentially replicating the issue that I pointed out. I have also tried different combinations of security flags (winreg.KEY_ALL_ACCESS, winreg.KEY_WOW64_64KEY, etc.), but keep getting the same results. -- ___ Python tracker <https://bugs.python.org/issue44046> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44046] When writing to the Registry using winreg, it currently allows you to write ONLY to HKEY_CURRENT_USERS.
Paul added the comment: @Eryk: GROUP INFORMATION - Group Name: Everyone Type: Well-known group SID:S-1-1-0 Attributes: Mandatory group, Enabled by default, Enabled group Group Name: NT AUTHORITY\Local account and member of Administrators group Type: Well-known group SID:S-1-5-114 Attributes: Group used for deny only Group Name: MACHINE_NAME\docker-users Type: Alias SID:S-1-5-21-3084499296-1678378808-3679662973-1002 Attributes: Mandatory group, Enabled by default, Enabled group Group Name: MACHINE_NAME\ORA_ASMDBA Type: Alias SID:S-1-5-21-3084499296-1678378808-3679662973-1028 Attributes: Mandatory group, Enabled by default, Enabled group Group Name: MACHINE_NAME\ORA_DBA Type: Alias SID:S-1-5-21-3084499296-1678378808-3679662973-1019 Attributes: Mandatory group, Enabled by default, Enabled group Group Name: MACHINE_NAME\ORA_OraDB18Home1_SYSBACKUP Type: Alias SID:S-1-5-21-3084499296-1678378808-3679662973-1025 Attributes: Mandatory group, Enabled by default, Enabled group Group Name: MACHINE_NAME\ORA_OraDB18Home1_SYSDG Type: Alias SID:S-1-5-21-3084499296-1678378808-3679662973-1026 Attributes: Mandatory group, Enabled by default, Enabled group Group Name: MACHINE_NAME\ORA_OraDB18Home1_SYSKM Type: Alias SID:S-1-5-21-3084499296-1678378808-3679662973-1027 Attributes: Mandatory group, Enabled by default, Enabled group Group Name: BUILTIN\Administrators Type: Alias SID:S-1-5-32-544 Attributes: Group used for deny only Group Name: BUILTIN\Users Type: Alias SID:S-1-5-32-545 Attributes: Mandatory group, Enabled by default, Enabled group Group Name: NT AUTHORITY\INTERACTIVE Type: Well-known group SID:S-1-5-4 Attributes: Mandatory group, Enabled by default, Enabled group Group Name: CONSOLE LOGON Type: Well-known group SID:S-1-2-1 Attributes: Mandatory group, Enabled by default, Enabled group Group Name: NT AUTHORITY\Authenticated Users Type: Well-known group SID:S-1-5-11 Attributes: Mandatory group, Enabled by default, Enabled group Group Name: NT AUTHORITY\This Organization Type: Well-known group SID:S-1-5-15 Attributes: Mandatory group, Enabled by default, Enabled group Group Name: NT AUTHORITY\Local account Type: Well-known group SID:S-1-5-113 Attributes: Mandatory group, Enabled by default, Enabled group Group Name: LOCAL Type: Well-known group SID:S-1-2-0 Attributes: Mandatory group, Enabled by default, Enabled group Group Name: NT AUTHORITY\NTLM Authentication Type: Well-known group SID:S-1-5-64-10 Attributes: Mandatory group, Enabled by default, Enabled group Group Name: Mandatory Label\Medium Mandatory Level Type: Label SID:S-1-16-8192 Attributes: -- ___ Python tracker <https://bugs.python.org/issue44046> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44046] When writing to the Registry using winreg, it currently allows you to write ONLY to HKEY_CURRENT_USERS.
Paul added the comment: "The most easy way to do is right click on the application you're running the code from, click Run as Administrator and then run the code in that application. You'll not get any WinError. And also being in the Administrators group doesn't mean whatever application you run has the permission. You have the permission to do manually not the application right? I'm saying this because I'm the admin of my computer still Python raises WinError if I run it normally. Being the admin of my computer I still have to run Python in elevated mode. Windows considers Python as a third party app and it'll not give admin access to it so easily." @Shreyan, Yes, totally understand that. I am also keenly aware of this, because often times for low-level, environment-related solutions in Visual Studio, I have had to set Visual Studio IDE to run as administrator in order for certain operations to function properly. In the case of Python, I am currently using VSCode. When I have set code.exe to run as administrator, and configure python.exe to run as administrator under Compatibility mode, then both VS Code and Python starts acting strange. From the IDE, I can no longer run in debug mode, and any output that is generated is launched in a separate console window, which is viewable only briefly. So... what I also tried was just eliminating VS Code from the equation at the moment. I run a DOS console as Administrator, then just call Python directly along with the .py, and the WinError 5 stops getting thrown. However, the other thing I noticed is that when python.exe is set to run as administrator under Compatibility Mode, and you reopen solution in VS Code, I noticed that the winregistry library stops getting recognized. -- ___ Python tracker <https://bugs.python.org/issue44046> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44046] When writing to the Registry using winreg, it currently allows you to write ONLY to HKEY_CURRENT_USERS.
Paul added the comment: "Here's something you should know about Windows, even if a local account is in the Administrators group, it still has restrictions on what it can do, it just has the power to elevate itself without requiring login credentials (VIA UAC prompts)." @William: Sure, I understand that, which is also why I have UAC prompts disabled. Also, there are additional security settings that most people do not know about Administrator accounts: There are permission settings that go much further than Administrator and disabling of UAC, which is also providing your user account "system" level permissions, and "Act as part of the operating system". This pretty much puts your account in "god mode" where you can do all kinds of things that most of us probably shouldn't need to do for most situations, but it is there when needed. In any case, it should not be necessary to get Python permissions to execute write / update methods in Python against HKLM hive. I definitely don't need to do much to get it to work in C# or C++ just fine. -- ___ Python tracker <https://bugs.python.org/issue44046> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45432] sys.argv is processed strangely under Windows
New submission from Paul : here is my test file: ''' import sys print(sys.argv) ''' when I then try 'python test.py ^test' the ^ character is stripped away, this doesn't happen on Linux. This also doesn't happen if I put ^test in quotes (only ") the ' quotes don't work -- components: Windows messages: 403656 nosy: paul.moore, paulhippler21, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: sys.argv is processed strangely under Windows type: behavior versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue45432> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45432] sys.argv is processed strangely under Windows
Paul added the comment: oh ok. thx -- ___ Python tracker <https://bugs.python.org/issue45432> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40764] Conflation of Counter with Multiset
New submission from Paul :
The collections docs state: "Several mathematical operations are provided for
combining Counter objects to produce multisets (counters that have counts
greater than zero)."
I am surprised at the clear level of decision into conflating counters with
multisets. Why break all functionality for negative counts in favour of
multisets? Why not create a Multiset object for multisets?
One example use of negative counts is in factorisation
(https://bugs.python.org/msg368298 will be surprised counters don't count)
18 = 2**1 * 3**2 --> x18 = Counter({2: 1, 3: 2})
4 = 2**2 --> x4 = Counter({2: 2})
To compute 18/4 in this representation (which I believe is exactly precisely a
count), one would expect
18/4 = 2**-1 * 3**2 --> x4_5 = x18 - x4 = Counter({2: -1, 3: 2})
But instead,
x18 - x4 = Counter({3: 2}) = 9 ???
This is just an example. The use case for negative counts is plain and obvious.
The question is: why does collections break counter behaviour in favour of
conflation with multisets? Why not have two objects: Counter for counters and
Multiset for multisets?
--
components: Library (Lib)
messages: 369867
nosy: wpk-
priority: normal
severity: normal
status: open
title: Conflation of Counter with Multiset
type: behavior
versions: Python 3.10, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python
3.9
___
Python tracker
<https://bugs.python.org/issue40764>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31947] names=None case is not handled by EnumMeta._create_ method
New submission from Paul : It seems to me that this method should not have names=None default value in signature, because that case is not handled, nor is it described as a possible value in the docstring. Seems like maybe a copy and paste from __call__, which has basically same signature, but names=None is valid and handled there. -- messages: 305591 nosy: anentropic priority: normal pull_requests: 4251 severity: normal status: open title: names=None case is not handled by EnumMeta._create_ method type: behavior versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8 ___ Python tracker <https://bugs.python.org/issue31947> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35215] Replacing CPython memory allocation
New submission from paul : Hi all, I am trying to replace the version of malloc/free...etc with my own function suit. I am have issues with loading the initial library setup. I am looking for wisdom from the community as to why this may be the case. Facts: - i just grabbed the latest cpython repo - my memory suit seem to be working from independent testing on other code - i am working on linux - i went into obmalloc.c and replaced the malloc, free, realloc, calloc with my own functions. - i changed the mmap/unmap to use my malloc and free in obmalloc.c - my allocated produces aligned allocations. - i dump the exceptions text being generated to see what is happening: EXCEPTION:: module 'sys' has no attribute '__file__' EXCEPTION:: type object 'BuiltinImporter' has no attribute '_ORIGIN' EXCEPTION:: module 'sys' has no attribute '__cached__' EXCEPTION:: module 'sys' has no attribute '__path__' EXCEPTION:: module 'builtins' has no attribute '__file__' EXCEPTION:: type object 'BuiltinImporter' has no attribute '_ORIGIN' EXCEPTION:: module 'builtins' has no attribute '__cached__' EXCEPTION:: module 'builtins' has no attribute '__path__' EXCEPTION:: module '_frozen_importlib' has no attribute '__file__' EXCEPTION:: type object 'FrozenImporter' has no attribute '_ORIGIN' EXCEPTION:: module '_frozen_importlib' has no attribute '__cached__' EXCEPTION:: module '_frozen_importlib' has no attribute '__path__' EXCEPTION:: module '_imp' has no attribute '__file__' EXCEPTION:: type object 'BuiltinImporter' has no attribute '_ORIGIN' EXCEPTION:: module '_imp' has no attribute '__cached__' EXCEPTION:: module '_imp' has no attribute '__path__' EXCEPTION:: name '_bootstrap' is not defined EXCEPTION:: name '_bootstrap' is not defined EXCEPTION:: name '_bootstrap' is not defined EXCEPTION:: name '_bootstrap' is not defined Fatal Python error: initfsencoding: failed to get the Python codec of the filesystem encoding Traceback (most recent call last): File "/home/paul/fresh_cpython/debug/../Lib/encodings/__init__.py", line 31, in File "", line 989, in _find_and_load File "", line 973, in _find_and_load_unlocked File "", line 671, in _load_unlocked File "", line 773, in exec_module File "", line 909, in get_code File "", line 966, in get_data OSError: [Errno 14] Bad address Aborted -- components: Library (Lib) messages: 329725 nosy: paul.har...@rakuten.com priority: normal severity: normal status: open title: Replacing CPython memory allocation type: enhancement versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue35215> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35215] Replacing CPython memory allocation
paul added the comment: I suspect that this _bootstrap library is not being loaded correctly, and i can only assume that this is somehow because of my memory suit, but i am not really sure where to start hunting, as there is a lot of code. My goal is just to replace malloc. If there is a better way or i am making some wrong assumption, please let me know. Best, Paul -- ___ Python tracker <https://bugs.python.org/issue35215> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35215] Replacing CPython memory allocation
paul added the comment: Hi guys, First, thanks for the prompt replies. @matrixise: If there is i can't find it, but i am happy to accept it as a possibility. Equally, maybe there is an assumption about memory in the cpython implementation somewhere that goes against my memory allocator -> this is obviously not cpythons fault. The point of this 'issue' is to ask the community to suggest how i can go about finding where the problem is. As i said, there is a lot of code :) @pablogsal: Yeah. I had a look in there. In cpython, i have made all allocations from obmalloc use my memory allocation functions, including the arenas. Perhaps a starting point would be if anyone has ideas about how to go about debugging this?? best, Paul -- ___ Python tracker <https://bugs.python.org/issue35215> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18291] codecs.open interprets space as line ends
New submission from Paul:
I hope I am writing in the right place.
When using codecs.open with UTF-8 encoding, it seems characters \x12, \x13, and
\x14 are interpreted as end-of-line.
Example code:
>>> with open('unicodetest.txt', 'w') as f:
>>> f.write('a'+chr(28)+'b'+chr(29)+'c'+chr(30)+'d'+chr(31)+'e')
>>> with open('unicodetest.txt', 'r') as f:
>>> for i,l in enumerate(f):
>>> print i, l
0 a\x12b\x13c\x14d\x15e
The point here is that it reads it as one line, as I would expect. But using
codecs.open with UTF-8 encoding it reads it as many lines:
>>> import codecs
>>> with codecs.open('unicodetest.txt', 'r', 'UTF-8') as f:
>>> for i,l in enumerate(f):
>>> print i, l
0 a\x12
1 b\x13
2 c\x14
3 d\x15e
The characters \x12 through \x15 are described as "Information Separator Four"
through "One" (in that order). As far as I can see they never mark line ends.
Also interestingly, \x15 isn't interpreted as such.
As a sidenote, I tested and verified that io.open is correct (but when reading
loads of data it appears to be 5 times slower than codecs):
>>> import io
>>> with io.open('unicodetest.txt', encoding='UTF-8') as f:
>>> for i,l in enumerate(f):
>>> print i, l
0 a\x12b\x13c\x14d\x15e
--
components: IO, Unicode
messages: 191758
nosy: ezio.melotti, wpk
priority: normal
severity: normal
status: open
title: codecs.open interprets space as line ends
type: behavior
versions: Python 2.6, Python 2.7
___
Python tracker
<http://bugs.python.org/issue18291>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18291] codecs.open interprets space as line ends
Paul added the comment: Sorry for bringing that up as I suppose it is unrelated to the bug I am reporting, but you can an example file attached with timings. -- Added file: http://bugs.python.org/file30688/codecs-io-example.py ___ Python tracker <http://bugs.python.org/issue18291> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18291] codecs.open interprets space as line ends
Paul added the comment: You're absolutely right. I tested it on another machine now, with Python 2.7.3 installed and it is actually twice as fast as codecs. Thanks. So I guess there is little interest in fixing codecs because io is the preferred package for reading unicode files. -- ___ Python tracker <http://bugs.python.org/issue18291> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18291] codecs.open interprets space as line ends
Paul added the comment: Right, #7643 indeed seems to be exactly about the issue I described here (for as much as I know unicode which isn't all that much). So maybe they should be merged. The issue was closed March 2010, is that after 2.7.3 was released? By the way, where I wrote \x12, \x13, \x14, and \x15, I should have written \x1c, \x1d, \x1e, \x1f (the hex representation of characters 28 to 31). Lost in translation, I guess. -- ___ Python tracker <http://bugs.python.org/issue18291> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15565] pdb displays runt Exception strings
New submission from Paul:
In Python 2.6, pdb doesn't show exception strings properly:
#somecode.py
import pdb
pdb.set_trace()
raise Exception('This is a message that contains a lot of characters and is
very long indeed.')
#terminal
> somecode.py
-> raise Exception('This is a message that contains a lot of characters and is
very long indeed.')
(Pdb) n
Exception: Exceptio...ndeed.',)
The pdb code assumes that sys.exc_info()[1] is a string. In fact it's an
Exception instance.
The solution I found was to use str()
#pdb.py line 186
print >>self.stdout, exc_type_name + ':', _saferepr(str(exc_value))
This may have been fixed already but I couldn't find any reference to it.
--
components: None
messages: 167550
nosy: powlo
priority: normal
severity: normal
status: open
title: pdb displays runt Exception strings
type: behavior
versions: Python 2.6
___
Python tracker
<http://bugs.python.org/issue15565>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22517] BufferedRWpair doesn't clear weakrefs
New submission from paul:
# static void
# bufferedrwpair_dealloc(rwpair *self)
# {
# _PyObject_GC_UNTRACK(self);
# Py_CLEAR(self->reader);
# Py_CLEAR(self->writer);
# Py_CLEAR(self->dict);
# Py_TYPE(self)->tp_free((PyObject *) self);
# }
#
# Weakrefs to this object contain stale pointer after BufferedRWPair is freed.
--
files: poc_brwpair_weakref.py
messages: 227835
nosy: pkt
priority: normal
severity: normal
status: open
title: BufferedRWpair doesn't clear weakrefs
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file36753/poc_brwpair_weakref.py
___
Python tracker
<http://bugs.python.org/issue22517>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22518] integer overflow in encoding unicode
New submission from paul:
# static PyObject *
# unicode_encode_ucs1(PyObject *unicode,
# const char *errors,
# unsigned int limit)
# {
# ...
# while (pos < size) {
# ...
# case 4: /* xmlcharrefreplace */
# /* determine replacement size */
# for (i = collstart, repsize = 0; i < collend; ++i) {
# Py_UCS4 ch = PyUnicode_READ(kind, data, i);
# ...
# else if (ch < 10)
# 1 repsize += 2+5+1;
# ...
# }
# 2 requiredsize = respos+repsize+(size-collend);
# if (requiredsize > ressize) {
# ...
# if (_PyBytes_Resize(&res, requiredsize))
# ...
# }
# /* generate replacement */
# for (i = collstart; i < collend; ++i) {
# 3 str += sprintf(str, "&#%d;", PyUnicode_READ(kind, data,
i));
# }
#
# 1. ch=0x<10, so repsize = (number of unicode chars in string)*8
#=2^29*2^3=2^32 == 0 (mod 2^32)
# 2. respos==0, collend==0, so requiredsize=repsize==0, so the destination
buffer
#isn't resized
# 3. overwrite
--
files: poc_encode_latin1.py
messages: 227837
nosy: pkt
priority: normal
severity: normal
status: open
title: integer overflow in encoding unicode
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file36754/poc_encode_latin1.py
___
Python tracker
<http://bugs.python.org/issue22518>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22519] integer overflow in computing byte's object representation
New submission from paul:
# PyBytes_Repr(PyObject *obj, int smartquotes)
# {
# PyBytesObject* op = (PyBytesObject*) obj;
# 1 Py_ssize_t i, length = Py_SIZE(op);
# size_t newsize, squotes, dquotes;
# ...
#
# /* Compute size of output string */
# newsize = 3; /* b'' */
# s = (unsigned char*)op->ob_sval;
# for (i = 0; i < length; i++) {
# ...
# default:
# if (s[i] < ' ' || s[i] >= 0x7f)
# 2 newsize += 4; /* \xHH */
# else
# newsize++;
# }
# }
# ...
# 3 if (newsize > (PY_SSIZE_T_MAX - sizeof(PyUnicodeObject) - 1)) {
# PyErr_SetString(PyExc_OverflowError,
# "bytes object is too large to make repr");
# return NULL;
# }
# 4 v = PyUnicode_New(newsize, 127);
# ...
# *p++ = 'b', *p++ = quote;
# for (i = 0; i < length; i++) {
# ...
# 5 *p++ = c;
# }
# *p++ = quote;
# 6 assert(_PyUnicode_CheckConsistency(v, 1));
# return v;
# }
#
# 1. length=2^30+1=1073741825
# 2. newsize=length*4+3=7 (overflow)
# 3. check is inefficient, because newsize=7
# 4. allocated buffer is too small
# 5. buffer overwrite
# 6. this assert will likely fail, since there is a good chance the allocated
#buffer is just before the huge one, so the huge one will overwrite itself.
--
files: poc_repr_bytes.py
messages: 227838
nosy: pkt
priority: normal
severity: normal
status: open
title: integer overflow in computing byte's object representation
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file36755/poc_repr_bytes.py
___
Python tracker
<http://bugs.python.org/issue22519>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22520] integer overflow in computing unicode's object representation
New submission from paul:
# unicode_repr(PyObject *unicode)
# {
# ...
# 1 isize = PyUnicode_GET_LENGTH(unicode);
# idata = PyUnicode_DATA(unicode);
#
# /* Compute length of output, quote characters, and
#maximum character */
# osize = 0;
# ...
# for (i = 0; i < isize; i++) {
# Py_UCS4 ch = PyUnicode_READ(ikind, idata, i);
# switch (ch) {
# ...
# default:
# /* Fast-path ASCII */
# if (ch < ' ' || ch == 0x7f)
# 2 osize += 4; /* \xHH */
# ...
# }
# }
#
# ...
# 3 repr = PyUnicode_New(osize, max);
# ...
# for (i = 0, o = 1; i < isize; i++) {
# Py_UCS4 ch = PyUnicode_READ(ikind, idata, i);
# ...
# else {
# 4 PyUnicode_WRITE(okind, odata, o++, ch);
# }
# }
# }
# }
# /* Closing quote already added at the beginning */
# 5 assert(_PyUnicode_CheckConsistency(repr, 1));
# return repr;
# }
#
# 1. isize=2^30+1
# 2. osize=isize*4=4
# 3. allocated buffer is too small
# 4. heap overflow
# 5. this assert will likely fail, since there is a good chance the allocated
#buffer is just before the huge one, so the huge one will overwrite itself.
--
files: poc_repr_unicode.py
messages: 227839
nosy: pkt
priority: normal
severity: normal
status: open
title: integer overflow in computing unicode's object representation
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file36756/poc_repr_unicode.py
___
Python tracker
<http://bugs.python.org/issue22520>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22517] BufferedRWpair doesn't clear weakrefs
paul added the comment: Why did the type changed from security to crash? -- ___ Python tracker <http://bugs.python.org/issue22517> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22643] Integer overflow in case_operation
New submission from paul:
Crashes python 3.4.1.
# Objects\unicodeobject.c
#
# static PyObject *
# case_operation(PyObject *self,
#Py_ssize_t (*perform)(int, void *, Py_ssize_t, Py_UCS4 *,
Py_UCS4 *))
# {
# PyObject *res = NULL;
# Py_ssize_t length, newlength = 0;
# int kind, outkind;
# (...)
# 1 length = PyUnicode_GET_LENGTH(self);
# 2 tmp = PyMem_MALLOC(sizeof(Py_UCS4) * 3 * length);
# (...)
# 3 newlength = perform(kind, data, length, tmp, &maxchar);
#
# 1. there are no safety checks
# 2. 12*length overflows
# 3. perform() writes to tmp buffer, which is too small to hold the result
--
files: poc_case_op.py
messages: 229455
nosy: pkt
priority: normal
severity: normal
status: open
title: Integer overflow in case_operation
type: security
versions: Python 3.4
Added file: http://bugs.python.org/file36941/poc_case_op.py
___
Python tracker
<http://bugs.python.org/issue22643>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23361] integer overflow in winapi_createprocess
New submission from paul:
winapi_createprocess takes env_mapping dictionary as a parameter, mapping
variables to their env. values. Dictionary with pathologically large values
will cause an integer overflow during computation of total space required to
store all key-value pairs
File: Modules\_winapi.c
static PyObject*
getenvironment(PyObject* environment)
{
Py_ssize_t i, envsize, totalsize;
...
envsize = PyMapping_Length(environment);
keys = PyMapping_Keys(environment);
values = PyMapping_Values(environment);
if (!keys || !values)
goto error;
totalsize = 1; /* trailing null character */
for (i = 0; i < envsize; i++) {
PyObject* key = PyList_GET_ITEM(keys, i);
PyObject* value = PyList_GET_ITEM(values, i);
if (! PyUnicode_Check(key) || ! PyUnicode_Check(value)) {
PyErr_SetString(PyExc_TypeError,
"environment can only contain strings");
goto error;
}
totalsize += PyUnicode_GET_LENGTH(key) + 1;/* +1 for '=' */
1 totalsize += PyUnicode_GET_LENGTH(value) + 1; /* +1 for '\0' */
}
2 buffer = PyMem_Malloc(totalsize * sizeof(Py_UCS4));
if (! buffer)
goto error;
p = buffer;
3 end = buffer + totalsize;
4 for (i = 0; i < envsize; i++) {
PyObject* key = PyList_GET_ITEM(keys, i);
PyObject* value = PyList_GET_ITEM(values, i);
X if (!PyUnicode_AsUCS4(key, p, end - p, 0))
goto error;
p += PyUnicode_GET_LENGTH(key);
X *p++ = '=';
X if (!PyUnicode_AsUCS4(value, p, end - p, 0))
goto error;
p += PyUnicode_GET_LENGTH(value);
X *p++ = '\0';
}
1. no overflow checks. We can set totalsize to 2^30, with a crafted dictionary.
2. totalsize*4 == 0, so buffer is 0-bytes long
3. end = buffer+2^30
4. envsize == len(env_mapping). We can make this variable as large as we like.
X. write past the buffer's end. Note size checks in PyUnicode_AsUCS4 are
inefficient, because the size variable (end-p) is very large.
--
messages: 235168
nosy: pkt
priority: normal
severity: normal
status: open
title: integer overflow in winapi_createprocess
type: crash
versions: Python 3.4
___
Python tracker
<http://bugs.python.org/issue23361>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23362] integer overflow in string translate
New submission from paul:
# Bug
# ---
#
# PyObject *
# _PyUnicode_TranslateCharmap(PyObject *input,
# PyObject *mapping,
# const char *errors)
# {
# ...
# size = PyUnicode_GET_LENGTH(input);
# ...
# osize = size;
# 1 output = PyMem_Malloc(osize * sizeof(Py_UCS4));
#
# 1. Input size = 2^30, so osize*sizeof(Py_UCS4)=2^32==0 (modulo 2^32) and
malloc
#allocates a 0 byte buffer
#
# Crash
# -
#
# Breakpoint 2, _PyUnicode_TranslateCharmap (
# input='aa...', mapping={97: 'b'}, errors=0x828c82b "ignore") at
Objects/unicodeobject.c:8597
# 8597{
# ...
# 8636output = PyMem_Malloc(osize * sizeof(Py_UCS4));
# (gdb) print osize
# $1 = 1073741824
# (gdb) print osize*4
# $2 = 0
# (gdb) c
# Continuing.
#
# Program received signal SIGSEGV, Segmentation fault.
# 0x0814aed2 in charmaptranslate_output (
# input='aa...', ipos=51302, mapping={97: 'b'}, output=0xbfc40860,
osize=0xbfc40864, opos=0xbfc40868,
# res=0xbfc40874) at Objects/unicodeobject.c:8574
# 8574(*output)[(*opos)++] = PyUnicode_READ_CHAR(*res, 0);
#
# OS info
# ---
#
# % ./python -V
# Python 3.4.1
#
# % uname -a
# Linux ubuntu 3.8.0-29-generic #42~precise1-Ubuntu SMP Wed Aug 14 15:31:16 UTC
2013 i686 i686 i386 GNU/Linux
#
s="a"*(2**30)
s.translate({ord('a'): 'b'})
--
files: poc_translate.py
messages: 235169
nosy: pkt
priority: normal
severity: normal
status: open
title: integer overflow in string translate
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file37961/poc_translate.py
___
Python tracker
<http://bugs.python.org/issue23362>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23363] integer overflow in itertools.permutations
New submission from paul:
# Bug
# ---
#
# static PyObject *
# permutations_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
# {
# ...
# 1 cycles = PyMem_Malloc(r * sizeof(Py_ssize_t));
# ...
# for (i=0 ; i0, so we write well beyond the buffer's end
#
# Crash
# -
#
# Breakpoint 1, permutations_new (type=0x83394e0 ,
args=('A', 1073741824), kwds=0x0) at ./Modules/itertoolsmodule.c:3012
# ...
# 3044indices = PyMem_Malloc(n * sizeof(Py_ssize_t));
# (gdb) print r
# $2 = 1073741824
# (gdb) print r*4
# $3 = 0
# (gdb) c
# Continuing.
#
# Program received signal SIGSEGV, Segmentation fault.
# 0x08230900 in permutations_new (type=0x83394e0 ,
args=('A', 1073741824), kwds=0x0) at ./Modules/itertoolsmodule.c:3054
# 3054cycles[i] = n - i;
#
# OS info
# ---
#
# % ./python -V
# Python 3.4.1
#
# % uname -a
# Linux ubuntu 3.8.0-29-generic #42~precise1-Ubuntu SMP Wed Aug 14 15:31:16 UTC
2013 i686 i686 i386 GNU/Linux
#
import itertools as it
it.permutations("A", 2**30)
--
files: poc_permutations.py
messages: 235170
nosy: pkt
priority: normal
severity: normal
status: open
title: integer overflow in itertools.permutations
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file37962/poc_permutations.py
___
Python tracker
<http://bugs.python.org/issue23363>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23364] integer overflow in itertools.product
New submission from paul:
# Bug
# ---
#
# static PyObject *
# product_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
# {
# ...
# 1 nargs = (repeat == 0) ? 0 : PyTuple_GET_SIZE(args);
# 2 npools = nargs * repeat;
#
# 3 indices = PyMem_Malloc(npools * sizeof(Py_ssize_t));
# ...
#
# 4 for (i=0; i < nargs ; ++i) {
# ...
# indices[i] = 0;
# }
#
# 1. nargs is the number of functions arguments (not counting the keyword arg).
#We set this value to 2^16 using argument unpacking (*args).
# 2. We set the 'repeat' keyword argument to 2^16, so npools=2^32==0 (modulo
2^32)
# 3. npools*4=0, so malloc allocates a 0 byte buffer
# 4. nargs=2^16, so the loop writes well beyond the buffer's end
#
# Breakpoint 1, product_new (type=0x8338c80 ,
# args=('a', ...(truncated), kwds={'repeat': 65536})
# at ./Modules/itertoolsmodule.c:1998
# ...
# 2021nargs = (repeat == 0) ? 0 : PyTuple_GET_SIZE(args);
# (gdb) n
# 2022npools = nargs * repeat;
# (gdb) print nargs
# $14 = 65536
# (gdb) print repeat
# $15 = 65536
# (gdb) n
# 2024indices = PyMem_Malloc(npools * sizeof(Py_ssize_t));
# (gdb) print npools
# $16 = 0
# (gdb) c
# Continuing.
#
# Crash
# -
#
# We crash in a different place, because there was sufficient allocated memory
# after the "indices" buffer.
#
# Program received signal SIGSEGV, Segmentation fault.
# 0x08313940 in PyTuple_Type ()
# (gdb) bt
# #0 0x08313940 in PyTuple_Type ()
# Python Exception 'utf8' codec can't
decode byte 0xc8 in position 1: invalid continuation byte:
# #1 0x080f27c7 in PyObject_Hash (v=) at Objects/object.c:747
# Python Exception 'utf8' codec can't
decode byte 0xc8 in position 1: invalid continuation byte:
# Python Exception 'utf8' codec can't
decode byte 0xc8 in position 1: invalid continuation byte:
# #2 0x080e132f in PyDict_GetItem (op=, key=) at Objects/dictobject.c:1070
# #2 0x080e132f in PyDict_GetItem (op=, key=) at Objects/dictobject.c:1070
# Python Exception 'utf8' codec can't
decode byte 0xc8 in position 1: invalid continuation byte:
# #3 0x080e5261 in _PyDict_GetItemId (dp=, key=0x832bd20
) at Objects/dictobject.c:2729
# #4 0x0806f0e8 in _PySys_GetObjectId (key=0x832bd20 )
at ./Python/sysmodule.c:57
# #5 0x081bb52a in PyEval_EvalFrameEx (f=Frame 0x404ea1ac, for file ,
line 1, in (), throwflag=0) at Python/ceval.c:1848
# Python Exception 'utf8' codec can't
decode byte 0xc8 in position 1: invalid continuation byte:
# Python Exception 'utf8' codec can't
decode byte 0xc8 in position 1: invalid continuation byte:
# #6 0x081c8574 in PyEval_EvalCodeEx (_co=,
globals=, locals=, args=0x0, argcount=0, kws=0x0, kwcount=0,
# defs=0x0, defcount=0, kwdefs=0x0, closure=0x0) at Python/ceval.c:3578
# Python Exception 'utf8' codec can't
decode byte 0xc8 in position 1: invalid continuation byte:
# Python Exception 'utf8' codec can't
decode byte 0xc8 in position 1: invalid continuation byte:
# #7 0x081b51ef in PyEval_EvalCode (co=, globals=,
locals=) at Python/ceval.c:773
# Python Exception 'utf8' codec can't
decode byte 0xc8 in position 1: invalid continuation byte:
# Python Exception 'utf8' codec can't
decode byte 0xc8 in position 1: invalid continuation byte:
# #8 0x08065e89 in run_mod (mod=0x9ea5758, filename='', globals=,
locals=, flags=0xbf85fbc0, arena=0x9e64220)
# at Python/pythonrun.c:2180
# #9 0x080637fd in PyRun_InteractiveOneObject (fp=0x40231ac0 <_IO_2_1_stdin_>,
filename='', flags=0xbf85fbc0)
# at Python/pythonrun.c:1445
# #10 0x08063243 in PyRun_InteractiveLoopFlags (fp=0x40231ac0 <_IO_2_1_stdin_>,
filename_str=0x826bc06 "", flags=0xbf85fbc0)
# at Python/pythonrun.c:1324
# #11 0x0806305f in PyRun_AnyFileExFlags (fp=0x40231ac0 <_IO_2_1_stdin_>,
filename=0x826bc06 "", closeit=0, flags=0xbf85fbc0)
# at Python/pythonrun.c:1286
# #12 0x08079e8a in run_file (fp=0x40231ac0 <_IO_2_1_stdin_>, filename=0x0,
p_cf=0xbf85fbc0) at Modules/main.c:319
# #13 0x0807a988 in Py_Main (argc=1, argv=0x9e45010) at Modules/main.c:751
# #14 0x0805dc34 in main (argc=1, argv=0xbf85fd04) at ./Modules/python.c:69
#
# OS info
# ---
#
# % ./python -V
# Python 3.4.1
#
# % uname -a
# Linux ubuntu 3.8.0-29-generic #42~precise1-Ubuntu SMP Wed Aug 14 15:31:16 UTC
2013 i686 i686 i386 GNU/Linux
#
import itertools as it
args=["a"]*(2**16)
it.product(*args, repeat=2**16)
--
files: poc_product.py
messages: 235172
nosy: pkt
priority: normal
severity: normal
status: open
title: integer overflow in itertools.product
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file37963/poc_product.py
___
Python tracker
<http://bugs.python.org/issue23364>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23365] integer overflow in itertools.combinations_with_replacement
New submission from paul:
# Bug
# ---
#
# static PyObject *
# cwr_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
# {
# ...
# 1 indices = PyMem_Malloc(r * sizeof(Py_ssize_t));
# ...
# for (i=0 ; i0, so we write well beyond the buffer's end
#
# Crash
# -
#
# Breakpoint 1, cwr_new (type=0x83392a0 , args=('AA', 1073741824),
kwds=0x0) at ./Modules/itertoolsmodule.c:2684
# 2684PyObject *pool = NULL;
# ...
# 2703indices = PyMem_Malloc(r * sizeof(Py_ssize_t));
# (gdb) print r
# $1 = 1073741824
# (gdb) print r*4
# $2 = 0
# (gdb) c
# Continuing.
#
# Program received signal SIGSEGV, Segmentation fault.
# 0x0822fdcd in cwr_new (type=0x83392a0 , args=('AA', 1073741824),
kwds=0x0) at ./Modules/itertoolsmodule.c:2710
# 2710indices[i] = 0;
#
# OS info
# ---
#
# % ./python -V
# Python 3.4.1
#
# % uname -a
# Linux ubuntu 3.8.0-29-generic #42~precise1-Ubuntu SMP Wed Aug 14 15:31:16 UTC
2013 i686 i686 i386 GNU/Linux
#
import itertools as it
it.combinations_with_replacement("AA", 2**30)
--
files: poc_cwr.py
messages: 235173
nosy: pkt
priority: normal
severity: normal
status: open
title: integer overflow in itertools.combinations_with_replacement
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file37964/poc_cwr.py
___
Python tracker
<http://bugs.python.org/issue23365>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23366] integer overflow in itertools.combinations
New submission from paul:
# Bug
# ---
#
# static PyObject *
# combinations_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
# {
# ...
#
# 1 indices = PyMem_Malloc(r * sizeof(Py_ssize_t));
# ...
#
# for (i=0 ; i0, so we write well beyond the buffer's end
#
# Crash
# -
#
# Breakpoint 1, combinations_new (type=0x83390c0 ,
args=('AA', 1073741824), kwds=0x0)
# at ./Modules/itertoolsmodule.c:2343
# 2343PyObject *pool = NULL;
# ...
# (gdb) n
# 2362indices = PyMem_Malloc(r * sizeof(Py_ssize_t));
# (gdb) print r
# $1 = 1073741824
# (gdb) print r*4
# $2 = 0
# (gdb) c
# Continuing.
#
# Program received signal SIGSEGV, Segmentation fault.
# 0x0822f359 in combinations_new (type=0x83390c0 ,
args=('AA', 1073741824), kwds=0x0)
# at ./Modules/itertoolsmodule.c:2369
# 2369indices[i] = i;
# OS info
# ---
#
# % ./python -V
# Python 3.4.1
#
# % uname -a
# Linux ubuntu 3.8.0-29-generic #42~precise1-Ubuntu SMP Wed Aug 14 15:31:16 UTC
2013 i686 i686 i386 GNU/Linux
#
import itertools as it
it.combinations("AA", 2**30)
--
files: poc_combinations.py
messages: 235174
nosy: pkt
priority: normal
severity: normal
status: open
title: integer overflow in itertools.combinations
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file37965/poc_combinations.py
___
Python tracker
<http://bugs.python.org/issue23366>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23367] integer overflow in unicodedata.normalize
New submission from paul:
# Bug
# ---
#
# static PyObject*
# unicodedata_normalize(PyObject *self, PyObject *args)
# {
# ...
# if (strcmp(form, "NFKC") == 0) {
# if (is_normalized(self, input, 1, 1)) {
# Py_INCREF(input);
# return input;
# }
# return nfc_nfkc(self, input, 1);
#
# We need to pass the is_normalized() check (repeated \xa0 char takes care of
# that). nfc_nfkc calls:
#
# static PyObject*
# nfd_nfkd(PyObject *self, PyObject *input, int k)
# {
# ...
# Py_ssize_t space, isize;
# ...
# isize = PyUnicode_GET_LENGTH(input);
# /* Overallocate at most 10 characters. */
# space = (isize > 10 ? 10 : isize) + isize;
# osize = space;
# 1 output = PyMem_Malloc(space * sizeof(Py_UCS4));
#
# 1. if isize=2^30, then space=2^30+10, so space*sizeof(Py_UCS4)=(2^30+10)*4 ==
#40 (modulo 2^32), so PyMem_Malloc allocates buffer too small to hold the
#result.
#
# Crash
# -
#
# nfd_nfkd (self=, input='...', k=1) at
/home/p/Python-3.4.1/Modules/unicodedata.c:552
# 552 stackptr = 0;
# (gdb) n
# 553 isize = PyUnicode_GET_LENGTH(input);
# (gdb) n
# 555 space = (isize > 10 ? 10 : isize) + isize;
# (gdb) n
# 556 osize = space;
# (gdb) n
# 557 output = PyMem_Malloc(space * sizeof(Py_UCS4));
# (gdb) print space
# $9 = 1073741834
# (gdb) print space*4
# $10 = 40
# (gdb) c
# Continuing.
#
# Program received signal SIGSEGV, Segmentation fault.
# 0x40579cbb in nfd_nfkd (self=, input='', k=1) at
/home/p/Python-3.4.1/Modules/unicodedata.c:614
# 614 output[o++] = code;
#
# OS info
# ---
#
# % ./python -V
# Python 3.4.1
#
# % uname -a
# Linux ubuntu 3.8.0-29-generic #42~precise1-Ubuntu SMP Wed Aug 14 15:31:16 UTC
2013 i686 i686 i386 GNU/Linux
import unicodedata as ud
s="\xa0"*(2**30)
ud.normalize("NFKC", s)
--
files: poc_unidata_normalize.py
messages: 235175
nosy: pkt
priority: normal
severity: normal
status: open
title: integer overflow in unicodedata.normalize
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file37966/poc_unidata_normalize.py
___
Python tracker
<http://bugs.python.org/issue23367>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23368] integer overflow in _PyUnicode_AsKind
New submission from paul:
# Bug
# ---
#
# void*
# _PyUnicode_AsKind(PyObject *s, unsigned int kind)
# {
# Py_ssize_t len;
# ...
# len = PyUnicode_GET_LENGTH(s);
# ...
# switch (kind) {
# ...
# case PyUnicode_4BYTE_KIND:
# 1 result = PyMem_Malloc(len * sizeof(Py_UCS4));
# ...
# else {
# assert(skind == PyUnicode_1BYTE_KIND);
# 2 _PyUnicode_CONVERT_BYTES(
# Py_UCS1, Py_UCS4,
# PyUnicode_1BYTE_DATA(s),
# PyUnicode_1BYTE_DATA(s) + len,
# result);
# }
#
# 1. len equals 2^30, so len*sizeof(Py_UCS4)=2^30*2^2=2^32, which gets casted
#down to 0, since PyMem_Malloc takes size_t as the parameter. Resulting
buffer
#is 0 bytes big.
# 2. chars from the source string s (which are 1 byte long) are expanded to 4
#bytes and copied to the 'result' buffer, which is too small to hold them
all
#
# Stack trace
# ---
#
# Breakpoint 2, _PyUnicode_AsKind (
# s='a...', kind=4) at Objects/unicodeobject.c:2176
# 2176if (PyUnicode_READY(s) == -1)
# (gdb) n
# 2179len = PyUnicode_GET_LENGTH(s);
# (gdb) n
# 2180skind = PyUnicode_KIND(s);
# (gdb) n
# 2181if (skind >= kind) {
# (gdb) n
# 2185switch (kind) {
# (gdb) n
# 2198result = PyMem_Malloc(len * sizeof(Py_UCS4));
# (gdb) print len
# $10 = 1073741824
# (gdb) print skind
# $11 = 1
# (gdb) print kind
# $12 = 4
# (gdb) print len*4
# $13 = 0
# (gdb) c
# Continuing.
#
# Program received signal SIGSEGV, Segmentation fault.
# 0x08130b56 in _PyUnicode_AsKind (
# s='a...', kind=4) at Objects/unicodeobject.c:2210
# 2210_PyUnicode_CONVERT_BYTES(
#
# OS info
# ---
#
# % ./python -V
# Python 3.4.1
#
# % uname -a
# Linux ubuntu 3.8.0-29-generic #42~precise1-Ubuntu SMP Wed Aug 14 15:31:16 UTC
2013 i686 i686 i386 GNU/Linux
#
# POC
# ---
txt=b"\x0a\x0a\x0a\x00"
uni=txt.decode("utf-32")
sub="a"*(2**30)
uni.count(sub)
--
files: poc_askind.py
messages: 235176
nosy: pkt
priority: normal
severity: normal
status: open
title: integer overflow in _PyUnicode_AsKind
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file37967/poc_askind.py
___
Python tracker
<http://bugs.python.org/issue23368>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23369] integer overflow in _json.encode_basestring_ascii
New submission from paul:
# static PyObject *
# ascii_escape_unicode(PyObject *pystr)
# {
# ...
#
# input_chars = PyUnicode_GET_LENGTH(pystr);
# input = PyUnicode_DATA(pystr);
# kind = PyUnicode_KIND(pystr);
#
# /* Compute the output size */
# for (i = 0, output_size = 2; i < input_chars; i++) {
# Py_UCS4 c = PyUnicode_READ(kind, input, i);
# if (S_CHAR(c))
# output_size++;
# else {
# switch(c) {
# ...
# default:
# 1 output_size += c >= 0x1 ? 12 : 6;
# ...
#
# 2 rval = PyUnicode_New(output_size, 127);
#
# 1. if c is \u then output_size += 6. There are no overflow checks on this
#variable, so we can overflow it with a sufficiently long (2**32/6+1 chars)
#string
# 2. rval buffer is too small to hold the result
#
# Crash:
# --
#
# Breakpoint 3, ascii_escape_unicode (pystr='...') at
/home/p/Python-3.4.1/Modules/_json.c:198
# 198 rval = PyUnicode_New(output_size, 127);
# (gdb) print output_size
# $9 = 4
# (gdb) c
# Continuing.
#
# Program received signal SIGSEGV, Segmentation fault.
# 0x4057888f in ascii_escape_unichar (c=65535,
# output=0x40572358 "...",
# chars=19624) at /home/p/Python-3.4.1/Modules/_json.c:155
# 155 output[chars++] = Py_hexdigits[(c >> 8) & 0xf];
#
# OS info
# ---
#
# % ./python -V
# Python 3.4.1
#
# % uname -a
# Linux ubuntu 3.8.0-29-generic #42~precise1-Ubuntu SMP Wed Aug 14 15:31:16 UTC
2013 i686 i686 i386 GNU/Linux
#
from _json import encode_basestring_ascii as enc
s="\u"*int((2**32)/6+1)
enc(s)
--
files: poc_ascii_escape.py
messages: 235177
nosy: pkt
priority: normal
severity: normal
status: open
title: integer overflow in _json.encode_basestring_ascii
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file37968/poc_ascii_escape.py
___
Python tracker
<http://bugs.python.org/issue23369>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23364] integer overflow in itertools.product
paul added the comment: Why do you think this test needs 16GiB? -- ___ Python tracker <http://bugs.python.org/issue23364> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23364] integer overflow in itertools.product
paul added the comment: You mean 64bit? On 32 it'll overflow and that's the point. -- ___ Python tracker <http://bugs.python.org/issue23364> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23361] integer overflow in winapi_createprocess
paul added the comment: ping -- ___ Python tracker <http://bugs.python.org/issue23361> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23361] integer overflow in winapi_createprocess
paul added the comment: ping -- ___ Python tracker <http://bugs.python.org/issue23361> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23490] allocation (and overwrite) of a 0 byte buffer
New submission from paul:
# Bug
# ---
#
# Py_UNICODE *
# PyUnicode_AsUnicodeAndSize(PyObject *unicode, Py_ssize_t *size)
# {
# ...
# #endif
# wchar_t *w;
# wchar_t *wchar_end;
#
# ...
# 1 _PyUnicode_WSTR(unicode) = (wchar_t *)
PyObject_MALLOC(sizeof(wchar_t) *
# (_PyUnicode_LENGTH(unicode)
+ 1));
# ...
# w = _PyUnicode_WSTR(unicode);
# 2 wchar_end = w + _PyUnicode_LENGTH(unicode);
#
# if (PyUnicode_KIND(unicode) == PyUnicode_1BYTE_KIND) {
# one_byte = PyUnicode_1BYTE_DATA(unicode);
# 3 for (; w < wchar_end; ++one_byte, ++w)
# *w = *one_byte;
# /* null-terminate the wstr */
# 4 *w = 0;
# }
#
# 1. if length(unicode)==2**30-1, then malloced buffer has size equal to
#4*(2^30-1+1)=2^32 == 0 (modulo 2^32)
# 2. wchar_end is equal to w-4 because of pointer arithmetic (nonexplicit
#multiplication by 4)
# 3. w > wchar_end, so we don't enter the loop
# 4. 4 byte write to a 0 size buffer
#
# GDB output
# --
#
# 3860_PyUnicode_WSTR(unicode) = (wchar_t *)
PyObject_MALLOC(sizeof(wchar_t) *
# ...
# (gdb) print sizeof(wchar_t)*(((PyASCIIObject*)(unicode))->length+1)
# $21 = 0
# ...
# (gdb) n
# 3868w = _PyUnicode_WSTR(unicode);
# (gdb) n
# 3869wchar_end = w + _PyUnicode_LENGTH(unicode);
# (gdb) n
# 3871if (PyUnicode_KIND(unicode) == PyUnicode_1BYTE_KIND) {
# (gdb) print w
# $22 = 0x805fc028 L"\xfbfbfbfb\xced0"
# (gdb) print wchar_end
# $23 = 0x805fc024 L"\xfbfbfb6f\xfbfbfbfb\xced0"
# ...
# 3876*w = 0;
#
# )
# OS info
# ---
#
# % ./python -V
# Python 3.4.1
#
# % uname -a
# Linux ubuntu 3.8.0-29-generic #42~precise1-Ubuntu SMP Wed Aug 14 15:31:16 UTC
2013 i686 i686 i386 GNU/Linux
import locale
s='a'*(2**30-1)
locale.strxfrm(s)
--
files: poc_strxfrm.py
messages: 236275
nosy: pkt
priority: normal
severity: normal
status: open
title: allocation (and overwrite) of a 0 byte buffer
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file38186/poc_strxfrm.py
___
Python tracker
<http://bugs.python.org/issue23490>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23490] allocation (and overwrite) of a 0 byte buffer
paul added the comment: And a nice error: Debug memory block at address p=0x805fc028: API 'o' 0 bytes originally requested The 3 pad bytes at p-3 are FORBIDDENBYTE, as expected. The 4 pad bytes at tail=0x805fc028 are not all FORBIDDENBYTE (0xfb): at tail+0: 0x00 *** OUCH at tail+1: 0x00 *** OUCH at tail+2: 0x00 *** OUCH at tail+3: 0x00 *** OUCH The block was made by call #53454 to debug malloc/realloc. Fatal Python error: bad trailing pad byte -- ___ Python tracker <http://bugs.python.org/issue23490> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24407] Use after free in PyDict_merge
New submission from paul:
# PyDict_Merge:
#
# 1 for (i = 0, n = DK_SIZE(other->ma_keys); i < n; i++) {
# ...
# 3 entry = &other->ma_keys->dk_entries[i];
# ...
# 2 if (insertdict(mp, entry->me_key,
#entry->me_hash,
#value) != 0)
# return -1;
# ...
# }
#
# 1. n is set once
# 2. it's possible to run a custom __eq__ method from inside the insertdict.
#__eq__ clears the "other" dict. "n" variables is now out of date
# 3. out of bounds read
#
# CRASH:
# --
#
# * thread #1: tid = 27715, 0x080d1c1d python`insertdict(mp=0xb71d66f4,
key=0x61682044, hash=543582496, value=0xb71d6664) + 132 at dictobject.c:819,
name = 'python', stop reason = invalid address (fault address: 0x61682050)
# frame #0: 0x080d1c1d python`insertdict(mp=0xb71d66f4, key=0x61682044,
hash=543582496, value=0xb71d6664) + 132 at dictobject.c:819
#816 if (ep == NULL) {
#817 return -1;
#818 }
# -> 819 assert(PyUnicode_CheckExact(key) || mp->ma_keys->dk_lookup ==
lookdict);
#820 Py_INCREF(value);
#821 MAINTAIN_TRACKING(mp, key, value);
#822 old_value = *value_addr;
#
--
files: dict_merge.py
messages: 245001
nosy: pkt
priority: normal
severity: normal
status: open
title: Use after free in PyDict_merge
type: crash
versions: Python 3.5
Added file: http://bugs.python.org/file39659/dict_merge.py
___
Python tracker
<http://bugs.python.org/issue24407>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24098] Multiple use after frees in obj2ast_* methods
paul added the comment: ping -- ___ Python tracker <http://bugs.python.org/issue24098> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24407] Use after free in PyDict_merge
paul added the comment: ping -- ___ Python tracker <http://bugs.python.org/issue24407> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24103] Use after free in xmlparser_setevents (1)
paul added the comment: ping -- ___ Python tracker <http://bugs.python.org/issue24103> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24104] Use after free in xmlparser_setevents (2)
paul added the comment: ping -- ___ Python tracker <http://bugs.python.org/issue24104> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24097] Use after free in PyObject_GetState
paul added the comment: ping -- ___ Python tracker <http://bugs.python.org/issue24097> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24407] Use after free in PyDict_merge
paul added the comment: ping -- ___ Python tracker <http://bugs.python.org/issue24407> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24103] Use after free in xmlparser_setevents (1)
paul added the comment: ping -- ___ Python tracker <http://bugs.python.org/issue24103> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24104] Use after free in xmlparser_setevents (2)
paul added the comment: ping -- ___ Python tracker <http://bugs.python.org/issue24104> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24098] Multiple use after frees in obj2ast_* methods
paul added the comment: ping -- ___ Python tracker <http://bugs.python.org/issue24098> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24407] Use after free in PyDict_merge
paul added the comment: ping -- ___ Python tracker <http://bugs.python.org/issue24407> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24683] Type confusion in json encoding
New submission from paul: on-35dm-i386-linux-gnu.so`encoder_listencode_list(s=0xb6f90394, acc=0xbfc42c28, seq=0xb6f2361c, indent_level=1) + 655 at _json.c:1800 # frame #2: 0xb6e4366d _json.cpython-35dm-i386-linux-gnu.so`encoder_listencode_obj(s=0xb6f90394, acc=0xbfc42c28, obj=0xb6f2361c, indent_level=1) + 733 at _json.c:1554 # frame #3: 0xb6e3fc4f _json.cpython-35dm-i386-linux-gnu.so`encoder_call(self=0xb6f90394, args=0xb7049304, kwds=0x) + 319 at _json.c:1386 # frame #4: 0x080c5758 python`PyObject_Call(func=0xb6f90394, arg=0xb7049304, kw=0x) + 264 at abstract.c:2149 # # This is a type confusion bug. encoder->markers can be initialized to an # arbitrary object (string in this POC). PyDict_Contains trusts the caller that # "op" is a dictionary without checking. Some callers can't be trusted :) -- messages: 247093 nosy: pkt priority: normal severity: normal status: open title: Type confusion in json encoding type: crash versions: Python 3.5 ___ Python tracker <http://bugs.python.org/issue24683> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24684] Type confusion in socket module
New submission from paul: eck(idna)); # (gdb) # # Program received signal SIGABRT, Aborted. # 0xb77a6d4c in __kernel_vsyscall () # # "host" argument can be set to a subclass of unicode with a custom "encode" # method. "encode" returns unexpected type. assert is not compiled in release # mode, so this will lead to a type confusion later on. -- files: poc_getaddr.py messages: 247094 nosy: pkt priority: normal severity: normal status: open title: Type confusion in socket module type: crash versions: Python 3.5 Added file: http://bugs.python.org/file39974/poc_getaddr.py ___ Python tracker <http://bugs.python.org/issue24684> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24683] Type confusion in json encoding
paul added the comment: Sorry, I uploaded a test case. -- ___ Python tracker <http://bugs.python.org/issue24683> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24683] Type confusion in json encoding
Changes by paul : Added file: http://bugs.python.org/file39975/json_markers.py ___ Python tracker <http://bugs.python.org/issue24683> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24684] socket.getaddrinfo(host) doesn't ensure that host.encode() returns a byte string
paul added the comment: @haypo: I'd be happy to implement all my fuzzer ideas if my bugs were patched in a timely manner. At this moment I have multiple bugs submitted over 2 months ago, which still aren't patched. Without patches, hackerone won't accept these issues, so my incentive to work on python is removed. -- ___ Python tracker <http://bugs.python.org/issue24684> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24683] Type confusion in json encoding
paul added the comment: resolution: not a bug ^ because of private API? -- ___ Python tracker <http://bugs.python.org/issue24683> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25143] 3.5 install fails poorly on Windows XP
New submission from Paul: Running the installer on an unsupported system (such as Windows XP)should fail gracefully and not just leave the user hanging. https://mail.python.org/pipermail/python-list/2015-September/696789.html -- components: Installation messages: 250852 nosy: pwat...@phs.org priority: normal severity: normal status: open title: 3.5 install fails poorly on Windows XP type: behavior versions: Python 3.5 ___ Python tracker <http://bugs.python.org/issue25143> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25143] 3.5 install fails poorly on Windows XP
Paul added the comment: Installing on an unsupported platform should fail gracefully. https://mail.python.org/pipermail/python-list/2015-September/696789.html -- ___ Python tracker <http://bugs.python.org/issue25143> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24683] Type confusion in json encoding
paul added the comment: Proof of EIP control. -- Added file: http://bugs.python.org/file41719/eip.py ___ Python tracker <http://bugs.python.org/issue24683> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24683] Type confusion in json encoding
paul added the comment: GDB dump of running ./python eip.py ___ eax:37A317DD ebx:B7A54268 ecx:BFFFE22C edx:11223344 eflags:00010217 esi:B7A61060 edi:B7AA6714 esp:BFFFE20C ebp:B7A317DC eip:11223344 cs:0073 ds:007B es:007B fs: gs:0033 ss:007Bo d I t s z A P C [007B:BFFFE20C]-[stack] BFFFE23C : 10 FA A1 B7 60 10 A6 B7 - 68 42 A5 B7 00 60 A2 B7 `...hB...`.. BFFFE22C : 60 17 A6 B7 10 68 2B 08 - 00 60 A2 B7 DC 17 A3 B7 `h+..`.. BFFFE21C : 2C E2 FF BF DC 17 A3 B7 - 3C E2 FF BF 00 00 00 00 ,...<... BFFFE20C : AE 07 0D 08 60 10 A6 B7 - 68 42 A5 B7 DD 17 A3 37 `...hB.7 [0073:11223344]-[ code] => 0x11223344: Error while running hook_stop: Cannot access memory at address 0x11223344 0x11223344 in ?? () -- ___ Python tracker <http://bugs.python.org/issue24683> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24683] Type confusion in json encoding
paul added the comment: Can you try on 2.7 branch? -- ___ Python tracker <http://bugs.python.org/issue24683> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24683] Type confusion in json encoding
paul added the comment: Sorry, I wasn't clear enough. This POC is a proof that the original bug can be used for EIP control. I just checked and it works as advertised on 2.7 revision: https://hg.python.org/cpython/rev/2d39777f3477 - it's a parent of https://hg.python.org/cpython/rev/0a1266ef1b5d containing the patch for this issue. I added this file, because I submitted a bug on hackerone claiming EIP control. -- ___ Python tracker <http://bugs.python.org/issue24683> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24091] Use after free in Element.extend (1)
New submission from paul:
# 1055for (i = 0; i < seqlen; i++) {
# (gdb) n
# 1056PyObject* element = PySequence_Fast_GET_ITEM(seq, i);
# (gdb) n
# 1057if (!PyObject_IsInstance(element, (PyObject *)&Element_Type))
{
# (gdb) print *element
# $19 = {_ob_next = 0x4060e6fc, _ob_prev = 0x4056cd8c, ob_refcnt = 1, ob_type =
0x406de3e4}
# (gdb) n
# 1066if (element_add_subelement(self, element) < 0) {
# (gdb) print *element
# $20 = {_ob_next = 0xdbdbdbdb, _ob_prev = 0xdbdbdbdb, ob_refcnt = -606348325,
ob_type = 0xdbdbdbdb}
#
# Fatal Python error: /home/p/Python-3.4.1/Modules/_elementtree.c:267 object at
0x4056c4cc has negative ref count -606348326
#
# "element" is removed in __getattribute__ method.
--
files: poc_elt_extend1.py
messages: 242305
nosy: pkt
priority: normal
severity: normal
status: open
title: Use after free in Element.extend (1)
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file39240/poc_elt_extend1.py
___
Python tracker
<http://bugs.python.org/issue24091>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24092] Use after free in Element.extend (2)
New submission from paul:
# Program received signal SIGSEGV, Segmentation fault.
# 0x4063cf19 in element_extend (self=0x405ddf74, args=([],)) at
/home/p/Python-3.4.1/Modules/_elementtree.c:1056
# 1056PyObject* element = PySequence_Fast_GET_ITEM(seq, i);
# (gdb) print i
# $3 = 1337
# (gdb) print *(PyListObject*)seq
# $4 = {ob_base = {ob_base = {_ob_next = 0x406373ec, _ob_prev = 0x405ddf74,
ob_refcnt = 3, ob_type = 0x830e1c0 },
# ob_size = 0}, ob_item = 0x0, allocated = 0}
#
# Controlled read (resulting from a use after free). "seq" is cleared in a
custom
# destructor (Y.__del__()).
#
--
files: poc_elt_extend2.py
messages: 242306
nosy: pkt
priority: normal
severity: normal
status: open
title: Use after free in Element.extend (2)
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file39241/poc_elt_extend2.py
___
Python tracker
<http://bugs.python.org/issue24092>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24093] Use after free in Element.remove
New submission from paul:
# Program received signal SIGABRT, Aborted.
# 0x40022424 in __kernel_vsyscall ()
# (gdb) bt
# #0 0x40022424 in __kernel_vsyscall ()
# #1 0x400bb1df in raise () from /lib/i386-linux-gnu/libc.so.6
# #2 0x400be825 in abort () from /lib/i386-linux-gnu/libc.so.6
# #3 0x08067030 in Py_FatalError (
# msg=0xbfed7a20 "/home/p/Python-3.4.1/Modules/_elementtree.c:1436 object
at 0x405743ec has negative ref count -606348326")
# at Python/pythonrun.c:2633
# #4 0x080f1374 in _Py_NegativeRefcount (fname=0x40646100
"/home/p/Python-3.4.1/Modules/_elementtree.c", lineno=1436,
# op=) at Objects/object.c:203
# #5 0x4063dfa6 in element_remove (self=0x40583c34,
args=(,))
# at /home/p/Python-3.4.1/Modules/_elementtree.c:1436
# (gdb) frame 5
# #5 0x4063dfa6 in element_remove (self=0x40583c34,
args=(,))
# at /home/p/Python-3.4.1/Modules/_elementtree.c:1436
# 1436Py_DECREF(self->extra->children[i]);
# (gdb) print i
# $1 = 1
# (gdb) print *(PyObject*)self->extra->children
# $3 = {_ob_next = 0x4057437c, _ob_prev = 0x405743ec, ob_refcnt = 1079461180,
ob_type = 0x4057461c}
#
# Fatal Python error: /home/p/Python-3.4.1/Modules/_elementtree.c:1436 object
at 0x405743ec has negative ref count -606348326
#
# "self->extra->children" is cleared in custom __eq__ method. Py_DECREF handles
# stale pointer. Use after free.
#
--
files: poc_elt_remove.py
messages: 242307
nosy: pkt
priority: normal
severity: normal
status: open
title: Use after free in Element.remove
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file39242/poc_elt_remove.py
___
Python tracker
<http://bugs.python.org/issue24093>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24094] Use after free during json encoding (PyType_IsSubtype)
New submission from paul:
# Breakpoint 1, encoder_listencode_dict (s=0x405b23fc, acc=0xbfc4038c, dct=, indent_level=0)
# at /home/p/Python-3.4.1/Modules/_json.c:1540
# 1540items = PyMapping_Keys(dct);
# (gdb) n
# 1541if (items == NULL)
# (gdb) print *items
# $1 = {_ob_next = 0x405c8af4, _ob_prev = 0x4059006c, ob_refcnt = 2, ob_type =
0x830e1c0 }
# (gdb) n
# 1543if (!PyList_Check(items)) {
# (gdb) n
# 1547if (PyList_Sort(items) < 0)
# (gdb) n
# 1549nitems = PyList_GET_SIZE(items);
# (gdb) n
# 1550for (i = 0; i < nitems; i++) {
# (gdb) print nitems
# $2 = 1122
# (gdb) n
# 1552key = PyList_GET_ITEM(items, i);
# (gdb) n
# 1553value = PyDict_GetItem(dct, key);
# (gdb) print *key
# $3 = {_ob_next = 0x4058eedc, _ob_prev = 0x40590d1c, ob_refcnt = 1, ob_type =
0x405afd1c}
# (gdb) n
#
# Program received signal SIGSEGV, Segmentation fault.
# 0x08108825 in PyType_IsSubtype (a=0xdbdbdbdb, b=0x830f1a0 ) at
Objects/typeobject.c:1292
# 1292mro = a->tp_mro;
# (gdb) bt
# #0 0x08108825 in PyType_IsSubtype (a=0xdbdbdbdb, b=0x830f1a0 )
at Objects/typeobject.c:1292
# #1 0x080f22d6 in do_richcompare (v=1337, w=,
op=2) at Objects/object.c:643
# #2 0x080f263d in PyObject_RichCompare (v=1337, w=, op=2) at Objects/object.c:701
# #3 0x080f26ce in PyObject_RichCompareBool (v=1337, w=, op=2) at Objects/object.c:723
# #4 0x080df7b5 in lookdict (mp=0x405c8b34, key=, hash=1337, value_addr=0xbfc40200)
# at Objects/dictobject.c:485
# #5 0x080e145d in PyDict_GetItem (op=, key=) at Objects/dictobject.c:1095
# #6 0x405bf6f9 in encoder_listencode_dict (s=0x405b23fc, acc=0xbfc4038c,
dct=, indent_level=0)
#
# Deleting the object in __hash__() method triggers an use after free in
PyType_IsSubtype.
--
files: poc_enc_dict1.py
messages: 242308
nosy: pkt
priority: normal
severity: normal
status: open
title: Use after free during json encoding (PyType_IsSubtype)
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file39243/poc_enc_dict1.py
___
Python tracker
<http://bugs.python.org/issue24094>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24095] Use after free during json encoding a dict (2)
New submission from paul:
# Breakpoint 1, encoder_listencode_dict (s=0x405b23fc, acc=0xbfaf96ec, dct=, indent_level=0)
# at /home/p/Python-3.4.1/Modules/_json.c:1540
# 1540items = PyMapping_Keys(dct);
# (gdb) print *items
# $1 = {_ob_next = 0x4059029c, _ob_prev = 0x405c8ab4, ob_refcnt = 1, ob_type =
0x830f1a0 }
# (gdb) n
# 1541if (items == NULL)
# (gdb) n
# 1543if (!PyList_Check(items)) {
# (gdb) n
# 1547if (PyList_Sort(items) < 0)
# (gdb) n
# 1549nitems = PyList_GET_SIZE(items);
# (gdb) n
# 1550for (i = 0; i < nitems; i++) {
# (gdb) n
# 1552key = PyList_GET_ITEM(items, i);
# (gdb) n
# 1553value = PyDict_GetItem(dct, key);
# (gdb) n
# 1554item = PyTuple_Pack(2, key, value);
# (gdb) print *key
# $2 = {_ob_next = 0xdbdbdbdb, _ob_prev = 0xdbdbdbdb, ob_refcnt = -606348325,
ob_type = 0xdbdbdbdb}
# (gdb) n
#
# Program received signal SIGSEGV, Segmentation fault.
# 0x08104047 in PyTuple_Pack (n=2) at Objects/tupleobject.c:216
# 216 Py_INCREF(o);
#
# We circumvent use after free bug in PyType_IsSubtype (poc_enc_dict1.py) by
# returning -1 from the __hash__() method. This way PyDict_GetItem bails
# quickly, without triggering the problematic code.
# PyTuple_Pack handles a stale "key" pointer and crashes. Use after free.
--
files: poc_enc_dict2.py
messages: 242309
nosy: pkt
priority: normal
severity: normal
status: open
title: Use after free during json encoding a dict (2)
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file39244/poc_enc_dict2.py
___
Python tracker
<http://bugs.python.org/issue24095>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24096] Use after free during json encoding a dict (3)
New submission from paul:
# Program received signal SIGSEGV, Segmentation fault.
# 0x40036740 in encoder_listencode_dict (s=0x405b43fc, acc=0xbf86438c, dct=, indent_level=0)
# at /home/p/Python-3.4.1/Modules/_json.c:1557
# 1557PyList_SET_ITEM(items, i, item);
# (gdb) print *(PyListObject*)items
# $1 = {ob_base = {ob_base = {_ob_next = 0x405bcab4, _ob_prev = 0x40591184,
ob_refcnt = 2, ob_type = 0x830e1c0 },
# ob_size = 0}, ob_item = 0x0, allocated = 0}
# (gdb) print i
# $2 = 112233
#
# "items" was cleared in __hash__, so we get a wild write at a controlled
address.
--
files: poc_enc_dict3.py
messages: 242311
nosy: pkt
priority: normal
severity: normal
status: open
title: Use after free during json encoding a dict (3)
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file39246/poc_enc_dict3.py
___
Python tracker
<http://bugs.python.org/issue24096>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24096] Use after free in get_filter
paul added the comment:
# Program received signal SIGSEGV, Segmentation fault.
# 0x080f2c17 in PyObject_GetAttr (v=,
name='match') at Objects/object.c:872
# 872 if (tp->tp_getattro != NULL)
# (gdb) bt
# #0 0x080f2c17 in PyObject_GetAttr (v=,
name='match') at Objects/object.c:872
# #1 0x080f2b42 in _PyObject_GetAttrId (v=,
name=0x8328354 ) at Objects/object.c:835
# #2 0x0809c3a6 in _PyObject_CallMethodId (o=,
name=0x8328354 , format=0x829552c "O")
# at Objects/abstract.c:2215
# #3 0x0817e48b in check_matched (obj=, arg='c')
at Python/_warnings.c:28
# #4 0x0817e88b in get_filter (category=, text='',
lineno=4, module='c', item=0xbfa87c88)
# (gdb) frame 4
# #4 0x0817e88b in get_filter (category=, text='',
lineno=4, module='c', item=0xbfa87c88)
# at Python/_warnings.c:152
# 152 good_mod = check_matched(mod, module);
# (gdb) print *mod
# $1 = {_ob_next = 0xdbdbdbdb, _ob_prev = 0xdbdbdbdb, ob_refcnt = -606348325,
ob_type = 0xdbdbdbdb}
#
# "mod" object is deleted in "match" method. Use after free.
#
--
title: Use after free during json encoding a dict (3) -> Use after free in
get_filter
Added file: http://bugs.python.org/file39247/poc_get_filter.py
___
Python tracker
<http://bugs.python.org/issue24096>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24097] Use after free in PyObject_GetState
New submission from paul:
# Program received signal SIGSEGV, Segmentation fault.
# 0x080f27b2 in PyObject_Hash (v=) at
Objects/object.c:746
# 746 if (tp->tp_hash != NULL)
# (gdb) bt
# #0 0x080f27b2 in PyObject_Hash (v=) at
Objects/object.c:746
# #1 0x080e1717 in PyDict_SetItem (op={}, key=,
value=None) at Objects/dictobject.c:1201
# #2 0x0810e8a0 in _PyObject_GetState (obj=) at
Objects/typeobject.c:3657
# #3 0x081100e9 in reduce_2 (obj=) at
Objects/typeobject.c:3949
# #4 0x08110551 in _common_reduce (self=, proto=2) at
Objects/typeobject.c:4012
# #5 0x08110641 in object_reduce (self=, args=(2,)) at
Objects/typeobject.c:4032
#
# (gdb) frame 2
# #2 0x0810e8a0 in _PyObject_GetState (obj=) at
Objects/typeobject.c:3657
# 3657int err = PyDict_SetItem(slots, name, value);
# (gdb) print *name
# $1 = {_ob_next = 0xdbdbdbdb, _ob_prev = 0xdbdbdbdb, ob_refcnt = -606348325,
ob_type = 0xdbdbdbdb}
#
# "name" is freed inside __getattr__ and is later used by PyDict_SetItem.
--
files: poc_getstate.py
messages: 242313
nosy: pkt
priority: normal
severity: normal
status: open
title: Use after free in PyObject_GetState
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file39248/poc_getstate.py
___
Python tracker
<http://bugs.python.org/issue24097>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24098] Multiple use after frees in obj2ast_* methods
New submission from paul:
# 3617for (i = 0; i < len; i++) {
# (gdb) print *(PyListObject*)tmp
# $1 = {ob_base = {ob_base = {_ob_next = 0x4056f8f4, _ob_prev = 0x4057329c,
ob_refcnt = 2, ob_type = 0x830e1c0 },
# ob_size = 1337}, ob_item = 0x8491ae0, allocated = 1432}
# (gdb) n
# 3619res = obj2ast_stmt(PyList_GET_ITEM(tmp, i), &value,
arena);
# (gdb) n
# 3620if (res != 0) goto failed;
# (gdb) print *(PyListObject*)tmp
# $2 = {ob_base = {ob_base = {_ob_next = 0x4056f8f4, _ob_prev = 0x4057329c,
ob_refcnt = 2, ob_type = 0x830e1c0 },
# ob_size = 1}, ob_item = 0x8491ae0, allocated = 4}
# (gdb) c
# Continuing.
#
# Program received signal SIGSEGV, Segmentation fault.
# 0x080f2c17 in PyObject_GetAttr (v=,
name='lineno') at Objects/object.c:872
# 872 if (tp->tp_getattro != NULL)
#
# Objects freed in __getattr__ are used later in the loop above. There are two
# bugs actually. One is the use-after-free and the second is using a stale size
# variable "len" to control the for(...) loop. "body" can be mutated inside
# obj2ast_stmt.
This construct:
for (i = 0; i < len; i++) {
stmt_ty value;
res = obj2ast_stmt(PyList_GET_ITEM(tmp, i), &value, arena);
if (res != 0) goto failed;
asdl_seq_SET(body, i, value);
}
is repeated multiple times in multiple obj2ast_ methods. It contains two bugs:
1. tmp[i] isn't protected from deletion inside python code (refcnt is not
increased by GET_ITEM),
2. tmp's length can drop below "len" resulting in an OOB read, because the loop
counter is static.
--
files: poc_obj2mod.py
messages: 242315
nosy: pkt
priority: normal
severity: normal
status: open
title: Multiple use after frees in obj2ast_* methods
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file39249/poc_obj2mod.py
___
Python tracker
<http://bugs.python.org/issue24098>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24100] Use after free in siftdown (2)
New submission from paul:
# _siftdown(PyListObject *heap, Py_ssize_t startpos, Py_ssize_t pos)
# ...
# while (pos > startpos){
# parentpos = (pos - 1) >> 1;
# parent = PyList_GET_ITEM(heap, parentpos);
# 1 cmp = PyObject_RichCompareBool(newitem, parent, Py_LT);
# ...
# 2 if (size != PyList_GET_SIZE(heap)) {
# Py_DECREF(newitem);
# PyErr_SetString(PyExc_RuntimeError,
# "list changed size during iteration");
# return -1;
# }
# if (cmp == 0)
# 3 break;
# ...
# }
# 4 Py_DECREF(PyList_GET_ITEM(heap, pos));
# 5 PyList_SET_ITEM(heap, pos, newitem);
#
# 1. custom compare function replaces object at index "pos" with a fresh
#instance with refcnt==1
# 2. check is ineffective, since mutation was done without altering size
# 3. break out of the loop
# 4. refcnt drops to 0 and __del__ method is called. Destructed clears the heap
# 5. SET_ITEM doesn't do any bounds checking and does a wild write.
#
# "pos" is under our control and is restricted only by the amount of free
# memory. pos==X requires heap of size X-1.
#
# gX global var is necessary. Without it, python crashes in debug checks inside
# Py_ForgetReference. Seems like clearing L puts objects in a bad state.
#
# GDB
# ---
# Program received signal SIGSEGV, Segmentation fault.
# 0x4002ed73 in _siftdown (heap=0x4058edfc, startpos=0, pos=112233) at
/home/p/Python-3.4.1/Modules/_heapqmodule.c:58
# 58 PyList_SET_ITEM(heap, pos, newitem);
# (gdb) print *heap
# $1 = {ob_base = {ob_base = {_ob_next = 0x405913f4, _ob_prev = 0x4058ee6c,
ob_refcnt = 2, ob_type = 0x830e1c0 },
# ob_size = 0}, ob_item = 0x0, allocated = 0}
# (gdb) print pos
# $2 = 112233
--
files: poc_siftdown2.py
messages: 242317
nosy: pkt
priority: normal
severity: normal
status: open
title: Use after free in siftdown (2)
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file39251/poc_siftdown2.py
___
Python tracker
<http://bugs.python.org/issue24100>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24101] Use after free in siftup
New submission from paul:
# Breakpoint 1, _siftup (heap=0x4056b344, pos=65534) at
/home/p/Python-3.4.1/Modules/_heapqmodule.c:121
# warning: Source file is more recent than executable.
# 121 Py_DECREF(PyList_GET_ITEM(heap, pos));
# (gdb) print *heap->ob_item[pos]
# $1 = {_ob_next = 0x41812058, _ob_prev = 0x831159c , ob_refcnt = 1,
ob_type = 0x4058fd1c}
# (gdb) n
# 122 PyList_SET_ITEM(heap, pos, newitem);
# (gdb) print *heap->ob_item[pos]
# Cannot access memory at address 0x3fff8
# (gdb) print *heap
# $2 = {ob_base = {ob_base = {_ob_next = 0x4059c0b4, _ob_prev = 0x405903b4,
ob_refcnt = 2, ob_type = 0x830e1c0 },
# ob_size = 0}, ob_item = 0x0, allocated = 0}
# (gdb) n
#
# Program received signal SIGSEGV, Segmentation fault.
# 0x4002f150 in _siftup (heap=0x4056b344, pos=65534) at
/home/p/Python-3.4.1/Modules/_heapqmodule.c:122
# 122 PyList_SET_ITEM(heap, pos, newitem);
--
files: poc_siftup.py
messages: 242318
nosy: pkt
priority: normal
severity: normal
status: open
title: Use after free in siftup
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file39252/poc_siftup.py
___
Python tracker
<http://bugs.python.org/issue24101>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24102] Multiple type confusions in unicode error handlers
New submission from paul:
# Breakpoint 1, PyUnicodeEncodeError_GetEnd (exc=,
end=0xbf9e8f7c) at Objects/exceptions.c:1643
# 1643PyObject *obj = get_unicode(((PyUnicodeErrorObject *)exc)->object,
# (gdb) s
# get_unicode (attr=, name=0x82765ea "object") at
Objects/exceptions.c:1516
# 1516if (!attr) {
# (gdb) print *attr
# $4 = {_ob_next = 0xfefefefe, _ob_prev = 0xfefefefe, ob_refcnt = -16843010,
ob_type = 0xfefefefe}
# (gdb) c
# Continuing.
#
# Program received signal SIGSEGV, Segmentation fault.
# 0x080bc7d9 in get_unicode (attr=, name=0x82765ea
"object") at Objects/exceptions.c:1521
# 1521if (!PyUnicode_Check(attr)) {
#
# Type confusion. IsInstance check is ineffective because of custom
# __getattribute__ method. Contents of string instance is interpreted as
# an exception object.
--
files: poc_unicode_errors.py
messages: 242319
nosy: pkt
priority: normal
severity: normal
status: open
title: Multiple type confusions in unicode error handlers
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file39253/poc_unicode_errors.py
___
Python tracker
<http://bugs.python.org/issue24102>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24103] Use after free in xmlparser_setevents (1)
New submission from paul:
# xmlparser_setevents(XMLParserObject *self, PyObject* args)
# {
# ...
# /* clear out existing events */
# Py_CLEAR(target->start_event_obj);
# 1 Py_CLEAR(target->end_event_obj);
# Py_CLEAR(target->start_ns_event_obj);
# Py_CLEAR(target->end_ns_event_obj);
#
# ...
#
# seqlen = PySequence_Size(events_seq);
# for (i = 0; i < seqlen; ++i) {
# 3 PyObject *event_name_obj = PySequence_Fast_GET_ITEM(events_seq, i);
# ...
#
# if (event_name == NULL) {
# ...
# return NULL;
# } else if (strcmp(event_name, "start") == 0) {
# ...
# } else if (strcmp(event_name, "end") == 0) {
# Py_INCREF(event_name_obj);
# 2 Py_XDECREF(target->end_event_obj);
# target->end_event_obj = event_name_obj;
# }
# ...
# }
# ...
# }
#
# This one leverages nested _setevents invocations. First invocation sets
# target->end_event_obj to S1 instance. On seconds invocation,
# target->end_event_obj has refcnt==1, so DECREF at line 1 triggers
S1.__del__().
# Destructor invokes _setevents again and sets target->end_event_obj to a S3
# instance (with refcnt==1). After we return from nested call at line 1,
# execution continues until it hits an "end" element. At line 2 S3.__del__() is
# called and it deallocates "events_seq". This triggers a controlled OOB (we can
# call it a use after free too) read at line 3. We can control a PyObject
pointer.
#
# Program received signal SIGSEGV, Segmentation fault.
# 0x4068563b in xmlparser_setevents (self=0x40669e4c, args=([], [])) at
/home/p/Python-3.4.1/Modules/_elementtree.c:3560
# 3560PyObject *event_name_obj =
PySequence_Fast_GET_ITEM(events_seq, i);
# (gdb) print i
# $1 = 1337
# (gdb) print *(PyListObject*)events_seq
# $2 = {ob_base = {ob_base = {_ob_next = 0x40669df4, _ob_prev = 0x4055f814,
ob_refcnt = 3, ob_type = 0x830e1c0 },
# ob_size = 0}, ob_item = 0x0, allocated = 0}
#
--
files: poc_xml_setevents1.py
messages: 242320
nosy: pkt
priority: normal
severity: normal
status: open
title: Use after free in xmlparser_setevents (1)
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file39254/poc_xml_setevents1.py
___
Python tracker
<http://bugs.python.org/issue24103>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24104] Use after free in xmlparser_setevents (2)
New submission from paul:
# Program received signal SIGSEGV, Segmentation fault.
# 0x4068565c in xmlparser_setevents (self=0x4064b13c, args=([], []))
# at /home/p/Python-3.4.1/Modules/_elementtree.c:3562
# 3562if (PyUnicode_Check(event_name_obj)) {
# (gdb) print *event_name_obj
# $6 = {_ob_next = 0xdbdbdbdb, _ob_prev = 0xdbdbdbdb, ob_refcnt = -606348325,
ob_type = 0xdbdbdbdb}
#
#
# "event_name_obj" is deleted inside a custom destructor. Use after free.
--
files: poc_xml_setevents2.py
messages: 242321
nosy: pkt
priority: normal
severity: normal
status: open
title: Use after free in xmlparser_setevents (2)
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file39255/poc_xml_setevents2.py
___
Python tracker
<http://bugs.python.org/issue24104>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24099] Use after free in siftdown (1)
New submission from paul:
# _siftdown(PyListObject *heap, Py_ssize_t startpos, Py_ssize_t pos)
# ...
# newitem = PyList_GET_ITEM(heap, pos);
# Py_INCREF(newitem);
# /* Follow the path to the root, moving parents down until finding
#a place newitem fits. */
# while (pos > startpos){
# parentpos = (pos - 1) >> 1;
# 1 parent = PyList_GET_ITEM(heap, parentpos);
# 2 cmp = PyObject_RichCompareBool(newitem, parent, Py_LT);
# if (cmp == -1) {
# Py_DECREF(newitem);
# return -1;
# }
# 3 if (size != PyList_GET_SIZE(heap)) {
# Py_DECREF(newitem);
# PyErr_SetString(PyExc_RuntimeError,
# "list changed size during iteration");
# return -1;
# }
# if (cmp == 0)
# break;
# 4 Py_INCREF(parent);
# ...
#
# 1. parent isn't protected (refcnt==1)
# 2. custom compare function deletes all objects in "heap" and repopulates it
with
#fresh instances. "parent" is freed
# 3. check is ineffective. Heap was mutated while preserving its size
# 4. use after free. Crash will manifest itself later.
--
files: poc_siftdown1.py
messages: 242316
nosy: pkt
priority: normal
severity: normal
status: open
title: Use after free in siftdown (1)
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file39250/poc_siftdown1.py
___
Python tracker
<http://bugs.python.org/issue24099>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24105] Use after free during json encoding a dict (3)
New submission from paul:
# Program received signal SIGSEGV, Segmentation fault.
# 0x40036740 in encoder_listencode_dict (s=0x405b43fc, acc=0xbf86438c, dct=, indent_level=0)
# at /home/p/Python-3.4.1/Modules/_json.c:1557
# 1557PyList_SET_ITEM(items, i, item);
# (gdb) print *(PyListObject*)items
# $1 = {ob_base = {ob_base = {_ob_next = 0x405bcab4, _ob_prev = 0x40591184,
ob_refcnt = 2, ob_type = 0x830e1c0 },
# ob_size = 0}, ob_item = 0x0, allocated = 0}
# (gdb) print i
# $2 = 112233
#
# "items" was cleared in __hash__, so we get a wild write at a controlled
address.
--
files: poc_enc_dict3.py
messages: 242322
nosy: pkt
priority: normal
severity: normal
status: open
title: Use after free during json encoding a dict (3)
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file39256/poc_enc_dict3.py
___
Python tracker
<http://bugs.python.org/issue24105>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24096] Use after free in get_filter
Changes by paul : Removed file: http://bugs.python.org/file39246/poc_enc_dict3.py ___ Python tracker <http://bugs.python.org/issue24096> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24096] Use after free in get_filter
paul added the comment: Issue for poc_enc_dict3.py is here: https://bugs.python.org/issue24105 Please ignore first and third message. -- ___ Python tracker <http://bugs.python.org/issue24096> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24105] Use after free during json encoding a dict (3)
paul added the comment: You want multiple bugs in the same module grouped in one issue? -- ___ Python tracker <http://bugs.python.org/issue24105> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
