[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords
Eugene Kapun added the comment: Actually, this can't be fixed without modifying C API methods PyArg_ParseTuple and PyArg_ParseTupleAndKeywords, because it's possible to make an object deallocated before PyArg_ParseTuple returns, so Py_INCREF immediately after parsing would be already too late. Here are my test cases: test-resource.py - in Modules/resource.c, and python-bug-01.patch won't work against it. test-ctypes.py - in Modules/_ctypes/_ctypes.c. test-functools.py - in Modules/_functoolsmodule.c (py3k only). -- components: +Interpreter Core -Extension Modules nosy: +abacabadabacaba title: Reference counting bug in setrlimit -> Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords Added file: http://bugs.python.org/file19616/test-resource.py ___ Python tracker <http://bugs.python.org/issue6083> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords
Changes by Eugene Kapun : Added file: http://bugs.python.org/file19617/test-ctypes.py ___ Python tracker <http://bugs.python.org/issue6083> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords
Changes by Eugene Kapun : Added file: http://bugs.python.org/file19618/test-functools.py ___ Python tracker <http://bugs.python.org/issue6083> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8401] Strange behavior of bytearray slice assignment
New submission from Eugene Kapun :
>>> a = bytearray()
>>> a[:] = 0 # Is it a feature?
>>> a
bytearray(b'')
>>> a[:] = 10 # If so, why not document it?
>>> a
bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')
>>> a[:] = -1
Traceback (most recent call last):
File "", line 1, in
ValueError: negative count
>>> a[:] = -10 # This should raise ValueError, not
>>> TypeError.
Traceback (most recent call last):
File "", line 1, in
TypeError: 'int' object is not iterable
>>> a[:] = 100
Traceback (most recent call last):
File "", line 1, in
MemoryError
>>> a[:] = 10 # This should raise OverflowError, not
>>> TypeError.
Traceback (most recent call last):
File "", line 1, in
TypeError: 'int' object is not iterable
>>> a[:] = [] # Are some empty sequences better than others?
>>> a[:] = ()
>>> a[:] = list("")
>>> a[:] = ""
Traceback (most recent call last):
File "", line 1, in
TypeError: string argument without an encoding
--
components: Interpreter Core
messages: 103133
nosy: abacabadabacaba
severity: normal
status: open
title: Strange behavior of bytearray slice assignment
type: behavior
versions: Python 3.1
___
Python tracker
<http://bugs.python.org/issue8401>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8401] Strange behavior of bytearray slice assignment
Eugene Kapun added the comment: Empty string is an iterable of integers in the range 0 <= x < 256, so it should be allowed. >>> all(isinstance(x, int) and 0 <= x < 256 for x in "") True >>> bytearray()[:] = "" Traceback (most recent call last): File "", line 1, in TypeError: string argument without an encoding -- ___ Python tracker <http://bugs.python.org/issue8401> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8417] bytes and bytearray constructors raise TypeError for very large ints
New submission from Eugene Kapun : >>> bytes(10) b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' >>> bytes(10 ** 100) Traceback (most recent call last): File "", line 1, in TypeError: 'int' object is not iterable -- components: Interpreter Core messages: 103300 nosy: abacabadabacaba severity: normal status: open title: bytes and bytearray constructors raise TypeError for very large ints type: behavior versions: Python 3.1 ___ Python tracker <http://bugs.python.org/issue8417> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1766304] improve xrange.__contains__
Changes by Eugene Kapun : -- nosy: +abacabadabacaba ___ Python tracker <http://bugs.python.org/issue1766304> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8401] Strange behavior of bytearray slice assignment
Eugene Kapun added the comment: > Not really, chars are not ints Yes, however, empty string contains exactly zero chars. > and anyway the empty string fall in the first case. Strings aren't mentioned in documentation of bytearray slice assignment. However, I think that bytearray constructor should accept empty string too, without an encoding, for consistency. -- ___ Python tracker <http://bugs.python.org/issue8401> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8401] Strange behavior of bytearray slice assignment
Eugene Kapun added the comment: __doc__ of bytearray says: > bytearray(iterable_of_ints) -> bytearray > bytearray(string, encoding[, errors]) -> bytearray > bytearray(bytes_or_bytearray) -> mutable copy of bytes_or_bytearray > bytearray(memory_view) -> bytearray So, unless an encoding is given, empty string should be interpreted as an iterable of ints. BTW, documentation and docstring should be made consistent with each other. -- ___ Python tracker <http://bugs.python.org/issue8401> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8420] set_lookkey is unsafe
New submission from Eugene Kapun :
I've noticed that set_lookkey (in Objects/setobject.c) does some unsafe things:
Objects/setobject.c:
> if (entry->hash == hash) {
> startkey = entry->key;
> Py_INCREF(startkey);
> cmp = PyObject_RichCompareBool(startkey, key, Py_EQ);
> Py_DECREF(startkey);
At this point, object pointed to by startkey could be deallocated, and then new
object may be allocated at the same address.
> if (cmp < 0)
> return NULL;
> if (table == so->table && entry->key == startkey) {
At this point, the table may be reallocated at the same address but with
different (possibly smaller) size, so entry->key may be in deallocated memory.
Also, entry->key may be equal to startkey but still point to an object other
than one key was compared with.
> if (cmp > 0)
> return entry;
> }
> else {
> /* The compare did major nasty stuff to the
>* set: start over.
>*/
> return set_lookkey(so, key, hash);
This can lead to infinite recursion.
> }
--
components: Interpreter Core
messages: 10
nosy: abacabadabacaba
severity: normal
status: open
title: set_lookkey is unsafe
versions: Python 3.1
___
Python tracker
<http://bugs.python.org/issue8420>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8425] a -= b should be fast if a is a small set and b is a large set
New submission from Eugene Kapun : >>> small_set = set(range(2000)) >>> large_set = set(range(2000)) >>> large_set -= small_set # Fast >>> small_set -= large_set # Slow, should be fast >>> small_set = small_set - large_set # Fast -- components: Interpreter Core messages: 103343 nosy: abacabadabacaba severity: normal status: open title: a -= b should be fast if a is a small set and b is a large set type: resource usage versions: Python 3.1 ___ Python tracker <http://bugs.python.org/issue8425> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8401] Strange behavior of bytearray slice assignment
Eugene Kapun added the comment:
-1 on special-casing string without an encoding. Current code does (almost)
this:
...
if argument_is_a_string:
if not encoding_is_given: # Special case
raise TypeError("string argument without an encoding")
encode_argument()
return
if encoding_is_given:
raise TypeError("encoding or errors without a string argument")
...
IMO, it should do this instead:
...
if encoding_is_given:
if not argument_is_a_string:
raise TypeError("encoding or errors without a string argument")
encode_argument()
return
...
This way, bytearray("") would work without any special cases.
--
___
Python tracker
<http://bugs.python.org/issue8401>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8420] Objects/setobject.c contains unsafe code
Eugene Kapun added the comment:
I've found more unsafe code in Objects/setobject.c.
This code makes Python 3.1.2 segfault by using a bug in function set_merge:
class bad:
def __eq__(self, other):
if be_bad:
set2.clear()
raise Exception
return self is other
def __hash__(self):
return 0
be_bad = False
set1 = {bad()}
set2 = {bad() for i in range(2000)}
be_bad = True
set1.update(set2)
Function set_symmetric_difference_update has a similar bug.
Another bug in set_symmetric_difference_update:
class bad:
def __init__(self):
print("Creating", id(self))
def __del__(self):
print("Deleting", id(self))
def __eq__(self, other):
print("Comparing", id(self), "and", id(other))
if be_bad:
dict2.clear()
return self is other
def __hash__(self):
return 0
be_bad = False
set1 = {bad()}
dict2 = {bad(): None}
be_bad = True
set1.symmetric_difference_update(dict2)
--
title: set_lookkey is unsafe -> Objects/setobject.c contains unsafe code
___
Python tracker
<http://bugs.python.org/issue8420>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8435] It is possible to observe a mutating frozenset
New submission from Eugene Kapun :
This code shows that frozensets aren't really immutable. The same frozenset is
printed twice, with different content. Buggy functions are set_contains,
set_remove and set_discard, all in Objects/setobject.c
class bad:
def __eq__(self, other):
global f2
f2 = other
print_f2()
s1.add("querty")
return self is other
def __hash__(self):
return hash(f1)
def print_f2():
print(id(f2), repr(f2))
f1 = frozenset((1, 2, 3))
s1 = set(f1)
s1 in {bad()}
print_f2()
--
components: Interpreter Core
messages: 103426
nosy: abacabadabacaba
severity: normal
status: open
title: It is possible to observe a mutating frozenset
type: behavior
versions: Python 3.1
___
Python tracker
<http://bugs.python.org/issue8435>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8420] Objects/setobject.c contains unsafe code
Changes by Eugene Kapun : -- type: -> crash ___ Python tracker <http://bugs.python.org/issue8420> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8436] set.__init__ accepts keyword args
New submission from Eugene Kapun : >>> list().__init__(a=0) Traceback (most recent call last): File "", line 1, in TypeError: 'a' is an invalid keyword argument for this function >>> set().__init__(a=0) -- components: Interpreter Core messages: 103427 nosy: abacabadabacaba severity: normal status: open title: set.__init__ accepts keyword args type: behavior versions: Python 3.1 ___ Python tracker <http://bugs.python.org/issue8436> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8420] Objects/setobject.c contains unsafe code
Eugene Kapun added the comment:
This patch still assumes that if so->table didn't change then the table wasn't
reallocated (see http://en.wikipedia.org/wiki/ABA_problem). One solution is to
check that so->mask didn't change as well. Also, checking that refcnt > 1 is
redundant because if entry->key == startkey then there are at least two
references: one from entry->key and another from startkey.
These functions have a bug that may cause them to refer to deallocated memory
when both arguments are sets: set_intersection, set_isdisjoint,
set_difference_update_internal, set_difference,
set_symmetric_difference_update, set_issubset.
These functions may also do the same if the first argument is a set and the
second argument is a dict: set_difference, set_symmetric_difference_update.
Bugs in set_repr:
> keys = PySequence_List((PyObject *)so);
> if (keys == NULL)
> goto done;
>
> listrepr = PyObject_Repr(keys);
> Py_DECREF(keys);
List pointed to by keys is already deallocated at this point.
> if (listrepr == NULL) {
> Py_DECREF(keys);
But this code tries to DECREF it.
> goto done;
> }
> newsize = PyUnicode_GET_SIZE(listrepr);
> result = PyUnicode_FromUnicode(NULL, newsize);
> if (result) {
> u = PyUnicode_AS_UNICODE(result);
> *u++ = '{';
> /* Omit the brackets from the listrepr */
> Py_UNICODE_COPY(u, PyUnicode_AS_UNICODE(listrepr)+1,
> PyUnicode_GET_SIZE(listrepr)-2);
> u += newsize-2;
> *u++ = '}';
> }
> Py_DECREF(listrepr);
> if (Py_TYPE(so) != &PySet_Type) {
result may be NULL here.
> PyObject *tmp = PyUnicode_FromFormat("%s(%U)",
>Py_TYPE(so)->tp_name,
>result);
I think PyUnicode_FromFormat won't like it.
> Py_DECREF(result);
> result = tmp;
> }
--
___
Python tracker
<http://bugs.python.org/issue8420>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8420] Objects/setobject.c contains unsafe code
Eugene Kapun added the comment: > > One solution is to check that so->mask didn't > > change as well. > > I saw that and agree it would make a tighter check, but haven't convinced > myself that it is necessary. Without this check, it is possible that the comparison shrinks the table, so entry becomes out of bounds. However, if both so->table and so->mask didn't change then entry is still a pointer to one of table elements so it can be used safely. > > Also, checking that refcnt > 1 is redundant > > because if entry->key == startkey then there > > are at least two references: one from entry->key > > and another from startkey. > > It is a meaningful check. We have our own INCREF > and one for the key being in the table. If the > count is 1, then it means that the comparison > check deleted the key from the table or replaced > its value (see issue 1517). If the comparison deleted or changed the key then the check entry->key == startkey would fail so refcnt check won't be reached. Checking refcounts is also bad because someone else may have references to the key. > I don't follow why you think keys is already deallocated. > When assigned by PySequence_List() without a NULL return, the refcnt is one. > The call to PyObject_Repr(keys) does not change the refcnt of keys, > so the Py_DECREF(keys) is correct. Look at the code again. If listrepr happens to be NULL, you do Py_DECREF(keys) twice (this bug is only present in py3k branch). -- ___ Python tracker <http://bugs.python.org/issue8420> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8420] Objects/setobject.c contains unsafe code
Eugene Kapun added the comment: This code crashes python by using another bug in set_repr. This only affects py3k. This code relies on out-of-memory condition, so run it like: $ (ulimit -v 65536 && python3 test.py) Otherwise, it will eat all your free memory before crashing. val = "a" * 1 class big: def __repr__(self): return val i = 16 while True: repr(frozenset(big() for j in range(i))) i = (i * 5) >> 2 -- ___ Python tracker <http://bugs.python.org/issue8420> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8420] Objects/setobject.c contains unsafe code
Changes by Eugene Kapun : Added file: http://bugs.python.org/file16981/repr.diff ___ Python tracker <http://bugs.python.org/issue8420> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
