[issue1230540] sys.excepthook doesn't work in threads
Changes by Nikolaus Rath : -- nosy: +Nikratio ___ Python tracker <http://bugs.python.org/issue1230540> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12960] threading.Condition is not a class
Nikolaus Rath added the comment: On 09/13/2011 07:41 PM, STINNER Victor wrote: > > STINNER Victor added the comment: > >> According to >> http://docs.python.org/library/threading.html#condition-objects, >> threading.Condition is a class. > > Nope, it's a factory, and it's written in the doc: > > "threading.Condition() > A factory function that returns a new condition variable object. A condition > variable allows one or more threads to wait until they are notified by > another thread. > > See Condition Objects." Yes, but further down it still says: """ class threading.Condition([lock]) If the lock argument is given and not None, [] """ Best, -Nikolaus -- ___ Python tracker <http://bugs.python.org/issue12960> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12960] threading.Condition is not a class
Nikolaus Rath added the comment: On 09/14/2011 04:29 AM, STINNER Victor wrote: > > STINNER Victor added the comment: > >> Yes, but further down it still says: >> >> """ >> class threading.Condition([lock]) >> >> If the lock argument is given and not None, [] >> """ > > What do you suggest? Replace it by class threading._Condition? I don't have an optimal solution that would fit into the prescribed layout. I think the best we can do is keep calling it class threading Condition, but mention in the very first sentence that it isn't actually a class: class threading.Condition([lock]): threading.Condition is not actually a class but a factory function. The returned instance, however, is guaranteed to have the behaviour of a threading.Condition class as described here. If the lock argument... Best, -Nikolaus -- ___ Python tracker <http://bugs.python.org/issue12960> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13585] Add contextlib.CleanupManager
New submission from Nikolaus Rath :
I'd like to propose addding the CleanupManager class described in
http://article.gmane.org/gmane.comp.python.ideas/12447 to the contextlib
module. The idea is to add a general-purpose context manager to manage (python
or non-python) resources that don't come with their own context manager.
Example code:
with CleanupManager() als mngr:
tmpdir = tempfile.mkdtemp()
mngr.register(shutil.rmtree(tmpdir))
# do stuff with tmpdir
# shutil.rmtree() get's called automatically when the block is over
Note that mkdtemp() could of course also be changed to become its own context
manager. The idea is to provide a general facility for this kind of problem, so
it doesn't have to be reinvented whenever a module provides a ressource without
its own context manager. Other possible uses are of course ressources that are
completely external to Python,
e.g. anything allocated with a subprocess (think of
subprocess.check_call('mount'))/
I'll be happy to make a proper patch with documentation and testcases from
Jan's code. As a matter of fact, I'll probably start working out it right now,
so please let me know quickly if this doesn't have a chance of getting accepted.
--
components: Library (Lib)
messages: 149268
nosy: Nikratio
priority: normal
severity: normal
status: open
title: Add contextlib.CleanupManager
type: feature request
versions: Python 3.3, Python 3.4
___
Python tracker
<http://bugs.python.org/issue13585>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13585] Add contextlib.CleanupManager
Nikolaus Rath added the comment: Here's the first part of the patch with the implementation. I'll add tests and documentation as soon as someone confirms that the idea & API is okay. -- keywords: +patch Added file: http://bugs.python.org/file23923/CleanupManager_patch_v1.diff ___ Python tracker <http://bugs.python.org/issue13585> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12704] Language References does not specify exception raised by final yield()
Nikolaus Rath added the comment: Hmm. Does the total lack responses mean agreement, disagreement or lack of interest? I'm attaching a patch against Python 3.3 in the hope of moving this forward. -- keywords: +patch Added file: http://bugs.python.org/file23924/patch_v1.diff ___ Python tracker <http://bugs.python.org/issue12704> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12704] Language Reference: Clarify behaviour of yield when generator is not resumed
Changes by Nikolaus Rath : -- title: Language References does not specify exception raised by final yield() -> Language Reference: Clarify behaviour of yield when generator is not resumed ___ Python tracker <http://bugs.python.org/issue12704> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13585] Add contextlib.CleanupManager
Nikolaus Rath added the comment: Not sure what you mean with "posted as a recipe" -- are you thinking of a specific website/mailing list? Example: which one do you mean? The one in the issue or the one in the patch? With statement: what advantages do you have in mind? Try/finally: I think the patch and the discussion in python-ideas talk about the advantage over try/finally. IMO the two most important points are: (1) avoids deep and pointless indendation for multiple ressources, (2) keeps logically connected lines (allocation+cleanup) closely together in the source instead of splitting them far apart like try/finally. error-prone: not sure if I understand you correctly. If there is an error prior to registration, the callback will not be called (that's a feature). To what kind of errors could that lead? Sorry for basically asking you to re-explain every sentence, but I honestly don't understand most of your message. -- ___ Python tracker <http://bugs.python.org/issue13585> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13585] Add contextlib.CleanupManager
Nikolaus Rath added the comment: On 12/11/2011 10:17 PM, Raymond Hettinger wrote: > > Raymond Hettinger added the comment: > > ''' > Example code: > > with CleanupManager() als mngr: > tmpdir = tempfile.mkdtemp() > mngr.register(shutil.rmtree(tmpdir)) <-- this makes the call right away > # do stuff with tmpdir > ''' Oh, of course. That is fixed in the patch. I couldn't find a way to edit the message in the tracker. > The part of my note that should be clear is that the idea and code need to > prove itself before being added to the standard library. So far, there has > been zero demand for this and I've not seen code like it being used in the > wild. AFAICT, it is not demonstrably better than a straight-forward > try/finally. I think it has the same advantages over try...finally as any use of 'with' has. CleanupManager would allow to use 'with' instead of 'try..finally' in many cases where this is currently not possible, so unless the introduction of 'with' itself was a mistake, I think this is just taking a step along the path that has already been chosen. Best, -Nikolaus -- ___ Python tracker <http://bugs.python.org/issue13585> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13585] Add contextlib.CleanupManager
Changes by Nikolaus Rath : Added file: http://bugs.python.org/file23933/CleanupManager_patch_v2.diff ___ Python tracker <http://bugs.python.org/issue13585> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13585] Add contextlib.CleanupManager
Nikolaus Rath added the comment: On 12/12/2011 03:31 PM, Sven Marnach wrote: > Adding this as a cookbook recipe first seems like a good idea. This is the second time that this is mentioned, so I would be very grateful if someone could elucidate what "adding as a cookbook recipe" means :-). Is there an official Python cookbook somewhere? Best, -Nikolaus -- ___ Python tracker <http://bugs.python.org/issue13585> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13585] Add contextlib.CleanupManager
Nikolaus Rath added the comment: On 12/21/2011 11:46 AM, Éric Araujo wrote: > > Éric Araujo added the comment: > >> The idea is to add a general-purpose context manager to manage (python >> or non-python) resources that don't come with their own context manager. > > In the passage I quoted, I don’t understand what is meant by “non-Python > resources”. Think about e.g. mounting a file system. Best, -Nikolaus -- ___ Python tracker <http://bugs.python.org/issue13585> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13585] Add contextlib.CleanupManager
Nikolaus Rath added the comment: On 12/21/2011 12:03 PM, Éric Araujo wrote: > > Éric Araujo added the comment: > >>> In the passage I quoted, I don’t understand what is meant by “non-Python >>> resources”. >> Think about e.g. mounting a file system. > > Ah, ok. In that case there would still be a Python-level object (just like a > Python file object will also release the OS-level handle when it’s closed). I don't think so. subprocess.check_call(['mount', 'bla', '/mnt']) Allocates the resource (mounts the file system) but doesn't leave you with any Python object. Best, -Nikolaus -- ___ Python tracker <http://bugs.python.org/issue13585> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5689] Support xz compression in tarfile module
Changes by Nikolaus Rath : -- nosy: -Nikratio ___ Python tracker <http://bugs.python.org/issue5689> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12704] Language References does not specify exception raised by final yield()
New submission from Nikolaus Rath : >From http://docs.python.org/reference/simple_stmts.html#the-yield-statement: "As of Python version 2.5, the yield statement is now allowed in the try clause of a try ... finally construct. If the generator is not resumed before it is finalized (by reaching a zero reference count or by being garbage collected), the generator-iterator’s close() method will be called, allowing any pending finally clauses to execute." This strongly suggests that the last-executed yield statement will raise an exception if the generator is closed. If this is the case, it would be great if the documentation could be extended to say what exception is raised. If this is not the case, it would be great if whatever magic is happening could be documented as well. -- assignee: docs@python components: Documentation messages: 141724 nosy: Nikratio, docs@python priority: normal severity: normal status: open title: Language References does not specify exception raised by final yield() type: feature request versions: Python 2.6, Python 2.7 ___ Python tracker <http://bugs.python.org/issue12704> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12704] Language References does not specify exception raised by final yield()
Nikolaus Rath added the comment: >From http://www.python.org/dev/peps/pep-0342/ I believe that the last yield >will raise GeneratorExit. So my suggestion is to replace the above mentioned >paragraph with: """ As of Python version 2.5, yield is an expression rather than a statement and allowed in the try clause of a try ... finally construct. If the generator is not resumed before it is finalized (by reaching a zero reference count or by being garbage collected), the generator-iterator’s close() method will be called, and the yield expression in the generator function will raise GeneratorExit. If the generator function raises GeneratorExit (either directly or by not catching it), future calls to the next() method of the generator iterator will raise StopIteration. GeneratorExit exceptions raised by the generator function are catched internally and do not result in a call to sys.excepthook. """ -- ___ Python tracker <http://bugs.python.org/issue12704> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1346874] httplib simply ignores CONTINUE
Changes by Nikolaus Rath : -- nosy: +Nikratio ___ Python tracker <http://bugs.python.org/issue1346874> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12960] threading.Condition is not a class
New submission from Nikolaus Rath : According to http://docs.python.org/library/threading.html#condition-objects, threading.Condition is a class. However, in fact it turns out to be function that constructs a _Condition instance. I don't know the reason for that, but it seems to me that either Condition should be a class, or the reason for it being a function should be documented so that people who I would like to inherit from Conditionknow if that is not supported, or if they can safely inherit from _Condition instead. -- assignee: docs@python components: Documentation messages: 143864 nosy: Nikratio, docs@python priority: normal severity: normal status: open title: threading.Condition is not a class type: behavior versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4 ___ Python tracker <http://bugs.python.org/issue12960> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12704] Language References does not specify exception raised by final yield()
Changes by Nikolaus Rath : -- versions: +Python 3.1, Python 3.2, Python 3.3, Python 3.4 ___ Python tracker <http://bugs.python.org/issue12704> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9606] logging filter is ignored when added to root handler
New submission from Nikolaus Rath :
I believe the following should print only one log message, not two:
import logging
class Filter(object):
def filter(self, record):
return record.name == 'foo'
logger = logging.getLogger()
formatter = logging.Formatter('%(message)s')
handler = logging.StreamHandler()
handler.setFormatter(formatter)
logger.addFilter(Filter())
logger.addHandler(handler)
logging.getLogger('foo').warn('foo!')
logging.getLogger('bar').warn('bar!')
If the filter is added to the handler instead of the logger, everything works
as expected.
If this is desired behaviour, please consider this a bug against the
documentation. In that case I propose to extend the documentation of
Logger.addFilter() as follows: "Note that the filter will act only on log
messages that are generated by this logger, and not on messages that have been
generated by descendant loggers."
--
components: Library (Lib)
messages: 113932
nosy: Nikratio
priority: normal
severity: normal
status: open
title: logging filter is ignored when added to root handler
type: behavior
versions: Python 2.6
___
Python tracker
<http://bugs.python.org/issue9606>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9606] logging filter is not applied to messages from descendant loggers
Changes by Nikolaus Rath : -- title: logging filter is ignored when added to root handler -> logging filter is not applied to messages from descendant loggers ___ Python tracker <http://bugs.python.org/issue9606> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6627] threading.local() does not work with C-created threads
Nikolaus Rath added the comment: @Swapnil: the rules you quote are correct for the C extension, but do not apply when using ctypes, because ctypes is doing the required initializations automatically. However, if Amaury is correct, ctypes performs the initializations in a way that break the threading.local functionality. I think the best way to address this bug would therefore be to add a warning to the ctypes documentation that C created threads will not support threading.local(). -- assignee: -> d...@python components: +Documentation -Library (Lib) nosy: +d...@python status: closed -> open ___ Python tracker <http://bugs.python.org/issue6627> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6627] threading.local() does not work with C-created threads
Changes by Nikolaus Rath : -- resolution: invalid -> ___ Python tracker <http://bugs.python.org/issue6627> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6627] threading.local() does not work with C-created threads
Nikolaus Rath added the comment: No, I am not saying that the behaviour of ctypes is wrong. It just happens to have some effects on threading.local that I think should be documented. That's why I reassigned this as a documentation bug. Please reconsider closing this bug. I'm also happy to change the type to "Feature request". As an aside: I think in most cases one uses ctypes to call functions that are already compiled, so changing the source is not an option. -- ___ Python tracker <http://bugs.python.org/issue6627> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6627] threading.local() does not work with C-created threads
Nikolaus Rath added the comment: To be a bit more constructive, why not add something like this in paragraph to http://docs.python.org/library/ctypes.html#callback-functions: "Note that if the callback function is called in a new thread that has been created outside of Python's control (i.e., by the foreign code that calls the callback), ctypes creates a new dummy Python thread on every invocation. That means that values stored with `threading.local` will *not* survive across different callbacks. -- ___ Python tracker <http://bugs.python.org/issue6627> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6627] threading.local() does not work with C-created threads
Nikolaus Rath added the comment: One point of ctypes is to save the user the trouble of having to create a full blown C extension, so I don't think it's reasonable to expect a ctypes user to have read the full C API documentation as well. Only a very small subset of the page that you gave is actually relevant for use with ctypes. Why not put this information where a ctypes user can find it easily? -- ___ Python tracker <http://bugs.python.org/issue6627> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10058] Unclear PyString_AsStringAndSize return value
New submission from Nikolaus Rath : http://docs.python.org/c-api/string.html says about the return value of AsStringAndSize: "If length is NULL, the resulting buffer may not contain NUL characters; if it does, the function returns -1 and a TypeError is raised." "If string is not a string object at all, PyString_AsStringAndSize() returns -1 and raises TypeError." This makes me wonder what the return code is if a) the function succeeds and b) it encounteres some other problem (i.e. out of memory when it tries to encode a unicode string into its default encoding). I guess that the return value is 0 on success and -1 on all errors, but it would be nice if the documentation would be clear about this. -- assignee: d...@python components: Documentation messages: 118294 nosy: Nikratio, d...@python priority: normal severity: normal status: open title: Unclear PyString_AsStringAndSize return value type: feature request versions: Python 2.7 ___ Python tracker <http://bugs.python.org/issue10058> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10058] C-API Intro should be more explicit about error return codes
Nikolaus Rath added the comment: Georg, this is an important piece of information, but I think it is not yet clearly stated in the documentation. Here is what http://docs.python.org/c-api/intro.html says about return codes: "In general, when a function encounters an error, it sets an exception, discards any object references that it owns, and returns an error indicator — usually NULL or -1. A few functions return a Boolean true/false result, with false indicating an error. Very few functions return no explicit error indicator or have an ambiguous return value, and require explicit testing for errors with PyErr_Occurred()." If this is meant to say that a function returns *only* -1 or NULL in case of an error, *unless documented otherwise* I suggest that the wording be changed to reflect that. Maybe like this: "In general, when a function encounters an error, it sets an exception, discards any object references that it owns, and returns either NULL or -1 (depending on what is compatible with the function's type). If a functions documentation does not say otherwise, the function can be assumed to follow this convention. Nevertheless, a few functions return a Boolean true/false result, with false indicating an error, and very few functions return no explicit error indicator or have an ambiguous return value, and require explicit testing for errors with PyErr_Occurred()." Wouldn't that be a reasonable change? -- title: Unclear PyString_AsStringAndSize return value -> C-API Intro should be more explicit about error return codes ___ Python tracker <http://bugs.python.org/issue10058> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7736] ctypes freezes/deadlocks process
New submission from Nikolaus Rath :
The following is a very subtle bug and it took me a couple of days to reduce it
to a manageable test case. So please bear with me even if it's tedious.
The problem is that in some cases the use of ctypes for a callback function
freezes the entire process. Even 'gdb -p' freezes when trying to attach to the
affected PID. The process can not be killed with SIGTERM.
Please consider the attached example. It needs the fuse headers installed to
work (package libfuse-dev under Debian).
Step 1: Compile hello_ll.c with
gcc -Wall -g3 -O0 -fPIC `pkg-config fuse --cflags` -c hello_ll.c
gcc -Wall -g3 -O0 -shared `pkg-config fuse --libs` -o hello_ll.so hello_ll.o
into a shared library. The do_mount() function will be called with ctypes from
call_hello.py.
Step 2: In call_hello.py you need to adjust the import of libfuse to match your
installed FUSE version (2.6 or 2.8). The libfuse2{6,8}.py files have been
autogenerated with ctypeslib. I can provide versions for other FUSE versions as
well if necessary. Note that the problem is most likely not with the
definitions in this file, I checked all of them carefully by hand.
Step 3: Create an empty directory 'mnt' in the working directory
Step 4: Run call_hello.py. It will mount a simple example FUSE file system in
mnt, read the directory, umount the fs and exit.
Step 5: Uncomment line 36 of call_hello.py. This will substitute the C function
hello_ll_opendir() (defined in hello_ll.c) by the Python function
fuse_opendir(). Note that both functions do exactly the same thing, they just
call the fuse function fuse_reply_err.
Step 6: Run call_hello.py again. The program will hang completely. You can
salvage the mount point with "fusermount -u -z mnt".
Step 7: Comment out line 47 in call_hello.py, so that the process itself does
not try to access the mountpoint anymore.
Step 8: Run call_hello.py again. Observe that the file system mounts and
umounts fine. Accessing the mountpoint with a different process does not
trigger the freeze either.
In summary, the process hangs only if:
1) The opendir function is implemented as a ctypes callback
and
2) the mounting process itself tries to access the mountpoint
I suspect that the ctypes callback may somehow deadlock with the thread that
tries to access the mountpoint.
If someone can tell me how to get around the hanging gdb process, i'll be happy
to provide a backtrace. I have already installed debugging symbols for both
Python and libfuse.
Thank you for taking the time to read up to this point :-)
--
assignee: theller
components: ctypes
messages: 98038
nosy: Nikratio, theller
severity: normal
status: open
title: ctypes freezes/deadlocks process
type: crash
versions: Python 2.6
___
Python tracker
<http://bugs.python.org/issue7736>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7736] ctypes freezes/deadlocks process
Changes by Nikolaus Rath : Added file: http://bugs.python.org/file15943/call_hello.py ___ Python tracker <http://bugs.python.org/issue7736> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7736] ctypes freezes/deadlocks process
Changes by Nikolaus Rath : Added file: http://bugs.python.org/file15944/hello_ll.c ___ Python tracker <http://bugs.python.org/issue7736> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7736] ctypes freezes/deadlocks process
Changes by Nikolaus Rath : Added file: http://bugs.python.org/file15945/libfuse26.py ___ Python tracker <http://bugs.python.org/issue7736> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7736] ctypes freezes/deadlocks process
Changes by Nikolaus Rath : Added file: http://bugs.python.org/file15946/libfuse28.py ___ Python tracker <http://bugs.python.org/issue7736> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7736] ctypes freezes/deadlocks process
Changes by Nikolaus Rath : Added file: http://bugs.python.org/file15947/call_hello.py ___ Python tracker <http://bugs.python.org/issue7736> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7736] ctypes freezes/deadlocks process
Changes by Nikolaus Rath : Removed file: http://bugs.python.org/file15943/call_hello.py ___ Python tracker <http://bugs.python.org/issue7736> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7736] ctypes freezes/deadlocks process
Nikolaus Rath added the comment: Reproduced with Python 3.1 -- versions: +Python 3.1 ___ Python tracker <http://bugs.python.org/issue7736> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7736] ctypes freezes/deadlocks process
Changes by Nikolaus Rath : Removed file: http://bugs.python.org/file15947/call_hello.py ___ Python tracker <http://bugs.python.org/issue7736> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7736] ctypes freezes/deadlocks process
Changes by Nikolaus Rath : Added file: http://bugs.python.org/file15948/call_hello.py ___ Python tracker <http://bugs.python.org/issue7736> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7736] ctypes freezes/deadlocks process
Nikolaus Rath added the comment: Wow, great! Thanks for looking into this. How did you manage to get the backtrace? As I said, on my system gdb -p just hangs with when attaching. I'm changing the component, since it seems that it's the os module that's at fault and not ctypes. -- components: +Library (Lib) -ctypes versions: +Python 2.7 ___ Python tracker <http://bugs.python.org/issue7736> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7736] os.listdir hangs since opendir() and closedir() do not release GIL
Changes by Nikolaus Rath : -- title: ctypes freezes/deadlocks process -> os.listdir hangs since opendir() and closedir() do not release GIL ___ Python tracker <http://bugs.python.org/issue7736> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7736] os.listdir hangs since opendir() and closedir() do not release GIL
Nikolaus Rath added the comment: In this simple example, FUSE does not fork and does not start any threads. Note that PyGILState_Ensure() cannot do anything here. What happens is this: - call_hello.py calls FUSE in a new thread, releasing the GIL. - FUSE mounts the file system and waits for requests - Meanwhile, in the main thread, call_hello.py calls opendir(), but does not release the GIL - FUSE receives the opendir() system call and calls the appropriate callback function - If the callback function is implemented in C, everything works fine. - If the callback function is implemented in Python, the FUSE thread tries to acquire the GIL to call the (Python) opendir() handler. But it cannot do so, because the GIL is still held by the main thread (which is waiting for the opendir syscall to return) => deadlock. -- ___ Python tracker <http://bugs.python.org/issue7736> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7736] os.listdir hangs since opendir() and closedir() do not release GIL
Nikolaus Rath added the comment: I have used both of them in the past, but in the end I wrote my own bindings (currently only available as part of the http://code.google.com/p/s3ql source code, but I intend to factor it out at some point), since neither fuse.py (http://code.google.com/p/fusepy) nor python-fuse (fuse.sf.net) support the FUSE low level API. That said, I am reasonably sure that in both of them are affected by this bug too. However, it may not show up in practive very often, because the high level FUSE API by default forks into a background process before mounting the file system. -- ___ Python tracker <http://bugs.python.org/issue7736> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7736] os.listdir hangs since opendir() and closedir() do not release GIL
Nikolaus Rath added the comment: On 01/20/2010 07:19 AM, Antoine Pitrou wrote: > Ah, thanks for the explanation. Yes indeed the patch looks ok for the > job. You should just be aware that similar problems may appear with > other system calls. I don't think we have ever considered that common C > calls such as opendir() could call back into Python code through > libraries such as FUSE. Well, now that I know what to look for, tracking down more of these problems should be significantly faster and easier. Are you generally going to accept similar patches for other unprotected syscalls? Still, I'd be extremly grateful if someone could tell me the trick how to create a backtrace in such a deadlock situation... Or am I the only one for which a simple "gdb -a" does not work? Best, -Nikolaus -- title: os.listdir hangs since opendir() and closedir() do not release GIL -> os.listdir hangs since opendir() and closedir() do not release GIL ___ Python tracker <http://bugs.python.org/issue7736> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7736] os.listdir hangs since opendir() and closedir() do not release GIL
Nikolaus Rath added the comment: The patch works fine for me too. Also, I did not discover any other such problems for other syscalls (but I did not systematically try all os.* functions). -- ___ Python tracker <http://bugs.python.org/issue7736> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7760] use_errno=True does not work
New submission from Nikolaus Rath : On my system (Ubuntu Karmic, Python 2.6.4 (r264:75706, Dec 7 2009, 18:45:15), Kernel 2.6.31-17-generic, libc6 2.10.1-0ubuntu16) the attached test script produces the following output: Traceback (most recent call last): File "test1.py", line 27, in raise OSError(err, os.strerror(err), path) OSError: [Errno 0] Success: '/does/not/exist' So the function call failed, but the errno provided by ctypes is zero. (There are two variants of construction the getxattr() foreign function in the script and both produce the same result). -- assignee: theller components: ctypes files: test1.py messages: 98171 nosy: Nikratio, theller severity: normal status: open title: use_errno=True does not work type: behavior versions: Python 2.6, Python 3.1 Added file: http://bugs.python.org/file15976/test1.py ___ Python tracker <http://bugs.python.org/issue7760> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7760] use_errno=True does not work
Nikolaus Rath added the comment: I can confirm that without the path it works for me too. But I have to admit that I don't really understand your explanation. Should I generally not use full paths with CDLL? Or just in the case of libc? In either case, I think the ctypes dokumentation could be more explicit about this. -- ___ Python tracker <http://bugs.python.org/issue7760> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7931] Python hangs after last thread has exited
Nikolaus Rath added the comment: Here is a short testcase that reproduces the problem. If you run the script it daemonizes correctly, but then the daemon does not terminate but hangs forever. The crucial part seems to be to daemonize the program while there is more than one thread running. -- Added file: http://bugs.python.org/file16228/test.py ___ Python tracker <http://bugs.python.org/issue7931> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7931] Interpreter does not terminate if daemonized while running multithreaded
Changes by Nikolaus Rath : -- title: Python hangs after last thread has exited -> Interpreter does not terminate if daemonized while running multithreaded ___ Python tracker <http://bugs.python.org/issue7931> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7736] os.listdir hangs since opendir() and closedir() do not release GIL
Nikolaus Rath added the comment: Does this patch still need review? Both Martin and Antoine already commented that the patch is ok, so it'd be great if someone could actually apply it... -- ___ Python tracker <http://bugs.python.org/issue7736> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Changes by Nikolaus Rath : -- nosy: +Nikratio ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8487] os.mknod() fails on NFS mounted directories
New submission from Nikolaus Rath :
$ cat test.py
#!/usr/bin/env python
import os
import stat
dbfile = './testfile.test'
with open(dbfile, 'w') as fh:
print('Opened file for writing')
os.unlink(dbfile)
os.mknod(dbfile, stat.S_IRUSR | stat.S_IWUSR | stat.S_IFREG)
print('Mknod\'ed file')
[cl...@ih ~]$ cd tmp <-- nfs mounted on a 64bit Fedora box
[cl...@ih tmp]$ ~/tmp/test.py
Opened file for writing
Traceback (most recent call last):
File "/home/cliff/tmp/test.py", line 9, in
os.mknod(dbfile, stat.S_IRUSR | stat.S_IWUSR | stat.S_IFREG)
OSError: [Errno 2] No such file or directory
[cl...@ih tmp]$ cd /tmp <-- locally mounted on a HD
[cl...@ih tmp]$ ~/tmp/test.py
Opened file for writing
Mknod'ed file
I think the mknod() call really shouldn't fail if it tries to create an
ordinary file that can be created with open() with problems.
--
components: IO
messages: 103860
nosy: Nikratio
severity: normal
status: open
title: os.mknod() fails on NFS mounted directories
type: behavior
versions: Python 2.6
___
Python tracker
<http://bugs.python.org/issue8487>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17681] Work with an extra field of gzip and zip files
Change by Nikolaus Rath : -- nosy: -nikratio ___ Python tracker <https://bugs.python.org/issue17681> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34624] -W option and PYTHONWARNINGS env variable does not accept module regexes
Change by Nikolaus Rath : -- nosy: -nikratio ___ Python tracker <https://bugs.python.org/issue34624> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17852] Built-in module _io can lose data from buffered files in reference cycles
Change by Nikolaus Rath : -- nosy: -nikratio ___ Python tracker <https://bugs.python.org/issue17852> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35561] Valgrind reports Syscall param epoll_ctl(event) points to uninitialised byte(s)
Nikolaus Rath added the comment: Is there a good reason to not explicitly initialize them nevertheless? Valgrind is a common tool, and this false positive will affect everyone attempting to run it on a Python extension module. -- ___ Python tracker <https://bugs.python.org/issue35561> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31122] SSLContext.wrap_socket() throws OSError with errno == 0
New submission from Nikolaus Rath: With a particularly atrocious network connection, I often get the following exception: File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 503, in connect self._sock = self.ssl_context.wrap_socket(self._sock, server_hostname=server_hostname) File "/usr/lib/python3.5/ssl.py", line 385, in wrap_socket _context=self) File "/usr/lib/python3.5/ssl.py", line 760, in __init__ self.do_handshake() File "/usr/lib/python3.5/ssl.py", line 996, in do_handshake self._sslobj.do_handshake() File "/usr/lib/python3.5/ssl.py", line 641, in do_handshake self._sslobj.do_handshake() OSError: [Errno 0] Error I don't think an error with errno == 0 should ever be raised by Python. -- assignee: christian.heimes components: SSL messages: 299759 nosy: christian.heimes, nikratio priority: normal severity: normal status: open title: SSLContext.wrap_socket() throws OSError with errno == 0 type: behavior versions: Python 3.5 ___ Python tracker <http://bugs.python.org/issue31122> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31122] SSLContext.wrap_socket() throws OSError with errno == 0
Nikolaus Rath added the comment: Regarding "atrocious connection": I wish I knew, but I have no control of the connection. All I can tell is that there are frequent disconnects, occasional latency spikes, my remote ip address seems to change frequently (while the apparent local one stays as-is, so presumably some NAT in between), temporary bandwidth drops, etc. -- ___ Python tracker <http://bugs.python.org/issue31122> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35561] Valgrind reports Syscall param epoll_ctl(event) points to uninitialised byte(s)
New submission from Nikolaus Rath : With current git master, configured --with-valgrind --with-pydebug, I get: ==31074== Command: /home/nikratio/clones/cpython/python /home/nikratio/in-progress/pyfuse3/test/../examples/hello.py /tmp/pytest-of-nikratio/pytest-11/test_hello_hello_py_0 ==31074== ==31074== Syscall param epoll_ctl(event) points to uninitialised byte(s) ==31074==at 0x584906A: epoll_ctl (syscall-template.S:84) ==31074==by 0xBDAA493: pyepoll_internal_ctl (selectmodule.c:1392) ==31074==by 0xBDAA59F: select_epoll_register_impl (selectmodule.c:1438) ==31074==by 0xBDAA5F8: select_epoll_register (selectmodule.c.h:599) ==31074==by 0x174E15: _PyMethodDef_RawFastCallKeywords (call.c:658) ==31074==by 0x300BCA: _PyMethodDescr_FastCallKeywords (descrobject.c:290) ==31074==by 0x21FC05: call_function (ceval.c:4611) ==31074==by 0x22B5E7: _PyEval_EvalFrameDefault (ceval.c:3183) ==31074==by 0x2206FF: PyEval_EvalFrameEx (ceval.c:533) ==31074==by 0x173B61: function_code_fastcall (call.c:285) ==31074==by 0x174737: _PyFunction_FastCallKeywords (call.c:410) ==31074==by 0x21FDF4: call_function (ceval.c:4634) ==31074== Address 0xffeffeb4c is on thread 1's stack ==31074== in frame #1, created by pyepoll_internal_ctl (selectmodule.c:1379) To reproduce: $ python-dev -m pip install --user pyfuse3 # for dependencies $ git clone https://github.com/libfuse/pyfuse3.git $ valgrind --trace-children=yes "--trace-children-skip=*mount*" python-dev -m pytest test/ pyfuse3 provides a C extension module, but I believe the problem is in the interpreter core as the stacktrace does not include anything from the extension. -- components: Interpreter Core messages: 332348 nosy: nikratio priority: normal severity: normal status: open title: Valgrind reports Syscall param epoll_ctl(event) points to uninitialised byte(s) type: compile error versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue35561> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35561] Valgrind reports Syscall param epoll_ctl(event) points to uninitialised byte(s)
Nikolaus Rath added the comment: Same error with 3.7.1 and 3.6.7 (though line numbers differ slightly). -- ___ Python tracker <https://bugs.python.org/issue35561> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35561] Valgrind reports Syscall param epoll_ctl(event) points to uninitialised byte(s)
Nikolaus Rath added the comment: $ ./configure CFLAGS="-O0 -g" --with-valgrind && make -j8 still gives ==13281== Memcheck, a memory error detector ==13281== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al. ==13281== Using Valgrind-3.12.0.SVN and LibVEX; rerun with -h for copyright info ==13281== Command: /home/nikratio/clones/cpython/python /home/nikratio/in-progress/pyfuse3/test/../examples/hello.py /tmp/pytest-of-nikratio/pytest-15/test_hello_hello_py_0 ==13281== ==13281== Syscall param epoll_ctl(event) points to uninitialised byte(s) ==13281==at 0x584906A: epoll_ctl (syscall-template.S:84) ==13281==by 0xB5C07FA: pyepoll_internal_ctl (selectmodule.c:1392) ==13281==by 0xB5C08CB: select_epoll_register_impl (selectmodule.c:1438) ==13281==by 0xB5C112A: select_epoll_register (selectmodule.c.h:599) ==13281==by 0x173031: _PyMethodDef_RawFastCallKeywords (call.c:658) ==13281==by 0x2FEFCD: _PyMethodDescr_FastCallKeywords (descrobject.c:290) ==13281==by 0x21ED25: call_function (ceval.c:4611) ==13281==by 0x21AB4E: _PyEval_EvalFrameDefault (ceval.c:3183) ==13281==by 0x21007C: PyEval_EvalFrameEx (ceval.c:533) ==13281==by 0x17245F: function_code_fastcall (call.c:285) ==13281==by 0x1728B5: _PyFunction_FastCallKeywords (call.c:410) ==13281==by 0x21EDF4: call_function (ceval.c:4634) ==13281== Address 0xffeff2d94 is on thread 1's stack ==13281== in frame #1, created by pyepoll_internal_ctl (selectmodule.c:1379) -- ___ Python tracker <https://bugs.python.org/issue35561> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20177] Derby #8: Convert 28 sites to Argument Clinic across 2 files
Nikolaus Rath added the comment: Sorry, no. I have long lost context and interest in this. -- ___ Python tracker <https://bugs.python.org/issue20177> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17852] Built-in module _io can lose data from buffered files at exit
Nikolaus Rath added the comment: Given the apparent difficulties getting this right, how about not attempting to implicitly flush buffers in the finalizer at all? This means scripts relying on this will break, but in contrast to the current behavior they will break consistently so it's easy to find and fix the problem. -- ___ Python tracker <https://bugs.python.org/issue17852> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1230540] sys.excepthook doesn't work in threads
Change by Nikolaus Rath : -- nosy: -nikratio ___ Python tracker <https://bugs.python.org/issue1230540> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34920] PYTHONWARNINGS entries are escaped
New submission from Nikolaus Rath : According to https://docs.python.org/3/library/warnings.html#describing-warning-filters: "The meaning of each of these fields [of PYTHONWARNINGS] is as described in ." The description of the "The Warnings filter" says "module is a string containing a regular expression that the module name must match." However, when parsing PYTHONWARNINGS, the warnings module explicitly escapes the module field. -- components: Library (Lib) messages: 327272 nosy: nikratio priority: normal severity: normal status: open title: PYTHONWARNINGS entries are escaped type: behavior versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue34920> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34920] PYTHONWARNINGS entries are escaped
Nikolaus Rath added the comment: yes, this is a duplicate. -- ___ Python tracker <https://bugs.python.org/issue34920> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34624] -W option and PYTHONWARNINGS env variable does not accept module regexes
Change by Nikolaus Rath : -- nosy: +nikratio title: -W option does not accept module regexes -> -W option and PYTHONWARNINGS env variable does not accept module regexes ___ Python tracker <https://bugs.python.org/issue34624> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18116] getpass.getpass() triggers ResourceWarning
New submission from Nikolaus Rath:
[0] nikratio@vostro:~/tmp$ cat bugme.py
#!python
import getpass
import warnings
warnings.simplefilter('default')
getpass.getpass("What's up?")
[0] nikratio@vostro:~/tmp$ python3 --version
Python 3.3.2
[0] nikratio@vostro:~/tmp$ python3 bugme.py
/usr/lib/python3.3/os.py:1043: ResourceWarning: unclosed file <_io.FileIO
name=3 mode='rb+'>
return io.open(fd, *args, **kwargs)
What's up?
--
components: Library (Lib)
messages: 190458
nosy: Nikratio
priority: normal
severity: normal
status: open
title: getpass.getpass() triggers ResourceWarning
type: resource usage
versions: Python 3.3
___
Python tracker
<http://bugs.python.org/issue18116>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18116] getpass.getpass() triggers ResourceWarning
Nikolaus Rath added the comment: No, it doesn't. -- ___ Python tracker <http://bugs.python.org/issue18116> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18116] getpass.getpass() triggers ResourceWarning
Nikolaus Rath added the comment: Yes, I'm pretty sure: [0] nikratio@vostro:~/tmp$ cp /usr/lib/python3.3/getpass.py . [0] nikratio@vostro:~/tmp$ patch -p2 < issue18116.diff patching file getpass.py Hunk #1 succeeded at 57 (offset -1 lines). [0] nikratio@vostro:~/tmp$ python3 bugme.py /usr/lib/python3.3/os.py:1043: ResourceWarning: unclosed file <_io.FileIO name=3 mode='rb+'> return io.open(fd, *args, **kwargs) What's up? # Test if we're using the patched getpass.py... [0] nikratio@vostro:~/tmp$ vim getpass.py [0] nikratio@vostro:~/tmp$ python3 bugme.py Hello /usr/lib/python3.3/os.py:1043: ResourceWarning: unclosed file <_io.FileIO name=3 mode='rb+'> return io.open(fd, *args, **kwargs) What's up? -- ___ Python tracker <http://bugs.python.org/issue18116> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17814] Popen.stdin/stdout/stderr documentation should mention object type
Changes by Nikolaus Rath : -- keywords: +patch Added file: http://bugs.python.org/file30736/subprocess_rst.diff ___ Python tracker <http://bugs.python.org/issue17814> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17814] Popen.stdin/stdout/stderr documentation should mention object type
Nikolaus Rath added the comment: patch attached. -- ___ Python tracker <http://bugs.python.org/issue17814> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18334] type(name, bases, dict) does not call metaclass' __prepare__ attribute
New submission from Nikolaus Rath:
When using the three parameter form of type to create a new class, and any of
the base classes has a metaclass with a __prepare__ function, the __prepare__
function is not executed:
>>> class CustomMetaclass(type):
... @classmethod
... def __prepare__(cls, name, bases):
... return { 'prepared_for': name }
...
>>> class ParentClass(metaclass=CustomMetaclass):
... pass
...
>>> class ClassOne(ParentClass):
... pass
...
>>> ClassTwo = type('ClassTwo', (ParentClass,), {})
>>> ClassOne.prepared_for
'ClassOne'
>>> ClassTwo.prepared_for
'ParentClass'
>>> 'prepared_for' in ClassOne.__dict__
True
>>> 'prepared_for' in ClassTwo.__dict__
False
I am not sure if that is intended behavior or not. I am attaching a doc patch
for the case that this is intended.
--
components: Interpreter Core
files: type_doc_patch.diff
keywords: patch
messages: 192099
nosy: Nikratio
priority: normal
severity: normal
status: open
title: type(name, bases, dict) does not call metaclass' __prepare__ attribute
type: behavior
versions: Python 3.3
Added file: http://bugs.python.org/file30737/type_doc_patch.diff
___
Python tracker
<http://bugs.python.org/issue18334>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18334] type(name, bases, dict) does not call metaclass' __prepare__ attribute
Nikolaus Rath added the comment: In that cases, maybe type(name, cls, clsdict) should actually raise an error if there's a metaclass with __prepare__ involved? Presumably that would break only code that was already broken, but it would convert previously hidden behavioral bugs into an explicit expressions raised at the right point. -- ___ Python tracker <http://bugs.python.org/issue18334> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18524] BufferedReader.read1() documentation/implementation difference
New submission from Nikolaus Rath: The read1() docstring says: "Reads up to n bytes, with at most one read() system call." However, in the implementation read1() calls peek() to refill the buffer if necessary, and then returns whatever is available in the buffer. This means that read1() will only return up to n bytes if n is smaller than the buffer size, otherwise it will return at most bytes. With the current implementation, running a loop that calls obj.read1(n) for large n is therefore much less performant than a loop that calls obj.raw.read(n). I think a call to read1(n) with empty buffer should always attempt to read n bytes from the raw stream, i.e. the implementation should be changed to match the documentation. -- components: IO messages: 193496 nosy: Nikratio priority: normal severity: normal status: open title: BufferedReader.read1() documentation/implementation difference versions: Python 3.3 ___ Python tracker <http://bugs.python.org/issue18524> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15805] Add stdout redirection tool to contextlib
Nikolaus Rath added the comment: I think stdout redirection is very useful, and I'm actually have a very similar context manager in my own code. However, I'd like to raise the question if using a context manager for this purpose should really be "officially blessed" in this way. To me, the with statement signals that effects are constrained to the managed block. But redirecting sys.stdout is a change with global scope - the redirection is not restricted to execution of the with block, it affects every other thread that's running at the same time. This effect is obvious if you wrote the redirection context manager yourself, but if you're using code from the standard library, this may be surprising. I don't have a better proposal, but I just wanted to mention this... -- nosy: +Nikratio ___ Python tracker <http://bugs.python.org/issue15805> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18524] BufferedReader.read1() documentation/implementation difference
Nikolaus Rath added the comment:
On 07/24/2013 07:35 AM, Antoine Pitrou wrote:
>
> Antoine Pitrou added the comment:
>
>> This means that read1() will only return up to n bytes if n is smaller
>> than the buffer size, otherwise it will return at most
>> bytes.
>
> Did you actually observe such behaviour? If so, this is a bug.
Yes, that's what I see:
$ python3 bug.py
Read 20 bytes using read()
Read 10 bytes using read1()
$ cat bug.py
#!/usr/bin/env python3
import io
raw = io.BytesIO(bytes(200))
buffered = io.BufferedReader(raw, 10)
data = buffered.read(20)
print('Read %d bytes using read()' % len(data))
data = buffered.read1(20)
print('Read %d bytes using read1()' % len(data))
There should be no problem reading 20 bytes with a single call to
BytesIO.read(), yet the buffered reader returns only 10 bytes.
--
___
Python tracker
<http://bugs.python.org/issue18524>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18524] BufferedReader.read1() documentation/implementation difference
Nikolaus Rath added the comment: On 07/24/2013 02:28 PM, Antoine Pitrou wrote: > > Antoine Pitrou added the comment: > > Le mercredi 24 juillet 2013 à 18:24 +0000, Nikolaus Rath a écrit : >> There should be no problem reading 20 bytes with a single call to >> BytesIO.read(), yet the buffered reader returns only 10 bytes. > > Er. Your bug report seems to imply that read1(n) can return *more* than > n bytes, which would be a bug. > That it may return less than n bytes is correct, though. I admit the bug title may not have been chosen carefully enough. The documentation is correct that read1(n) never returns more than n bytes. My complaint is that the actual bound is stricter than this, band read1(n) will never return more than min(n, bufsize) bytes. To me, this seems unnecessary and counterintuitive. Why am I restricted by the buffer size if the underlying raw device could provide the requested amount with one call? Looking at the source, this restriction seems caused by read1() calling peek(). I think it would be straightforward to just delegate a read1(n) call to raw stream's read(n) method. This would improve performance, and also make the documentation more accurate. Did I miss something? Best, Nikolaus -- ___ Python tracker <http://bugs.python.org/issue18524> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18524] BufferedReader.read1() documentation/implementation difference
Nikolaus Rath added the comment:
On 07/24/2013 02:51 PM, Antoine Pitrou wrote:
>
> Antoine Pitrou added the comment:
>
> Le mercredi 24 juillet 2013 à 21:45 +0000, Nikolaus Rath a écrit :
>> The documentation is correct that read1(n) never returns more than n bytes.
>>
>> My complaint is that the actual bound is stricter than this, band
>> read1(n) will never return more than min(n, bufsize) bytes.
>
> That's not really true. If the buffer is empty, read1(n) will try to
> read at most n bytes from the raw stream. So:
> - if the buffer is not empty, the whole buffer is returned and 0 raw I/O
> call is issued
> - if the buffer is empty, 1 raw I/O call is issued and at most 1 byte is
> returned
>
> Therefore, if you call read1(n) in a loop, all calls but the first will
> really try to read *n* bytes from the raw stream (because the first call
> will have emptied the buffer).
I agree that this is how it *should* work. But that's not what's
happening in practice:
> $ python3
Python 3.2.3 (default, Feb 20 2013, 14:44:27)
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import io
>>> raw = io.BytesIO(bytes(200))
>>> buffered = io.BufferedReader(raw, 10)
>>> while True:
... buf = buffered.read1(20)
... print('Read %d bytes' % len(buf))
... if not buf:
...break
...
Read 10 bytes
Read 10 bytes
Read 10 bytes
Read 10 bytes
Read 10 bytes
Read 10 bytes
Read 10 bytes
Read 10 bytes
[...]
--
___
Python tracker
<http://bugs.python.org/issue18524>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18524] BufferedReader.read1() documentation/implementation difference
Nikolaus Rath added the comment: On 07/24/2013 03:04 PM, Antoine Pitrou wrote: > However, 3.2 didn't get that improvement, sorry. See changeset 27bf3d0b8e5f > and issue #13393. > > -- > resolution: -> duplicate > status: open -> closed Duh. So much back and forth just to find it's already fixed. Python is moving too fast these days... :-). -Nikolaus -- ___ Python tracker <http://bugs.python.org/issue18524> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18574] BaseHTTPRequestHandler.handle_expect_100() sends invalid response
New submission from Nikolaus Rath: The handle_expect_100() implementation in BaseHTTPRequestHandler currently calls send_response_only(100) followed by flush_headers(). However, even a 1xx response is always followed by a (potentially empty) set of response headers (cf. http://tools.ietf.org/html/rfc2616#section-9.9). Therefore, clients will block waiting for an additional '\r\n' before sending the buffer. The attached patch fixes the problem. -- components: Library (Lib) files: handle_expect.diff keywords: patch messages: 193800 nosy: Nikratio priority: normal severity: normal status: open title: BaseHTTPRequestHandler.handle_expect_100() sends invalid response versions: Python 3.3, Python 3.4, Python 3.5 Added file: http://bugs.python.org/file31057/handle_expect.diff ___ Python tracker <http://bugs.python.org/issue18574> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18574] BaseHTTPRequestHandler.handle_expect_100() sends invalid response
Nikolaus Rath added the comment: The correct link is http://tools.ietf.org/html/rfc2616#section-10.1: 10.1 Informational 1xx This class of status code indicates a provisional response, consisting only of the Status-Line and optional headers, and is terminated by an empty line. [...] Currently handle_expect_100() does not send this empty line. -- ___ Python tracker <http://bugs.python.org/issue18574> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18861] Problems with recursive automatic exception chaining
New submission from Nikolaus Rath:
Consider this:
$ python3 test_exc.py
Traceback (most recent call last):
File "test_exc.py", line 14, in
fail1()
File "test_exc.py", line 11, in fail1
fail2()
File "test_exc.py", line 5, in fail2
raise RuntimeError('Third') from None
RuntimeError: Third
$ cat test_exc.py
def fail2():
try:
raise RuntimeError('Second')
except RuntimeError:
raise RuntimeError('Third') from None
def fail1():
try:
raise RuntimeError('First')
except:
fail2()
raise
fail1()
Any exception raised in fail2() is the immediate consequence of the 'First'
exception should thus be chained to the 'First' exception.
However, if somewhere in the call stack under fail2() an exception is caught
and re-raised from None (to convert between exception types), this also results
in a loss of the chain to the initial exception.
The correct stacktrace (in my opinion) would be:
Traceback (most recent call last):
File "test_exc.py", line 9, in fail1
raise RuntimeError('First')
RuntimeError: First
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "test_exc.py", line 14, in
fail1()
File "test_exc.py", line 11, in fail1
fail2()
File "test_exc.py", line 5, in fail2
raise RuntimeError('Third')
RuntimeError: Third
--
components: Interpreter Core
messages: 196344
nosy: Nikratio
priority: normal
severity: normal
status: open
title: Problems with recursive automatic exception chaining
type: behavior
versions: Python 3.3
___
Python tracker
<http://bugs.python.org/issue18861>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18861] Problems with recursive automatic exception chaining
Nikolaus Rath added the comment: The second paragraph should of course read "...consequence of the 'First' exception *and* should thus be chained..." -- ___ Python tracker <http://bugs.python.org/issue18861> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12704] Language Reference: Clarify behaviour of yield when generator is not resumed
Nikolaus Rath added the comment: *ping* -- ___ Python tracker <http://bugs.python.org/issue12704> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12704] Language Reference: Clarify behaviour of yield when generator is not resumed
Nikolaus Rath added the comment: On 09/04/2013 06:03 AM, Eli Bendersky wrote: > Why guess... did you try it in the code? I don't follow... why guess what? And try what in code? > it would be nice to have a short code sample here demonstrating what's > happening. The paragraph you're quoting seems obscure to me, with or without > the fix. Sounds reasonable. I'll revise the patch and add an example. Best, -Nikolaus -- »Time flies like an arrow, fruit flies like a Banana.« PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C -- ___ Python tracker <http://bugs.python.org/issue12704> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12704] Language Reference: Clarify behaviour of yield when generator is not resumed
Nikolaus Rath added the comment: I've attached an updated patch. I agree with Tim that the docs on the yield expression are already very good, so instead of extending the yield statement documentation I have shortened it to instead reduce the overlap that already exists. In the yield expression documentation, I have replaced :keyword:`yield` with an un-tagged "yield" (because the former results in a link to the yield statement) and added '.. or yield statement' to the parts that apply to both yield expressions and statements. I have also added hyperlinks to the relevant descriptions of the generator methods. -- Added file: http://bugs.python.org/file31688/yield-docpatch.diff ___ Python tracker <http://bugs.python.org/issue12704> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15633] httplib.response is not closed after all data has been read
New submission from Nikolaus Rath: Occasionally, the isclosed() method of HTTPResponse instances returns False, even though a preceding read() call returned '' (or something else that evalues to False). This is a problem, because then the corresponding HTTPConnection can still be used to send another request, but an attempt to retrieve its response result it in ResponseNotReady() exception. I have not found anything special about the requests for which this happens. I also looked at the httplib code and found no way for this to happen, but obviously I am missing something. Maybe someone more familiar with the codebase can tell what's happening there. The problem happens to rarely to be able to run packet capture, but I can provide attributes of the relevant httplib instances if desired. -- components: Library (Lib) messages: 168068 nosy: Nikratio priority: normal severity: normal status: open title: httplib.response is not closed after all data has been read type: behavior versions: Python 2.7 ___ Python tracker <http://bugs.python.org/issue15633> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15633] httplib.response is not closed after all data has been read
Nikolaus Rath added the comment: The problem seems to occur regularly, just with large intervals in between. Is there any specific information that might help here? I am not quite sure what to save when the problem happens the next time. Can the response object be pickled? -- ___ Python tracker <http://bugs.python.org/issue15633> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15633] httplib.response is not closed after all data has been read
Nikolaus Rath added the comment: The call should be read(size), with size a number. I will make sure to check size, "fp", "length", "chunked" and "chunk_left" when it happens the next time. -- ___ Python tracker <http://bugs.python.org/issue15633> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15633] httplib.response is not closed after all data has been read
Nikolaus Rath added the comment: This is just a heads-up that I'm still trying to debug this. So far, the problem doesn't seem to have shown up again (for reference, the original report for this is http://code.google.com/p/s3ql/issues/detail?id=358). -- ___ Python tracker <http://bugs.python.org/issue15633> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1738] filecmp.dircmp does exact match only
Nikolaus Rath added the comment: I don't think that we can just introduce path normalization in phase0. Even though I agree that this would be the proper way to do it when reimplementing from scratch, it breaks backward compatibility. There also is a small mistake in that the *match* attribute should also be used for subdirectories in the `phase4` method. Other than that, this patch looks good to me. I fixed the above issues, rebased on current hg tip, and added some missing markup in the documentation. After inspecting the code, it seems that there is no difference between directory entries being "hidden" by the *hide* parameter, and being "ignored* by the *ignore* parameter, so I also updated the documentation make this less confusing. I could not reproduce the test failure reported by Mark, but this is most likely because I could not find out on what base revision to apply his patch. I think this is ready for commit. -- nosy: +nikratio Added file: http://bugs.python.org/file3/issue1738.diff ___ Python tracker <http://bugs.python.org/issue1738> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1738] Add match parameter to filecmp.dircmp to ignore name patterns
Changes by Nikolaus Rath : -- title: filecmp.dircmp does exact match only -> Add match parameter to filecmp.dircmp to ignore name patterns versions: +Python 3.5 -Python 3.2 ___ Python tracker <http://bugs.python.org/issue1738> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1738] Add match parameter to filecmp.dircmp to ignore using patterns
Changes by Nikolaus Rath : -- title: Add match parameter to filecmp.dircmp to ignore name patterns -> Add match parameter to filecmp.dircmp to ignore using patterns ___ Python tracker <http://bugs.python.org/issue1738> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20951] SSLSocket.send() returns 0 for non-blocking socket
New submission from Nikolaus Rath: When using non-blocking operation, the SSLSocket.send method returns 0 if no data can be sent at this point. This is counterintuitive, because in the same situation (write to non-blocking socket that isn't ready for IO): * A regular (non-SSL) socket raises BlockingIOError * libc's send(2) does not return 0, but -EAGAIN or -EWOULDBLOCK. * OpenSSL's ssl_write does not return 0, but returns an SSL_ERROR_WANT_WRITE error * The ssl module's documentation describes the SSLWantWrite exception as "A subclass of SSLError raised by a non-blocking SSL socket when trying to read or write data, but more data needs to be sent on the underlying TCP transport before the request can be fulfilled." * Consistent with that, trying to *read* from a non-blocking SSLSocket when no data is ready raises SSLWantRead, instead of returning zero. This behavior also makes it more complicated to write code that works with both SSLSockets and regular sockets. Since the current behavior undocumented at best (and contradicting the documentation at worst), can we change this in Python 3.5? -- components: Library (Lib) messages: 213759 nosy: christian.heimes, giampaolo.rodola, janssen, nikratio, pitrou priority: normal severity: normal status: open title: SSLSocket.send() returns 0 for non-blocking socket type: behavior versions: Python 3.5 ___ Python tracker <http://bugs.python.org/issue20951> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20951] SSLSocket.send() returns 0 for non-blocking socket
Nikolaus Rath added the comment: This is actually seems to be not just an inconvience, but a real bug: since SSLSocket.sendall() uses SSLSocket.send() internally, the former method will busy-loop when called on a non-blocking socket. Note also that the .sendto and .write methods already behave consistently and raise SSLWantWrite. It seems it's really just the send() method that is the lone outlier. The attached patch changes ssl.send to raise SSLWantWrite instead of returning zero. The full testsuite still runs fine. I'm a bit sceptical though, because the code looks as if send() was deliberately written to catch the SSLWantWrite exception and return zero instead.. Can anyone familiar with the code comment on this? -- keywords: +patch Added file: http://bugs.python.org/file34448/issue20951.diff ___ Python tracker <http://bugs.python.org/issue20951> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20951] SSLSocket.send() returns 0 for non-blocking socket
Nikolaus Rath added the comment: Antoine, do you know that there are frameworks out there using this, or is that a guess? asyncio, for example, seems to expect an SSLWantWrite exception as well. (it also works with a zero return, but it's not clear from the code if that's by design or by a chance). -- ___ Python tracker <http://bugs.python.org/issue20951> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20951] SSLSocket.send() returns 0 for non-blocking socket
Nikolaus Rath added the comment: Twisted does not seem to rely on it either (there's no mention of SSLWant* in the source at all, and without that, you can't possibly have support for non-blocking ssl sockets). -- ___ Python tracker <http://bugs.python.org/issue20951> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20951] SSLSocket.send() returns 0 for non-blocking socket
Nikolaus Rath added the comment: gevent is calling _sslobject.write() directly, so it would not be affected by any change. -- ___ Python tracker <http://bugs.python.org/issue20951> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20951] SSLSocket.send() returns 0 for non-blocking socket
Nikolaus Rath added the comment: Tornado uses SSLSocket.send(), and it looks as if a SSLWantWrite exception is not caught but would propagate, so this would probably break. -- ___ Python tracker <http://bugs.python.org/issue20951> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20951] SSLSocket.send() returns 0 for non-blocking socket
Nikolaus Rath added the comment: More info on twisted: it uses PyOpenSSL rather than the stdlib ssl module, so it's not affected at all. -- ___ Python tracker <http://bugs.python.org/issue20951> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
