[issue38007] Regression: name of PyType_Spec::slots conflicts with Qt macro
New submission from Leslie : In Feb 20, 2006, in commit https://github.com/python/cpython/commit/c255c7bef7621596869f56d887ac3ccd5b536708 the bug https://bugs.python.org/issue1086854 had been fixed. In Dec 3, 2010, in commit https://github.com/python/cpython/commit/4d0d471a8031de90a2b1ce99c4ac4780e60b3bc9 basically the same issue has been introduced again, because it added a "slots" member to the added PyType_Spec struct. The proposed solution is the same like in the earlier commit, which is explained in https://bugs.python.org/msg23759 : protect PyType_Spec struct members with prefix: ts_name, ts_doc, ts_basicsize, ts_itemsize, ts_flags, ts_slots -- messages: 350977 nosy: Leslie priority: normal severity: normal status: open title: Regression: name of PyType_Spec::slots conflicts with Qt macro type: compile error versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue38007> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38007] Regression: name of PyType_Spec::slots conflicts with Qt macro
Change by Leslie : -- keywords: +patch pull_requests: +15309 stage: -> patch review pull_request: https://github.com/python/cpython/pull/15644 ___ Python tracker <https://bugs.python.org/issue38007> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38007] Regression: name of PyType_Spec::slots conflicts with Qt macro
Leslie added the comment: Some history and explanation of this problem: - Qt extends C++ with the following keywords: "slots", "signals", "emit" - This extension is compiled with a Meta-Object-Compiler (MOC) into standard C++. - Since code using Qt extensions (i.e. headers) generally used "mixed" with standard C++ code, "standard" compiler must be able to compile the Qt specific part, too -> the extension keywords are declared as empty macros (i.e. #define slots) The consequence is that if Qt-based headers are used together with any 3rd party headers, the 3rd party code cannot contain the extension keywords (slots, signals, emit) as C/C++ tokens, because the preprocessor will "delete" them as a result of macro expansion, since the keywords are defined as empty macros. -> The code won't compile because of syntax errors. This caused bug https://bugs.python.org/issue1086854 , which was fixed in https://github.com/python/cpython/commit/c255c7bef7621596869f56d887ac3ccd5b536708 The fix renamed the "slots" struct member in Python, thus the conflict was resolved. Note: the Qt library is, like Python, old, and used in a huge number of projects and products. E.g. Qt is the base platform for KDE development. It's a matter of point of view, which of the two, Qt or Python should be "fixed". In my PR I used the same solution: renaming the conflicting member. - Since this time this solution was not welcome, and obviously I'm not the first facing this issue, I was searching for another solutions. I found that Qt has already provided a solution for this problem. There are 2 macros: QT_NO_SIGNALS_SLOTS_KEYWORDS and QT_NO_KEYWORDS , which "turn off" defining the extension keywords as macros. Instead the QT_SLOTS, Q_SIGNALS macros should be used, which very likely do not interfere with any 3rd party library... Though, these are "official" Qt macros, not very well documented: https://bugreports.qt.io/browse/QTBUG-70564 :) So by defining QT_NO_KEYWORDS I could resolve this whole issue. Thanks guys for your patience and attetion! :) -- ___ Python tracker <https://bugs.python.org/issue38007> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22213] Make pyvenv style virtual environments easier to configure when embedding Python
Leslie added the comment: I just can say that sorting this issue (and PEP-0432) out would be great! I run into this issue when embedding CPython in a Windows app, and want to use some pre-installed Python, which is not part of the install package... So beside pyenv venvs, please keep Windows devs in mind, too! :) -- nosy: +Leslie ___ Python tracker <https://bugs.python.org/issue22213> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16140] subprocess.Popen the os.close calls in _execute_child can raise an EBADF exception
Changes by Benno Leslie : -- nosy: +bennoleslie ___ Python tracker <http://bugs.python.org/issue16140> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16140] subprocess.Popen the os.close calls in _execute_child can raise an EBADF exception
Benno Leslie added the comment: Regarding #2 my understanding is that the FDs are already always wrapped. E.g: at line http://hg.python.org/cpython/file/b9ac3c44a4eb/Lib/subprocess.py#l798 it shows these always being wrapped (assuming the file descriptor is not -1). For my test case on 3.2.3, replacing the os.close loop with: if p2cwrite != -1: self.stdin.close() if c2pread != -1: self.stdout.close() if errread != -1: self.stderr.close() This fixed all my stability problems and races, and can't (as far as I can tell) cause any other problems; however this is a very subtle module, so I'm eager to understand if this would cause any undesirable side-effects. -- ___ Python tracker <http://bugs.python.org/issue16140> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21164] print unicode in Windows console
New submission from Leslie Klein: The console behaves by encoding a str (using the sys.getdefaultencoding()) and then decodes the bytes to a glyph for rendering. The decoder used is 'cp437'. Apparently, there is no way to override that! See ipython notebook for summary and example of the issue. nbviewer.ipython.org/gist/LeslieK/10013426 -- components: Windows messages: 215675 nosy: LeslieK priority: normal severity: normal status: open title: print unicode in Windows console type: behavior versions: Python 3.3 ___ Python tracker <http://bugs.python.org/issue21164> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21164] print unicode in Windows console
Leslie Klein added the comment:
I start ipython in Windows PowerShell. The "console" I am referring to is
ipython running in WindowsPowerShell.
I do not use the DOS cmd.exe
When running ipython notebook from WindowsPowerShell -- no problem printing
unicode.
When running in PTVS (Python Tools for Visual Studio) debugger -- no
problem printing unicode in Output window, or Interactive Python window.
Problem does appear in Console window (which is the DOS window I believe).
On Mon, Apr 7, 2014 at 7:08 AM, STINNER Victor wrote:
>
> STINNER Victor added the comment:
>
> Hi, when you say "The console" is the the old MS-DOS command ("Windows
> console") opened when you run the "cmd.exe" program? Or IDLE or another
> console?
>
> For the Windows console, see the old issue #1602 which is not fixed yet.
>
> On Windows, sys.stdout.encoding is your OEM code page, which is usually
> different than the ANSI code page (locale.getpreferredencoding()).
>
> --
> nosy: +haypo
>
> ___
> Python tracker
> <http://bugs.python.org/issue21164>
> ___
>
--
___
Python tracker
<http://bugs.python.org/issue21164>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16833] httpc.lient delayed ack / Nagle algorithm optimisation performs badly for large messages
New submission from Benno Leslie: he http.client HTTPConnection._send_output method has an optimization for avoiding bad interactions between delayed-ack and the Nagle algorithm: http://hg.python.org/cpython/file/f32f67d26035/Lib/http/client.py#l884 Unfortunately this interacts rather poorly if the case where the message_body is a bytes instance and is rather large. If the message_body is bytes it is appended to the headers, which causes a copy of the data. When message_body is large this duplication of data can cause a significant spike in memory usage. (In my particular case I was uploading a 200MB file to 30 hosts at the same leading to memory spikes over 6GB. [There is a short thread discussing this issue on python-dev; Subject: "http.client Nagle/delayed-ack optimization"; Date: Dec 15, 2012] -- components: Library (Lib) files: http_opt.diff keywords: patch messages: 178728 nosy: bennoleslie priority: normal severity: normal status: open title: httpc.lient delayed ack / Nagle algorithm optimisation performs badly for large messages type: performance versions: Python 3.3 Added file: http://bugs.python.org/file28518/http_opt.diff ___ Python tracker <http://bugs.python.org/issue16833> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16833] httpc.lient delayed ack / Nagle algorithm optimisation performs badly for large messages
Benno Leslie added the comment: I've updated the patch based on Charles-François and Antoine's feedback. Primarily this increase the estimated MSS value to 16KiB. Additionally tests are added and comments improved. Thanks for the feedback. -- Added file: http://bugs.python.org/file28521/http_opt_v2.diff ___ Python tracker <http://bugs.python.org/issue16833> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16866] libainstall doesn't create all directories
New submission from Benno Leslie: make libainstall fails when $(BINDIR) does not exist. -- components: Build files: ainstall.diff keywords: patch messages: 179103 nosy: bennoleslie priority: normal severity: normal status: open title: libainstall doesn't create all directories type: compile error versions: Python 3.4 Added file: http://bugs.python.org/file28569/ainstall.diff ___ Python tracker <http://bugs.python.org/issue16866> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16866] libainstall doesn't create $(BINDIR) directory
Changes by Benno Leslie : -- title: libainstall doesn't create all directories -> libainstall doesn't create $(BINDIR) directory ___ Python tracker <http://bugs.python.org/issue16866> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16867] setup.py fails if there are no extensions
New submission from Benno Leslie: When building if there are zero extensions setup.py fails (max of a zero length list is undefined.) Although not a very common case, there are potential speciality cases where there will be zero extensions. -- components: Build files: zero_length_extensions.diff keywords: patch messages: 179104 nosy: bennoleslie priority: normal severity: normal status: open title: setup.py fails if there are no extensions versions: Python 3.4 Added file: http://bugs.python.org/file28570/zero_length_extensions.diff ___ Python tracker <http://bugs.python.org/issue16867> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16869] makesetup should support .S source files
New submission from Benno Leslie: It is useful to be able to build .S files as built-in modules (in particular if you want ctypes as a built-in you need .S files) The patch enables .S files to be specified in Setup.dist files. -- components: Build files: makesetup-asm.diff keywords: patch messages: 179111 nosy: bennoleslie priority: normal severity: normal status: open title: makesetup should support .S source files versions: Python 3.4 Added file: http://bugs.python.org/file28574/makesetup-asm.diff ___ Python tracker <http://bugs.python.org/issue16869> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16866] libainstall doesn't create $(BINDIR) directory
Benno Leslie added the comment: I was using this in the case where I just want to link against libpython.a and for me it is a limited case where I don't really need the functionality of python-config; so for me this is certainly the best approach. But I concede this use case is probably a very little general interest. If the goal is to have a working python-config, then I guess libainstall should depend on bininstall. I'm not too fussed which approach is taken, I just think that (make && make libainstall) should work. I'm happy to fashion a patch for whichever approach is thought to be best. -- ___ Python tracker <http://bugs.python.org/issue16866> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16869] makesetup should support .S source files
Benno Leslie added the comment: Thanks for the comments Senthil. I'll improve the patch fixing the bug, adding support for .s and updating the docs. -- ___ Python tracker <http://bugs.python.org/issue16869> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16866] libainstall doesn't create $(BINDIR) directory
Benno Leslie added the comment: In a similar manner the bininstall target relies on $(LIBPC), but does not create that. This makes me consider if the libainstall target should be installing pkg-config sciprt at all (and whether bininstall should be installing the .pc files). It is hard for me to determine what the exact intended goals of each of these targets is, so I can't really come up with the right fix. Naively for both targets ensuring that the directory exists solves the symptom, but to be it looks like there is probably a greater underlying thing to determine here. -- ___ Python tracker <http://bugs.python.org/issue16866> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6759] zipfile.ZipExtFile.read() is missing universal newline support
New submission from Ryan Leslie :
The zipfile.ZipFile.open() behavior with mode 'U' or 'rU' is not quite
as advertised in
http://docs.python.org/library/zipfile.html#zipfile.ZipFile.open
Here is an example:
$ echo -ne "This is an example\r\nWhich demonstrates a problem\r\nwith
ZipFile.open(..., 'U')\r\n" > foo.txt
$ cat -v foo.txt
This is an example^M
Which demonstrates a problem^M
with ZipFile.open(..., 'U')^M
$ zip foo.zip foo.txt
adding: foo.txt (deflated 1%)
$ python
Python 2.6.2 (r262:71600, Aug 21 2009, 17:52:12)
[GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> open("foo.txt", 'U').read()
"This is an example\nWhich demonstrates a problem\nwith
ZipFile.open(..., 'U')\n"
>>> from zipfile import ZipFile
>>> ZipFile("foo.zip").open("foo.txt", 'U').read()
"This is an example\r\nWhich demonstrates a problem\r\nwith
ZipFile.open(..., 'U')\r\n"
>>>
The open() method was added here:
http://bugs.python.org/issue1121142
The cause is that the universal newline implementation is specific to
readline(), which also implements readlines() and next() as well.
Support was never added for read(), which is independent.
Note that test_zipfile.UniversalNewlineTests.readTest() passes. This is
suspect because it's explicitly coded to *not* expect translation of new
line sequences.
--
components: Library (Lib)
messages: 91854
nosy: ryles
severity: normal
status: open
title: zipfile.ZipExtFile.read() is missing universal newline support
versions: Python 2.6
___
Python tracker
<http://bugs.python.org/issue6759>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6759] zipfile.ZipExtFile.read() is missing universal newline support
Ryan Leslie added the comment: Hi Art, Thanks for working on this. I've taken a look at the patch. The fix to read_test looks correct. Of course, I would consider a more descriptive variable name than 'b'. The changes to read() are an improvement, but I think we need to be careful when we replace "\r\n" with "\n". Basically, we've turned two characters into one and are now potentially one character short of 'size' bytes. This doesn't match the behavior of file.read(). Another thing to work out is the lack of the 'newlines' attribute, discussed in PEP 278. I've noted that bz2 seems to do a pretty good job with universal newline handling: python/trunk/Modules/bz2module.c. It's in C, however, and I don't think there's actually anything in the library doing UL in pure Python. -- ___ Python tracker <http://bugs.python.org/issue6759> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7052] "from logging import *" causes an error under Ubuntu Karmic
Ryan Leslie added the comment: Looks like a merge has gone bad. NullHandler has existed for a while on trunk but is not present in the 2.6.3 tag (__all__ was updated to include it, however): /python/tags/r263/Lib/logging/__init__.py -- nosy: +ryles ___ Python tracker <http://bugs.python.org/issue7052> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7157] Fix Download Current Documentation link
New submission from Ryan Leslie : http://docs.python.org/download.html shows this: Download Python 2.6.4c1 Documentation We don't package the documentation for development releases for download. Downloads will be available for the final release. This is not really acceptable for someone navigating through python.org to the Documentation page. The latest 2.6 reference, tutorial, etc, are all available, and so the archives with the latest PDF versions should be too. -- assignee: georg.brandl components: Documentation messages: 94158 nosy: georg.brandl, ryles severity: normal status: open title: Fix Download Current Documentation link versions: Python 2.6 ___ Python tracker <http://bugs.python.org/issue7157> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7169] zipfile leaves a file handle open if file is zero size
Ryan Leslie added the comment: I expect this should already be fixed by the commit in http://bugs.python.org/issue6511 BadZipFile will now be raised for empty files rather than IOError, and so ZipFile._GetContents() should now also close the file. The fix was committed to trunk, but I don't see it merged into 2.6. -- nosy: +amaury.forgeotdarc, ryles ___ Python tracker <http://bugs.python.org/issue7169> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7169] zipfile leaves a file handle open if file is zero size
Ryan Leslie added the comment: Yes, I think this fix should have been included in the 2.6 branch. I subscribed Amaury to look into that when I last updated. -- ___ Python tracker <http://bugs.python.org/issue7169> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7249] Consider allowing io.BytesIO sizes to be passed as 'long' in 2.6
New submission from Ryan Leslie :
py> StringIO.StringIO("foo").read(long(1))
'f'
py> io.BytesIO("foo").read(long(1))
Traceback (most recent call last):
File "", line 1, in
TypeError: integer argument expected, got 'long'
This is known to cause problems when reading zip data from a BytesIO
object with zipfile. See this recent thread on comp.lang.python:
http://groups.google.com/group/comp.lang.python/browse_thread/thread/337c1b8a48e8acae/
--
components: Library (Lib)
messages: 94818
nosy: ryles
severity: normal
status: open
title: Consider allowing io.BytesIO sizes to be passed as 'long' in 2.6
type: behavior
versions: Python 2.6
___
Python tracker
<http://bugs.python.org/issue7249>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5971] logging.Handler.handlerError() may raise IOError in traceback.print_exception()
New submission from Ryan Leslie : When using the logging package, if a StreamHandler is configured with stderr and stderr is redirected to a pipe which no longer has readers, then StreamHandler.emit() will result in an IOError for "Broken pipe". The exception will be handled in logging.Handler.handleError(), which by default will call traceback.print_exception() with file=sys.stderr. This will cause in a second IOError exception which will not be caught within the logging code. Unless the user placed their log calls such as logging.info() in a try/except block (unlikely), the end result is termination of the process. While the logging code itself is certainly not the cause of the underlying problem, it does seem that the intent of the default handleError() was to eat exceptions, and possibly print them, without disturbing the application code. As the docstring correctly points out, the application can probably survive without the logging. To work around this issue without writing a custom handler, raiseExceptions can be set to false. But then the user would miss log trace when other types of errors not affecting stderr occurred. That is, I think handleError() does the right thing in trying to print the error, but suggest that if the print results in an IOError (or certain types of IOError), then it should be ignored, or optionally ignorable. -- components: Library (Lib) messages: 87469 nosy: ryles severity: normal status: open title: logging.Handler.handlerError() may raise IOError in traceback.print_exception() type: behavior versions: Python 2.6 ___ Python tracker <http://bugs.python.org/issue5971> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5979] strptime() gives inconsistent exceptions
New submission from Ryan Leslie :
e.g.
>>> from datetime import datetime
>>>
>>> datetime.strptime("19951001", "%Y%m%d")
datetime.datetime(1995, 10, 1, 0, 0)
>>>
>>> datetime.strptime("19951000", "%Y%m%d") # day = 0, month < 11
...
ValueError: time data '19951000' does not match format '%Y%m%d'
>>>
>>> datetime.strptime("19951100", "%Y%m%d") # day = 0, month >= 11
...
ValueError: unconverted data remains: 0
>>>
The exception messages are not really a serious issue, but note that the
latter one can be confusing for users.
However, there seems to be some underlying issues with the choice to
recognize single digit months with double digit days, which can make
date strings ambiguous:
Consider "19951100" from above with the last '0' removed.
>>> datetime.strptime("1995110", "%Y%m%d")
datetime.datetime(1995, 1, 10, 0, 0)
>>>
In this case, strptime has treated the middle '1' as the month,
resulting in 1995-01-10. This hints at why the second exception from
above gives a strange message: with the extra '0' the day portion of
"19951100" (00) is invalid, and strptime falls back on parsing the first
7 characters as above, and then failing due to the remaining '0'.
This seems a little dangerous. For instance:
timestamp = "19951101-23:20:18"
datestamp=timestamp[:7] # Oops, needed to use :8 to get up to index 7.
reallyImportantButWrongDate = datetime.strptime(datestamp, "%Y%m%d")
Note that in this case strptime() from glibc would instead result in an
error, which IMHO is the right thing to do.
I do realize, though, that changing the behavior of strptime() could
devastate some existing code.
--
components: Library (Lib)
messages: 87505
nosy: ryles
severity: normal
status: open
title: strptime() gives inconsistent exceptions
type: behavior
versions: Python 2.4, Python 2.5, Python 2.6, Python 2.7, Python 3.0, Python 3.1
___
Python tracker
<http://bugs.python.org/issue5979>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6056] socket.setdefaulttimeout affecting multiprocessing Manager
New submission from Ryan Leslie :
Terminal 1:
Python 2.6.1 (r261:67515, Apr 2 2009, 18:25:55)
[GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from multiprocessing.managers import SyncManager
>>> manager = SyncManager(authkey="mykey")
>>> manager.start()
>>> queue = manager.Queue()
>>> import pickle
>>> pickle.dump(queue, open("myqueue.pkl", "w"))
>>>
Terminal 2:
Python 2.6.1 (r261:67515, Apr 2 2009, 18:25:55)
[GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> socket.setdefaulttimeout(30)
>>> import multiprocessing, pickle
>>> multiprocessing.current_process().authkey = "mykey"
>>> queue = pickle.load(open("myqueue.pkl"))
Traceback (most recent call last):
File "", line 1, in
File "python2.6/pickle.py", line 1370, in load
return Unpickler(file).load()
File "python2.6/pickle.py", line 858, in load
dispatch[key](self)
File "python2.6/pickle.py", line 1133, in load_reduce
value = func(*args)
File "python2.6/multiprocessing/managers.py", line 845, in RebuildProxy
return func(token, serializer, incref=incref, **kwds)
File "python2.6/multiprocessing/managers.py", line 894, in AutoProxy
incref=incref)
File "python2.6/multiprocessing/managers.py", line 700, in __init__
self._incref()
File "python2.6/multiprocessing/managers.py", line 749, in _incref
conn = self._Client(self._token.address, authkey=self._authkey)
File "python2.6/multiprocessing/connection.py", line 140, in Client
answer_challenge(c, authkey)
File "python2.6/multiprocessing/connection.py", line 376, in
answer_challenge
response = connection.recv_bytes(256)# reject large message
IOError: [Errno 11] Resource temporarily unavailable
>>>
This works as expected without socket.setdefaulttimeout(). However, the
timeout is useful since if the listening process on terminal 1 goes to
sleep, e.g. ^Z, it would avoid blocking.
I suspect the cause is similar to http://bugs.python.org/issue976613
--
components: Library (Lib)
messages: 88040
nosy: ryles
severity: normal
status: open
title: socket.setdefaulttimeout affecting multiprocessing Manager
versions: Python 2.6
___
Python tracker
<http://bugs.python.org/issue6056>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6056] socket.setdefaulttimeout affecting multiprocessing Manager
Ryan Leslie added the comment: Yeah, storing pickled queues in the file system makes for some easy IPC :) It wasn't a very original idea, I took the pickling comments in the documentation at face value: http://docs.python.org/library/multiprocessing.html#proxy-objects So, from what I can tell this issue is related to the mixing of standard python socket I/O with multiprocessing socket I/O, with state not being carried from the former to the latter. In multiprocessing/connection.py, SocketClient() creates a familiar python socket object which, when a default timeout has been set in the module, will be made non-blocking. In addition, the timeout is remembered in the socket object, and when calling socket.recv(), the function internal_select() will use this to perform the expected poll() or select(). However, after a connection is established, SocketClient() will not use python's socket implementation any further, and instead pass its low-level socket descriptor to a multiprocessing Connection object. This object has its own specialized socket I/O implementation, which is not at all aware of the timeout previously associated with the socket. As a result no select/poll occurs and, due to the socket's non-blocking status, recv() may return EAGAIN immediately. I suspect this is what's happening. There might be a number of ways to make SocketClient() more timeout friendly, but possibility could be to simply check if the python socket has a timeout associated, and if so, use connection.poll() in addition to connection.recv(). There may be other places in the code where similar changes would occur. You obviously have more experience with this code base so I'll be curious to see the outcome. -- type: -> behavior ___ Python tracker <http://bugs.python.org/issue6056> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6082] os.path.sameopenfile reports that standard streams are the same
New submission from Ryan Leslie : Python 2.6.1 (r261:67515, Apr 2 2009, 18:25:55) [GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import sys, os >>> os.path.sameopenfile(sys.stdin.fileno(), sys.stdout.fileno()) True >>> os.path.sameopenfile(sys.stdout.fileno(), sys.stderr.fileno()) True >>> null = open(os.devnull) >>> os.path.sameopenfile(sys.stdin.fileno(), null.fileno()) False >>> # That worked. -- components: Library (Lib) messages: 88174 nosy: ryles severity: normal status: open title: os.path.sameopenfile reports that standard streams are the same type: behavior versions: Python 2.6 ___ Python tracker <http://bugs.python.org/issue6082> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6082] os.path.sameopenfile reports that standard streams are the same
Ryan Leslie added the comment:
Thanks for the quick response, Philip. Makes even more sense now that I see:
Python 2.6.1 (r261:67515, Apr 2 2009, 18:25:55)
[GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> open('/dev/stdout').readline()
hello
'hello\n'
>>> open('/dev/stdin').readline()
hello
'hello\n'
>>>
--
status: closed -> open
___
Python tracker
<http://bugs.python.org/issue6082>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6082] os.path.sameopenfile reports that standard streams are the same
Changes by Ryan Leslie : -- status: open -> closed ___ Python tracker <http://bugs.python.org/issue6082> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6362] multiprocessing: handling of errno after signals in sem_acquire()
New submission from Ryan Leslie :
While developing an application, an inconsistency was noted where,
depending on the particular signal handler in use,
multiprocessing.Queue.put() may (or may not) raise OSError() after
sys.exit() was called by the handler. The following example, which was
tested with Python 2.6.1 on Linux, demonstrates this.
#!/usr/bin/env python
import multiprocessing
import signal
import sys
def handleKill(signum, frame):
#sys.stdout.write("Exit requested by signal.\n")
print "Exit requested by signal."
sys.exit(1)
signal.signal(signal.SIGTERM, handleKill)
queue = multiprocessing.Queue(maxsize=1)
queue.put(None)
queue.put(None)
When the script is run, the process will block (as expected) on the
second queue.put(). If (from another terminal) I send the process
SIGTERM, I consistently see:
$ ./q.py
Exit requested by signal.
$
Now, if I modify the above program by commenting out the 'print', and
uncommenting the 'sys.stdout' (a very subtle change), I would expect
the result to be the same when killing the process. Instead, I
consistently see:
$ ./q.py
Exit requested by signal.
Traceback (most recent call last):
File "./q.py", line 15, in
queue.put(None)
File "python2.6/multiprocessing/queues.py", line 75, in put
if not self._sem.acquire(block, timeout):
OSError: [Errno 0] Error
$
After debugging this further, the issue appears to be in
semlock_acquire() or semaphore.c in Modules/_multiprocessing:
http://svn.python.org/view/python/trunk/Modules/_multiprocessing/semaphore.c?revision=71009&view=markup
The relevant code from (the Unix version of) semlock_acquire() is:
do {
Py_BEGIN_ALLOW_THREADS
if (blocking && timeout_obj == Py_None)
res = sem_wait(self->handle);
else if (!blocking)
res = sem_trywait(self->handle);
else
res = sem_timedwait(self->handle, &deadline);
Py_END_ALLOW_THREADS
if (res == MP_EXCEPTION_HAS_BEEN_SET)
break;
} while (res < 0 && errno == EINTR && !PyErr_CheckSignals());
if (res < 0) {
if (errno == EAGAIN || errno == ETIMEDOUT)
Py_RETURN_FALSE;
else if (errno == EINTR)
return NULL;
else
return PyErr_SetFromErrno(PyExc_OSError);
}
In both versions of the program (print vs. sys.stdout), sem_wait() is
being interrupted and is returning -1 with errno set to EINTR. This is
what I expected. Also, in both cases it seems that the loop is
(correctly) terminating with PyErr_CheckSignals() returning non-zero.
This makes sense too; the call is executing our signal handler, and then
returning -1 since our particular handler raises SystemExit.
However, I suspect that depending on the exact code executed
for the signal handler, errno may or may not wind up being reset in
some nested call of PyErr_CheckSignals(). I believe that the
error checking code below the do-while (where sem_wait() is called),
needed errno to have the value set by sem_wait(), and the author
wasn't expecting anything else to have changed it. In the "print"
version, errno apparently winds up unchanged with EINTR, resulting in
the `return NULL' statement. In the "sys.stdout" version (and probably
many others), errno winds up being reset to 0, and the error handling
results in the `return PyErr_SetFromErrno(PyExc_OSError)' statement.
To patch this up, we can probably just save errno as, say, `wait_errno'
at the end of the loop body, and then use it within the error handling
block that follows. However, the rest of the code should probably be
checked for this type of issue.
--
components: Library (Lib)
messages: 89804
nosy: ryles
severity: normal
status: open
title: multiprocessing: handling of errno after signals in sem_acquire()
type: behavior
versions: Python 2.6, Python 2.7, Python 3.0, Python 3.1, Python 3.2
___
Python tracker
<http://bugs.python.org/issue6362>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6362] multiprocessing: handling of errno after signals in sem_acquire()
Changes by Ryan Leslie : -- nosy: +jnoller ___ Python tracker <http://bugs.python.org/issue6362> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24450] Add gi_yieldfrom calculated property to generator object
New submission from Benno Leslie: When a co-routine has delegated computation to another co-routine via "yield from", it is useful to be able to access the delegated-to co-routine. This proposed enhancement would add a new calculated property to the generator object called "gi_yieldfrom", which returns the delegated-to co-routine (or None). -- components: Interpreter Core messages: 245338 nosy: bennoleslie priority: normal severity: normal status: open title: Add gi_yieldfrom calculated property to generator object type: enhancement versions: Python 3.5 ___ Python tracker <http://bugs.python.org/issue24450> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24450] Add gi_yieldfrom calculated property to generator object
Benno Leslie added the comment: Attached is my first attempt at implementing this feature. It includes to implementation, updates to documentation, and added test cases. I'm not a regular contributor so look forward to any feedback on improving the patch. -- keywords: +patch Added file: http://bugs.python.org/file39706/gi_yieldfrom.v0.patch ___ Python tracker <http://bugs.python.org/issue24450> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24450] Add gi_yieldfrom calculated property to generator object
Benno Leslie added the comment: I've tried to address all the issues raised in the review of the first patch. -- Added file: http://bugs.python.org/file39725/gi_yieldfrom.v1.patch ___ Python tracker <http://bugs.python.org/issue24450> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25687] Error during test case and tearDown
New submission from Benno Leslie: I'm not sure if this should be considered a bug, but the behaviour is surprising and not clearly documented. I a have a very simple test that has an error during both the test case, and during tearDown. """ import unittest class Test(unittest.TestCase): def test_a(self): asdf def tearDown(self): asdf if __name__ == '__main__': unittest.main() """ When this occurs it is surprising (to me) that the output is: """ Ran 1 test in 0.000s FAILED (errors=2) """ In particular, the fact that has more errors than there are tests that have been run. Obviously in this very simple example it is clear what has happened, however in a test suite that has hundreds of test cases it is somewhat frustrating to have the number of failing test cases over-reported. (And of course in the real-world test suite that led to this the tearDown doesn't fail on every single test case like in this simplified example). Although there are definitely two errors occurring in my example, in all other cases, only the first error would be reported. e.g.: an error in setUp wouldn't run the test case, and only the first (potential) error in the testcase itself would occur. I think that either: 1/ The documentation of the tearDown method should clearly indicate that an error in tearDown would cause multiple errors to be reported for the single testCase, or. 2/ Change the implementation so that if there is an exception during tearDown, and there was already an error in the test case only the first error is reported. 2 is probably a non-starter given backwards compatibility concerns (even assuming anyone else thinks this is the best behaviour in this corner case). I'd be happy to try and draft something for #1 if that is the best action. -- components: Library (Lib) messages: 255045 nosy: bennoleslie priority: normal severity: normal status: open title: Error during test case and tearDown type: behavior versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4 ___ Python tracker <http://bugs.python.org/issue25687> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19662] smtpd.py should not decode utf-8
New submission from Leslie P. Polzer: http://hg.python.org/cpython/file/3.3/Lib/smtpd.py#l289 as of now decodes incoming bytes as UTF-8. An SMTP server must not attempt to interpret characters beyond ASCII, however. Originally mail servers were not 8-bit clean, meaning they would only guarantee the lower 7 bits of each octet to be preserved. However even then they were not expected to choke on any input because of attempts to decode it into a specific extended charset. Whenever a mail server does not need to interpret data (like base64-encoded auth information) it is simply left alone and passed through. I am not aware of the reasons that caused the current state, but to correct this behavior and make it possible to support the 8BITMIME feature I suggest decoding received bytes as latin1, leaving it to the user to reinterpret it as UTF-8 or whatever charset they need. Any other simple extended encoding could be used for this, but latin1 is the default in asynchat. The documentation should also mention charset handling. I'll be happy to submit a patch for both code and docs. -- components: Library (Lib) messages: 203467 nosy: skypher priority: normal severity: normal status: open title: smtpd.py should not decode utf-8 type: enhancement versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3 ___ Python tracker <http://bugs.python.org/issue19662> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12816] smtpd uses library outside of the standard libraries
Changes by Leslie P. Polzer : -- nosy: +lpolzer ___ Python tracker <http://bugs.python.org/issue12816> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16462] smtpd should return greeting
Changes by Leslie P. Polzer : -- nosy: +lpolzer ___ Python tracker <http://bugs.python.org/issue16462> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8503] smtpd SMTPServer does not allow domain filtering
Changes by Leslie P. Polzer : -- nosy: +lpolzer ___ Python tracker <http://bugs.python.org/issue8503> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3802] smtpd.py __getaddr insufficient handling
Changes by Leslie P. Polzer : -- nosy: +lpolzer ___ Python tracker <http://bugs.python.org/issue3802> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19662] smtpd.py should not decode utf-8
Leslie P. Polzer added the comment: Patch attached. This also adds some more charset clarification to the docs and corrects a minor spelling issue. It is also conceivable that we add a charset attribute to the class. This should have the safe default of latin1, and some notes in the docs that setting this to utf-8 (and probably other utf-* encodings) is not really standards-compliant. -- keywords: +patch Added file: http://bugs.python.org/file32719/smtpd_charset_latin1.diff ___ Python tracker <http://bugs.python.org/issue19662> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19662] smtpd.py should not decode utf-8
Leslie P. Polzer added the comment: Since this is my first contribution I'm not entirely sure about the fine details of backwards compatibility in Python, so please forgive me if I'm totally missing the mark here. There are facilities in smtpd's parent class asynchat that perform the necessary conversions automatically if the user sets an encoding, so smtpd should be adjusted to rely on that and thus give the user the opportunity to choose for themselves. Then it boils down to breaking backwards compatibility by setting a default encoding, which could be none as you suggest or latin1 as I suggest; either will probably be painful for current users. My take here is that whoever is using this code for their SMTP server and hasn't given the encoding issues any thought will need to take a look at their code in that respect anyway, so IMHO a break with compatibility might be a bit painful but necessary. If you agree then I will gladly rework the patch to have smtpd work with an underlying byte stream by default, rejecting anything non-ASCII where necessary. Later patches could bring 8BITMIME support to smtpd, with charset conversion as specified by the MIME metadata. -- ___ Python tracker <http://bugs.python.org/issue19662> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19678] smtpd.py: channel should be passed to process_message
New submission from Leslie P. Polzer: process_message needs to have access to the channel state since it needs to make decisions based on the authentication or transport associated with the channel. It should be either the first or the last arg. I can provide a patch for this. Should backwards compatibility be achieved by using the inspect module to check the signature of process_message and call it accordingly? -- components: Library (Lib), email messages: 203609 nosy: barry, lpolzer, r.david.murray priority: normal severity: normal status: open title: smtpd.py: channel should be passed to process_message type: enhancement versions: Python 3.4 ___ Python tracker <http://bugs.python.org/issue19678> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19679] smtpd.py: implement ESMTP status messages
New submission from Leslie P. Polzer: ESMTP status messages (of the form "xab x.y.z test") can be added easily to the current status message strings emitted by the SMTP server and channel classes. They are not harmful if the user's server only intends to support plain HELO-SMTP I will provide a patch within due time unless someone disagrees. -- components: Library (Lib), email messages: 203610 nosy: barry, lpolzer, r.david.murray priority: normal severity: normal status: open title: smtpd.py: implement ESMTP status messages type: enhancement versions: Python 3.4 ___ Python tracker <http://bugs.python.org/issue19679> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19679] smtpd.py (SMTPChannel): get rid of "conn" attribute
Leslie P. Polzer added the comment: The contents of "conn" are already stored by asyncore's "socket" attribute, so there doesn't seem to be a need to keep it around. We should deprecate its usage and refer the user to the "socket" attribute. Furthermore I suggest renaming the "conn" argument to "socket" to make its semantics clearer. -- title: smtpd.py: implement ESMTP status messages -> smtpd.py (SMTPChannel): get rid of "conn" attribute ___ Python tracker <http://bugs.python.org/issue19679> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19679] smtpd.py (SMTPChannel): implement enhanced status codes
Leslie P. Polzer added the comment: Sorry for the confusion, the second comment (#msg203622) should actually have been a separate ticket. Since you'd like to preserve "conn" I will just change the title of this ticket. -- title: smtpd.py (SMTPChannel): get rid of "conn" attribute -> smtpd.py (SMTPChannel): implement enhanced status codes ___ Python tracker <http://bugs.python.org/issue19679> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19679] smtpd.py (SMTPChannel): implement enhanced status codes
Leslie P. Polzer added the comment: I am indeed referring to the enhanced status codes proposed in RFC 3463. This would just entail adding information to the status codes, converting them from the format " " to " ". In this it doesn't seem necessary to differentiate by HELO/EHLO; neither is it demanded by the standard nor does it result in an incompatible form of traditional HELO status messages. HELO clients should just interpret the first three digits and regard the rest as part of the human readable informal section. -- ___ Python tracker <http://bugs.python.org/issue19679> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19679] smtpd.py (SMTPChannel): implement enhanced status codes
Leslie P. Polzer added the comment: Sounds reasonable, I will propose a patch soon. Thanks! -- ___ Python tracker <http://bugs.python.org/issue19679> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
