[issue46667] SequenceMatcher & autojunk - false negative
New submission from Jonathan :
The following two strings are identical other than the text "UNIQUESTRING".
UNIQUESTRING is at the start of first and at the end of second.
Running the below gives the following output:
0.99830220713073
0.99830220713073
0.023769100169779286 # ratio
0.99830220713073
0.99830220713073
0.023769100169779286 # ratio
As you can see, Ratio is basically 0. Remove either of the UNIQUESTRING pieces
and it goes up to 0.98 (correct)... Remove both and you get 1.0 (correct)
```
from difflib import SequenceMatcher
first = """
UNIQUESTRING
Lorem Ipsum is simply dummy text of the printing and typesetting industry.
Lorem Ipsum has been the industry's standard dummy text ever since the 1500s,
when an unknown printer took a galley of type and scrambled it to make a type
specimen book. It has survived not only five centuries, but also the leap into
electronic typesetting, remaining essentially unchanged. It was popularised in
the 1960s with the release of Letraset sheets containing Lorem Ipsum passages,
and more recently with desktop publishing software like Aldus PageMaker
including versions of Lorem Ipsum
"""
second = """
Lorem Ipsum is simply dummy text of the printing and typesetting industry.
Lorem Ipsum has been the industry's standard dummy text ever since the 1500s,
when an unknown printer took a galley of type and scrambled it to make a type
specimen book. It has survived not only five centuries, but also the leap into
electronic typesetting, remaining essentially unchanged. It was popularised in
the 1960s with the release of Letraset sheets containing Lorem Ipsum passages,
and more recently with desktop publishing software like Aldus PageMaker
including versions of Lorem Ipsum UNIQUESTRING
"""
sm = SequenceMatcher(None, first, second, autojunk=False)
print(sm.real_quick_ratio())
print(sm.quick_ratio())
print(sm.ratio())
print()
sm2 = SequenceMatcher(None, second, first, autojunk=False)
print(sm2.real_quick_ratio())
print(sm2.quick_ratio())
print(sm2.ratio())
```
If I add `autojunk=False`, then I get a correct looking ratio (0.98...),
however from my reading of the autojunk docs, UNIQUESTRING shouldn't be
triggering it. Furthermore, looking in the code, as far as I can see autojunk
is having no effect...
Autojunk considers these items to be "popular" in that string:
`{'n', 'p', 'a', 'h', 'e', 'u', 'I', 'r', 'k', 'g', 'y', 'm', 'c', 'd', 't',
'l', 'o', 's', ' ', 'i'}`
If I remove UNIQUESTRING from `first`, this is the autojunk popular set:
`{'c', 'p', 'a', 'u', 'r', 'm', 'k', 'g', 'I', 'd', ' ', 'o', 'h', 't', 'e',
'i', 'l', 's', 'y', 'n'}`
They're identical!
In both scenarios, `b2j` is also identical.
I don't pretend to understand what the module is doing in any detail, but this
certainly seems like a false positive/negative.
Python 3.8.10
--
components: Library (Lib)
messages: 412673
nosy: jonathan-lp
priority: normal
severity: normal
status: open
title: SequenceMatcher & autojunk - false negative
type: behavior
versions: Python 3.8
___
Python tracker
<https://bugs.python.org/issue46667>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46667] SequenceMatcher & autojunk - false negative
Jonathan added the comment: (Like the idiot I am, the example code is wrong. `autojunk` parameter should *not* be set for either of them to get the stated wrong results). In place of "UNIQUESTRING", any unique 3 character string triggers it (QQQ, EEE, ZQU...). And in those cases you get a ratio of 0.008! (and 0.993 in the other direction!) -- ___ Python tracker <https://bugs.python.org/issue46667> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46667] SequenceMatcher & autojunk - false negative
Jonathan added the comment: Gah. I mean 0.008 in both directions. I'm just going to be quiet now. :-) -- ___ Python tracker <https://bugs.python.org/issue46667> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46667] SequenceMatcher & autojunk - false negative
Jonathan added the comment: I still don't get how UNIQUESTRING is the longest even with autojunk=True, but that's an implementation detail and I'll trust you that it's working as expected. Given this, I'd suggest the following then: * `Autojunk=False` should be the default unless there's some reason to believe SequenceMatcher is mostly used for code comparisons. * If - for whatever reason - the default can't be changed, I'd suggest a nice big docs "Warning" (at a minimum a "Note") saying something like "The default autojunk=True is not suitable for normal string comparison. See autojunk for more information". * Human-friendly doc explanation for autojunk. The current explanation is only going to be helpful to the tiny fraction of users who understand the algorithm. Your explanation is a good start: "Autojunk was introduced as a way to greatly speed comparing files of code, viewing them as sequences of lines. But it more often backfires when comparing strings (viewed as sequences of characters)" Put simply: The current docs aren't helpful to users who don't have text matching expertise, nor do they emphasise the huge caveat that autojunk=True raises. -- ___ Python tracker <https://bugs.python.org/issue46667> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46802] Wrong result unpacking binary data with ctypes bitfield.
New submission from Jonathan : I have issues unpacking binary data, produced by C++. The appended jupyter notebook shows the problem. It is also uploaded to github gist: https://gist.github.com/helo9/04125ae67b493e505d5dce4b254a2ccc -- components: ctypes files: ctypes_bitfield_problem.ipynb messages: 413559 nosy: helo9 priority: normal severity: normal status: open title: Wrong result unpacking binary data with ctypes bitfield. type: behavior versions: Python 3.10 Added file: https://bugs.python.org/file50633/ctypes_bitfield_problem.ipynb ___ Python tracker <https://bugs.python.org/issue46802> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46802] Wrong result unpacking binary data with ctypes bitfield.
Change by Jonathan : -- nosy: -helo9 ___ Python tracker <https://bugs.python.org/issue46802> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46802] Wrong result unpacking binary data with ctypes bitfield.
Jonathan added the comment: True, have to admit, that I forgot to search first, that really looks like it is the same problem, especially when looking at https://bugs.python.org/msg289212. Would say this one can be closed. -- nosy: +helo9 stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue46802> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2628] ftplib Persistent data connection
Changes by Jonathan <[EMAIL PROTECTED]>: Added file: http://bugs.python.org/file10311/ftplib.py.blockmode.patch.2 __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2628> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2628] ftplib Persistent data connection
Changes by Jonathan <[EMAIL PROTECTED]>: Added file: http://bugs.python.org/file10312/ftplib.rst.blockmode.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2628> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2628] ftplib Persistent data connection
Jonathan <[EMAIL PROTECTED]> added the comment: I've attached two new files. The first swaps the array.array usage for struct.unpack. The second simply modifies the rst documentation. I'm not sure how we'd do any tests for FTP without making use of an actual server. In a quick check of servers, MadGoat (for OpenVMS) was the only BLOCK-supporting server I found; neither vsftpd nor proftpd support BLOCK. (I didn't check wuftpd.) Sadly, I've no publicly accessible servers available to me for others to test against. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2628> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2628] ftplib Persistent data connection
New submission from Jonathan <[EMAIL PROTECTED]>: About a year ago I found myself fighting a broken FTP server that couldn't handle multiple passive data transfers through a firewall or NATed connection. Thankfully, this same problem server supports block transmission mode, which allows a client to create a single data connection for transferring multiple files. I've attached a patch (and sample debug output) for the latest trunk. Might this be useful to anyone else in any way? I realize any MODE option rather than S is widely unsupported and possibly an edge case, but getting this into trunk would be preferable. We've been running this code under Python2.3 for nearly a year -- the jobs run several times per hour -- and are extremely happy with the results. -- components: Library (Lib) files: ftplib.py.blockmode.patch keywords: patch messages: 65454 nosy: jbell severity: normal status: open title: ftplib Persistent data connection type: feature request versions: Python 2.6 Added file: http://bugs.python.org/file10025/ftplib.py.blockmode.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2628> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2628] ftplib Persistent data connection
Jonathan <[EMAIL PROTECTED]> added the comment: Here's the debug output when the transfers are going well. Added file: http://bugs.python.org/file10026/debug.log __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2628> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2628] ftplib Persistent data connection
Jonathan <[EMAIL PROTECTED]> added the comment: Yeah, I'm glad to see a test suite. I've only skimmed the test, but it seems like an excellent starting point. Initial thoughts for updating the tests: - Need a cmd_mode to handle the MODE command. - Suite cmd_retr logic needs to keep dtp connection open and write a simple header depending on MODE. Due to the Thanksgiving holiday, I may be without network access (or time) until next week, however. ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2628> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37111] Logging - Inconsistent behaviour when handling unicode
New submission from Jonathan :
Python is inconsistent in how it handles errors that have some unicode
characters. It works to screen but fails to log
This works:
```
>>> import logging
>>> logging.error('จุด1')
ERROR:root:จุด1
```
The following breaks:
```
>>> import logging
>>> logging.basicConfig(filename='c:\\my_log.log')
>>> logging.error('จุด1')
```
This raises a unicode error:
UnicodeEncodeError: 'charmap' codec can't encode characters in position 11-13:
character maps to
Python 3.6.3
Given that the file created by the logger is utf-8, it's unclear why it doesn't
work.
I found a workaround by using a Handler, but surely the loggers should all work
the same way so that people don't get unpleasant surprises that even more
painful to debug when things only break in certain logging modes?
--
messages: 344053
nosy: jonathan-lp
priority: normal
severity: normal
status: open
title: Logging - Inconsistent behaviour when handling unicode
versions: Python 3.6
___
Python tracker
<https://bugs.python.org/issue37111>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37111] Logging - Inconsistent behaviour when handling unicode
Jonathan added the comment: It definitely claims to be "utf-8" in NotePad++. I've attached it if you want to double-check. (Windows 7) -- Added file: https://bugs.python.org/file48380/my_log.log ___ Python tracker <https://bugs.python.org/issue37111> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37111] Logging - Inconsistent behaviour when handling unicode
Jonathan added the comment: Thank you for your comments but this wasn't a question and I maintain this is a bug, or at least undesirable behaviour. I'd consider it a bug in my own software. Reasoning: * It's an inconsistent default with the logging to screen. This causes more complexity for users when their bug is intermittent. * Despite your assertion, it's not documented anywhere on the logging docs (I did check before creating this bug when trying to figure out what's going on) - The word "utf" or "unicode" doesn't appear on the logging page, or any of the two tutorials, or the logging.handlers page. There's something in the cookbook but that's about BOM's. * Much of the world's native characters won't log to ASCII Per this page: https://docs.python.org/3/howto/unicode.html "UTF-8 is one of the most commonly used encodings, and Python often defaults to using it." > People have been using logging, on Windows, without problems, for years, > often using utf-8 to encode their log files. I'm afraid this line of reasoning is suffering from selection bias, cherry picking, confirmation bias, and probably some others too. Clearly people have had problems before because it was from one of those folks I took the solution. Doing something as basic as logging unicode shouldn't require knowledge of "handlers" - that's failing "simple is better than complex". -- ___ Python tracker <https://bugs.python.org/issue37111> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37111] Logging - Inconsistent behaviour when handling unicode
Jonathan added the comment: > Did you look at the basicConfig documentation before raising this issue? This seems like an attempt at victim blaming. But yes, I did. In fact, this is now the third time I've looked at that page - once before raising this issue, once before my previous reply, and now. I note that your example and nothing like your example is anywhere on that page. The word "encoding" doesn't appear anywhere on the page either. Sure "stream" is on there, but then you need to know about streams and make the association with logging which I apparently don't. You have to remember not everyone has your level of proficiency in the language. In fact, most Python users don't. Lets put this another way - is there a reason NOT to have Unicode logging as the default? Clearly Unicode was important enough for Guido-et-al to decide to throw Python 2 under the bus. I've listed the advantages of changing it, what are the disadvantages? -- ___ Python tracker <https://bugs.python.org/issue37111> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37111] Logging - Inconsistent behaviour when handling unicode
Jonathan added the comment: > I have no idea what you mean by this. I don't see how I can be clearer. What are the reasons for NOT implementing logging to file be unicode as a default? Logging to screen is unicode as a default. What are the reasons for not wanting consistency? > A simple Internet search for "basicConfig encoding" yields for me as the > second result this Stack Overflow question Indeed, and it was from that question I got my solution in fact. The problem was the 30-60 minutes I wasted before that trying to figure out why my program was crashing and why it was only crashing *sometimes*. I'd written the logging part of the program a year ago and not really touched it since, so the logging module being a possible culprit was not even in my mind when the program crashed. > As my example illustrated, it's quite easy to log Unicode in log files. Yes, things are easy when you know it's necessary. It's the process of discovery that's an unnecessary waste of people's time. That's why I raised this and that's why I would consider this a bug in my own software. It's inconsistent, it invites problems, and it wastes peoples time. -- ___ Python tracker <https://bugs.python.org/issue37111> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37111] Logging - Inconsistent behaviour when handling unicode
Jonathan added the comment: > Learning is not a waste of time. You're entitled to your opinion, but this is > not a bug in logging. We'll have to agree to disagree. I agree and value learning a great deal. However learning should happen on your own time, NOT when a program crashes randomly and tries taking you down the rabbit hole. I like learning but not about unrelated things when I'm trying to do useful work. Fine, if you don't consider this a bug, consider it a feature request. "User would like Python logging of Unicode characters to be consistent" is not an unreasonable request. -- status: closed -> open type: behavior -> enhancement ___ Python tracker <https://bugs.python.org/issue37111> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37563] Documentation - default for StreamHandler
New submission from Jonathan : https://docs.python.org/2/library/logging.handlers.html https://docs.python.org/3/library/logging.handlers.html Both say: """class logging.StreamHandler(stream=None) Returns a new instance of the StreamHandler class. If stream is specified, the instance will use it for logging output; otherwise, sys.stderr will be used.""" Surely that means from an user perspective that the default is actually `sys.stderr`, not `None`? -- assignee: docs@python components: Documentation messages: 347677 nosy: docs@python, jonathan-lp priority: normal severity: normal status: open title: Documentation - default for StreamHandler versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue37563> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37563] Documentation - default for StreamHandler
Change by Jonathan : -- status: -> open ___ Python tracker <https://bugs.python.org/issue37563> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37563] Documentation - default for StreamHandler
Jonathan added the comment: > The devil is in the detail. If stream=sys.stderr is specified, that takes > effect at import time. If stream=None is specified and the implementation > chooses to treat that as sys.stderr, that takes effect at the time of the > call. The two are not equivalent. But this isn't what the prose says at all. You're right, the prose clearly say that the default is sys.stderr, however the code doesn't show that, and many people won't read the prose (I don't a lot of the time), they'll only look at the code snippet because that's all they think they need. The code-snippet claims that the default is None, which from a user perspective isn't true. Again I point out that the documentation is for users, not implementers. We users Do. Not. Care. about how wonderfully clever your implementation is, we care about how it actually works. Whatever Rube-Goldbergian implementation details there are behind the scenes are of no interest to us. Yet again: There's a standard for documenting defaults for keyword arguments, I would ask that it please be used consistently to help us users. Fine, lets try this another way - does anyone else have opinions on this? What's the convention for documentation defaults? -- ___ Python tracker <https://bugs.python.org/issue37563> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37563] Documentation - default for StreamHandler
Jonathan added the comment: > I'm not sure your tone is particularly constructive here. Apologies, my bad. > Which code are you looking at? The documentation code: `class logging.StreamHandler(stream=None)`. Sorry, I don't know what you'd call that. I'm not referring to the code proper. > As far as I can remember, you're the first person to bring this up since > logging was added to Python in 2003. This is a fallacy. Just because no-one else has reported it doesn't mean it hasn't caused a problem. I mean, I'm sure there are plenty of spelling errors/typos in the docs that no-one has reported for years, it doesn't mean they shouldn't be fixed when raised. It's also assuming you have seen and remember every single bug report related to this from the past 16 years which, nothing personal, seems incredibly unlikely given how poor humans are at remembering things in the first place. > And are you presuming to speak for all Python users here? I'm presuming to speak for end-users yes, why not? I did ask for other input too you'll note. After a few decades of practice I'm fairly decent at getting into the headspace of users (of which I am one in this case), and I know it's something many developers don't really do well. A common mistake we developers make is to assume that everyone knows what we know and thinks like us. -- ___ Python tracker <https://bugs.python.org/issue37563> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37563] Documentation - default for StreamHandler
Jonathan added the comment: >What fallacy? You appeared to be saying (to paraphrase) "no-one else has ever reported this, so it's never been a problem". That's a fallacy. > I was responding to "does anyone else have opinions on this?" I was asking if anyone else wanted to chime in with an opinion. > There are numerous examples in the stdlib where None is passed in and some > other value (e.g. 'utf-8' for an encoding) are used as a default Then for clarity's purpose I'd suggest those be changed too, but that's another ticket. -- ___ Python tracker <https://bugs.python.org/issue37563> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35301] python.exe crashes - lzma?
New submission from Jonathan : Python 3.6.3 on Windows 7 64 bit. I keep getting intermittent crashes of Python.exe with my project. The error is below and as best I can see seems to be indicating the issue with the LZMA module. All I'm doing is using it to compress web-pages (typically XML ones). Problem is, I can't replicate it. When I try and run the process again, the page that previously crashed it now works. I've had this happen three times so far (out of about 150,000 pages downloaded and compressed). My use of LZMA is quite simply: from lzma import compress compress(page_content, preset=5) The pages it has crashed on so far (but which work fine now): http://www.geoportal.rlp.de/mapbender/php/wms.php?inspire=1&layer_id=35833&request=GetCapabilities&service=WMS&version=1.3.0 https://maps.runnymede.gov.uk/geoserver/planning/lp_ancient_woodland/wms?request=GetCapabilities&service=WMS&version=1.3.0 http://serviziogc.regione.fvg.it/geoserver/GEOLOGIA/wms?request=GetCapabilities&service=WMS&version=1.3.0 I took a mini dump (attached) and full dump using Process Explorer. The full dump is 100MB compressed (1GB uncompressed). Problem signature: Problem Event Name: APPCRASH Application Name: python.exe Application Version: 3.6.3150.1013 Application Timestamp:59d3d3a3 Fault Module Name:_lzma.pyd Fault Module Version: 3.6.3150.1013 Fault Module Timestamp: 59d3d343 Exception Code: c005 Exception Offset: 0002e368 OS Version: 6.1.7601.2.1.0.768.3 Locale ID:2057 Additional Information 1: d00b Additional Information 2: d00be1cd6ce69f4b081e66c649a14f90 Additional Information 3: 8c17 Additional Information 4: 8c17b9c4b6a39b7e31bf71b3b1374f1b -- files: python-mini.7z messages: 330327 nosy: jonathan-lp priority: normal severity: normal status: open title: python.exe crashes - lzma? type: crash versions: Python 3.6 Added file: https://bugs.python.org/file47944/python-mini.7z ___ Python tracker <https://bugs.python.org/issue35301> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35301] python.exe crashes - lzma?
Jonathan added the comment: In total my python.exe process crashed 6 times with the exact same error, compressing about 60,000 pages, so about 1 in 10,000 failed. However, I have ran the same process monthly for the last several months and it has never done that before despite processing mostly the same sites and compressing similar quantities of pages. The code is completely unchanged from last month. Upon reflection, the only thing that has changed since then is that I am now running BOINC (https://boinc.berkeley.edu/) in the background on the machine. So I have just re-ran the entire process with BOINC suspended. Not a single crash! It seems that somehow BOINC and Python are interacting to cause Python to crash. -- ___ Python tracker <https://bugs.python.org/issue35301> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36120] Regression - Concurrent Futures
New submission from Jonathan : I'm using Concurrent Futures to run some work in parallel (futures.ProcessPoolExecutor) on windows 7 x64. The code works fine in 3.6.3, and 3.5.x before that. I've just upgraded to 3.7.2 and it's giving me these errors: Process SpawnProcess-6: Traceback (most recent call last): File "c:\_libs\Python37\lib\multiprocessing\process.py", line 297, in _bootstrap self.run() File "c:\_libs\Python37\lib\multiprocessing\process.py", line 99, in run self._target(*self._args, **self._kwargs) File "c:\_libs\Python37\lib\concurrent\futures\process.py", line 226, in _process_worker call_item = call_queue.get(block=True) File "c:\_libs\Python37\lib\multiprocessing\queues.py", line 93, in get with self._rlock: File "c:\_libs\Python37\lib\multiprocessing\synchronize.py", line 95, in __enter__ return self._semlock.__enter__() PermissionError: [WinError 5] Access is denied If I switch back to the 3.6.3 venv it works fine again. -- messages: 336649 nosy: jonathan-lp priority: normal severity: normal status: open title: Regression - Concurrent Futures versions: Python 3.7 ___ Python tracker <https://bugs.python.org/issue36120> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36120] Regression - Concurrent Futures
Jonathan added the comment: There's also this error too: Traceback (most recent call last): File "c:\_libs\Python37\lib\multiprocessing\process.py", line 297, in _bootstrap self.run() File "c:\_libs\Python37\lib\multiprocessing\process.py", line 99, in run self._target(*self._args, **self._kwargs) File "c:\_libs\Python37\lib\concurrent\futures\process.py", line 226, in _process_worker call_item = call_queue.get(block=True) File "c:\_libs\Python37\lib\multiprocessing\queues.py", line 94, in get res = self._recv_bytes() File "c:\_libs\Python37\lib\multiprocessing\synchronize.py", line 98, in __exit__ return self._semlock.__exit__(*args) OSError: [WinError 6] The handle is invalid -- ___ Python tracker <https://bugs.python.org/issue36120> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36120] Regression - Concurrent Futures
Jonathan added the comment: The "ProcessPoolExecutor Example" on this page breaks for me: https://docs.python.org/3/library/concurrent.futures.html -- ___ Python tracker <https://bugs.python.org/issue36120> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32595] Deque with iterable object as one object
New submission from Jonathan : Creating a deque with an iterable object creates extra overhead if you want to insert it as one element. e.g: import timeit test1 = ''' str = "x" * 10 lst = [str] ''' test2 = ''' str = "x" * 10 ''' print(timeit.timeit(test1, number=10)) print(timeit.timeit(test2, number=10)) If I want to add a string to a deque i will have to put it in a list first, this causes 23% slower performance in my case. There should be a deque where this overhead is not needed because in most cases this will be used in performance demanding cases. -- components: Library (Lib) messages: 310256 nosy: jonathandaniel priority: normal severity: normal status: open title: Deque with iterable object as one object type: performance versions: Python 2.7 ___ Python tracker <https://bugs.python.org/issue32595> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32595] Deque with iterable object as one object
Jonathan added the comment:
Hello,
I dont know why but yesterday when i appended it behaved like extend.
I should sleep more.
2018-01-18 20:27 GMT+00:00 Raymond Hettinger :
>
> Raymond Hettinger added the comment:
>
> Sorry Jonathan, this is the way the python containers work if they take an
> iterable input. In the case of a str, it is not possible for us to know
> whether you mean for deque('abc') to go it as three arguments or as one.
>
> FWIW, if you don't what to put the single element in a list, the API
> provides the append() method for adding scalars and extend() method for
> adding iterables:
>
>d = deque()
>d.append('abc')
>d.extend('abc')
>
> Note that lists behave the same way.
>
> --
> resolution: -> not a bug
> stage: -> resolved
> status: open -> closed
>
> ___
> Python tracker
> <https://bugs.python.org/issue32595>
> ___
>
--
___
Python tracker
<https://bugs.python.org/issue32595>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31727] FTP_TLS errors when use certain subcommands
Jonathan added the comment: Hi Christian - Can you not reproduce it from the code included in my original report? I think that's pretty much all that should be needed, as it's all I was using when I discovered the issue. I figure the hard part will be finding a FTP_TLS server to test with. -- status: pending -> open ___ Python tracker <https://bugs.python.org/issue31727> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33034] urllib.parse.urlparse and urlsplit not raising ValueError for bad port
Change by Jonathan : -- versions: +Python 3.5 ___ Python tracker <https://bugs.python.org/issue33034> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33034] urllib.parse.urlparse and urlsplit not raising ValueError for bad port
New submission from Jonathan :
(Confirmed in 2.7.14, 3.5.4, and 3.6.3)
I have this really bad URL from a crawl:
"http://Server=sde; Service=sde:oracle$sde:oracle11g:geopp; User=bodem;
Version=SDE.DEFAULT"
if I try and parse it with wither urlparse or urlsplit it works - no errors.
But when I try and get the port, I get a ValueError.
> from urllib.parse import urlparse
> r = urlparse('http://Server=sde; Service=sde:oracle$sde:oracle11g:geopp;
> User=bodem; Version=SDE.DEFAULT')
ParseResult(scheme='http', netloc='Server=sde;
Service=sde:oracle$sde:oracle11g:geopp; User=bodem; Version=SDE.DEFAULT',
path='', params='', query='', fragment='')
Ok, great, now to use the result:
> print(r.port)
Traceback (most recent call last):
File "", line 1, in
File "E:\Software\_libs\Python36\lib\urllib\parse.py", line 167, in port
port = int(port, 10)
ValueError: invalid literal for int() with base 10:
'oracle$sde:oracle11g:geopp; User=bodem; Version=SDE.DEFAULT'
I'm not a Python Guru, but to me at least it's inconsistent with how every
other Python Function works. In all other builtin functions I've used it would
fail with the exception when I ran the function, not when I try and get the
results. This caused a good few minutes of head-scratching while I tried to
debug why my try/except wasn't catching it.
This inconsistency makes the results more difficult to use. Now a user needs to
wrap all calls to the *results* in a try/except, or write an entire function
just to "read" the results into a won't-except tuple/dict. Seems sub-optimal.
(May relate to: https://bugs.python.org/issue20059)
--
messages: 313475
nosy: jonathan-lp
priority: normal
severity: normal
status: open
title: urllib.parse.urlparse and urlsplit not raising ValueError for bad port
versions: Python 2.7, Python 3.6
___
Python tracker
<https://bugs.python.org/issue33034>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33034] urllib.parse.urlparse and urlsplit not raising ValueError for bad port
Jonathan added the comment:
Arguably the same logic applies to out-of-range ports:
> a = urlsplit('http://example.com:444')
> a.port
Traceback (most recent call last):
File "", line 1, in
File "E:\Software\_libs\Python36\lib\urllib\parse.py", line 169, in port
raise ValueError("Port out of range 0-65535")
ValueError: Port out of range 0-65535
--
___
Python tracker
<https://bugs.python.org/issue33034>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33034] urllib.parse.urlparse and urlsplit not raising ValueError for bad port
Jonathan added the comment:
Interesting conversation
As I see it, there are two ways to solve this, both discussed above:
A) Python can compute and ValueError at parse-time
B) Python can ValueError at property-call time. (Current method)
But both have Advantages/Disadvantages.
A - Pros) - The function is more consistent with all the other Python builtins
(well, the one's I've dealt with).
A - Pros) - Not carrying around a "bad" port.
A - Cons) - As Matt suggests, we could be "taking something from the user"
because they may want the other values. (although in that case, the current
semi-related behaviour: "Unmatched square brackets in the netloc attribute will
raise a ValueError" is also potentially taking something from the user).
B - Pros) - User gets the other values even if they don't get the port.
B - Cons) - User needs to have more Try/Excepts in the code (whereever you may
access the property), or to write their own wrapper function.
Given the above, of the those options I still think option (A) is better. The
other values may have a reduced worth if the port is invalid (depending on the
user's goal).
May I suggest a third option:
C) A flag for urlsplit/urlparse to indicate we don't want to do port
validation, and to just return whatever it thinks is there.
(example.com:3293849038 would return 3293849038. example.com:gibberish would
return "gibberish" etc).
This way the user can choose what behaviour they want with the bad port and
test the value of the port themselves if they care (something I was going to do
anyway before I discovered it was included in the builtin). Some of the url
quoting functions have a similar flag ("errors") for handling bad data
transparently or not.
--
___
Python tracker
<https://bugs.python.org/issue33034>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33034] urllib.parse.urlparse and urlsplit not raising ValueError for bad port
Jonathan added the comment: Thanks for the thoughts. If only the exception message changes, that doesn't really address the issue that caused me to raise this - namely that it seems to be inconsistent with how almost every other Python builtin function I've used works. I have to defer to you folks who know how feasible changing any of that is - I can see your reasoning. One quick observation - on glancing over the patch, it seems to only be for urlparse, but this happens to urlsplit too. Or does urlsplit import from that function (as I said, I only glanced). -- ___ Python tracker <https://bugs.python.org/issue33034> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33172] Update built-in version of SQLite3
New submission from Jonathan : The current version of SQLite (in Python 3.6) is 3.7.17 which was released almost 5 years ago - https://www.sqlite.org/releaselog/3_7_17.html Given that user updating of the version of SQLite used by Python is something of a pain (and the process is different across platforms (*and* different again for virtual-envs across platforms)), can the built-in version please be updated to a more recent version? This will allow usage of new SQLite features and users can benefit from a lot of performance enhancements/optimisations too. SQLite has excellent backwards compatibility, so except for any regressions (and they run over a hundred million tests per release to keep them to a minimum), any newer version will be backwards compatible with that version. Thanks -- messages: 314610 nosy: jonathan-lp priority: normal severity: normal status: open title: Update built-in version of SQLite3 versions: Python 3.6 ___ Python tracker <https://bugs.python.org/issue33172> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31727] FTP_TLS errors when
New submission from Jonathan :
Using Python 3.6.3.
The below issue only happens with FTP_TLS. It works fine via a plain FTP
connection.
I am connecting to an FTP to using `ftplib` via FTP_TLS.
ftps = ftplib.FTP_TLS('example.com', timeout=5)
# TLS is more secure than SSL
ftps.ssl_version = ssl.PROTOCOL_TLS
# login before securing control channel
ftps.login('usern...@example.com', 'password')
# switch to secure data connection
ftps.prot_p()
# Explicitly set Passive mode
ftps.set_pasv(True)
This all works. I can send `ftps.mkd('mydir')` (make directory) and
`ftps.cwd('mydir')` (change working dir), and both work fine.
But if I send **any** of these (they're all basically synonyms as far as I can
tell):
ftps.dir()
ftps.nlst()
ftps.retrlines('LIST')
ftps.retrlines('MLSD')
Then I get back an exception (below also includes all ftplib debug info as
well; generally matches up with what FileZilla shows too):
*cmd* 'AUTH TLS'
*put* 'AUTH TLS\r\n'
*get* '234 AUTH TLS OK.\n'
*resp* '234 AUTH TLS OK.'
*cmd* 'USER usern...@example.com'
*put* 'USER usern...@example.com\r\n'
*get* '331 User usern...@example.com OK. Password required\n'
*resp* '331 User usern...@example.com OK. Password required'
*cmd* 'PASS **'
*put* 'PASS **\r\n'
*get* '230 OK. Current restricted directory is /\n'
*resp* '230 OK. Current restricted directory is /'
*cmd* 'PBSZ 0'
*put* 'PBSZ 0\r\n'
*get* '200 PBSZ=0\n'
*resp* '200 PBSZ=0'
*cmd* 'PROT P'
*put* 'PROT P\r\n'
*get* '200 Data protection level set to "private"\n'
*resp* '200 Data protection level set to "private"'
*cmd* 'MKD mydir'
*put* 'MKD mydir\r\n'
*get* '257 "mydir" : The directory was successfully created\n'
*resp* '257 "mydir" : The directory was successfully created'
*cmd* 'CWD mydir'
*put* 'CWD mydir\r\n'
*get* '250 OK. Current directory is /mydir\n'
*resp* '250 OK. Current directory is /mydir'
*cmd* 'TYPE A'
*put* 'TYPE A\r\n'
*get* '200 TYPE is now ASCII\n'
*resp* '200 TYPE is now ASCII'
*cmd* 'PASV'
*put* 'PASV\r\n'
*get* '227 Entering Passive Mode (8,8,8,8,8,8)\n'
*resp* '227 Entering Passive Mode (8,8,8,8,8,8)'
*cmd* 'MLSD'
*put* 'MLSD\r\n'
*get* '150 Accepted data connection\n'
*resp* '150 Accepted data connection'
Traceback (most recent call last):
File "c:\my_script.py", line 384, in run_ftps
ftps.retrlines('MLSD')
File "c:\libs\Python36\lib\ftplib.py", line 485, in retrlines
conn.unwrap()
File "C:\libs\Python36\lib\ssl.py", line 1051, in unwrap
s = self._sslobj.unwrap()
File "C:\libs\Python36\lib\ssl.py", line 698, in unwrap
return self._sslobj.shutdown()
OSError: [Errno 0] Error
The same FTP command (LIST) works fine via filezilla.
The closest thing I can find with googling is this:
https://bugs.python.org/msg253161 - and I'm not sure if it's related or
relevant.
Short version: What does "OSError: [Errno 0] Error" actually mean, and how do I
list my directory contents?
--
messages: 303914
nosy: jonathan-lp
priority: normal
severity: normal
status: open
title: FTP_TLS errors when
versions: Python 3.6
___
Python tracker
<https://bugs.python.org/issue31727>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31727] FTP_TLS errors when
Jonathan added the comment: Just tested this with Python 3.7.0a1. I'm afraid it makes no difference. Exact same error: *cmd* 'LIST' *put* 'LIST\r\n' *get* '150 Accepted data connection\n' *resp* '150 Accepted data connection' Traceback (most recent call last): File "c:\backup_script.py", line 385, in run_ftps ftps.dir() File "c:\Python37\lib\ftplib.py", line 575, in dir self.retrlines(cmd, func) File "c:\Python37\lib\ftplib.py", line 485, in retrlines conn.unwrap() File "c:\Python37\lib\ssl.py", line 1059, in unwrap s = self._sslobj.unwrap() File "c:\Python37\lib\ssl.py", line 706, in unwrap return self._sslobj.shutdown() OSError: [Errno 0] Error -- ___ Python tracker <https://bugs.python.org/issue31727> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34011] Default preference not given to venv DLL's
New submission from Jonathan : I don't know if this is a bug or an odd design decision or just something that hasn't been considered or maybe I'm missing something. On Windows and I create a venv with Python 3.6.3: > python -m venv venv This creates a subdirectory called /venv. Inside this, there's: ./venv/Scripts/sqlite3.dll This is the sqlite library - except it's not, because Python isn't using this file. If I upgrade this library by replacing it with a newer sqlite3.dll version, Python keep using the original version of the library. Turns out, Python is by default reading the DLL in the root Python install: c:\Python36\DLLs\sqlite3.dll Now, I can change that file and sure enough my VENV (*all* VENVs) then get the newer version of SQLite, or I can delete that file and the VENV's will all use their local versions, or I can possibly play with some sys.path to try and get the VENV version loaded first. But this raises a few questions: 1) This seems like a rather odd default - copying a file that is never used by default. 2) Surely either the correct option is to use the VENV version by default? 3) Otherwise, what's the point in copying across this DLL file into the VENV by default? -- messages: 320765 nosy: jonathan-lp priority: normal severity: normal status: open title: Default preference not given to venv DLL's versions: Python 3.6 ___ Python tracker <https://bugs.python.org/issue34011> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7292] Multiprocessing Joinable race condition?
Jonathan added the comment: I can't even find the code I was having issues with anymore and I'm not doing anything related to this right now. So, unless Ramchandra can still reproduce this I'm going to say go ahead and close it. -- status: pending -> open ___ Python tracker <http://bugs.python.org/issue7292> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7292] Multiprocessing Joinable race condition?
New submission from Jonathan :
In multiprocessing.JoinableQueue when task_done is called
self._unfinished_tasks.acquire(False) is called non-blocking. My
program reliably crashes with the "task_done() called too many times"
message. If I change the .acquire() call to True (blocking) then my
program no longer crashes and I have not noticed any adverse effects. I
don't know if this would be considered a race condition or something
else but it does lead to crashes even in correct use scenarios.
(Code snippet follows for context, line 292 is the critical one for me)
def task_done(self):
self._cond.acquire()
try:
if not self._unfinished_tasks.acquire(False):
raise ValueError('task_done() called too many times')
if self._unfinished_tasks._semlock._is_zero():
--
components: None
messages: 95059
nosy: jonat...@kc8onw.net
severity: normal
status: open
title: Multiprocessing Joinable race condition?
type: behavior
versions: Python 3.1
___
Python tracker
<http://bugs.python.org/issue7292>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4938] Pdb cannot access doctest source in postmortem
Jonathan added the comment:
#!/usr/bin/env python
# This does what you want. Change pydb for pdb if you prefer it.
import pydb
import doctest
import sys
sys.excepthook = pydb.exception_hook
try:
doctest.testfile("story.txt",verbose=False,raise_on_error=True)
except doctest.UnexpectedException, failure:
pass
exc_info = failure.exc_info
raise exc_info[0], exc_info[1], exc_info[2]
--
nosy: +jonathan.cervidae
___
Python tracker
<http://bugs.python.org/issue4938>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4938] Pdb cannot access doctest source in postmortem
Jonathan added the comment:
#!/usr/bin/env python
# Slight mistake in last post
import pydb
import doctest
import sys
sys.excepthook = pydb.exception_hook
try:
doctest.testfile("story.txt",verbose=False,raise_on_error=True)
except doctest.UnexpectedException, failure:
exc_info = failure.exc_info
raise exc_info[0], exc_info[1], exc_info[2]
--
___
Python tracker
<http://bugs.python.org/issue4938>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5997] strftime is broken
New submission from Jonathan :
[...@jaydee Development]$ cat is-strftime-broken.py
#!/usr/bin/env python
import subprocess
import time
date_process = subprocess.Popen(
("date", "+%x"), stdout=subprocess.PIPE)
from_date_command = date_process.communicate()[0].rstrip()
from_strftime = time.strftime("%x")
print "Date command returns %s, strftime returns %s" % (
from_date_command, from_strftime )
[...@jaydee Development]$ python ./is-strftime-broken.py
Date command returns 11/05/09, strftime returns 05/11/09
--
components: Library (Lib)
messages: 87582
nosy: jonathan.cervidae
severity: normal
status: open
title: strftime is broken
type: behavior
versions: Python 2.5
___
Python tracker
<http://bugs.python.org/issue5997>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5997] strftime is broken
Jonathan added the comment: Works perfectly now, thank you and sorry for the inaccurate report. -- ___ Python tracker <http://bugs.python.org/issue5997> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6165] strftime incorrectly processes DST flag when passed time touples
New submission from Jonathan :
>>> import time
>>> time.strftime("%FT%T%z")
'2009-06-01T18:35:42+0100'
>>> # Expect to see +0100
>>> time.strftime("%FT%T%z",time.localtime())
'2009-06-01T18:35:42+'
>>> time.strftime("%FT%T%Z",time.localtime())
'2009-06-01T18:35:42BST'
>>> made_up_time = list(time.localtime())
>>> made_up_time
[2009, 6, 1, 18, 35, 48, 0, 152, 1]
>>> made_up_time[1] = 2
>>> made_up_time=tuple(made_up_time)
>>> time.strftime("%FT%T%z",made_up_time)
'2009-02-01T18:35:48+'
>>> # Expect to see GMT or UTC, whatever strftime wants to call it.
>>> time.strftime("%FT%T%Z",made_up_time)
'2009-02-01T18:35:48BST'
>>>
--
components: Extension Modules
messages: 88659
nosy: jonathan.cervidae
severity: normal
status: open
title: strftime incorrectly processes DST flag when passed time touples
type: behavior
versions: Python 2.5
___
Python tracker
<http://bugs.python.org/issue6165>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6165] strftime incorrectly processes DST flag when passed time touples
Jonathan added the comment: Actually, I didn't change the DST flag in the second test, the second commented bug is invalid, only the first one is correct. -- ___ Python tracker <http://bugs.python.org/issue6165> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6165] strftime incorrectly processes DST flag when passed time touples
Jonathan added the comment:
kludged_zone = ("+" if time.altzone < 0 else '-') +
time.strftime("%H%M",time.gmtime(abs(time.altzone)))
time_zone_format_string = time_zone_format.replace("%z", kludged_zone)
--
___
Python tracker
<http://bugs.python.org/issue6165>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46181] Destroying an expaned Combobox prevents Entry focus until Alt+Tab
New submission from Jonathan Lahav : Happens on Windows. Observation: When an expanded Combobox is destroyerd, widgets in the window can't get focus until Alt+Tab forth and back. Buttons can still be clicked, but focus can't be obtained by widgets, entries fro example, not by clicking nor by the Tab or arrow keys. The attached file contains a minimal reproduction example. Motivation: I develop the GUI for a complex application at work which needs to recreate its GUI layout upon a combobox selection, thus destroying the combobox as well. -- components: Tkinter files: combobug.py messages: 409196 nosy: j.lahav priority: normal severity: normal status: open title: Destroying an expaned Combobox prevents Entry focus until Alt+Tab type: behavior versions: Python 3.8 Added file: https://bugs.python.org/file50522/combobug.py ___ Python tracker <https://bugs.python.org/issue46181> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46181] Destroying an expaned Combobox prevents Entry focus until Alt+Tab
Jonathan Lahav added the comment: Here's a discussion about the issue. I asked about it in comp.lang.tcl: https://groups.google.com/g/comp.lang.tcl/c/C-uQIH-wP5w Someone there explains what's happening. -- ___ Python tracker <https://bugs.python.org/issue46181> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46910] Expect IndentationError, get SyntaxError: 'break' outside loop
New submission from Jonathan Fine :
This arises from a request for help made by Nguyễn Ngọc Tiến to the visually
impaired programmers lists, see
https://www.freelists.org/post/program-l/python,48. Please keep this in mind.
Nguyễn asked for help with the syntax error created by
===
count = 0
while count < 1:
count = count + 1
print(count)
break
else:
print("no break")
===
When I saved this to a file and ran it I got:
===
$ python3.8 funny_break_error.py
File "funny_break_error.py", line 6
else:
^
IndentationError: unexpected indent
===
However, remove the last two lines and you get the more helpful error
===
$ python3.8 funny_break_error.py
File "funny_break_error.py", line 5
break
^
SyntaxError: 'break' outside loop
===
Python3.6 and 3.7 also behave as above.
Note. I've heard that blind Python programmers prefer a single space to denote
indent. I think this is because they hear the leading spaces via a screen
reader, rather than see the indent with their eyes.
--
components: Parser
files: funny_break_error.py
messages: 414424
nosy: jfine2358, lys.nikolaou, pablogsal
priority: normal
severity: normal
status: open
title: Expect IndentationError, get SyntaxError: 'break' outside loop
type: behavior
versions: Python 3.7, Python 3.8
Added file: https://bugs.python.org/file50655/funny_break_error.py
___
Python tracker
<https://bugs.python.org/issue46910>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46910] Expect IndentationError, get SyntaxError: 'break' outside loop
Jonathan Fine added the comment: Many thanks Pablo for the clear explanation. I'd prefer that the issue remain open, as there's an important user experience issue here. I suspect there are other similar examples of how the compiler error messages could be improved. Here's a change that doesn't seem to be too hard, that could fix the problem at hand. The IndentationError occurred at a known location in the input string. So as part of error reporting truncate the input string and try to compile that. In other words, make a good faith attempt to find an earlier error. I've attached a funny_break_error_fix.py which is a first draft implementation of this idea. Here's the output: === $ python3 funny_break_error_fix.py funny_break_error.py unexpected indent (funny_break_error.py, line 6) Traceback (most recent call last): File "funny_break_error_fix.py", line 3, in compile_fix compile(source, filename, 'exec') File "funny_break_error.py", line 6 else: ^ IndentationError: unexpected indent During handling of the above exception, another exception occurred: Traceback (most recent call last): File "funny_break_error_fix.py", line 18, in compile_fix(src.read(), filename, 'exec') File "funny_break_error_fix.py", line 9, in compile_fix compile(new_source, filename, 'exec') File "funny_break_error.py", line 5 break ^ SyntaxError: 'break' outside loop === And in this case we've got hold of the first error (at the cost of compiling part of the source file twice). Many thanks again for the clear explanation, which I found most helpful when formulating the above fix. -- Added file: https://bugs.python.org/file50656/funny_break_error_fix.py ___ Python tracker <https://bugs.python.org/issue46910> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46910] Expect IndentationError, get SyntaxError: 'break' outside loop
Jonathan Fine added the comment: My main concern is that the door not be closed on improving the user experience relating to this behaviour of the compiler. This issue was raised as a bug for the compiler (which is C-coded). I'd be very happy for this issue to be closed as 'not a bug' for the compiler, provided the door is left open for Python-coded improvements for the user experience. I suggest that the issue title be changed to: The two-pass compile(bad_src, ...) sometimes does not report first error in bad_src These two changes to the details of closure would be sufficient to meet my concern. I hope they can be accepted. By the way, I see these improvements being done as a third-party pure-Python module outside Python's Standard Library, at least until they've reached a wide measure of community acceptance. -- ___ Python tracker <https://bugs.python.org/issue46910> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1336] subprocess.Popen hangs when child writes to stderr
New submission from Jonathan Amsterdam:
This is under Linux (2.6).
I occasionally see subprocess.Popen() fail to return, and I have
finally figured out roughly what's going on. It involves the GC and
stderr.
1. os.fork()
2. Parent blocks reading from errpipe_read (subprocess.py:982)
3. In child, a GC occurs before the exec.
4. The GC attempts to free a file descriptor, calling file_dealloc.
5. That function gets an error closing the descriptor ("close failed:
[Errno 9] Bad file descriptor\n," is the string I extracted via gdb).
6. It attempts to write the error to stderr and blocks. Since it never
execs or writes to errpipe_write, both child and parent are hung.
Here is the pstack output on the child:
#0 0x006587a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
#1 0x0072f11b in __write_nocancel () from /lib/tls/libc.so.6
#2 0x006d409f in _IO_new_file_write () from /lib/tls/libc.so.6
#3 0x006d42ec in _IO_new_file_xsputn () from /lib/tls/libc.so.6
#4 0x006afce9 in buffered_vfprintf () from /lib/tls/libc.so.6
#5 0x006afe8b in vfprintf () from /lib/tls/libc.so.6
#6 0x080dd4af in mywrite ()
#7 0x080dd505 in PySys_WriteStderr ()
#8 0x08064cd0 in file_dealloc ()
#9 0x08079c88 in dict_dealloc ()
#10 0x0808931d in subtype_dealloc ()
#11 0x08079af3 in PyDict_Clear ()
#12 0x0807bb6a in dict_tp_clear ()
#13 0x080e0ead in collect ()
#14 0x080e1912 in _PyObject_GC_New ()
#15 0x0805e50b in PyInstance_NewRaw ()
#16 0x0805e5d7 in PyInstance_New ()
#17 0x0805bdc0 in PyObject_Call ()
#18 0x080aecef in PyEval_CallObjectWithKeywords ()
#19 0x080ca975 in PyErr_NormalizeException ()
#20 0x080b492c in PyEval_EvalFrame ()
#21 0x080b5eb2 in PyEval_EvalCodeEx ()
#22 0x080b3c83 in PyEval_EvalFrame ()
#23 0x080b5734 in PyEval_EvalFrame ()
#24 0x080b5eb2 in PyEval_EvalCodeEx ()
#25 0x080fce92 in function_call ()
#26 0x0805bdc0 in PyObject_Call ()
#27 0x080641e5 in instancemethod_call ()
#28 0x0805bdc0 in PyObject_Call ()
#29 0x0808ffce in slot_tp_init ()
#30 0x08088b3a in type_call ()
#31 0x0805bdc0 in PyObject_Call ()
#32 0x080b0f05 in PyEval_EvalFrame ()
#33 0x080b5eb2 in PyEval_EvalCodeEx ()
#34 0x080fce92 in function_call ()
#35 0x0805bdc0 in PyObject_Call ()
#36 0x080641e5 in instancemethod_call ()
#37 0x0805bdc0 in PyObject_Call ()
#38 0x0808ffce in slot_tp_init ()
#39 0x08088b3a in type_call ()
#40 0x0805bdc0 in PyObject_Call ()
#41 0x080b0f05 in PyEval_EvalFrame ()
#42 0x080b5734 in PyEval_EvalFrame ()
#43 0x080b5eb2 in PyEval_EvalCodeEx ()
#44 0x080fce92 in function_call ()
#45 0x0805bdc0 in PyObject_Call ()
#46 0x080641e5 in instancemethod_call ()
#47 0x0805bdc0 in PyObject_Call ()
#48 0x0808ffce in slot_tp_init ()
#49 0x08088b3a in type_call ()
#50 0x0805bdc0 in PyObject_Call ()
#51 0x080b0f05 in PyEval_EvalFrame ()
#52 0x080b5eb2 in PyEval_EvalCodeEx ()
#53 0x080fce92 in function_call ()
#54 0x0805bdc0 in PyObject_Call ()
#55 0x080b075f in PyEval_EvalFrame ()
#56 0x080b5734 in PyEval_EvalFrame ()
#57 0x080b5734 in PyEval_EvalFrame ()
#58 0x080b5734 in PyEval_EvalFrame ()
#59 0x080b5eb2 in PyEval_EvalCodeEx ()
#60 0x080b3c83 in PyEval_EvalFrame ()
#61 0x080b5734 in PyEval_EvalFrame ()
#62 0x080b5734 in PyEval_EvalFrame ()
#63 0x080b5eb2 in PyEval_EvalCodeEx ()
#64 0x080b3c83 in PyEval_EvalFrame ()
#65 0x080b5eb2 in PyEval_EvalCodeEx ()
#66 0x080b3c83 in PyEval_EvalFrame ()
#67 0x080b5eb2 in PyEval_EvalCodeEx ()
#68 0x080b3c83 in PyEval_EvalFrame ()
#69 0x080b5734 in PyEval_EvalFrame ()
#70 0x080b5734 in PyEval_EvalFrame ()
#71 0x080b5734 in PyEval_EvalFrame ()
#72 0x080b5734 in PyEval_EvalFrame ()
#73 0x080b5734 in PyEval_EvalFrame ()
#74 0x080b5eb2 in PyEval_EvalCodeEx ()
#75 0x080fce92 in function_call ()
#76 0x0805bdc0 in PyObject_Call ()
#77 0x080b075f in PyEval_EvalFrame ()
#78 0x080b5eb2 in PyEval_EvalCodeEx ()
#79 0x080b3c83 in PyEval_EvalFrame ()
#80 0x080b5eb2 in PyEval_EvalCodeEx ()
#81 0x080b3c83 in PyEval_EvalFrame ()
#82 0x080b5eb2 in PyEval_EvalCodeEx ()
#83 0x080b3c83 in PyEval_EvalFrame ()
#84 0x080b5734 in PyEval_EvalFrame ()
#85 0x080b5734 in PyEval_EvalFrame ()
#86 0x080b5eb2 in PyEval_EvalCodeEx ()
#87 0x080b601a in PyEval_EvalCode ()
#88 0x080d9ff4 in PyRun_FileExFlags ()
#89 0x080da8d6 in PyRun_SimpleFileExFlags ()
#90 0x08055617 in Py_Main ()
#91 0x08054e3f in main ()
--
components: Library (Lib)
messages: 56790
nosy: jba
severity: normal
status: open
title: subprocess.Popen hangs when child writes to stderr
type: crash
versions: Python 2.4
__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1336>
__
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1336] subprocess.Popen hangs when child writes to stderr
Jonathan Amsterdam added the comment: It's an honor to hear from you, BDFL. I'm just a Python user, not a developer, so the time it would take me to check out, compile, test, etc. would dwarf the change itself (removing two lines from file_dealloc). There is another solution, following Java: file_dealloc could raise an exception. Then the error would appear in normal contexts, but (assuming the GC ignores exceptions in finalization) would be ignored in GC. There is also a simpler, less invasive change you could consider: disabling GC in the child inside subprocess.Popen. On 10/26/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: > > Guido van Rossum added the comment: > > I don't think we can prevent GC from occurring between fork and exec -- > it's legal to just call os.fork() and execute Python code in the > subprocess forever. I think the right solution might be to ignore > errors in file_close(). Can you try to whip up a patch for that and > test it? > > -- > nosy: +gvanrossum > > __ > Tracker <[EMAIL PROTECTED]> > <http://bugs.python.org/issue1336> > __ > __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1336> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12930] reindent.py inserts spaces in multiline literals
Jonathan Rogers added the comment: I don't think reindent.py should change any bytes inside string literals since it can't know anything about what those strings mean or how they'll be used by the program at run time. Unfortunately, it starts out by unconditionally calling the .expandtabs() method on each input line, so tab characters are lost. The only change to a string literal I can imagine that would be safe is to replace tab characters with '\t'. I am trying to use reindent.py on Python source files which include triple-quoted, multi-line string literals containing makefile and Python snippets. In both cases, running reindent.py changes the meaning of of that contained in the literal. -- nosy: +Jonathan.Rogers ___ Python tracker <http://bugs.python.org/issue12930> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12930] reindent.py inserts spaces in multiline literals
Jonathan Rogers added the comment: Rather than expanding tab characters inside string literals, it's safer to replace them with '\t'. -- Added file: http://bugs.python.org/file24120/save_strings.patch ___ Python tracker <http://bugs.python.org/issue12930> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9998] ctypes find_library should search LD_LIBRARY_PATH on linux
Jonathan Niehof added the comment: Yaroslav: does the patch cause problems, or the original issue? If the former, could you be specific so I can try and fix it? -- ___ Python tracker <http://bugs.python.org/issue9998> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9993] shutil.move fails on symlink source
Jonathan Niehof added the comment: Éric, here's a quick docs-only patch against current default...does this do the job? -- Added file: http://bugs.python.org/file23002/shutil_move_doc.patch ___ Python tracker <http://bugs.python.org/issue9993> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9993] shutil.move fails on symlink source
Jonathan Niehof added the comment: Éric: I think copying a relative symlink should also be relative, and that's the behaviour of this patch. That was the use case that tripped me up with the original behaviour of shutil.move: a relative symlink which was dangling in its original location, but not once moved. (This was, believe it or not, intentional design) This is also the behaviour when src and dst are on the same filesystem. Basically my intention was to remove the distinction between "same filesystem" and "different filesystem." -- ___ Python tracker <http://bugs.python.org/issue9993> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9991] xmlrpc client ssl check faulty
New submission from Jonathan Niehof : This has been reported before (#6494, #7987) and closed as "You need to build Python with SSL support." However, this error is raised even if ssl support is included. The SSL check in xmlrpc.client for Python 3.1 is: if not hasattr(socket, "ssl") but ssl was removed from socket and into its own class for Py3k. So one workaround, provided ssl support is available: import socket if not hasattr(socket, 'ssl') socket.ssl = None import xmlrpc.client at which point everything works fine. The attached patch fixes the bug, checking for ssl support based on the existence of http.client.HTTPSConnection, which is similar to the check in Python 2.7. -- components: Library (Lib) files: xmlrpc_client_ssl_check.patch keywords: patch messages: 117667 nosy: jniehof priority: normal severity: normal status: open title: xmlrpc client ssl check faulty type: crash versions: Python 3.1, Python 3.2 Added file: http://bugs.python.org/file19061/xmlrpc_client_ssl_check.patch ___ Python tracker <http://bugs.python.org/issue9991> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9993] shutil.move fails on symlink source
New submission from Jonathan Niehof : shutil.move does not behave as I expect when moving a symlink across filesystems. (This is when src itself is a symlink, not when it is a directory tree including symlinks.) -If src is a symlink to file, rather than moving the symlink, it copies the contents of the file -If src is a symlink to a directory, rather than moving the symlink, it copies the contents of the directory to a new directory -If src is a dangling symlink, it errors out Attached patch against the py3k branch adds tests for these cases and adds the expected behaviour, which is to recreate the symlink. (I have encountered this on 2.6 - current SVN; it is probably in 2.5 as well but I have not confirmed.) -- components: Library (Lib) files: shutil_move_symlinks.patch keywords: patch messages: 117673 nosy: jniehof priority: normal severity: normal status: open title: shutil.move fails on symlink source type: behavior versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2 Added file: http://bugs.python.org/file19063/shutil_move_symlinks.patch ___ Python tracker <http://bugs.python.org/issue9993> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9998] find_library should search LD_LIBRARY_PATH on linux
New submission from Jonathan Niehof : It's come up on occasion (#2936, http://osdir.com/ml/python.ctypes/2008-05/msg00046.html) that ctypes find_library doesn't search LD_LIBRARY_PATH for libraries, which is different behaviour from the runtime linker. The attached patch adds this search. Unfortunately I can't conceive of a reasonable unit test for this (compiling a shared library in setUp seems a bit overkill.) -- assignee: theller components: ctypes files: ctypes_ld_lib_path.patch keywords: patch messages: 117738 nosy: jniehof, theller priority: normal severity: normal status: open title: find_library should search LD_LIBRARY_PATH on linux type: feature request versions: Python 2.7, Python 3.1, Python 3.2 Added file: http://bugs.python.org/file19068/ctypes_ld_lib_path.patch ___ Python tracker <http://bugs.python.org/issue9998> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9999] test_shutil cross-file-system tests are fragile (may not test what they pruport to test)
Jonathan Niehof added the comment: Might it make sense to skip-decorate those tests which require cross-filesystem? Put a test above the TestMove definition which compares tempfile.gettempdir() and os.path.dirname(__file__). I don't know of an out-of-the-box function to see if two paths are on the same filesystem; I suppose one approach would be to walk up the directory tree of each, checking ismount. (Looks like somebody's tried it: http://stackoverflow.com/questions/1138383/python-get-mount-point-on-windows-or-linux) -- nosy: +jniehof ___ Python tracker <http://bugs.python.org/issue> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9991] xmlrpc client ssl check faulty
Changes by Jonathan Niehof : -- nosy: +Jelly.Chen, lister171254 ___ Python tracker <http://bugs.python.org/issue9991> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11295] On Windows, Python crashes on ANSI / Windows-formatted source files
New submission from Jonathan Hayward : So far as I can tell, Python 2.7 crashes on at least some ANSI / Windows \r\n-delimited source files. More specifically, as invoked by Apache as a CGI script, the source file line: import cgi generated an error logged by Apache, complaining that the module "cgi\r" couldn't be imported. Email sent as follows: -- Today I was visiting with a friend and installing OSS on his computer. On an x86_64 Windows 7 box, he had already installed Apache, and I installed Python with a python.org installer. I opened up Notepad and created a "Hello world" Python CGI script, put it in the cgi-bin directory, saw it crash, and looked in the logs. The log message complained that I had tried to "import cgi\r": in other words, Python on Windows was choking because the file I made in Notepad used "\r\n" for line breaks. (The equivalent script made with vim, and presumably "\n" for line breaks worked predictably.) Isn't it a defect/design flaw for Python on Windows to choke on "\r\n"-separated files? Should I file a bug, or is this a side effect of decisions that are now non-negotiable? (I wasn't thrilled, after trying to sell my friend on the idea that Python is a good language with a low barrier to entry, to find that it choked on a Notepad-edited "Hello world!" CGI script.) -- components: Interpreter Core messages: 129142 nosy: JonathanHayward priority: normal severity: normal status: open title: On Windows, Python crashes on ANSI / Windows-formatted source files type: behavior versions: Python 2.7 ___ Python tracker <http://bugs.python.org/issue11295> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11295] On Windows, Python crashes on ANSI / Windows-formatted source files
Jonathan Hayward added the comment: Thank you; noted. I'm closing the bug for now at least; I'll reopen it if need be. -- resolution: -> invalid status: open -> closed ___ Python tracker <http://bugs.python.org/issue11295> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11407] unittest.TestCase.run should return result
Jonathan Hartley added the comment: Attached patch fixes this. TestCase.run now returns its TestResult object, regardless of whether it was passed in or created internally. Test assertions added for this. Invoking an instance of TestCase already returned the return value of .run, so the result is correctly propagated. New test added that verifies that .__call__ delegates to .run. Fix for an unrelated minor typo in the docs also bundled into this patch: Fixed the versionchanged:: markup for class TestCase. This is my first patch submission, please help me out if I mess it up. Thanks. -- keywords: +patch nosy: +tartley Added file: http://bugs.python.org/file21131/issue11407.patch ___ Python tracker <http://bugs.python.org/issue11407> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11796] list and generator expressions in a class definition fail if expression condition refers to a class variable
Jonathan Hartley added the comment: Is also exhibited by other class variable being used in the 'output' clause of the list comprehension: >>> class C: ... x = 3 ... z = [z*x for z in range(4)] Traceback (most recent call last): File "", line 1, in File "", line 3, in C File "", line 3, in -- nosy: +jonathan.hartley versions: +Python 3.1 ___ Python tracker <http://bugs.python.org/issue11796> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11835] python (x64) ctypes incorrectly pass structures parameter
Changes by Jonathan White : -- nosy: +jwhitecl ___ Python tracker <http://bugs.python.org/issue11835> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9228] Make changes in the PATH and PATHEXT on installation
Changes by Jonathan Hartley : -- nosy: +jonathan.hartley ___ Python tracker <http://bugs.python.org/issue9228> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3561] Windows installer should add Python and Scripts directories to the PATH environment variable
Changes by Jonathan Hartley : -- nosy: +jonathan.hartley ___ Python tracker <http://bugs.python.org/issue3561> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2636] Regexp 2.7 (modifications to current re 2.2.2)
Jonathan Halcrow added the comment: I'm having a problem using the current version (0.1.20110504) with python 2.5 on OSX 10.5. When I try to import regex I get the following import error: dlopen(/python2.5/site-packages/_regex.so, 2): Symbol not found: _re_is_same_char_ign Referenced from: /python2.5/site-packages/_regex.so Expected in: dynamic lookup -- ___ Python tracker <http://bugs.python.org/issue2636> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2636] Regexp 2.7 (modifications to current re 2.2.2)
Jonathan Halcrow added the comment: It seems that _regex_unicode.c is missing from setup.py, adding it to ext_modules fixes my previous issue. -- ___ Python tracker <http://bugs.python.org/issue2636> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10254] unicodedata.normalize('NFC', s) regression
Jonathan Halcrow added the comment: I think I've come across a related problem. I am experiencing a segfault when NFC-normalizing a certain string [1]. The crash occurs with 2.7.1 in OS X (built from source with homebrew). Here is the backtrace: #0 0x0025a96e in _PyUnicode_Resize () #1 0x00601673 in nfc_nfkc () #2 0x00601bb7 in unicodedata_normalize () #3 0x0029834b in PyEval_EvalFrameEx () #4 0x00299f13 in PyEval_EvalCodeEx () #5 0x0029a0fe in PyEval_EvalCode () #6 0x002bd5f0 in PyRun_FileExFlags () #7 0x002be430 in PyRun_SimpleFileExFlags () #8 0x002d5bd6 in Py_Main () #9 0x1f8f in _start () #10 0x1ebd in start () [1] http://pastebin.com/cfNd2QEz -- nosy: +jhalcrow ___ Python tracker <http://bugs.python.org/issue10254> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4681] mmap offset should be off_t instead of ssize_t, and size calculation needs corrected
Changes by Jonathan Beezley : -- nosy: +jbeezley ___ Python tracker <http://bugs.python.org/issue4681> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11221] all() returns wrong result when the parameters are non-encapsulated list-comprehension
New submission from Jonathan Livni : all( (x<=y) for x,y in zip(L, L[1:]) ) all([(x<=y) for x,y in zip(L, L[1:])]) Both lines of code above check if L is a non-decreasing list. Both should return the same results. But under some conditions, they don't. I've encountered this with a list of Decimal numbers. This is 100% reproducible on my Win7 64bit vanilla Python 2.6.6 32bit setup, alas I cannot share the specific code that generates this difference. See attached screenshot from Eclipse Pydev debugger. -- components: Windows files: Eclipse.JPG messages: 128628 nosy: Jonathan.Livni priority: normal severity: normal status: open title: all() returns wrong result when the parameters are non-encapsulated list-comprehension versions: Python 2.6 Added file: http://bugs.python.org/file20766/Eclipse.JPG ___ Python tracker <http://bugs.python.org/issue11221> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11221] all() returns wrong result when the parameters are non-encapsulated list-comprehension
Jonathan Livni added the comment:
The exact list of decimals doesn't help - I tried taking the list and
reproducing the bug with the following short script, but the problem did not
reproduced:
from decimal import Decimal
L = [Decimal('6.700'), Decimal('6.800'), Decimal('7.140'), Decimal('7.460'),
Decimal('7.735'), Decimal('8.160'), Decimal('8.280'), Decimal('8.355'),
Decimal('8.710'), Decimal('9.640'), Decimal('10.155'), Decimal('10.460'),
Decimal('10.810'), Decimal('11.875'), Decimal('12.310'), Decimal('12.315'),
Decimal('13.250'), Decimal('13.205'), Decimal('13.750'), Decimal('14.245'),
Decimal('14.805'), Decimal('15.385'), Decimal('15.955'), Decimal('16.365'),
Decimal('16.960'), Decimal('17.500'), Decimal('19.445')]
print all(x<=y for x, y in zip(L, L[1:]))
The script above rightfully printed False.
The decimal list above was taken from the pydev debugger session where I found
the bug.
In the original script I do not mess around with Decimal at all. I just cast
too and from float and use simple arithmetics with it.
--
___
Python tracker
<http://bugs.python.org/issue11221>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11221] all() returns wrong result when the parameters are non-encapsulated list-comprehension
Jonathan Livni added the comment: Another note - the original problematic code looks like this: def non_decreasing(L): return all(x<=y for x, y in zip(L, L[1:])) Changing it to: def non_decreasing(L): def f(L): return [x<=y for x, y in zip(L, L[1:])] return all(f(L)) also worked around the bug -- ___ Python tracker <http://bugs.python.org/issue11221> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11221] all() returns wrong result when the parameters are non-encapsulated list-comprehension
Jonathan Livni added the comment:
The script I used is a single file single threaded code - but - It uses
django's ORM to get the data from a MySQL database.
I've reduced the code path to this:
import sys,os
sys.path.append(os.path.dirname(os.getcwdu()))
os.environ['DJANGO_SETTINGS_MODULE']='my_app.settings'
from django.core.management import setup_environ
from my_app import settings
setup_environ(settings)
from my_app.convert.models import SomeModel
from operator import itemgetter
from decimal import Decimal
def non_decreasing(L):
return all(x<=y for x, y in zip(L, L[1:]))
raw_data =
SomeModel.objects.filter(the_date=the_date,col1__gt=Decimal('0.2'),col2__gt=Decimal('0.2'),col3__gt=0,col4__gt=0,col5__gte=2).order_by('col6','col7','col8').values_list('col6','col7','col8','col1','col3','col2','col4')
data=defaultdict(list)
for d in raw_data:
data[d[0],d[1]].append(d[2:])
for (exp,t),d in data.iteritems():
col8s = map(itemgetter(0),d)
mids = [(x[3]+x[4])/Decimal('2.0') for x in d]
if not non_decreasing(mids):
raise Exception
--
___
Python tracker
<http://bugs.python.org/issue11221>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11221] all() returns wrong result when the parameters are non-encapsulated list-comprehension
Jonathan Livni added the comment: from pylab import * There lies the rub? -- ___ Python tracker <http://bugs.python.org/issue11221> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11221] all() returns wrong result when the parameters are non-encapsulated list-comprehension
Jonathan Livni added the comment: Let my foolishness be a reminder to all not to use "from [module] import *" After saying that - I believe overloading a built in Python function in a popular package\module is a mistake! I still don't know if pylab's all() is erroneous or if it's correct functionality. I'll open a ticket there. -- ___ Python tracker <http://bugs.python.org/issue11221> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3071] The ValueError raised by failing to unpack sequence should have more information.
New submission from Jonathan Lange <[EMAIL PROTECTED]>: Here's the current message: Python 2.5.2 (r252:60911, Apr 21 2008, 11:12:42) [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> [foo] = [2, 3] Traceback (most recent call last): File "", line 1, in ValueError: too many values to unpack It would be good if the message of the ValueError contained information about how many values were expected and how many values were given. -- components: Interpreter Core messages: 67885 nosy: jml severity: normal status: open title: The ValueError raised by failing to unpack sequence should have more information. type: feature request versions: Python 2.3, Python 2.4, Python 2.5 ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue3071> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1291446] SSLObject breaks read semantics
Jonathan Ellis <[EMAIL PROTECTED]> added the comment: This is incorrect. Perhaps you are thinking of a raw socket read; a _file-like-object_ read is supposed to return the amount of data requested, unless (a) the socket is in non-blocking mode, or (b) if EOF is reached first. Normal socket.makefile observes this, but SSLObject does not. ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1291446> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1291446] SSLObject breaks read semantics
Jonathan Ellis <[EMAIL PROTECTED]> added the comment: s/raw socket read/raw socket recv/ ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1291446> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1291446] SSLObject breaks read semantics
Jonathan Ellis <[EMAIL PROTECTED]> added the comment: Here is the exact SSLObject.read documentation from 2.5 (although the bug was filed against 2.4, and 2.6 will be out soon, the docs are the same): - read([n]) If n is provided, read n bytes from the SSL connection, otherwise read until EOF. The return value is a string of the bytes read. - This is not ambiguous. Similarly, help(file.read) is not ambiguous. (The "at most" part in the first line of file.read is later explained to apply to non-blocking reads.) If you want to claim "well, it's not a file-like object" then (a) it shouldn't have file-like methods (socket-like send and recv are the obvious choices instead of write and read), (b) you need to fix your docs. But since god knows how many programs are out there expecting the semantics explained by the existing docs, I think just fixing the bug in the code is better than defining away the problem. (Obviously socket.makefile won't work on an object that doesn't implement a socket-like interface, so that's a non-option.) ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1291446> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1291446] SSLObject breaks read semantics
Jonathan Ellis <[EMAIL PROTECTED]> added the comment: Ah, great. I was wondering why you kept talking about SSLSocket instead of SSLObject. "New API in 2.6" is good enough for me. Thanks! ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1291446> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8232] webbrowser open(), open_new(), and open_new_tab() Broken Functionality
New submission from Jonathan Chao :
webbrowser.open(), webbrowser.open_new(), and webbrowser.open_new_tab() all do
the exact same thing, regardless of the flags that I set. In Firefox,
open('www.google.com', new=0), open_new('www.google.com'), and
open_new_tab('www.google.com') all open either three new www.google.com tabs
(if "Open new windows in a new tab instead" is selected in FF options) or three
new www.google.com windows (if "Open new windows in a new tab instead" is not
selected in FF options). In Internet Explorer, three new www.google.com tabs
are created.
The issue exhibits itself the same way whether or not I have the browser open
before running the script.
Environment was a Windows Vista 32-bit machine, running Python 3.1.2.
Example script reads:
import webbrowser
import time
ff = webbrowser.get('firefox')
ff.open('www.google.com', new=0)
time.sleep(3)
ff.open_new('www.google.com')
time.sleep(3)
ff.open_new_tab('www.google.com')
--
components: Library (Lib)
messages: 101725
nosy: joncwchao
severity: normal
status: open
title: webbrowser open(), open_new(), and open_new_tab() Broken Functionality
type: behavior
versions: Python 3.1
___
Python tracker
<http://bugs.python.org/issue8232>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1822] email.mime.multipart.MIMEMultipart.is_multipart() returns false before items have been added.
New submission from Jonathan Share: Steps to reproduce == >>> from email.mime.multipart import MIMEMultipart >>> foo = MIMEMultipart() >>> foo.is_multipart() False Expected Result === True should be returned from MIMEMultipart.is_multipart() Notes = Looking at the implementation of is_multipart() in email.Message it would appear that if self._payload was initialised to an empty list in the constructor of MIMEMultipart when _subparts is None, this would be a sufficient fix for this issue. However, from an outsider looking into this code for the first time, this doesn't look like the best architecture, shouldn't the issue of whether a message has multiple parts, and logic specific to this be handled through inheritance. With the current implementation the superclass makes assumptions about how a subclass is implemented, this just feels wrong. Have I missed something, is there a good reason for things being as they are today? Feel free to take the discussion to the python-dev list, I have just subscribed, and I am interested in fixing this issue myself in the next bug day if someone can answer my questions above. -- components: Library (Lib) messages: 59895 nosy: Sharebear severity: normal status: open title: email.mime.multipart.MIMEMultipart.is_multipart() returns false before items have been added. type: behavior versions: Python 2.5 __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1822> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1823] Possible to set invalid Content-Transfer-Encoding on email.mime.multipart.MIMEMultipart
New submission from Jonathan Share:
Steps to Reproduce
==
>>> from email.mime.multipart import MIMEMultipart
>>> from email.mime.text import MIMEText
>>> multipart = MIMEMultipart()
>>> multipart.set_charset('UTF-8')
>>> text = MIMEText("sample text")
>>> multipart.attach(text)
>>> print multipart.as_string()
MIME-Version: 1.0
Content-Type: multipart/mixed; charset="utf-8";
boundary="===0973828728=="
Content-Transfer-Encoding: base64
--===0973828728==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
sample text
--===0973828728==--
>>> multipart = MIMEMultipart()
>>> multipart.attach(text)
>>> multipart.set_charset('UTF-8')
Traceback (most recent call last):
File "", line 1, in
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
email/message.py", line 262, in set_charset
self._payload = charset.body_encode(self._payload)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
email/charset.py", line 384, in body_encode
return email.base64mime.body_encode(s)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
email/base64mime.py", line 148, in encode
enc = b2a_base64(s[i:i + max_unencoded])
TypeError: b2a_base64() argument 1 must be string or read-only buffer,
not list
Explanation
===
The first example above demonstrates that if you call set_charset('UTF-
8') on a MIMEMultipart instance before adding child parts then it is
possible to generate a multipart/* message with an illegal Content-
Transfer-Encoding as specified by RFC 2045[1] "If an entity is
of type "multipart" the Content-Transfer-Encoding is not permitted to
have any value other than "7bit", "8bit" or "binary"."
In the second example, I demonstrate that if you try and call
set_charset after adding child parts, the code exceptions. The user
should at least be provided with a more targeted exception.
Notes
=
Where should this be fixed? The smallest fix would be to add a check to
set_charset to see if it is dealing with with a multipart message but
as I express in issue1822 I feel the better design would be to move
this subtype specific logic into the appropriate subclass.
Again, this is something I'm willing to work on in next saturday's bug
day if I can get some feedback on my architectural concerns.
[1] http://tools.ietf.org/html/rfc2045#section-6.4
--
components: Library (Lib)
messages: 59896
nosy: Sharebear
severity: normal
status: open
title: Possible to set invalid Content-Transfer-Encoding on
email.mime.multipart.MIMEMultipart
type: behavior
versions: Python 2.5
__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1823>
__
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1823] Possible to set invalid Content-Transfer-Encoding on email.mime.multipart.MIMEMultipart
Jonathan Share added the comment: Martin, I can almost agree with you _if_ I was setting the Content-Transfer- Encoding myself, however I am not. I am setting the charset and the library chooses an appropriate Content-Transfer-Encoding to represent the mime part with. Currently I can't see any way other than reading the source or writing a test case (and that would require understanding what the email.mime module was doing "under the hood") for a developer to find out which Content-Transfer-Encoding was going to be used. Also, just from a usability point of view I would expect that creating an invalid mime part would be a little more difficult. Especially considering the fix should be as small as adding "if not encoding in valid encodings: raise SensibleException". __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1823> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1822] email.mime.multipart.MIMEMultipart.is_multipart() returns false before items have been added.
Jonathan Share added the comment: Attaching a patch for the quick fix I proposed below. I would still like to see some feedback regarding making the design of the mime module more object oriented. email.Message really shouldn't be making assumtions about how subclasses represent their state. Added file: http://bugs.python.org/file9214/issue1822.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1822> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1822] email.mime.multipart.MIMEMultipart.is_multipart() returns false before items have been added.
Changes by Jonathan Share: -- nosy: +facundobatista __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1822> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1823] Possible to set invalid Content-Transfer-Encoding on email.mime.multipart.MIMEMultipart
Jonathan Share added the comment: I'm beginning to realise this is slightly bigger than I first thought ;-) Trying to make a nice test case for this issue, I thought it would be a good idea for the parser to register a defect for invalid content- transfer-encoding so I can test against that in the test case rather than fragile substring tests. Unfortunately the parser code isn't the easiest code to get your head around on a first look. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1823> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1823] Possible to set invalid Content-Transfer-Encoding on email.mime.multipart.MIMEMultipart
Jonathan Share added the comment: Run out of time to look at this today. In order to write a nice test case for this issue I need the parser to notice this error in messages. I've filed issue1874 for the parser not reporting the invalid cte in the msg.defects __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1823> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1874] email parser does not register a defect for invalid Content-Transfer-Encoding on multipart messages
New submission from Jonathan Share: Although the documentation of FeedParser states that "It will populate a message object's defects attribute with a list of any problems it found in a message." no defect is found in the test case I am about to upload. The message in the test case is broken because it specifies a Content-Transfer-Encoding that is not valid for multipart/* messages[1]. I've spent some time today looking at the parser and cannot see where the parser is doing anything with the Content-Transfer-Encoding of a multipart message, leading me to believe that there might be another bug here with regards to not honoring Content-Transfer-Encoding at all for multipart/* messages but I don't have any more time today to look at it, I'll have to go away and read the rfc really well. If someone can guide me on how to get this test to pass then I can probably get a good test case and fix written for issue1823 as well. [1] http://tools.ietf.org/html/rfc2045#section-6.4 -- components: Library (Lib) messages: 60207 nosy: Sharebear, barry severity: normal status: open title: email parser does not register a defect for invalid Content-Transfer-Encoding on multipart messages type: behavior versions: Python 2.5, Python 2.6, Python 3.0 __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1874> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1874] email parser does not register a defect for invalid Content-Transfer-Encoding on multipart messages
Changes by Jonathan Share: Added file: http://bugs.python.org/file9227/issue1874.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1874> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1343] XMLGenerator: nice elements
Jonathan Hitchcock <[EMAIL PROTECTED]> added the comment: The attached patch makes this new feature optional, by passing the "empty_element_tag" option to the constructor. -- nosy: +vhata Added file: http://bugs.python.org/file10261/xml.sax.saxutils.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1343> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
