[issue17807] Generator cleanup without tp_del

2013-04-21 Thread Phil Connell

Changes by Phil Connell :


--
nosy: +isoschiz, pconnell -pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17807] Generator cleanup without tp_del

2013-04-21 Thread Phil Connell

Changes by Phil Connell :


--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17810] Implement PEP 3154 (pickle protocol 4)

2013-04-21 Thread Andrew Svetlov

Changes by Andrew Svetlov :


--
nosy: +asvetlov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17776] IDLE Internationalization

2013-04-21 Thread Damien Marié

Damien Marié added the comment:

Here is a new patch featuring:
_ a setting to disable idle i18n
_ a documentation

Things needed:
_ taking into account Windows (where IDLE is mainly used)
_ a much in-depth translation of the interface: Context-menu, dialogs, ...
_ unit-testing it

To test it by yourself without touching your /usr/share/local you can modify 
the binddomain() (in i18n.py) to another dir:
Like " gettext.bindtextdomain('idlelib',"/home/you/your_trans_dir/") "

And put in this dir the "en" dir in the tar file.

The .mo generation is explained in the module documentation.

So, here is a tar archive with:
_ a screenshot of the patch in action
_ the patch
_ the trans dir to try it by yourself
_ the .po file (thanks to Olivier Berger)

--
Added file: http://bugs.python.org/file29965/patch.tar.gz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17810] Implement PEP 3154 (pickle protocol 4)

2013-04-21 Thread Antoine Pitrou

Changes by Antoine Pitrou :


--
keywords: +patch
Added file: http://bugs.python.org/file29966/9f1be171da08.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17804] streaming struct unpacking

2013-04-21 Thread Mark Dickinson

Mark Dickinson added the comment:

This seems like an attractive idea.  There's definitely a need for repeated 
unpacking with the same pattern, and I agree that putting the repetition into 
the pattern is suboptimal (not least from the point of view of caching structs).

One thing that feels a bit unorthogonal is that this is doing two things at 
once:  both allowing for repetition of a pattern, and also adding the lazy 
iteration.  I'd guess that there's also a use-case for allowing repetition but 
not returning an iterator;  but then that's easily covered by list(iter_unpack).

+1 from me.

Hmm;  the name.  'iterunpack'?  'iter_unpack'?  'unpack_stream'?  'unpack_all'?

Would we want something similar for packing, too?  I guess that's effectively 
covered by b''.join(s.pack(item) for item in ...).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17810] Implement PEP 3154 (pickle protocol 4)

2013-04-21 Thread Antoine Pitrou

Antoine Pitrou added the comment:

Thank you for reviving this :)
A couple of questions:
- why ADDITEM in addition to ADDITEMS? I don't think single-element sets are an 
important use case (as opposed to, say, single-element tuples)
- what is the purpose of STACK_GLOBAL? I would say memoization of common names 
but you pass memoize=False

> For example, I think Stefan's idea, which is not specified in the
> PEP, to eliminate PUT opcodes is interesting. His proposal was to
> emit an implicit PUT opcode after each object pickled and make the
> Pickler and Unpickler classes agree on the scheme.

Are the savings worth it?
I've tried pickletools.optimize() on two objects:

- a typical data dict (http.client.responses). The pickle length decreases from 
1155 to 1063 (8% shrink); unpickling is faster by 4%.

- a Logger object (logging.getLogger("foobar"). The pickle length decreases 
from 427 to 389 (9% shrink); unpickling is faster by 2%.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17808] No code example for Event object in threading module

2013-04-21 Thread Antoine Pitrou

Antoine Pitrou added the comment:

An example should generally show something interesting or non-obvious, which 
isn't the case here. IMHO an Event is simple enough to use that it doesn't need 
an example; furthermore, the example you are proposing doesn't really showcase 
anything interesting, functionally :-)

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17065] Fix sporadic buildbot failures for test_winreg

2013-04-21 Thread Jeremy Kloth

Jeremy Kloth added the comment:

Not to sound needy, but could the patch be looked into being integrated soon?  
This problem had only occurred once or twice a month however it has caused 
failures three times just in the last week.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17670] Improve str.expandtabs() doc

2013-04-21 Thread Eli Bendersky

Eli Bendersky added the comment:

LGTM!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue9682] socket.create_connection error message for domain subpart with invalid length is very confusing

2013-04-21 Thread Mike Milkin

Mike Milkin added the comment:

Sure ill modify the patch, thanks for the feedback.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17065] Fix sporadic buildbot failures for test_winreg

2013-04-21 Thread Roundup Robot

Roundup Robot added the comment:

New changeset 0882960fa6df by R David Murray in branch '3.3':
#17065: Use process-unique key for winreg test.
http://hg.python.org/cpython/rev/0882960fa6df

New changeset c7806d1b09eb by R David Murray in branch 'default':
Merge #17065: Use process-unique key for winreg test.
http://hg.python.org/cpython/rev/c7806d1b09eb

New changeset c0cb78bedc2b by R David Murray in branch '2.7':
#17065: Use process-unique key for winreg test.
http://hg.python.org/cpython/rev/c0cb78bedc2b

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17065] Fix sporadic buildbot failures for test_winreg

2013-04-21 Thread R. David Murray

R. David Murray added the comment:

Not being a windows dev I couldn't easily test the patch, so hopefully this 
commit won't break the buildbots :)

--
nosy: +r.david.murray
resolution:  -> fixed
stage: patch review -> committed/rejected
status: open -> closed
versions:  -Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17065] Fix sporadic buildbot failures for test_winreg

2013-04-21 Thread Jeremy Kloth

Jeremy Kloth added the comment:

Thank you!  There are no failures due to the patch and now its just a wait and 
see if test_winreg will misbehave again.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17810] Implement PEP 3154 (pickle protocol 4)

2013-04-21 Thread Serhiy Storchaka

Changes by Serhiy Storchaka :


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17804] streaming struct unpacking

2013-04-21 Thread Serhiy Storchaka

Serhiy Storchaka added the comment:

> Well, according to a quick benchmark, iter_unpack() is 3x to 6x faster than 
> the grouper() + unpack() recipe.
> (it's also a bit more user-friendly)

Yes, It's mainly because a grouper written on Python. When it will be 
implemented in C, the difference will be less. This function will be useful 
beside struct. Note that in my patch for issue17618 struct.Struct("!{}I") is 
not used.

As for extending Struct, what you think about a more powerful feature? About a 
method which returns not an iterator, but an iterable and indexable sequence. 
Here is a sample Python implementation.

--
Added file: http://bugs.python.org/file29967/struct_array_view.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17804] streaming struct unpacking

2013-04-21 Thread Antoine Pitrou

Antoine Pitrou added the comment:

> Yes, It's mainly because a grouper written on Python. When it will be
> implemented in C, the difference will be less. This function will be
> useful beside struct.

I'm not against adding useful C tools to itertools, but you may have to
convince Raymond ;)

> As for extending Struct, what you think about a more powerful feature?
> About a method which returns not an iterator, but an iterable and
> indexable sequence. Here is a sample Python implementation.

I'll take a look, but the question is how complex a C implementation
would be.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17810] Implement PEP 3154 (pickle protocol 4)

2013-04-21 Thread Serhiy Storchaka

Serhiy Storchaka added the comment:

Link to the previous attempt: issue15642.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16231] pickle persistent_id return value

2013-04-21 Thread Serhiy Storchaka

Serhiy Storchaka added the comment:

It was fixed for Python 3 in 0ae50aa7d97c. Should it be fixed in 2.7 too or 
close the issue as won't fix? Note that cPickle tests the return value of 
persistent_id only for None.

--
nosy: +alexandre.vassalotti, pitrou, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17810] Implement PEP 3154 (pickle protocol 4)

2013-04-21 Thread Serhiy Storchaka

Serhiy Storchaka added the comment:

Memoization consumes memory during pickling. For now every memoized object 
requires memory for:

dict's entity;
an id() integer object;
a 2-element tuple;
a pickle's index (an integer object).

It's about 80 bytes on 32-bit platform (and twice as this on 64-bit). For data 
which contains a lot of floats it can be cumbersome.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17348] Unicode - encoding seems to be lost for inputs of unicode chars

2013-04-21 Thread Tomoki Imai

Tomoki Imai added the comment:

NO,this thread should not be closed!
This is IDLE Bug.I found, IDLE has issue in using unicode literal.

In normal interpreter in console.
>>> u"こんにちは"
u'\u3053\u3093\u306b\u3061\u306f'

In IDLE.
>>> u"こんにちは"
u'\xe3\x81\x93\xe3\x82\x93\xe3\x81\xab\xe3\x81\xa1\xe3\x81\xaf'

I take a look IDLE codes, found bug in IDLE.
In idlelib/PyShell.py.

def runsource(self, source):
"Extend base class method: Stuff the source in the line cache first"
filename = self.stuffsource(source)
self.more = 0
self.save_warnings_filters = warnings.filters[:]
warnings.filterwarnings(action="error", category=SyntaxWarning)
print(source,len(source))

if isinstance(source, types.UnicodeType):
from idlelib import IOBinding
try:
source = source.encode(IOBinding.encoding)
except UnicodeError:
self.tkconsole.resetoutput()
self.write("Unsupported characters in input\n")
return
try:
print(source,len(source))
# InteractiveInterpreter.runsource() calls its runcode() method,
# which is overridden (see below)
return InteractiveInterpreter.runsource(self, source, filename)
finally:
if self.save_warnings_filters is not None:
warnings.filters[:] = self.save_warnings_filters
self.save_warnings_filters = None


This codes change u"こんにちは" to 
u'\xe3\x81\x93\xe3\x82\x93\xe3\x81\xab\xe3\x81\xa1\xe3\x81\xaf'
I commented out  following lines.

if isinstance(source, types.UnicodeType):
from idlelib import IOBinding
try:
source = source.encode(IOBinding.encoding)
except UnicodeError:
self.tkconsole.resetoutput()
self.write("Unsupported characters in input\n")
return

And now works.
Not well tested, I'll do unittest in GSoC (if I can).

--
components: +IDLE -Unicode
keywords: +patch
nosy: +Tomoki.Imai
type:  -> behavior
Added file: http://bugs.python.org/file29968/PyShell.py.20130422.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17618] base85 encoding

2013-04-21 Thread Serhiy Storchaka

Serhiy Storchaka added the comment:

As for interface, I think 'adobe' flag should be false by default. It makes 
encoder simpler. ascii85 encoder in Go's standard library doesn't wrap nor add 
Adobe's brackets. btoa/atob functions looks redundant as we can just use 
a85encode/a85decoder with appropriate options. Perhaps we should get rid from 
'adobe' flag in a85decode and autodetect it. And perhaps to do the same with 
other a85decode's options.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17618] base85 encoding

2013-04-21 Thread Martin Morrison

Martin Morrison added the comment:

On 21 Apr 2013, at 17:38, Serhiy Storchaka  wrote:
> Serhiy Storchaka added the comment:
> 
> As for interface, I think 'adobe' flag should be false by default. It makes 
> encoder simpler. ascii85 encoder in Go's standard library doesn't wrap nor 
> add Adobe's brackets. btoa/atob functions looks redundant as we can just use 
> a85encode/a85decoder with appropriate options. Perhaps we should get rid from 
> 'adobe' flag in a85decode and autodetect it. And perhaps to do the same with 
> other a85decode's options.

The problem with autodetecting is that it makes it impossible for an 
application to use this library to verify that something is encoded in a 
specific way. Explicit is better than implicit. 

Otherwise, your changes look good to me.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17810] Implement PEP 3154 (pickle protocol 4)

2013-04-21 Thread Antoine Pitrou

Antoine Pitrou added the comment:

> Memoization consumes memory during pickling. For now every memoized
> object requires memory for:
> 
> dict's entity;
> an id() integer object;
> a 2-element tuple;
> a pickle's index (an integer object).
> 
> It's about 80 bytes on 32-bit platform (and twice as this on 64-bit).

As far as I understand, Alexandre doesn't propose to suppress
memoization, only to make it implicit. Therefore the memory overhead
would be the same (but the pickle would have less opcodes).

> For data which contains a lot of floats it can be cumbersome.

Apparently, floats don't get memoized:

>>> pickletools.dis(pickle.dumps([1.0, 2.0]))
0: \x80 PROTO  3
2: ]EMPTY_LIST
3: qBINPUT 0
5: (MARK
6: GBINFLOAT   1.0
   15: GBINFLOAT   2.0
   24: eAPPENDS(MARK at 5)
   25: .STOP

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17764] Support http.server passing bind address via commend line argument

2013-04-21 Thread Malte Swart

Malte Swart added the comment:

I have updated the patch and added a paragraph for this option to the 
documentation. 

Shall I add this issue to the changelog list for python 3.4.0 alpha 1?

--
Added file: http://bugs.python.org/file29969/http-server-bind-arg2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17795] backwards-incompatible change in SysLogHandler with unix domain sockets

2013-04-21 Thread Vinay Sajip

Changes by Vinay Sajip :


Added file: http://bugs.python.org/file29970/cd970801b061.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17795] backwards-incompatible change in SysLogHandler with unix domain sockets

2013-04-21 Thread Vinay Sajip

Vinay Sajip added the comment:

Okay, I made the change to default socktype=None. Please try out the latest 
patch (ideally on all Python versions you can test with) to confirm it's OK. 
Then I can apply to 2.7/3.2/3.3/default. Thanks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17348] Unicode - encoding seems to be lost for inputs of unicode chars

2013-04-21 Thread R. David Murray

R. David Murray added the comment:

I believe you have indeed understood what the original poster was reporting.

However, those lines date back a long time (2002 or earlier).  They exist in 
Python2 only, and there they have a purpose, so they can't just be deleted.

My guess is the problem is a conflict between the locale setting and the 
encoding used when the character string is input into IDLE.

For me, if I cut and paste that string into the idle shell in python2, it shows 
up as the unicode escape characters (meaning IDLE is doing the correct 
conversion at input time on my system).  In Python3 it looks the same, except 
that the echoed output shows the expected glyphs instead of the unicode escapes 
as it does in Python2, which is as expected.

My only locale setting, by the way, is LC_CTYPE=en_US.UTF-8.  What is your 
setting?

I don't know if there is a better way for idle to behave in the modern era or 
not.  Perhaps it should be using utf-8 by default instead of the locale?  Do 
you know how (and in what charset) your system is generating the characters you 
type into idle?

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17348] Unicode - encoding seems to be lost for inputs of unicode chars in IDLE

2013-04-21 Thread R. David Murray

Changes by R. David Murray :


--
title: Unicode - encoding seems to be lost for inputs of unicode chars -> 
Unicode - encoding seems to be lost for inputs of unicode chars in IDLE

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17670] Improve str.expandtabs() doc

2013-04-21 Thread Roundup Robot

Roundup Robot added the comment:

New changeset 6a02d2af814f by Ned Deily in branch '2.7':
Issue #17670: Provide an example of expandtabs() usage.
http://hg.python.org/cpython/rev/6a02d2af814f

New changeset 5b6ccab52a4d by Ned Deily in branch '3.3':
Issue #17670: Provide an example of expandtabs() usage.
http://hg.python.org/cpython/rev/5b6ccab52a4d

New changeset 1a6cb6d8591a by Ned Deily in branch 'default':
Issue #17670: merge from 3.3
http://hg.python.org/cpython/rev/1a6cb6d8591a

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17670] Improve str.expandtabs() doc

2013-04-21 Thread Ned Deily

Changes by Ned Deily :


--
resolution:  -> fixed
stage: commit review -> committed/rejected
status: open -> closed
type: enhancement -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17764] Support http.server passing bind address via commend line argument

2013-04-21 Thread R. David Murray

R. David Murray added the comment:

Thanks for proposing this and working on it Malte.  Could you please submit a 
contributor agreement? (http://www.python.org/psf/contrib).

We will add the Misc/NEWS entry when we commit the patch; that file changes so 
rapidly that any patch to it quickly becomes outdated.  You could add an entry 
to Doc/whatsnew/3.4, though, if you were so moved.  If not we'll add that too 
when we commit.

For the doc patch, I think the first sentence would be clearer if it said "By 
default the server binds to all available interfaces."  I'll probably tweak a 
couple other words for better English phrasing when I commit it, as well.

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17742] Add _PyBytesWriter API

2013-04-21 Thread Antoine Pitrou

Antoine Pitrou added the comment:

The last patch increases the size of the code substantially. I'm still 
wondering what the benefits are.

$ hg di --stat
 Include/bytesobject.h  |   90 ++
 Misc/NEWS  |3 +
 Objects/bytesobject.c  |  144 ++
 Objects/stringlib/codecs.h |  109 
 Objects/unicodeobject.c|  267 -
 5 files changed, 368 insertions(+), 245 deletions(-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17403] Robotparser fails to parse some robots.txt

2013-04-21 Thread R. David Murray

R. David Murray added the comment:

Lucaz pointed out on IRC that the problem is that the current robotparser is 
implementing an outdated robots.txt standard.  He may work on fixing that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17403] Robotparser fails to parse some robots.txt

2013-04-21 Thread R. David Murray

Changes by R. David Murray :


--
keywords:  -easy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue15575] Tutorial is unclear on multiple imports of a module.

2013-04-21 Thread Roundup Robot

Roundup Robot added the comment:

New changeset 9df9931fae96 by R David Murray in branch '3.3':
#15575: Clarify tutorial description of when modules are executed.
http://hg.python.org/cpython/rev/9df9931fae96

New changeset dac847938326 by R David Murray in branch 'default':
#15575: Clarify tutorial description of when modules are executed.
http://hg.python.org/cpython/rev/dac847938326

New changeset a1421d28393b by R David Murray in branch '2.7':
#15575: Clarify tutorial description of when modules are executed.
http://hg.python.org/cpython/rev/a1421d28393b

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue15575] Tutorial is unclear on multiple imports of a module.

2013-04-21 Thread R. David Murray

R. David Murray added the comment:

Thanks, James.  I wound up going with a different wording for the 
"elaboration": since the concept of running a python file as a script is 
mentioned just a bit earlier, I added a parenthetical that the statements are 
also executed if the module is run as a script.

--
resolution:  -> fixed
stage: needs patch -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17811] Improve os.readv() and os.writev() documentation and docstring

2013-04-21 Thread Nikolaus Rath

New submission from Nikolaus Rath:

The os.writev and os.readv functions are currently documented as:

os.writev(fd, buffers)

Write the contents of buffers to file descriptor fd, where buffers is an 
arbitrary sequence of buffers. Returns the total number of bytes written.

os.readv(fd, buffers)

Read from a file descriptor into a number of writable buffers. buffers is 
an arbitrary sequence of writable buffers. Returns the total number of bytes 
read.


This is rather confusing, mostly because it's not clear what can be passed as 
*buffer* (since buffer objects don't exist in Python 3 anymore), and (as a 
consequence) how the input will be distributed among the list of buffers.

Reading the code, it seems to me that any object that implements the buffer 
protocol would be fine, but that still doesn't help much in practice. From the 
memoryview documentation, I can infer that `bytes` implements the buffer 
protocol, but is also immutable, so how can something be written into it?

I'd be happy to write a patch to the documentation, but someone would need to 
explain to me first what kind of buffers are actually acceptable (and I suspect 
this will be different for readv and writev).

--
assignee: docs@python
components: Documentation
messages: 187526
nosy: Nikratio, docs@python
priority: normal
severity: normal
status: open
title: Improve os.readv() and os.writev() documentation and docstring
type: enhancement
versions: Python 3.3, Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17812] Quadratic complexity in b32encode

2013-04-21 Thread Serhiy Storchaka

New submission from Serhiy Storchaka:

b32encode accumulates encoded data in a bytes object and this operation has 
quadratic complexity.

Here is a patch, which fixes this issue by accumulating in a list.

--
components: Library (Lib)
files: base32_fix.patch
keywords: patch
messages: 187527
nosy: pitrou, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Quadratic complexity in b32encode
type: performance
versions: Python 3.3, Python 3.4
Added file: http://bugs.python.org/file29971/base32_fix.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17811] Improve os.readv() and os.writev() documentation and docstring

2013-04-21 Thread Nikolaus Rath

Nikolaus Rath added the comment:

Here's a first attempt at improvement based on my guess:

os.writev(fd, buffers)

Write the contents of buffers to file descriptor fd, where buffers is an 
arbitrary sequence of buffers. In this context, a buffer may be any Python 
object that provides a Memory View, i.e. that may be used to instantiate a 
`memoryview` object (e.g. a bytes or bytearray object). writev writes the 
contents of each memory view to the file descriptor and returns the total 
number of bytes written.

os.readv(fd, buffers)

Read from a file descriptor into a number of writable buffers. buffers is 
an arbitrary sequence of writable buffers. In this context, a buffer may be any 
Python object that provides a writeable Memory View, i.e. that may be used to 
instantiate a `memoryview` object whose `read_only` attribute is false (for 
example, a bytearray object). Writeable memory views have a fixed size, and 
readv will transfer data into each buffer until it is full and then move on to 
the next buffer in the sequence to hold the rest of the data. readv returns the 
total number of bytes read (which may be less than the total capacity of all 
the buffers).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17812] Quadratic complexity in b32encode

2013-04-21 Thread Serhiy Storchaka

Serhiy Storchaka added the comment:

And here are other patch, which not only fixes an issue with quadratic 
complexity, but optimize b32encode and b32decode about 2.5 times.

Microbenchmarks:

./python -m timeit -r 1 -n 10 -s "from base64 import b32encode as encode; data 
= open('python', 'rb').read(101)"  "encode(data)"
./python -m timeit -r 1 -n 1 -s "from base64 import b32encode as encode, 
b32decode as decode; data = encode(open('python', 'rb').read(101))"  
"decode(data)"

Results:
   First patch  Second patch
b32encode1.25 sec 486 msec
b32decode2.08 sec 835 msec

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17812] Quadratic complexity in b32encode

2013-04-21 Thread Serhiy Storchaka

Changes by Serhiy Storchaka :


Added file: http://bugs.python.org/file29972/base32_optimize.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17811] Improve os.readv() and os.writev() documentation and docstring

2013-04-21 Thread R. David Murray

R. David Murray added the comment:

Well, the documentation is technically precise.  I'd even managed to forget 
that buffer objects existed in Python2 :)

As you observed, in Python3 a buffer is something that implements the buffer 
protocol.  What I would do is link the word 'buffer' to 
http://docs.python.org/3/c-api/buffer.html.  Perhaps mentioning bytes and 
bytearray as examples of read-only and writeable buffers would be worthwhile, 
but they are mentioned in the first paragraph of that section so it may not be 
necessary.

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17811] Improve os.readv() and os.writev() documentation and docstring

2013-04-21 Thread Nikolaus Rath

Nikolaus Rath added the comment:

What section do you mean? bytearray is not mentioned anywhere in
http://docs.python.org/3.4/library/os.html.

I think the problem with just linking to the C API section is that it doesn't 
help people that are only using pure Python. You can't look at a Python object 
and tell whether it implements the buffer protocol or not. Therefore, I think 
it would be important to give at least a reference to memoryview as the Python 
equivalent of the buffer protocol.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17813] lzma and bz2 decompress methods lack max_size attribute

2013-04-21 Thread Nikolaus Rath

New submission from Nikolaus Rath:

The zlib Decompress.decompress has a max_length parameter that limits the size 
of the returned uncompressed data.

The lzma and bz2 decompress methods do not have such a parameter.

Therefore, it is not possible to decompress untrusted lzma or bz2 data without 
becoming susceptible to a DoS attack, as the attacker can force allocation of 
gigantic buffers by sending just a tiny amount of compressed data:

$ dd if=/dev/zero bs=128k count=10k | bzip2 -9 > nasty.bz2
10240+0 records in
10240+0 records out
1342177280 bytes (1.3 GB) copied, 11.0892 s, 121 MB/s
$ dir nasty.bz2 
-rw-rw-r-- 1 nikratio nikratio 977 Apr 21 14:58 nasty.bz2


It would be great if the lzma and bz2 decompressor methods could also get a 
max_length parameters to make this situation less threatening.

--
components: Library (Lib)
messages: 187532
nosy: Nikratio
priority: normal
severity: normal
status: open
title: lzma and bz2 decompress methods lack max_size attribute
type: behavior
versions: Python 3.3, Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17813] lzma and bz2 decompress methods lack max_size parameter

2013-04-21 Thread Nikolaus Rath

Changes by Nikolaus Rath :


--
title: lzma and bz2 decompress methods lack max_size attribute -> lzma and bz2 
decompress methods lack max_size parameter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17813] lzma and bz2 decompress methods lack max_size parameter

2013-04-21 Thread Serhiy Storchaka

Changes by Serhiy Storchaka :


--
resolution:  -> duplicate
superseder:  -> gzip, bz2, lzma: add option to limit output size

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue14398] bz2.BZ2DEcompressor.decompress fail on large files

2013-04-21 Thread Nadeem Vawda

Nadeem Vawda added the comment:

Hmm, so actually most of the bugs fixed in 2.7 and 3.2 weren't present
in 3.3 and 3.4, and those versions already had tests equivalent to the
tests I added for 2.7/3.2.

As for the changes that I did make to 3.3/3.4:

- two of the three cover cases that only occur if the output data is
  larger than ~32GiB. Even if we have a buildbot with enough memory for
  it (which I don't think we do), actually running such tests would take
  forever and then some.

- the third is for a condition that's actually pretty much impossible to
  trigger - grow_buffer() has to be called on a buffer that is already at
  least 8*((size_t)-1)/9 bytes long. On a 64-bit system this is
  astronomically large, while on a 32-bit system the OS will probably
  have reserved more than 1/9th of the virtual address space for itself,
  so it won't be possible to allocate a large enough buffer.

--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue15955] gzip, bz2, lzma: add option to limit output size

2013-04-21 Thread Nikolaus Rath

Nikolaus Rath added the comment:

The lack of output size limiting has security implications as well.

Without being able to limit the size of the uncompressed data returned per 
call, it is not possible to decompress untrusted lzma or bz2 data without 
becoming susceptible to a DoS attack, as the attacker can force allocation of 
gigantic buffers by sending just a tiny amount of compressed data.

--
nosy: +Nikratio

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17814] Popen.stdin/stdout/stderr documentation should mention object type

2013-04-21 Thread Nikolaus Rath

New submission from Nikolaus Rath:

The subprocess documentation currently just says that Popen.stdin et all are 
"file objects", which is linked to the glossary entry. This isn't very helpful, 
as it doesn't tell whether the streams are bytes or text streams.

Suggested patch:

diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst
--- a/Doc/library/subprocess.rst
+++ b/Doc/library/subprocess.rst
@@ -677,21 +677,29 @@
 
 .. attribute:: Popen.stdin
 
-   If the *stdin* argument was :data:`PIPE`, this attribute is a :term:`file
-   object` that provides input to the child process.  Otherwise, it is 
``None``.
+   If the *stdin* argument was :data:`PIPE`, this attribute is a
+   :class:`io.BufferedWriter` (if the *universal_newlines* argument
+   was False) or :class:`io.TextIOWrapper` (if the
+   *universal_newlines* argument was True) object that provides input
+   to the child process. Otherwise, it is ``None``.
 
 
 .. attribute:: Popen.stdout
 
-   If the *stdout* argument was :data:`PIPE`, this attribute is a :term:`file
-   object` that provides output from the child process.  Otherwise, it is 
``None``.
+   If the *stdout* argument was :data:`PIPE`, this attribute is a
+   :class:`io.BufferedReader` (if the *universal_newlines* argument
+   was False) or :class:`io.TextIOWrapper` (if the
+   *universal_newlines* argument was True) object that provides output
+   from the child process. Otherwise, it is ``None``.
 
 
 .. attribute:: Popen.stderr
 
-   If the *stderr* argument was :data:`PIPE`, this attribute is a :term:`file
-   object` that provides error output from the child process.  Otherwise, it is
-   ``None``.
+   If the *stderr* argument was :data:`PIPE`, this attribute is a
+   :class:`io.BufferedReader` (if the *universal_newlines* argument
+   was False) or :class:`io.TextIOWrapper` (if the
+   *universal_newlines* argument was True) object that provides output
+   from the child process. Otherwise, it is ``None``.
 
 
 .. attribute:: Popen.pid

--
assignee: docs@python
components: Documentation
messages: 187535
nosy: Nikratio, docs@python
priority: normal
severity: normal
status: open
title: Popen.stdin/stdout/stderr documentation should mention object type
type: enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17814] Popen.stdin/stdout/stderr documentation should mention object type

2013-04-21 Thread Ezio Melotti

Ezio Melotti added the comment:

I think the specific classes are an implementation detail, but mentioning if 
the file object is a bytes or text stream seems reasonable.
(Also it's better if you attach patches, instead of pasting them in the 
message.)

--
nosy: +ezio.melotti
stage:  -> needs patch
versions: +Python 2.7, Python 3.3, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17348] Unicode - encoding seems to be lost for inputs of unicode chars in IDLE

2013-04-21 Thread Tomoki Imai

Tomoki Imai added the comment:

Thanks.

I noticed Terry used python3 to confirm this problem...

I am Japanese, but using English environment.
Here is my locale settings. And I'm using Linux.
konomi:tomoki% locale
LANG=en_US.utf8
LC_CTYPE=en_US.UTF-8
LC_NUMERIC="en_US.utf8"
LC_TIME="en_US.utf8"
LC_COLLATE="en_US.utf8"
LC_MONETARY="en_US.utf8"
LC_MESSAGES="en_US.utf8"
LC_PAPER="en_US.utf8"
LC_NAME="en_US.utf8"
LC_ADDRESS="en_US.utf8"
LC_TELEPHONE="en_US.utf8"
LC_MEASUREMENT="en_US.utf8"
LC_IDENTIFICATION="en_US.utf8"
LC_ALL=

All strings used internally should be unicode type.
In Japan, many many charset is here.(cp932,euc-jp,...).
And, they causes problems in Python2 without converting it to unicode type.
Remember, unicode type and "utf-8" is not same.

When I type into Tkinter's Entry and get Entry's value,it returned me unicode.
And deleted code converts unicode to str type.
They are unified in Python3.(unicode become str,and str become byte).
So, these lines are not in Python3 codes.

I typed these strings using "Input Method"(am using uim).
https://code.google.com/p/uim/
But, I don't know how uim generate these characters.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17814] Popen.stdin/stdout/stderr documentation should mention object type

2013-04-21 Thread Nikolaus Rath

Nikolaus Rath added the comment:

I think it would also be rather important to know if the streams are buffered 
or not.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue15392] Create a unittest framework for IDLE

2013-04-21 Thread Jayakrishnan

Jayakrishnan added the comment:

The aimed versions for this unit test frame work is python 3.3, 3.4. So as Nick 
said, unittest.mock may have no issues on this.
As you said 3rd party modules seems not a better way.But the link you provided 
( https://pypi.python.org/pypi/mock ) says ..  "mock is now part of the 
Python standard library, available as unittest.mock in Python 3.3 onwards." 
And yes, GUI testings 
While I was writting my proposal on this project Todd.Rovito mentioned me  " .. 
you should still complete some GUI tests so future developers will have a 
model. "

--
nosy: +JayKrish

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17808] No code example for Event object in threading module

2013-04-21 Thread Andriy Mysyk

Andriy Mysyk added the comment:

Thank you for the feedback, Antoine. 

The example shows how to essentially kill threads through an event facilitated 
request, something that I consider to be useful and non-obvious.

I trust that you and others will make the right decision and will keep your 
feedback in mind for any future patches.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17348] Unicode - encoding seems to be lost for inputs of unicode chars in IDLE

2013-04-21 Thread R. David Murray

R. David Murray added the comment:

Well, it does seem to me that there is something wrong here.  Your fix may even 
be correct, but I'd hesitate to apply it without someone understanding why 
those lines were added in the first place.  (I *think* they were added by 
Martin von Loewis, but I'm not 100% sure since the commit was part of a block 
of changes by different authors and I'm just guessing based on the comment on 
the commit.)

I'm reopening the issue.  I'll have to leave it to the idle team (or you) to 
figure out what the correct fix is that doesn't break anything :)

--
nosy: +roger.serwy
resolution: invalid -> 
stage: committed/rejected -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17348] Unicode - encoding seems to be lost for inputs of unicode chars in IDLE

2013-04-21 Thread Terry J. Reedy

Terry J. Reedy added the comment:

When discussing problematical behavior, one should specify OS and exact Python 
version, including bugfix number. If at all possible, one should use the latest 
bugfix release with all released bugfixes. 2.7.3 came out 10+ months before the 
original report. I do not presume without evidence that it has the same 
behavior as the 2.7.2. The recently released 2.7.4 has another year of 
bugfixes, so it might also behave differently.

Looking again at the original report, I see that the false issue of lost 
encoding obscured to me a real problem: ord(u'€') is 8364, not 128. Does 2.7.4 
make the same error for that input? What does it do with u"こんにちは"?

(Note, on the Windows console, both keying and viewing unicode chars is 
problematical, apparently more so that with the *nix consoles. If I could not 
paste, u"こんにちは", I would most likely just key 
u'\u3053\u3093\u306b\u3061\u306f'.)

I believe the underlying problem is that a Python 2 program is a stream of 
bytes while a Python 3 program is a stream of unicode codepoints. So in Python 
2, a unicode literal has to be encoded to bytes before being decoded back to 
unicode codepoints in a unicode string object.

David, I presume this is why you say we cannot just toss out the encoding to 
bytes. I presume that you are also suggesting that the encoding and subsequent 
decoding are done with different codecs because of locale issues. Might 
IOBinding.encoding be miscalculated?

For ascii codepoints, the encoding and decoding is typically a null operation. 
This means that \u escapes, as opposed to non-ascii codepoints, should not 
get mangled before being interpreted during the creation of the unicode object. 
Using such escapes is one solution to the problem.

Another is to use Python 3. That *is* the generic answer to many Python 2.x 
unicode problems. In 3.3.1:
>>> u"こんにちは"
'こんにちは'
problem solved ;-).

In other words, fixing 2.7-only unicode bugs has fairly low priority in 
general. However, if there is an easy fix here that Roger thinks is safe, it 
can be applied.

--
resolution:  -> invalid
stage:  -> committed/rejected

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17717] Set up nasm from external.bat

2013-04-21 Thread John Ehresman

John Ehresman added the comment:

What I'd like is for external to set up all the dependencies needed to build 
python and run the test suite.  Yes, nasm can be downloaded and set up 
separately, but that's true of all of the libraries that external.bat downloads.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17795] backwards-incompatible change in SysLogHandler with unix domain sockets

2013-04-21 Thread Mike Lundy

Mike Lundy added the comment:

I've tested in our full dev setup, and it seems to work fine; I've also tested 
on my laptop, no problems there either.

Unfortunately, that's python 2.7.4 in both cases. I don't really have a python3 
setup I can bump up to test.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue15392] Create a unittest framework for IDLE

2013-04-21 Thread Jayakrishnan

Jayakrishnan added the comment:

There is a need of a proper design in whether to put tests in test sub 
directory or to create idlelib/test directory.

For my GSoc proposal initial draft, I suggested to start with  
Put tests in test/test_idle directory (like test/test_email would be the best 
practice).
Test for the presence of idle and skip if idle is not available.
Move the existing,pending tests for idle ( 
http://bugs.python.org/file27613/issue16226_test.patch , 
http://bugs.python.org/file24075/test_idlelib.py,)  into this directory.
(which my patch 15392_idletestframework_1.patch did for me)

Considering the point terry.reedy mentioned, "Adding test/test_idle will not be 
too much use until issue #10652 is resolved so it would actually run with -m 
test", my view is, the issue #10652 on this message is on a healthy patch 
review, so it would actually run with -m test in near-future
Expecting your suggestions on this design decisions.

--
keywords: +patch
Added file: http://bugs.python.org/file29973/15392_idletestframework_1.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17348] Unicode - encoding seems to be lost for inputs of unicode chars in IDLE

2013-04-21 Thread Tomoki Imai

Tomoki Imai added the comment:

Sorry.I forgot to note my environment.

I'm using Arch Linux.
$ uname -a
Linux manaka 3.8.7-1-ARCH #1 SMP PREEMPT Sat Apr 13 09:01:47 CEST 2013 x86_64 
GNU/Linux

And python version is here.
$ python --version
Python 2.7.4

IDLE's version is same, 2.7.4 downloaded from following link.
http://www.python.org/download/releases/2.7.4/

In IDLE,I repeated original author's attempts.

Python 2.7.4 (default, Apr  6 2013, 19:20:36)
[GCC 4.8.0] on linux2
Type "copyright", "credits" or "license()" for more information.
>>> c = u'€'
>>> ord(c)

Traceback (most recent call last):
  File "", line 1, in 
ord(c)
TypeError: ord() expected a character, but string of length 3 found
>>> c.encode('utf-8')
'\xc3\xa2\xc2\x82\xc2\xac'
>>> c
u'\xe2\x82\xac'
>>> print c
€
>>> c = u'\u20ac'
>>> ord(c)
8364
>>> c.encode('utf-8')
'\xe2\x82\xac'
>>> c
u'\u20ac'
>>> print c
€
>>>

I have a problem.But it is different from original.
After my fix.

Python 2.7.4 (default, Apr  6 2013, 19:20:36)
[GCC 4.8.0] on linux2
Type "copyright", "credits" or "license()" for more information.
>>> c = u'€'
>>> ord(c)
8364
>>> c.encode('utf-8')
'\xe2\x82\xac'
>>> c
u'\u20ac'
>>> print c
€
>>>

It works.

Using unicode escape is one solution.
But, we Japanese can type u'こんにちは' just in 10 or 5 key types.
And other people who use unicode literals for their language have same 
situation.
Why IDLE users (probably beginner) use such workaround ?

Of cource, using Python3 is best way.
All beginner should start from Python3 now.
But, there are people including me who have to use python2 because of libraries 
.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17348] Unicode - encoding seems to be lost for inputs of unicode chars in IDLE

2013-04-21 Thread Ned Deily

Ned Deily added the comment:

I think this is another case where confusion is introduced by the behavior of 
Python 2 interactive mode with regard to encodings.  In 2.x Python/pythonrun.c, 
depending on a number of factors the interactive loop will try to set a more 
useful encoding on stdin, stdout, and stderr for any of them assigned to a 
terminal (isatty)  This is why the examples using the interpreter (not IDLE) 
work when the locale env variables (LANG, et al) are set to one that supports 
Unicode (like "en_US.utf8").  If you changed them to a non-Unicode locale (like 
"C") and ran the interpreter examples, they wouldn't work.  Even clearer, if 
you used a Unicode-aware text editor to write the examples to files without a  
# -*- coding: utf-8 -*- directive and tried to run them with 
/usr/bin/python2.7, you'd get a syntax error.  I suspect the issue here is that 
2.x IDLE PyShell does not exactly mimic the behavior of the interpreter 
interactive loop about setting encodings for std*.  A side issue is the 
limitation o
 f Tk's support of non-BMP characters; because of that, it may not possible to 
fully mimic the interactive interpreter.

--
nosy: +ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17348] Unicode - encoding seems to be lost for inputs of unicode chars in IDLE

2013-04-21 Thread Ned Deily

Ned Deily added the comment:

Also see Issue15809 in which Martin proposed the same patch but then explained 
why it isn't totally correct.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17348] Unicode - encoding seems to be lost for inputs of unicode chars in IDLE

2013-04-21 Thread Ned Deily

Changes by Ned Deily :


--
Removed message: http://bugs.python.org/msg187547

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17348] Unicode - encoding seems to be lost for inputs of unicode chars in IDLE

2013-04-21 Thread R. David Murray

R. David Murray added the comment:

For those of us without fancy input methods, I was able to see the same problem 
using a simple non-ascii letter (an accented a: á.  You will note that my stdin 
encoding ought to be utf-8, so I'm not sure why it fails (but I didn't check 
that).  Removing the lines in question makes it work, but as Martin says in the 
referenced issue I won't be surprised if that breaks working with a file that 
has a non-utf8 coding cookie.

If Martin sees no way to make it work in Python2 it makes it pretty unlikely 
we'll find a fix :(.

--
resolution: invalid -> 
stage: committed/rejected -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com