Re: Strange behaviour with os.linesep

[Vincent Vande Vyvre]

Le 23/07/2013 17:25, Steven D'Aprano a écrit :

On Tue, 23 Jul 2013 13:42:13 +0200, Vincent Vande Vyvre wrote:


On Windows a script where de endline are the system line sep, the files
are open with a double line in Eric4, Notepad++ or Gedit but they are
correctly displayed in the MS Bloc-Notes.

I suspect the problem lies with Eric4, Notepad++ and Gedit. Do you
perhaps have to manually tell them that the file uses Windows line
separators?

I recommend opening the file in a hex editor and seeing for yourself what
line separators are used.



Example with this code:
--
# -*- coding: utf-8 -*-

import os
L_SEP = os.linesep

def write():
  strings = ['# -*- coding: utf-8 -*-\n',
  'import os\n',
  'import sys\n']
  with open('writetest.py', 'w') as outf:
  for s in strings:
  outf.write(s.replace('\n', L_SEP))

write()
--

The syntax `s.replace('\n', L_SEP)`is required for portability.

I don't think it is. Behaviour is a little different between Python 2 and
3, but by default, Python uses "Universal Newlines". When you open a file
in text mode, arbitrary line separators should be automatically
translated to \n when reading, and \n will be automatically translated to
os.line_sep when writing.


http://docs.python.org/3/library/functions.html#open
http://docs.python.org/2/library/functions.html#open

Some further discussion here:

http://stackoverflow.com/questions/12193047/is-universal-newlines-mode-
supposed-to-be-default-behaviour-for-open-in-python



In fact, in my code, the original file is open in binary mode, the line 
separator is translate to \n and it is parsed by the module tokenise.


I'm not a Windows user but my code must be run also on Win, this is the 
reason of the usage of os.linesep in writting.


So, now I found the solution, just write the file in binary mode and now 
it is correctly open with all the editors.


Thanks all

--
Vincent V.V.
Oqapy  . Qarte 
 . PaQager 

--
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behaviour with os.linesep

[Chris Angelico]

On Wed, Jul 24, 2013 at 5:02 PM, Vincent Vande Vyvre
 wrote:
> In fact, in my code, the original file is open in binary mode, the line
> separator is translate to \n and it is parsed by the module tokenise.
>
> I'm not a Windows user but my code must be run also on Win, this is the
> reason of the usage of os.linesep in writting.
>
> So, now I found the solution, just write the file in binary mode and now it
> is correctly open with all the editors.
>

Sounds to me like the problem was double-translation - you in your
code turned \n into \r\n, but then the file was written in text mode,
doing the same translation, so your lines were terminated with \r\r\n.
That would result in what you're seeing (including the oddity that
some editors will show the file differently).

You may find it easier to write the file in text mode and let the
underlying system do the translation for you. Chances are that'll be
correct.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

i want to run user specific cronjobs so where to specify user in ubuntu?

[navnathgadakh]

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: i want to run user specific cronjobs so where to specify user in ubuntu?

[Chris Angelico]

On Wed, Jul 24, 2013 at 5:47 PM,   wrote:
>
> --
> http://mail.python.org/mailman/listinfo/python-list

Not a Python question. I'm sure Google can help you with this one -
just take three words from your question, 'user specific cronjobs',
and you'll get plenty of advice.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Contact information for Jim Hugunin?

[Larry Hastings]

Does anybody have an email address (or anything, really) for Jim 
Hugunin?  He left Google in May and appears to have dropped off the face 
of the internet.  Please email me privately.


I swear I will use the information only for good and never for evil,


//arry/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Contact information for Jim Hugunin?

[Albert Hopkins]

On Mon, Jul 22, 2013, at 05:33 PM, Larry Hastings wrote:
> 
> 
> Does anybody have an email address (or anything, really) for Jim 
> Hugunin?  He left Google in May and appears to have dropped off the face 
> of the internet.  Please email me privately.
> 
> I swear I will use the information only for good and never for evil,

Is that your definition of "good" and "evil" or mine?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: how: embed + extend to control my running app?

[Gregory Ewing]

David M. Cotter wrote:

For Mac, I understand i need to "create" (?) a python.dylib,


If your Python was installed as a framework, you should
already have one. Just link your application with "-framework Python".


Now for Windows: same thing, i think i must create a .dll, right?


Again, you should already have a python.dll in your installation
somewhere.

--
Greg
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: dict & dict.keys()

[Oscar Benjamin]

On Jul 24, 2013 7:25 AM, "Peter Otten" <__pete...@web.de> wrote:
>
> Ethan Furman wrote:
>
> > So, my question boils down to:  in Python 3 how is dict.keys() different
> > from dict?  What are the use cases?
>
> I just grepped through /usr/lib/python3, and could not identify a single
> line where some_object.keys() wasn't either wrapped in a list (or set,
> sorted, max) call, or iterated over.
>
> To me it looks like views are a solution waiting for a problem.

What do you mean? Why would you want to create a temporary list just to
iterate over it explicitly or implicitly (set, sorted, max,...)?

Oscar
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: dict & dict.keys()

[Neil Cerutti]

On 2013-07-24, Peter Otten <__pete...@web.de> wrote:
>> So, my question boils down to:  in Python 3 how is dict.keys()
>> different from dict?  What are the use cases?
>
> I just grepped through /usr/lib/python3, and could not identify
> a single line where some_object.keys() wasn't either wrapped in
> a list (or set, sorted, max) call, or iterated over.
>
> To me it looks like views are a solution waiting for a problem.

Here's a case of using keys as a set-like view from my own
"production" code (i.e., I used it once for one important job):

seen = set()
students = {}
dates = []

for fname in sorted(glob.glob("currentterm201320?.csv")):
print(fname, end="\n\t")
date = get_date(fname)
dates.append(date)
term = fname[-11:-4]
r = reg.registration(term, path=".")
regs = r.keys()
for alt_id in regs & seen:
students[alt_id].append(r[alt_id])
for alt_id in seen - regs:
students[alt_id].append(None)
for alt_id in regs - seen:
students[alt_id] = [None]*(len(dates)-1) + [r[alt_id]]
seen.add(alt_id)

It was a very nice way to to do three different things depending
on the student sin the set I was working with, compared to a
registration list:

Granted the line was originally "regs = set(regs.keys())" before
it occurred to me that it sucked to take what must be equivalent
to a set, convert to a list, and then back to set again.

Thanks to the set-like view of dict.keys it worked just like one
might hope.

Looking at it again "seen" might be a redundant parallel version
of students.keys().

-- 
Neil Cerutti
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: dict & dict.keys()

[Peter Otten]

Oscar Benjamin wrote:

> On Jul 24, 2013 7:25 AM, "Peter Otten" <__pete...@web.de> wrote:
>>
>> Ethan Furman wrote:
>>
>> > So, my question boils down to:  in Python 3 how is dict.keys()
>> > different
>> > from dict?  What are the use cases?
>>
>> I just grepped through /usr/lib/python3, and could not identify a single
>> line where some_object.keys() wasn't either wrapped in a list (or set,
>> sorted, max) call, or iterated over.
>>
>> To me it looks like views are a solution waiting for a problem.
> 
> What do you mean? Why would you want to create a temporary list just to
> iterate over it explicitly or implicitly (set, sorted, max,...)?

I mean I don't understand the necessity of views when all actual usecases 
need iterators. The 2.x iterkeys()/iteritems()/itervalues() methods didn't 
create lists either.

Do you have 2.x code lying around where you get a significant advantage by 
picking some_dict.viewkeys() over some_dict.iterkeys()? I could construct 
one

>>> d = dict(a=1, b=2, c=3)
>>> e = dict(b=4, c=5, d=6)
>>> d.viewkeys() & e.viewkeys()
set(['c', 'b'])

but have not seen it in the wild.

My guess is that most non-hardcore users don't even know about viewkeys(). 
By the way, my favourite idiom to iterate over the keys in both Python 2 and 
3 is -- for example -- max(some_dict) rather than 
max(some_dict.whateverkeys()).

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: hex dump w/ or w/out utf-8 chars

[wxjmfauth]

I do not find the thread, where a Python core dev spoke
about French, so I'm putting here.

This stupid Flexible String Representation splits Unicode
in chunks and one of these chunks is latin-1 (iso-8859-1).

If we consider that latin-1 is unusable for 17 (seventeen)
European languages based on the latin alphabet, one can not
say Python is really well prepared.

Most of the problems are coming from the extensive usage of
diacritics in these languages. Thanks to the FSR again,
working with normalized forms does not work very well. At
least, there is some consistency.

Now, if we consider that most of the new characters will
be part of the BMP ("daily" used chars), it is hard to
present Python as a modern language. It sticks more
to the past and it not really prepared for the future,
the acceptance of new chars like ẞ or the new Turkish lira
sign ((U+20BA).

>>> sys.getsizeof('š')
40
>>> sys.getsizeof('0')
26

14 bytes to encode a non-latin-1 char is not so bad.


jmf

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[wxjmfauth]

Le samedi 13 juillet 2013 01:13:47 UTC+2, Michael Torrie a écrit :
> On 07/12/2013 09:59 AM, Joshua Landau wrote:
> 
> > If you're interested, the basic of it is that strings now use a
> 
> > variable number of bytes to encode their values depending on whether
> 
> > values outside of the ASCII range and some other range are used, as an
> 
> > optimisation.
> 
> 
> 
> Variable number of bytes is a problematic way to saying it.  UTF-8 is a
> 
> variable-number-of-bytes encoding scheme where each character can be 1,
> 
> 2, 4, or more bytes, depending on the unicode character.  As you can
> 
> imagine this sort of encoding scheme would be very slow to do slicing
> 
> with (looking up a character at a certain position).  Python uses
> 
> fixed-width encoding schemes, so they preserve the O(n) lookup speeds,
> 
> but python will use 1, 2, or 4 bytes per every character in the string,
> 
> depending on what is needed.  Just in case the OP might have
> 
> misunderstood what you are saying.
> 
> 
> 
> jmf sees the case where a string is promoted from one width to another,
> 
> and thinks that the brief slowdown in string operations to accomplish
> 
> this is a problem.  In reality I have never seen anyone use the types of
> 
> string operations his pseudo benchmarks use, and in general Python 3's
> 
> string behavior is pretty fast.  And apparently much more correct than
> 
> if jmf's ideas of unicode were implemented.

--

Sorry, you are not understanding Unicode. What is a Unicode
Transformation Format (UTF), what is the goal of a UTF and
why it is important for an implementation to work with a UTF.

Short example. Writing an editor with something like the
FSR is simply impossible (properly).

jmf
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[Chris Angelico]

On Wed, Jul 24, 2013 at 11:40 PM,   wrote:
> Short example. Writing an editor with something like the
> FSR is simply impossible (properly).

jmf, have you ever written an editor with *any* string representation?
Are you speaking from any level of experience at all?

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[David Hutto]

I've screwed up plenty of times in python, but can write code like a pro
when I'm feeling better(on SSI and medicaid). An editor can be built
simply, but it's preference that makes the difference. Some might have used
tkinter, gtk. wxpython or other methods for the task.

I think the main issue in responding is your library preference, or widget
set preference. These can make you right with some in your response, or
wrong with others that have a preferable gui library that coincides with
one's personal cognitive structure that makes t




On Wed, Jul 24, 2013 at 9:48 AM, Chris Angelico  wrote:

> On Wed, Jul 24, 2013 at 11:40 PM,   wrote:
> > Short example. Writing an editor with something like the
> > FSR is simply impossible (properly).
>
> jmf, have you ever written an editor with *any* string representation?
> Are you speaking from any level of experience at all?
>
> ChrisA
> --
> http://mail.python.org/mailman/listinfo/python-list
>



-- 
Best Regards,
David Hutto
*CEO:* *http://www.hitwebdevelopment.com*
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[David Hutto]

I've screwed up plenty of times in python, but can write code like a pro
when I'm feeling better(on SSI and medicaid). An editor can be built
simply, but it's preference that makes the difference. Some might have used
tkinter, gtk. wxpython or other methods for the task.

I think the main issue in responding is your library preference, or widget
set preference. These can make you right with some in your response, or
wrong with others that have a preferable gui library that coincides with
one's personal cognitive structure that makes it more usable in relation to
how you learned a preferable gui kit.


On Wed, Jul 24, 2013 at 10:17 AM, David Hutto wrote:

> I've screwed up plenty of times in python, but can write code like a pro
> when I'm feeling better(on SSI and medicaid). An editor can be built
> simply, but it's preference that makes the difference. Some might have used
> tkinter, gtk. wxpython or other methods for the task.
>
> I think the main issue in responding is your library preference, or widget
> set preference. These can make you right with some in your response, or
> wrong with others that have a preferable gui library that coincides with
> one's personal cognitive structure that makes t
>
>
>
>
> On Wed, Jul 24, 2013 at 9:48 AM, Chris Angelico  wrote:
>
>> On Wed, Jul 24, 2013 at 11:40 PM,   wrote:
>> > Short example. Writing an editor with something like the
>> > FSR is simply impossible (properly).
>>
>> jmf, have you ever written an editor with *any* string representation?
>> Are you speaking from any level of experience at all?
>>
>> ChrisA
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>>
>
>
>
> --
> Best Regards,
> David Hutto
> *CEO:* *http://www.hitwebdevelopment.com*
>



-- 
Best Regards,
David Hutto
*CEO:* *http://www.hitwebdevelopment.com*
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[Chris Angelico]

On Thu, Jul 25, 2013 at 12:17 AM, David Hutto  wrote:
> I've screwed up plenty of times in python, but can write code like a pro
> when I'm feeling better(on SSI and medicaid). An editor can be built simply,
> but it's preference that makes the difference. Some might have used tkinter,
> gtk. wxpython or other methods for the task.
>
> I think the main issue in responding is your library preference, or widget
> set preference. These can make you right with some in your response, or
> wrong with others that have a preferable gui library that coincides with
> one's personal cognitive structure that makes t

jmf's point is more about writing the editor widget (Scintilla, as
opposed to SciTE), which most people will never bother to do. I've
written several text editors, always by embedding someone else's
widget, and therefore not concerning myself with its internal string
representation. Frankly, Python's strings are a *terrible* internal
representation for an editor widget - not because of PEP 393, but
simply because they are immutable, and every keypress would result in
a rebuilding of the string. On the flip side, I could quite plausibly
imagine using a list of strings; whenever text gets inserted, the
string gets split at that point, and a new string created for the
insert (which also means that an Undo operation simply removes one
entire string). In this usage, the FSR is beneficial, as it's possible
to have different strings at different widths.

But mainly, I'm just wondering how many people here have any basis
from which to argue the point he's trying to make. I doubt most of us
have (a) implemented an editor widget, or (b) tested multiple
different internal representations to learn the true pros and cons of
each. And even if any of us had, that still wouldn't have any bearing
on PEP 393, which is about applications, not editor widgets. As stated
above, Python strings before AND after PEP 393 are poor choices for an
editor, ergo arguing from that standpoint is pretty useless. Not that
that bothers jmf...

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Simple Python script as SMTP server for outgoing e-mails?

[Kevin Walzer]

On 7/23/13 5:53 PM, Gilles wrote:

On Mon, 22 Jul 2013 10:14:15 -0400, Kevin Walzer 
wrote:

http://www.hmailserver.com


Thanks. hMailServer was one of the apps I checked, and I was just
making sure there weren't something simpler, considering my needs,
ideally something like Mongoose MTA.

Regardless, because of the SPAM anti-measures mentioned above, it
seems like I was over-optimistic about running an MTA and sending
e-mails from my home computer :-/



The reason I mentioned hMailServer is that I host my own mail server for 
my business--I have a static IP address, and correctly configured 
DNS--and so I'm able to send out large batches of e-mails to customers 
from my network without being blocked by my ISP. I'm running a Mac 
server so my mail system is the typical Unix setup (Postfix, etc.), but 
hMailServer was the closest thing I've found for Windows.


Configuring your own server isn't cheap in terms of time even if you use 
FOSS components, and your ISP may charge more for a static IP, so I 
completely understand if you don't want to go that route.


--Kevin

--
Kevin Walzer
Code by Kevin/Mobile Code by Kevin
http://www.codebykevin.com
http://www.wtmobilesoftware.com
--
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[Michael Torrie]

On 07/24/2013 07:40 AM, wxjmfa...@gmail.com wrote:
> Sorry, you are not understanding Unicode. What is a Unicode
> Transformation Format (UTF), what is the goal of a UTF and
> why it is important for an implementation to work with a UTF.

Really?  Enlighten me.

Personally, I would never use UTF as a representation *in memory* for a
unicode string if it were up to me.  Why?  Because UTF characters are
not uniform in byte width so accessing positions within the string is
terribly slow and has to always be done by starting at the beginning of
the string.  That's at minimum O(n) compared to FSR's O(1).  Surely you
understand this.  Do you dispute this fact?

UTF is a great choice for interchange, though, and indeed that's what it
was designed for.

Are you calling for UTF to be adopted as the internal, in-memory
representation of unicode?  Or would you simply settle for UCS-4?
Please be clear here.  What are you saying?

> Short example. Writing an editor with something like the
> FSR is simply impossible (properly).

How? FSR is just an implementation detail.  It could be UCS-4 and it
would also work.


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: dict & dict.keys()

[Skip Montanaro]

> What do you mean? Why would you want to create a temporary list just to
> iterate over it explicitly or implicitly (set, sorted, max,...)?

Because while iterating over the keys, he might also want to add or
delete keys to/from the dict.  You can't do that while iterating over
them in-place.

This example demonstrates the issue and also shows that the
modification actually takes place:

>>> d = dict(zip(range(10), range(10, 0, -1)))
>>> d
{0: 10, 1: 9, 2: 8, 3: 7, 4: 6, 5: 5, 6: 4, 7: 3, 8: 2, 9: 1}
>>> for k in d:
...   if k == 3:
... del d[k+1]
...
Traceback (most recent call last):
  File "", line 1, in 
RuntimeError: dictionary changed size during iteration
>>> for k in list(d):
...   if k == 3:
... del d[k+1]
...
Traceback (most recent call last):
  File "", line 3, in 
KeyError: 4
>>> d.keys()
dict_keys([0, 1, 2, 3, 5, 6, 7, 8, 9])

Skip
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[Michael Torrie]

On 07/24/2013 08:34 AM, Chris Angelico wrote:
> Frankly, Python's strings are a *terrible* internal representation
> for an editor widget - not because of PEP 393, but simply because
> they are immutable, and every keypress would result in a rebuilding
> of the string. On the flip side, I could quite plausibly imagine
> using a list of strings; whenever text gets inserted, the string gets
> split at that point, and a new string created for the insert (which
> also means that an Undo operation simply removes one entire string).
> In this usage, the FSR is beneficial, as it's possible to have
> different strings at different widths.

Very good point.  Seems like this is exactly what is tripping up jmf in
general.  His pseudo benchmarks are bogus for this exact reason. No one
uses python strings in this fashion.  Editors certainly would not.  But
then again his argument in the past does not mention editors.  But it
makes me wonder if jmf is using python strings appropriately, or even
realizes they are immutable.

> But mainly, I'm just wondering how many people here have any basis 
> from which to argue the point he's trying to make. I doubt most of
> us have (a) implemented an editor widget, or (b) tested multiple 
> different internal representations to learn the true pros and cons
> of each. 

Maybe, but simply thinking logically, FSR and UCS-4 are equivalent in
pros and cons, and the cons of using UCS-2 (the old narrow builds) are
well known.  UCS-2 simply cannot represent all of unicode correctly.
This is in the PEP of course.

His most recent argument that Python should use UTF as a representation
is very strange to be honest.  The cons of UTF are apparent and widely
known.  The main con is that UTF strings are O(n) for indexing a
position within the string.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: dict & dict.keys()

[Ian Kelly]

On Tue, Jul 23, 2013 at 8:11 PM, Steven D'Aprano
 wrote:
> Basically, views are set-like, not list-like.

The keys and items views are set-like.  The values view is not.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[Chris Angelico]

On Thu, Jul 25, 2013 at 12:47 AM, Michael Torrie  wrote:
> On 07/24/2013 07:40 AM, wxjmfa...@gmail.com wrote:
>> Sorry, you are not understanding Unicode. What is a Unicode
>> Transformation Format (UTF), what is the goal of a UTF and
>> why it is important for an implementation to work with a UTF.
>
> Really?  Enlighten me.
>
> Personally, I would never use UTF as a representation *in memory* for a
> unicode string if it were up to me.  Why?  Because UTF characters are
> not uniform in byte width so accessing positions within the string is
> terribly slow and has to always be done by starting at the beginning of
> the string.  That's at minimum O(n) compared to FSR's O(1).  Surely you
> understand this.  Do you dispute this fact?

Take care here; UTF is a general term for Unicode Translation Formats,
of which one (UTF-32) is fixed-width. Every other UTF-n is variable
width, though, so your point still stands. UTF-32 is the basis for
Python's FSR.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: dict & dict.keys()

[Ian Kelly]

On Wed, Jul 24, 2013 at 8:58 AM, Skip Montanaro  wrote:
>> What do you mean? Why would you want to create a temporary list just to
>> iterate over it explicitly or implicitly (set, sorted, max,...)?
>
> Because while iterating over the keys, he might also want to add or
> delete keys to/from the dict.  You can't do that while iterating over
> them in-place.

None of the (set, sorted, max, ...) cases will add or delete keys
while iterating.  For the occasional for loop where the programmer
does want to do that, you can still explicitly create a temporary list
with list().
-- 
http://mail.python.org/mailman/listinfo/python-list

[ANN] pyspread 0.2.4

[Martin Manns]

==
pyspread 0.2.4
==


Pyspread 0.2.4 is released.

Besides Linux, the new version also runs on Windows (Windows 7 64bit
and Windows XP 32bit tested).


About pyspread
==

Pyspread is a non-traditional spreadsheet application that is based on
and written in the programming language Python. 

The goal of pyspread is to be the most pythonic spreadsheet application.

Pyspread is free software. It is released under the GPL v3.

Project website: http://manns.github.com/pyspread/


What is new in 0.2.4


 + Windows compatibility
 + More charts and chart options (box plots, histograms, pie charts)
 + Export of matplotlib charts to svg, png, pdf, eps and ps
 + Import from UTF-8 encoded csv files (no other encoding, yet)
 + More than 64k characters supported in one cell
 + Insertion and deletion of rows and columns now affect only the
   current table
 + High quality in-cell bitmaps images (with transparency) supported
 + Iterable data can be pasted into multiple cells via Paste As...
 + Pyspread can now be started from other Python applications
 + Partly localized in Nowegian Nynorsk and Bokmaal

Known issues


 + Selection mode is disabled in Windows.
 + "Find & Replace All" is slow in grids with much data unless pyspread
   is minimized.
 + Paste As... does not work if dates are present in the data structure.
 + Sometimes, pressing redo when there is nothing left to redo has
   undesired effects such as redoing an operation again.
 + When updating from a non-release version (from git), the file 
   ~/.pyspreadrc (in Windows pyspread's registry entry) may have to be
   deleted.

Enjoy

Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Converting a list of lists to a single list

[Terry Reedy]

On 7/23/2013 7:02 PM, Terry Reedy wrote:

On 7/23/2013 5:52 PM, st...@divillo.com wrote:

I think that itertools may be able to do what I want but I have not
been able to figure out how.


What you want is a flattened product with unchanged components of the 
successive products omitted in the flattening. The omission is the 
difficulty.



A recursive generator suffices.


But see below for how to use itertools.product.


I want to convert an arbitrary number of lists with an arbitrary
number of elements in each list into a single list as follows.


While others answered the Python2-oriented question ("How do I produce a 
list from a list of lists"), I answered the Python-3 oriented question 
of how to produce an iterator from an iterable of iterables. This scales 
better to an input with lots of long sequences. There is usually no need 
to manifest the output as a list, as the typical use of the list will be 
to iterate it.



def crossflat(lofl):
 if lofl:
 first = lofl.pop(0)
 for o in first:
yield o
yield from crossflat(lofl.copy())

A0, A1, A2 = 100, 101, 102
B0, B1, B2 = 10, 11, 12
C0, C1, C2 = 0, 1, 2
LL = [[A0, A1, A2], [B0, B1, B2], [C0, C1, C2]]
cfLL = list(crossflat(LL))
print(cfLL)
assert cfLL == [
A0, B0, C0, C1, C2, B1, C0, C1, C2, B2, C0, C1, C2,
A1, B0, C0, C1, C2, B1, C0, C1, C2, B2, C0, C1, C2,
A2, B0, C0, C1, C2, B1, C0, C1, C2, B2, C0, C1, C2]

passes


Here is filtered flattened product version. I think it clumsier than 
directly producing the items wanted, but it is good to know of this 
approach as a backup.


from itertools import product

def flatprod(iofi):  # iterable of iterables
lofi = list(iofi)
now = [object()] * len(lofi)
for new in product(*lofi):
i = 0
while now[i] == new[i]:
i += 1
yield from new[i:]
now = new

cfLL = list(flatprod(LL))

Same assert as before passes.

--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behaviour with os.linesep

[Terry Reedy]

On 7/23/2013 7:41 PM, Dennis Lee Bieber wrote:

On 23 Jul 2013 15:25:12 GMT, Steven D'Aprano
 declaimed the following:


On Tue, 23 Jul 2013 13:42:13 +0200, Vincent Vande Vyvre wrote:


On Windows a script where de endline are the system line sep, the files
are open with a double line in Eric4, Notepad++ or Gedit but they are
correctly displayed in the MS Bloc-Notes.


I suspect the problem lies with Eric4, Notepad++ and Gedit. Do you
perhaps have to manually tell them that the file uses Windows line
separators?


I suspect the problem likes in the file written. Notepad++ works fine 
with \r\n or \n on input and can produce either on output.



Don't know about those, but SciTE I know has both an menu option for
line ending (, , ), and one for "convert line endings"




--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: how: embed + extend to control my running app?

[David M. Cotter]

well, umm, gosh, now i feel quite silly.  that was easy.  okay that's done.

next: i'd like to redirect the output of any "print" statements to my C 
function:

>   voidLog(const unsigned char *utf8_cstrP);

on the mac, python output sys.stdout goes into the debug console if you're in 
the debugger, and to the "console app" if not.  On windows, i don't think it 
goes anywhere at all?

So: i really want it to go to my own log file (via my Log() function).  now, 
can i specify "please output to this FILE*" ?, i looked at all the python c 
headers but found nothing about redirecting the output.

I see "PySys_GetFile()" which will get what it points to, but what i want is a 
"PySys_SetFile()" so i can set it.

the only alternative seems to be:
>   PyObject*logObjectP = create ???;
>   
>   ERR(PySys_SetObject("stdout", logObjectP));

if that's the only way, how to create the logObjectP such that it redirects the 
write() python function to my Log() C function?

i tried this:

const char *s_printFunc = 
"import sys\n"
"class CustomPrint():\n"
"   def __init__(self):\n"
"   self.old_stdout=sys.stdout\n"
"\n"
"   def write(self, text):\n"
"   self.old_stdout.write('foobar')\n"
"   text = text.rstrip()\n"
"   if len(text) == 0:\n"
"   return\n"
"   self.old_stdout.write('custom Print--->' + text + 
'\n')\n";

OSStatusCPython_PreAlloc(const char *utf8Z)
{
OSStatuserr = noErr;
PyCompilerFlags flags;
PyObject*logObjectP = NULL;

Py_SetProgramName(const_cast(utf8Z));
Py_Initialize();

flags.cf_flags  = PyCF_SOURCE_IS_UTF8;
logObjectP  = Py_CompileStringFlags(s_printFunc, 
"CustomPrint", Py_single_input, &flags);

ERR_NULL(logObjectP, tsmUnsupScriptLanguageErr);

if (!err) {
ERR(PySys_SetObject("stdout", logObjectP));
ERR(PySys_SetObject("stderr", logObjectP));
Py_DECREF(logObjectP);
}

return err;
}

voidCPython_PostDispose()
{
Py_Finalize();
}

voidCPython_Test()
{
PyRun_SimpleString(
"from time import time, ctime\n"
"print 'Today is', ctime(time())\n");
}

-
and when i run CPython_Test(), there is no output at all.  If i comment out the 
entire Py_CompileStringFlags() line, then the output works fine (going to 
stdout as expected), so i'm not sure what i'm doing wrong
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: dict & dict.keys()

[Ethan Furman]

On 07/24/2013 05:51 AM, Oscar Benjamin wrote:


On Jul 24, 2013 7:25 AM, "Peter Otten" <__pete...@web.de 
> wrote:


Ethan Furman wrote:

> So, my question boils down to:  in Python 3 how is dict.keys() different
> from dict?  What are the use cases?

I just grepped through /usr/lib/python3, and could not identify a single
line where some_object.keys() wasn't either wrapped in a list (or set,
sorted, max) call, or iterated over.

To me it looks like views are a solution waiting for a problem.


What do you mean? Why would you want to create a temporary list just to iterate 
over it explicitly or implicitly (set,
sorted, max,...)?


You wouldn't.  But you don't need .keys() for that either as you can just use 
the dict itself.

My point is that in 2.x .keys() did something different from the dict, while in 
3.x it appears to me that they are the same.

Peter's point is that in the stdlib the new functionality of .keys() is never 
used, not even once.

--
~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list

Re: how: embed + extend to control my running app?

[Chris Angelico]

On Thu, Jul 25, 2013 at 2:10 AM, David M. Cotter  wrote:
> So: i really want it to go to my own log file (via my Log() function).  now, 
> can i specify "please output to this FILE*" ?, i looked at all the python c 
> headers but found nothing about redirecting the output.


Are you able to simply redirect the OS-level stdout handle, or would
that disrupt your own code? That might be an easier way to log to a
file.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: dict & dict.keys()

[Oscar Benjamin]

On Jul 24, 2013 2:27 PM, "Peter Otten" <__pete...@web.de> wrote:
>
> Oscar Benjamin wrote:
>
> > On Jul 24, 2013 7:25 AM, "Peter Otten" <__pete...@web.de> wrote:
> >>
> >> Ethan Furman wrote:
> >>
> >> > So, my question boils down to:  in Python 3 how is dict.keys()
> >> > different
> >> > from dict?  What are the use cases?
> >>
> >> I just grepped through /usr/lib/python3, and could not identify a
single
> >> line where some_object.keys() wasn't either wrapped in a list (or set,
> >> sorted, max) call, or iterated over.
> >>
> >> To me it looks like views are a solution waiting for a problem.
> >
> > What do you mean? Why would you want to create a temporary list just to
> > iterate over it explicitly or implicitly (set, sorted, max,...)?
>
> I mean I don't understand the necessity of views when all actual usecases
> need iterators. The 2.x iterkeys()/iteritems()/itervalues() methods didn't
> create lists either.

Oh, okay. I see what you mean.

>
> Do you have 2.x code lying around where you get a significant advantage by
> picking some_dict.viewkeys() over some_dict.iterkeys()?

No. I don't think I've ever used viewkeys. I noticed it once, didn't see an
immediate use and forgot about it but...

> I could construct
> one
>
> >>> d = dict(a=1, b=2, c=3)
> >>> e = dict(b=4, c=5, d=6)
> >>> d.viewkeys() & e.viewkeys()
> set(['c', 'b'])

that might be useful.

>
> but have not seen it in the wild.

> My guess is that most non-hardcore users don't even know about viewkeys().
> By the way, my favourite idiom to iterate over the keys in both Python 2
and
> 3 is -- for example -- max(some_dict) rather than
> max(some_dict.whateverkeys()).

Agreed.

Earlier I saw that I had list(some_dict) in some code. Not sure why but
maybe because it's the same in Python 2 and 3.

Oscar
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: dict & dict.keys()

[Chris Angelico]

On Thu, Jul 25, 2013 at 1:57 AM, Ethan Furman  wrote:
> On 07/24/2013 05:51 AM, Oscar Benjamin wrote:
>> What do you mean? Why would you want to create a temporary list just to
>> iterate over it explicitly or implicitly (set,
>> sorted, max,...)?
>
> You wouldn't.  But you don't need .keys() for that either as you can just
> use the dict itself.

Side point: Why is iterating over a dict equivalent to .keys() rather
than .items()? It feels odd that, with both options viable, the
implicit version iterates over half the dict instead of all of it.
Obviously it can't be changed now, even if .items() were the better
choice, but I'm curious as to the reason for the decision.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: dict & dict.keys()

[Terry Reedy]

On 7/24/2013 12:34 PM, Chris Angelico wrote:


Side point: Why is iterating over a dict equivalent to .keys() rather
than .items()? It feels odd that, with both options viable, the
implicit version iterates over half the dict instead of all of it.
Obviously it can't be changed now, even if .items() were the better
choice, but I'm curious as to the reason for the decision.


Both were considered and I think there were and are two somewhat-linked 
practical reasons. First, iterating over keys in more common than 
iterating over items. The more common one should be the default.


Second, people ask much more often if 'key' is in dict than if 'key, 
value' is in dict. This is true as well for keyed reference books such 
as phone books, dictionaries, encyclopedias, and for the same reason. 
This is  coupled with the fact that the default meaning of 'item in 
collection' is that iterating over 'collection' eventually produces 
'item' or a value equal to 'item'.


--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: dict & dict.keys()

[Stefan Behnel]

Chris Angelico, 24.07.2013 18:34:
> On Thu, Jul 25, 2013 at 1:57 AM, Ethan Furman wrote:
>> On 07/24/2013 05:51 AM, Oscar Benjamin wrote:
>>> What do you mean? Why would you want to create a temporary list just to
>>> iterate over it explicitly or implicitly (set,
>>> sorted, max,...)?
>>
>> You wouldn't.  But you don't need .keys() for that either as you can just
>> use the dict itself.
> 
> Side point: Why is iterating over a dict equivalent to .keys() rather
> than .items()? It feels odd that, with both options viable, the
> implicit version iterates over half the dict instead of all of it.
> Obviously it can't be changed now, even if .items() were the better
> choice, but I'm curious as to the reason for the decision.

The reason is that you can easily get at the values when iterating over the
keys, or simply decide not to care about them and be happy with the keys
only. Note that there are also many use cases that need all keys but not
all values. If iteration always returned an item tuple by default, many use
cases would have to resort to using .keys() in order to be efficient. And
for the simple case, you'd have to type more, either the additional .keys()
or the useless tuple unpacking. So, the reasoning is that iteration should
do the basic thing that still allows you to do everything, instead of doing
everything and pushing unnecessary work on the users by default.

Stefan


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: dict & dict.keys()

[Stefan Behnel]

Peter Otten, 24.07.2013 08:23:
> Ethan Furman wrote:
>> So, my question boils down to:  in Python 3 how is dict.keys() different
>> from dict?  What are the use cases?
> 
> I just grepped through /usr/lib/python3, and could not identify a single 
> line where some_object.keys() wasn't either wrapped in a list (or set, 
> sorted, max) call, or iterated over.

In case that directory mainly consists of the standard library, then you
shouldn't forget that most of the code in there predates Python 3 by ages
and was only adapted to work with the new syntax/semantics, not rewritten
in a "better" way.

Even if it's not just the stdlib, that argument still holds. There is still
fairly little code out there that was specifically written for Py2.6+, as
opposed to just being adapted.


> To me it looks like views are a solution waiting for a problem.

They reduce the API overhead. Previously, you needed values() and
itervalues(), with values() being not more than a special case of what
itervalues() provides anyway. Now it's just one method that gives you
everything. It simply has corrected the tradeoff from two special purpose
APIs to one general purpose API, that's all.

Stefan


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: how: embed + extend to control my running app?

[David M. Cotter]

i don't use stdout in my own code, my code goes to my own log file.  i want the 
output from any python code to go to my existing log file, so log statements 
from my app and any python code are intermingled in that one file.

my updated code is here, which now bridges my python print function to my C 
function:
http://karaoke.kjams.com/wiki/Python

but it seems that my custom "s_printFunc" is never called ?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[Terry Reedy]

On 7/24/2013 11:00 AM, Michael Torrie wrote:

On 07/24/2013 08:34 AM, Chris Angelico wrote:

Frankly, Python's strings are a *terrible* internal representation
for an editor widget - not because of PEP 393, but simply because
they are immutable, and every keypress would result in a rebuilding
of the string. On the flip side, I could quite plausibly imagine
using a list of strings;


I used exactly this, a list of strings, for a Python-coded text-only 
mock editor to replace the tk Text widget in idle tests. It works fine 
for the purpose. For small test texts, the inefficiency of immutable 
strings is not relevant.


Tk apparently uses a C-coded btree rather than a Python list. All 
details are hidden, unless one finds and reads the source ;-), but but 
it uses C arrays rather than Python strings.



In this usage, the FSR is beneficial, as it's possible to have
different strings at different widths.


For my purpose, the mock Text works the same in 2.7 and 3.3+.


Maybe, but simply thinking logically, FSR and UCS-4 are equivalent in
pros and cons,


They both have the pro that indexing is direct *and correct*. The cons 
are different.



and the cons of using UCS-2 (the old narrow builds) are
well known.  UCS-2 simply cannot represent all of unicode correctly.


Python's narrow builds, at least for several releases, were in between 
USC-2 and UTF-16 in that they used surrogates to represent all unicodes 
but did not correct indexing for the presence of astral chars. This is a 
nuisance for those who do use astral chars, such as emotes and CJK name 
chars, on an everyday basis.


--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: dict & dict.keys()

[Chris Angelico]

On Thu, Jul 25, 2013 at 3:17 AM, Terry Reedy  wrote:
> On 7/24/2013 12:34 PM, Chris Angelico wrote:
>
>> Side point: Why is iterating over a dict equivalent to .keys() rather
>> than .items()? It feels odd that, with both options viable, the
>> implicit version iterates over half the dict instead of all of it.
>> Obviously it can't be changed now, even if .items() were the better
>> choice, but I'm curious as to the reason for the decision.
>
> This is
> coupled with the fact that the default meaning of 'item in collection' is
> that iterating over 'collection' eventually produces 'item' or a value equal
> to 'item'.

Ahh, that makes sense. I never thought of iteration and 'in' being
connected like that, but yes, that's a solid reason for doing it that
way.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dihedral

[Stefan Behnel]

Fábio Santos, 16.07.2013 00:54:
>> On 07/15/2013 08:36 AM, Steven D'Aprano wrote:
>>>
>>> Devyn,
>>>
>>> 8 Dihedral is our resident bot, not a human being. Nobody knows who
>>> controls it, and why they are running it, but we are pretty certain that
>>> it is a bot responding mechanically to keywords in people's posts.
>>>
>>> It's a very clever bot, but still a bot. About one post in four is
>>> meaningless jargon, the other three are relevant enough to fool people
>>> into thinking that maybe it is a human being. It had me fooled for a long
>>> time.
> 
> Does this mean he passes the Turing test?

I'd say that "it" appears more correct. Or is there any indication of a
specific bot gender? (I sure might have missed it...)

Note that being of a specific gender is definitely not required to pass the
Turing test. I'm not even sure pretending to have one is required.

Stefan


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Converting a list of lists to a single list

[steve]

Wow, thanks everyone. Very helpful indeed!

On Tuesday, July 23, 2013 2:52:21 PM UTC-7, st...@divillo.com wrote:
> I think that itertools may be able to do what I want but I have not been able 
> to figure out how.
> 
> 
> 
> I want to convert an arbitrary number of lists with an arbitrary number of 
> elements in each list into a single list as follows.
> 
> 
> 
> Say I have three lists:
> 
> 
> 
> [[A0,A1,A2], [B0,B1,B2] [C0,C1,C2]]
> 
> 
> 
> I would like to convert those to a single list that looks like this:
> 
> 
> 
> [A0,B0,C0,C1,C2,B1,C0,C1,C2,B2,C0,C1,C2,A1,B0,C0,C1,C2,B1,C0,C1,C2,B2,C0,C1,C2,A2,B0,C0,C1,C2,B1,C0,C1,C2,B2,C0,C1,C2]
> 
> 
> 
> An easier way to visualize the pattern I want is as a tree.
> 
> 
> 
> A0
> 
>   B0
> 
>   C0
> 
>   C1
> 
>   C2
> 
>   B1
> 
>   C0
> 
>   C1
> 
>   C2
> 
>   B2
> 
>   C0
> 
>   C1
> 
>   C2
> 
> A1
> 
>   B0
> 
>   C0
> 
>   C1
> 
>   C2
> 
>   B1
> 
>   C0
> 
>   C1
> 
>   C2
> 
>   B2
> 
>   C0
> 
>   C1
> 
>   C2
> 
> A2
> 
>   B0
> 
>   C0
> 
>   C1
> 
>   C2
> 
>   B1
> 
>   C0
> 
>   C1
> 
>   C2
> 
>   B2
> 
>   C0
> 
>   C1
> 
>   C2

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[Chris Angelico]

On Thu, Jul 25, 2013 at 3:52 AM, Terry Reedy  wrote:
> On 7/24/2013 11:00 AM, Michael Torrie wrote:
>>
>> On 07/24/2013 08:34 AM, Chris Angelico wrote:
>>>
>>> Frankly, Python's strings are a *terrible* internal representation
>>> for an editor widget - not because of PEP 393, but simply because
>>> they are immutable, and every keypress would result in a rebuilding
>>> of the string. On the flip side, I could quite plausibly imagine
>>> using a list of strings;
>
>
> I used exactly this, a list of strings, for a Python-coded text-only mock
> editor to replace the tk Text widget in idle tests. It works fine for the
> purpose. For small test texts, the inefficiency of immutable strings is not
> relevant.
>
> Tk apparently uses a C-coded btree rather than a Python list. All details
> are hidden, unless one finds and reads the source ;-), but but it uses C
> arrays rather than Python strings.
>
>
>>> In this usage, the FSR is beneficial, as it's possible to have
>>> different strings at different widths.
>
>
> For my purpose, the mock Text works the same in 2.7 and 3.3+.

Thanks for that report! And yes, it's going to behave exactly the same
way, because its underlying structure is an ordered list of ordered
lists of Unicode codepoints, ergo 3.3/PEP 393 is merely a question of
performance. But if you put your code onto a narrow build, you'll have
issues as seen below.

>> Maybe, but simply thinking logically, FSR and UCS-4 are equivalent in
>> pros and cons,
>
> They both have the pro that indexing is direct *and correct*. The cons are
> different.

They're close enough, though. It's simply a performance tradeoff - use
the memory all the time, or take a bit of overhead to give yourself
the option of using less memory. The difference is negligible compared
to...

>> and the cons of using UCS-2 (the old narrow builds) are
>> well known.  UCS-2 simply cannot represent all of unicode correctly.
>
> Python's narrow builds, at least for several releases, were in between USC-2
> and UTF-16 in that they used surrogates to represent all unicodes but did
> not correct indexing for the presence of astral chars. This is a nuisance
> for those who do use astral chars, such as emotes and CJK name chars, on an
> everyday basis.

... this. If nobody had ever thought of doing a multi-format string
representation, I could well imagine the Python core devs debating
whether the cost of UTF-32 strings is worth the correctness and
consistency improvements... and most likely concluding that narrow
builds get abolished. And if any other language (eg ECMAScript)
decides to move from UTF-16 to UTF-32, I would wholeheartedly support
the move, even if it broke code to do so. To my mind, exposing UTF-16
surrogates to the application is a bug to be fixed, not a feature to
be maintained. But since we can get the best of both worlds with only
a small amount of overhead, I really don't see why anyone should be
objecting.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dihedral

[Chris Angelico]

On Thu, Jul 25, 2013 at 4:08 AM, Stefan Behnel  wrote:
> Fábio Santos, 16.07.2013 00:54:
>> Does this mean he passes the Turing test?
>
> I'd say that "it" appears more correct. Or is there any indication of a
> specific bot gender? (I sure might have missed it...)
>
> Note that being of a specific gender is definitely not required to pass the
> Turing test. I'm not even sure pretending to have one is required.

Gender:  [ ] Male   [ ] Female   [ ] Both   [ ] Neither   [ ] Robot   [ ] Other

In absence of specific information to the contrary, I'll usually refer
to computers and programs by masculine pronouns; no particular reason
for it, just the same sort of convention that has most ships and boats
being "she".

And you're quite right, gender and the pretense thereto are completely
optional in a Turing test, even though the Turing test was derived
from a similar concept involving gender.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: dict & dict.keys()

[Ethan Furman]

On 07/24/2013 10:23 AM, Stefan Behnel wrote:

Peter Otten, 24.07.2013 08:23:

Ethan Furman wrote:


So, my question boils down to:  in Python 3 how is dict.keys() different
from dict?  What are the use cases?


To me it looks like views are a solution waiting for a problem.


They reduce the API overhead. Previously, you needed values() and
itervalues(), with values() being not more than a special case of what
itervalues() provides anyway. Now it's just one method that gives you
everything. It simply has corrected the tradeoff from two special purpose
APIs to one general purpose API, that's all.


I started this thread for two reasons:

  1) Increase awareness that using `list(dict)` is a cross-version replacement 
for `dict.keys()`

  2) Hopefully learn something about when a view is useful.

So far #2 is pretty much a failure.  Only one use-case so far (and it feels pretty rare).  But hey, I have learned that 
while some set operations are allowed (&, ^, |, .isdisjoint()), others are not (.remove(), .discard(), .union(), etc.).


The old .keys(), .values(), and .items() (and their .iter...() variations) did something commonly useful.  Of what 
common use are these views?


--
~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list

Re: how: embed + extend to control my running app?

[David M. Cotter]

> http://karaoke.kjams.com/wiki/Python
nevermind, i got it, it's working now (see link for code)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: dict & dict.keys()

[Stefan Behnel]

Ethan Furman, 24.07.2013 20:31:
> On 07/24/2013 10:23 AM, Stefan Behnel wrote:
>> Peter Otten, 24.07.2013 08:23:
>>> Ethan Furman wrote:

 So, my question boils down to:  in Python 3 how is dict.keys() different
 from dict?  What are the use cases?
>>>
>>> To me it looks like views are a solution waiting for a problem.
>>
>> They reduce the API overhead. Previously, you needed values() and
>> itervalues(), with values() being not more than a special case of what
>> itervalues() provides anyway. Now it's just one method that gives you
>> everything. It simply has corrected the tradeoff from two special purpose
>> APIs to one general purpose API, that's all.
> 
> I started this thread for two reasons:
> 
>   1) Increase awareness that using `list(dict)` is a cross-version
> replacement for `dict.keys()`
> 
>   2) Hopefully learn something about when a view is useful.
> 
> So far #2 is pretty much a failure.

I think the question is: how else would you implement an interface that
doesn't restrict itself to returning a list? I mean, previously, the
following was totally inefficient in terms of memory:

value in d.values()

It now avoids creating an intermediate list copy of the values, thus
running with no additional memory overhead (well, a constant, ok, but
definitely not linear) and keeps users from resorting to the much more
unfriendly

for v in d.itervalues():
if v == value:
return True
else:
return False

in order to achieve the same thing. You can now even efficiently do this
for items, i.e.

(key, value) in d.items()

That's equivalent to "d[key] == value", but uses a different protocol,
meaning that you don't have to make a copy of the dict items in order to
pass it into something that works on a set or iterable of 2-tuples (which
is a way more generic interface than requiring a dict as input). These
things chain much more cleanly now, without first having to explain the
difference between items() and iteritems() and when to use which.

It's all about replacing the old copy-to-list interface by something that
is efficiently processable step by step. All of this started back when
iterators became a part of the language, then generators, and now dict
views. They may not be the hugest feature ever, but they definitely fit
into the language much better and much more cleanly than the old
copy-to-list way.

Ask yourself, if they had been there in Python 1.x, would you even have
thought about making the iter*() methods a part of the language? Would you
really have wanted a shorter way to create a list of dict values than
list(d.values())?

Stefan


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: how: embed + extend to control my running app?

[David M. Cotter]

> > Now for Windows: same thing, i think i must create a .dll, right?
> you should already have a python.dll in your installation

i can find "python27.lib" in the "libs" folder, but there is no 
"python27_d.lib", and there is no "python27.dll" in the DLLs folder?

are there instructions for creating (or finding) these for Windows?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: dict & dict.keys()

[Ethan Furman]

On 07/24/2013 12:59 PM, Stefan Behnel wrote:


I think the question is: how else would you implement an interface that
doesn't restrict itself to returning a list? I mean, previously, the
following was totally inefficient in terms of memory:

 value in d.values()

It now avoids creating an intermediate list copy of the values, thus
running with no additional memory overhead (well, a constant, ok, but
definitely not linear) and keeps users from resorting to the much more
unfriendly

 for v in d.itervalues():
 if v == value:
 return True
 else:
 return False

in order to achieve the same thing. You can now even efficiently do this
for items, i.e.

 (key, value) in d.items()

That's equivalent to "d[key] == value", but uses a different protocol,
meaning that you don't have to make a copy of the dict items in order to
pass it into something that works on a set or iterable of 2-tuples (which
is a way more generic interface than requiring a dict as input). These
things chain much more cleanly now, without first having to explain the
difference between items() and iteritems() and when to use which.

It's all about replacing the old copy-to-list interface by something that
is efficiently processable step by step. All of this started back when
iterators became a part of the language, then generators, and now dict
views. They may not be the hugest feature ever, but they definitely fit
into the language much better and much more cleanly than the old
copy-to-list way.


Thank you. :)

--
~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list

RE: Python 3: dict & dict.keys()

[Prasad, Ramit]

Stefan Behnel wrote:
> Ethan Furman, 24.07.2013 20:31:
> > On 07/24/2013 10:23 AM, Stefan Behnel wrote:
> >> Peter Otten, 24.07.2013 08:23:
> >>> Ethan Furman wrote:
> 
>  So, my question boils down to:  in Python 3 how is dict.keys() different
>  from dict?  What are the use cases?
> >>>
> >>> To me it looks like views are a solution waiting for a problem.
> >>
> >> They reduce the API overhead. Previously, you needed values() and
> >> itervalues(), with values() being not more than a special case of what
> >> itervalues() provides anyway. Now it's just one method that gives you
> >> everything. It simply has corrected the tradeoff from two special purpose
> >> APIs to one general purpose API, that's all.
> >
> > I started this thread for two reasons:
> >
> >   1) Increase awareness that using `list(dict)` is a cross-version
> > replacement for `dict.keys()`
> >
> >   2) Hopefully learn something about when a view is useful.
> >
> > So far #2 is pretty much a failure.
> 
> I think the question is: how else would you implement an interface that
> doesn't restrict itself to returning a list? I mean, previously, the
> following was totally inefficient in terms of memory:
> 
> value in d.values()
> 
> It now avoids creating an intermediate list copy of the values, thus
> running with no additional memory overhead (well, a constant, ok, but
> definitely not linear) and keeps users from resorting to the much more
> unfriendly
> 
> for v in d.itervalues():
> if v == value:
> return True
> else:
> return False
> 
> in order to achieve the same thing. You can now even efficiently do this
> for items, i.e.
> 
> (key, value) in d.items()
> 
> That's equivalent to "d[key] == value", but uses a different protocol,
> meaning that you don't have to make a copy of the dict items in order to
> pass it into something that works on a set or iterable of 2-tuples (which
> is a way more generic interface than requiring a dict as input). These
> things chain much more cleanly now, without first having to explain the
> difference between items() and iteritems() and when to use which.
> 
> It's all about replacing the old copy-to-list interface by something that
> is efficiently processable step by step. All of this started back when
> iterators became a part of the language, then generators, and now dict
> views. They may not be the hugest feature ever, but they definitely fit
> into the language much better and much more cleanly than the old
> copy-to-list way.
> 
> Ask yourself, if they had been there in Python 1.x, would you even have
> thought about making the iter*() methods a part of the language? Would you
> really have wanted a shorter way to create a list of dict values than
> list(d.values())?
> 
> Stefan
> 

I am still not clear on the advantage of views vs. iterators. What
makes d.viewkeys() better than d.iterkeys()? Why did they decide
not to rename d.iterkeys() to d.keys() and instead use d.viewkeys()?
Is the iteration over a set operation on keys really that common a 
use case?


Ramit




This email is confidential and subject to important disclaimers and conditions 
including on offers for the purchase or sale of securities, accuracy and 
completeness of information, viruses, confidentiality, legal privilege, and 
legal entity disclaimers, available at 
http://www.jpmorgan.com/pages/disclosures/email.  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: dict & dict.keys()

[Christian Heimes]

Am 24.07.2013 18:34, schrieb Chris Angelico:
> Side point: Why is iterating over a dict equivalent to .keys() rather
> than .items()? It feels odd that, with both options viable, the
> implicit version iterates over half the dict instead of all of it.
> Obviously it can't be changed now, even if .items() were the better
> choice, but I'm curious as to the reason for the decision.

Consider this:

if key in dict:
...

for key in dict:
   ...

It would be rather surprising if "in" as containment checks operates on
keys and "in" as iterator returns (key, value) tuples.

Christian


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Beginner. 2d rotation gives unexpected results.

[Joshua Landau]

On 23 July 2013 13:34,  wrote:

> Hello!
> This is my first post, nice to meet you all!
> I`m biology student from Russia, trying to learn python to perform some
>
> simple simulations.
>
> Here`s my first problem.
> I`m trying to perform some simple 2d vector rotations in pygame, in order
>
> to learn the basics of linear algebra and 2d transformations. So far i
>
> understand matrix multiplication pretty well, and probably all my math is
>
> right. Eventually i`m planning to write Poly class, and use it to rotate
>
> and translate some simple shapes. But when i try and write it in the
>
> program, i get very weird results, like all points of rectangle with
>
> coordinates [0,0],[0,100],[100,0],[100,100] start to go spiral and
>
> eventually shrink to the center. Although even Excel calculations with
>
> this formulas give me right result.
> I use Python 3.3 on Windows Xp.
> What is wrong with my code?
>
> [code]import pygame
> import math as m
>

GAH!

Why on earth would you do such a thing?
Just "import math", there's no need to obfuscate your code.


> black = ( 0, 0, 0)
> white = ( 255, 255, 255)
> green = ( 0, 255, 0)
> red = ( 255, 0, 0)
>

It's probably better to do:

black = pygame.Color("Black")
white = pygame.Color("white")
green = pygame.Color("green")
red   = pygame.Color("red")


> class Poly():
> pos = [100,100] #x and y coordinates of a point

rot = m.radians(1) #rotation in degrees
>

*Do not do this*

This is because classes have shared values -- these "pos" and "rot" values
are shared within the class.

>>> class P:
... n = []
... def more_n(self):
... self.n.append(len(self.n))
...
...
...
>>> one_P = P()
>>> two_P = P()
>>>
>>> one_P.more_n()
>>> one_P.more_n()
>>> one_P.more_n()
>>>
>>> two_P.n
[0, 1, 2]


Normally you want to set these at initialisation:

class Poly():
def __init__(self, pos=None, rot=math.radians(1)):
self.pos = [100, 100] if pos is None else pos
self.rot = rot
super().__init__(self)


> def draw(self): #draw point
> pygame.draw.circle(screen,white,self.pos,10,0)
>

Add some spaces, dude.

I was going to say:

> Also, use keyword arguments instead of throwing around "10" and "0" with
no context:
> def draw(self):
> pygame.draw.circle(screen, white, self.pos, radius=10, width=0)
> Pygame-ists will know that "width" means border_width, by now. Pygame
isn't known for it's clean design ;).

But pygame, being brilliant (not) decided that it won't let you.

def rotate(self): # rotation method
> sin = m.sin(self.rot) #calculationg sin and cos
> cos = m.cos(self.rot)
> x_rot = int(self.pos[0]*cos-self.pos[1]*sin) #mulpitplicating

vector to rotation matrix
> y_rot = int(self.pos[0]*sin+self.pos[1]*cos)
>
> self.pos[0] = x_rot #set new coordinates to a point
> self.pos[1] = y_rot
>


A lot of your comments are ridiculous. This one is particularly so:
#mulpitplicating vector to rotation matrix. Don't add comments that talk
about lines. Here is a quick guide for when to use comments:

1) API usage, when docstrings aren't usable

2) When something funny or unexpected occurs in the code, such as:

# Goes down to 0 (does not include end-point)
for i in range(foo, -1, -1): ...

3) To explain large-scale methodologies and algorithms

Other than this, are you trying to obfuscate this line? HINT: ADD SPACES ;).


A big problem here (this solves your problem) is your int(...) conversions.
Do *not* store important information less accurately than it deserves. This
is one very important instance. Only convert to int when you need to.

Another thing: "sin = m.sin(self.rot)"??? Really? Write "sin_1deg = ..."
instead, at least.


> a = Poly() #Some simple sample points giving rectangle
> b = Poly()
> c = Poly()
> d = Poly()
>
> b.pos = [0,100]
> c.pos = [100,0]
> d.pos = [0,0]
>

Use:

a = Poly()
b = Poly([0, 100])
c = Poly([100, 0])
d = Poly([0, 0])

pygame.init()
> size = [700,500]
> screen = pygame.display.set_mode(size)
> done = False
> clock = pygame.time.Clock()
> while done == False:
>

"while not done"

Also, just use "while True" and a "break". I know some C-people or what not
think this is "evil" or something, but they're wrong.

Personally, I actually like using:

while "rotating the squares":
...

instead of "while True". It's no slower and it's free documentation.


> for event in pygame.event.get():
> if event.type == pygame.QUIT:
> done = True
>
> a.rotate() #perform rotation
> b.rotate()
> c.rotate()
> d.rotate()
>
> screen.fill(black)
>
> a.draw() #draw point
> b.draw()
> c.draw()
> d.draw()
> pygame.display.flip()
> clock.tick(30)
>
> pygame.quit()[/code]
>

You don't need pygame.quit(). You *only* want pygame.quit() if you're
quitting Pygame *before* the program is finished.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python testing tools

[Y Yen]

On Friday, July 19, 2013 3:56:24 PM UTC-7, cutems93 wrote:
> I am currently doing some research on testing software for Python. I found 
> that there are many different types of testing tools. These are what I've 
> found.
> 
> 
> 
> 1.Unit test   
> 
> 2.Mock test
> 
> 3.Fuzz test
> 
> 4.Web test
> 
> 5.Acceptance/business logic test
> 
> 6.GUI test
> 
> 7.Source code checking
> 
> 8.Code coverage
> 
> 9.Continuous integration
> 
> 10.Automatic test runners
> 
> 11.Test fixtures
> 
> 
> 
> I know web and GUI testing tools are for specific uses. For instance, if you 
> are not working with GUI or web pages, you don't need those testing tools. 
> Other than these two, do you use all of the other nine testing tools? I think 
> many of you are using unit testing tools, such as unittest and doctest, and 
> source code checking tools, like pylint or pychecker. Do you guys use 
> #2,3,5,8,9,10 and 11 often?
> 
> 
> 
> Thanks!
> 
> 
> 
> -Min S.

I found the python's unittest framework lack good reporting tools. 

For my project (www.srcmap.com), I want something that does high level features 
integration tests, scriptable, generate report in HTML table format that label 
pass ->Green, fail -> Red.   Separate by scripts, commands runs, etc.

  In python, it is so trivial to write you own.  It parse the json test scripts 
and execute the commands inside one at a time.  It works very well. 

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[Terry Reedy]

On 7/24/2013 2:15 PM, Chris Angelico wrote:

On Thu, Jul 25, 2013 at 3:52 AM, Terry Reedy  wrote:



For my purpose, the mock Text works the same in 2.7 and 3.3+.


Thanks for that report! And yes, it's going to behave exactly the same
way, because its underlying structure is an ordered list of ordered
lists of Unicode codepoints, ergo 3.3/PEP 393 is merely a question of
performance. But if you put your code onto a narrow build, you'll have
issues as seen below.


I carefully said 'For my purpose', which is to replace the tk Text 
widget. Up to 8.5, Tk's text is something like Python's narrow-build 
unicode.


If put astral chars into the toy editor, then yes, it would not work on 
narrow builds, but would on 3.3+.


 ...

> If nobody had ever thought of doing a multi-format string

representation, I could well imagine the Python core devs debating
whether the cost of UTF-32 strings is worth the correctness and
consistency improvements... and most likely concluding that narrow
builds get abolished. And if any other language (eg ECMAScript)
decides to move from UTF-16 to UTF-32, I would wholeheartedly support
the move, even if it broke code to do so.


Making a UTF-16 implementation correct requires converting abstract 
'character' array indexes to concrete double byte array indexes. The 
simple O(n) method of scanning the string from the beginning for each 
index operation is too slow. When PEP393 was being discussed, I devised 
a much faster way to do the conversion.


The key idea is to add an auxiliary array of the abstract indexes of the 
astral chars in the abstract array. This is easily created when the 
string is created and can be done afterward with one linear scan (which 
is how I experimented with Python code). The length of that array is the 
number of surrogate pairs in the concrete 16-bit codepoint array. 
Subtracting that number from the length of the concrete array gives the 
length of the abstract array.


Given a target index of a character in the abstract array, use the 
auxiliary array to determine k, the number of astral characters that 
precede the target character. That can be done with either a O(k) linear 
scan or O(log k) binary search. Add 2 * k to the abstract index to get 
the corresponding index in the concrete array. When slicing a string 
with i0 and i1, slice the auxiliary array with k0 and k1 and adjusting 
the contained indexes downward to get the corresponding auxiliary array.



To my mind, exposing UTF-16 surrogates to the application is a bug

> to be fixed, not a feature to be maintained.

It is definitely not a feature, but a proper UTF-16 implementation would 
not expose them except to codecs, just as with the PEP 393 
implementation. (In both cases, I am excluding the sys size function as 
'exposing to the application'.)


> But since we can get the best of both worlds with only

a small amount of overhead, I really don't see why anyone should be
objecting.


I presume you are referring to the PEP 393 1-2-4 byte implementation. 
Given how well it has been optimized, I think it was the right choice 
for Python. But a language that now uses USC2 or defective UTF-16 on all 
platforms might find the auxiliary array an easier fix.


--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[Chris Angelico]

On Thu, Jul 25, 2013 at 8:09 AM, Terry Reedy  wrote:
> On 7/24/2013 2:15 PM, Chris Angelico wrote:
>> To my mind, exposing UTF-16 surrogates to the application is a bug
>> to be fixed, not a feature to be maintained.
>
> It is definitely not a feature, but a proper UTF-16 implementation would not
> expose them except to codecs, just as with the PEP 393 implementation. (In
> both cases, I am excluding the sys size function as 'exposing to the
> application'.)
>
>> But since we can get the best of both worlds with only
>> a small amount of overhead, I really don't see why anyone should be
>> objecting.
>
> I presume you are referring to the PEP 393 1-2-4 byte implementation. Given
> how well it has been optimized, I think it was the right choice for Python.
> But a language that now uses USC2 or defective UTF-16 on all platforms might
> find the auxiliary array an easier fix.
>

I'm referring here to objections like jmf's, and also to threads like this:

http://mozilla.6506.n7.nabble.com/Flexible-String-Representation-full-Unicode-for-ES6-td267585.html

According to the ECMAScript people, UTF-16 and exposing surrogates to
the application is a critical feature to be maintained. I disagree.
But it's not my language, so I'm stuck with it. (I ended up writing a
little wrapper function in C that detects unpaired surrogates, but
that still doesn't deal with the possibility that character indexing
can create a new character that was never there to start with.)

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[Michael Torrie]

On 07/24/2013 04:19 PM, Chris Angelico wrote:
> I'm referring here to objections like jmf's, and also to threads like this:
> 
> http://mozilla.6506.n7.nabble.com/Flexible-String-Representation-full-Unicode-for-ES6-td267585.html
> 
> According to the ECMAScript people, UTF-16 and exposing surrogates to
> the application is a critical feature to be maintained. I disagree.
> But it's not my language, so I'm stuck with it. (I ended up writing a
> little wrapper function in C that detects unpaired surrogates, but
> that still doesn't deal with the possibility that character indexing
> can create a new character that was never there to start with.)

This is starting to drift off topic here now, but after reading your
comments on that post, and others objections, I don't fully understand
why making strings simply "unicode" in javascript breaks compatibility
with older scripts.  What operations are performed on strings that
making unicode an abstract type would break?  Is it just in the input
and output of text that must be decoded and encode?  Why should a script
care about the internal representation of unicode strings?  Is it
because the incorrect behavior of UTF-16 and the exposed surrogates (and
subsequent incorrect indexing) are actually depended on by some scripts?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[Chris Angelico]

On Thu, Jul 25, 2013 at 8:59 AM, Michael Torrie  wrote:
> I don't fully understand
> why making strings simply "unicode" in javascript breaks compatibility
> with older scripts.  What operations are performed on strings that
> making unicode an abstract type would break?


Imagine this in JavaScript and Python (apart from  the fact that JS
doesn't do backslash escapes past 0x1):

a = "asdf\U00012345qwer";
b = a[[..10];

What will this do? It depends on whether UTF-16 is used or not.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: dict & dict.keys()

[Terry Reedy]

On 7/24/2013 4:34 PM, Prasad, Ramit wrote:


I am still not clear on the advantage of views vs. iterators.


A1: Views are iterables that can be iterated more than once. Therefore, 
they can be passed to a function that re-iterates its inputs, or to 
multiple functions. They support 'x in view' as efficiently as possible. 
Think about how you would write the non-view equivalent of '(0,None) in 
somedict.views())'. When set-like, views support some set operations. 
For .keys, which are always set-like, these operations are easy to 
implement as dicts are based on a hashed array of keys.


Q2: What is the advantage of views vs. lists?

A2: They do not take up space that is not needed. They can be converted 
to lists, to get all the features of lists, but not vice versa.



What makes d.viewkeys() better than d.iterkeys()? Why did they decide
not to rename d.iterkeys() to d.keys() and instead use d.viewkeys()?


This is historically the wrong way to phrase the question. The 2.7 
.viewxyz methods were *not* used to make the 3.x .xyz methods. It was 
the other way around. 3.0 came out with view methods replacing both list 
and iter methods just after 2.6, after a couple of years of design, and 
a year and a half before 2.7. The view methods were backported from 3.1 
to 2.7, with 'view' added to the name to avoid name conflicts, to make 
it easier to write code that would either run on both 2.7 and 3.x or be 
converted with 2to3.


A better question is: 'When 3.0 was designed, why were views invented 
for the .xyz methods rather than just renaming the .iterxyz methods. The 
advantages given above are the answer. View methods replace both list 
and iterator methods and are more flexible than either and directly or 
indirectly have all the advantages of both.


My question is why some people are fussing so much because Python 
developers gave them one thing that is better than either of the two 
things it replaces?


The mis-phrased question above illustrates why people new to Python 
should use the latest 3.x and ignore 2.x unless they must use 2.x 
libraries. 2.7 has all the old stuff, for back compatibility, and as 
much of the new stuff in 3.1 as seemed sensible, for forward 
compatibility. Thus it has lots of confusing duplication, and in this 
case, triplication


--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: dict & dict.keys()

[Ethan Furman]

On 07/24/2013 01:34 PM, Prasad, Ramit wrote:


I am still not clear on the advantage of views vs. iterators. What
makes d.viewkeys() better than d.iterkeys()? Why did they decide
not to rename d.iterkeys() to d.keys() and instead use d.viewkeys()?
Is the iteration over a set operation on keys really that common a
use case?


From a practical standpoint, iterkeys() is a one-shot deal, while viewkeys() 
can be iterated over multiple times:

--> d = {1: 'one', 2: 'two', 3: 'three'}

--> di = d.iterkeys()

--> list(di)
[1, 2, 3]

--> list(di)
[]

--> dv = d.viewkeys()

--> list(dv)
[1, 2, 3]

--> list(dv)
[1, 2, 3]

And views are not sets -- they just support a couple set-like operations.

--
~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list

Re: Beginner. 2d rotation gives unexpected results.

[Terry Reedy]

On 7/24/2013 5:17 PM, Joshua Landau wrote:


import math as m


GAH!

Why on earth would you do such a thing?


for the same reason people do 'import tkinter as tk': to minimize typing 
and maximize clarity. In this case,

  from math import sin, cos, radians
also works well

--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: how: embed + extend to control my running app?

[David M. Cotter]

update: okay so the python27.dll is in /windows/system32 so ignore that

i've set my include directory correct, so i can compile

i've set my "additional libraries" directory to the "libs" directory (where the 
".lib" files are.  (note: NOT including "Lib" directory, cuz that's full of .py 
files and folders) (note: NOT including "DLLs" directory, cuz, why would i?)

No need to specify "additional dependencies" for the .lib file, cuz the 
pyconfig.h file does that.

but there is no "python27_d.dll" anywhere to be found, so i hacked pyconfig.h 
to get rid of the "_d".

so it all compiles.

but it won't link:

LNK2001: unresolved external symbol __imp___Py_RefTotal
LNK2001: unresolved external symbol __imp___Py_NoneStruct
LNK2019: unresolved external symbol __imp__PyArg_ParseTuple
LNK2019: unresolved external symbol __imp__PyFloat_FromDouble
LNK2019: unresolved external symbol __imp__PyString_FromString
LNK2019: unresolved external symbol __imp__PyRun_SimpleStringFlags
LNK2019: unresolved external symbol __imp__Py_InitModule4TraceRefs
LNK2019: unresolved external symbol __imp__Py_Initialize
LNK2019: unresolved external symbol __imp__Py_SetProgramName
LNK2019: unresolved external symbol __imp__Py_Finalize
LNK2019: unresolved external symbol __imp__PyRun_SimpleFileExFlags

what, pray tell, am i doing wrong?  *hopeful face*
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: dict & dict.keys()

[Ethan Furman]

On 07/23/2013 07:11 PM, Steven D'Aprano wrote:

On Tue, 23 Jul 2013 18:16:08 -0700, Ethan Furman wrote:


So now, in Python 3, .keys(), .values(), even .items() all return these
'view' thingies.

And everything I thought I knew about when to use one or the other went
out the window.


Surely not. The fundamental behaviour of Python's data model hasn't
changed.


Poetic effect.  Dramatic license.  Blah blah.  ;)



Repeat after me: "In Python 2, d.keys() returns a list of keys, so if I
want a list of keys in Python 3, call list explicitly list(d.keys())."


Actually, I would recommend `list(d)`, which also works the same in both 2 and 
3.

--
~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list

Re: how: embed + extend to control my running app?

[Dave Angel]

On 07/24/2013 08:51 PM, David M. Cotter wrote:

update: okay so the python27.dll is in /windows/system32 so ignore that

i've set my include directory correct, so i can compile

i've set my "additional libraries" directory to the "libs" directory (where the ".lib" files are.  
(note: NOT including "Lib" directory, cuz that's full of .py files and folders) (note: NOT including "DLLs" 
directory, cuz, why would i?)

No need to specify "additional dependencies" for the .lib file, cuz the 
pyconfig.h file does that.

but there is no "python27_d.dll" anywhere to be found, so i hacked pyconfig.h to get rid 
of the "_d".

so it all compiles.

but it won't link:

LNK2001: unresolved external symbol __imp___Py_RefTotal
LNK2001: unresolved external symbol __imp___Py_NoneStruct
LNK2019: unresolved external symbol __imp__PyArg_ParseTuple
LNK2019: unresolved external symbol __imp__PyFloat_FromDouble
LNK2019: unresolved external symbol __imp__PyString_FromString
LNK2019: unresolved external symbol __imp__PyRun_SimpleStringFlags
LNK2019: unresolved external symbol __imp__Py_InitModule4TraceRefs
LNK2019: unresolved external symbol __imp__Py_Initialize
LNK2019: unresolved external symbol __imp__Py_SetProgramName
LNK2019: unresolved external symbol __imp__Py_Finalize
LNK2019: unresolved external symbol __imp__PyRun_SimpleFileExFlags

what, pray tell, am i doing wrong?  *hopeful face*



Digging *far* back in my Windows memory, those look like imports.  You 
probably need the import lib for the Python.dll.  Probably called 
somethng like python.lib.   You could check that by doing a dumpbin of 
python.dll and searching for those entry points.


An import lib in Windows simply tells the linker that those symbols will 
be resolved at runtime, and from a particular dll. They can also change 
the names to be used (removing the __imp__ prefix) and even specify a 
numeric entry point (to slow down people who reverse engineer these things).


If I recall right, there's a way in Microsoft's toolset to create an 
import lib from a dll, assuming the dll doesn't restrict itself to those 
numeric thingies.


As for the _d suffix, that's commonly used to specify debug versions of 
things.  They would have extra symbol information, and less optimized 
code so that it's easier to use a debugger on them.



--
DaveA

--
http://mail.python.org/mailman/listinfo/python-list

I have a little problem with Django when I am trying to create a new app

[peins0242]

Hello everyone I'm watching a tutorial on how to create a project on Django...

django-admin.py startproject carabali

when I  run this code on terminal.. happens :


http://nsae01.casimages.net/img/2013/07/25/130725021220676239.png


There's a mistake or something but I can't figure out, help me please
-- 
http://mail.python.org/mailman/listinfo/python-list

High Availability with neo4j-embedded in Python -- Is that possible?

[jotavrj]

I'm almost switching from Python to Groovy because I can't find a way to use 
the neo4j-embedded's Highly Available mode. But no, I certainly don't want 
Java... I'm using JPype.

Is there a way to do it?

Kind regards,
Jota Junior
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: dict & dict.keys()

[Ben Finney]

Ethan Furman  writes:

> On 07/23/2013 07:11 PM, Steven D'Aprano wrote:
> > On Tue, 23 Jul 2013 18:16:08 -0700, Ethan Furman wrote:
> >> And everything I thought I knew about when to use one or the other went
> >> out the window.
> >
> > Surely not. The fundamental behaviour of Python's data model hasn't
> > changed.
>
> Poetic effect.  Dramatic license.  Blah blah.  ;)

Text-only medium. Clarity of communication. Et cetera. :-)

-- 
 \   “Two hands working can do more than a thousand clasped in |
  `\   prayer.” —Anonymous |
_o__)  |
Ben Finney

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: I have a little problem with Django when I am trying to create a new app

[Chris Angelico]

On Thu, Jul 25, 2013 at 12:03 PM,   wrote:
> Hello everyone I'm watching a tutorial on how to create a project on Django...
>
> django-admin.py startproject carabali
>
> when I  run this code on terminal.. happens :
>
>
> http://nsae01.casimages.net/img/2013/07/25/130725021220676239.png
>
>
> There's a mistake or something but I can't figure out, help me please

It's saying that it can't find the zlib module. What Linux
distribution is this? You may need to 'apt-get install' / 'yum
install' / 'pacman -S' another package to make this work - possibly
it'll be called 'zlib' or 'python-zlib' or something. I would
recommend a Google search for your Linux distribution and the words
'python importerror zlib'; there's a high chance that'll tell you what
you need to install.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

[ANN] PyWeek 17 will run in the first week of September (1st to 8th) - write a game in Python in a week

[Richard Jones]

Hi all,

The Python Game Programming Challenge  will run
its 17th challenge during the first week of September, from the 1st to
the 8th.

The PyWeek challenge:

1. Invites entrants to write a game in one week from scratch either as
an individual or in a team,
2. Is intended to be challenging and fun,
3. Will increase the public body of game tools, code and expertise,
4. Will let a lot of people actually finish a game, and
5. May inspire new projects (with ready made teams!)

Check out the help page for how to compete (and prepare) and the
growing resources message board post:

   http://pyweek.org/s/help/
   http://pyweek.org/d/4008/
-- 
http://mail.python.org/mailman/listinfo/python-list

Python3, GUIs, game APIs, PyGLet, 2to3...?

[John Ladasky]

I am teaching Python 3 to a few beginning computer programming students.  Being 
high-school age boys, they are, unsurprisingly, interested in games.  I want to 
introduce them to real-time programming and GUI in the most painless way 
possible.

I know that Python comes bundled with Tkinter.  Aside from the fact that I 
dislike its look and feel, Tkinter is not a beginners' GUI tool.  You have to 
write a fair amount of boiler-plate code, and you have to understand classes 
pretty well.  My students are not at the OOP level yet.  Yet Python's turtle 
package, which is built on Tkinter, manages to avoid a lot of that complexity, 
at least at first.  I am looking for a tool which gives my students the 
simplicity of turtle, but which works with 2D raster graphics.  And, it must 
work on Py3.

Deep in the middle of another thread, I had PyGLet recommended to me.  So I 
tried to get PyGLet working on Python 3, which it supposedly will do.  It 
installs on Py3, but importing pyglet fails.  The test program also fails.  The 
tracebacks were showing me that the package code is full of Python 2.x print 
statements.  I started fixing them manually and, after a while, gave up.

https://groups.google.com/d/msg/comp.lang.python/J64gfFg3ZKw/hH-lXurR70EJ

There may be other Py3-incompatible code in the PyGLet package that I haven't 
encountered yet.  Thus I have looked at the Python docs for the "2to3" utility. 
 2to3 probably does what I want, except for one thing: recursive operations on 
subfolders.  Do I really have to manually traverse the directory tree of the 
package, modify one folder's worth of .py files, and then repeat ad nauseam?  
Yes, I could write a script to do that -- but shouldn't that functionality be 
built into 2to3?

Any advice that you folks might offer -- either about getting 2to3 to execute 
recursively, or about installing any GUI with a shallow learning curve 
installed on Py3, would be appreciated.  Thanks.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[Serhiy Storchaka]

24.07.13 21:15, Chris Angelico написав(ла):

To my mind, exposing UTF-16
surrogates to the application is a bug to be fixed, not a feature to
be maintained.


Python 3 uses code points from U+DC80 to U+DCFF (which are in surrogates 
area) to represent undecodable bytes with surrogateescape error handler.



--
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: dict & dict.keys()

[Steven D'Aprano]

On Wed, 24 Jul 2013 08:57:11 -0700, Ethan Furman wrote:

> My point is that in 2.x .keys() did something different from the dict,
> while in 3.x it appears to me that they are the same.

Then you aren't looking very closely. d.keys() returns a set-like view 
into the dict, which is great for comparing elements:


py> d1 = dict.fromkeys([1, 2, 3, 4])
py> d2 = dict.fromkeys([3, 4, 5, 6])
py> d1.keys() & d2.keys()  # keys that are in both
{3, 4}
py> d1.keys() ^ d2.keys()  # keys not in both
{1, 2, 5, 6}
py> d1.keys() - d2.keys()  # keys only in d1
{1, 2}
py> d2.keys() - d1.keys()  # keys only in d2
{5, 6}


Dicts aren't sets, and don't support set methods:

py> d1 - d2
Traceback (most recent call last):
  File "", line 1, in 
TypeError: unsupported operand type(s) for -: 'dict' and 'dict'


> Peter's point is that in the stdlib the new functionality of .keys() is
> never used, not even once.

The standard library is not the universe of Python code, and most of the 
standard library predates Python 3. Some of it goes back to Python 1 
idioms. In general, working code doesn't get updated until it stops 
working.

I have code that manually walks over each dict and extracts keys that are 
in both, or one but not the other. Once I drop support for Python 2.6, I 
throw that code away and just use views. But until then, I'm stuck doing 
it the horrible way. Judging by a naive grep of my code, you might think 
I never used views. I do, I just have to reinvent the wheel.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: dict & dict.keys()

[Steven D'Aprano]

On Wed, 24 Jul 2013 13:17:12 -0400, Terry Reedy wrote:

> On 7/24/2013 12:34 PM, Chris Angelico wrote:
> 
>> Side point: Why is iterating over a dict equivalent to .keys() rather
>> than .items()? It feels odd that, with both options viable, the
>> implicit version iterates over half the dict instead of all of it.
>> Obviously it can't be changed now, even if .items() were the better
>> choice, but I'm curious as to the reason for the decision.
> 
> Both were considered and I think there were and are two somewhat-linked
> practical reasons. First, iterating over keys in more common than
> iterating over items. The more common one should be the default.
> 
> Second, people ask much more often if 'key' is in dict than if 'key,
> value' is in dict. This is true as well for keyed reference books such
> as phone books, dictionaries, encyclopedias, and for the same reason.
> This is  coupled with the fact that the default meaning of 'item in
> collection' is that iterating over 'collection' eventually produces
> 'item' or a value equal to 'item'.

That second point was the deciding factor when direct iteration over 
dicts was added. has_key() was deprecated in favour of "key in dict", and 
that pretty much forced iteration to go over keys by default. The 
reasoning is, "x in y" ought to be equivalent to:

for tmp in y:
if x == tmp: return True
return False

There's probably even a PEP about this, if anyone is less lazy/busy and 
can be bothered looking for it.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[Steven D'Aprano]

On Wed, 24 Jul 2013 09:00:39 -0600, Michael Torrie wrote about JMF:

> His most recent argument that Python should use UTF as a representation
> is very strange to be honest.

He's not arguing for anything, he is just hating on anything that gives 
even the tiniest benefit to ASCII users. This isn't about Python 3.3. 
hurting non-ASCII users, because that is demonstrably untrue: they are 
*better off* in Python 3.3. This is about denying even a tiny benefit to 
ASCII users.

In Python 3.3, non-ASCII users have these advantages compared to previous 
versions:

- strings will usually take less memory, and aside from trivial changes 
to the object header, they never take more memory than a wide build would 
use;

- consequently nearly all objects will take less memory (especially 
builtins and standard library objects, which are all ASCII), since 
objects contain dozens of internal strings (attribute and method names in 
__dict__, class name, etc.);

- consequently whole-application benchmarks show most applications will 
use significantly less memory, which leads to faster speeds;

- you cannot break surrogate pairs apart by accident, which you can do in 
narrow builds;

- in previous versions, code which works when run in a wide build may 
fail in a narrow build, but that is no longer an issue since the 
distinction between wide and narrow builds is gone;

- Latin1 users, which includes JMF himself, will likewise see memory 
savings, since Latin1 strings will take half the size of narrow builds 
and a quarter the size of wide builds.


The cost of all these benefits is a small overhead when creating a string 
in the first place, and some purely internal added complication to the 
string implementation.

I'm the first to argue against complication unless there is a 
corresponding benefit. This is a case where the benefit has proven itself 
doubly: Python 3.3's Unicode implementation is *more correct* than 
before, and it uses less memory to do so.

> The cons of UTF are apparent and widely
> known.  The main con is that UTF strings are O(n) for indexing a
> position within the string.

Not so for UTF-32.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: dict & dict.keys()

[Steven D'Aprano]

On Wed, 24 Jul 2013 17:59:43 -0700, Ethan Furman wrote:

>> Repeat after me: "In Python 2, d.keys() returns a list of keys, so if I
>> want a list of keys in Python 3, call list explicitly list(d.keys())."
> 
> Actually, I would recommend `list(d)`, which also works the same in both
> 2 and 3.

Fair point.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[Chris Angelico]

On Thu, Jul 25, 2013 at 3:49 PM, Serhiy Storchaka  wrote:
> 24.07.13 21:15, Chris Angelico написав(ла):
>
>> To my mind, exposing UTF-16
>> surrogates to the application is a bug to be fixed, not a feature to
>> be maintained.
>
>
> Python 3 uses code points from U+DC80 to U+DCFF (which are in surrogates
> area) to represent undecodable bytes with surrogateescape error handler.

That's a deliberate and conscious use of the codepoints; that's not
what I'm talking about here. Suppose you read a UTF-8 stream of bytes
from a file, and decode them into your language's standard string
type. At this point, you should be working with a string of Unicode
codepoints:

"\22\341\210\264\360\222\215\205"

-->

"\x12\u1234\U00012345"

The incoming byte stream has a length of 8, the resulting character
stream has a length of 3. Now, if the language wants to use UTF-16
internally, it's free to do so:

0012 1234 d808 df45

When I referred to exposing surrogates to the application, this is
what I'm talking about. If decoding the above byte stream results in a
length 4 string where the last two are \xd808 and \xdf45, then it's
exposing them. If it's a length 3 string where the last is \U00012345,
then it's hiding them. To be honest, I don't imagine I'll ever see a
language that stores strings in UTF-16 and then exposes them to the
application as UTF-32; there's very little point. But such *is*
possible, and if it's working closely with libraries that demand
UTF-16, it might well make sense to do things that way.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: dict & dict.keys()

[Chris Angelico]

On Thu, Jul 25, 2013 at 3:48 PM, Steven D'Aprano
 wrote:
> Dicts aren't sets, and don't support set methods:
>
> py> d1 - d2
> Traceback (most recent call last):
>   File "", line 1, in 
> TypeError: unsupported operand type(s) for -: 'dict' and 'dict'

I wouldn't take this as particularly significant, though. A future
version of Python could add that support (and it might well be very
useful), without breaking any of the effects of views.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: dict & dict.keys()

[alex23]

On 25/07/2013 4:31 AM, Ethan Furman wrote:

   2) Hopefully learn something about when a view is useful.


I haven't seeen this mentioned - forgive me if it's a repeat - but views 
are constant references to whichever set they represent.


Python 2.7:

>>> dd = dict(a=1,b=2,c=3)
>>> keys = dd.keys()
>>> 'a' in keys
True
>>> dd['d'] = 4
>>> 'd' in keys
False

Python 3.3:
>>> dd = dict(a=1,b=2,c=3)
>>> keys = dd.keys()
>>> 'a' in keys
True
>>> dd['d'] = 4
>>> 'd' in keys
True

If part of my code is only interested in what keys or values are 
present, it doesn't need to be given a reference to the full dictionary, 
just to whichever view it cares about.

--
http://mail.python.org/mailman/listinfo/python-list