Re: [Tutor] Process list elements as consecutive pairs

[Peter Otten]

Rüdiger Wolf wrote:

> I am trying to Process list elements as consecutive pairs  into
> consecutive pairs.
> Any pythonic suggestions?
> 
> listin = [1,2,3,4,5,6,7,8,9,10]
> I want to process as consecutive pairs
> 1,2
> 3,4
> 5,6
> 7,8
> 9,10

>>> listin = [1,2,3,4,5,6,7,8,9,10]
>>> it = iter(listin)
>>> zip(it, it)
[(1, 2), (3, 4), (5, 6), (7, 8), (9, 10)]

If listin as an odd length the last item will be lost.

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] writing csv files

[Peter Otten]

prasad rao wrote:

> hello!
> 
> I got a problem writing csv file.
> 
> 1)
>   csvw=csv.writer(open('/home/prasad/kkm','w'),
> dialect='excel',fieldnames=names)
> Traceback (most recent call last):
>   File "", line 1, in 
> TypeError: 'fieldnames' is an invalid keyword argument for this function
> 
> 2)
>  for x in csvr:
> ...y=lambda x: ''.join([x.split()[3],x.split()[-3],x.split()[-6]])
> ...csvw.write(y)
> ...
> Traceback (most recent call last):
>   File "", line 3, in 
> AttributeError: '_csv.writer' object has no attribute 'write'
> 
> At  http://www.python.org/dev/peps/pep-0305/  they are using  the function
> write
>  and the argument fieldnames.
> 
> Where is the problem?Please someone show me  a way to do it.

The PEP is not uptodate. If you are using Python 2.6 have a look at

http://docs.python.org/library/csv.html

You can find the documentation for other versions of Python via 
http://docs.python.org/index.html

If you are using 2.6 the following might work:

# prepare rows
rows = (row.split() for row in csvr)
rows = ((row[3], row[-3], row[-6]) for row in rows)

with open(filename, "wb") as out:
w = csv.writer(out, dialect="excel")
w.writerow(names) # write header
w.writerows(rows) # write data

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] list of dicts <-> dict of lists?

[Peter Otten]

David Perlman wrote:

> Using the csv.DictReader and csv.DictWriter lets you read and write
> lists of dictionaries from files containing tabular data.  I have a
> system that naturally generates tabular data in the form of a
> dictionary of lists: the dictionary key is the name of the column, and
> then the value is a list which contains the data for that column.
> Sort of like a spreadsheet.  I would like to use csv.DictWriter to
> write out this data but that requires a list of dicts.
> 
> I can convert the dict of lists to a list of dicts, and thus make it
> compatible with csv.DictWriter, with the following ugly comprehensions:
> 
>  >>> y
> {'a': [1, 2, 3], 'c': [7, 8, 9], 'b': [4, 5, 6]}
>  >>> [dict([(i,y[i][j]) for i in y.keys()]) for j in
> range(len(y[y.keys()[0]]))]
> [{'a': 1, 'c': 7, 'b': 4}, {'a': 2, 'c': 8, 'b': 5}, {'a': 3, 'c': 9,
> 'b': 6}]
> 
> ...but I'm wondering if anyone knows of a more elegant way, perhaps
> something built-in suited for this purpose...
> 
> I am aware that any solution will only work if the lists in the dict
> are all the same length.  :)

I think it's simpler and therefore more appropriate to use a normal 
csv.writer here:

>>> import csv
>>> import sys
>>> data = {'a': [1, 2, 3], 'c': [7, 8, 9], 'b': [4, 5, 6]}
>>> writer = csv.writer(sys.stdout)
>>> writer.writerow(data.keys())
a,c,b
>>> writer.writerows(zip(*data.values()))
1,7,4
2,8,5
3,9,6

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] list of dicts <-> dict of lists?

[Peter Otten]

David Perlman wrote:

> Oh, except one problem: the csv.DictWriter lets you tell it what order
> you want the columns output in.  With your version, they just show up
> in whatever order Python wants them.

That's not hard to fix:

>>> fieldnames = "abca"
>>> cols = [data[fn] for fn in fieldnames]
>>> writer.writerows(zip(*cols))
1,4,7,1
2,5,8,2
3,6,9,3

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] accented characters to unaccented

[Peter Otten]

KB SU wrote:

> Hi,
> 
> I have open url and read like following:
> 
> $import urllib
> $txt = urllib.urlopen("http://www.terme-catez.si";).read()
> $txt

> If you see above, in junk of HTLM, there is text like 'Terme
> \xc4\x8cate\xc5\xbe'  (original is 'Terme Čatež'). Now, I want to convert
> code like '\xc4\x8c' or '\xc5\xbe' to unaccented chars so that 'Terme
> \xc4\x8cate\xc5\xbe' become 'Terme Catez'. Is there any way convert from
> whole HTML.

First convert to unicode with 

txt = txt.decode("utf-8") and then follow

http://effbot.org/zone/unicode-convert.htm


Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Looking for duplicates within a list

[Peter Otten]

Ken G. wrote:

> I have been working on this problem for several days and I am not making
> any progress.  I have a group of 18 number, in ascending order, within a
> list.  They ranged from 1 to 39.  Some numbers are duplicated as much as
> three times or as few as none.
> 
> I started with one list containing the numbers.  For example, they are
> listed as like below:
> 
> a = [1, 2, 3, 3, 4]
> 
> I started off with using a loop:
> 
> for j in range (0, 5):
> x = a[0] # for example, 1
> 
> How would I compare '1' with 2, 3, 3, 4?
> 
> Do I need another duplicated list such as b = a and compare a[0] with
> either b[0], b[1], b[2], b[3], b[4]?
> 
> Or do I compare a[0] with a[1], a[2], a[3], a[4]?
> 
> In any event, if a number is listed more than once, I would like to know
> how many times, such as 2 or 3 times.  For example, '3' is listed twice
> within a list.

I haven't seen a solution that takes advantage of the fact that the numbers 
are sorted, so here's one:

>>> items = [1, 2, 3, 3, 4]
>>> value = items[0]   
>>> n = 0
>>> result = []
>>> for item in items:
... if value != item:
... print value, "-->", n
... value = item
... n = 1   
... else:   
... n += 1  
... 
1 --> 1 
2 --> 1
3 --> 2
>>> print value, "-->", n
4 --> 1

Python has a cool feature called generator that allows you to encapsulate 
the above nicely:

>>> def count_runs(items):
... it = iter(items)
... value = next(it)
... n = 1 # next consumes one item
... for item in it: # the for loop starts with the second item
... if value != item:
... yield value, n
... value = item
... n = 1
... else:
... n += 1
... yield value, n
...
>>> for v, n in count_runs([1, 1, 1, 2, 2, 3, 4, 4, 4, 4]):
... print v, "-->", n
...
1 --> 3
2 --> 2
3 --> 1
4 --> 4

Finally, there is itertools.groupby() which allows you to simplify the 
implementation count_runs():

>>> from itertools import groupby
>>> def count_runs2(items):
... for key, group in groupby(items):
... yield key, sum(1 for _ in group)
...
>>> for v, n in count_runs2([1, 1, 1, 2, 2, 3, 4, 4, 4, 4]):
... print v, "-->", n
...
1 --> 3
2 --> 2
3 --> 1
4 --> 4

Peter


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Converting audio samples from 16-bit signed int to float?

[Peter Otten]

Joe Veldhuis wrote:

> Hello all. I'm writing a program that needs to capture audio from a
> soundcard and run FFTs to determine peak frequency for further processing.
> The soundcard's native capture format is 16-bit little-endian signed
> integer samples (values 0-65535), and of course the FFT function requires
> floating-point values (-1.0 - +1.0).
> 
> So, what is the most efficient way to do the necessary conversion? I'm
> using the pyalsaaudio module to access the soundcard, if it matters.

Using numpy should be pretty efficient. Assuming you get your data as a 
bytestring:

>>> import numpy
>>> data = "\x00\x80\x00\x00\xff\x7f"
>>> a = numpy.fromstring(data, dtype=">> a
array([-32768,  0,  32767], dtype=int16)
>>> a/32768.
array([-1.,  0.,  0.6948])

Or, if you don't find the half-open interval acceptable:

>>> (a+.5)/32767.5
array([ -1.e+00,   1.52590219e-05,   1.e+00])

If you want to use the full interval and avoid moving the origin:

>>> (a>0)*a/32767.+(a<0)*a/32768.
array([-1.,  0.,  1.])

(There may be a better way as I'm not very familiar with numpy and found the 
above by trial and error)

Peter



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] split struggle

[Peter Otten]

Richard D. Moores wrote:

> Please see my Python 3.1 code pasted at
> .
> 
> This does what I want, which is to do one of:
> 1. print all the elements of the list, lst.
> 2. print "Done" when "" is entered.
> 3. print the elements of lst whose indexes are entered.
> (sorry if all this is obvious)
> 
> Now, the code works, but isn't there a better way to do what I want?
> I've been away from Python for a while, and have gotten rusty.

Although the code gets slightly more complex having a look at the cmd module 
may be worthwhile. Once you have set up the bases you can add commands by 
adding a do_xxx() method assuming 'xxx' is the name of the command. Tab 
completion works (for commands) and you get a rudimentary help function for 
free.

$ cat indexes.py
import cmd

class Cmd(cmd.Cmd):
prompt = "Enter a command -> "
def precmd(self, line):
if line.strip().isdigit():
# treat a lone integer as a shortcut
# for the 'one' command
return "one " + line
if not line:
return "EOF"
return line
def do_all(self, line):
"print all items"
for i, item in enumerate(items):
print_item(i)
def do_one(self, line):
i = int(line)
print_item(i)
def do_EOF(self, line):
print("That's all, folks")
return True

items = [100,101,102,103]

def print_item(i):
print(i, "->", items[i])

Cmd().cmdloop("Enter a command or type 'help' for help")
$ python3 indexes.py
Enter a command or type 'help' for help
Enter a command -> help

Documented commands (type help ):

all

Undocumented commands:
==
EOF  help  one

Enter a command -> one 1
1 -> 101
Enter a command -> help all
print all items
Enter a command -> all
0 -> 100
1 -> 101
2 -> 102
3 -> 103
Enter a command -> 3
3 -> 103
Enter a command ->
That's all, folks
$



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] I don't understand this code

[Peter Otten]

ZUXOXUS wrote:

> Hi,
> 
> I am a true beginner in programming, and im learning with
> inventwithpython.com.
> 
> There's something I dont understand, and i would really appreciate any
> help.
> 
> In chapter 9, the one about the Hangman game, I don't get the block of
> code in line 61
> 
> 59.  words = 'ant baboon badger bat bear'
> 60.
> 
>1. def getRandomWord(wordList):
>2. # This function returns a random string from the passed list of
>strings.
>3. wordIndex = random.randint(0, len(wordList) - 1)
>4. return wordList[wordIndex]
> 
> 
> The thing is, the "passed list of strings" is called "words", not
> "wordList", so I see it shouldn't work.
> 
> On the other hand, the variable "wordList" is defined nowhere!
> 
> The code is ok, because the program runs ok. So there is somethings that i
> dont get.
> 
> Thank you very much in advance.

A value can have a different name inside a function when it is passed as a 
parameter. Consider the following session:

>>> def get_first_word(whatever_you_like):
... return whatever_you_like[0]
...
>>> names = "peter paul mary".split()
>>> words = "nobody expects the spanish inquisition".split()
>>> get_first_word(names)
'peter'
>>> get_first_word(words)
'nobody'

Both the 'names' and the 'words' list of strings are referred to as 
'whatever_you_like' inside the get_first_word() function.

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] A file containing a string of 1 billion random digits.

[Peter Otten]

bob gailer wrote:

> Check this out:
> 
> import random, time
> s = time.time()
> cycles = 1000
> d = "0123456789"*100
> f = open("numbers.txt", "w")
> for i in xrange(n):
>l = []
>l.extend(random.sample(d, 1000))
>f.write(''.join(l))
> f.close()
> print time.time() - s

Note that this is not random. E. g. the start sequence "0"*101 should have a 
likelyhood of 1/10**101 but is impossible to generate with your setup.

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] A file containing a string of 1 billion random digits.

[Peter Otten]

Richard D. Moores wrote:

> On Mon, Jul 19, 2010 at 04:51, Peter Otten <__pete...@web.de> wrote:
>> bob gailer wrote:
>>
>>> Check this out:
>>>
>>> import random, time
>>> s = time.time()
>>> cycles = 1000
>>> d = "0123456789"*100
>>> f = open("numbers.txt", "w")
>>> for i in xrange(n):
>>> l = []
>>> l.extend(random.sample(d, 1000))
>>> f.write(''.join(l))
>>> f.close()
>>> print time.time() - s
>>
>> Note that this is not random. E. g. the start sequence "0"*101 should
>> have a likelyhood of 1/10**101 but is impossible to generate with your
>> setup.
> I not sure exactly what you mean, because I don't fully understand
> that '*' (despite Alan's patient explanation), but if you run
> 
> import random
> cycles = 10
> d = "0123456789"*10
> for i in range(cycles):
> l = []
> l.extend(random.sample(d, 100))
> s = (''.join(l))
> if s[:4] == '0101':
> print(s)
> 
> You'll see a bunch of strings that begin with "0101"
> 
> Or if you run
> 
> import random
> cycles = 50
> d = "0123456789"*10
> for i in range(cycles):
> l = []
> l.extend(random.sample(d, 100))
> s = (''.join(l))
> if s[:1] == '0':
> print(s)
> 
> You'll see some that begin with '0'.
> 
> Am I on the right track?

No. If you fire up your python interpreter you can do

>>> "0"*10
'00'

i. e. "0"*101 is a sequence of 101 zeros. Because a sample can pick every 
item in the population only once and there are only 100 zeros, at most 100 
of them can be drawn, and the more are drawn the less likely it becomes that 
another one is drawn. The simplest demo is probably

random.sample([0, 1], 2)

Possible returns are [0, 1] and [1, 0], but for true randomness you want [1, 
1] and [0, 0], too. The more often the items are repeated the less 
pronounced that bias becomes, e. g.

random.sample([0, 1, 0, 1], 2)

can produce all combinations, but [0, 1] is twice as likely as [0, 0] 
because once the first 0 is drawn there is only one 0 left, but two 1s.
Here's a demonstration:

>>> from collections import defaultdict
>>> d = defaultdict(int)
>>> for i in range(1000):
... d[tuple(random.sample([0, 1]*2, 2))] += 1
...
>>> dict(d)
{(0, 1): 333, (1, 0): 308, (0, 0): 174, (1, 1): 185}

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] "x and y" means "if x is false, then x, else y"??

[Peter Otten]

Lie Ryan wrote:

> On 07/05/10 22:23, Adam Bark wrote:
> 
>> 
>> I should add that this is how something like:
>> 
>> if x != y:
>> do_something()
>> 
>> works, if expects a True or False (this isn't always true but works for
>> comparison operators expressions such as this).
>> 
> 
>  "if" expects an expression that can be converted to True or False
> by calling its __bool__()/__nonzero__(); in case of missing
> __bool__/__nonzero__, then the object is considered True. 

Don't forget about __len__()

>>> class A:
... def __init__(self, n): self.n = n
... def __len__(self): return self.n
...
>>> "yes" if A(1) else "no"
'yes'
>>> "yes" if A(0) else "no"
'no'

Bonus:

>>> "yes" if A(-1) else "no"
Traceback (most recent call last):
  File "", line 1, in 
ValueError: __nonzero__ should return >= 0

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] decorators

[Peter Otten]

Mary Morris wrote:

> I'm trying to compile a list of decorators from the source code at my
> office.
> I did this by doing a
> 
> candidate_line.find("@")
> 
> because all of our decorators start with the @ symbol.  The problem I'm
> having is that the email addresses that are included in the comments are
> getting included in the list that is getting returned.
> I was thinking I could do a candidate_line.find(".com") to set the email
> addresses apart, but how do I tell the computer to not include the lines
> it finds with ".com" in them in the list?

You can use the tokenize module to do the heavy lifting, see

http://docs.python.org/library/tokenize.html

Here's an example:

$ cat find_decos.py
import tokenize
from collections import defaultdict

class Token:
def __init__(self, token):
self.string = token[1]
self.lineno = token[2][0]

def find_decos(instream, filename, decos):
tokens = (Token(token) for token in   
  tokenize.generate_tokens(instream.readline))
for token in tokens:  
if token.string == "@":   
lineno = token.lineno 
qname = [next(tokens).string] 
for token in tokens:  
if token.string == ".":   
qname.append(next(tokens).string) 
else: 
break 
decos[".".join(qname)].append((lineno, filename))

def main():
import sys
files = sys.argv[1:]
if not files:   
# read filenames from stdin
files = (line.strip() for line in sys.stdin)

decorators = defaultdict(list)
for filename in files:
with open(filename) as instream:
find_decos(instream, filename, decorators)
for name in sorted(decorators):
print name
for location in decorators[name]:
print "%8d %s" % location

if __name__ == "__main__":
main()

if False:
def f():
"""
@not_a_decorator
"""
return g
# @not_a_decorator

@alpha
def first(x):
return "u...@example.com"

@beta
@gamma . one
def second():
pass

@delta.two.three.four(*args)
@epsilon(42)
def third():
pass

The if False:... suite is of course not a necessary part of the script, it's 
just a trick to cram in a few decorators for the script to find when you run 
it over itself:

$ python find_decos.py find_decos.py
alpha
  50 find_decos.py
beta
  54 find_decos.py
delta.two.three.four
  59 find_decos.py
epsilon
  60 find_decos.py
gamma.one
  55 find_decos.py

Alternatively you can feed filenames via stdin:

$ find /usr/lib/python2.6 -name \*.py | python find_decos.py | tail
 429 /usr/lib/python2.6/dist-
packages/usbcreator/frontends/kde/frontend.py
 434 /usr/lib/python2.6/dist-
packages/usbcreator/frontends/kde/frontend.py
 446 /usr/lib/python2.6/dist-
packages/usbcreator/frontends/kde/frontend.py
threaded
 166 /usr/lib/python2.6/dist-
packages/softwareproperties/kde/DialogMirror.py
withResolverLog
 572 /usr/lib/python2.6/dist-packages/DistUpgrade/DistUpgradeCache.py
 858 /usr/lib/python2.6/dist-packages/DistUpgrade/DistUpgradeCache.py
wraps
  81 /usr/lib/python2.6/contextlib.py
$

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] finding duplicates within a tuple of tuples

[Peter Otten]

Norman Khine wrote:

> hello,
> 
> i have this tuple:
> 
> http://paste.lisp.org/+2F4X
> 
> i have this, which does what i want:
> 
> from collections import defaultdict
> 
> d = defaultdict(set)
> for id, url in result:
> d[url].add(id)
> for url in sorted(d):
> if len(d[url]) > 1:
> print('%d -- %s' % (len(d[url]), url))
> 
> so here the code checks for duplicate urls and counts the number of
> occurences.
> 
> but i am sort of stuck in that i want to now update the id of the
> related table and update the
> 
> basically i have two tables:
> 
> id, url
> 24715L, 'http://aqoon.local/muesli/2-muesli-tropical-500g.html'
> 24719L, 'http://aqoon.local/muesli/2-muesli-tropical-500g.html'
> 
> id, tid,
> 1, 24715L
> 2, 24719L
> 
> so i want to first update t(2)'s tid to t(1)'s id for each duplicate
> and then delete the row id = 24719L

You can use another dictionary that maps ids associated with the same url to 
a canonical id.

from collections import defaultdict

url_table = [
(24715,"http://aqoon.local/muesli/2-muesli-tropical-500g.html";),
(24719,"http://aqoon.local/muesli/2-muesli-tropical-500g.html";),
(24720,"http://example.com/index.html";)
]

id_table = [
(1, 24715),
(2, 24719),
(3, 24720)
]

dupes = defaultdict(set)
for uid, url in url_table:
dupes[url].add(uid)

lookup = {}
for synonyms in dupes.itervalues():
if len(synonyms) > 1:
canonical = min(synonyms)
for alias in synonyms:
assert alias not in lookup
lookup[alias] = canonical

ids = [(id, lookup.get(uid, uid)) for id, uid in id_table]
print ids
urls = [(min(synonyms), url) for url, synonyms in dupes.iteritems()]
print urls

Note that if you use a database for these tables you can avoid creating 
duplicates in the first place.

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] finding duplicates within a tuple of tuples

[Peter Otten]

Norman Khine wrote:

> Here is the latest version http://pastie.org/1066582 can this be
> further improved?

> # get all the duplicates and clean the products table
> main.execute("SELECT product_Id, url FROM %s.product WHERE url != ''" % 
db)
> results = main.fetchall()
> 
> d = defaultdict(set)
> for id, url in results:
>   d[url].add(id)
> 
> for ids in d.itervalues():
>   if len(ids) > 1:
>   # we now hove the first product_id added
>   canonical = min(ids)
>   ids = list(ids)
>   ids.pop(ids.index(canonical))
>   for id in ids:
>   update_product_id = 'UPDATE 
> oneproduct.productList_product_assoc SET 
productList_id=%s WHERE productList_id=%s'
>   main.execute(update_product_id, (id, canonical))
>   org.commit()
> main.close()

Yes; do it in SQL. Here's my attempt:

# Disclaimer: I stopped at the first point where it seemed to work; don't 
# apply the following on valuable data without extensive prior tests.
 
import sqlite3 
WIDTH = 80
db = sqlite3.connect(":memory:")

url_table = [
(24715,"http://aqoon.local/muesli/2-muesli-tropical-500g.html";),
(24719,"http://aqoon.local/muesli/2-muesli-tropical-500g.html";),
(24720,"http://example.com/index.html";)
]
cursor = db.cursor()
cursor.execute("create table alpha (id, url)")
cursor.executemany("insert into alpha values (?, ?)", url_table)

c2 = db.cursor()

id_table = [
(1, 24715),
(2, 24719),
(3, 24720)
]

cursor.execute("create table beta (id, alpha_id)")
cursor.executemany("insert into beta values (?, ?)", id_table)

def show(name):
print name.center(WIDTH, "-")
for row in cursor.execute("select * from %s" % name):
print row
print

print " BEFORE ".center(WIDTH, "=")
show("alpha")
show("beta")

cursor.execute("""
create view gamma as 
select min(a.id) new_id, b.id old_id from alpha a, alpha b 
where a.url = b.url group by a.url
""")
cursor.execute("""
update beta 
set alpha_id = (select new_id from gamma where alpha_id = old_id) 
where (select new_id from gamma where alpha_id = old_id) is not Null
""")

cursor.execute("""
delete from alpha 
where id not in (select min(b.id) 
from alpha b where alpha.url = b.url)
""")

print " AFTER ".center(WIDTH, "=")
show("alpha")
show("beta")

A database expert could probably simplify that a bit.

Again: duplicate records are best not created rather than removed. If you 
can create a unique index for the url column and alter your insertion code 
appropriately.

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Conflict with encoding in console view and file dump

[Peter Otten]

Alex Baraibar wrote:

> Hello, would you please look at the comments in the code and help me with
> an answer?
> Thanx.
> 
> # -*- coding: cp1252 -*-
> 
> def festivos():
> fest = [ 'Any Nou:\t\t\t1 de enero',
>  'Reis:\t\t\t\t6 de enero',
>  'Festa del Treball:\t\t1 de mayo',
>  'Sant Joan:\t\t\t24 de junio',
>  u'La Assumpció:\t\t\t15 de agosto',
>  'La Diada:\t\t\t11 de septiembre',
>  u'La Mercè:\t\t\t24 de septiembre',
>  'La Hispanitat:\t\t\t12 de octubre',
>  'Tots Sants:\t\t\t1 de novembre',
>  u'La Constitució:\t\t\t6 de desembre',
>  u'La Concepció:\t\t\t8 de desembre',
>  'Nadal:\t\t\t\t25 de desembre',
>  'Sant Esteve:\t\t\t26 de desembre' ]
> return fest
> 
> def separador( num, char ):
> return char * num
> 
> # --- Main ---
> dias = festivos()
> print "Los festivos fijos anuales son:\n"
> for element in dias:
> sep = separador( 50, '-' )
> 
> # If I turn off encoding latin-1, accented characters look
> # wrong when I output them to a file from the command line, but
> # if I turn on encoding, accented characters look
> # wrong in the console view.
> print element.encode( 'latin-1' )
> 
> print sep
> raw_input()

Hm, I would expect an exception when you redirect the output to a file.
If you add

import sys
print >> sys.stderr, sys.stdout.encoding

what does the above print 
(a) when you print to the console
(b) when you redirect to a file?

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] A unicode print question

[Peter Otten]

Joel Goldstick wrote:

> I was reading this: http://diveintopython.org/xml_processing/unicode.html
> 
> and tried the exercise:
> 
 s = u'La Pe\xf1a' [image: 1]
 print s   [image: 2]
> Traceback (innermost last):
>   File "", line 1, in ?
> UnicodeError: ASCII encoding error: ordinal not in range(128)
 print s.encode('latin-1') [image: 3]
> La Peña
> 
>  [image:
>  
[1]
> 
> But oddly enough, when I typed it into my python shell I did NOT get the
> UnicodeError, and I wonder why not:
> 
> Python 2.6.5 (r265:79063, Apr 16 2010, 13:09:56)
> [GCC 4.4.3] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
 s = u'La Pe\xf1a'
 print s
> La Peña
 import sys
 sys.getdefaultencoding()
> 'ascii'
>>
> 
> 
> Maybe an earlier version of python produces the error, but not 2.6.5?

This works a bit differently, probably since Python 2.3. See

http://docs.python.org/library/stdtypes.html#file.encoding

The "old" behaviour may still bite you when you redirect stdout.

$ python -c"print u'Pe\xf1a'"
Peña
$ python -c"print u'Pe\xf1a'" > /dev/null
Traceback (most recent call last):
  File "", line 1, in 
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf1' in position 
2: ordinal not in range(128)

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Maximum recursion depth problem.

[Peter Otten]

Wesley Brooks wrote:

> I'm having real difficulty understanding why the following is not
> working and hoped I've either missed something obvious of I'm doing
> something wrong!
> 
> class A:
> def break_down(self, value, base, broken_list=[]):

> I'm a little stumped as I don't think I'm using any global or class
> variables? Any help would be much appreciated.

You are on the right track, the default value for broken_list is 
evaluated only once; Modifications during an invocation of the 
break_down() method are visible when break_down() is called again later.

See also

http://docs.python.org/faq/design.html#why-are-default-values-shared-between-objects

Peter


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Conflict with encoding in console view and file dump

[Peter Otten]

Alex Baraibar wrote:

> Hi, Peter.
> Sorry for the delay, I've been out for a while.
> So I did what you suggested and it prints the following:
> 
> "
> import sys
> print >> sys.stderr, sys.stdout.encoding
> 
> what does the above print
> (a) when you print to the console? it prints cp850
> (b) when you redirect to a file? it prints None

That's as expected. 

You should normally get an exception like

Traceback (most recent call last):
  File "tmp_enc.py", line 33, in 
print element
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf3' in position 
11: ordinal not in range(128)

when you redirect your script's output to a file.

Did you change the default encoding? If you didn't you probably have an ill-
behaved application on your system that did it without asking.
 
> Any suggestions?

Check stdout.encoding, and if it is None (unknown encoding) wrap the stdout 
stream into a Writer. Your script with that modification:

# -*- coding: cp1252 -*-

def festivos():
fest = [ 'Any Nou:\t\t\t1 de enero',
 'Reis:\t\t\t\t6 de enero',
 'Festa del Treball:\t\t1 de mayo',
 'Sant Joan:\t\t\t24 de junio',
 u'La Assumpció:\t\t\t15 de agosto',
 'La Diada:\t\t\t11 de septiembre',
 u'La Mercè:\t\t\t24 de septiembre',
 'La Hispanitat:\t\t\t12 de octubre',
 'Tots Sants:\t\t\t1 de novembre',
 u'La Constitució:\t\t\t6 de desembre',
 u'La Concepció:\t\t\t8 de desembre',
 'Nadal:\t\t\t\t25 de desembre',
 'Sant Esteve:\t\t\t26 de desembre' ]
return fest

def separador( num, char ):
return char * num

# --- Main ---
import sys
if sys.stdout.encoding is None:
import codecs
Writer = codecs.getwriter("latin-1")
sys.stdout = Writer(sys.stdout)

dias = festivos()
print "Los festivos fijos anuales son:\n"
for element in dias:
sep = separador( 50, '-' ) # move that out of the loop
print element
print sep

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Conflict with encoding in console view and file dump

[Peter Otten]

Alex Baraibar wrote:

> I just wish I understood better what happened and why. I've looked into
> the Unicode section of the official docs, but I probably need to study it
> more in order to fully understand it, so...

Just in case you didn't come across it, there's a howto that covers the 
basics:

http://docs.python.org/howto/unicode

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] string conversion according to the terminal

[Peter Otten]

ANKUR AGGARWAL wrote:

> Hey- suppose we have a file name-"my file number"
> if we want to execute it into the terminal it would be like - my\ file\
> number
> 
> so wondering is there any one in the python that change the enter string
> into the terminal string one-
> like if user enter the file name with path- "my file number". i want to
> automatically convert it into "my\ file\ number"

If you want this to prepare an os.system() call you should instead use 
subprocess.call() with a list containing the command and a filename that is 
not escaped.

>>> import subprocess, os

Bad:

>>> os.system(r"ls -l my\ file\ number")
-rw-r--r-- 1 petto petto 0 2010-08-12 15:20 my file number
0

Good:

>>> subprocess.call(["ls", "-l", "my file number"])
-rw-r--r-- 1 petto petto 0 2010-08-12 15:20 my file number
0

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Immutable objects

[Peter Otten]

Nitin Das wrote:

> class mymut(object):
> 
>   def __setattr__(self,k,v):
>   if hasattr(self,k):
>   if self.__dict__.get(k) == None:
>   self.__dict__[k] = v
>   else:
>   raise TypeError("Cant Modify Attribute Value")
>   else:
>   raise TypeError("Immutable Object")
> 
> 
> class mm(mymut):
> x = ''
> y = ''
> def __init__(self,x,y):
> self.x = x
> self.y = y
> 
> 
> 
> p = mm(10,11)
> print p.x
> print p.y
> 
> 
> I have created this immutable object.Is there any other better
> implementation?

How about

>>> from collections import namedtuple
>>> Mm = namedtuple("Mm", "x y")
>>> p = Mm(10, 11)
>>> p.x
10
>>> p.y
11
>>> p.x = 42
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: can't set attribute
>>> p.z = 42
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: 'Mm' object has no attribute 'z'
>>>

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Immutable objects

[Peter Otten]

Peter Otten wrote:

> Nitin Das wrote:
> 
>> class mymut(object):
>> 
>>   def __setattr__(self,k,v):
>>   if hasattr(self,k):
>>   if self.__dict__.get(k) == None:
>>   self.__dict__[k] = v
>>   else:
>>   raise TypeError("Cant Modify Attribute Value")
>>   else:
>>   raise TypeError("Immutable Object")
>> 
>> 
>> class mm(mymut):
>> x = ''
>> y = ''
>> def __init__(self,x,y):
>> self.x = x
>> self.y = y
>> 
>> 
>> 
>> p = mm(10,11)
>> print p.x
>> print p.y
>> 
>> 
>> I have created this immutable object.Is there any other better
>> implementation?
> 
> How about
> 
>>>> from collections import namedtuple
>>>> Mm = namedtuple("Mm", "x y")
>>>> p = Mm(10, 11)
>>>> p.x
> 10
>>>> p.y
> 11
>>>> p.x = 42
> Traceback (most recent call last):
>   File "", line 1, in 
> AttributeError: can't set attribute
>>>> p.z = 42
> Traceback (most recent call last):
>   File "", line 1, in 
> AttributeError: 'Mm' object has no attribute 'z'
>>>>

By the way, you can see the class definition with

namedtuple("Mm", "x y", verbose=True)

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] List comprehension for dicts?

[Peter Otten]

Pete wrote:

> Hi,
> 
> I've been reading up on list comprehensions lately, all userful and
> powerful stuff - trying to wrap my brain around it :)
> 
> As the examples all seem to relate to lists, I was wondering if there is
> an elegant similar way to apply a function to all keys in a dictionary?
> 
> (without looping over it, of course)
> 
> I'm trying to convert all keys in a dict to uppercase, as in:
> 
> INPUT:
> age_dict = { 'pete': 42, 'ann': 25, 'carl': 30, 'amanda': 64 }
> 
> OUTPUT:
> age_dict = { 'PETE': 42, 'ANN': 25, 'CARL': 30, 'AMANDA': 64 }
> 
> I googled 'dictionary comprehension' but couldn't get any code to work
> with the examples given.

Python 2.4-2.6
>>> dict((k.upper(), v) for k, v in age_dict.iteritems())
{'PETE': 42, 'ANN': 25, 'AMANDA': 64, 'CARL': 30}

Python 2.7
>>> {k.upper(): v for k, v in age_dict.iteritems()}
{'PETE': 42, 'ANN': 25, 'AMANDA': 64, 'CARL': 30}


Python 3.x
>>> {k.upper(): v for k, v in age_dict.items()}
{'PETE': 42, 'ANN': 25, 'AMANDA': 64, 'CARL': 30}


items() instead of iteritems() works in 2.x, too, but is less efficient.

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] question about import statement

[Peter Otten]

Greg Bair wrote:

> On 08/26/2010 10:29 PM, Hugo Arts wrote:
>> On Thu, Aug 26, 2010 at 9:16 PM, Bill Allen  wrote:
>>>
>>> I did try that and of course it gave an error because it was necessary. 
>>> I
>>> just did not know why.   However, I later found an explanation on the
>>> web. Here it is:
>>>
>>> from tkinter import *
>>> from tkinter import ttk
>>>
>>> These two lines tell Python that our program needs two modules. The
>>> first, "tkinter", is the standard binding to Tk, which when loaded also
>>> causes the existing Tk library on your system to be loaded. The second,
>>> "ttk", is Python's binding to the newer "themed widgets" that were added
>>> to Tk in 8.5.
>>>
>> 
>> yeah, "from package import *" doesn't actually import every name from
>> a module. For example, by default, names starting with an underscore
>> are not imported. Alternatively, if you have a variable named __all__
>> in your module, and it's a list of names, only those names in the list
>> actually get imported when you do a "from x import *"
>> 
>> Hugo
> Why would the person who wrote this package choose to do it this way,
> though?  If it was something that people would use and not just an
> internal name, why hide it this way?

There are many programs out there that don't use ttk. For those it would be 
a waste of time and memory if the tkinter.ttk module were imported 
automatically. If you modify your example to

from tkinter import *
root = Tk()
button = Button(root, text="Hello World").grid()
root.mainloop()

it will still work (but maybe look a little different). If the ttk import 
were automatic the above script might not work on machines that don't have 
"themed widgets" even though it doesn't use them. 

In general it is good practice that modules introduce as few dependencies as 
possible.

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] __new__ over __init__

[Peter Otten]

Payal wrote:

> Can someone please give a very simple example of using __new__ wherein
> __init__ cannot be used?

Subclasses of immutable types, e. g. tuple:

>>> class A(tuple):
... def __init__(self, a, b):
... pass
...
>>> a = A(1,2)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: tuple() takes at most 1 argument (2 given)
>>> class A(tuple):
... def __new__(cls, a, b):
... return tuple.__new__(cls, (a, b))
...
>>> A(1, 2)
(1, 2)
>>> type(_)


Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] __new__ over __init__

[Peter Otten]

Alan Gauld wrote:

> "Payal"  wrote
>> b. What does type(_) mean?
> 
> The _ refers to the last evaluated result, in this case the tuple
> (1,2).
> Its a shorthand trick, I think it only works in the interpreter, I
> don't like
> it and never use it, but many do. (FWIW Perl has a similar shortcut
> and Perl fans use it a lot!)

_ does indeed only work in interactive mode. It is handy when you evaluate 
an expression and only then notice that you want to do something with the 
result. 

I added the type(_) line as an afterthought when I saw that the A instance 
was indistinguishable from a tuple.

A more forward-thinking demo would have been

>>> class A(tuple):
... def __new__(cls, a, b):
... return tuple.__new__(cls, (a, b))
...
>>> a = A(1, 2)
>>> a
(1, 2)
>>> type(a)


Peter



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Exception Handling and Stack traces

[Peter Otten]

Michael Powe wrote:

> I can't work out how to suppress stacktrace printing when exceptions
> are thrown.

[snip rant]

It might have been a good idea to read a tutorial like

http://docs.python.org/tutorial/errors.html#handling-exceptions

or ask before you got annoyed enough to write that rant ;)

To catch an exception you have to put the class into the except clause, not 
an instance. Basic example, using 2.6 syntax:

WRONG:

>>> try:
... 1/0
... except ZeroDivisionError("whatever"):
... print "caught"
...
Traceback (most recent call last):
  File "", line 2, in 
ZeroDivisionError: integer division or modulo by zero

CORRECT:

>>> try:
... 1/0
... except ZeroDivisionError as e:
... print "caught", e
...
caught integer division or modulo by zero

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Exception Handling and Stack traces

[Peter Otten]

Michael Powe wrote:

> On Fri, Sep 10, 2010 at 03:56:51PM +0200, Peter Otten wrote:
>> Michael Powe wrote:
>> 
>> > I can't work out how to suppress stacktrace printing when exceptions
>> > are thrown.
>> 
>> [snip rant]
>> 
>> It might have been a good idea to read a tutorial like
>> 
>> http://docs.python.org/tutorial/errors.html#handling-exceptions
>  
>> or ask before you got annoyed enough to write that rant ;)
> 
> Hello,
> 
> Thanks for the reply.  Stupid me, I just read a half dozen articles on
> the web about python exception handling, including some at
> docs.python.  At no point is the 'as' clause discussed as being
> required.

>> WRONG:
>> 
>> >>> try:
>> ... 1/0
>> ... except ZeroDivisionError("whatever"):
>> ... print "caught"
>> ...
>> Traceback (most recent call last):
>>   File "", line 2, in 
>> ZeroDivisionError: integer division or modulo by zero
>> 
>> CORRECT:
>> 
>> >>> try:
>> ... 1/0
>> ... except ZeroDivisionError as e:
>> ... print "caught", e
>> ...
>> caught integer division or modulo by zero

> Note that in section 8.3 of that article, the statement is made that
> if the exception matches the the exception type in the following
> format, the statements within the except clause are executed.
> 
> except URLError :
> # do something
> 
> That in fact, seems to me to be incorrect.  It is not my experience
> (e.g., print statements are not executed in the example I gave and the
> sys.exit() is not called).

Sorry, the as-clause is *not* necessary. The relevant difference between the 
correct and the wrong approach is that you must not instantiate the 
exception:

WRONG:

>>> try:
... 1/0
... except ZeroDivisionError("whatever"):
... print "caught"
...
Traceback (most recent call last):
  File "", line 2, in 
ZeroDivisionError: integer division or modulo by zero

CORRECT:

>>> try:
... 1/0
... except ZeroDivisionError:
... print "caught"
...
caught

I just put in the as-clause to show an easy way to print the exception. I 
did not anticipate that it would obscure the message.

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] hashlib problems

[Peter Otten]

Rance Hall wrote:

> Im wanting to use the builtin hashlib functions to encrypt passwords
> before storing them in a database.
> 
> According to documentation on the python.org site it should be as simple
> as
> 
> import hashlib
> 
> hashname = hashlib.sha234  (or whatever other hash method you want
> like md5 or whatever)
> 
> hashname.update(b"test")  the b is for byte since the documentation
> states that hashlib does not accept strings
> 
> hashname.hexdigest() or digest() will give you the hash encrypted
> value.  (hexdigest will encode the output in hex for use in email and
> string db storage)
> 
> 
> My problem is that hashname.update results in a:
> 
> AttributeError:  'built_in_function_or_method' object has no attribute
> 'update'
> 
> I get this error in python 2 and python 3
> 
> I get it on multiple operating systems, so I dont understand whats
> going wrong.  Either there is a bug in the module hashlib, or the
> module changed and the docs don't keep up.

Neither
 
> Either way I can not encrypt the password of my project for new users
> till I figure out whats going on.

hashlib.sha224 is indeed a function:

>>> f = hashlib.sha224
>>> f


You have to invoke it to get a hash object:

>>> g = f()
>>> g


Complete working example:

>>> import hashlib
>>> h = hashlib.sha224() # note the parens
>>> h.update(b"test")
>>> h.hexdigest()
'90a3ed9e32b2aaf4c61c410eb925426119e1a9dc53d4286ade99a809'

Look here for another one:

http://docs.python.org/dev/py3k/library/hashlib.html

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] exceptions problem

[Peter Otten]

Steven D'Aprano wrote:

> On Sat, 11 Sep 2010 09:56:41 am bob gailer wrote:
>> > I never thought that you can use a float and a integer to look if
>> > the number is a integer.
>>
>> You can't.
> 
> What? Of course you can.
> 
> def is_integer(x):
> """Return True if x is an integer."""
> try:
> return 1.0*x == int(x)
> except (TypeError, ValueError):
> return False

> The multiplication by 1.0 is not actually needed, but it doesn't hurt.
> 
> [thinks more carefully...] Actually it does hurt:
> 
 is_integer(Decimal(2))
> False

Another problem is the limited precision of floats:

>>> x = 10**22
>>> 1.0*x == x
True
>>> x *= 10
>>> 1.0*x == x
False
 
Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to numerically sort strings that start with numbers?

[Peter Otten]

Pete O'Connell wrote:

> theList = ["21 trewuuioi","3zxc","134445"]
> print sorted(theList)
> 
> Hi, the result of the sorted list above doesn't print in the order I
> want. Is there a straight forward way of getting python to print
> ['3zxc','21 trewuuioi','134445']
> rather than ['134445', '21 trewuuioi', '3zxc']?

You have to write a function that extracts the number and use it as the key 
argument for sorted() or list.sort():

>>> import re
>>> def extract_number(s):
... m = re.compile(r"-?\d+").match(s)
... if m is not None:
... return int(m.group())
...
>>> theList = ["21 trewuuioi","3zxc","134445"]
>>> sorted(theList, key=extract_number)
['3zxc', '21 trewuuioi', '134445']

Have a look as str.isdigit() if you want to write such a function without 
regular expressions.

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Regex comments

[Peter Otten]

Michael Powe wrote:

> The re module includes the option to comment a regular expression with
> the syntax (?#comment).  e.g.
> 
> p=r'(.*) (?PWT.dl)(?#parameter)=(?P[^&]+)(?#value).*'
> 
> Is there a mechanism for extracting these values from the match, in
> the way that group names are extracted?
> 
> I don't see one.

You could write a regular expression to extract them ;)

> The point would be that in my processing of the match, I could
> implement the comments as identifiers for the matched value.

But that's what the names are for, e. g.:

>>> re.compile(r'(.*) (?PWT.dl)=(?P[^&]+).*').search(
" WTxdl=yadda&ignored").groupdict()
{'parameter': 'WTxdl', 'value': 'yadda'}

Or am I missing something?

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] robots question

[Peter Otten]

Roelof Wobben wrote:

> As a exercise from this book ( Thinking like a computer scientist ) I have
> to make this programm on this
> page(http://openbookproject.net/thinkcs/python/english2e/ch12.html)
> Exercise 11

> def check_collisions(robots, junk, player):

> defeated = check_collisions(robot, player, junk)

Just look at the argument names: check_collisions() is probably expecting a 
/list/ of robots where you are passing it a single one.

> But now Im getting this error message :
> 
> Traceback (most recent call last):
>   File "/root/workspace/test2/src/test.py", line 120, in 
> play_game(2)
>   File "/root/workspace/test2/src/test.py", line 106, in play_game
> defeated = check_collisions(robot, player, junk)
>   File "/root/workspace/test2/src/test.py", line 73, in check_collisions
> for thing in robots + junk:
> TypeError: can only concatenate list (not "dict") to list
> 
> I understand that robots is a dict and junk is a list.
> 
> Is that right ?

Yes. You are getting a dict because a single robot's state is stored in a 
dictionary.

> And why does the book say that when this message is appearing.

The way the program is presented makes it hard to tell at what point your 
and Eckel's idea of the script start to differ without going through the 
entire chapter.

Aside: that's why real projects use version control systems. When a program 
stops working after a change there is always the option to go back to the 
state of the program before the bug was introduced.

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] FW: robots question

[Peter Otten]

Roelof Wobben wrote:

> I change everything to this :

> def check_collisions(robots, junk, player):

> defeated = check_collisions(robots, player, junk)

Do you see the problem?

> print "type robots", type(robots)
> print "type junk", type(junk)
> print "type player", type(player)

Adding print statements for debugging purposes is a good approach.

> And now Im getting this message :
> 
> ** Message: pygobject_register_sinkfunc is deprecated (GtkWindow)
> ** Message: pygobject_register_sinkfunc is deprecated (GtkInvisible)
> ** Message: pygobject_register_sinkfunc is deprecated (GtkObject)
> 
> type robotsTraceback (most recent call last):
>  
> type junk 
> type player 
>   File "/root/workspace/test2/src/test.py", line 125, in 
> play_game()
>   File "/root/workspace/test2/src/test.py", line 111, in play_game
> defeated = check_collisions(robots, player, junk)
>   File "/root/workspace/test2/src/test.py", line 74, in check_collisions
> for thing in robots + junk:
> TypeError: can only concatenate list (not "dict") to list
> 
> So far I can see the problem is that player is a dict and the rest is a
> list.
> Is this the correct conclusion ?

It may be correct but it's a low-level view. A more appropriate description 
would be that you are trying to concatenate a list of robots with a player.

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] robots question

[Peter Otten]

Roelof Wobben wrote:

> I changed a lot because of your suggestions.
> But one thing is still a puzzle.
> The robots don't move anymore.
> 
> What I have is this :

> robots = []
> place_robots(4)

> move_robots(robots, player)

You are only moving the robots in the 'robots' list. But that list is empty.

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Remove a dictionary entry

[Peter Otten]

M. 427 wrote:

> (I am very new to python)
> I built a dictionary d={} of lists similar to this :
> 
> d = {
> 'a': ['apricot', 'apple'],
> 'b': ['beach', 'bear', 'bottle'],
> 'c': ['cold', 'cook', 'coleslaw'],
> 'd': ['deep'],
> 'e': ['expression', 'elephant']
> }
> 
> Now i want to go through this dictionary and remove all rows containing
> only one entry. How should I do that?

You should never iterate over a list or dictionary and add or remove items 
to it at the same time. That is a recipe for disaster even if it doesn't 
fail explicitly.

Instead create a new dictionary that contains only the items you are 
interested in:

>>> d = {
... 'a': ['apricot', 'apple'],
... 'b': ['beach', 'bear', 'bottle'],
... 'c': ['cold', 'cook', 'coleslaw'],
... 'd': ['deep'],
... 'e': ['expression', 'elephant']
... }
>>> result = {}
>>> for k, v in d.iteritems():
... if len(v) > 1:
... result[k] = v
...
>>> import pprint
>>> pprint.pprint(result)
{'a': ['apricot', 'apple'],
 'b': ['beach', 'bear', 'bottle'],
 'c': ['cold', 'cook', 'coleslaw'],
 'e': ['expression', 'elephant']}

Peter

PS: Instead of using the pretty print module "pprint" I could have typed

>>> result
{'a': ['apricot', 'apple'], 'c': ['cold', 'cook', 'coleslaw'], 'b': 
['beach', 'bear', 'bottle'], 'e': ['expression', 'elephant']}

The only difference is that it looks messier.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Remove a dictionary entry

[Peter Otten]

M. 427 wrote:

> Version 4 : (2 steps)
> 
> # step 1 : list keys of unwanted rows
> sck=[] # list of single children keys in dictionary
> for k in d.keys() :
> if len(d[k]) < 2 :
> sck.append(k)
> # step 2 : delete all d rows whose key is listed in sck
> while len(sck) > 0 :
> del d[sck.pop()]
> 
> This works.
> Is this the optimal pythonic way of doing it?

Ceterum censeo: the pythonic way is to make a copy:

d = dict((k, v) for k, v in d.iteritems() if len(v) > 1)

As an optimization, if the dictionary is huge and there are relatively few 
items to be deleted you may fall back to the 2-step approach. I would write 
it

delenda = [k for k, v in d.iteritems() if len(v) < 2]
for k in delenda:
del d[k]

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Remove a dictionary entry

[Peter Otten]

Steven D'Aprano wrote:

> On Sat, 18 Sep 2010 07:13:13 pm Peter Otten wrote:
> 
>> You should never iterate over a list or dictionary and add or remove
>> items to it at the same time. That is a recipe for disaster even if
>> it doesn't fail explicitly.
> 
> That's a bit strong. It's quite possible to modify lists safely and
> correctly while iterating over them with a little bit of care.

You're of course not the intended audience of my advice.
 
> You know, for decades people were able to program in languages like C
> and Pascal and assembly, often on machines with tiny amounts of memory.
> When your machine has 64K of memory, and the OS and application uses
> half of it, you don't have the luxury of making a copy of a 20K list
> before modifying it. Back when I was a lad, we learned how to modify
> lists in place. It isn't hard. *wink*

When you do that you are usually operating on an abstraction level below 
Python.
 
> Even in Python, it is sometimes necessary to modify lists and even dicts
> in place while iterating over them. 98% of the time, making a copy is
> faster, simpler and more efficient, but learning how to safely modify
> data structures in place is a valuable skill to have.

If you have a huge list that you can only modify in place you may have 
chosen the wrong data structure.

> But I'm just talking about general principles here. In most cases, stick
> to Peter's advice to make a copy.

Hey, I can agree with that ;)

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Can this be done easly

[Peter Otten]

Roelof Wobben wrote:

> When I change everything to this :

> I get this message :
>  
> Traceback (most recent call last):
>   File "C:\Users\wobben\workspace\oefeningen\src\test.py", line 13, in
>   
> rechthoek = Rectangle (punt,20,30)
> TypeError: object.__new__() takes no parameters

Hint: why does this work:
  
>  def __init__(self, x=0, y=0):

...while this doesnt:

> def _init_(self, base_point, width=0, length=0):

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Can this be done easly

[Peter Otten]

Roelof Wobben wrote:

>> Hint: why does this work:
>>
>>> def __init__(self, x=0, y=0):
>>
>> ...while this doesnt:
>>
>>> def _init_(self, base_point, width=0, length=0):
>>
>> Peter

> Maybe because base_point has no value ?

No. One __init__ has two underscores (correct) on each side, the other 
_init_ only one (wrong).

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] FW: Can this be done easly

[Peter Otten]

Roelof Wobben wrote:

> For this exercise :
> 
> 3.Write a function named move_rect that takes a Rectangle and two
> parameters named dx and dy. It should change the location of the rectangle
> by adding dx to the x coordinate of corner and adding dy to the y
> coordinate of corner.
> 
> Is this one of the possible solutions :
> 
> class Point:
> def __init__(self, x=0, y=0):
> self.x = x
> self.y = y
>  
> class Rectangle(object):
> def __init__(self, base_point, width=0, length=0):
> self.base_point = base_point
> self.width = width
> self.length = length
> 
> def moverect(rectangle, dx, dy):
> rechthoek.base_point.y += dy
> rechthoek.base_point.x +=dx
> return rechthoek
> 
> punt = Point(3,4)
> rechthoek = Rectangle (punt,20,30)
> test = moverect (Rectangle, 4,3)
> print rechthoek.base_point.x

At first glance I'd say so. At second glance I see that you pass the class 
and not an instance to the moverect() routine. Your program only seems to 
work because you are not using the parameter rectangle but the global 
rechthoek variable and as soon as you are trying to move rectangles with a 
different variable name everything will break.
If I were to run your program I might even find more errors or problematic 
behaviours.

In the long run it's not a sustainable model to verify the correctness of 
your programs by asking someone on the internet who is just as likely to be 
wrong as you or might even fool you.

Instead add some tests. For example, you could precalculate the expected 
position and then check if the new position meets your expectation:

r = Rectangle(Point(3, 4), 20, 30)

moverect(r, 10, 11)

if r.base_point.x == 13 and r.base_point.y == 15:
print "looks good"
else:
print "needs work"

Because tests are needed very often there are libraries accompanying the 
interpreter (unittest, doctest) to formalize them and for simple runtime 
checks there is the assert statement. Instead of the if...else you could 
write

assert r.base_point.x == 13, "wrong x position %d" % r.base_point.x
assert r.base_point.y == 15, "wrong y position %d" % r.base_point.y

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] input and raw input

[Peter Otten]

Brian Jones wrote:

> No need for the 're' module. Even in the case where both can be used
> together, you can still just use string methods:
> 
 s
> '12, 13 14'
 s.replace(',', '').split(' ')
> ['12', '13', '14']

I think to replace "," with " " and then split() without explicit separator 
is slightly more robust. Compare:
 
>>> s = "12,34, 56  789"
>>> s.replace(",", " ").split()
['12', '34', '56', '789']
>>> s.replace(",", "").split(" ")
['1234', '56', '', '789']

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] if value in list of dictionaries

[Peter Otten]

Norman Khine wrote:

> thanks for the reply, i think i have it now, perhaps it could be done
> better

> >>> topics.sort(key=itemgetter('name'))
> >>> for i, t in enumerate(topics):
> ... for (k, v) in t.iteritems():
> ... if v == 'other':
> ... topics.append(topics.pop(i))
> ... 
 
You should never iterate over a list or dictionary and add or remove items 
to it at the same time. That is a recipe for disaster even if it doesn't 
fail explicitly*

As Christian Witts explains in the "trouble with list.remove() loop" thread 
you will not see all items.

I suggest that you use a better sort key instead:

>>> def sort_key(topic):
... name = topic["name"]
... return name == "other", name
...
>>> topics.sort(key=sort_key)

>>> pprint(topics)
[{'industry': 'travel',
  'name': 'assistant-manager',
  'value': 'Assistant Manager'},



 {'industry': 'travel', 'name': 'university', 'value': 'University'},
 {'industry': 'travel', 'name': 'other', 'value': 'Other'}]

The above sort_key() checks only the "name" key for an "other" value. It 
will return a (True, name) tuple if the name is "other" and (False, name) 
else. As 

False < True 

it ensures that topics with topic["name"] == "other" are placed after all 
others. If (like your attempt suggests) you want to check all values instead 
of just the one associated with the "name" key use

def sort_key(topic):
return "other" in topic.itervalues(), topic["name"]

Remember that both functions are case-sensitive.

Peter

(*) I'll leave it to Steven D'Aprano to add the fine print ;)

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] trouble with list.remove() loop

[Peter Otten]

D Ryan (2) wrote:

> Hello all,
> I am currently trying to write a program which can find the solution to a
> game of hangman.
> In a part of the program, a user inputs a letter, a tester tells him if
> the word contains that letter, and then if the answer is no, all words
> containing that letter are removed from the list of remaining candidates.
> However, the code I've written seems to remove some, but not all the words
> containing the letter. Strangely though, if i run it a few more times it
> gets some of the ones it missed the 1st time round, untill after enough
> iterations it gets all of them. I cant understand why it doesnt remove all
> of them the first time round. I have cut the offending code and formatted
> it to work on its own, and also to fit into a relatively short email.
> 
> # A sample list of words, one of which is the answer.
> candidates = ["abacus","amazing",
>   "ozimandias",
>   "a","alphanumeric",
>   "functioning"]
> 
> # In the following code, the user has guessed the letter 'a',
> # and the tester has told him that the letter 'a' is not in the word.
> 
> user_guess="a"
> tester_response="no"
> 
> # The following code should eliminate all words which contain the letter
> # 'a', leaving only the word 'functioning' in the list
> 
> if tester_response in ("No","no","n","N","NO"):
> for word in candidates:
> if user_guess in word:
> print word, "removed.."
> candidates.remove(word)
> print candidates
> 
> Running once gives this output
> 
> abacus removed..
> ozimandias removed..
> alphanumeric removed..
> ['amazing', 'a', 'functioning']
> 
> But if i run it again it successfully removes 'amazing, and the next time
> it removes 'a', leaving the correct answer. I'm perplexed by this strange
> behaviour and would be most appreciative of any help. I'm very new to
> python so apologies for any formatting/style errors, and also for the
> simplicity of the problem.

I'd like to point out a robust method to avoid such pitfalls: create a new 
list instead of modifying the old one:

old_candidates = candidates
candidates = []
for word in old_candidates:
if user_guess in word:
print word, "removed"
else:
candidates.append(word)
print candidates

Often you can simplify that to a list comprehension like

candidates = [word for word in candidates if user_guess not in word]

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] generating independent random numbers

[Peter Otten]

Carter Danforth wrote:

> Thanks for the replies, Dave and Joel. The reason I'm not just using the
> time or datetime modules for a random date is because it's restricted to
> 1970-2038; I'm pulling dates from 1600-3099. Thanks a lot for the pointer

The datetime module is not restricted to 1970...2038. It allows years 
1... (it uses the Gregorian calendar even before its adoption). 

>>> import datetime
>>> datetime.MINYEAR, datetime.MAXYEAR
(1, )
>>> datetime.date(1500, 2, 29)
Traceback (most recent call last):
  File "", line 1, in 
ValueError: day is out of range for month
>>> datetime.date(1600, 2, 29)
datetime.date(1600, 2, 29)

The range allowed by the time module is probably implementation dependent. I 
can do things like

>>> time.gmtime(-11670998400)
time.struct_time(tm_year=1600, tm_mon=2, tm_mday=29, tm_hour=0, tm_min=0, 
tm_sec=0, tm_wday=1, tm_yday=60, tm_isdst=0)

>>> time.gmtime(2**55)
time.struct_time(tm_year=1141709097, tm_mon=6, tm_mday=13, tm_hour=6, 
tm_min=26, tm_sec=8, tm_wday=6, tm_yday=164, tm_isdst=0)

>>> time.gmtime(2**56)
Traceback (most recent call last):
  File "", line 1, in 
ValueError: (75, 'Value too large for defined data type')

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] generating independent random numbers

[Peter Otten]

Ewald Ertl wrote:

>> Just an attempt from my side:
> The year 1500 didn't have a 29th of February, the 28th work for me but
> 29th also fails here.
 datetime.date( 1500, 2, 28 )
> datetime.date(1500, 2, 28)
 datetime.date( 1500, 2, 29 )
> Traceback (most recent call last):
> File "", line 1, in 
> ValueError: day is out of range for month

Yes, it's an expected "failure". I should have said that explicitly.

Peter 


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] how to extract data only after a certain condition is met

[Peter Otten]

Eduardo Vieira wrote:

> The other day I was writing a script to extract data from a file from
> the line where a text is found to the end of the file. The same
> functionality is this sed script:
> '1,/regexp/'d
> I couldn't put my head to work around this and came up with a solution
> using list slicing. But how can I do that? I was experimenting with a
> simple list and I came up with this. I wonder if I shouldn't you a
> "while" statement, but how?
> 
> a = ['m', 'a', 'r', 'i', 'g', 'o', 'l', 'd']
> b = True
> 
> for letter in a:
> if letter != 'i' and b:
> continue
> elif letter == 'i':
> b = False
> else:
> print letter
> 
> Ok. This works, but I wonder if I shouldn't you a "while" statement, but
> how?

I would use two for loops:

>>> a = ['m', 'a', 'r', 'i', 'g', 'o', 'l', 'd']
>>> ai = iter(a) # make a list iterator
>>> for letter in ai:
... if letter == "i": break
...
>>> for letter in ai:
... print letter
...
g
o
l
d

Normally a list iterator is created implicitly by writing

for item in some_list:
   ...

but here you have to make one explicitly because you want to reuse it in the 
second loop.

Alternatively, the itertools module has the building blocks for this and 
similar problems:

>>> from itertools import dropwhile, islice
>>> def not_an_i(letter):
... return letter != "i"
...
>>> for letter in dropwhile(not_an_i, a):
... print letter
...
i
g
o
l
d

OK, let's shave off the first item in the dropwhile(...) sequence:

>>> for letter in islice(dropwhile(not_an_i, a), 1, None):
... print letter
...
g
o
l
d

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Converting from unicode to nonstring

[Peter Otten]

David Hutto wrote:

> Hey Buddy Pals,

?

> I receive the following output from a sqlite db
> 
> (u'graph1', u'Line', u'222', u'BLUE', u'1,2,3,4', u'True', u'0,5,0,10')

How did the string u"1,2,3,4" get into the database in the first place?
The sqlite3 module offers a mechanism to convert data from and to Python 
(semi-)transparently:

import sqlite3
import json

sqlite3.register_adapter(list, json.dumps)
sqlite3.register_converter("list", json.loads)

db = sqlite3.connect(":memory:", detect_types=sqlite3.PARSE_DECLTYPES)
cursor = db.cursor()

cursor.execute("create table data (value list)")
cursor.execute("insert into data values (?)", ([11,22,33],))
for row in cursor.execute("select value from data"):
print row


See also:
http://docs.python.org/library/sqlite3.html#converting-sqlite-values-to-
custom-python-types

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] csv DictReader/Writer question

[Peter Otten]

Ara Kooser wrote:

> Hello all,
> 
>   I have a csv file (from a previous code output). It looks like this:
> Species2, Protein ID, E_value, Length, Hit From, Hit to, Protein ID2,
> Locus Tag, Start/Stop, Species
> Streptomyces sp. AA4,  ZP_05482482,  2.82936001e-140,  5256, 
> 1824,
> 2249\n, ZP_05482482,  StAA4_0101000304844,
> complement(NZ_ACEV0178.1:25146..40916)4,  Streptomyces sp. AA4: 0\n
> Streptomyces sp. AA4,  ZP_05482482,  8.03332997e-138,  5256,  123,
> 547\n, ZP_05482482,  StAA4_0101000304844,
> complement(NZ_ACEV0178.1:25146..40916)4,  Streptomyces sp. AA4: 0\n
> Streptomyces sp. AA4,  ZP_05482482,  1.08889e-124,  5256,  3539,  3956\n,
> ZP_05482482,  StAA4_0101000304844,
> complement(NZ_ACEV0178.1:25146..40916)4,  Streptomyces sp. AA4: 0\n
> 
> 
> I want to removing certain sections in each line so I wrote this code
> using csv.DictWriter:
> import csv
> data = csv.DictReader(open('strep_aa.txt'))
> 
> for x in data:
> del x['Species2']
> del x[' Protein ID2']
> print x
> 
>   When it prints to the screen everything works great:
> {' Hit From': '  1824', ' Hit to': '  2249\\n', ' Protein ID': '
> ZP_05482482', ' Locus Tag': '  StAA4_0101000304844', ' Start/Stop': '
> complement(NZ_ACEV0178.1:25146..40916)4', ' Species': '  Streptomyces
> sp. AA4: 0\\n', ' Length': '  5256', ' E_value': '
> 2.82936001e-140'}
> {' Hit From': '  123', ' Hit to': '  547\\n', ' Protein ID': '
> ZP_05482482', ' Locus Tag': '  StAA4_0101000304844', ' Start/Stop': '
> complement(NZ_ACEV0178.1:25146..40916)4', ' Species': '  Streptomyces
> sp. AA4: 0\\n', ' Length': '  5256', ' E_value': '
> 8.03332997e-138'}
> 
> What I don't know how to do is the export this back out a csv file and
> rearrange each key as a column header so it work look like this:
> Species  Protein ID  E Value  .
> 
> I thought csv.DictWriter would be the way to go since it writes
> dictionaries to text files. I was wondering how do I go about doing this?
> I don't really understand the syntax.

It's not about syntax, you have to read the docs for the csv module 
carefully to find the information that is relevant for your problem.
For starters you could try to find out what the skipinitialspace and 
extrasaction keyword parameters are supposed to do in the example below:

with open(source, "rb") as instream:
reader = csv.DictReader(instream, skipinitialspace=True)

destfieldnames = list(reader.fieldnames)
destfieldnames.remove("Species2")
destfieldnames.remove("Protein ID2")

with open(dest, "wb") as outstream:
writer = csv.DictWriter(outstream, destfieldnames, 
extrasaction="ignore")
writer.writer.writerow(destfieldnames)
writer.writerows(reader)

Can you find the line where I'm writing the headers to the destination file?

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] csv DictReader/Writer question

[Peter Otten]

Ara Kooser wrote:

>   Thank you for your response. I did try reading the documentation but I
> missed something (or several somethings in this case). So what I see in
> the code you supplied is:
> 
> with open(source, "rb") as instream:
>reader = csv.DictReader(instream, skipinitialspace=True)
> 
>destfieldnames = list(reader.fieldnames)
>destfieldnames.remove("Species2")
>destfieldnames.remove("Protein ID2")
> 
> So this reads the csv file in but I need to run it to see what
> skipinitialspace does. 

Your csv header looks like

Field1, Field2, Field3, ...

When you feed that to the DictReader you get fieldnames

["Field1", " Field2", " Field3", ...]

skipinitialspace advises the DictReader to remove the leading spaces, i. e. 
you get

["Field1", "Field2", "Field3", ...]

instead.

> Then it reads in the header line and removes the
> Species2 and Protein ID2. Does this remove all the data associated with
> those columns? For some reason I thought I had to bring these into a
> dictionary to manipulate them.

destfieldnames is the list of field names that will be written to the output 
file. I construct it by making a copy of the list of field names of the 
source and then removing the two names of the columns that you don't want in 
the output. Alternatively you could use a constant list like

destfieldnames = ["Field2", "Field5, "Field7"]

to handpick the columns.

>with open(dest, "wb") as outstream:
>writer = csv.DictWriter(outstream,
> destfieldnames,extrasaction="ignore")

The following line uses the csv.writer instance wrapped by the 
csv.DictWriter to write the header.

>writer.writer.writerow(destfieldnames)

The line below iterates over the rows in the source file and writes them 
into the output file.

>writer.writerows(reader)

A more verbose way to achieve the same thing would be

for row in reader:
writer.writerow(row)

Remember that row is a dictionary that has items that shall not be copied 
into the output? By default the DictWriter raises an exception if it 
encounters such extra items. But we told it to silently ignore them with 
extrasaction="ignore".
 
> I think the first line after the open writes the field names to the file
> and the follow lines write the data is that correct? I am going to run the
> code as soon as I get home.

Come back if you have more questions.

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Simple counter to determine frequencies of words in adocument

[Peter Otten]

Alan Gauld wrote:

> The loop is a bit clunky. it would be clearer just to iterate over
> a_list:
> 
> for item in a_list:
>words[item] = a_list.count(item)

This is a very inefficient approach because you repeat counting the number 
of occurrences of a word that appears N times N times:

>>> words = {}
>>> a_list = "in the garden on the bank behind the tree".split()
>>> for word in a_list:
... print "counting", word
... words[word] = a_list.count(word)
...
counting in
counting the # <-- 1
counting garden
counting on
counting the # <-- 2
counting bank
counting behind
counting the # <-- 3
counting tree

>>> words
{'on': 1, 'garden': 1, 'tree': 1, 'behind': 1, 'in': 1, 'the': 3, 'bank': 1}

To avoid the duplicate effort you can check if the word was already counted:

>>> words2 = {}
>>> for word in a_list:
... if word not in words2:
... print "counting", word
... words2[word] = a_list.count(word)
...
counting in
counting the
counting garden
counting on
counting bank
counting behind
counting tree
>>> words == words2
True

Inside the count() method you are still implicitly iterating over the entire 
list once for every distinct word. I would instead prefer counting manually 
while iterating over the list once. This has the advantage that it will even 
work if you don't keep the whole sequence of words in a list in memory (e. 
g. if you read them from a file one line at a time):

>>> words3 = {}
>>> for word in a_list:
... if word in words3:
... words3[word] += 1
... else:
... words3[word] = 1
...
>>> words3 == words
True

Finally there's collections.defaultdict or, in Python 2.7, 
collections.Counter when you are more interested in the result than the way 
to achieve it.

Peter



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] If os.path.lexists() isn't working properly

[Peter Otten]

Susana Iraiis Delgado Rodriguez wrote:

> Hello memebers:
> 
> I'm writing a python script to validate if files with extension .prj
> exist, if they exist it should write 1 or 0 into an excel cell. I've
> working to do this properly, but I'm not getting the results I need. The
> script doesn't find all the files, is like the files don't exist but they
> exist, besides I think is just reading some of the files that are in
> C:\Python 26. I really need a hand on this.
> 
> import os, time, socket, pylab
> from xlwt import Workbook
> from osgeo import ogr,gdal,osr
> from dbf import *
> #Register GAL drivers
> gdal.AllRegister()
> #Create an empty list
> file_list = []
> folders = None
> #Code to indicate directory
> for root, folders, files in os.walk( "C:\\" ):
>  file_list.extend(os.path.join(root,fi) for fi in files if
> fi.endswith(".shp"))
> #Open excel book
> wrkbk = Workbook()
> #Add excel sheet
> wksht = wrkbk.add_sheet('shp')
> wksht.row(0).write(0,'ruta')
> wksht.row(0).write(1,'archivo')
> wksht.row(0).write(2,'prj')
> for row, filepath in enumerate(file_list, start=1):
> wksht.row(row).write(0, filepath)
> (ruta, filename) = os.path.split(filepath)
>  wksht.row(row).write(1, filename)
>  n = os.path.splitext(filename)
>  p = n[0]+'.prj'

Add

   print "looking for", p

here. Does it show what you expect/want? In what directory will lexists() 
look for the file?

>  if os.path.lexists(p):

lexists()' claim to fame is that it "Returns True for broken symbolic 
links". Are you sure you prefer that over good old os.path.exists()?

>   wksht.row(row).write(2, 1)
>  else:
>   wksht.row(row).write(2, 0)
> wrkbk.save('shp3.xls')


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Random Number Question

[Peter Otten]

Alan Gauld wrote:

 import random
> 
> It tries to import itself, which then tries to import itself,
> which then. infinite loop time...

I think it's more like

- look up module random in sys.modules
- module random not found in sys.modules; locate the file random.py
- file random.py found, create a new empty module object and put it into
  sys.modules
- execute code loaded from random.py, encounter 'import random', look up
  random in sys.modules
- module random found in sys.modules, bind it to the name 'random' in itself
- continue execution of random.py code. This will succeed unless you try to
  access random.some_name for a name not already defined.

In short: you are more likely to get an AttributeError than infinite 
recursion.

$ cat module.py
print "importing", __name__
import module
print module.yadda
yadda = 42
$ python -c'import module'
importing module
Traceback (most recent call last):
  File "", line 1, in 
  File "module.py", line 3, in 
print module.yadda
AttributeError: 'module' object has no attribute 'yadda'

$ cat module2.py
print "importing", __name__
yadda = 42
import module2
print module2.yadda
$ python -c'import module2'
importing module2
42

While the latter "works" as in "runs without error" it still isn't a good 
idea.

By the way, you can generalise the above to arbitrary circular imports where 
module a imports b which imports c which imports ... which imports z which 
imports a.

Peter


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] if syntax expression help

[Peter Otten]

Rance Hall wrote:

> I have a user entered variable that I need to check to see if they
> entered one of the two legal values.
> 
> But I only need to check this if one other fact is true.
> 
> 
> we have a variable called "mode"  whose value is either "add" or
> "edit" based on how we where called.
> 
> we have a userentry variable tied to an imput function.
> 
> My current if statement looks like this:
> 
> if ((userentry.lower != "c" or userentry.lower != "i") and
> mode == "add"):

Hint:

>>> userentry = "YADDA"
>>> userentry.lower

>>> userentry.lower()
'yadda'


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] normalize an array

[Peter Otten]

John wrote:

> I know this is a simple problem, but I want to do it the most
> efficient way (that is vectorized...)
> 
> import numpy as np
> 
> a = np.array(([1,2,3,4],[1,.2,3,4],[1,22,3,4]))
> b = np.sum(a,axis=1)
> 
> for i,elem in enumerate(a):
> a[i,:] = elem/b[i]
 
> suggestions?

I'm not a numpy expert, but:

(a.transpose()/np.sum(a, axis=1)).transpose()

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] permutations?

[Peter Otten]

Alex Hall wrote:

> 1. Order matters; I meant to say that direction does not. That is, 123
> is not the same as 213, but 123 is the same as 321 since the second
> example is simply a reversal.
> 2. I am looking for all permutations and subpermutations (if that is a
> word) of 1-n where the list must have at least 2, but no more than n,
> unique numbers (so 1 is not valid, nor is 1231 since it repeats 1 and
> is too long for n=3).

>>> from itertools import permutations, chain
>>> def hall(items):
... items = list(items)
... maxwidth = len(items)
... return chain.from_iterable(
... (p for p in permutations(items, w) if p < p[::-1])
... for w in range(2, maxwidth+1))
...
>>> map("".join, hall("123"))
['12', '13', '23', '123', '132', '213']
>>> map("".join, hall("111"))
[]

Is that it? Be warned though that normally you'll get duplicates if items 
repeat:

>>> map("".join, hall("112"))
['12', '12', '112', '112']

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] SAVE FILE IN A SPECIFIC DIRECTORY

[Peter Otten]

Susana Iraiis Delgado Rodriguez wrote:

[UPPER CASE text is interpreted as shouting in emails and usenet posts. 
Please don't shout. Because spammers do it all the time it's more likely to 
make potential readers turn away from your writ rather than pay extra 
attention.]

> I'm trying to save files into a specific directory, the file will be
> generated from a python script. The script will look for some data stored
> in a directory which only allows to read files, it doesn't let the user
> write or modify information. But when I run the script I got the next
> error: Python 2.6.6 (r266:84297, Aug 24 2010, 18:46:32) [MSC v.1500 32 bit
> (Intel)] on
> win32
> Type "help", "copyright", "credits" or "license" for more information.
 import crawler_shp
> Traceback (most recent call last):
>   File "", line 1, in 
>   File "crawler_shp.py", line 103, in 
> a = open (filepath +t,"w+")
> IOError: [Errno 2] No such file or directory:
> 'C\\Python26\\BUFFER1000_bd.txt'

Looks like you forgot a colon after the "C". Also note that C:\\Python26 is 
a bad place to store your own stuff.

A general remark on your code: it looks like you spend quite some time on 
it, and the up-front time it takes to make it reader-friendly will save you 
and others a multiple in debugging cost. So please

- Use 4-space indents
- Choose descriptive names, not a, t, f, that even if understood in the 
context of an expression will be forgotten three lines below. Also 'wrksht' 
or 'wrkSht' have no advantage over 'worksheet'. Vowels are your friend ;)
- Cut dead wood. Remove variables or imports that aren't currently used. You 
can always reintroduce them later on.
- Bonus: split your code into functions. It may seem like extra work at 
first, but it makes it easy to test small chunks of simple code and then to 
build more complex scripts by combining these "known good" pieces.

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] role playing game - help needed

[Peter Otten]

Robert Sjöblom wrote:

>> Close, but remember that input() returns a string. You need numbers
>> so you need to convert strings to integers.
> 
> Actually, input() only accept integers, consider the following:
 input("input: ")
> input: d
> 
> Traceback (most recent call last):
> File "", line 1, in 
> input("input: ")
> File "", line 1, in 
> NameError: name 'd' is not defined

You are using Python 2.x where raw_input() was used to enter strings and 
input() behaved like

eval(raw_input())

From the above follows that input() in 2.x accepts arbitrary Python 
expressions:

Python 2.6.4 (r264:75706, Dec  7 2009, 18:43:55)
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> d = 42
>>> input()
d
42
>>> input()
d**2
1764
>>> input()
"*".join([str(d)]*5)
'42*42*42*42*42'

I think I drove the point home ;)

input() in Python 3.x on the other hand is similar to 2.x's raw_input():

Python 3.1.1+ (r311:74480, Nov  2 2009, 15:45:00)
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> input()
d
'd'
>>> input()
"*".join([str(d)]*5)
'"*".join([str(d)]*5)'

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] role playing game - help needed

[Peter Otten]

Al Stern wrote:

> Apologies for all my questions.  Up to this point I have been able to work
> out most of the challenges but I seem to have hit a wall.  Can't seem to
> make any progress and completely frustrated.
> 
> I looked at the 11/21 discussion.  From the documentation, I realized I
> needed to set the variables to view the keys and values.  Getting an error
> though.
> 
> attributes = {"strength": 0, "health": 0, "wisdom": 0, "dexterity": 0}
> MAX_POINTS = 30
> keys = attributes.viewkeys()
> values = attributes.viewvalues()
> 
> Traceback (most recent call last):
>   File "C:\Users\Public\Documents\My Python
>   programs\role_playing_game1.py",
> line 8, in 
> keys = attributes.viewkeys()
> AttributeError: 'dict' object has no attribute 'viewkeys'

The dictionary methods you are looking for are called keys() and values() 
not viewkeys() or viewvalues(). They do return view objects which may be 
causing the confusion. Have a look at the documentation at

http://docs.python.org/dev/py3k/library/stdtypes.html#dictionary-view-
objects

which shows a simple example. 
By the way you, can use the interactive interpreter to find out what 
attributes an object has to offer:

$ python3
Python 3.1.1+ (r311:74480, Nov  2 2009, 15:45:00)
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> d = {"a": 1, "b": 2}
>>> dir(d)
['__class__', '__contains__', '__delattr__', '__delitem__', '__doc__', 
'__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', 
'__gt__', '__hash__', '__init__', '__iter__', '__le__', '__len__', '__lt__', 
'__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', 
'__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', 
'clear', 'copy', 'fromkeys', 'get', 'items', 'keys', 'pop', 'popitem', 
'setdefault', 'update', 'values']

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] role playing game - help needed

[Peter Otten]

Al Stern wrote:

> I used the following code and got the following error.

The result of input is always a string. 
 
> attributes["strength"] = input("\nHow many points do you want to assign to
> strength?: ")

Say you type 42 when you run your script. Then the above assignment is 
effectively

attributes["strength"] = "42"

and when you loop over the values you try to add a string to an integer 
which is what Python complains about:

>>> 42 + "42"
Traceback (most recent call last):
  File "", line 1, in 
TypeError: unsupported operand type(s) for +: 'int' and 'str'

To get an integer you have to convert the string explicitly:

>>> 42 + int("42")
84

The best place to do that is as early as possible, even before you put the 
value into the dictionary:

attributes["strength"] = int(input(...))

(In a real application you'd guard against values that cannot be converted 
to integers)

> #point allocation
> point_total = 0
> for val in values:
> point_total += val
> print (point_total)
> 
> and get this error...
> 
> Traceback (most recent call last):
> File "C:\Users\Public\Documents\My Python programs\role_playing_game1.py",
> line 26, in 
> point_total += val
> TypeError: unsupported operand type(s) for +=: 'int' and 'str'


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Code evaluation inside of string fails with __get_item

[Peter Otten]

Tim Johnson wrote:

> This is a resend. I note that the original had an incorrect
> `reply-to' ID attached to it. (sorry)
> --
> I'm using Python 2.6.5.
> The following problem is coming from inside of a complex code base
> and involves an implementation that I have used for years, and is
> now failing to execute in certain conditions.
> This problem occurs with either of the follow two classes, which are
> 'lifted' from 'Python Cookbook'. Class code follows:
> class Eval:

> def __getitem__(self, key):

> Under certain circumstances, the embedded is code *not* executed.

The first step is to concentrate on just one class, at one point in the 
code. Then try to control the input to the method and find an argument where 
the code reproducibly fails.

> By inserting debugging stubs, I can see that the the Eval/Evalx
> instantiation does occur, but the overloaded function call to __get_item
> does *not* occur. I have also found that a seemingly unrelated event
> having to do with file I/O must be causing a side effect. However, to keep
> things simple, I am first asking the following question:
> What would cause __get_item__ not to be called? I can confirm by

Nothing, provided 

(1) the method is spelt __getitem__ (two leading and two trailing 
underscores)

(2) the left side is a python string with legal "%(...)s"-style format 
expressions. Given a format string

s = "%(s.upper())s" 

try to feed it a real dict

print s % {"s.upper()":"OK") # should print OK

to verify that that precondition is met.

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Generating a python file

[Peter Otten]

C.T. Matsumoto wrote:

> Is it possible to create files containing python code in the same sort
> of way that you can generate text files.
> 
> A simple example perhaps could be a persistent dictionary. Keys and
> values are written to the dictionary in a file, that can be imported
> later.

For simple human-readable dictionaries the established format is json, see

http://docs.python.org/library/json.htm

json files look similar to python dictionaries, but there are libraries to 
read or write them for other programming languages, too.

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Generating a python file

[Peter Otten]

Evans Anyokwu wrote:

> The page you linked to above was not found.
> Could you check the link again -

Sorry, I accidentally stripped off the trailing 'l' during cut-and-paste. 
The correct link is

http://docs.python.org/library/json.html

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] pyodbc/date values in MS Access

[Peter Otten]

Albert-Jan Roskam wrote:

> Hi,
> 
> I'm using pyodbc (Python 2.5) to insert records in an MS Access database.
> For security reasons, question marks should be used for string replacement
> [*]. The standard %s would make the code vulnerable to sql code injection.
> Problem is, string replacement in the Good Way somehow doesn't work when
> the values are dates. Below, snippet #1 does not work (Access says the
> inserted value is not consistent with the defined datatype), but #2 does.
> I tried various other ways (ie. DateValue, CDate, etc.) but none of them
> works. Is there a solution for this?
> 
> [*] see http://code.google.com/p/pyodbc/wiki/GettingStarted --> under
> 'Parameters'
> 
> ### 1
> sql = "INSERT INTO tblSomeTable (myDate) VALUES (?);"
> cursor.execute(sql, "#01/01/2010#")


(1) Try providing the date in ISO format "-mm-dd"

"2010-01-01"

or (even better if supported) as a date value

from datetime import date
date(2010, 1, 1)

(2) Wrap the value into a tuple which I think is required by the Python 
DBAPI.

cursor.execute(sql, ("2010-01-01",))
cursor.execute(sql, (date(2010, 1, 1),))

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Parameterized Queries failing on MySQL

[Peter Otten]

Shea Grove wrote:

>> I'm using pyodbc to interact with MS SQL Server and I'm starting to
>> support mysql.  My issue is that when I use a parameterized query, it
>> works for SQL Server, but crashes when I point to MySQL.  

I assume it raise an Exception.

>> Is there a different syntax that I should be using? or module?

MySQLdb uses a different paramstyle "format" (see 
http://www.python.org/dev/peps/pep-0249/ )

>> query='insert into RestaurantTable values (?, ?, ?, ?, ?, ?, ?, ?, ?)'

Make this

query = 'insert into RestaurantTable values (%s, %s, %s, %s, %s, %s, %s, %s, 
%s)'

>> mcursor.execute(query, params)

but *never* change the above to

mcursor.execute(query % params) #WRONG

which unfortunately will work most of the time with that paramstyle while 
making your app vulnerable to sql injection as demonstrated here: 
http://xkcd.com/327/

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to import python dictionary into MySQL table?

[Peter Otten]

Sean Carolan wrote:

>> I have a database with a table called "systems" that contains an
>> auto-increment id field, as well as fields for each of the keys in
>> mydata.keys().  But I can't seem to get the syntax to import
>> mydata.values() into the table.  I think the problem may be that some
>> of the items in my list are dictionaries or lists...
>>
>> What would be the quickest and most straightforward way to do this?
> 
> I got this working in case anyone else comes across the same problem.
> This function will pull cobbler *.json data into a MySQL database
> table.  The function assumes that you already have a database set up,
> and that you are dumping the data into the "systems" table:
> 
> def importJSON(targetfile):
> '''
> Imports JSON data from targetfile into MySQL database table.
> '''
> value_list = []
> rawdata = json.load(open(targetfile))
> for key in rawdata.keys():
> strvalue = str(rawdata[key]).replace("'",'"')

Is the above line really needed?

> value_list.append(strvalue)
> valtuple = tuple(value_list)
> sql = "INSERT INTO systems (comment, kickstart,
> name_servers_search, ks_meta, kernel_options_post, image,
> redhat_management_key, power_type, power_user, kernel_options, vi
> rt_file_size, mtime, template_files, gateway, uid, virt_cpus,
> hostname, virt_type, mgmt_classes, power_pass, netboot_enabled,
> profile, virt_bridge, parent, virt_path, interfaces, power_address,
> name_servers, name, owners, ctime, virt_ram, power_id, random_id,
> server, redhat_management_server, depth) VALUES (%s, %s, %s, %s, %s,
> %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s,
> %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s);"
> cursor.execute(sql, valtuple)

The order of key/value pairs in a dictionary is an implementation detail. I 
think it's dangerous to assume they will always be in sync with the column 
names as provided in the string constant.

Here's what I came up with when I tried to make a generalized version of the 
above:

def add_row(cursor, tablename, rowdict):
# XXX tablename not sanitized
# XXX test for allowed keys is case-sensitive

# filter out keys that are not column names
cursor.execute("describe %s" % tablename)
allowed_keys = set(row[0] for row in cursor.fetchall())
keys = allowed_keys.intersection(rowdict)

if len(rowdict) > len(keys):
unknown_keys = set(rowdict) - allowed_keys
print >> sys.stderr, "skipping keys:", ", ".join(unknown_keys)

columns = ", ".join(keys)
values_template = ", ".join(["%s"] * len(keys))

sql = "insert into %s (%s) values (%s)" % (
tablename, columns, values_template)
values = tuple(rowdict[key] for key in keys)
cursor.execute(sql, values)

filename = ...
tablename = ...
db = MySQLdb.connect(...)
cursor = db.cursor()
with open(filename) as instream:
row = json.load(instream)
add_row(cursor, tablename, row)

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] calling setters of superclasses

[Peter Otten]

Gregory, Matthew wrote:

> Hi all,
> 
> Consider the following classes where PositiveX should constrain the
> attribute _x to be positive and SmallX should further constrain the
> attribute _x to be less than 10.
> 
> class PositiveX(object):
> def __init__(self):
> self._x = 1
> @property
> def x(self):
> return self._x
> @x.setter
> def x(self, val):
> if val < 0:
> raise ValueError('Negatives not allowed')
> self._x = val
> 
> class SmallX(PositiveX):
> @property
> def x(self):
> return self._x
> @x.setter
> def x(self, val):
> # How do I call the superclass' @x.setter
> super(SmallX, self).__setattr__('x', val)
> if val > 10:
> raise ValueError('Big values not allowed')
> self._x = val
> 
> I thought I could call the superclass setter first to ensure the value was
> positive, but I'm getting an infinite recursion.  I also tried:
> 
>   super(SmallX, self).x = val
> 
> but I get this:
> 
>   AttributeError: 'super' object has no attribute 'x'
> 
> I'm fully confused and, therefore, likely doing something stupid.

I don't think /how/ you are trying it is stupid though I'm not so sure about 
/what/ .

I didn't get it to work with super() either, so here's Plan B till someone 
is going to enlighten both of us:

class SmallX(PositiveX):
@property
def x(self):
return self._x
@x.setter
def x(self, val):
if val > 10:
raise ValueError('Big values not allowed')
PositiveX.x.__set__(self, val)

Personally, I would more or less follow Alan's advice and do something like

class PositiveX(object):
def __init__(self):
self._x = 1
def check_x(self, val):
if val < 0:
raise ValueError('Negatives not allowed')
@property
def x(self):
return self._x
@x.setter
def x(self, val):
self.check_x(val)
self._x = val

class SmallX(PositiveX):
def check_x(self, val):
super(SmallX, self).check_x(val)
if val > 10:
raise ValueError('Big values not allowed')

Peter



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] doing maths on lists

[Peter Otten]

Dave Angel wrote:

> from math import cos
> 
> JME = 0.4
> L0A = [2.3, 4.65]
> L0B = [1.8, 2.2]
> L0C = [12.1, 4]
> limit = len(L0A)
> 
> L0 = sum(L0A[i]*cos(L0B[i]+L0C[i]*JME) for i in range(limit))

OP, if you are planning to do this "maths on lists" a lot you should have a 
look at numpy. With numpy Dave's snippet can be written as

from numpy import array, cos, sum

j = 0.4
a = array([2.3, 4.65])
b = array([1.8, 2.2])
c = array([12.1, 4])

print sum(a*cos(b+c*j))

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Hangman game.....problem putting strings in a list.....

[Peter Otten]

Yasin Yaqoobi wrote:

> # This is the line that gives me the error don't know why?
> guessed[index] = " " + (letter); ,TypeError: 'str' object does not
> support item assignment

I don't get this far because I run into

Traceback (most recent call last):
  File "hangman.py", line 69, in 
line = file.readline();
TypeError: descriptor 'readline' of 'file' object needs an argument

In the future please copy and paste the code you are actually running.
However:

> def initArray():
>  global guessed
>  print line
   print guessed
>  guessed =  guessed[0] * (len(line)-1)
   print guessed
>  print "this is new list " + guessed;

If you add these two print statements to your code you might find out 
yourself what's going wrong.

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] scraping and saving in file

[Peter Otten]

Tommy Kaas wrote:

> I’m trying to learn basic web scraping and starting from scratch. I’m
> using Activepython 2.6.6

> I have uploaded a simple table on my web page and try to scrape it and
> will save the result in a text file. I will separate the columns in the
> file with
> #.
 
> It works fine but besides # I also get spaces between the columns in the
> text file. How do I avoid that?

> This is the script:

> import urllib2
> from BeautifulSoup import BeautifulSoup
> f = open('tabeltest.txt', 'w')
> soup = 
BeautifulSoup(urllib2.urlopen('http://www.kaasogmulvad.dk/unv/python/tabelte
> st.htm').read())
 
> rows = soup.findAll('tr')

> for tr in rows:
> cols = tr.findAll('td')
> print >> f,
> cols[0].string,'#',cols[1].string,'#',cols[2].string,'#',cols[3].string
> 
> f.close()

> And the text file looks like this:

> Kommunenr # Kommune # Region # Regionsnr
> 101 # København # Hovedstaden # 1084
> 147 # Frederiksberg # Hovedstaden # 1084
> 151 # Ballerup # Hovedstaden # 1084
> 153 # Brøndby # Hovedstaden # 1084

The print statement automatically inserts spaces, so you can either resort 
to the write method

for i in range(4):
if i:
f.write("#")
f.write(cols[i].string)

which is a bit clumsy, or you build the complete line and then print it as a 
whole:

print >> f, "#".join(col.string for col in cols)

Note that you have non-ascii characters in your data -- I'm surprised that 
writing to a file works for you. I would expect that

import codecs
f = codecs.open("tmp.txt", "w", encoding="utf-8")

is needed to successfully write your data to a file

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] scraping and saving in file

[Peter Otten]

Tommy Kaas wrote:

> Steven D'Aprano wrote:
>> But in your case, the best way is not to use print at all. You are
>> writing
> to a
>> file -- write to the file directly, don't mess about with print.
>> Untested:
>> 
>> 
>> f = open('tabeltest.txt', 'w')
>> url = 'http://www.kaasogmulvad.dk/unv/python/tabeltest.htm'
>> soup = BeautifulSoup(urllib2.urlopen(url).read())
>> rows = soup.findAll('tr')
>> for tr in rows:
>>  cols = tr.findAll('td')
>>  output = "#".join(cols[i].string for i in (0, 1, 2, 3))
>>  f.write(output + '\n')  # don't forget the newline after each row
>> f.close()
> 
> Steven, thanks for the advice.
> I see the point. But now I have problems with the Danish characters. I get
> this:
> 
> Traceback (most recent call last):
>   File "C:/pythonlib/kursus/kommuner-regioner_ny.py", line 36, in 
> f.write(output + '\n')  # don't forget the newline after each row
> UnicodeEncodeError: 'ascii' codec can't encode character u'\xf8' in
> position 5: ordinal not in range(128)
> 
> I have tried to add # -*- coding: utf-8 -*- to the top of the script, but
> It doesn't help?

The coding cookie only affects unicode string constants in the source code, 
it doesn't change how the unicode data coming from BeautifulSoup is handled.
As I suspected in my other post you have to convert your data to a specific 
encoding (I use UTF-8 below) before you can write it to a file:

import urllib2 
import codecs
from BeautifulSoup import BeautifulSoup 

html = urllib2.urlopen(
'http://www.kaasogmulvad.dk/unv/python/tabeltest.htm').read()
soup = BeautifulSoup(html)

with codecs.open('tabeltest.txt', "w", encoding="utf-8") as f:
rows = soup.findAll('tr')
for tr in rows:
cols = tr.findAll('td')
print >> f, "#".join(col.string for col in cols)

The with statement implicitly closes the file, so you can avoid f.close() at 
the end of the script.

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] scraping and saving in file SOLVED

[Peter Otten]

Tommy Kaas wrote:

> With Stevens help about writing and Peters help about import codecs - and
> when I used \r\n instead of \r to give me new lines everything worked. I
> just thought that \n would be necessary? Thanks.
> Tommy

Newline handling varies across operating systems. If you are on Windows and 
open a file in text mode your program sees plain "\n",  but the data stored 
on disk is "\r\n". Most other OSes don't mess with newlines.

If you always want "\r\n" you can rely on the csv module to write your data, 
but the drawback is that you have to encode the strings manually:

import csv
import urllib2 
from BeautifulSoup import BeautifulSoup 

html = urllib2.urlopen(
'http://www.kaasogmulvad.dk/unv/python/tabeltest.htm').read()
soup = BeautifulSoup(html)

with open('tabeltest.txt', "wb") as f:
writer = csv.writer(f, delimiter="#")
rows = soup.findAll('tr')
for tr in rows:
cols = tr.findAll('td')
writer.writerow([unicode(col.string).encode("utf-8")
 for col in cols])

PS: It took me some time to figure out how deal with beautifulsoup's flavour 
of unicode:

>>> import BeautifulSoup as bs
>>> s = bs.NavigableString(u"älpha")
>>> s
u'\xe4lpha'
>>> s.encode("utf-8")
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/pymodules/python2.6/BeautifulSoup.py", line 430, in encode
return self.decode().encode(encoding)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in position 
0: ordinal not in range(128)
>>> unicode(s).encode("utf-8") # heureka
'\xc3\xa4lpha'


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Problem installing pyaudiere

[Peter Otten]

Danci Emanuel wrote:

> I am trying to install the pyaudiere package on linux and I get the
> following error: " AudioDevice.h:6: fatal error: audiere.h: No such file
> or directorycompilation terminated. " I am interested if anyone managed to
> solve this problem or if someone could suggest a package that could
> replace pyaudiere.

[Just gueessing]

If you want to install pyaudiere from source you probably need the 
development package for the underlying library. You may be able to install 
that with

sudo apt-get install libaudiere-dev

or the equivalent command and package for your distribution.

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Checkbox problem in Tkinter

[Peter Otten]

ANKUR AGGARWAL wrote:

> and i got the output as -
> [image: Screenshot.png]

The image doesn't survive.
 
> When i take the var as of string variable type i am  unable to edit the
> checkbox. It comes slected predefined and the widget in kindof Invisible
> and
> u can say uneditable. Can anybody tell me whats i am  doing wrong in
> this??? Its works fine if i take the var variable as the integer variable.
> I can deselect and select in that case and it gives me the desired result
> but in string case i am unable to do so as checkbox is invisible type.
> I am using Python 2.6

I can't reproduce that behaviour. The widget appears greyed initially, but 
is still editible. You could try to initialise the StringVar with onvalue or 
offvalue:

> from Tkinter import *
> root=Tk()
> var=StringVar()

  var.set("")

> c=Checkbutton(root,text="hello",variable=var,onvalue="",
> offvalue="")
> c.pack()
> root.mainloop()


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Checkbox problem in Tkinter

[Peter Otten]

Alan Gauld wrote:

> 
> "ANKUR AGGARWAL"  wrote
> 
>> from Tkinter import *
>> root=Tk()
>> var=StringVar()
>> 
c=Checkbutton(root,text="hello",variable=var,onvalue="",offvalue="")
>> c.pack()
>> root.mainloop()
> 
> FWIW I get a different problem, namely that the var does not seem
> to get updated nor does changling the var seem to alter the buuttons
> state. But at least the button displays OK with 'hello' in the label
> and
> I can select/deselect it with the mouse. its just the change to the
> variable that is broken.
> If I write an explicit event handler:
> 
> def f():
>   var = '' if var == '' else ''
> 
> Then var changes OK. Here is my code(from Pythonwin):
> 
 def f(): global var; var = 'on' if var == 'off' else 'off';print
 var
> ...
 top = Tk()
 var = StringVar()
 c = Checkbutton(top,text='txt', variable=var, onvalue='on',
 offvalue='off', command=f)
 c.pack()
 top.mainloop()
> off
> on
> off
> on
> off
> on

I think that is confusing. Tkinter is not supposed to rebind the global name 
'var', it invokes its set() method. Try

from Tkinter import *

root = Tk()
var = StringVar()

def command():
print var.get()

cb = Checkbutton(root, text="yadda", onvalue="on", offvalue="off", 
variable=var, command=command)
cb.pack()
root.mainloop()

You can also monitor changes to var's value directly:

from Tkinter import *

root = Tk()
var = StringVar()

def show(*args):
print var.get()
var.trace_variable('w', show)
cb = Checkbutton(root, text="yadda", onvalue="on", offvalue="off", 
variable=var)
cb.pack()
root.mainloop()



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Checkbox problem in Tkinter

[Peter Otten]

ANKUR AGGARWAL wrote:

> Hey I write this code up
> 
> from Tkinter import *
> root=Tk()
> var=StringVar()
> 
c=Checkbutton(root,text="hello",variable=var,onvalue="",offvalue="")
> c.pack()
> root.mainloop()
> 
> and i got the output as -
> [image: Screenshot.png]
> 
> When i take the var as of string variable type i am  unable to edit the
> checkbox. It comes slected predefined and the widget in kindof Invisible
> and
> u can say uneditable. Can anybody tell me whats i am  doing wrong in
> this??? Its works fine if i take the var variable as the integer variable.
> I can deselect and select in that case and it gives me the desired result
> but in string case i am unable to do so as checkbox is invisible type.
> I am using Python 2.6
> Thanks In Advance
> Ankur Aggarwal

The variable appears greyed out because initially var.get() == "", see

http://www.tcl.tk/man/tcl8.5/TkCmd/checkbutton.htm#M-tristatevalue


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Sorting a List

[Peter Otten]

Bill Campbell wrote:

> On Wed, Jan 12, 2011, Corey Richardson wrote:
>>Hello Tutors,
>>
>>I am generating XML definitions for animations to be used in a
>>FIFE-based game. I need to sort the frames of the animations, and I am
>>currently using:
>>sorted([image for image in os.listdir(path) if image.endswith('.png')])
>>
>>The final output in the XML is:
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>Having frame 10 and 11 between frame 1 and 2 is not desired behavior;
>>how can I sort all of these with walk_10.png and company (this isn't the
>>only animation like this) being after walk_9.png? A google search
>>doesn't return anything I could use. I tried just using
>>[image for image in os.listdir(path) if image.endswith('.png')],
>>but that doesn't appear to have any order.
> 
> This is discussed in the Python Cookbook from O'Reilly.  If I
> remember correctly, it deals exactly with the problem of sorting
> file names containing numbers as you have above.
> 
> Here's a bit of code I use in a module I wrote for dealing with
> RPM versions.
> 
> import re
> _digits = re.compile(r'(\d+)')
> 
> def ver(s):
> r = _digits.split(s)
> r[1::2] = map(lambda x: int(x), r[1::2])
> return(tuple(r))

Instead of map(lambda x: int(x), ...)
just use map(int, ...)

> class FileInfo(object):
> def __init__(self, fname)
> self.fname = fname
> self.cmp = ver(fname)
> 
> def __cmp__(self, othr):
> return cmp(self.cmp, othr.cmp)

You don't need the FileInfo object, you can make ver() the key argument to 
list.sort() or sorted():

sorted(filenames, key=ver)

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

[Tutor] Determinw Tkinter checkbox state, was Re: question

[Peter Otten]

Stinson, Wynn A Civ USAF AFMC 556 SMXS/MXDED wrote:

[In the future please take the time to choose a meaningful subject]

> Can someone give me some sample code to use to determine if a checkbox
> has been selected using Tkinter? thanks

Method 1: check the state of the underlying variable:

import Tkinter as tk

root = tk.Tk()

var = tk.IntVar()
cb = tk.Checkbutton(root, text="the lights are on", variable=var)
cb.pack()

def showstate():
if var.get():
print "the lights are on"
else:
print "the lights are off"

button = tk.Button(root, text="show state", command=showstate)
button.pack()

root.mainloop()

Method 2: trigger a function when the underlying variable changes

import Tkinter as tk

root = tk.Tk()

var = tk.IntVar()
cb = tk.Checkbutton(root, text="the lights are on", variable=var)
cb.pack()

def showstate(*args):
if var.get():
print "the lights are on"
else:
print "the lights are off"

var.trace_variable("w", showstate)
root.mainloop()


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] end parameter in 2.7.1?

[Peter Otten]

Bill DeBroglie wrote:

> Twice in two days...!
> 
> Using Mac OS X 10.5.8 and Python 2.7.1 but am following a book which
> is using Python 3.1. The author uses the end parameter in numerous
> programs but this doesn't seem to translate to 2.7. Any advice as to
> how I can specify the final string of the print function?

You can trick 2.7 into behaving similar to 3.x in that respect with a from 
__future__ import:

Python 2.7.1 (r271:86832, Jan  5 2011, 10:42:58)
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from __future__ import print_function
>>> print("you are asking for trouble", end="!\n")
you are asking for trouble!

However, I recommend that you install Python 3.1 in addition to 2.7 and use 
3.1 to learn the language. If you don't you'll run into a lot of tiny but 
frustrating differences.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Is it possible to tell, from which class an method was inherited from

[Peter Otten]

Jojo Mwebaze wrote:

> Is it possible to tell, from which class an method was inherited from.
> take an example below
> 
> class A:
>def foo():
>  pass
> class B(A):
>def boo(A):
>  pass
> class C(B):
>def coo()
>  pass
> class D(C):
>def doo()
>   pass
> 
 dir (D)
> ['__doc__', '__module__', 'boo', 'coo', 'doo', 'foo']
> 
> Is there any method to tell me form which classes boo, coo, foo where
> inherited from?

You can check the classes in method resolution order (mro):


import inspect

class A:
def foo(self):
pass
def bar(self):
pass

class B(A):
def bar(self):
pass

class C(B):
def baz(self):
pass

for methodname in dir(C) + ["spam"]:
for class_ in inspect.getmro(C):
if methodname in class_.__dict__:
print "%s.%s()" % (class_.__name__, methodname)
break
else:
print "no attribute named %r found" % methodname


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Decoding from strange symbols

[Peter Otten]

Oleg Oltar wrote:

> I am trying to decode a string I took from file:
> 
> file = open ("./Downloads/lamp-post.csv", 'r')
> data = file.readlines()
> data[0]
> 
> 
'\xff\xfeK\x00e\x00y\x00w\x00o\x00r\x00d\x00\t\x00C\x00o\x00m\x00p\x00e\x00t\x00i\x00t\x00i\x00o\x00n\x00\t\x00G\x00l\x00o\x00b\x00a\x00l\x00

> How do I convert this to something human readable?

If you stare at it long enough you'll see the usual ascii characters 
interspersed with zero-bytes shown by Python as "\x00".

This is an UTF-16 file. Open it with

import codecs
filename = "./Downloads/lamp-post.csv"
with codecs.open(filename, "r", encoding="utf-16") as file:
   for line in file:
   print line

Note that 'line' will now contain a unicode string instead of a byte string. 
If you want to write that to a file you have to encode it manually

line = u"äöü"
with open("tmp.txt", "w") as f:
f.write(line.encode("utf-8"))

or use codecs.open() again:

with codecs.open("tmp.txt", "w", encoding="utf-8") as f:
f.write(line)

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Is it possible to tell, from which class an method was inherited from

[Peter Otten]

Jojo Mwebaze wrote:

> Thanks guys for the responses,
> 
> inspect.classify_class_attrs(klass)
> 
> does the magic

Argh, undocumented functions. How did you find that gem?

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] not understanding a recursion example

[Peter Otten]

Bill Allen wrote:

> I am not understanding the following code (I did not write it).  It
> demonstrates walking a tree-like data structure using recursion.  It does
> run and produces reasonable output.  I particularly do not understand the
> "traverse.level" statements.  Can anyone give me an idea how this is
> working
> and the principles?  I would like understand recursive calls in Python
> better as I have not used the technique previously.

> def traverse(data):
> print(' ' * traverse.level + data['text'])
> for kid in data['kids']:
> traverse.level += 1
> traverse(kid)
> traverse.level -= 1
> 
> if __name__ == '__main__':
> traverse.level = 1
> traverse(data)


Functions are objects in python; you can tuck arbitrary attributes onto 
them, and the above uses a function attribute instead of a global variable.
A more standard way to write the above would be

def traverse(data):
global level
print(' ' * level + data['text'])
for kid in data['kids']:
level += 1
traverse(kid)
level -= 1

if __name__ == '__main__':
level = 1
traverse(data)

What it does: 
* call traverse with the outermost dictionary
* print the data["text"] value indented by the current global level.
* iterate over the data["kids"] list of dictionaries
* for each entry in that list
- increment indentation level
- invoke traverse which will print the data["text"] value.
  (Remember that traverse was called with the current value of kid, so
   in terms of the outer traverse() the inner traverse() is printing
   kid["text"]) Process the kid's kids in the same way.
- decrement the indentation level

However using global variables is generally a bad practice. 
It is easy to leave them in an inconsistent state, and if you are using 
multiple threads (i. e. invoke traverse() a second time while the first call 
hasn't finished) you'll end up with a big mess.

I would therefore write the traverse function as

def traverse(data, level):
print(' ' * level + data['text'])
for kid in data['kids']:
traverse(kid, level+1)

if __name__ == "__main__":
traverse(data, 1)

which I think may also be easier to understand.

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] ascii codec cannot encode character

[Peter Otten]

Alex Hall wrote:

> Hello again:
> I have never seen this message before. I am pulling xml from a site's
> api and printing it, testing the wrapper I am writing for the api. I
> have never seen this error until just now, in the twelfth result of my
> search:
> UnicodeEncodeError: 'ASCII' codec can't encode character u'\u2019' in
> position 42: ordinal not in range(128)
> 
> I tried making the strings Unicode by saying something like
> self.title=unicode(data.find("title").text)
> but the same error appeared. I found the manual chapter on this, but I
> am not sure I want to ignore since I do not know what this character
> (or others) might mean in the string. I am not clear on what 'replace'
> will do. Any suggestions?

You get a UnicodeEncodeError if you print a unicode string containing non-
ascii characters, and Python cannot determine the target's encoding:

$ cat tmp.py
# -*- coding: utf-8 -*-
print u'äöü'

$ python tmp.py
äöü
$ python tmp.py > tmp.txt
Traceback (most recent call last):
  File "tmp.py", line 2, in 
print u'äöü'
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-2: 
ordinal not in range(128)

The error occurs because by default Python 2 tries to convert unicode into 
bytes using the ascii codec.

One approach to tackle this is to check sys.stdout's encoding, and if it's 
unknown (None) wrap it into a codecs.Writer that can handle all characters 
that may occur. UTF-8 is usually a good choice, but other codecs are 
possible.

$ cat tmp2.py
# -*- coding: utf-8 -*-
import sys

if sys.stdout.encoding is None:
import codecs
Writer = codecs.getwriter("utf-8")
sys.stdout = Writer(sys.stdout)

print u'äöü'
$ python tmp2.py
äöü
$ python tmp2.py > tmp.txt
$ cat tmp.txt
äöü


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Wrapping my head around global variables!!

[Peter Otten]

Nevins Duret wrote:

> Hello Python collective,
> 
>  I am trying to wrap my head around what exactly is causing me
> not to get any output or error message in the following code:
> 
>> #!/usr/bin/env python3.1
>>
>> import random
>>
>> def main():
>>
>> def chosen_letter():
>>
>> chosen_letter = Consonant()
>> chosen_letter = Vowel()
>>
>> return chosen_letter

Hint: what function are you leaving with the above return statement?

I won't spoil you the fun of tackling the other problems with your code, one 
at a time. Come back here for those you can't handle yourself.

>> if __name__ == "__main__": main()


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] RE module is working ?

[Peter Otten]

Karim wrote:

> I am trying to subsitute a '""' pattern in '\"\"' namely escape 2
> consecutives double quotes:
> 
> * *In Python interpreter:*
> 
> $ python
> Python 2.7.1rc1 (r271rc1:86455, Nov 16 2010, 21:53:40)
> [GCC 4.4.3] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>  >>> expression = *' "" '*
>  >>> re.subn(*r'([^\\])?"', r'\1\\"', expression*)
> Traceback (most recent call last):
>File "", line 1, in 
>File "/home/karim/build/python/install/lib/python2.7/re.py", line
> 162, in subn
>  return _compile(pattern, flags).subn(repl, string, count)
>File "/home/karim/build/python/install/lib/python2.7/re.py", line
> 278, in filter
>  return sre_parse.expand_template(template, match)
>File "/home/karim/build/python/install/lib/python2.7/sre_parse.py",
> line 787, in expand_template
>  raise error, "unmatched group"
> sre_constants.error: unmatched group
> 
> But if I remove '?' I get the following:
> 
>  >>> re.subn(r'([^\\])"', r'\1\\"', expression)
> (' \\"" ', 1)
> 
> Only one substitution..._But this is not the same REGEX._ And the
> count=2 does nothing. By default all occurrence shoul be substituted.
> 
> * *On linux using my good old sed command, it is working with my '?'
>   (0-1 match):*
> 
> *$* echo *' "" '* | sed *'s/\([^\\]\)\?"/\1\\"/g*'*
>   \"\"
> 
> *Indeed what's the matter with RE module!?*

You should really fix the problem with your email program first; afterwards 
it's probably a good idea to try and explain your goal clearly, in plain 
English.

Yes. What Steven said ;)

Now to your question as stated: if you want to escape two consecutive double 
quotes that can be done with

s = s.replace('""', '\"\"')

but that's probably *not* what you want. Assuming you want to escape two 
consecutive double quotes and make sure that the first one isn't already 
escaped, this is my attempt:

>>> def sub(m):
... s = m.group()
... return r'\"\"' if s == '""' else s
...
>>> print re.compile(r'[\\].|""').sub(sub, r'\\\"" \\"" \"" "" \\\" \\" \"')
\\\"" \\\"\" \"" \"\" \\\" \\" \"

Compare that with

$ echo '\\\"" \\"" \"" "" \\\" \\" \"' | sed 's/\([^\\]\)\?"/\1\\"/g'
\\\"\" \\"\" \"\" \"\" " \\\" \\"

Concerning the exception and the discrepancy between sed and python's re, I 
suggest that you ask it again on comp.lang.python aka the python-list 
mailing list where at least one regex guru will read it.

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] RE module is working ?

[Peter Otten]

Karim wrote:

> Recall:
> 
>  >>> re.subn(r'([^\\])?"', r'\1\\"', expression)
> 
> Traceback (most recent call last):
>  File "", line 1, in
>  File "/home/karim/build/python/install/lib/python2.7/re.py", line
> 162, in subn
>return _compile(pattern, flags).subn(repl, string, count)
>  File "/home/karim/build/python/install/lib/python2.7/re.py", line
> 278, in filter
>return sre_parse.expand_template(template, match)
>  File "/home/karim/build/python/install/lib/python2.7/sre_parse.py",
> line 787, in expand_template
>raise error, "unmatched group"
> sre_constants.error: unmatched group
> 
> 
> Found the solution: '?' needs to be inside parenthesis (saved pattern)
> because outside we don't know if the saved match argument
> will exist or not namely '\1'.
> 
>  >>> re.subn(r'([^\\]?)"', r'\1\\"', expression)
> 
> (' \\"\\" ', 2)
> 
> sed unix command is more permissive: sed 's/\([^\\]\)\?"/\1\\"/g'
> because '?' can be outside parenthesis (saved pattern but escaped for
> sed). \1 seems to not cause issue when matching is found. Perhaps it is
> created only when match occurs.

Thanks for reporting the explanation.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] RE module is working ?

[Peter Otten]

Karim wrote:

> That is not the thing I want. I want to escape any " which are not
> already escaped.
> The sed regex  '/\([^\\]\)\?"/\1\\"/g' is exactly what I need (I have
> made regex on unix since 15 years).

Can the backslash be escaped, too? If so I don't think your regex does what 
you think it does.

r'\\\"' # escaped \ followed by escaped "

should not be altered, but:

$ echo '\\\"' | sed 's/\([^\\]\)\?"/\1\\"/g'
" # two escaped \ folloed by a " that is not escaped



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] making object iterable

[Peter Otten]

Alex Hall wrote:

> Hi all,
> I have a class which has a list as an attribute, meaning you must say
> something like:
> for result in obj.results: ...
> I want to make life a bit easier and let users just say:
> for result in obj: ...
> Here is what I have tried to make my class into an iterable one, yet I
> get an error saying that the class does not support indexing:

Please remember to always cut-and-paste code and traceback.
 
>  def __iter__(self):
>   #makes this class indexable instead of having to use its "results" list
>   return self.iterate()
> 
>  def iterate(self):
>   i=0
>   while iyield self.results[i]
>i+=1
> 
> I am not sure why this does not work, unless I am misunderstanding
> something about iterators and iterables? I thought an iterable, which
> is what I am shooting for, was an iterator on steroids, allowing
> indexing, multiple passes over the list, and all that. Do I want to
> make this an iterator instead of an iterable?

It *does* work:

>>> class A(object):
... def __init__(self, results):
... self.results = results
... def __iter__(self):
... return self.iterate()
... def iterate(self):
... i = 0
... while i < len(self.results):
... yield self.results[i]
... i += 1
...
>>> a = A("abc")
>>> for item in a:
... print item
...
a
b
c

Perhaps self.results is not what you think it is. Check by adding the 
apprpriate print statement.

By the way, there are simpler alternatives to delegate iteration to an 
attribute:

>>> class B(object):
... def __init__(self, results):
... self.results = results
... def __iter__(self):
... return iter(self.results)
...
>>> b = B("abc")
>>> for item in b:
... print item
...
a
b
c


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Splitting a string

[Peter Otten]

Christian Witts wrote:

> On 08/02/2011 15:04, tee chwee liong wrote:
>> hi all,
>>
>> thanks for the advice. i modified my code to be:
>>
>> c=('01101')
>> i=-1
>> try:
>> while 1:
>> i=c.index('0',i+1)
>> print "Lane fail",i
>> except ValueError:
>> print "All Lanes PASS"
>> pass
>>
>> when i run, the result is:
>>
>> >>>
>> Lane fail 0
>> Lane fail 3
>> All Lanes PASS
>>
>> Question: How do i prevent the code from executing the except
>> ValueError part. it seems to be checking bit by bit and when is see 1
>> it will execute the except part.
>> i just want the results to be:
>> >>>
>> Lane fail 0
>> Lane fail 3

> `while i < len(c)` instead of `while 1`

You have an off-by-one bug. But even if you fix that you'll still enter the 
except suite if the string c doesn't end with a "0". You need to keep track 
of the failed lanes, e. g.: 

for c in "011010", "111", "000", "", "1", "0", "001":
print c.center(20, "-")

i = -1
all_passed = True
try:
while True:
i = c.index('0', i+1)
print "Lane fail", i
all_passed = False
except ValueError:
pass
if all_passed:
print "All Lanes PASS"


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Odd behavior with eval, list comps, and name lookup

[Peter Otten]

John wrote:

> I noticed some odd behavior relating to eval(). First, a baseline case for
> behavior:
> 
 def test():
> ... x = 5
> ... return [a for a in range(10) if a == x]
> ...
 test()
> [5]
> 
> So far so good. Now let's try eval:
> 
 c = compile('[a for a in range(10) if a == x]', '', 'single')
 eval(c, globals(), {'x': 5})
> Traceback (most recent call last):
> File "", line 1, in 
> File "", line 1, in 
> File "", line 1, in 
> NameError: global name 'x' is not defined
> 
> Looks like 'x' is searched for only among the globals, bypassing the
> locals. The same behavior is seen between exec and eval, with and without
> compile, and between 3.1.3 and 3.2rc2. Given simpler code without a list
> comp, the locals are used just fine:
> 
 c = compile('a=5; a==x', '', 'single')
 eval(c, {}, {'x': 5})
> True
> 
> Could anyone help me understand these scoping rules? Thanks.

Except for class definitions the compiler statically determines whether a 
variable is global or local (or a closure).
List comprehensions and generator expressions are realized as functions; 
therefore they have their own local scope that you cannot provide via 
eval/exec. 

You could argue that in the following example

>>> exec("x = 2; print([a for a in range(3) if a == x])", {}, {})
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 1, in 
  File "", line 1, in 
NameError: global name 'x' is not defined

the assignment x = 2 should give the compiler a clue that x is supposed to 
be a local name, just like in

def f():
   x = 2
   print([a for a in range(3) if a == x])

My *guess* is that this is an implementation restriction rather than a 
conscious decision. It works for the common case of module-level code 
because there local and global namespace are identical:

>>> exec("x = 2; print([a for a in range(3) if a == x])", {})
[2]

Peter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] reading large text file as numpy array

[Peter Otten]

Jaidev Deshpande wrote:

> I have a large text file with more than 50k lines and about 784 floats in
> each line.
> 
> How can I read the whole file as a single numpy array?

Have you tried numpy.loadtxt()?


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python object

[Peter Otten]

Christopher Brookes wrote:

> Hi, i'm new in python.
> I'm trying to create a small fight program in object.
> 
> I've created __init__ (its works) but when i'm trying to display init
> param i'm getting param and "None" every time. Why ?
> 
> def GetAllAtrib(self):
> print '---'
> print self.name
> print self.description
> print self.type
> print '---'
> 
> give ->>
> 
> ---
> Klaitos
> Soldier very strong
> Soldier
> ---
> *None   *-- WHY ARE U HERE

My crystall ball says: because you invoke the above method with something 
like

print soldier.GetAllAttrib()

instead of just

soldier.GetAllAttrib()

If I'm guessing wrong please provide the relevant source code.
By the way, show_all_attributes() would be a better name for your method as 
it would make the problem in your code obvious.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to vectorize a constant function?

[Peter Otten]

Jose Amoreira wrote:

> I am trying to define a constant vectorized function, that is, one that
> returns 1. (for instance) when called with a scalar argument x and that
> returns array([1.,1.,1.]) when the argument is a three-element array, etc.

With the help of google I found

http://docs.scipy.org/doc/numpy/reference/generated/numpy.frompyfunc.html

>>> import numpy
>>> def f(x): return 42
...
>>> f = numpy.frompyfunc(f, 1, 1)
>>> f(1)
42
>>> f(numpy.zeros(7))
array([42, 42, 42, 42, 42, 42, 42], dtype=object)


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Code structure help

[Peter Otten]

Martin De Kauwe wrote:

> Note I have cross posted this as I have only just found this mailing list
> and perhaps it is the more appropriate place (I am not sure)?

I think your question is appropriate for both lists, it just wasn't sexy 
enough for anyone on c.l.py to answer ;)
 
> I have been working on re-writing a model in python. However I am not sure
> how easy on the eye my final structure is and would appreciate any
> constructive comments/suggestions. So broadly the model estimates how
> plants grow using a number of related sub functions which I have grouped
> into classes and they all live in separate files. My main issue at the
> moment is I think I have a lot of verbose class instances but I really
> can't see a better way to do it. Is there a better way? How do other
> people do similar things? I am talking largely about the instances in the
> method run_sim

Random remarks:

> pg = PlantGrowth(self.control, self.params, self.state,
> self.fluxes, self.met_data)

I'd prefer your code to be even more verbose here; no two-letter variables 
for anything that is non-generic. 

> # plant growth
> pg.grow(project_day, self.date, leafnc)

With the construction in mind that is actually

> # plant growth
> plant_growth.grow(project_day, self.date, leafnc)

but plant_grows.grow() does not make a lot of sense, and the comment is 
superfluous as it just undoes the abbreviation instead of explaining what is 
going on.

> for i in self.met_data.doy:
> project_day = i - 1

> self.increment_date()

I'd iterate over the project_day-s directly if possible

for project_day in self.project_days():
...

> # calculate model decay rates
> dc.decay_rates()

A lot of methods don't take any arguments and return nothing. I'm guessing 
that they modify the state that you passed to the initializer. I prefer 
these modifications to be explicit if feasible, e. g.

 state = dc.decay_rates(state)

where of course state is a placeholder for the actual variables that are 
necessary to do the calculations.

The big picture? I'll leave that for someone else.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] getting the last file in a folder (reliably)

[Peter Otten]

Steven D'Aprano wrote:

> files.sort()
> the_file_name = files[-1]

You don't need to sort if you want only one file:

the_file_name = max(files)


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Recursively flatten the list

[Peter Otten]

Dharmit Shah wrote:

> I am learning Python and yesterday I cam across a definition wherein I was
> supposed to flatten a list recursively. I am getting the solution properly
> but wanted to know if I can optimize the code further.
> 
> #!/usr/bin/env python
> new_list=[]
> def flatten(num_list):
> """
>   >>> flatten([2, 9, [2, 1, 13, 2], 8, [2, 6]])
>   [2, 9, 2, 1, 13, 2, 8, 2, 6]
>   >>> flatten([[9, [7, 1, 13, 2], 8], [7, 6]])
>   [9, 7, 1, 13, 2, 8, 7, 6]
>   >>> flatten([[9, [7, 1, 13, 2], 8], [2, 6]])
>   [9, 7, 1, 13, 2, 8, 2, 6]
>   >>> flatten([[5, [5, [1, 5], 5], 5], [5, 6]])
>   [5, 5, 1, 5, 5, 5, 5, 6]
> """
> global new_list
> for i in num_list:
> if type(i) == type([]):
> new_list = flatten(i)
> else:
> new_list.append(i)
> tmp = new_list
> new_list=[]
> return tmp
> 
> if __name__=="__main__":
> import doctest
> doctest.testmod()
> 
> PS - My knowledge of Python is still very basic and I am trying to dive
> into it as deeper as I can. Solutions on Stackoverflow.com were beyond my
> understandability.

You have a working solution, and you have tests to prove it. That's a good 
start!

Now let's think more tests to break it:

(1) Deeply nested lists:

items = []
for i in range(2000):
items = [items] 
flatten(items)

Python has a limited stacksize, i. e. allows only for a limited number of 
nested function calls. There's not much you can do about that other than 
switch to a non-recursive implementation of flatten(). Most of the time it's 
not a problem in practice; you should just know about it.

(2) Lists that contain themselves:

items = []
items.append(items)
flatten(items)

This one would run forever were it not for the limited stack size. You are  
even less likely to run into that in practice, but it might still be an 
interesting challenge for you to come up with a solution for the problem.

(3) Multithreaded programs:

Because you use a global variable you cannot run two instances of the 
function simultaneously. This problem leads to intermittent bugs that are 
really hard to find. You should try to eliminate the global new_list 
variable from your flatten() function.

Unfortunately the demo is a bit messy, so don't waste too much time on 
trying to understand it. You can revisit it later if you like. Just remember 
the rule of thumb: don't use global variables if you can avoid them.

if __name__=="__main__":
import threading
def set_flattened(num_list, index):
results[index] = flatten(num_list)

A = [5, [5, [1, 5], 5], 5], [5, 6]
B = [[50, [50, [10, 50], 50], 50], [50, 60]]
EXPECTED = [flatten(A), flatten(B)]
print "expected output:"
print EXPECTED
i = 1
results = [None, None]
while True:
# run flatten() in two threads simultaneously...
a = threading.Thread(target=set_flattened, args=(A, 0))
b = threading.Thread(target=set_flattened, args=(B, 1))

for t in a, b:
t.start()
for t in a, b:
t.join()

#... until we see an unexpected result
if results != EXPECTED:
print "The %dth attempt gave this preculiar result:" % i
print results
break
i += 1


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] script conversion windows -> linux error

[Peter Otten]

Rance Hall wrote:

> On Fri, Mar 25, 2011 at 1:54 PM, Walter Prins  wrote:
>>
>> On 25 March 2011 18:26, Rance Hall  wrote:
>>> config_version =  config.get('versions','configver',0)
>>> This line fails under 3.2 Linux with the error message:
>>> TypeError:  get() takes exactly 3 positional arguments (4 given)
>>> What could the 4th argument be?  I only see three.
>>> This same code worked on python 3.1 under windows.

>>> What am I missing?

> Likely explanation.  But I don't think it holds water when compared
> against the configparser documentation.

Speaking of the documentation,  I think that 

http://docs.python.org/py3k/library/configparser.html#configparser.ConfigParser.get

"""
get(section, option, raw=False[, vars, fallback])
[...]
Changed in version 3.2: Arguments raw, vars and fallback are keyword only to 
protect users from trying to use the third argument as the fallback fallback 
(especially when using the mapping protocol).
"""

makes it clear that instead of 

config.get(section, option, rawflag)  # 3.1

you now have to write 

config.get(section, option, raw=rawflag) # 3.2

because many people erroneously assumed that the third argument was the 
fallback value.

Peter


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Problem recognizing '{' character?

[Peter Otten]

Ben Hunter wrote:

> Hi,
> 
> I'm completing the Python lessons on YouTube that Google posted. At the
> end of section 2 of day 2, there is a task to identify files then put them
> in a zip file in any directory. The code is from the 'solution' folder, so
> it's not something I wrote. I suspect I have a problem with PATHS or
> environment variables. I'm new to programming in something as advanced as
> Python, but I do okay with VBA - so I just feel like there's a setting up
> issue somewhere. I'm on Windows 7, tried running this in Idle and from the
> command line.

The commands module used by zip_to() is an unfortunate choice for a tutorial 
because it is supposed to work with unix shells only.

> def zip_to(paths, zipfile):
>   """Zip up all of the given files into a new zip file with the given
> name."""
>   cmd = 'zip -j ' + zipfile + ' ' + ' '.join(paths)
>   print "Command I'm going to do:" + cmd
>   (status, output) = commands.getstatusoutput(cmd)
>   # If command had a problem (status is non-zero),
>   # print its output to stderr and exit.
>   if status:
> sys.stderr.write(output)
> sys.exit(1)

You can either try to install Cygwin to run your script unchanged or rewrite 
the above function to work with subprocess

import sys
from subprocess import Popen, PIPE

def zip_to(paths, zipfile):
command = ["zip", "-j", zipfile]
command.extend(paths)
process = Popen(command, stdout=PIPE, stderr=PIPE)
stdoutdata, stderrdata = process.communicate()
if process.returncode:
sys.stdout.write(stdoutdata)
sys.stderr.write(stderrdata)
sys.exit(1)

You'll still need a zip.exe on your system.
If you're ambitious, have a look at

http://docs.python.org/library/zipfile.html

a library that allows you to create zipfiles in python without the help of 
an external program.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor