Panda data read_csv returns 'TextFileReader' object

[Kurt]

Isn't the call pd.read_csv(filepath,...) suppose to return a dataframe, not 
this other object? I keep getting the following error when I try to view the 
attribute head.

AttributeError: 'TextFileReader' object has no attribute 'head'

I reference pandas as pd and df is suppose to define itself. I had this working 
at one time.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Panda data read_csv returns 'TextFileReader' object

[Kurt]

Python has a return object dependency on its input parameter call. I overlooked 
the fact specifying a "chunksize" parameter in the read_csv function returns a 
TextFileReader object, which is used for iteration purposes. I might need to 
work with that, as well as a ref manual such as Python for Data Analysis.

Kurt



On Friday, April 24, 2015 at 6:53:05 PM UTC-4, Dave Angel wrote:
> On 04/24/2015 04:04 PM, Kurt wrote:
> > Isn't the call pd.read_csv(filepath,...) suppose to return a dataframe, not 
> > this other object? I keep getting the following error when I try to view 
> > the attribute head.
> >
> > AttributeError: 'TextFileReader' object has no attribute 'head'
> >
> > I reference pandas as pd and df is suppose to define itself. I had this 
> > working at one time.
> >
> 
> Please supply the Python version, the pandas version, and the code you 
> ran.  It can also be useful to show how/where you installed pandas, like 
> the URL you used, or the apt-get, or the pip command, or whatever.
> 
> Then show the complete error traceback, not just a summary.  And if the 
> error really is on the line you partially supplied above, what's the 
> type and contents of filepath?  What are the other arguments' values?
> 
> The function I suspect you're trying to call is:
> 
> pandas.io.parsers.read_csv(filepath, ...)
> 
> but I can't tell from your description.
> 
> -- 
> DaveA
-- 
https://mail.python.org/mailman/listinfo/python-list

Functions on list items

[Kurt]

I am trying to process the following calendar and data attributes in a file:
Da Mo Yr AttrA AttrB AttrC...
I need to average AttrA for each of 365 Da days across Yr years. Then do the 
same for 27K files. Repeat for AttrB, AttrC etc.
Can I do the averaging with lists or do I need some Python Db module (if it 
exists)? 

Kurt
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Functions on list items

[Kurt]

> 
> Maybe http://pandas.pydata.org/ ???
> 
> 

Thanks. This reply is like that butterfly wing flap causing hurricanes a world 
away; big steerage away from proprietary stuff used in my org. Detox will take 
some time though.

K
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to detect the character encoding in a web page ?

[Kurt Mueller]

Am 24.12.2012 um 04:03 schrieb iMath:
> but how to let python do it for you ? 
> such as these 2 pages 
> http://python.org/ 
> http://msdn.microsoft.com/en-us/library/bb802962(v=office.12).aspx
> how to  detect the character encoding in these 2 pages  by python ?


If you have the html code, let 
chardetect.py 
do an educated guess for you.

http://pypi.python.org/pypi/chardet

Example:
$ wget -q -O - http://python.org/ | chardetect.py 
stdin: ISO-8859-2 with confidence 0.803579722043
$ 

$ wget -q -O - 
'http://msdn.microsoft.com/en-us/library/bb802962(v=office.12).aspx' | 
chardetect.py 
stdin: utf-8 with confidence 0.87625
$ 


Grüessli
-- 
kurt.alfred.muel...@gmail.com

-- 
http://mail.python.org/mailman/listinfo/python-list

How to modify this script?

[Kurt Hansen]

http://www.tuxradar.com/content/save-time-gedit-snippets:

To convert tab-separated text lines into a HTML-table:

$<
lines = $GEDIT_SELECTED_TEXT.split("\n");
output = '\n';

for line in lines:
output += '';

columns = line.split("\t");
for item in columns:
output += '' + item + ' '

output += '\n';

output += '';
return output
>

I would like to make a small modification (I'm not a programmer myself). 
Let's say I have these lines:


Price table
1  Green apple  $1
5  Green apples  $4
10  Green apples  $7

Since there's only one "field" in the first line, I want this output:

Price table

- insted of

Price table

How to? Thank you i advance.
--
Venlig hilsen
Kurt Hansen
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to modify this script?

[Kurt Hansen]

Den 06/01/13 13.52, Chris Angelico skrev:

On Sun, Jan 6, 2013 at 11:42 PM, Kurt Hansen  wrote:

Since there's only one "field" in the first line, I want this output:

Price table

- insted of

Price table

How to? Thank you i advance.



It's actually quite simple, as long as you don't mind the junk of
colspan="1" on all the other cells.


I do, but I would like to test anyway ;-)

 Just replace the innermost loop

with:

 for item in columns:
 output += '' +
item + ' '


"innermost"? I have replaced this with yours, but all the marked text 
are deleted:


for item in columns:
output += '' + item + ' '


Untested, but it ought to work - as long as you never have _more_
cells in the line.

--
Regards
Kurt Hansen
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to modify this script?

[Kurt Hansen]

Den 06/01/13 13.58, chaouche yacine skrev:

if len(columns) != 3:
colspan = 3 - len(columns) + 1
output += '' % (colspan) + item + ' '

I did not test. Use with caution.


I've tried to put it in several different places in the script, but with 
no luck; remember that I'm not experienced, so please tell me exactly 
where it's surposed to be inserted. Could you eventually show the 
complete modified script?




----
*From:* Kurt Hansen 
*To:* python-list@python.org
*Sent:* Sunday, January 6, 2013 1:42 PM
*Subject:* How to modify this script?

http://www.tuxradar.com/content/save-time-gedit-snippets:

To convert tab-separated text lines into a HTML-table:

$<
lines = $GEDIT_SELECTED_TEXT.split("\n");
output = '\n';

for line in lines:
 output += '';

 columns = line.split("\t");
 for item in columns:
 output += '' + item + ' '

 output += '\n';

output += '';
return output
 >

I would like to make a small modification (I'm not a programmer myself).
Let's say I have these lines:

Price table
1  Green apple  $1
5  Green apples  $4
10  Green apples  $7

Since there's only one "field" in the first line, I want this output:

Price table

- insted of

Price table

How to? Thank you i advance.
-- Venlig hilsen
Kurt Hansen
-- http://mail.python.org/mailman/listinfo/python-list





--
Venlig hilsen
Kurt Hansen
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to modify this script?

[Kurt Hansen]

Den 06/01/13 14.44, Chris Angelico wrote:

On Mon, Jan 7, 2013 at 12:34 AM, Kurt Hansen  wrote:

"innermost"? I have replaced this with yours, but all the marked text are
deleted:


Here's the full code, with my change:

$<
lines = $GEDIT_SELECTED_TEXT.split("\n");
output = '\n';


I'm sorry to bother you, Chris, but applying the snippet with your code 
in Gedit still just deletes the marked, tab-separated text in the editor.

--
Venlig hilsen
Kurt Hansen
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to modify this script?

[Kurt Hansen]

Den 06/01/13 15.01, chaouche yacine wrote:

Well, I'm not answering your question since I am rewriting the script,
because I prefer it this way :)

def addline(line):
 return "%s\n" % line

[cut]

I surpose I shall put your code between $< and >?


printed

 >>> 
Price table
1  Green apple  $1
5  Green apples  $4
10  Green apples  $7

 >>>


Aha, so you tested it yourself?

When running this in Gedit on four lines of tab-separated text the 
output is:


%s\n" % line

def addcolumn(item,nb_columns):
if nb_columns != 3:
return "%s" % (3 - nb_columns + 1, item)
return "%s" % item

output = "\n"
for line in file("data.txt"):
items = line.strip().split("\t")
columns = ""
for item in items :
columns += addcolumn(item,len(items))
output  += addline(columns)


output += ""
print output
>
--
Venlig hilsen
Kurt Hansen
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to modify this script?

[Kurt Hansen]

Den 06/01/13 15.20, Chris Angelico wrote:

On Mon, Jan 7, 2013 at 1:03 AM, Kurt Hansen  wrote:

I'm sorry to bother you, Chris, but applying the snippet with your code in
Gedit still just deletes the marked, tab-separated text in the editor.



Ah, whoops. That would be because I had a bug in the code (that's why
I commented that it was untested). Sorry about that! Here's a fixed
version:


[cut]>

Note that it's a single line:

output += '' + item + ' '

If your newsreader (or my poster) wraps it, you'll need to unwrap that
line, otherwise you'll get an IndentError.


Ahhh, I did'nt realize that. Now it works :-)


That version should work.


It certainly does. I'll keep it and use it until at better solution is 
found. In the meantime I can just remove any unnecessary "colspan="1" 
with a macro.


Thanks for your help.
--
Venlig hilsen
Kurt Hansen
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to modify this script?

[Kurt Hansen]

Den 06/01/13 15.22, Subimal Deb wrote:

Kurt,
Try this:


[cut]

I've tested it on my original example:

Price table
1   Green apple $1
5   Green apples$4
10  Green apples$7

With all four lines selected it makes an error. With only three (without 
the first line) it works all right.


The error message says:

Execution of the Python command (lines = 'Price table\n1\tGreen 
apple\t$1\n5\tGreen apples\t$4\n10\tGreen apples\t$7'.split("\n");

output = '\n';

for line in lines:
output += '';

columns = line.split("\t");
if len(columns)==1:
output += '', line, ''
else:
for item in columns:
output += '' + item + ' '

output += '\n';

output += '';
return output) failed: cannot concatenate 'str' and 'tuple' objects
--
Venlig hilsen
Kurt Hansen
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to modify this script?

[Kurt Hansen]

Den 06/01/13 15.52, Chris Angelico skrev:

On Mon, Jan 7, 2013 at 1:40 AM, Kurt Hansen  wrote:

failed: cannot concatenate 'str' and 'tuple' objects


The problem is this line:

output += '', line, ''

Change it to:

output += '' + line + ''


:-)

Something happened allright, but ...
Output with this change:

' + line + ''
else:
for item in columns:
output += '' + item + ' '

output += '\n';

output += '';
return output
>
--
Venlig hilsen
Kurt Hansen
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to modify this script?

[Kurt Hansen]

Den 06/01/13 16.12, chaouche yacine skrev:

I'm not confident this would run on gedit. It works on a python
interpreter if you have a file named data.txt in the same directory
containing your sample data.

It surely has to do with how gedit works then, because the "$" sign
isn't used in python, this business should be a gedit convention. And
sorry, I can't help on that, I'm not a user of gedit myself. Fortunately
others have answered and I beleive one of the solutions worked for you.


It does not seem to be the case :-(

Thank you for trying to help.


----
*From:* Kurt Hansen 
*To:* python-list@python.org
*Sent:* Sunday, January 6, 2013 3:21 PM
*Subject:* Re: How to modify this script?

Den 06/01/13 15.01, chaouche yacine wrote:
 > Well, I'm not answering your question since I am rewriting the script,
 > because I prefer it this way :)
 >
 > def addline(line):
 >  return "%s\n" % line
[cut]

I surpose I shall put your code between $< and >?

 > printed
 >
 >  >>> 
 > Price table
 > 1  Green apple  $1
 > 5  Green apples  $4
 > 10  Green apples  $7
 > 
 >  >>>

Aha, so you tested it yourself?

When running this in Gedit on four lines of tab-separated text the
output is:

%s\n" % line

def addcolumn(item,nb_columns):
 if nb_columns != 3:
 return "%s" % (3 - nb_columns + 1, item)
 return "%s" % item

output = "\n"
for line in file("data.txt"):
 items = line.strip().split("\t")
 columns = ""
 for item in items :
 columns += addcolumn(item,len(items))
 output  += addline(columns)


output += ""
print output
 >
-- Venlig hilsen
Kurt Hansen
-- http://mail.python.org/mailman/listinfo/python-list





--
Venlig hilsen
Kurt Hansen
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to modify this script?

[Kurt Hansen]

Den 08/01/13 16.31, chaouche yacine skrev:

Well tell me how do you use this script in gedit, are you using it as a
plugin ?


"Snippets" is a plugin, yes. It's included in the .app for Mac (v. 
2.30.2), but not activated af default.


Open "Tools" in the menu line and click "Manage snippets...". Here you 
can organize, add and edit snippets of texts. The feature olså has the 
ability to work with Python code inside the snippet content.


I am re-building a 15 years old homepage. The HTML code is handmade over 
the years and very varying, buggy etc., så I would like to renew the 
HTML for the table structure in an easy way.


Example: On this page: http://www.danacord.dk/frmsets/records/732-r.html 
I mark the content of the CD, copy it to the clipboard and paste it into 
the editing area in Gedit. cmd-a marks it all again and then I "run" the 
snippet upon the text, either using my self-defined hotkey or by pushing 
ctrl+space and select my snippet from a list.


The copied text is inserted as clean text without any HTML. The 
Python-snippet we are discussing recognizes tabs to separate the columns 
and adds the apprpriate HTML-code to it.

--
Regards
Kurt Hansen
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to modify this script?

[Kurt Hansen]

Den 06/01/13 15.20, Chris Angelico wrote:



That version should work.



Am 06.01.2013 15:30 schrieb Kurt Hansen:


It certainly does. I'll keep it and use it until at better solution is
found.


On 08/01/13 15.18, Thomas Rachel wrote:
>

That would be simple:

Replace

output += '' + item +
' '

with

if len(columns) >= 3:
 output += ''
else:
 output += ''
output += item + ' '

(untested as well; keep the indentation in mind!)


Thanks, Thomas, but ...

The script stops right bofore
= 3:
output += ''

This, and the rest of the script code is put out as the result, not my test.
--
Venlig hilsen
Kurt Hansen
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to modify this script?

[Kurt Hansen]

Den 09/01/13 11.23, chaouche yacine skrev:

I figrued it out. Copy/paste exactly these lines in the snippets tool.
You can bind it to a key as you may know, I bound it to Ctrl-E. So paste
it in a new snippet (keep the original in a safe place), bind to a key,
select the text you want to html-tableize and hit the key binding. In my
case it worked.

$<

[cut]

def addline(line):


Spooky behavior. Yes, the green-apple-example also works for me with 
your new script, BUT ...!


Try to copy the table content on this page:
http://www.danacord.dk/frmsets/records/732-r.html
which is a realistic scenario. That's whar I am doing these days.

Pasting it into Gedit and running the snippet blanks the edit area (on 
MY Mac at least).


And yes: I have pasted your code excatly and I've double-checked for 
linewraps. Everything is okay.


For your cenvenience I have put borders on the table online (see link 
above). You may ommit the rows after track 14. Not that it makes any 
differerence, but that block is surposed to be formatted differerent. I 
do that manually afterwards ... if not ... ;-)

--
Regards
Kurt Hansen
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to modify this script?

[Kurt Hansen]

Kurt wrote:
>
> Spooky behavior. Yes, the green-apple-example also works for me with
> your new script, BUT ...!
>
> Try to copy the table content on this page:
> http://www.danacord.dk/frmsets/records/732-r.html
> which is a realistic scenario. That's whar I am doing these days.
>
> Pasting it into Gedit and running the snippet blanks the edit area (on
> MY Mac at least).
>
> And yes: I have pasted your code excatly and I've double-checked for
> linewraps. Everything is okay.

Chaouche replied:


Indeed, the console shows a traceback where data is misinterpreted,
maybe due to my triple protective quotes around $GEDIT_SELECTED_TEXT.
Try without them, like so (it worked for me) :


Yes!!! :-)

Of course it would be nice if the script could be developed to take into 
account some of the antics I have made in my tables over the years, but 
with this script the basic codes of rows and columns is formatted 
properly; refinements of some collapsed fields with with line breaks 
a.o. is up to me now ;-).


Thanks to you and others who have participated in this thread.
--
Venlig hilsen
Kurt Hansen
--
http://mail.python.org/mailman/listinfo/python-list

Re: Donald E. Knuth in Python, cont'd

[Kurt Mueller]

Am 11.04.2012 16:38, schrieb mwil...@the-wire.com:

Antti J Ylikoski wrote:

I wrote about a straightforward way to program D. E. Knuth in Python,


Maybe it's godly coded...


$ python Easter.py
The year: 2012
(20, 'th ', 'APRIL, ', 2012)
$

AFAIK it was the 8th of April 2012?



Grüessli
--
Kurt Müller, m...@problemlos.ch

--
http://mail.python.org/mailman/listinfo/python-list

Re: Question About Command line arguments

[Kurt Smith]

On Fri, Jun 10, 2011 at 12:58 PM, Mark Phillips
 wrote:
> How do I write my script so it picks up argument from the output of commands
> that pipe input into my script?

def main():
import sys
print sys.stdin.read()

if __name__ == '__main__':
main()

$ echo "fred" | python script.py
fred
$
-- 
http://mail.python.org/mailman/listinfo/python-list

nltk in 3.7.2 for win 10 64 bit

[Türkcan Kurt]

is there a way to install nltk in win 10 64 bit in python 3.7.2 ?

--
Teşekkürler, iyi çalışmalar.

 


ODTU - BİDB

Lisanslı Yazılım Sorumlusu


swb...@metu.edu.tr

Tel: 210 3311

--
https://mail.python.org/mailman/listinfo/python-list

right adjusted strings containing umlauts

[Kurt Mueller]

I'd like to print strings right adjusted.
( Python 2.7.3, Linux 3.4.47-2.38-desktop )

from __future__ import print_function
print( '>{0:>3}<'.format( 'a' ) )
>  a<

But if the string contains an Umlaut:
print( '>{0:>3}<'.format( 'ä' ) )
> ä<

Same with % notation:
print( '>%3s<' % ( 'a' ) )
>  a<
print( '>%3s<' % ( 'ä' ) )
> ä<

For a string with no Umlaut it uses 3 characters, but for an Umlaut
it uses only 2 characters.

I guess it has to to with unicode.
How do I get it right?


TIA
-- 
Kurt Mueller

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: right adjusted strings containing umlauts

[Kurt Mueller]

Am 08.08.2013 16:43, schrieb jfhar...@gmail.com:
> On Thursday, 8 August 2013 15:23:46 UTC+1, Kurt Mueller  wrote:
>> I'd like to print strings right adjusted.
>> print( '>{0:>3}<'.format( 'ä' ) )
> 
> Make both strings unicode
> print( u'>{0:>3}<'.format( u'ä' ) )
> Why not use rjust for it though?
> u'ä'.rjust(3)

In real life there is a list of strings in output_list from a command like:
output_list = shlex.split( input_string, bool_cmnt, bool_posi, )
input_string is from a file, bool_* are either True or False
repr( output_list )
['\xc3\xb6', '\xc3\xbc', 'i', 's', 'f']
which should be printed right aligned.
using:
print( u'{0:>3} {1:>3} {2:>3} {3:>3} {4:>3}'.format( *output_list ) )
( In real life, the alignement and the width is variable )

How do I prepare output_list the pythonic way to be unicode strings?
What do I do, when input_strings/output_list has other codings like iso-8859-1?

TIA
-- 
Kurt Mueller

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: right adjusted strings containing umlauts

[Kurt Mueller]

Am 08.08.2013 17:44, schrieb Peter Otten:
> Kurt Mueller wrote:
>> What do I do, when input_strings/output_list has other codings like
>> iso-8859-1?
> 
> You have to know the actual encoding. With that information it's easy:
>>>> output_list
> ['\xc3\xb6', '\xc3\xbc', 'i', 's', 'f']
>>>> encoding = "utf-8"
>>>> output_list = [s.decode(encoding) for s in output_list]
>>>> print output_list
> [u'\xf6', u'\xfc', u'i', u's', u'f']

How do I get to know the actual encoding?
I read from stdin. There can be different encondings.
Usually utf8 but also iso-8859-1/latin9 are to be expected.
But sys.stdin.encoding sais always 'None'.


TIA
-- 
Kurt Mueller
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: right adjusted strings containing umlauts

[Kurt Mueller]

Now I have this small example:
--
#!/usr/bin/env python
# vim: set fileencoding=utf-8 :

from __future__ import print_function
import sys, shlex

print( repr( sys.stdin.encoding ) )

strg_form = u'{0:>3} {1:>3} {2:>3} {3:>3} {4:>3}'
for inpt_line in sys.stdin:
proc_line = shlex.split( inpt_line, False, True, )
encoding = "utf-8"
proc_line = [ strg.decode( encoding ) for strg in proc_line ]
print( strg_form.format( *proc_line ) )
--

$ echo -e "a b c d e\na ö u 1 2" | file -
/dev/stdin: UTF-8 Unicode text
$ echo -e "a b c d e\na ö u 1 2" | ./align_compact.py
None
  a   b   c   d   e
  a   ö   u   1   2
$ echo -e "a b c d e\na ö u 1 2" | recode utf8..latin9 | file -
/dev/stdin: ISO-8859 text
$ echo -e "a b c d e\na ö u 1 2" | recode utf8..latin9 | ./align_compact.py
None
  a   b   c   d   e
Traceback (most recent call last):
  File "./align_compact.py", line 13, in 
proc_line = [ strg.decode( encoding ) for strg in proc_line ]
  File "/usr/lib64/python2.7/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xf6 in position 0: invalid 
start byte
muk@mcp20:/sw/prog/scripts/text_manip>

How do I handle this two inputs?


TIA
-- 
Kurt Mueller

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: right adjusted strings containing umlauts

[Kurt Mueller]

Am 08.08.2013 18:37, schrieb Chris Angelico:
> On Thu, Aug 8, 2013 at 5:16 PM, Kurt Mueller
>  wrote:
>> Am 08.08.2013 17:44, schrieb Peter Otten:
>>> Kurt Mueller wrote:
>>>> What do I do, when input_strings/output_list has other codings like
>>>> iso-8859-1?
>>> You have to know the actual encoding. With that information it's easy:
>>>>>> output_list
>>> ['\xc3\xb6', '\xc3\xbc', 'i', 's', 'f']
>>>>>> encoding = "utf-8"
>>>>>> output_list = [s.decode(encoding) for s in output_list]
>>>>>> print output_list
>>> [u'\xf6', u'\xfc', u'i', u's', u'f']
>> How do I get to know the actual encoding?
>> I read from stdin. There can be different encondings.
>> Usually utf8 but also iso-8859-1/latin9 are to be expected.
>> But sys.stdin.encoding sais always 'None'.
> 
> If you can switch to Python 3, life becomes a LOT easier. The Python 3
> input() function (which does the same job as raw_input() from Python
> 2) returns a Unicode string, meaning that it takes care of encodings
> for you.

Because I cannot switch to Python 3 for now my life is not so easy:-)

For some text manipulation tasks I need a template to split lines
from stdin into a list of strings the way shlex.split() does it.
The encoding of the input can vary.
For further processing in Python I need the list of strings to be in unicode.

Here is template.py:

##
#!/usr/bin/env python
# vim: set fileencoding=utf-8 :
# split lines from stdin into a list of unicode strings
# Muk 2013-08-23
# Python 2.7.3

from __future__ import print_function
import sys
import shlex
import chardet

bool_cmnt = True  # shlex: skip comments
bool_posx = True  # shlex: posix mode (strings in quotes)

for inpt_line in sys.stdin:
print( 'inpt_line=' + repr( inpt_line ) )
enco_type = chardet.detect( inpt_line )[ 'encoding' ]   # 
{'encoding': 'EUC-JP', 'confidence': 0.99}
print( 'enco_type=' + repr( enco_type ) )
try:
strg_inpt = shlex.split( inpt_line, bool_cmnt, bool_posx, ) # shlex 
does not work on unicode
except Exception, errr: # usually 
'No closing quotation'
print( "error='%s' on inpt_line='%s'" % ( errr, inpt_line.rstrip(), ), 
file=sys.stderr, )
continue
print( 'strg_inpt=' + repr( strg_inpt ) )   # list of 
strings
strg_unic = [ strg.decode( enco_type ) for strg in strg_inpt ]  # decode 
the strings into unicode
print( 'strg_unic=' + repr( strg_unic ) )   # list of 
unicode strings
##

$ cat  | template.py


Comments are welcome.


TIA
-- 
Kurt Mueller
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: right adjusted strings containing umlauts

[Kurt Mueller]

Am 08.08.2013 18:37, schrieb Chris Angelico:
> On Thu, Aug 8, 2013 at 5:16 PM, Kurt Mueller
>  wrote:
>> Am 08.08.2013 17:44, schrieb Peter Otten:
>>> Kurt Mueller wrote:
>>>> What do I do, when input_strings/output_list has other codings like
>>>> iso-8859-1?
>>> You have to know the actual encoding. With that information it's easy:
>>>>>> output_list
>>> ['\xc3\xb6', '\xc3\xbc', 'i', 's', 'f']
>>>>>> encoding = "utf-8"
>>>>>> output_list = [s.decode(encoding) for s in output_list]
>>>>>> print output_list
>>> [u'\xf6', u'\xfc', u'i', u's', u'f']
>> How do I get to know the actual encoding?
>> I read from stdin. There can be different encondings.
>> Usually utf8 but also iso-8859-1/latin9 are to be expected.
>> But sys.stdin.encoding sais always 'None'.
> 
> If you can switch to Python 3, life becomes a LOT easier. The Python 3
> input() function (which does the same job as raw_input() from Python
> 2) returns a Unicode string, meaning that it takes care of encodings
> for you.

Because I cannot switch to Python 3 for now my life is not so easy:-)

For some text manipulation tasks I need a template to split lines
from stdin into a list of strings the way shlex.split() does it.
The encoding of the input can vary.
For further processing in Python I need the list of strings to be in unicode.

Here is template.py:

##
#!/usr/bin/env python
# vim: set fileencoding=utf-8 :
# split lines from stdin into a list of unicode strings
# Muk 2013-08-23
# Python 2.7.3

from __future__ import print_function
import sys
import shlex
import chardet

bool_cmnt = True  # shlex: skip comments
bool_posx = True  # shlex: posix mode (strings in quotes)

for inpt_line in sys.stdin:
print( 'inpt_line=' + repr( inpt_line ) )
enco_type = chardet.detect( inpt_line )[ 'encoding' ]   # 
{'encoding': 'EUC-JP', 'confidence': 0.99}
print( 'enco_type=' + repr( enco_type ) )
try:
strg_inpt = shlex.split( inpt_line, bool_cmnt, bool_posx, ) # shlex 
does not work on unicode
except Exception, errr: # usually 
'No closing quotation'
print( "error='%s' on inpt_line='%s'" % ( errr, inpt_line.rstrip(), ), 
file=sys.stderr, )
continue
print( 'strg_inpt=' + repr( strg_inpt ) )   # list of 
strings
strg_unic = [ strg.decode( enco_type ) for strg in strg_inpt ]  # decode 
the strings into unicode
print( 'strg_unic=' + repr( strg_unic ) )   # list of 
unicode strings
##

$ cat  | template.py


Comments are welcome.


TIA
-- 
Kurt Mueller

-- 
Kurt Mueller
-- 
http://mail.python.org/mailman/listinfo/python-list

split lines from stdin into a list of unicode strings

[Kurt Mueller]

This is a follow up to the Subject
"right adjusted strings containing umlauts"

For some text manipulation tasks I need a template to split lines
from stdin into a list of strings the way shlex.split() does it.
The encoding of the input can vary.
For further processing in Python I need the list of strings to be in unicode.

Here is template.py:

##
#!/usr/bin/env python
# vim: set fileencoding=utf-8 :
# split lines from stdin into a list of unicode strings
# Muk 2013-08-23
# Python 2.7.3

from __future__ import print_function
import sys
import shlex
import chardet

bool_cmnt = True  # shlex: skip comments
bool_posx = True  # shlex: posix mode (strings in quotes)

for inpt_line in sys.stdin:
print( 'inpt_line=' + repr( inpt_line ) )
enco_type = chardet.detect( inpt_line )[ 'encoding' ]   # 
{'encoding': 'EUC-JP', 'confidence': 0.99}
print( 'enco_type=' + repr( enco_type ) )
try:
strg_inpt = shlex.split( inpt_line, bool_cmnt, bool_posx, ) # shlex 
does not work on unicode
except Exception, errr: # usually 
'No closing quotation'
print( "error='%s' on inpt_line='%s'" % ( errr, inpt_line.rstrip(), ), 
file=sys.stderr, )
continue
print( 'strg_inpt=' + repr( strg_inpt ) )   # list of 
strings
strg_unic = [ strg.decode( enco_type ) for strg in strg_inpt ]  # decode 
the strings into unicode
print( 'strg_unic=' + repr( strg_unic ) )   # list of 
unicode strings
##

$ cat  | template.py


Comments are welcome.


TIA
-- 
Kurt Mueller



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: split lines from stdin into a list of unicode strings

[Kurt Mueller]

Am 29.08.2013 11:12, schrieb Peter Otten:
> kurt.alfred.muel...@gmail.com wrote:
>> On Wednesday, August 28, 2013 1:13:36 PM UTC+2, Dave Angel wrote:
>>> On 28/8/2013 04:32, Kurt Mueller wrote:
>>>> For some text manipulation tasks I need a template to split lines
>>>> from stdin into a list of strings the way shlex.split() does it.
>>>> The encoding of the input can vary.

> You can compromise and read ahead a limited number of lines. Here's my demo 
> script (The interesting part is detect_encoding(), I got a bit distracted by 
> unrelated stuff...). The script does one extra decode/encode cycle -- it 
> should be easy to avoid that if you run into performance issues.

Thanks Peter!

I see the idea. It limits the buffersize/memory usage for the detection.


I have to say that I am a bit disapointed by the chardet library.
The encoding for the single character 'ü'
is detected as {'confidence': 0.99, 'encoding': 'EUC-JP'},
whereas "file" says:
$ echo "ü" | file -i -
/dev/stdin: text/plain; charset=utf-8
$

"ü" is a character I use very often, as it is in my name: "Müller":-)


I try to use the "python-magic" library which has a similar functionality
as chardet and is used by the "file" unix-command and it is expandable
with a magicfile, see "man file".


My magic_test script:
---
#!/usr/bin/env python
# vim: set fileencoding=utf-8 :
from __future__ import print_function
import magic
strg_chck = 'ü'
magc_enco = magic.open( magic.MAGIC_MIME_ENCODING )
magc_enco.load()
print( strg_chck + ' encoding=' + magc_enco.buffer( strg_chck ) )
magc_enco.close()
---
$ magic_test
ü encoding=utf-8

python-magic seems to me a bit more reliable.


Cheers
-- 
Kurt Mueller

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: split lines from stdin into a list of unicode strings

[Kurt Mueller]

Am 29.08.2013 11:12, schrieb Peter Otten:
> kurt.alfred.muel...@gmail.com wrote:
>> On Wednesday, August 28, 2013 1:13:36 PM UTC+2, Dave Angel wrote:
>>> On 28/8/2013 04:32, Kurt Mueller wrote:
>>>> For some text manipulation tasks I need a template to split lines
>>>> from stdin into a list of strings the way shlex.split() does it.
>>>> The encoding of the input can vary.

> You can compromise and read ahead a limited number of lines. Here's my demo 
> script (The interesting part is detect_encoding(), I got a bit distracted by 
> unrelated stuff...). The script does one extra decode/encode cycle -- it 
> should be easy to avoid that if you run into performance issues.

I took your script as a template.
But I used the libmagic library (pyhton-magic) instead of chardet.
See http://linux.die.net/man/3/libmagic
and https://github.com/ahupp/python-magic
( I made tests with files of different size, up to 1.2 [GB] )

I had following issues:

- I a real file, the encoding was detected as 'ascii' for detect_lines=1000.
  In line 1002 there was an umlaut character. So then the line.decode(encoding) 
failed.
  I think to add the errors parameter, line.decode(encoding, errors='replace')

- If the buffer was bigger than about some Megabytes, the returned encoding
  from libmagic was always None. The big files had very long lines ( more than 
4k per line ).
  So with detect_lines=1000 this limit was exceeded.

- The magic.buffer() ( the equivalent of chardet.detect() ) takes about 2 
seconds
  per megabyte buffer.



-- 
Kurt Mueller
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: split lines from stdin into a list of unicode strings

[Kurt Mueller]

Am 05.09.2013 10:33, schrieb Peter Otten:
> Kurt Mueller wrote:
>> Am 29.08.2013 11:12, schrieb Peter Otten:
>>> kurt.alfred.muel...@gmail.com wrote:
>>>> On Wednesday, August 28, 2013 1:13:36 PM UTC+2, Dave Angel wrote:
>>>>> On 28/8/2013 04:32, Kurt Mueller wrote:
>>>>>> For some text manipulation tasks I need a template to split lines
>>>>>> from stdin into a list of strings the way shlex.split() does it.
>>>>>> The encoding of the input can vary.
>> I took your script as a template.
>> But I used the libmagic library (pyhton-magic) instead of chardet.
>> See http://linux.die.net/man/3/libmagic
>> and https://github.com/ahupp/python-magic
>> ( I made tests with files of different size, up to 1.2 [GB] )
>> I had following issues:
>> - I a real file, the encoding was detected as 'ascii' for
>> detect_lines=1000.
>>   In line 1002 there was an umlaut character. So then the
>>   line.decode(encoding) failed. I think to add the errors parameter,
>>   line.decode(encoding, errors='replace')
> 
> Tough luck ;) You could try and tackle the problem by skipping leading 
> ascii-only lines. Untested:
> 
> def detect_encoding(instream, encoding, detect_lines, skip_ascii=True):
> if encoding is None:
> encoding = instream.encoding
> if encoding is None:
> if skip_ascii:
> try:
> for line in instream:
> yield line.decode("ascii")
> except UnicodeDecodeError:
> pass
> else:
> return
> head = [line]
> head.extend(islice(instream, detect_lines-1))
> encoding =  chardet.detect("".join(head))["encoding"]
> instream = chain(head, instream)
> for line in instream:
> yield line.decode(encoding)

I find this solution as a generator very nice.
With just some small modifications it runs fine for now.
( line is undefined if skip_ascii is False. )

For ascii only files chardet or libmagic will not be bothered.
And the detect_lines comes not in charge, until there are
some non ascii characters.

--
def decode_stream_lines( inpt_strm, enco_type, numb_inpt, skip_asci=True, ):
if enco_type is None:
enco_type = inpt_strm.encoding
if enco_type is None:
line_head = []
if skip_asci:
try:
for line in inpt_strm:
yield line.decode( 'ascii' )
except UnicodeDecodeError:
line_head = [ line ] # last line was not ascii
else:
return # all lines were ascii
line_head.extend( islice( inpt_strm, numb_inpt - 1 ) )
magc_enco = magic.open( magic.MAGIC_MIME_ENCODING )
magc_enco.load()
enco_type = magc_enco.buffer( "".join( line_head ) )
magc_enco.close()
print( I_AM + '-ERROR: enco_type=' + repr( enco_type ), 
file=sys.stderr, )
if  enco_type.rfind( 'binary' ) >= 0: # binary, 
application/mswordbinary, application/vnd.ms-excelbinary and the like
return
inpt_strm = chain( line_head, inpt_strm )
for line in inpt_strm:
yield line.decode( enco_type, errors='replace' )
--


Thank you very much!
-- 
Kurt Mueller
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to turn a string into a list of integers?

[Kurt Mueller]

Am 05.09.2014 um 10:42 schrieb c...@isbd.net:

> Joshua Landau  wrote:
>> On 3 September 2014 15:48,   wrote:
>>> Peter Otten <__pete...@web.de> wrote:
>>>>>>> [ord(c) for c in "This is a string"]
>>>> [84, 104, 105, 115, 32, 105, 115, 32, 97, 32, 115, 116, 114, 105, 110, 103]
>>>> 
>>>> There are other ways, but you have to describe the use case and your Python
>>>> version for us to recommend the most appropriate.
>>>> 
>>> That looks OK to me.  It's just for outputting a string to the block
>>> write command in python-smbus which expects an integer array.
>> 
>> Just be careful about Unicode characters.
> 
> I have to avoid them completely because I'm sending the string to a
> character LCD with a limited 8-bit only character set.


Could someone please explain the following behavior to me:
Python 2.7.7, MacOS 10.9 Mavericks

>>> import sys
>>> sys.getdefaultencoding()
'ascii'
>>> [ord(c) for c in 'AÄ']
[65, 195, 132]
>>> [ord(c) for c in u'AÄ']
[65, 196]

My obviously wrong understanding:
‚AÄ‘ in ‚ascii‘ are two characters
 one with ord A=65 and
 one with ord Ä=196 ISO8859-1 
 —-> why [65, 195, 132]
u’AÄ’ is an Unicode string
 —-> why [65, 196]

It is just the other way round as I would expect.



Thank you
-- 
Kurt Mueller, kurt.alfred.muel...@gmail.com

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to turn a string into a list of integers?

[Kurt Mueller]

Am 05.09.2014 um 20:25 schrieb Chris “Kwpolska” Warrick :
> On Sep 5, 2014 7:57 PM, "Kurt Mueller"  wrote:
> > Could someone please explain the following behavior to me:
> > Python 2.7.7, MacOS 10.9 Mavericks
> >
> > >>> import sys
> > >>> sys.getdefaultencoding()
> > 'ascii'
> > >>> [ord(c) for c in 'AÄ']
> > [65, 195, 132]
> > >>> [ord(c) for c in u'AÄ']
> > [65, 196]
> >
> > My obviously wrong understanding:
> > ‚AÄ‘ in ‚ascii‘ are two characters
> >  one with ord A=65 and
> >  one with ord Ä=196 ISO8859-1 
> >  —-> why [65, 195, 132]
> > u’AÄ’ is an Unicode string
> >  —-> why [65, 196]
> >
> > It is just the other way round as I would expect.
> 
> Basically, the first string is just a bunch of bytes, as provided by your 
> terminal — which sounds like UTF-8 (perfectly logical in 2014).  The second 
> one is converted into a real Unicode representation. The codepoint for Ä is 
> U+00C4 (196 decimal). It's just a coincidence that it also matches latin1 aka 
> ISO 8859-1 as Unicode starts with all 256 latin1 codepoints. Please kindly 
> forget encodings other than UTF-8.

So:
‘AÄ’ is an UTF-8 string represented by 3 bytes:
A -> 41   -> 65  first byte decimal
Ä -> c384 -> 195 and 132 second and third byte decimal

u’AÄ’ is an Unicode string represented by 2 bytes?:
A -> U+0041 -> 65 first byte decimal, 00 is omitted or not yielded by ord()?
Ä -> U+00C4 -> 196 second byte decimal, 00 is ommited or not yielded by ord()?


> BTW: ASCII covers only the first 128 bytes.

ACK
-- 
Kurt Mueller, kurt.alfred.muel...@gmail.com

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to turn a string into a list of integers?

[Kurt Mueller]

Am 05.09.2014 um 21:16 schrieb Kurt Mueller :
> Am 05.09.2014 um 20:25 schrieb Chris “Kwpolska” Warrick :
>> On Sep 5, 2014 7:57 PM, "Kurt Mueller"  wrote:
>>> Could someone please explain the following behavior to me:
>>> Python 2.7.7, MacOS 10.9 Mavericks
>>> 
>>>>>> import sys
>>>>>> sys.getdefaultencoding()
>>> 'ascii'
>>>>>> [ord(c) for c in 'AÄ']
>>> [65, 195, 132]
>>>>>> [ord(c) for c in u'AÄ']
>>> [65, 196]
>>> 
>>> My obviously wrong understanding:
>>> ‚AÄ‘ in ‚ascii‘ are two characters
>>> one with ord A=65 and
>>> one with ord Ä=196 ISO8859-1 
>>> —-> why [65, 195, 132]
>>> u’AÄ’ is an Unicode string
>>> —-> why [65, 196]
>>> 
>>> It is just the other way round as I would expect.
>> 
>> Basically, the first string is just a bunch of bytes, as provided by your 
>> terminal — which sounds like UTF-8 (perfectly logical in 2014).  The second 
>> one is converted into a real Unicode representation. The codepoint for Ä is 
>> U+00C4 (196 decimal). It's just a coincidence that it also matches latin1 
>> aka ISO 8859-1 as Unicode starts with all 256 latin1 codepoints. Please 
>> kindly forget encodings other than UTF-8.
> 
> So:
> ‘AÄ’ is an UTF-8 string represented by 3 bytes:
> A -> 41   -> 65  first byte decimal
> Ä -> c384 -> 195 and 132 second and third byte decimal
> 
> u’AÄ’ is an Unicode string represented by 2 bytes?:
> A -> U+0041 -> 65 first byte decimal, 00 is omitted or not yielded by ord()?
> Ä -> U+00C4 -> 196 second byte decimal, 00 is ommited or not yielded by ord()?

After reading the ord() manual:
The second case should read:
u’AÄ’ is an Unicode string represented by 2 unicode characters:
If Python was built with UCS2 Unicode, then the character’s code point must
be in the range [0..65535, 16 bits, U-..U-]
A -> U+0041 ->  65 first  character decimal (code point)
Ä -> U+00C4 -> 196 second character decimal (code point)


Am I right now?

-- 
Kurt Mueller, kurt.alfred.muel...@gmail.com

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to turn a string into a list of integers?

[Kurt Mueller]

Am 06.09.2014 um 07:47 schrieb Steven D'Aprano 
:
> Kurt Mueller wrote:
>> Could someone please explain the following behavior to me:
>> Python 2.7.7, MacOS 10.9 Mavericks

[snip]
Thanks for the detailed explanation. I think I understand a bit better now.


Now the part of the two Python builds is still somewhat unclear to me.

> If you could peer under the hood, and see what implementation Python uses to
> store that string, you would see something version dependent. In Python
> 2.7, you would see an object more or less something vaguely like this:
> 
> [object header containing various fields]
> [length = 2]
> [array of bytes = 0x0041 0x00C4]
> 
> 
> That's for a so-called "narrow build" of Python. If you have a "wide build",
> it will something like this:
> 
> [object header containing various fields]
> [length = 2]
> [array of bytes = 0x0041 0x00C4]
> 
> In Python 3.3, "narrow builds" and "wide builds" are gone, and you'll have
> something conceptually like this:
> 
> [object header containing various fields]
> [length = 2]
> [tag = one byte per character]
> [array of bytes = 0x41 0xC4]
> 
> Some other implementations of Python could use UTF-8 internally:
> 
> [object header containing various fields]
> [length = 2]
> [array of bytes = 0x41 0xC3 0x84]
> 
> 
> or even something more complex. But the important thing is, regardless of
> the internal implementation, Python guarantees that a Unicode string is
> treated as a fixed array of code points. Each code point has a value
> between 0 and, not 127, not 255, not 65535, but 1114111.



In Python 2.7:

As I learned from the ord() manual:
If a unicode argument is given and Python was built with UCS2 Unicode,
(I suppose this is the narrow build in your terms),
then the character’s code point must be in the range [0..65535] inclusive;

I understand: In a UCS2 build each character of a Unicode string uses
16 Bits and can represent code points from U-..U-.



>From the unichr(i) manual I learn:
The valid range for the argument depends how Python was configured
– it may be either UCS2 [0..0x] or UCS4 [0..0x10].

I understand: narrow build is UCS2, wide build is UCS4
- In a UCS2 build each character of an Unicode string uses 16 Bits and has 
  code points from U-..U- (0..65535)
- In a UCS4 build each character of an Unicode string uses 32 Bits and has 
  code points from U-..U-0010 (0..1114111)


Am I right?
-- 
Kurt Mueller, kurt.alfred.muel...@gmail.com

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to turn a string into a list of integers?

[Kurt Mueller]

Am 06.09.2014 um 20:19 schrieb Steven D'Aprano 
:
> Kurt Mueller wrote:
> [...]
>> Now the part of the two Python builds is still somewhat unclear to me.
> [...]
>> In Python 2.7:
>> As I learned from the ord() manual:
>> If a unicode argument is given and Python was built with UCS2 Unicode,
> Where does the manual mention UCS-2? As far as I know, no version of Python
> uses that.

https://docs.python.org/2/library/functions.html?highlight=ord#ord


[snip] very detailed explanation of narrow/wide build, UCS-2/UCS-4, 
UTF-16/UTF-32



> Remember, though, these internal representations are (nearly) irrelevant to
> Python code. In Python code, you just consider that a Unicode string is an
> array of ordinal values from 0x0 to 0x10, each representing a single
> code point U+ to U+10. The only reason I say "nearly" is that
> narrow builds don't *quite* work right if the string contains surrogate
> pairs.

So I can interpret your last section:
Processing any Unicode string will work with small and wide
python 2.7 builds and also with python >3.3?
( parts of small build python will not work with values over 0x )
( strings with surrogate pairs will not work correctly on small build python )



Many thanks for your detailed answer!
-- 
Kurt Mueller, kurt.alfred.muel...@gmail.com

-- 
https://mail.python.org/mailman/listinfo/python-list

do not use stunnix

[kurt . michalke]

Various chanels and groups are flooded with information about a thing
called stunnix web server. Do not ever consider using it. It will in
spite of all the marketing blah give you more pain than you can bear.
We licensed it for a CD-ROM and ended up having to issue it three (!)
times.

Cheers,
Kurt

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange set of errors

[Kurt Smith]

Sorry, forgot to "Reply to all."

On 8/3/07, Stephen Webb <[EMAIL PROTECTED]> wrote:
> Greetings all,

<>

> Also, I've been having trouble with the plot function in matplotlib. For
> example, I enter the following in the terminal:
>
>  >>> from pylab import *
> >>> plot([1,2,3])
> []
>

I can help you with your second problem: matplotlib is doing what it
should, all you need to do is tell it to show() the figure you created
(with the plot on it.)  Note -- if you call the show() function, you
will turn control over to the backend (in my case, TkAgg), and lose
the ability to issue interactive commands afterwards.  A solution is
to use the ion() (stands for "interactive-on") function call before
issuing plotting commands:

>>> from pylab import *
>>> ion()
>>> plot([1,2,3])
[]
>>> # image is shown here

See http://matplotlib.sourceforge.net/interactive.html for an
exhaustive explanation.

> Every time I run the plot([1,2,3]) I get a different ending number that
> seems to vary randomly.

The "[]" is what the
plot() function returns -- a list of instances of the Line2D class,
and it tells you their locations in memory (hence the hex number
starting with 0x).  The hex number can be used to uniquely identify
this object, as in the id(object) call.

Hope this helps,

Kurt
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange set of errors

[Kurt Smith]

On 8/3/07, Stephen Webb <[EMAIL PROTECTED]> wrote:
> Greetings all,
>
> I've recently begun using Python to do scientific computation, and I wrote
> the following script to find approximate eigenvalues for a semi-infinite
> matrix:
>
>
>  from pylab import *
> from numpy import *
> from scipy import *
>
> def bandstructure(N,s):
>
> b = s/4.0
>
> jmax = 10 + N**2
>
> spectrum1 = [0]*2*N
> spectrum2 = [0]*2*N
> spectrum3 = [0]*2*N
>
>
> for k in arange(1, 2*N+1, 1):
>
> A = zeros( (jmax,jmax) )
>
> i = 0
> while i <= jmax-1:
> if i <= jmax-2:
> A[i,i+1] = b
> A[i+1,i] = b
> A[i,i] = ((k + 2.0*i*N)/N)**2
> i = i+1
> else:
> A[i,i] = ((k + 2.0*i*N)/N)**2
> i = i+1
>
> #This portion of the code builds a matrix twice as large to check
> against
>
> B = zeros( (2*jmax,2*jmax) )
>
> i = 0
> while i <= 2*jmax-1:
> if i <= 2*jmax-2:
> B[i,i+1] = b
> B[i+1,i] = b
> B[i,i] = ((k + 2.0*i*N)/N)**2
> i = i+1
> else:
> B[i,i] = ((k + 2.0*i*N)/N)**2
> i = i+1
>
> x = linalg.eigvals(A)
> y = linalg.eigvals(B)
>
> j = 1
> while j<=3:
> if abs(y[j]-x[j]) <= 10.0**(-5):
> j = j + 1
> else:
> print 'jmax not large enough to obtain accurate results'
>
> spectrum1[k-1] = x[0] + 0.5*s
> spectrum2[k-1] = x[1] + 0.5*s
> spectrum3[k-1] = x[2] + 0.5*s
>
> plot (k, spectrum1, k, spectrum2, k, spectrum3)
>
> xlabel('k (energy level)')
> ylabel('E/E_r')
> title('Finite Size Band Structure, N = %d, s = %f' % (N, s))
> grid(true)
>
>
> When I run this script, I get the following message, which I can't figure
> out:
>
>  Traceback (most recent call last):
>   File "", line 1, in 
>   File "bandstruc.py", line 61, in bandstructure
> plot (k, spectrum1, k, spectrum2, k, spectrum3)
>   File
> "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/matplotlib/pylab.py",
> line 2028, in plot
> ret =  gca().plot(*args, **kwargs)
>   File
> "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/matplotlib/axes.py",
> line 2535, in plot
> for line in self._get_lines(*args, **kwargs):
>   File
> "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/matplotlib/axes.py",
> line 437, in _grab_next_args
> for seg in self._plot_2_args(remaining[:2], **kwargs):
>   File
> "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/matplotlib/axes.py",
> line 337, in _plot_2_args
> x, y, multicol = self._xy_from_xy(x, y)
>   File
> "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/matplotlib/axes.py",
> line 266, in _xy_from_xy
> nrx, ncx = x.shape
>  ValueError: need more than 0 values to unpack
>


Basically, you're trying to plot spectrum[123] vs. k: the spectra are
lists, but k is an integer.  I made some modifications to get things
working more smoothly.  Note the ion() function call at the beginning,
and the use of "indx = arange(1,2*N+1)" instead of plotting vs. k.

There's probably some other things that could be cleaned up, but I'm
short on time right now; this will get your code working at least.

Hope this helps,
(From a fellow physicist)
Kurt

##

from pylab import *
from numpy import *

ion()

def bandstructure(N,s):

b = s/4.0

jmax = 10 + N**2

spectrum1 = [0]*2*N
spectrum2 = [0]*2*N
spectrum3 = [0]*2*N


for k in arange(1, 2*N+1, 1):

A = zeros( (jmax,jmax) )

i = 0
while i <= jmax-1:
if i <= jmax-2:
A[i,i+1] = b
A[i+1,i] = b
A[i,i] = ((k + 2.0*i*N)/N)**2
i = i+1
else:
A[i,i] = ((k + 2.0*i*N)/N)**2
i = i+1

#This portion of the code builds a matrix twice as large to
check against

B = zeros( (2*jmax,2*jmax) )

i = 0
while i <= 2*jmax-1:
if i <= 2*jmax-2:
B[i,i+1] = b
B[i+1,i] = b
B[i,i] = ((k + 2.0*i*N)/N)**2
i = i+1
e

Re: Drawing a graph

[Kurt Smith]

On 8/12/07, Ghirai <[EMAIL PROTECTED]> wrote:
> Hello list,
>
> I need to draw a graph, 2 axes, 2D, nothing fancy.
> One of the axes is time, the other one is a series of integers.
>
> I don't care much about the output format.
>
> Are there any specialized libraries for this, or should i use PIL?
>
> Thanks.
>
> --
> Regards,
> Ghirai.
> --
> http://mail.python.org/mailman/listinfo/python-list
>

Checkout matplotlib: http://matplotlib.sourceforge.net/

Powerful (esp. with IPython), clean implementation, very nice output,
IMHO.  It does depend on some other things; primarily numpy and some
sort of backend for displaying figures (usually can use a default
backend -- e.g. Tkinter).  Otherwise you can plot directly to file.
May not be what you're looking for if you want something quick.

HTH,

Kurt
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to assign a function to another function

[Kurt Smith]

On 9/17/07, Stefano Esposito <[EMAIL PROTECTED]> wrote:
> Hi all
>
> what i'm trying to do is this:
>
> >>>def foo ():
> ... return None
> ...
> >>>def bar ():
> ... print "called bar"
> ...
> >>>def assigner ():
> ... foo = bar
> ...

You need to tell "assigner()" that foo doesn't belong to the local
(function) namespace, but rather comes from the global namespace:

In [1]: def foo():
   ...: return None
   ...:

In [2]: def bar():
   ...: print "called bar"
   ...:
   ...:

In [3]: def assigner():
   ...: global foo
   ...: foo = bar
   ...:
   ...:

In [4]: assigner()

In [5]: foo()
called bar

Kurt
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: What does the syntax [::-1] really mean?

[Kurt Smith]

On 10/4/07, Casey <[EMAIL PROTECTED]> wrote:
[snippage]
>
> Following the reference to section 3.2 provides a (non-rigorous)
> description of what a slice object is, in terms of the extended
> slicing semantics.  But it doesn't shed any additional light on the
> meaning of [::-1].
>
> >From this, I would expect that x[::-1] would be identical to x[n:0:-1]
> (n and 0 being the "end" values, with the order switched due to the
> negative step value).  But the clause that "(but never including j)"
> means that x[n:0:-1] excludes the 1st element of x, x[0].  A quick
> test in ipython confirms that "abc"[3:0:-1] => "cb", not "cba".
> Changing the "end" value  to x[n:-1:-1] results in an empty string.

Check it out:

>>> 'abc'[3:None:-1]
'cba'
>>>

The second argument to the slice object, if negative, will convert
this index to (len(x)+j), which is why 'abc'[3:-1:-1] == 'abc'[2:2:-1]
== ''.  If you pass in None for the 1st or second index, the slice
object will compute the right bounds based on the __len__ of the
object, and do the right thing.  When there is no value specified,
None is assumed.


>
> So my question is: "what exactly is [::-1] shorthand for"?  Or is it a
> special case, in which case why isn't it  defined as such in the
> library?

obj[::-1] == obj[None:None:-1] == obj[slice(None,None,-1)]

>>> 'abc'[::-1]
'cba'
>>> 'abc'[None:None:-1]
'cba'
>>> 'abc'[slice(None,None,-1)]
'cba'

Taking a look at the slice class:

 |  indices(...)
 |  S.indices(len) -> (start, stop, stride)
 |
 |  Assuming a sequence of length len, calculate the start and stop
 |  indices, and the stride length of the extended slice described by
 |  S. Out of bounds indices are clipped in a manner consistent with the
 |  handling of normal slices.

Kurt
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Newbi Q: Recursively reverse lists but NOT strings?

[Kurt Smith]

On 10/15/07, Gary Herron <[EMAIL PROTECTED]> wrote:
> Dmitri O.Kondratiev wrote:
> > Gary, thanks for lots of info!
> > Python strings are not lists! I got it now. That's a pity, I need two
> > different functions: one to reverse a list and one to reverse a string:
> True, they are not both lists, but they *are* both sequences, with some
> things in common.  In particular xs[::-1] will reverse both types of
> objects.  And even if you roll you own reversal function, you don't need
> two.  One will do.
>
> Gary Herron
>
> >
> > def reverseList(xs):
> > if xs == []:
> > return xs
> > else:
> > return (reverseList (xs[1:])) + [xs[0]]
> >
> > def reverseStr(str):
> > if str == "":
> > return str
> > else:
> > return (reverseStr (str[1:])) + str[0]
> >
> > Ok. Now regarding in-place reversal of a list:
> >
> > >>> l = [1,2,3]
> > >>> l
> > [1, 2, 3]
> > >>> l.reverse()
> > >>> l
> > [3, 2, 1]
> >
> > That was, as I expected. Good.
> >
> > Then why this ? :
> >
> > >>> ls = [1,2,3].reverse()
> > >>> ls
> > >>>
> > >>> print [1,2,3].reverse()
> > None
> > >>>
> > I mean, why ls is empty after assignment?
> >
> > Also, I couldn't find in the Python docs what this form of slicing means:
> > xs[::-1]  ?
> >
> > It works for creating a reversed copy of either a string or a list,
> > but what does '::-1' syntax means?

mylist[::-1] is interpreted as mylist[slice(None,None,-1)], and the
slice object has a method, 'indices' that computes the right endpoints
for what you want to do.

So:

>>> myseq = (1,2,3)
>>> myseq[::-1]
(3, 2, 1)
>>> myseq[slice(None,None,-1)]
(3, 2, 1)
>>> myseq[None:None:-1]
(3, 2, 1)

etc.



> >
> > Thanks,
> >
> > Dmitri O. Kondratiev
> > [EMAIL PROTECTED] 
> > http://www.geocities.com/dkondr
> >
> > On 10/15/07, *Gary Herron* < [EMAIL PROTECTED]
> > > wrote:
> >
> > Dmitri O.Kondratiev wrote:
> > >
> > > The function I wrote (below) reverses lists all right:
> > >
> > > def reverse(xs):
> > > if xs == []:
> > > return []
> > > else:
> > > return (reverse (xs[1:])) + [xs[0]]
> > >
> > >
> > > >>> reverse ([1,2,3])
> > > [3, 2, 1]
> > > >>>
> > >
> > >
> > > Yet when I try to reverse a string I  get:
> > >
> > > >>> reverse ("abc")
> > >
> > > ...
> > > ...
> > > ...
> > >
> > >   File "C:\wks\python-wks\reverse.py", line 5, in reverse
> > >
> > > return (reverse (xs[1:])) + [xs[0]]
> > >
> > >   File "C:\wks\python-wks\reverse.py", line 5, in reverse
> > >
> > > return (reverse (xs[1:])) + [xs[0]]
> > >
> > >   File "C:\wks\python-wks\reverse.py", line 2, in reverse
> > >
> > > if xs == []:
> > >
> > > RuntimeError: maximum recursion depth exceeded in cmp
> > >
> > > >>>
> > >
> > > What's wrong? Why recursion never stops?
> > >
> > If you are doing this as an python-learning exercise, then read
> > on.   If
> > you are doing this reversal for real code, then try:
> >
> >   xs.reverse() for in-place reversal of a list (but not a string), or
> >   result = xs[::-1] for creating a reversed copy of either a
> > string or a
> > list
> >
> >
> > Your recursion stops when xs == [], but when you're stripping
> > characters
> > off a string,  like 'abc', the remaining portion will be 'bc',
> > then 'c',
> > than '', but never [] so you 'll never stop.
> >
> > Try:
> >
> > if xs == []:
> > return []
> > elif xs == '':
> > return ''
> > else:
> > ...
> >
> >
> > Gary Herron
> >
> >
> > >
> > > Thanks,
> > > Dima
> >
> >
> >
> >
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

New subclass vs option in __init__

[Kurt Smith]

Hi List:

Class inheritance noob here.

For context, I have the following base class and subclass:

class Base(object):
def __init__(self, val):
self.val = val

class Derived1(Base):
def __init__(self, val):
super(Derived1, self).__init__(val)

I'm curious as to other's thoughts on the following: when
incorporating optional behavior differences for a subclass, do you a)
make a new subclass (e.g., 'Derived2') and override (and add new)
methods that would encapsulate the new behavior, or b) keep the same
subclass around (i.e., 'Derived1'), but add an initialization option
that would specify the different behavior, and check for the value of
this option in the different methods?

It would seem that there are cases where one would be preferable over
the other: a) when the new behavior would modify a large portion of
the existing subclass, making a new subclass would be ideal; b) when
the new behavior changes only slightly the existing subclass, perhaps
a simple default option in the subclass's __init__ method would be
best.  Where is the tipping point?  Since one cannot predict what
direction the new behavior might take things, should one usually err
on the side of a new subclass?  Is option b) just being lazy?  Is a)
too verbose in many situations?

Kurt
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: subprocess.popen function with quotes

[Kurt Smith]

On Wed, Mar 26, 2008 at 12:15 AM, skunkwerk <[EMAIL PROTECTED]> wrote:
> On Mar 25, 9:25 pm, "Gabriel Genellina" <[EMAIL PROTECTED]>
>  wrote:
>  > En Wed, 26 Mar 2008 00:39:05 -0300, skunkwerk <[EMAIL PROTECTED]>
>  > escribió:
>
> >
>  > >>i'm trying to call subprocess.popen on the 'rename' function in
>  > >> linux.  When I run the command from the shell, like so:
>  >
>  > >> rename -vn 's/\.htm$/\.html/' *.htm
>  >
>  > >> it works fine... however when I try to do it in python like so:
>  > >> p = subprocess.Popen(["rename","-vn","'s/\.htm$/
>  > >> \.html/'","*.htm"],stdout=subprocess.PIPE,stderr=subprocess.PIPE)
>  >
>  > >> print p.communicate()[0]
>  >
>  > >> nothing gets printed out (even for p.communicate()[1])
>  >
>
> > I'd try with:
>  >
>  > p = subprocess.Popen(["rename", "-vn", r"'s/\.htm$/\.html/'", "*.htm"],
>  >stdout=subprocess.PIPE, stderr=subprocess.PIPE,
>  >shell=True)
>  >
>  > (note that I added shell=True and I'm using a raw string to specify the
>  > reg.expr.)
>  >
>  > --
>  > Gabriel Genellina
>
>  Thanks Gabriel,
>I tried the new command and one with the raw string and single
>  quotes, but it is still giving me the same results (no output).  any
>  other suggestions?

I had similar problems passing quoted arguments to grep -- I don't
have rename on my system so I can't do an exact translation.

First version, passing argument to grep that would normally have to be
quoted if typed in shell:

In [1]: from subprocess import *

In [2]: p1 = Popen(['ls'], stdout=PIPE)

In [3]: p2 = Popen(['grep', '[0-9]\{7\}'], stdin=p1.stdout,
stdout=PIPE) # note that the grep regex isn't double quoted...

In [4]: output = p2.communicate()[0]

In [5]: print output
cur0046700.png
cur0046700_1.png
cur0046750.png
dendat0046700.png
dendat0046700_1.png
dendat0046750.png

And we see that everything is hunky dory.

Now, trying to pass grep a quoted argument:

In [10]: p1 = Popen(['ls'], stdout=PIPE)

In [11]: p2 = Popen(['grep', '"[0-9]\{7\}"'], stdin=p1.stdout, stdout=PIPE)

In [12]: output = p2.communicate()[0]

In [13]: print output


In [14]:

And we get nothing.  N.B. that's a single-quote double-quote string
argument to grep in the second example, not a triple single quote.

Incidentally, a triple quoted string will work as well.

Moral from this example:  when passing arguments that would normally
be quoted to be safe from the shell's expansion, etc,  don't.  Just
pass it using Popen(['cmd', 'arg-that-would-normally-be-quoted']) and
it will be passed directly to the function without the shell's
intervention.

Since the argument is passed directly to the command (rename in your
case, grep in this one) quoting to preserve special characters isn't
needed, and the quotes will be passed as part of the argument, giving
the null results you got above.

Kurt
-- 
http://mail.python.org/mailman/listinfo/python-list

MySQLdb and MySQL stored functions

[Kurt Forrester]

Hello All,


I am running 
 - Ubuntu 8.10
 - Python 2.5.2
 - MySQLdb (1, 2, 2, 'final', 0)
 - MySQL Server/Client 5.0.67

I am trying to write an authentication script for a python application
that connects to a MySQL database. The database has a table named `user`
which has the fields `id`, `alias` and `password` as well as a stored
function `authenticate` as detailed below:

CREATE definer=`ro...@`localhost` FUNCTION `authenticate`(a TEXT, p
TEXT) RETURNS int(11)
BEGIN
DECLARE STATUS INT DEFAULT -1;
SELECT id INTO STATUS FROM user WHERE alias = a 
AND password = p;
RETURN STATUS;
END

table:  `user`
`id` = 1
`alias` = 'captain'
`password' = 'a'

I have been executing the following query from various connections:
`SELECT authenticate('captain', 'a')` (this is what is in the table and
should return 1)
and
`SELECT authenticate('captain', 'aa')` (this is a incorrect version of
the password and should return -1)

I have tried running this query from the MySQL Query Browser and it
returns results as expected.
I have also tried query from python using the _mysql module and this
also returns results as expected.
However, when I try to use the MySQLdb module it returns an incorrect
value (it returns 1).

I wish to use the DB API 2.0 compliant module for flexibility. Therefore
I am trying to work out why the MySQLdb does not return the value as
expected (that is as it is returned by the Query Browser).

Any help would be greatly appreciated.

Kurt
--
http://mail.python.org/mailman/listinfo/python-list

Re: Fortran array in python (f2py?)...

[Kurt Smith]

On Sat, Feb 14, 2009 at 2:06 PM, tripp  wrote:

> OK.  It sounds like it would be easiest for me, then, to dump the
> arrays to a binary file (much faster than dumping it to a text) from
> the fortran program.  Then use f2py to load a fortran module to read
> it.?.


I've done something similar and have written some wrapper functions (in pure
python) that read in fortran binary arrays and puts them into a numpy
array.  You have to deal with fortran records, which (for the fortran
compiler I'm using) puts a 4-byte record length indicator at the beginning
and end of each record, with the raw binary data between.  The issue is
mildly complicated if you have to deal with endianness incompatibilites
between computers.  IIRC, the format of the records is compiler dependent,
although I've found that gfortran, g77 and xlfortran are consistent with
each other.

If you like I'd be happy to send you the code.

Kurt
--
http://mail.python.org/mailman/listinfo/python-list

Re: number theory libraries / project euler

[Kurt Smith]

On Wed, Feb 18, 2009 at 2:18 PM, eliben  wrote:
>
> Hello,
>
> What are some good & recommended number theory libs for Python (or
> accessible interfaces to C libs), for things like primes,
> factorization, etc. Naturally, speed is of utmost importance here.
>
> In other words, which Python libraries and tools to you use to help
> you solve Project Euler problems :-) ?

There's Sage: http://www.sagemath.org/ -- I believe it aims to do
everything that Mathematica can do and more, and I know it has some
number theory libs, too.  I haven't had the occasion to use it myself.

Much of the fun of project euler problems is 'rolling your own,' or
implementing a classic algorithm from a description of it on
Wikipedia.  A good sieve of eratosthenes in pure Python (that I tend
to use quite often) is here:

http://code.activestate.com/recipes/117119/

You can find some combinatorics in numpy, I believe.

Often the solutions are one-to-five liners, if you've thought the
problem through.  Too much dependence on external libraries robs the
project euler problems of their fun, IMO.

Best,

Kurt
--
http://mail.python.org/mailman/listinfo/python-list

Re: Regular expression bug?

[Kurt Smith]

On Thu, Feb 19, 2009 at 12:55 PM, Ron Garret  wrote:
> I'm trying to split a CamelCase string into its constituent components.
> This kind of works:
>
>>>> re.split('[a-z][A-Z]', 'fooBarBaz')
> ['fo', 'a', 'az']
>
> but it consumes the boundary characters.  To fix this I tried using
> lookahead and lookbehind patterns instead, but it doesn't work:
>
>>>> re.split('((?<=[a-z])(?=[A-Z]))', 'fooBarBaz')
> ['fooBarBaz']
>
> However, it does seem to work with findall:
>
>>>> re.findall('(?<=[a-z])(?=[A-Z])', 'fooBarBaz')
> ['', '']
>
> So the regular expression seems to be doing the Right Thing.  Is this a
> bug in re.split, or am I missing something?

>From what I can tell, re.split can't split on zero-length boundaries.
It needs something to split on, like str.split.  Is this a bug?
Possibly.  The docs for re.split say:

Split the source string by the occurrences of the pattern,
returning a list containing the resulting substrings.

Note that it does not say that zero-length matches won't work.

I can work around the problem thusly:

re.sub(r'(?<=[a-z])(?=[A-Z])', '_', 'fooBarBaz').split('_')

Which is ugly.  I reckon you can use re.findall with a pattern that
matches the components and not the boundaries, but you have to take
care of the beginning and end as special cases.

Kurt
--
http://mail.python.org/mailman/listinfo/python-list

Re: can multi-core improve single funciton?

[Kurt Smith]

On Fri, Feb 20, 2009 at 4:27 PM, Grant Edwards  wrote:
> On one hand, the upshot of that is that by finding an
> appropriate library module you might gain some of the same
> benefits as removing the GIL.
>
> On the other hand, that doesn't help if you're doing something
> original enough that nobody has written a library to handle
> large chunks of it.
>
> And on the grasping hand, I find that most of us vastly
> overestimate the originality of what we're doing.

+1 The Mote in God's Eye / The Gripping Hand reference!

Kurt
--
http://mail.python.org/mailman/listinfo/python-list

Re: getting object instead of string from dir()

[Kurt Smith]

On Wed, Dec 17, 2008 at 1:52 PM, Rominsky  wrote:

> On Dec 17, 10:59 am, Christian Heimes  wrote:
> > Rominsky schrieb:
> >
> > > I am trying to use dir to generate a list of methods, variables, etc.
> > > I would like to be able to go through the list and seperate the
> > > objects by type using the type() command, but the dir command returns
> > > a list of strings.  When I ask for the type of an element, the answer
> > > is always string.  How do I point at the variables themselves.  A
> > > quick example is:
> >
> > > a = 5
> > > b = 2.0
> > > c = 'c'
> >
> > > lst = dir()
> >
> > > for el in lst:
> > > print type(el)
> >
> > for name, obj in vars().iteritems():
> > print name, obj
> >
> > Christian
>
> I do have some understanding of the pythonic methodology of
> programming, though by far I still don't consider myself an expert.
> The problem at hand is that I am coming from a matlab world and trying
> to drag my coworkers with me.  I have gotten a lot of them excited
> about using python for this work, but the biggest gripe everytime is
> they want their matlab ide.  I am trying to experiment with making
> similar pieces of the ide, in particular I am working on the workspace
> window which lists all the current variables in the namespace, along
> with their type, size, value, etc  I am trying to create a python
> equivalent.  I can get dir to list all the variables names in a list
> of strings, but I am trying to get more info them.  hence the desire


Are you familiar with the ipython console?

http://ipython.scipy.org/moin/

It is quite powerful; in particular, the %who and %whos 'magic functions'
will do much of what you'd like:

[501]$ ipython
Python 2.5.2 (r252:60911, Jul 31 2008, 17:31:22)
Type "copyright", "credits" or "license" for more information.

IPython 0.8.1 -- An enhanced Interactive Python.
?   -> Introduction to IPython's features.
%magic  -> Information about IPython's 'magic' % functions.
help-> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]: a = 'foo'

In [2]: b = 'bar'

In [3]: c = 5.234

In [4]: import os

In [5]: d = os

In [6]: whos
Variable   Type  Data/Info
--
a  str   foo
b  str   bar
c  float 5.234
d  module
os module

In [7]: import numpy as np

In [8]: aa = np.zeros(100)

In [9]: whos
Variable   Type   Data/Info
---
a  strfoo
aa ndarray100: 100 elems, type `float64`, 800 bytes
b  strbar
c  float  5.234
d  module 
np module ages/numpy/__init__.pyc'>
os module 

And I trust you've heard of numpy, scipy and matplotlib?

http://www.scipy.org/

http://matplotlib.sourceforge.net/

 http://numpy.scipy.org/

Cheers,

Kurt
--
http://mail.python.org/mailman/listinfo/python-list

Minor Typo in doc

[Kurt Mueller]

Hi



There is a minor typo in the new doc in:
http://www.python.org/doc/2.6/library/signal.html

--
signal.SIG_DFL¶
This is one of two standard signal handling options;
it will simply perform the default function for the signal.
For example, on most systems the default action for SIGQUIT
is to dump core and exit, while the default action for
SIGCLD
is to simply ignore it.
--

SIGCLD
should be
SIGCHLD


Should I make a bug report in http://bugs.python.org?




Grüessli
-- 
Kurt Müller, m...@problemlos.ch

--
http://mail.python.org/mailman/listinfo/python-list

Re: Minor Typo in doc

[Kurt Mueller]

Steve Holden schrieb:
> Kurt Mueller wrote:
>> Hi
>> There is a minor typo in the new doc in:
>> http://www.python.org/doc/2.6/library/signal.html
>> --
>> signal.SIG_DFL¶
>> This is one of two standard signal handling options;
>> it will simply perform the default function for the signal.
>> For example, on most systems the default action for SIGQUIT
>> is to dump core and exit, while the default action for
>> SIGCLD
>> is to simply ignore it.
>> --
>> SIGCLD
>> should be
>> SIGCHLD
>> Should I make a bug report in http://bugs.python.org

> Yes. The documentation give you a link to follow from the bottom of
> each page.
> regards
> Steve

Yesterday evening I made a bug report on bugs.python.org.
(I opened a issue).
This morning it has already been fixed.
(It has been assigned to benjamin.peterson.
 He has fixed it.
 The issue has been closed.)


> Benjamin Peterson  added the comment:
> Thanks for the report! Fixed in r67848.
> --
> nosy: +benjamin.peterson
> resolution: -> fixed
> status: open -> closed

So, that is a great community!
isn't it?
Thanks to all who make this possible.



Grüessli
-- 
Kurt Mueller

-- 
Kurt Müller, m...@problemlos.ch 


--
http://mail.python.org/mailman/listinfo/python-list

Re: HARD REAL TIME PYTHON

[Kurt Mueller]

Am 07.10.2008 um 11:44 schrieb Diez B. Roggisch:

Kurt Mueller wrote:

David,
As others mentioned before, python is not the right tool for "HARD
REAL TIME".
But: Maybe you can isolate the part of your application that needs
"HARD REAL TIME".
   Then implement this part in an approriate Environment (Language,
OS, HW).
   Then implement the rest of your application which is not "HARD
REAL TIME"
   in python.
I've done this using RTAI + ctypes. Of course the hard realtime  
tasks are

written in C - but only the absolutely minimal core.
Works like a charm.


(Btw, what is this application like)


Yes.
The key is to
!*isolate*!
the part of the application that needs "HARD REAL TIME".
Thats what you have done with the "absolutely minimal core"
in RTAI + ctypes.



Sometimes it is even questionable if an application really
needs "HARD REAL TIME".
sometimes it is enough to have "soft real time" or even
"fast enough" is enough.

Good enough is good enough!

"HARD REAL TIME" is mostly expensive.



Grüessli
--
Kurt Müller, [EMAIL PROTECTED]

--
http://mail.python.org/mailman/listinfo/python-list

Re: HARD REAL TIME PYTHON

[Kurt Mueller]

David,


Am 07.10.2008 um 01:25 schrieb Blubaugh, David A.:

I have done some additional research into the possibility of utilizing
Python for hard real time development.  I have seen on various  
websites

where this has been discussed before on the internet.  However, I was
wondering as to how successful anyone has truly been in developing a
program project either in windows or in Linux that was or extremely
close to real time constraints? For example is it possible to  
develop a

python program that can address an interrupt or execute an operation
within 70 Hz or less?? Are there any additional considerations that I
should investigate first regarding this matter??



As others mentioned before, python is not the right tool for "HARD  
REAL TIME".


But: Maybe you can isolate the part of your application that needs  
"HARD REAL TIME".
  Then implement this part in an approriate Environment (Language,  
OS, HW).
  Then implement the rest of your application which is not "HARD  
REAL TIME"

  in python.

To be more helpful, we should know what you mean by "HARD REAL TIME".
  Do you mean:
  - Handle at least 70 interrupt per second("SPEED")
  - If one fails, this is catastrophic for the application ("HARD")
  - Deliver an response to an interrupt within 5-10[ms]("REAL  
TIME")


see http://en.wikipedia.org/wiki/Real-time


Grüessli
--
Kurt Müller, [EMAIL PROTECTED]

--
http://mail.python.org/mailman/listinfo/python-list

Re: HARD REAL TIME PYTHON

[Kurt Mueller]

Am 08.10.2008 um 06:59 schrieb Hendrik van Rooyen:

"Blubaugh, David A."  wrote:
I have done some additional research into the possibility of  
utilizing
Python for hard real time development.  I have seen on various  
websites

where this has been discussed before on the internet.  However, I was
wondering as to how successful anyone has truly been in developing a
program project either in windows or in Linux that was or extremely
close to real time constraints? For example is it possible to  
develop a

python program that can address an interrupt or execute an operation
within 70 Hz or less?? Are there any additional considerations that I
should investigate first regarding this matter??

[...]

If I run it between 2 PC's, faking the I/O by writing to a disk,
I sometimes get up to 250 such "pings" per second. Going full
duplex would pass a lot more info, but you lose the stimulus-
response nature, which is kind of nice to have in a control
environment.
Its not real time, but its not exactly yesterday's stuff either.


OK, this is gives an impression of SPEED.



We have also used python to do the HMI for an Injection
Moulding machine, talking to a custom controller with
an 8031 and an ARM on it via an RS-422 link running at
115200 - and the python keeps the link fully occupied


In your application the REAL-TIME requirements
are isolated and implemented in the custom controller.
And the HMI which has to be fast enough (SPEED, not HARD-REAL-TIME)
you implemented in python probably on a standard hardware/OS.

That is exactly what I suggested the OP to consider.



So don't be afraid - go for it!


Unless we do not know more details of the requirements
for the OPs application I think it is a bit early
to give this advice, althoug Python is a great
programming language in many aspects.

The only requirements we know of the OPs program project are:
- able to address interrupts (python can)
- execute an operation within 70[Hz] or less (python can)
But we do not know what operation has to be done at this
rate.

My concern is to point out, that the terms
SPEED and REAL-TIME and HARD-REAL-TIME
should not be misused or misunderstood.

Read:
http://en.wikipedia.org/wiki/Real-time


Grüessli
--
Kurt Müller, [EMAIL PROTECTED]

--
http://mail.python.org/mailman/listinfo/python-list

unicode .replace not working - why?

[Kurt Peters]

I'm using the code below to read a pdf document, and it has no line feeds 
or carriage returns in the imported text.  I'm therefore trying to just 
replace the symbol that looks like it would be an end of line (found by 
examining the characters in the "for loop") unichr(167).
  Unfortunately, the replace isn't working, does anyone know what I'm 
doing wrong?  I tried a number of things so I left comments in place as a 
subset of the bunch of things I tried to no avail.

Any help?
Kurt

#!/usr/bin/python
# -*- coding: utf-8 -+-
from pyPdf import PdfFileWriter, PdfFileReader
import unicodedata
fileencoding = "utf-16-LE" #"iso-8859-1" # "utf-8"
doc = PdfFileReader(file(r"C:\Documents and Settings\kpeters\My Documents
\SUA.pdf", "rb")

# print the title of document1.pdf
print "title = %s" % (doc.getDocumentInfo().title)
print "Subject:", doc.getDocumentInfo().subject
print "PDF Version:", doc.getDocumentInfo().producer
page4 = doc.getPage(3)
textu= page4.extractText()
#textu=textu.decode(fileencoding)
print type(textu)
#print type(textu.encode(fileencoding))
#textu=textu.encode(fileencoding)  #Converts to str
fn = unichr(167)
print('The char is %s' % fn)
textu.replace(unichr(167),'\n')
#print unicodedata.bidirectional(fn)   unichr(167)
for i, c in enumerate(textu):
if (i!=302):
print('# %d has char %s, ord: %d , char: %s, category %s, and 
Name: %s' % (i, c, ord(c), unichr(ord(c)), unicodedata.category(c), 
unicodedata.name(c)))

#if (ord(c)==167):
#print('Found it!')
#textu[i]='\n'
print('')
print textu
print textu.encode(fileencoding)
--
http://mail.python.org/mailman/listinfo/python-list

Re: unicode .replace not working - why?

[Kurt Peters]

I had done that about 21 revisions ago.  Nevertheless, why would you think 
that would work, when the code as shown doesn't?
kurt


"Dennis Lee Bieber" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
> On Sat, 11 Oct 2008 15:05:43 -0500, Kurt Peters
> <[EMAIL PROTECTED]> declaimed the following in comp.lang.python:
>
>
>> textu.replace(unichr(167),'\n')
>
> Might I suggest:
>
> textu = textu.replace(fn, "\n") #you already created fn as the character
> -- 
> Wulfraed Dennis Lee Bieber KD6MOG
> [EMAIL PROTECTED] [EMAIL PROTECTED]
> HTTP://wlfraed.home.netcom.com/
> (Bestiaria Support Staff: [EMAIL PROTECTED])
> HTTP://www.bestiaria.com/ 


--
http://mail.python.org/mailman/listinfo/python-list

Re: unicode .replace not working - why?

[Kurt Peters]

Thanks...

  On a side note, do you really think the function call wouldn't interpret 
the unichr before the function call?
Kurt


"Peter Otten" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
> Kurt Peters wrote:
>
>> I had done that about 21 revisions ago.
>
> If you litter your module with code that is commented out it is hard to 
> keep
> track of what works and what doesn't.
>
>> Nevertheless, why would you think
>> that would work, when the code as shown doesn't?
>
> Because he knows Python? Why don't /you/ try it before asking that 
> question?
>
> A good place to do "exploratory" programming is Python's interactive
> interpreter. Here's a sample session:
>
> Python 2.5.1 (r251:54863, Jul 31 2008, 23:17:43)
> [GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> from pyPdf import PdfFileReader as PFR
>>>> doc = PFR(open("SUA.pdf"))
>>>> text = doc.getPage(3).extractText()
>>>> type(text)
> 
>>>> text[:200]
> u'2/16/087400.8P Table of Contents - Continued  Section 
> Page
> \   xa773.49  New Hampshire (NH) 50
> \xa773.50  New Jersey (NJ) 50 \xa773.51  New Mex
> ico (NM) 51 \xa773.52  New York (NY) 56 \xa773.53  North '
>>>> print text[:200].replace(u"\xa7", u"\n")
> 2/16/087400.8P Table of Contents - Continued  Section Page
> 73.49  New Hampshire (NH) 50
> 73.50  New Jersey (NJ) 50
> 73.51  New Mexico (NM) 51
> 73.52  New York (NY) 56
> 73.53  North
>
> Peter 


--
http://mail.python.org/mailman/listinfo/python-list

Re: unicode .replace not working - why?

[Kurt Peters]

Thanks,
  clearly though, my "For loop" shows a character using ord(167), and using 
print repr(textu), it shows the character \xa7 (as does Peter Oten's post). 
So you can see what I see, here's the document I'm using - the Special Use 
Airspace document at
http://www.faa.gov/airports_airtraffic/air_traffic/publications/
which is = JO 7400.8P (PDF)

if you just look at page three, it shows those unusual characters.
Once again, using a "simple" replace, doesn't seem to work.  I can't seem to 
figure out how to get it to work, despite all the great posts attempting to 
shed some light on the subject.

Regards,
Kurt


"John Machin" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
On Oct 12, 7:05 am, Kurt Peters <[EMAIL PROTECTED]> wrote:
> I'm using the code below to read a pdf document, and it has no line feeds
> or carriage returns in the imported text. I'm therefore trying to just
> replace the symbol that looks like it would be an end of line (found by
> examining the characters in the "for loop") unichr(167).
> Unfortunately, the replace isn't working, does anyone know what I'm
> doing wrong? I tried a number of things so I left comments in place as a
> subset of the bunch of things I tried to no avail.

This is the first time I've ever looked inside a PDF file, and *only*
one file, but:

import pyPdf, sys
filename = sys.argv[1]
doc = pyPdf.PdfFileReader(open(filename, "rb"))
for pageno in range(doc.getNumPages()):
page = doc.getPage(pageno)
textu = page.extractText()
print "pageno", pageno
print type(textu)
print repr(textu)

gives me  and text with lots of \n at places where
you'd expect them.

The only problem I can see is that where I see (and expect) quotation
marks (U+201C and U+201D) when viewing the file with Acrobat Reader,
the repr is showing \ufb01 and \ufb02. Similar problems with em-dashes
and apostrophes. I had a bit of a poke around:

1. repr(result of FlateDecode) includes *both* the raw bytes \x93 and
\x94, *and* the octal escapes \\223 and \\224 (which pyPdf translates
into \x93 and \x94).

2. Then pyPdf appears to push these through a fixed transformation
table (_pdfDocEncoding in generic.py) and they become \ufb01 and
\ufb02.

3. However:
|>>> '\x93\x94'.decode('cp1252') # as suspected
|u'\u201c\u201d' # as expected
|>>>

AFAICT there is only one reference to encoding in the pyPdf docs: "if
pyPdf was unable to decode the string's text encoding" ...

Cheers,
John 


--
http://mail.python.org/mailman/listinfo/python-list

Re: 'Hidden Features of Python'

[Kurt Smith]

On Fri, Oct 17, 2008 at 11:48 AM, Joe Strout <[EMAIL PROTECTED]> wrote:
> On Oct 17, 2008, at 10:35 AM, coldpizza wrote:
>
>> If you are using and IDE, such as Eclipse, PyScripter, etc, then CTR
>> +click on 'this' should do the trick.
>> In ipython you can do 'import this' and then type 'this??' Or if you
>> are *not* lazy, you could try locating the file in the Python tree.
>
> Oh!  They're actually talking about a module literally called 'this'!  I
> thought that was just a placeholder, like "foo".
>
> Interesting (and useful).
>
> As for examining the source though, it seems like it could be shortened up
> quite a bit now -- in fact all the source except the assignment to s could
> be replaced with the one-liner
>
>  print s.encode('rot13')

Methinks you miss the point -- read the Zen, and apply its principles
to the "this" module.  I think they managed to break pretty much all
of them, probably to illustrate a point, all in good fun.  Tim Peters
rocks!

For more fun with the Zen, see this thread:

http://mail.python.org/pipermail/python-bugs-list/2008-July/055857.html

Best,

Kurt
--
http://mail.python.org/mailman/listinfo/python-list

Re: unicode .replace not working - why?

[Kurt Peters]

Thanks,
  The "distraction" was my problem.  I replaced the textu.replace as you 
suggested and it works fine.
Kurt

On Sun, 12 Oct 2008 19:53:09 -0700, Mark Tolonen wrote:

> In your original code:
> 
>textu.replace(unichr(167),'\n')
> 
> as Dennis suggested (but maybe you were distracted by his 'fn'
> replacement, so I'll leave it out):
> 
>textu = textu.replace(unichr(167),'\n')
> 
> .replace does not modify the string in place.  It returns the modified
> string, so you have to reassign it.
> 
> -Mark
> 
> "Kurt Peters" <[EMAIL PROTECTED]> wrote in message
> news:[EMAIL PROTECTED]
>> Thanks,
>>  clearly though, my "For loop" shows a character using ord(167), and
>>  using
>> print repr(textu), it shows the character \xa7 (as does Peter Oten's
>> post). So you can see what I see, here's the document I'm using - the
>> Special Use Airspace document at
>> http://www.faa.gov/airports_airtraffic/air_traffic/publications/ which
>> is = JO 7400.8P (PDF)
>>
>> if you just look at page three, it shows those unusual characters. Once
>> again, using a "simple" replace, doesn't seem to work.  I can't seem to
>> figure out how to get it to work, despite all the great posts
>> attempting to shed some light on the subject.
>>
>> Regards,
>> Kurt
>>
>>
>> "John Machin" <[EMAIL PROTECTED]> wrote in message
>> news:42f39e4c-
[EMAIL PROTECTED]
>> On Oct 12, 7:05 am, Kurt Peters <[EMAIL PROTECTED]> wrote:
>>> I'm using the code below to read a pdf document, and it has no line
>>> feeds or carriage returns in the imported text. I'm therefore trying
>>> to just replace the symbol that looks like it would be an end of line
>>> (found by examining the characters in the "for loop") unichr(167).
>>> Unfortunately, the replace isn't working, does anyone know what I'm
>>> doing wrong? I tried a number of things so I left comments in place as
>>> a subset of the bunch of things I tried to no avail.
>>
>> This is the first time I've ever looked inside a PDF file, and *only*
>> one file, but:
>>
>> import pyPdf, sys
>> filename = sys.argv[1]
>> doc = pyPdf.PdfFileReader(open(filename, "rb")) for pageno in
>> range(doc.getNumPages()):
>>page = doc.getPage(pageno)
>>textu = page.extractText()
>>print "pageno", pageno
>>print type(textu)
>>print repr(textu)
>>
>> gives me  and text with lots of \n at places where
>> you'd expect them.
>>
>> The only problem I can see is that where I see (and expect) quotation
>> marks (U+201C and U+201D) when viewing the file with Acrobat Reader,
>> the repr is showing \ufb01 and \ufb02. Similar problems with em-dashes
>> and apostrophes. I had a bit of a poke around:
>>
>> 1. repr(result of FlateDecode) includes *both* the raw bytes \x93 and
>> \x94, *and* the octal escapes \\223 and \\224 (which pyPdf translates
>> into \x93 and \x94).
>>
>> 2. Then pyPdf appears to push these through a fixed transformation
>> table (_pdfDocEncoding in generic.py) and they become \ufb01 and
>> \ufb02.
>>
>> 3. However:
>> |>>> '\x93\x94'.decode('cp1252') # as suspected |u'\u201c\u201d' # as
>> expected
>> |>>>
>>
>> AFAICT there is only one reference to encoding in the pyPdf docs: "if
>> pyPdf was unable to decode the string's text encoding" ...
>>
>> Cheers,
>> John
>>

--
http://mail.python.org/mailman/listinfo/python-list

Re: "xxx.has_key(a)" vs "a in xxx"

[Kurt Smith]

On Thu, Oct 30, 2008 at 9:37 AM, Łukasz Ligowski <[EMAIL PROTECTED]>wrote:

> Hi,
>
> There is small inconsistency (or I don't understand it right) between
> python
> 2.5 docs and python 2.6 docs.
>
> 2.5 docs say that:
>  "a.has_key(k) Equivalent to k in a, use that form in new code"


Meaning: don't use 'a.has_key(k)'.


>
>
> 2.6 docs say that:
>  "dict.has_key(key) is equivalent to key in d, but deprecated."


Meaning, 'd.has_key(key)' is deprecated -- use 'key in d'.


>
>
> which is true?


Both.

[269]$ python2.6 -3
Python 2.6 (r26:66714, Oct  2 2008, 12:46:52)
[GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> D = {'foo':'bar'}
>>> 'foo' in D
True
>>> D.has_key('foo')
__main__:1: DeprecationWarning: dict.has_key() not supported in 3.x; use the
in operator
True
>>>



>
>
> L
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Data Coding suggestions

[Kurt Smith]

On Sat, Feb 28, 2009 at 10:08 AM, steven.oldner  wrote:

>
> Thanks guys.  While shopping today I've thought of a few more columns
> for my data so my first item will be building the 3 DB tables and a
> way to populate them.  Since this was intended to automate what I do
> on a weekly basis, I didn't think about adding recipes since I know
> what I need for the family, but that's a good touch.
>
> Item 1. Build 3 db tables
> Item 2. Build app to populate tables.
> Item 3. Build app to read tables and print lists.
>
> Anything else?

You might take a look at the source code for the Gourmet Recipe Manager

http://grecipe-manager.sourceforge.net/

It's written in python, has a persistent database (not sure if using
sqlite3) and you might be able to adapt it to your needs.

We use it for our shopping lists here and it's great.

Kurt
--
http://mail.python.org/mailman/listinfo/python-list

Re: Indentations and future evolution of languages

[Kurt Smith]

On Tue, Mar 10, 2009 at 12:39 PM,   wrote:
>
>
>    John> The only complaint I have there is that mixing tabs and spaces for
>    John> indentation should be detected and treated as a syntax error.
>
> Guido's time machine strikes again (fixed in Python 3.x):
>
>    % python3.0 ~/tmp/mixed.py
>      File "/home/titan/skipm/tmp/mixed.py", line 3
>        print(a)
>               ^
>    TabError: inconsistent use of tabs and spaces in indentation

Or just use the '-tt' command line switch to force indentation consistency:

ksm...@work:~/tmp [366]$ python2.5 -tt mixed.py
  File "mixed.py", line 6
print a
  ^
TabError: inconsistent use of tabs and spaces in indentation


Kurt
--
http://mail.python.org/mailman/listinfo/python-list

Re: Memory efficient tuple storage

[Kurt Smith]

On Fri, Mar 13, 2009 at 10:59 AM, psaff...@googlemail.com
 wrote:
> I'm reading in some rather large files (28 files each of 130MB). Each
> file is a genome coordinate (chromosome (string) and position (int))
> and a data point (float). I want to read these into a list of
> coordinates (each a tuple of (chromosome, position)) and a list of
> data points.
>
> This has taught me that Python lists are not memory efficient, because
> if I use lists it gets through 100MB a second until it hits the swap
> space and I have 8GB physical memory in this machine. I can use Python
> or numpy arrays for the data points, which is much more manageable.
> However, I still need the coordinates. If I don't keep them in a list,
> where can I keep them?

Assuming your data is in a plaintext file something like
'genomedata.txt' below, the following will load it into a numpy array
with a customized dtype.  You can access the different fields by name
('chromo', 'position', and 'dpoint' -- change to your liking).  Don't
know if this works or not; might give it a try.

===

[186]$ cat genomedata.txt
gene1 120189 5.34849
gene2 84040 903873.1
gene3 300822 -21002.2020

[187]$ cat g2arr.py
import numpy as np

def g2arr(fname):
# the 'S100' should be modified to be large enough for your string field.
dt = np.dtype({'names': ['chromo', 'position', 'dpoint'],
'formats': ['S100', np.int, np.float]})
return np.loadtxt(fname, delimiter=' ', dtype=dt)

if __name__ == '__main__':
arr = g2arr('genomedata.txt')
print arr
print arr['chromo']
print arr['position']
print arr['dpoint']

=

Take a look at the np.loadtxt and np.dtype documentation.

Kurt
--
http://mail.python.org/mailman/listinfo/python-list

Re: Memory efficient tuple storage

[Kurt Smith]

On Fri, Mar 13, 2009 at 11:33 AM, Kurt Smith  wrote:
[snip OP]
>
> Assuming your data is in a plaintext file something like
> 'genomedata.txt' below, the following will load it into a numpy array
> with a customized dtype.  You can access the different fields by name
> ('chromo', 'position', and 'dpoint' -- change to your liking).  Don't
> know if this works or not; might give it a try.

To clarify -- I don't know if this will work for your particular
problem, but I do know that it will read in the array correctly and
cut down on memory usage in the final array size.

Specifically, if you use a dtype with 'S50', 'i4' and 'f8' (see the
numpy dtype docs) -- that's 50 bytes for your chromosome string, 4
bytes for the position and 8 bytes for the data point -- each entry
will use just 50 + 4 + 8 bytes, and the numpy array will have just
enough memory allocated for all of these records.  The datatypes
stored in the array will be a char array for the string, a C int and a
C double; it won't use the corresponding python datatypes which have a
bunch of other memory usage associated with them.

Hope this helps,

Kurt
--
http://mail.python.org/mailman/listinfo/python-list

Re: Memory efficient tuple storage

[Kurt Smith]

On Fri, Mar 13, 2009 at 1:13 PM, psaff...@googlemail.com
 wrote:
> Thanks for all the replies.
>
[snip]
>
> The numpy solution does work, but it uses more than 1GB of memory for
> one of my 130MB files. I'm using
>
> np.dtype({'names': ['chromo', 'position', 'dpoint'], 'formats': ['S6',
> 'i4', 'f8']})
>
> so shouldn't it use 18 bytes per line? The file has 5832443 lines,
> which by my arithmetic is around 100MB...?

I made a mock up file with 5832443 lines, each line consisting of

abcdef 100 100.0

and ran the g2arr() function with 'S6' for the string.  While running
(which took really long), the memory usage spiked on my computer to
around 800MB, but once g2arr() returned, the memory usage went to
around 200MB.  The number of bytes consumed by the array is 105MB
(using arr.nbytes).  From looking at the loadtxt routine in numpy, it
looks like there are a zillion objects created (string objects for
splitting each line, temporary ints floats and strings for type
conversions, etc) while in the routine which are garbage collected
upon return.  I'm not well versed in Python's internal memory
managment system, but from what I understand, practically all that
memory is either returned to the OS or held onto by Python for future
use by other objects after the routine returns.  But the only memory
in use by the array is the ~100MB for the raw data.

Making 5 copies of the array (using numpy.copy(arr)) bumps total
memory usage (from top) up to 700MB, which is 117MB per array or so.
The total memory reported by summing the arr.nbytes is 630MB (105MB /
array), so there isn't that much memory wasted.  Basically, the numpy
solution will pack the data into an array of C structs with the fields
as indicated by the dtype parameter.

Perhaps a database solution as mentioned in other posts would suit you
better; if the temporary spike in memory usage is unacceptable you
could try to roll your own loadtxt function that would be leaner and
meaner.  I suggest the numpy solution for its ease and efficient use
of memory.

Kurt
--
http://mail.python.org/mailman/listinfo/python-list

string processing question

[Kurt Mueller]

Hi,


on a Linux system and python 2.5.1 I have the
following behaviour which I do not understand:



case 1
> python -c 'a="ä"; print a ; print a.center(6,"-") ; b=unicode(a, "utf8"); 
> print b.center(6,"-")'
ä
--ä--
--ä---
>


case 2
- an UnicodeEncodeError in this case:
> python -c 'a="ä"; print a ; print a.center(20,"-") ; b=unicode(a, "utf8"); 
> print b.center(20,"-")' | cat
Traceback (most recent call last):
  File "", line 1, in 
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in position 9: 
ordinal not in range(128)
ä
--ä--
>


The behaviour changes if I pipe the output to another prog or to a file.
and
centering with the string a is not correct, but with string b.



Could somebody please explain this to me?




Thanks in advance
-- 
Kurt Müller, m...@problemlos.ch

--
http://mail.python.org/mailman/listinfo/python-list

Re: string processing question

[Kurt Mueller]

Paul McGuire schrieb:
> --
> Weird. What happens if you change the second print statement to:
> print b.center(6,u"-")
Same behavior.


I have an even more minimal example:


:> python -c 'print unicode("ä", "utf8")'
ä

:> python -c 'print unicode("ä", "utf8")' | cat
Traceback (most recent call last):
  File "", line 1, in 
UnicodeEncodeError: 'ascii' codec can't encode characters in position
0-1: ordinal not in range(128)


Just the difference of having piped the output to another program or to
a file.
Maybe we leave the other issue with the different centering for the moment.

My goal is to have my python programs unicode enabled.





TIA
-- 
Kurt Mueller

-- 
Kurt Müller, m...@problemlos.ch 


--
http://mail.python.org/mailman/listinfo/python-list

Re: string processing question

[Kurt Mueller]

Scott David Daniels schrieb:
> To discover what is happening, try something like:
> python -c 'for a in "ä", unicode("ä"): print len(a), a'
>
> I suspect that in your encoding, "ä" is two bytes long, and in
> unicode it is converted to to a single character.

:> python -c 'for a in "ä", unicode("ä", "utf8"): print len(a), a'
2 ä
1 ä
:>

Yes it is. That is one of the two problems I see.
The solution for this is to unicode(, ) each string.


I'd like to have my python programs unicode enabled.




:> python -c 'for a in "ä", unicode("ä"): print len(a), a'
Traceback (most recent call last):
  File "", line 1, in 
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 0:
ordinal not in range(128)

It seems that the default encoding is "ascii", so unicode() cannot cope
with "ä".
If I specify "utf8" for the encoding, unicode() works.

:> python -c 'for a in "ä", unicode("ä", "utf8"): print len(a), a'
2 ä
1 ä
:> 


But the print statement yelds an UnicodeEncodeError
if I pipe the output to a program or a file.

:> python -c 'for a in "ä", unicode("ä", "utf8"): print len(a), a' | cat
Traceback (most recent call last):
  File "", line 1, in 
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in
position 0: ordinal not in range(128)
2 ä
1 :>


So it seems to me, that piping the output changes the behavior of the
print statement:

:> python -c 'for a in "ä", unicode("ä", "utf8", "ignore"): print a,
len(a), type(a)'
ä 2 
ä 1 

:> python -c 'for a in "ä", unicode("ä", "utf8", "ignore"): print a,
len(a), type(a)'  | cat
Traceback (most recent call last):
  File "", line 1, in 
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in
position 0: ordinal not in range(128)
ä 2 
:>




How can I achieve that my python programs are unicode enabled:
- Input strings can have different encodings (mostly ascii, latin_1 or utf8)
- My python programs should always output "utf8".

Is that a good idea??



TIA
-- 
Kurt Müller, m...@problemlos.ch

--
http://mail.python.org/mailman/listinfo/python-list

Re: string processing question

[Kurt Mueller]

Sion Arrowsmith wrote:
> Kurt Mueller  wrote:
>> :> python -c 'print unicode("ä", "utf8")'
>> ä
>> :> python -c 'print unicode("ä", "utf8")' | cat
>> Traceback (most recent call last):
>> File "", line 1, in 
>> UnicodeEncodeError: 'ascii' codec can't encode characters in position
>> 0-1: ordinal not in range(128)
> $ python -c 'import sys; print sys.stdout.encoding'
> UTF-8
> $ python -c 'import sys; print sys.stdout.encoding' | cat
> None
>
> If print gets a Unicode string, it does an implicit
> .encode(sys.stdout.encoding or sys.getdefaultencoding()) on it.
> If you want your output to be guaranteed UTF-8, you'll need to
> explicitly .encode("utf8") it yourself.

This works now correct with and without piping:

python -c 'a=unicode("ä", "utf8") ; print (a.encode("utf8"))'



In my python source code I have these two lines first:
#!/usr/bin/env python
# vim: set fileencoding=utf-8 :

So the source code itself and the strings in the source code
are interpreted as utf-8.

But from the command line python interprets the code
as 'latin_1' I presume. That is why I have to convert
the "ä" with unicode().
Am I right?


> (I dare say this is slightly different in 3.x .)
I heard about it but I wait to go to 3.x until its time to...




Thanks
-- 
Kurt Mueller

-- 
Kurt Müller, m...@problemlos.ch 


--
http://mail.python.org/mailman/listinfo/python-list

Re: Why there is a parameter named "self" for classmethod function?

[Kurt Symanzik]

Jianchun Zhou  wrote on 2009-05-07 10:49:33 AM 
+0800

I have a sample code as bellow:

#!/usr/bin/env python

class Hello:
def __init__(self):
print "Hello __init__"
@classmethod
def print_hello(self):
print "hello"

Hello.print_hello()

If I move "self" parameter of print_hello away, this code fragment won't 
work.


I am wondering when Hello.print_hello() executes, what value will "self" 
be asigned?


The self variable above with be populated with a reference to the class, 
so it would be more appropriately named cls such as:


@classmethod
def print_hello(cls):
print "hello"

But you might consider decorating the method as a static method instead 
since in your example you are not using the parameter at all.  A static 
method would not require a parameter.


@staticmethod
def print_hello():
print "hello"

Kurt

--
Kurt Symanzik
k...@kbsymanzik.org
Skype id: ksymanzik
http://kbsymanzik.org
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to catch str exception?

[Kurt Symanzik]

"anuraguni...@yahoo.com"  wrote on 2009-05-15 
4:13:15 PM +0800

import sys
try:
raise "xxx"
except str,e:
print "1",e # is not caught here
except:# is caught here
print "2",sys.exc_type,sys.exc_value

In the above code a string exception is raised which though deprecated
but still a 3rd party library I use uses it. So how can I catch such
exception without relying on catch all, which could be bad.

system: Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52) [GCC 4.2.3
(Ubuntu 4.2.3-2ubuntu7)] on linux2


Try this, i.e. catch the exact string:

except "xxx":


Kurt

--
Kurt Symanzik
k...@kbsymanzik.org
Skype id: ksymanzik
http://kbsymanzik.org
--
http://mail.python.org/mailman/listinfo/python-list

Re: Books for learning how to write "big" programs

[Kurt Smith]

On Thu, May 22, 2008 at 10:55 AM, duli <[EMAIL PROTECTED]> wrote:
> Hi:
> I would like recommendations for books (in any language, not
> necessarily C++, C, python) which have walkthroughs for developing
> a big software project ? So starting from inception, problem
> definition, design, coding and final delivery on a single theme
> or application.

The bigger the project, the more likely it is that you'll have
documentation on how to use it (for a language or library, how to use
the features in your program) but to take the time to write up a
dead-tree book on the project's "inception, problem definition,
design, coding and final delivery" is not likely well spent.  Anyone
who has the expertise to write such a book would probably be spending
his time working on the next phase of the project itself.

Someone will probably respond with an amazon link to a book that does
exactly what you're asking, in which case, I will stand corrected.
But I'll be surprised.

>
> Most of the code I have written and books that I have read deal with
> toy programs and I am looking for something a bit more
> comprehensive.  For example, maybe a complete compiler written in C++
> for some language, or a complete web server or implementing
> .net libraries in some language (just a few examples of the scale of
> things I am interested in learning).

It seems to me the reason toy programs are so prevalent is because
they illustrate a (few) well defined ideas in a short amount of code.
A big project, necessarily, brings together all kinds of stuff, much
of which may not interest the author at all, and so doesn't motivate
him to write a book about it.

Compilers, web servers & .NET libraries are *widely* varying areas.
You may have interest in them all, but to significantly contribute to
any requires a fair amount of expertise and specialization.

The best route I've found to learn how to organize & program large
scale applications is this: find a cutting edge program that interests
you and that is open source.  Download its source, and read the code.
Diagram it.  Map it out.  Read the comments.  Join the mailing list
(probably the developer's list), lurk for a while, and ask questions
about why they organized things the way they did.  Get the overall big
picture and learn from it.  Better yet, find out what pitfalls they
found and avoided (or fell into).  Compare their approach &
organization with another competing project.  This is the wonder of
open source software -- you have access to everything, and can learn
from all the expertise the developers put into their opus.

You can learn the basics from books, but nothing beats analyzing a
species in the wild.

Kurt
--
http://mail.python.org/mailman/listinfo/python-list

String split with " and/or ' and/or \

[Kurt Mueller]

How to (super)split a string (literal) containing " and/or ' and/or \.

example:

' a  "  b b   "  c\ c '.supersplit(' ')
->
['a', '  b b   ', 'c c']


Thanks and Grüessli
--
Kurt Müller:
[EMAIL PROTECTED]

--
http://mail.python.org/mailman/listinfo/python-list

Re: String split with " and/or ' and/or \

[Kurt Mueller]

Peter Otten schrieb:

Kurt Mueller wrote:

How to (super)split a string (literal) containing " and/or ' and/or
\.
example:

' a  "  b b   "  c\ c '.supersplit(' ')
->
['a', '  b b   ', 'c c']



import shlex
shlex.split(' a  "  b b   "  c\ c ')

['a', '  b b   ', 'c c']


Thanks Peter
Thanks Paul

shlex is what I was looking for.


Grüessli
--
Kurt Müller, [EMAIL PROTECTED]

--
http://mail.python.org/mailman/listinfo/python-list

Re: Histogram of floating point values.

[Kurt Smith]

On Fri, Jul 25, 2008 at 5:02 PM, aditya shukla
<[EMAIL PROTECTED]> wrote:
> Hello folks,
>
> I have a list say
>
> data=[0.99,0.98,0.98,0.98,0.97,0.93,0.92,0.92,0.83,0.66,0.50,0.50]
>
> i am trying to plot histogram of these values
>
> i have installed numpy and matplotlib  and this is what i am doing*
>  import numpy
>  import pylab
>  from numpy import *
>  from pylab import *
>
> input_hist=array(data)
> pylab.hist(input_hist,bins=0.1)
> and this is the error that i am getting
>
> (array([], dtype=int32), array([ 0.5]), )
>
>
> does this mean that i cannot plot a histogram of floating point values ? or
> is there a way around

the 'bins' argument to pylab.hist() is supposed to be an integer or a
list of the bins' lower edges.  The default value is 10, more than
that gives smaller bins, as one would expect.  Take a look at the
pylab.hist documentation (you can do 'print pylab.hist.__doc__' from
the command interpreter).

You should have no problem plotting a hist of floats.  Try this:

import numpy
import pylab
from numpy import *
from pylab import *

data=[0.99,0.98,0.98,0.98,0.97,0.93,0.92,0.92,0.83,0.66,0.50,0.50]

input_hist=array(data)
pylab.hist(input_hist)
pylab.show()

The last line will display the actual histogram.  See the difference
pylab.show and pylab.ion functions.

In the future, it is advisable to post these questions to the
matplotlib or the numpy/scipy users mailing lists.

Kurt



>
> Thanks in advance
>
> Aditya
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Getting pid of a remote process

[Kurt Mueller]

srinivasan srinivas schrieb:

Thanks a lot.
But i am wondeing will it return correct pid if more than one instance of 
 run on the remote machine??
Thanks,
Srini


On UNIX-like OS:

If you start the process in the background, you can
get the PID with:

:~> ssh  'ls -l & echo PID=$!'  | grep PID
PID=30596
:~>



see:
man bash
-> Special Parameters



Grüessli
--
Kurt Müller, [EMAIL PROTECTED]

--
http://mail.python.org/mailman/listinfo/python-list

Re: Read and write binary data

[Kurt Smith]

On Sun, Sep 7, 2008 at 5:41 PM, Mars creature <[EMAIL PROTECTED]> wrote:
> Hi guys,
>  I am new to Python, and thinking about migrating to it from matlab
> as it is a really cool language. Right now, I am trying to figure out

If you're trying to migrate from matlab to python I'd take a look at numpy:

http://numpy.scipy.org/

And scipy which is built on top of numpy:

http://www.scipy.org/

There is a plotting/numerical computation package known as matplotlib
that does its best to parallel matlab commands:

http://matplotlib.sourceforge.net/

There is support for reading and writing binary data, even fortran records.

And last but not least, you can always take a look at the scipy and
numpy mailing lists.

Good luck,

Kurt

> how to control read and write binary data, like
> 'formatted','stream','big-endian','little-edian' etc.. as in fortran.
> I googled, but can not find a clear answer. Anyone has clue where can
> I learn it? Thanks!!
> Jinbo
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Using jython to call python procedures/methods

[Kurt Smith]

On Wed, Jan 20, 2010 at 9:32 AM, KB  wrote:
> Hi there,
>
> I have an application that only publishes a Java API. I can use jython
> to access java classes, but jython currently (to the best of my
> knowledge) does not support numpy/scipy.
>
> Ideally I would like to have jython call a "native" python routine
> where I have the numpy/scipy procedures already written.
>
> Does anyone have any experience with this? Is it possible?

I have no experience with these technologies, and others can point you
to more detailed info, but you could try using XML-RPC (see the docs
for the xmlrpclib module in the standard library) or SOAP (Google it).

These would be better than rolling your own.

Perhaps there's a more Pythonic solution though?

Kurt

>
> I had toyed with the idea of having jython/java write the data to a
> file/database and then manually kick off the python process, but
> ideally I would like this as automated as possible.
>
> Thanks in advance.
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Over(joy)riding

[Kurt Smith]

On Wed, Feb 17, 2010 at 9:08 AM, mk  wrote:
> Bruno Desthuilliers wrote:
>>
>> mk a écrit :
>>>
>>> P.S. Method resolution order in Python makes me want to kill small
>>> kittens.
>>
>> mro is only a "problem" when using MI.
>
> Oh sure! And I have the impression that multiple inheritance is not used all
> that often. What (some) Python code I've read in open source projects
> typically uses single inheritance.
>
> Nevertheless MI is there in Python, might be useful in some cases and Guido
> put it there for some reason, so Citizens Aspiring To Become Pythonistas
> like me work to learn it.

In case you're not familiar with it, MI allows you to have mixins &
traits.  They work very well if the mixin superclasses don't have any
clashes with the other superclasses, so each mixin adds its own unique
set of methods to the derived class.  Then you don't have to worry
about mro and all that (at least as much).

Here's an article on it, with examples:

http://www.linuxjournal.com/node/4540/print

Michele Simionato has some criticism of MI pitfalls and has come up
with the straits module to remedy it -- you might be interested.  He
goes into detail here:

http://www.artima.com/weblogs/viewpost.jsp?thread=246488

Kurt
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Interesting talk on Python vs. Ruby and how he would like Python to have just a bit more syntactic flexibility.

[Kurt Smith]

On Thu, Feb 18, 2010 at 10:46 PM, Steve Howell  wrote:
> On Feb 18, 2:49 pm, Jonathan Gardner 
> wrote:
>> On Feb 18, 8:15 am, Steve Howell  wrote:
>>
>>
>>
>> >     def print_numbers()
>> >         [1, 2, 3, 4, 5, 6].map { |n|
>> >             [n * n, n * n * n]
>> >         }.reject { |square, cube|
>> >             square == 25 || cube == 64
>> >         }.map { |square, cube|
>> >             cube
>> >         }.each { |n|
>> >             puts n
>> >         }
>> >     end
>>
>> If this style of programming were useful, we would all be writing Lisp
>> today. As it turned out, Lisp is incredibly difficult to read and
>> understand, even for experienced Lispers. I am pleased that Python is
>> not following Lisp in that regard.
>>
>> for n in range(1,6):
>>     square = n*n
>>     cube = n*n*n
>>     if square == 25 or cube == 64: continue
>>     print cube
>
> There's definitely a cognitive dissonance between imperative
> programming and functional programming.  It's hard for programmers
> used to programming in an imperative style to appreciate a functional
> approach, because functional solutions often read "upside down" in the
> actual source code and common algebraic notation:
>
>    def compute_squares_and_cubes(lst):
>        return [(n * n, n * n * n) for n in lst]
>
>    def reject_bad_values(lst):
>        return [(square, cube) for (square, cube) \
>            in lst if not (square == 25 or cube == 64)]
>
>    def cubes_only(lst):
>        return [cube for square, cube in lst]
>
>    def print_results(lst):
>        # 1. compute_squares_and_cubes
>        # 2. reject_bad_values
>        # 3. take cubes_only
>        # 4. print values
>        for item in \
>            cubes_only( # 3
>                reject_bad_values( # 2
>                    compute_squares_and_cubes(lst))): # 1
>            print item # 4
>
> You can, of course, restore the natural order of operations to read
> top-down with appropriate use of intermediate locals:
>
>    def print_results(lst):
>        lst2 = compute_squares_and_cubes(lst)
>        lst3 = reject_bad_values(lst2)
>        lst4 = cubes_only(lst3)
>        for item in lst4:
>            print item
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>

# sent the original to the wrong place -- resending to python-list.

Somewhat off topic, but only somewhat:  you could use coroutines to
get a pipeline effect.

#--8<-
# Shamelessly lifted from David Beazley's
#  http://www.dabeaz.com/coroutines/

def coroutine(co):
   def _inner(*args, **kwargs):
   gen = co(*args, **kwargs)
   gen.next()
   return gen
   return _inner

def squares_and_cubes(lst, target):
   for n in lst:
   target.send((n * n, n * n * n))

@coroutine
def reject_bad_values(target):
   while True:
   square, cube = (yield)
   if not (square == 25 or cube == 64):
   target.send((square, cube))

@coroutine
def cubes_only(target):
   while True:
   square, cube = (yield)
   target.send(cube)

@coroutine
def print_results():
   while True:
   print (yield)

squares_and_cubes(range(10),
   reject_bad_values(
   cubes_only(
   print_results()
   )
   )
   )
#--8<-
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Need advice on starting a Python group

[Kurt Smith]

On Thu, Mar 11, 2010 at 8:57 AM, gb345  wrote:
>
>
>
> I'm hoping to get advice from anyone with prior experience setting
> up a Python group.
>
> A friend of mine and I have been trying to start a
> scientific-programming-oriented Python group in our school (of
> medecine and bio research), with not much success.
>
> The main problem is attendance.  Even though a *ton* of people have
> told us that it's a great idea, that they're *very* interested,
> and have asked to be added to our mailing list, the attendance to
> our first few meeting has never been more than 5, including my
> friend and I.  Last time just he and I showed up.
>
> The second problem is getting content.  The format we'd envisioned
> for this group was centered around code review (though not limited
> to it).  The idea was that at every meeting a different member
> would show some code.  This could be for any of a number of reasons,
> such as, for example, 1) illustrate a cool module or technique; 2)
> present a scientific research problem and how they used Python to
> solve it, or get help solving it; 3) get general feedback (e.g. on
> code clarity, software usability, module architecture, etc.).  But
> in principle just about anything is OK: e.g. a talk on favorite
> Python resources, or a comparison of Python with some other language,
> or an overview of Python gotchas would all be fair game.
>
> Also, we stressed that the talks were not expected to be polished:
> no need for PowerPoint slides, etc.  Just project any old code onto
> the screen, and talk about it, or scribble stuff on the chalkboard.
>
> Still, we have a hard time finding volunteers.
>
> And even when we've had volunteers, hardly anyone shows up!
>
> Any suggestions would be appreciated.
>
> GB
>
> P.S.  There's a Python Meetup we could go to, but it does not fit
> the bill for us: it doesn't meet often enough, it's sort of out of
> the way, and has practically no one doing scientific programming.
> --
> http://mail.python.org/mailman/listinfo/python-list
>

There's a general Scientific Computing interest group that gets
together here at the University of Wisconsin-Madison, and it has a
significant Python component & focus.  They put on a Python bootcamp
this January that was a huge success.

http://hackerwithin.org/cgi-bin/hackerwithin.fcgi/wiki

They have bi-weekly meetings, sometimes it's of the 'come and share on
X topic,' although many times its 'local guest speaker is coming to
speak about Y'.  My impression is that the latter meetings grabbed a
number of people around campus -- 'hey, I need to do Y, I'll see what
the speaker has to say,' and then they started coming for the
show-and-tell meetings.  My recommendation would be to provide
something of value every meeting, the more specific the better.
'Python' in this regard is a bit open ended.  You'd likely get more
involvement if you had meetings that focused on, e.g., parallel
computing (and have examples in python (mpi4py), and have someone come
and talk about MPI or something), or scientific data formats (with
examples of pyhdf5 or pytables...), or you could advertise a tutorial
on some scipy & numpy features and their advantages over using
matlab/octave/idl.

It's more work than show-and-tell meetings, but look at it as priming the pump.

There is much interest around here re: Python in science, but many
have only heard about it, some have dabbled but put it on the shelf,
others couldn't get it to work (they're scientists and used to
prepackaged software that works out of the box -- if it doesn't, it's
somebody else's problem), many others can't justify the time it would
take to learn it when they already have something else working.  Until
something with value comes along (like your meeting with specific
topics) to change their minds, an open-ended meeting won't appeal much
to them.

Just some thoughts, and an example of what's worked here.  Personally
I tend to make it to the meetings with a specific topic, and end up
skipping the ones that are more open-ended.

Kurt
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Glob in python which supports the ** wildcard

[Kurt Mueller]

HI,


Am 22.11.2010 um 23:05 schrieb Stefan Sonnenberg-Carstens:
> Am 22.11.2010 22:43, schrieb Martin Lundberg;
>> 
>> I want to be able to let the user enter paths like this:
>> apps/name/**/*.js
>> and then find all the matching files in apps/name and all its
>> subdirectories. However I found out that Python's glob function
>> doesn't support the recursive ** wildcard. Is there any 3rd party glob
>> function which do support **?
>> 
> os.walk() or os.path.walk() can be used.
> You need to traverse the file system.
> AFAIK there is no support for this.


Or python only:
--
#!/usr/bin/env python
import os, fnmatch
# generator:
def find_files(directory, pattern):
for root, dirs, files in os.walk(directory):
for basename in files:
if fnmatch.fnmatch(basename, pattern):
filename = os.path.join(root, basename)
yield filename
# process each file as it is found:
for filename in find_files('apps/name', '*.js'):
print 'found java source:', filename
--
Found at
http://stackoverflow.com/questions/2186525/use-a-glob-to-find-files-recursively-in-python

Have a nice day
-- 
kurt.alfred.muel...@gmail.com

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Glob in python which supports the ** wildcard

[Kurt Mueller]

Hi,


Am 22.11.2010 um 23:05 schrieb Stefan Sonnenberg-Carstens:
> Am 22.11.2010 22:43, schrieb Martin Lundberg:
>> I want to be able to let the user enter paths like this:
>> apps/name/**/*.js
>> and then find all the matching files in apps/name and all its
>> subdirectories. However I found out that Python's glob function
>> doesn't support the recursive ** wildcard. Is there any 3rd party glob
>> function which do support **?
> os.walk() or os.path.walk() can be used.
> You need to traverse the file system.
> AFAIK there is no support for this.


If you are a lucky Unix/Linux/MacOS user:
---
#!/usr/bin/env python
# find files
import os
cmd = 'find apps/name/ -type f -name "*.js" -print' # find is a standard 
Unix tool
for filename in os.popen(cmd).readlines():  # run find command
# do something with filename 
---

find is very powerful, really.


Have anice day
-- 
kurt.alfred.muel...@gmail.com

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: while True or while 1

[Kurt Mueller]

Am 14.12.2010 11:33, schrieb Hans-Peter Jansen:
> On Tuesday 14 December 2010, 10:19:04 Gregory Ewing wrote:
>> Steven D'Aprano wrote:
>>>>>> while True:
>>>
>>> ... print "Looping"
>>> ... True = 0
>>
>> Just remember that if you use that inside a function, you'll
>> have to initialise True to True before... er, wait a moment,
>> that won't work... ah, I know:
>>
>> def f(true = True):
>> True = true
>> while True:
>> ...
>> True = False
>
> Thankfully, with Python 3 this code falls flat on its face.
>
> If I would have to _consume_ code like that more often,
> it would require me to also use a vomit resistant keyboard cover..
>
> Pete


True yesterday, today and in the future:


Yesterday:
"Pilate said to him, True? what is true?
 Having said this he went out again to the Jews
 and said to them, I see no wrong in him."

Today:
We are so thankful that today we are free
to define "True" ourselves using Python 2.x.

Future:
Be warned, the future gets darker!


;-)


Grüessli
-- 
Kurt Mueller

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to decide between PHP and Python

[Kurt Smith]

On Thu, Jan 6, 2011 at 3:32 PM, Alan Meyer  wrote:
> On 1/5/2011 11:40 AM, Tomasz Rola wrote:
>>
>> On Tue, 4 Jan 2011, Roy Smith wrote:
>>
>>> There.  Now that I've tossed some gasoline on the language wars fire,
>>> I'll duck and run in the other direction :-)
>>
>> May I suggest a better strategy? Run first, duck next :-).
>
> Or more precisely:
>
>   ((run) duck)

If you're going to mock another language, you might as well get it right :-)

If that's Lisp code, it should be:

(funcall (run) duck)

see: http://hornbeck.wordpress.com/2009/07/05/lisp-1-vs-lisp-2/

It'll work unchanged for Scheme, though.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to use Python well?

[Kurt Smith]

On Wed, Feb 16, 2011 at 12:35 PM, snorble  wrote:
> I use Python a lot, but not well. I usually start by writing a small
> script, no classes or modules. Then I add more content to the loops,
> and repeat. It's a bit of a trial and error learning phase, making
> sure I'm using the third party modules correctly, and so on. I end up
> with a working script, but by the end it looks messy, unorganized, and
> feels hacked together. I feel like in order to reuse it or expand it
> in the future, I need to take what I learned and rewrite it from
> scratch.
>
> If I peeked over a Python expert's shoulder while they developed
> something new, how would their habits differ? Do they start with
> classes from the start?
>
> I guess I'm looking for something similar to "Large Scale C++ Software
> Design" for Python. Or even just a walkthrough of someone competent
> writing something from scratch. I'm not necessarily looking for a
> finished product that is well written. I'm more interested in, "I have
> an idea for a script/program, and here is how I get from point A to
> point B."
>
> Or maybe I'm looking for is best practices for how to organize the
> structure of a Python program. I love Python and I just want to be
> able to use it well.

Try this:

http://www.refactoring.com/

Not a silver bullet, but a good place to start.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Tool for browsing python code

[Kurt Smith]

On Tue, Jun 16, 2009 at 7:48 AM, Lucas P Melo wrote:
> Is there any tool for browsing python code? (I'm having a hard time trying
> to figure this out)
> Anything like cscope with vim would be great.

Check out pycscope:

http://pypi.python.org/pypi/pycscope/0.3

I use it myself, and it works fine.  Better than ctags/etags for python.

Kurt
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Converting Python code to C/C++

[Kurt Smith]

On Mon, Jun 22, 2009 at 9:49 PM, Andras
Pikler wrote:
> Hi!
>
>
>
> Short: I need to turn a Python program that I (mostly) wrote into C code,
> and I am at a loss.
>
>
>
> Long: I’m doing research/programming for a professor, and we are working
> with MIDI files (a type of simple music file). The research deals with
> generating variations from a musical melody; currently, my Python code uses
> a Python midi package I found online to read the notes in question from a
> midi file, about 350 lines of my own code to generate a variation based on
> these notes and the professor’s algorithms, and finally the package again to
> write the new melody to another midi file.
>
>
>
> Now, my professor would like to have this exact code in C/C++, as she
> believes C is more compatible with MATLAB, and wants the code to be
> available in multiple languages in case a programmer works for her in the
> future who knows C but not Python. While I know a tiny bit of C (emphasis on
> the tiny), I would much prefer if there were some sort of automatic compiler
> I could use to turn my Python code into C than taking a week or two or three
> to learn the minimum I need about C, find a way to access MIDI files in it,
> and rewrite all of my code.
>
>
>
> After some googling, I found and tried Shedskin, but it doesn’t work, as the
> Python midi package I’m using uses modules which Shedskin does not support.
> Otherwise, I haven’t found much. Is there anything out there to help me do
> this? If not, from anyone who has experience in this regard, how daunting
> should I expect this to be?


Taking on C from a cold start and being able to handle the ins and
outs of interfacing with Python isn't something that's feasible in
'two or three weeks'.  Here are a couple of options -- take 'em or
leave 'em:

1) Put the code in Cython: http://www.cython.org/  (full disclosure:
I'm doing a GSoC project with Cython).  It will convert pretty much
any python code into C code (even closures are supported in the most
recent version, I think), and the C code can then be compiled into an
extension module.

The only problem with the above is the C code isn't, at first blush,
easy to read.  Nor is it supposed to be changed by the user.  So that
leads us to option...

2) Write the core functionality in C yourself, and then wrap those C
functions in Cython.  You'll want to take a look at the documentation:

http://docs.cython.org/

and, more specifically on wrapping C code:

http://docs.cython.org/docs/external_C_code.html

I don't think you'll be able to avoid learning C, though.

Kurt
-- 
http://mail.python.org/mailman/listinfo/python-list

A superclass using a child classes' methods

[Kurt Schwehr]

I'm trying to build an OO system for encoding and decoding
datapackets.  I'd like the parent class to have an encode function
that uses each of the child classes' packing methods.  It appears that
this works for attributes of children accessed by the parent, but not
for methods.  Is that right?  For attributes I found this example,
where the alphabet attribute is set in the child, but used in the
parent.

http://www.pasteur.fr/formation/infobio/python/ch19s04.html

Should I just set an attribute in the child class and then call the
super's functionality making is pull the data from the attribute?

Thanks,
-kurt
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: A superclass using a child classes' methods

[Kurt Schwehr]

Jean-Michel,

Thanks for the excellent response setting me straight.  Now to figure
out what dumb thing I did to make my code not work...

-kurt

On Jun 24, 12:23 pm, Jean-Michel Pichavant 
wrote:
> Kurt Schwehr wrote:
> > I'm trying to build an OO system for encoding and decoding
> > datapackets.  I'd like the parent class to have an encode function
> > that uses each of the child classes' packing methods.  It appears that
> > this works for attributes of children accessed by the parent, but not
> > for methods.
>

[clear example of it working]

>
> Declaring the foo method at the Parent level is not required. But it's a
> good practice: explicit > implicit and it helps to write proper
> documentation.
>
> Jean-Michel

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why not enforce four space indentations in version 3.x?

[Kurt Smith]

On Fri, Jul 10, 2009 at 2:22 PM, walterbyrd wrote:
> I believe Guido himself has said that all indentions should be four
> spaces - no tabs.
>
> Since backward compatibility is being thrown away anyway, why not
> enforce the four space rule?
>
> At least that way, when I get python code from somebody else, I would
> know what I am looking at, without having to do a hex dump, or
> something.

What you propose has already been (forcefully) rejected:

http://www.python.org/dev/peps/pep-0666/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: ANN: Shed Skin 0.2, an experimental (restricted) Python-to-C++ compiler

[Kurt Smith]

On Wed, Jul 22, 2009 at 2:48 AM, Bearophile wrote:
> greg:
>> Posting benchmark times for Pyrex or Cython is pretty
>> meaningless without showing the exact code that was
>> used, since times can vary enormously depending on
>> how much you C-ify things.
>
> Was this link, shown by William, not enough?
> http://hg.flibuste.net/libre/games/cheval/file/46797c3a5136/chevalx.pyx#l1

I took a stab at converting the recent psyco-optimized code to cython,
and got a speedup.

gcj4.3.31.39s
gcc4.3.31.55s
cython 11.2 1.91s
psyco   1.94s
javac  1.5.0_19 2.00s
python 2.5.4168.37s

It was just a matter of cdef-ing all the arrays & integers --
bearophile already did the hard work :-)

Here's the cython code; all the others are from the repo.

#
DEF NMOVES = 8
DEF SIDE = 5
DEF SQR_SIDE = SIDE * SIDE

cdef int circuit[SQR_SIDE]
cdef int nsolutions = 0

cdef int movex[NMOVES]
cdef int movey[NMOVES]
py_movex = [-1,-2,-2,-1,+1,+2,+2,+1]
py_movey = [-2,-1,+1,+2,+2,+1,-1,-2]
for i in range(NMOVES):
   movex[i] = py_movex[i]
   movey[i] = py_movey[i]
shift = [x * SIDE + y for x,y in zip(py_movex, py_movey)]
cdef int shift_0 = shift[0]
cdef int shift_1 = shift[1]
cdef int shift_2 = shift[2]
cdef int shift_3 = shift[3]
cdef int shift_4 = shift[4]
cdef int shift_5 = shift[5]
cdef int shift_6 = shift[6]
cdef int shift_7 = shift[7]

def showCircuit():
   print
   for x in xrange(SIDE):
   x_SIDE = x * SIDE
   for y in xrange(SIDE):
   if SQR_SIDE < 100:
   print "%02d " % circuit[x_SIDE + y],
   else:
   print "%03d " % circuit[x_SIDE + y],
   print

cdef void solve(int nb, int x, int y,
   int SIDE=SIDE, int SQR_SIDE=SQR_SIDE, int *circuit=circuit,
   int shift_0=shift_0,
   int shift_1=shift_1,
   int shift_2=shift_2,
   int shift_3=shift_3,
   int shift_4=shift_4,
   int shift_5=shift_5,
   int shift_6=shift_6,
   int shift_7=shift_7,
   ):
   global nsolutions

   cdef int newx, newy
   cdef int pos = x * SIDE + y
   circuit[pos] = nb
   if nb == SQR_SIDE:
   #showCircuit()
   nsolutions += 1
   circuit[pos] = 0
   return

   newx = x + -1
   if newx >= 0 and newx < SIDE:
   newy = y + -2
   if newy >= 0 and newy < SIDE and not circuit[pos + shift_0]:
   solve(nb+1, newx, newy)

   newx = x + -2
   if newx >= 0 and newx < SIDE:
   newy = y + -1
   if newy >= 0 and newy < SIDE and not circuit[pos + shift_1]:
   solve(nb+1, newx, newy)

   newx = x + -2
   if newx >= 0 and newx < SIDE:
   newy = y + 1
   if newy >= 0 and newy < SIDE and not circuit[pos + shift_2]:
   solve(nb+1, newx, newy)

   newx = x + -1
   if newx >= 0 and newx < SIDE:
   newy = y + 2
   if newy >= 0 and newy < SIDE and not circuit[pos + shift_3]:
   solve(nb+1, newx, newy)

   newx = x + 1
   if newx >= 0 and newx < SIDE:
   newy = y + 2
   if newy >= 0 and newy < SIDE and not circuit[pos + shift_4]:
   solve(nb+1, newx, newy)

   newx = x + 2
   if newx >= 0 and newx < SIDE:
   newy = y + 1
   if newy >= 0 and newy < SIDE and not circuit[pos + shift_5]:
   solve(nb+1, newx, newy)

   newx = x + 2
   if newx >= 0 and newx < SIDE:
   newy = y + -1
   if newy >= 0 and newy < SIDE and not circuit[pos + shift_6]:
   solve(nb+1, newx, newy)

   newx = x + 1
   if newx >= 0 and newx < SIDE:
   newy = y + -2
   if newy >= 0 and newy < SIDE and not circuit[pos + shift_7]:
   solve(nb+1, newx, newy)

   circuit[pos] = 0

def main():
   print "Search for side=%d" % SIDE
   cdef int x,y
   for x in range(SIDE):
   for y in range(SIDE):
   solve(1, x, y);
   print "\n%dx%d case, %d solutions." % (SIDE, SIDE, nsolutions)

def run():
   import time
   s=time.time()
   main()
   print time.time()-s
#
-- 
http://mail.python.org/mailman/listinfo/python-list

Pulling arrays from database for plotting

[Kurt Schwehr]

Hi all,

What's the best way to pull arrays from a database for plotting?
Right now, this is what I do, but can it be done simpler / more
efficiently?

ipython -pylab
import sqlite3
cx = sqlite3.connect('20080407.decimated.db3')
a = array( [tuple(row) for row in cx.execute('SELECT cg_offset,
delta_t_sec FROM bs_time WHERE recvr=2;')] )
x = a[:,0]
y = a[:,1]
plot(x,y)

However, if I try to plot it directly using transpose, it hangs:

import sqlite3
cx = sqlite3.connect('20080407.decimated.db3')
plot(array( [tuple(row) for row in cx.execute('SELECT cg_offset,
delta_t_sec FROM bs_time WHERE recvr=2;')] ).transpose())

It is that plot just doesn't know what to do with a 2D array?  Then I
tried this and it works, but is long and confusing to the uninitiated.

plot(*array( [tuple(row) for row in cx.execute('SELECT cg_offset,
delta_t_sec FROM bs_time WHERE recvr=2;')] ).T)



In [4]: a
Out[4]:
array([[  2.4000e+01,   0.e+00],
   [  2.5000e+01,  -1.e+00],
   [  3.4000e+01,   0.e+00],
   ...,
   [  8.6384e+04,   2.e+01],
   [  8.6394e+04,   2.e+01],
   [  8.6404e+04,   2.e+01]])

In [5]: a.transpose()
Out[5]:
array([[  2.4000e+01,   2.5000e+01,   3.4000e+01, ...,
  8.6384e+04,   8.6394e+04,   8.6404e+04],
   [  0.e+00,  -1.e+00,   0.e+00, ...,
  2.e+01,   2.e+01,   2.e+01]])



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python/Fortran interoperability

[Kurt Smith]

On Sun, Aug 23, 2009 at 9:21 AM, Stefan Behnel wrote:
> n...@cam.ac.uk wrote:
>> I am interested in surveying people who want to interoperate between
>> Fortran and Python to find out what they would like to be able to do
>> more conveniently, especially with regard to types not supported for C
>> interoperability by the current Fortran standard.  Any suggestions as to
>> other ways that I could survey such people (Usenet is no longer as
>> ubiquitous as it used to be) would be welcomed.
>
> You might want to ask also on the Cython, NumPy and SciPy mailing lists.
> NumPy and SciPy have a rather large audience of scientific developers, and
> Cython has a running sub-project on providing better Fortran integration
> (which might be of interest to you anyway).

Thanks for the mention, Stefan.  For those who are interested, here's
my blog summarizing the status of 'fwrap,' a Fortran wrapper utility
for the C, Cython & Python languages.

http://fortrancython.wordpress.com/

Linked there are my talk slides & presentation at last week's SciPy
2009 conference, which gives a good overview.

Kurt
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: An assessment of the Unicode standard

[Kurt Mueller]

Am 01.09.2009 um 09:39 schrieb Terry Reedy:


But this same problem also extends into monies, nation states, units
of measure, etc.


There is, of course, an international system of measure. The US is  
the only major holdout. (I recall Burma, or somesuch, is another.)  
An interesting proposition would be for the US to adopt the metric  
system in exchange for the rest of the world adopting simplified  
basic English as a common language.


The SI-system is nearly universally employed.
Three principal exceptions are Burma (Myanmar), Liberia, and the  
United States.
The United Kingdom has officially adopted the International System of  
Units

but not with the intention of replacing customary measures entirely.
<>


When I was a student, they told us, that in a couple of years there  
will be

the SI-system only, because most countries accepted it in their laws.
So we should adopt it.
That was in the early 70ties.
Only this year we have to deliver results of technical processes to  
british

and US companies. They still want them in their "crazy outdated" units.


The other thing would be the US to adopt a "simplified basic English".
I would not be astonished, that british people would state,
that they already do :-)



Grüessli-- 
http://mail.python.org/mailman/listinfo/python-list

Weekly Python Patch/Bug Summary

[Kurt B. Kaiser]

Patch / Bug Summary
___

Patches :  337 open ( -6) /  2941 closed (+14) /  3278 total ( +8)
Bugs:  908 open ( +0) /  5262 closed (+17) /  6170 total (+17)
RFE :  194 open ( +5) /   187 closed ( +2) /   381 total ( +7)

New / Reopened Patches
__

use LIST_APPEND opcode for X.append() method calls  (2005-09-16)
   http://python.org/sf/1292625  opened by  Neal Norwitz

python.sty correction - verbatim environment  (2005-09-17)
   http://python.org/sf/1293788  opened by  Wojciech Smigaj

python.sty: [EMAIL PROTECTED] correction  (2005-09-17)
   http://python.org/sf/1293790  opened by  Wojciech Smigaj

cjkcodecs does not initialize type pointer  (2005-09-20)
   http://python.org/sf/1297028  opened by  Eric Huss

"Fatal Python error" from cStringIO's writelines  (2005-09-22)
CLOSED http://python.org/sf/1298449  opened by  Andrew Bennetts

Unchecked return value in cStringIO  (2005-09-22)
CLOSED http://python.org/sf/1298499  opened by  Andrew Bennetts

sysmodule.c: realpath() is unsafe  (2005-09-22)
   http://python.org/sf/1298813  opened by  Mihai Ibanescu

vendor-packages directory.  (2005-09-22)
   http://python.org/sf/1298835  opened by  Rich Burridge

Patches Closed
__

Simple webbrowser fix for netscape -remote  (2004-12-02)
   http://python.org/sf/1077979  closed by  birkenfeld

Replacement for webbrowser.py which has problems.  (2004-05-16)
   http://python.org/sf/954628  closed by  birkenfeld

Multiple webbrowser.py bug fixes / improvements  (2003-04-27)
   http://python.org/sf/728278  closed by  birkenfeld

python -c readlink()s and stat()s '-c'  (2005-02-09)
   http://python.org/sf/1119423  closed by  birkenfeld

Bugfix for dircheck() in Objects/fileobject.c  (2004-04-30)
   http://python.org/sf/944928  closed by  birkenfeld

tkinter hello world example bug  (2005-08-31)
   http://python.org/sf/1277677  closed by  birkenfeld

Trivial typo in unicodedata._getcode  (2005-06-03)
   http://python.org/sf/1213831  closed by  loewis

Cache lines in StreamReader.readlines  (2005-08-24)
   http://python.org/sf/1268314  closed by  loewis

locale.getdefaultencoding: LANGUAGE not correctly parsed  (2005-03-20)
   http://python.org/sf/1166938  closed by  doko

locale.getdefaultencoding: precedence of LANGUAGE / LANG  (2005-03-20)
   http://python.org/sf/1166948  closed by  doko

locale: 'utf' is not a known encoding  (2005-03-20)
   http://python.org/sf/1166957  closed by  doko

"Fatal Python error" from cStringIO's writelines  (2005-09-22)
   http://python.org/sf/1298449  closed by  mwh

Unchecked return value in cStringIO  (2005-09-22)
   http://python.org/sf/1298499  closed by  mwh

New / Reopened Bugs
___

Installation of waste by MacPython installer  (2005-09-15)
CLOSED http://python.org/sf/1291662  opened by  Freek Dijkstra

The _ssl build process for 2.3.5 is broken  (2005-09-16)
   http://python.org/sf/1292634  opened by  Robert Cheung

doctest runner cannot handle non-ascii characters   (2005-09-17)
   http://python.org/sf/1293741  opened by  GRISEL

Distutils writes keywords comma-separated  (2005-09-17)
   http://python.org/sf/1294032  opened by  Martijn Pieters

Error in metaclass search order  (2005-09-18)
   http://python.org/sf/1294232  opened by  Pedro Werneck

email.Parser.FeedParser leak  (2005-09-18)
CLOSED http://python.org/sf/1294453  opened by  George Giannakopoulos

Problems with /usr/lib64 builds.  (2005-09-19)
   http://python.org/sf/1294959  opened by  Sean Reifschneider

termios.c in qnx4.25  (2005-09-19)
   http://python.org/sf/1295179  opened by  kbob_ru

cookielib undefined local variable error  (2005-09-19)
CLOSED http://python.org/sf/1295432  opened by  Yusuke Shinyama

expat symbols should be namespaced in pyexpat  (2005-09-19)
   http://python.org/sf/1295808  opened by  Trent Mick

inspect.getsource() misses single line blocks.   (2005-09-19)
   http://python.org/sf/1295909  opened by  Ron Adam

MemoryError in httplib  (2005-09-20)
   http://python.org/sf/1296004  opened by  Yue Zhang

Python/C API Reference Manual, 7.2.1.1 Boolean Objects  (2005-09-20)
CLOSED http://python.org/sf/1296321  opened by  Li Daobing

expat crash python  (2005-09-20)
   http://python.org/sf/1296433  opened by  Mike Rozhnov

Call by object reference sometimes call by value  (2005-09-20)
   http://python.org/sf/1296434  opened by  Alan G

datetime.replace could take a dict  (2005-09-20)
   http://python.org/sf/1296581  opened by  Tom Lynn

Incorrect return type for search() method  (2005-09-21)
   http://python.org/sf/1297059  opened by  Noah Spurrier

hashable and mutable functions  (2005-09-21)
CLOSED http://python.org/sf/1297986  opened by  ChristianJ

shlex does not support unicode  (2005-09-21)
   http://python.org/sf/1298120  opened by  Wai Yip Tung

_socket module not build under cygwin  (2005-09-22

Weekly Python Patch/Bug Summary

[Kurt B. Kaiser]

Patch / Bug Summary
___

Patches :  337 open ( +0) /  2947 closed ( +6) /  3284 total ( +6)
Bugs:  912 open ( +4) /  5278 closed (+16) /  6190 total (+20)
RFE :  195 open ( +1) /   187 closed ( +0) /   382 total ( +1)

New / Reopened Patches
__

fix for distutils "upload" command  (2005-09-23)
   http://python.org/sf/1299675  opened by  Richard Jones

xdrlib.py: pack_fstring() did not use null bytes for padding  (2005-09-23)
CLOSED http://python.org/sf/1300515  opened by  Don Quijote

desktop module (providing startfile as open, all platforms)  (2005-09-23)
   http://python.org/sf/1301512  opened by  Paul Boddie

Update documentation for __builtins__  (2005-09-24)
   http://python.org/sf/1303595  opened by  Collin Winter

Strobe Timer for guis  (2005-09-26)
   http://python.org/sf/1304895  opened by  Brett Rosen

pyexpat.c: Two line fix for decoding crash  (2005-09-29)
   http://python.org/sf/1309009  opened by  Evan Jones

Patches Closed
__

cjkcodecs does not initialize type pointer  (2005-09-20)
   http://python.org/sf/1297028  closed by  nnorwitz

xdrlib.py: pack_fstring() did not use null bytes for padding  (2005-09-23)
   http://python.org/sf/1300515  closed by  birkenfeld

vendor-packages directory.  (2005-09-22)
   http://python.org/sf/1298835  closed by  richburridge

PyOS_InputHook inconsistency on Windows  (2004-10-19)
   http://python.org/sf/1049855  closed by  loewis

FreeBSD 5.3 and upward has thread-safe getaddrinfo(3)  (2005-09-13)
   http://python.org/sf/1288833  closed by  perky

use LIST_APPEND opcode for X.append() method calls  (2005-09-16)
   http://python.org/sf/1292625  closed by  nnorwitz

New / Reopened Bugs
___

A command history for the idle interactive shell  (2005-09-24)
   http://python.org/sf/1302267  opened by  Björn Lindqvist

2.4.1 windows MSI has no _socket  (2005-09-24)
   http://python.org/sf/1302793  opened by  IamPaul

SSL-ed sockets don't close correct?  (2004-06-24)
   http://python.org/sf/978833  reopened by  kxroberto

Bypassing __dict__ readonlyness  (2005-09-24)
   http://python.org/sf/1303614  opened by  Armin Rigo

traceback on trying to load a file  (2005-09-24)
   http://python.org/sf/1303673  opened by  nanotube

Extended slice bug (or feature?)  (2005-09-25)
CLOSED http://python.org/sf/1303928  reopened by  mwh

Extended slice bug (or feature?)  (2005-09-25)
CLOSED http://python.org/sf/1303928  opened by  David M. Beazley

2.3.5 configure / make infinite loop  (2005-09-25)
   http://python.org/sf/1304179  opened by  Michael Schwartz

windows unicode write is messing up the EOL.  (2005-09-27)
CLOSED http://python.org/sf/1305706  opened by  Istvan Visegradi

Syntax error in Python Library Reference, 6.1.4. Files ...  (2005-09-27)
CLOSED http://python.org/sf/1306043  opened by  Alexander Schliep

Python 2.4.2c1 fails to build for 64-bit Solaris 9/10  (2005-09-27)
CLOSED http://python.org/sf/1306211  opened by  John Stone

Add 64-bit Solaris 9 build instructions to README  (2005-09-27)
   http://python.org/sf/1306248  opened by  John Stone

Python 2.4.2c1 fails to build on 64-bit Solaris 10  (2005-09-27)
   http://python.org/sf/1306253  opened by  John Stone

PyString_AsStringAndSize() return value documented wroing  (2005-09-28)
CLOSED http://python.org/sf/1306449  opened by  Gregory Bond

compile() converts "filename" parameter to StringType  (2005-09-28)
   http://python.org/sf/1306484  opened by  Vágvölgyi Attila

Augmented assigment to mutable objects in tuples fail  (2005-09-28)
CLOSED http://python.org/sf/1306777  opened by  Mattias Engdegård

=, not =  (2005-09-28)
CLOSED http://python.org/sf/1307188  opened by  Davide Andrea

Datagram Socket Timeouts  (2005-09-28)
   http://python.org/sf/1307357  opened by  Tom Vrankar

subprocess.Popen locks on Cygwin  (2005-09-29)
   http://python.org/sf/1307798  opened by  Jacek Poplawski

PCbuild vcproj project files need a cleanup  (2005-09-29)
   http://python.org/sf/1307806  opened by  Adal Chiriliuc

Unsatisfied symbols: _PyGILState_NoteThreadState (code)  (2005-09-29)
   http://python.org/sf/1307978  opened by  Goetz Isenmann

Datagram Socket Timeouts  (2005-09-29)
   http://python.org/sf/1308042  opened by  Tom Vrankar

Py_BuildValue (C/API): "K" format  (2005-09-29)
CLOSED http://python.org/sf/1308740  opened by  Yair Chuchem

Bugs Closed
___

Incorrect return type for search() method  (2005-09-20)
   http://python.org/sf/1297059  closed by  nnorwitz

build fails on Solaris 10 for Objects/complexobject.c  (2005-09-22)
   http://python.org/sf/1299187  closed by  nnorwitz

test_ioctl fails  (2003-07-26)
   http://python.org/sf/777867  closed by  birkenfeld

test_ioctl fails  (2003-07-31)
   http://python.org/sf/780576  closed by  birkenfeld

subprocesss module retains older license header  (2005-02-17)
   http:/