Re: KeyboardInterrupt catch does not shut down the socketserver

[Igor Katson]

Gabriel Genellina wrote:

En Fri, 15 May 2009 09:04:05 -0300, Igor Katson escribió:

Lawrence D'Oliveiro wrote:
In message , 
Igor Katson wrote:

Lawrence D'Oliveiro wrote:
In message , 
Igor Katson wrote:



I have problems in getting a SocketServer to shutdown.


Do you want to do a shutdown or a close?


I want the server close the socket ...



You want to do a close, do a close, not a shutdown 
.



Shutdown implies closing the listening socket, doesn't it?


No (perhaps it should, but that is another issue). There is a
documentation bug; BaseServer.shutdown is documented as "Tells the
serve_forever() loop to stop and waits until it does." [1]
The docstring is much more explicit: """Stops the serve_forever loop.
Blocks until the loop has finished. This must be called while
serve_forever() is running in another thread, or it will deadlock."""

So, if you have a single-threaded server, *don't* use shutdown(). And, to
orderly close the listening socket, use server_close() instead. Your code
would become:

   from SocketServer import TCPServer, BaseRequestHandler

server = TCPServer(('localhost',1234), BaseRequestHandler)
try:
  server.serve_forever()
except KeyboardInterrupt:
  print "^C detected"
  pass
finally:
  print "server_close()"
  server.server_close()
print "bye"

(I've opened http://bugs.python.org/issue6031 )

[1]
http://docs.python.org/dev/library/socketserver.html?highlight=baseserver#SocketServer.BaseServer.shutdown 



Hmm. Gabriel, could you please show the same for the threaded version? 
This one deadlocks:


from SocketServer import TCPServer, BaseRequestHandler
from threading import Thread

server = TCPServer(('localhost',1234), BaseRequestHandler)
try:
   run_thread = Thread(target=server.serve_forever)
   run_thread.start()
   run_thread.join()
except KeyboardInterrupt:
   print "^C detected"
   pass
finally:
   print "server_shutdown()"
   kill_thread = Thread(target=server.shutdown)
   kill_thread.start()
print "bye"
--
http://mail.python.org/mailman/listinfo/python-list

Re: Appending traceback from exception in child thread

[Diez B. Roggisch]

Edd schrieb:

Hi folks,

I have a some threadpool code that works like this :

tp = ThreadPool(number_of_threads)
futures = [tp.future(t) for t in tasks] # each task is callable
for f in futures:
print f.value() # <-- may propagate an exception

The idea being that a Future object represents a value that may or may
not have been computed yet and the .value() method will block until
that value is ready.

As the comment on the last line indicates, looking at the .value() of
a Future may give the return value of the associated task, or it may
also propagate an exception that was raised in a child thread.

Inside the implementation I store caught exceptions with code that
looks something like:

try:
self.return_value = self.task()
except:
self.exception = sys.exc_info()[1]

The problem I have is that when an exception is propagated in to the
parent thread by re-raising it, the part of the traceback from the
child thread is lost. So if the program dies due to such an exception,
I can't see what caused it by examining the printed traceback.

To solve this problem, I could of course grab the traceback in the
child thread and create some kind of custom exception that stashes the
trace inside. This could then be caught on the fringes of my code in
order to combine the tracebacks of parent and child together before
printing it. But I was wondering if there might be a nicer way that
keeps the original exception object. Perhaps something akin to gluing
the tracebacks together at the point where the exception is re-raised?


If you are only interested in the original traceback, you can get that 
with the traceback.format function.


Then you don't need to re-raise it in the main-thread, or if so, use the 
formatted stacktrace as payload to an exception.


Diez
--
http://mail.python.org/mailman/listinfo/python-list

Re: Appending traceback from exception in child thread

[Edd]

On May 16, 4:02 am, "Gabriel Genellina" 
wrote:

> You could use the 3-argument form of the raise 
> statement:http://docs.python.org/reference/simple_stmts.html#the-raise-statement

Ah! When did that get there? :)

> There is a problem: remember that the traceback object keeps a reference  
> to all previous frames -- that means, all local variables along the whole  
> stack of called functions are kept alive until you delete the traceback  
> object. And depending on your application, that might involve thousand of  
> objects "artificially" alive for an undeterminate period of time...

Yes, that may be problematic, I'll have to experiment.

Thanks to everyone's help here, I have plenty of alternatives if it
turns out to be problematic. Your comments are all greatly appreciated
-- thanks!

Kind regards,

Edd
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Need advice on distributing small module

[Nick Craig-Wood]

kj  wrote:
>  The module has only one non-standard dependency, described by the
>  following code:
> 
>  if sys.version_info[:2] >= (2, 6):
>  import json
>  else:
>  import simplejson as json

I think

try:
import json
except ImportError:
import simplejson as json

Is more pythonic...  You aren't relying on what came with particular
python versions which may not be true in jython/ironpython/etc.

-- 
Nick Craig-Wood  -- http://www.craig-wood.com/nick
-- 
http://mail.python.org/mailman/listinfo/python-list

How to get Exif data from a jpeg file

[Arnaud Delobelle]

Hi all,

I need to get the creation date from a jpeg file in Python.  Googling
brought up a several references to apparently defunct modules.  The best
way I have been able to find so far is something like this:

from PIL import Image
img = Image.open('img.jpg')
exif_data = img._getexif()
creation_date = exif_data[36867]

Where 36867 is the exif tag for the creation date data (which I found by
ooking at PIL.ExifTags.TAGS).  But this doesn't even seem to be
documented in the PIL docs.  Is there a more natural way to do this?

Thanks.

-- 
Arnaud
-- 
http://mail.python.org/mailman/listinfo/python-list

using Winpdb in an embedded python interpreter

[Adrian Genaid]

Hi there, 

I have a problem using Winpdb in Pythonscripts running in a program
which uses an embedded Python interpreter. After linking all needed
modules statically to python, "import rpdb2" can be done and leads to
no error.
But on using the next line "rpdb2.start_embedded_debugger('password')".
an error occurs:

Exception in thread ioserver:
Traceback (most recent call last):
  File "C:\Program Files\Python30\lib\threading.py", line 507, in
_bootstrap_inn er
self.run()
  File "S:\dev\adge\build\r3804\Debug_Win32\py\rpdb2.py", line 4570, in
run threading.Thread.run(self)
  File "C:\Program Files\Python30\lib\threading.py", line 462, in run
self._target(*self._args, **self._kwargs)
  File "S:\dev\adge\build\r3804\Debug_Win32\py\rpdb2.py", line 9639, in
run (self.m_port, self.m_server) = self.__StartXMLRPCServer()
  File "S:\dev\adge\build\r3804\Debug_Win32\py\rpdb2.py", line 9730, in
__StartX MLRPCServer
server = CXMLRPCServer((host, port), logRequests = 0)
  File "C:\Program Files\Python30\lib\xmlrpc\server.py", line 530, in
__init__ socketserver.TCPServer.__init__(self, addr, requestHandler,
bind_and_activat e)
  File "C:\Program Files\Python30\lib\socketserver.py", line 400, in
__init__ self.server_bind()
  File "C:\Program Files\Python30\lib\socketserver.py", line 411, in
server_bind

self.socket.bind(self.server_address)
LookupError: unknown encoding: idna

Does someone have an idea why this error occurs?

Funny thing is that module "socketserver" is taken from "C:\Program
Files\Python30\lib\xmlrpc\server.py", although the module search path
does not contain "C:\Program Files\Python30\lib". Maybe this has
something to do with the problem...

Thanks,
Adrian
-- 
http://mail.python.org/mailman/listinfo/python-list

Options for creating a statistics screen on a Mac with Python?

[Phillip B Oldham]

I've come into possession of a mac mini and a large LCD tv at the
office. I'd like to set it up in the corner to pull statistics from
our various servers (load, uptimes, etc) and display them in a
graphical format, full-screen, with a reasonable refresh rate (say
every 30 seconds). There will be quite a few statistics, and we'll be
adding new servers over time, and since there won't be the option to
scroll a full-screen web-page won't work.

What then would be my options to create a full-screen display of
statistics, and an attractive graphical way, using Python on OS X?
-- 
http://mail.python.org/mailman/listinfo/python-list

open(os.path.join(os.path.dirname(__file__), '../www/bin/picture.png'), 'rb')

[gert]

open(os.path.join(os.path.dirname(__file__),'../www/bin/picture.png'),
'rb')
how do you do this on windows (py3) so it still works on linux ?
-- 
http://mail.python.org/mailman/listinfo/python-list

Generic web parser

[S.Selvam]

Hi all,

I have to design web parser which will visit the given list of websites and
need to fetch a particular set of details.
It has to be so generic that even if we add new websites, it must fetch
those details if available anywhere.
So it must be something like a framework.

Though i have done some parsers ,but they will parse for a given format(For.
eg It will get the data from  tag).But here each website may have
different format and the information may available within any tags.

I know its a tough task for me,but i feel with python it should be possible.
My request is, if such thing is already available please let me know ,also
your suggestions are welcome.

Note: I planned to use BeautifulSoup for parsing.

-- 
Yours,
S.Selvam
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Options for creating a statistics screen on a Mac with Python?

[Diez B. Roggisch]

Phillip B Oldham schrieb:

I've come into possession of a mac mini and a large LCD tv at the
office. I'd like to set it up in the corner to pull statistics from
our various servers (load, uptimes, etc) and display them in a
graphical format, full-screen, with a reasonable refresh rate (say
every 30 seconds). There will be quite a few statistics, and we'll be
adding new servers over time, and since there won't be the option to
scroll a full-screen web-page won't work.

What then would be my options to create a full-screen display of
statistics, and an attractive graphical way, using Python on OS X?


matplotlib should serve you well. Or you use some web-based solution 
with a javascript-based chart library. This might be even a bit easier.



Diez
--
http://mail.python.org/mailman/listinfo/python-list

Re: open(os.path.join(os.path.dirname(__file__), '../www/bin/picture.png'), 'rb')

[Diez B. Roggisch]

gert schrieb:

open(os.path.join(os.path.dirname(__file__),'../www/bin/picture.png'),
'rb')
how do you do this on windows (py3) so it still works on linux ?


os.path.join("..", "www", "bin", "picture.png")

Or use os.sep.

Diez
--
http://mail.python.org/mailman/listinfo/python-list

Re: open(os.path.join(os.path.dirname(__file__), '../www/bin/picture.png'), 'rb')

[MRAB]

Diez B. Roggisch wrote:

gert schrieb:

open(os.path.join(os.path.dirname(__file__),'../www/bin/picture.png'),
'rb')
how do you do this on windows (py3) so it still works on linux ?


os.path.join("..", "www", "bin", "picture.png")

Or use os.sep.


Windows should understand '/' as well as '\'.
--
http://mail.python.org/mailman/listinfo/python-list

Re: open(os.path.join(os.path.dirname(__file__), '../www/bin/picture.png'), 'rb')

[gert]

On May 16, 3:16 pm, "Diez B. Roggisch"  wrote:
> gert schrieb:
>
> > open(os.path.join(os.path.dirname(__file__),'../www/bin/picture.png'),
> > 'rb')
> > how do you do this on windows (py3) so it still works on linux ?
>
> os.path.join("..", "www", "bin", "picture.png")
>
> Or use os.sep.
>
> Diez

IOError: [Errno 2] No such file or directory: 'C:\\Users\\gert\\Desktop
\\koen\\a
ppwsgi/wsgi\\..\\www\\bin\\picture.png'

The os.path.dirname(__file__) is still in posix mode ?

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: open(os.path.join(os.path.dirname(__file__), '../www/bin/picture.png'), 'rb')

[gert]

On May 16, 3:40 pm, gert  wrote:
> On May 16, 3:16 pm, "Diez B. Roggisch"  wrote:
>
> > gert schrieb:
>
> > > open(os.path.join(os.path.dirname(__file__),'../www/bin/picture.png'),
> > > 'rb')
> > > how do you do this on windows (py3) so it still works on linux ?
>
> > os.path.join("..", "www", "bin", "picture.png")
>
> > Or use os.sep.
>
> > Diez
>
> IOError: [Errno 2] No such file or directory: 'C:\\Users\\gert\\Desktop
> \\koen\\a
> ppwsgi/wsgi\\..\\www\\bin\\picture.png'
>
> The os.path.dirname(__file__) is still in posix mode ?

DOH!!! forgot to actually put the file in the directory :)
works both ways never mind the question, thanks.
-- 
http://mail.python.org/mailman/listinfo/python-list

xml menu pygtk

[Seb]

I'm trying to construct a menu from an xml file. However my recursive
algorithm isn't doing what I want it too. I've been starring at this
for too long. Any help appreciated :)

I get the following error but the problem is more of a logical nature.
./gnomeAppletMenu.py:40: GtkWarning: gtk_menu_shell_insert: assertion
`GTK_IS_MENU_ITEM (child)' failed
  menu_bar.append(menu)

I'm pretty sure I fuck up around line 27-30

xml file:
http://pastie.org/480045

python code:
http://pastie.org/480042


best regards,
Seb
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: xml menu pygtk

[MRAB]

Seb wrote:

I'm trying to construct a menu from an xml file. However my recursive
algorithm isn't doing what I want it too. I've been starring at this
for too long. Any help appreciated :)

I get the following error but the problem is more of a logical nature.
./gnomeAppletMenu.py:40: GtkWarning: gtk_menu_shell_insert: assertion
`GTK_IS_MENU_ITEM (child)' failed
  menu_bar.append(menu)

I'm pretty sure I fuck up around line 27-30

xml file:
http://pastie.org/480045

python code:
http://pastie.org/480042


The traceback suggests to me that you can add only a 'MenuItem' to a
'MenuBar', but create_menu() is returning a 'Menu'.
--
http://mail.python.org/mailman/listinfo/python-list

Re: xml menu pygtk

[Seb]

On May 16, 4:20 pm, MRAB  wrote:
> Seb wrote:
> > I'm trying to construct a menu from an xml file. However my recursive
> > algorithm isn't doing what I want it too. I've been starring at this
> > for too long. Any help appreciated :)
>
> > I get the following error but the problem is more of a logical nature.
> > ./gnomeAppletMenu.py:40: GtkWarning: gtk_menu_shell_insert: assertion
> > `GTK_IS_MENU_ITEM (child)' failed
> >   menu_bar.append(menu)
>
> > I'm pretty sure I fuck up around line 27-30
>
> > xml file:
> >http://pastie.org/480045
>
> > python code:
> >http://pastie.org/480042
>
> The traceback suggests to me that you can add only a 'MenuItem' to a
> 'MenuBar', but create_menu() is returning a 'Menu'.

Doesn't a MenuBar consists of Menu's?

best regards,
Seb
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Options for creating a statistics screen on a Mac with Python?

[Phillip B Oldham]

On May 16, 2:15 pm, "Diez B. Roggisch"  wrote:
> matplotlib should serve you well.

Thanks, I'll check that out.

> Or you use some web-based solution with a javascript-based chart library.

I'd rather stay away from web-based solutions; since this is going to
be running 24/7 and has a fixed a screen size. I'd prefer something
that has few dependencies, if possible hooking into the OS X platform
for graphics.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: xml menu pygtk

[MRAB]

Seb wrote:

On May 16, 4:20 pm, MRAB  wrote:

Seb wrote:

I'm trying to construct a menu from an xml file. However my recursive
algorithm isn't doing what I want it too. I've been starring at this
for too long. Any help appreciated :)
I get the following error but the problem is more of a logical nature.
./gnomeAppletMenu.py:40: GtkWarning: gtk_menu_shell_insert: assertion
`GTK_IS_MENU_ITEM (child)' failed
  menu_bar.append(menu)
I'm pretty sure I fuck up around line 27-30
xml file:
http://pastie.org/480045
python code:
http://pastie.org/480042

The traceback suggests to me that you can add only a 'MenuItem' to a
'MenuBar', but create_menu() is returning a 'Menu'.


Doesn't a MenuBar consists of Menu's?


I've had a look at some examples on the web and it looks like you need
to:

1. Set the menu on a menu item (the 'set_submenu' method);
2. Append the menu item onto the menu bar.

What you're actually doing is:

1. Setting the menu on a menu item;
2. Appending the menu item onto a menu;
3. Trying to append the menu onto the menu bar.

Result: exception.

HTH
--
http://mail.python.org/mailman/listinfo/python-list

Re: xml menu pygtk

[Seb]

On May 16, 5:04 pm, MRAB  wrote:
> Seb wrote:
> > On May 16, 4:20 pm, MRAB  wrote:
> >> Seb wrote:
> >>> I'm trying to construct a menu from an xml file. However my recursive
> >>> algorithm isn't doing what I want it too. I've been starring at this
> >>> for too long. Any help appreciated :)
> >>> I get the following error but the problem is more of a logical nature.
> >>> ./gnomeAppletMenu.py:40: GtkWarning: gtk_menu_shell_insert: assertion
> >>> `GTK_IS_MENU_ITEM (child)' failed
> >>>   menu_bar.append(menu)
> >>> I'm pretty sure I fuck up around line 27-30
> >>> xml file:
> >>>http://pastie.org/480045
> >>> python code:
> >>>http://pastie.org/480042
> >> The traceback suggests to me that you can add only a 'MenuItem' to a
> >> 'MenuBar', but create_menu() is returning a 'Menu'.
>
> > Doesn't a MenuBar consists of Menu's?
>
> I've had a look at some examples on the web and it looks like you need
> to:
>
> 1. Set the menu on a menu item (the 'set_submenu' method);
> 2. Append the menu item onto the menu bar.
>
> What you're actually doing is:
>
> 1. Setting the menu on a menu item;
> 2. Appending the menu item onto a menu;
> 3. Trying to append the menu onto the menu bar.
>
> Result: exception.
>
> HTH


Ok, now it looks like below. I don't get any errors but the dictionary
entry show up as a toplevel menu... damn!!

def create_menu(node):
menus = []
for child in node.childNodes:
if child.localName == "item":
menus.append(gtk.MenuItem(child.getAttribute("name")))
if child.localName == "seperator":
menus.append(gtk.SeparatorMenuItem())
if child.localName == "menu": #if child.childNodes:
menuitem = gtk.MenuItem(child.getAttribute("name"))
menu = gtk.Menu()
for mi in create_menu(child):   # for each menuitem
menu.append(mi) # append each menuitem 
to menu
menuitem.set_submenu(menu)  # set menu as submenu 
of menuitem
menus.append(menuitem)
return menus

def factory(applet, iid):
doc = minidom.parse("menu.xml")
rootNode = doc.documentElement

menu_bar = gtk.MenuBar()
for cn in rootNode.childNodes:  # for each menu under menus
for menu in create_menu(cn):# for each menu in list
menu_bar.append(menu)   # append each menu

applet.add(menu_bar)
applet.show_all()
return True
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Parsing Strings in Enclosed in Curly Braces

[Aahz]

In article <180531ca-33aa-47b9-9c69-5b5973f6b...@v35g2000pro.googlegroups.com>,
John Machin   wrote:
>
>Neat trick. However, from 2.6.2:
>
 help(sum)
>Help on built-in function sum in module __builtin__:
>
>sum(...)
>sum(sequence[, start]) -> value
>
>Returns the sum of a sequence of numbers (NOT strings) plus the
>value
>of parameter 'start' (which defaults to 0).  When the sequence is
>empty, returns start.
>
>Since when is a list a number? Perhaps the help needs clarification,
>in line with the docs.

The primary use-case for sum() is numbers, with a special exception to
prohibit using strings.  Only strings are prohibited to allow using sum()
with user-defined classes.  That makes it a little difficult to document
precisely in a summary; unless you can come up with a specific better
wording, the docs will probably stay as-is.  If you do come up with
something better, please file it on bugs.python.org.
-- 
Aahz (a...@pythoncraft.com)   <*> http://www.pythoncraft.com/

"In 1968 it took the computing power of 2 C-64's to fly a rocket to the moon.
Now, in 1998 it takes the Power of a Pentium 200 to run Microsoft Windows 98.
Something must have gone wrong."  --/bin/fortune
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: xml menu pygtk

[Seb]

On May 16, 5:04 pm, MRAB  wrote:
> Seb wrote:
> > On May 16, 4:20 pm, MRAB  wrote:
> >> Seb wrote:
> >>> I'm trying to construct a menu from an xml file. However my recursive
> >>> algorithm isn't doing what I want it too. I've been starring at this
> >>> for too long. Any help appreciated :)
> >>> I get the following error but the problem is more of a logical nature.
> >>> ./gnomeAppletMenu.py:40: GtkWarning: gtk_menu_shell_insert: assertion
> >>> `GTK_IS_MENU_ITEM (child)' failed
> >>>   menu_bar.append(menu)
> >>> I'm pretty sure I fuck up around line 27-30
> >>> xml file:
> >>>http://pastie.org/480045
> >>> python code:
> >>>http://pastie.org/480042
> >> The traceback suggests to me that you can add only a 'MenuItem' to a
> >> 'MenuBar', but create_menu() is returning a 'Menu'.
>
> > Doesn't a MenuBar consists of Menu's?
>
> I've had a look at some examples on the web and it looks like you need
> to:
>
> 1. Set the menu on a menu item (the 'set_submenu' method);
> 2. Append the menu item onto the menu bar.
>
> What you're actually doing is:
>
> 1. Setting the menu on a menu item;
> 2. Appending the menu item onto a menu;
> 3. Trying to append the menu onto the menu bar.
>
> Result: exception.
>
> HTH

I managed to get it to work. I thanks a lot for your help.

best regards,
Seb
-- 
http://mail.python.org/mailman/listinfo/python-list

tarfile doesn't work with tgz files?

[Robert Dailey]

Hi,

I'm not a big expert on the tarfile component, but I assumed that .tgz
files were short for .tar.gz and the format was the same. When I try
to extract a .tgz file using tarfile in Python 3.0 on Windows, I get
the following error:

  File "C:\Python30\lib\tarfile.py", line 1630, in open
raise ReadError("file could not be opened successfully")
tarfile.ReadError: file could not be opened successfully

I'm assuming this is because the .tgz format is not what I think it is
and is not supported. How can I extract .tgz files using Python 3.0?
Note that I probably can't depend on third party python modules, since
I have yet to find many that support Python 3.0.

Thanks.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Case insensitive version of string.Template?

[python]

Gabriel,


If you can guarantee the identifiers are all lowercase inside the  
dictionary:

class lowercase_dict(dict):
 def __getitem__(self, name):
 return dict.__getitem__(self, name.lower())

import string
values = lowercase_dict( name='John Doe', phone='999-555-1212' )
output = string.Template( 'My name is $NAME and my phone is $Phone.'  
).safe_substitute( values )


Brilliant!! Using a custom dictionary solves my problem and gives me
some ideas on similar challenges.

Thank you!

Regards,
Malcolm
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: tarfile doesn't work with tgz files?

[Kushal Kumaran]

On Sat, May 16, 2009 at 10:03 PM, Robert Dailey  wrote:
> Hi,
>
> I'm not a big expert on the tarfile component, but I assumed that .tgz
> files were short for .tar.gz and the format was the same. When I try
> to extract a .tgz file using tarfile in Python 3.0 on Windows, I get
> the following error:
>
>  File "C:\Python30\lib\tarfile.py", line 1630, in open
>    raise ReadError("file could not be opened successfully")
> tarfile.ReadError: file could not be opened successfully
>
> I'm assuming this is because the .tgz format is not what I think it is
> and is not supported. How can I extract .tgz files using Python 3.0?
> Note that I probably can't depend on third party python modules, since
> I have yet to find many that support Python 3.0.
>

Is the file itself correct?  Are you able to extract the files using
any compression program?  Do you have a version of tar installed that
you can try extracting the file with?  If you don't, a Windows version
of tar will be available with cygwin.

-- 
kushal
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: tarfile doesn't work with tgz files?

[Gary Herron]

Robert Dailey wrote:

Hi,

I'm not a big expert on the tarfile component, but I assumed that .tgz
files were short for .tar.gz and the format was the same. 


That's correct.


When I try
to extract a .tgz file using tarfile in Python 3.0 on Windows, I get
the following error:

  File "C:\Python30\lib\tarfile.py", line 1630, in open
raise ReadError("file could not be opened successfully")
tarfile.ReadError: file could not be opened successfully

I'm assuming this is because the .tgz format is not what I think it is
and is not supported.


Not true.  It's supported.

 How can I extract .tgz files using Python 3.0?
Note that I probably can't depend on third party python modules, since
I have yet to find many that support Python 3.0.

Thanks.
  


It's hard to debug your code when you don't show us your code,  so I 
won't try.


I can however supply a bit of working code:

>>> import tarfile
>>> f=tarfile.open('f.tgz')
>>> f.list()
-rw-r--r-- gherron/gherron703 2009-03-14 15:59:46 t.py
-rw-r--r-- gherron/gherron566 2009-05-15 09:13:08 note-bzr-hg
-rw-r--r-- gherron/gherron  11563 2009-04-24 09:33:12 .emacs

So the module correctly reads the tar-gzip file, and finds its contents.

Now, what is it you were trying to do?


Gary Herron



--
http://mail.python.org/mailman/listinfo/python-list

Re: tarfile doesn't work with tgz files?

[Robert Dailey]

On May 16, 11:33 am, Robert Dailey  wrote:
> Hi,
>
> I'm not a big expert on the tarfile component, but I assumed that .tgz
> files were short for .tar.gz and the format was the same. When I try
> to extract a .tgz file using tarfile in Python 3.0 on Windows, I get
> the following error:
>
>   File "C:\Python30\lib\tarfile.py", line 1630, in open
>     raise ReadError("file could not be opened successfully")
> tarfile.ReadError: file could not be opened successfully
>
> I'm assuming this is because the .tgz format is not what I think it is
> and is not supported. How can I extract .tgz files using Python 3.0?
> Note that I probably can't depend on third party python modules, since
> I have yet to find many that support Python 3.0.
>
> Thanks.

Correction, it is the following line that breaks:

tarfile.open( file )

When I try to *open* the file it fails, not extract. Sorry for the
mixup.
-- 
http://mail.python.org/mailman/listinfo/python-list

Swapping superclass from a module

[Emanuele D'Arrigo]

Hi everybody,

let's assume I have a module with loads of classes inheriting from one
class, from the same module, i.e.:

## myFile.py
class SuperClass(object)
class SubClass1(SuperClass)
class SubClass2(SuperClass)
class SubClass3(SuperClass)

In a separate file I also have:

## myOtherFile.py
class NewSuperClass(object)

Now, let's also assume that myFile.py cannot be changed or it's
impractical to do so. Is there a way to replace the SuperClass at
runtime, so that when I instantiate one of the subclasses
NewSuperClass is used instead of the original SuperClass provided by
the first module module?

That was the generic case. Would the solution change much if
NewSuperClass was actually inheriting from SuperClass, effectively
wedging itself between the SuperClass and the SubClasses?

Manu



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: tarfile doesn't work with tgz files?

[Robert Dailey]

On May 16, 11:49 am, Gary Herron  wrote:
> Robert Dailey wrote:
> > Hi,
>
> > I'm not a big expert on the tarfile component, but I assumed that .tgz
> > files were short for .tar.gz and the format was the same.
>
> That's correct.
>
> > When I try
> > to extract a .tgz file using tarfile in Python 3.0 on Windows, I get
> > the following error:
>
> >   File "C:\Python30\lib\tarfile.py", line 1630, in open
> >     raise ReadError("file could not be opened successfully")
> > tarfile.ReadError: file could not be opened successfully
>
> > I'm assuming this is because the .tgz format is not what I think it is
> > and is not supported.
>
> Not true.  It's supported.
>
> >  How can I extract .tgz files using Python 3.0?
> > Note that I probably can't depend on third party python modules, since
> > I have yet to find many that support Python 3.0.
>
> > Thanks.
>
> It's hard to debug your code when you don't show us your code,  so I
> won't try.
>
> I can however supply a bit of working code:
>
>  >>> import tarfile
>  >>> f=tarfile.open('f.tgz')
>  >>> f.list()
> -rw-r--r-- gherron/gherron        703 2009-03-14 15:59:46 t.py
> -rw-r--r-- gherron/gherron        566 2009-05-15 09:13:08 note-bzr-hg
> -rw-r--r-- gherron/gherron      11563 2009-04-24 09:33:12 .emacs
>
> So the module correctly reads the tar-gzip file, and finds its contents.
>
> Now, what is it you were trying to do?
>
> Gary Herron

I wasn't sure if my code was relevant, since there really isn't much
to show. Here is the convenience function I've created for extracting
the contents of a tar.gz or tgz file on Windows:

def untar( file, dst ):
   print( file )
   print( dst )
   tarball = tarfile.open( file )
   tarball.extractall( dst )
   tarball.close()

The respective results of the 2 print statements above are:

D:\IT\personal\layout-editor\temp\tbb21_20080605oss_src.tgz
D:\IT\personal\layout-editor\temp

And the full output I get:

Traceback (most recent call last):
  File "D:\IT\personal\layout-editor\third_party.py", line 257, in

tbb.Process()
  File "D:\IT\personal\layout-editor\third_party.py", line 247, in
Process
install_operation.Process( self, 'tbb' )
  File "D:\IT\personal\layout-editor\third_party.py", line 160, in
Process
build_operation.Process( self, name )
  File "D:\IT\personal\layout-editor\third_party.py", line 138, in
Process
self.Extract()
  File "D:\IT\personal\layout-editor\third_party.py", line 125, in
Extract
extract( file, self._outdir )
  File "D:\IT\personal\layout-editor\third_party.py", line 43, in
untar
tarball = tarfile.open( file )
  File "C:\Python30\lib\tarfile.py", line 1630, in open
raise ReadError("file could not be opened successfully")
tarfile.ReadError: file could not be opened successfully
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: tarfile doesn't work with tgz files?

[Robert Dailey]

On May 16, 11:46 am, Kushal Kumaran  wrote:
> On Sat, May 16, 2009 at 10:03 PM, Robert Dailey  wrote:
> > Hi,
>
> > I'm not a big expert on the tarfile component, but I assumed that .tgz
> > files were short for .tar.gz and the format was the same. When I try
> > to extract a .tgz file using tarfile in Python 3.0 on Windows, I get
> > the following error:
>
> >  File "C:\Python30\lib\tarfile.py", line 1630, in open
> >    raise ReadError("file could not be opened successfully")
> > tarfile.ReadError: file could not be opened successfully
>
> > I'm assuming this is because the .tgz format is not what I think it is
> > and is not supported. How can I extract .tgz files using Python 3.0?
> > Note that I probably can't depend on third party python modules, since
> > I have yet to find many that support Python 3.0.
>
> Is the file itself correct?  Are you able to extract the files using
> any compression program?  Do you have a version of tar installed that
> you can try extracting the file with?  If you don't, a Windows version
> of tar will be available with cygwin.
>
> --
> kushal

Yes, I can open and extract it using WinRAR
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: tarfile doesn't work with tgz files?

[Robert Dailey]

On May 16, 11:49 am, Gary Herron  wrote:
> Robert Dailey wrote:
> > Hi,
>
> > I'm not a big expert on the tarfile component, but I assumed that .tgz
> > files were short for .tar.gz and the format was the same.
>
> That's correct.
>
> > When I try
> > to extract a .tgz file using tarfile in Python 3.0 on Windows, I get
> > the following error:
>
> >   File "C:\Python30\lib\tarfile.py", line 1630, in open
> >     raise ReadError("file could not be opened successfully")
> > tarfile.ReadError: file could not be opened successfully
>
> > I'm assuming this is because the .tgz format is not what I think it is
> > and is not supported.
>
> Not true.  It's supported.
>
> >  How can I extract .tgz files using Python 3.0?
> > Note that I probably can't depend on third party python modules, since
> > I have yet to find many that support Python 3.0.
>
> > Thanks.
>
> It's hard to debug your code when you don't show us your code,  so I
> won't try.
>
> I can however supply a bit of working code:
>
>  >>> import tarfile
>  >>> f=tarfile.open('f.tgz')
>  >>> f.list()
> -rw-r--r-- gherron/gherron        703 2009-03-14 15:59:46 t.py
> -rw-r--r-- gherron/gherron        566 2009-05-15 09:13:08 note-bzr-hg
> -rw-r--r-- gherron/gherron      11563 2009-04-24 09:33:12 .emacs
>
> So the module correctly reads the tar-gzip file, and finds its contents.
>
> Now, what is it you were trying to do?
>
> Gary Herron

Okay so I managed to figure out the problem.

Initially, I manually downloaded the TGZ above through Chrome and then
opened it in WinRAR and extracted the files. Everything worked fine.

Then, I ran my Python script which uses urlretrieve() to download the
file and then tarfile is used to extract it. However, urlretrieve()
was not downloading the full file because of a bug that was corrupting
my URL.

Sorry for the confusion, and thanks to everyone for the very prompt
help!
-- 
http://mail.python.org/mailman/listinfo/python-list

Python code-bloat tool-- warning n00b stuff...

[anand j]

Hi,
I am looking for a bunch of rules or a tool that takes the code for my
python class and checks the amount of code bloat and points out where i can
improve. I am a n00b to python and built an application linking wordnet and
graph packages. but somehow have this nagging feeling my code is too bloated
with too many functions. might just be paranoia , but worth an
investigation i guess.

Sorry if this is a repeat/trivial question, I could not find any
comprehensive links on google or the mailing list archive that is within my
gmail. Currently, trying to download the other archives and index using
whoosh and try searching it.

Thanks

==
Anand J
Center for Behaviour and Cognitive Sciences
University of Allahabad
Allahabad-211002
http://sites.google.com/a/cbcs.ac.in/students/anand
==
The man who is really serious,
with the urge to find out what truth is,
has no style at all. He lives only in what is.
 ~Bruce Lee
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: ConfigParser examples?

[Esmail]

Hi Gabriel,

Gabriel Genellina wrote:


The (excellent!) article series by Doug Hellmann, "Python Module of the 
Week", covers ConfigParser (and almost the whole standard library by now).


http://www.doughellmann.com/PyMOTW


Thanks for the pointer, I hadn't come across this site before, it
looks like a place with a lot of useful information!

Esmail


--
http://mail.python.org/mailman/listinfo/python-list

Re: Options for creating a statistics screen on a Mac with Python?

[Nick Craig-Wood]

Diez B. Roggisch  wrote:
>  Phillip B Oldham schrieb:
> > I've come into possession of a mac mini and a large LCD tv at the
> > office. I'd like to set it up in the corner to pull statistics from
> > our various servers (load, uptimes, etc) and display them in a
> > graphical format, full-screen, with a reasonable refresh rate (say
> > every 30 seconds). There will be quite a few statistics, and we'll be
> > adding new servers over time, and since there won't be the option to
> > scroll a full-screen web-page won't work.
> > 
> > What then would be my options to create a full-screen display of
> > statistics, and an attractive graphical way, using Python on OS X?
> 
>  matplotlib should serve you well. Or you use some web-based solution 
>  with a javascript-based chart library. This might be even a bit
>  easier.

I'd use pygame for a really clean full screen display.

Probably a bit more work, but you'll get something really cool at the
end of it!

here is how to use matplotlib on a pygame surface

  http://www.pygame.org/wiki/MatplotlibPygame

-- 
Nick Craig-Wood  -- http://www.craig-wood.com/nick
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: tarfile doesn't work with tgz files?

[draeath]

On 2009-05-16, Robert Dailey  wrote:
> On May 16, 11:49 am, Gary Herron  wrote:
>> Robert Dailey wrote:
>> > Hi,
>>
>> > I'm not a big expert on the tarfile component, but I assumed that .tgz
>> > files were short for .tar.gz and the format was the same.
>>
>> That's correct.
>>
>> > When I try
>> > to extract a .tgz file using tarfile in Python 3.0 on Windows, I get
>> > the following error:
>>
>> >   File "C:\Python30\lib\tarfile.py", line 1630, in open
>> >     raise ReadError("file could not be opened successfully")
>> > tarfile.ReadError: file could not be opened successfully
>>
>> > I'm assuming this is because the .tgz format is not what I think it is
>> > and is not supported.
>>
>> Not true.  It's supported.
>>
>> >  How can I extract .tgz files using Python 3.0?
>> > Note that I probably can't depend on third party python modules, since
>> > I have yet to find many that support Python 3.0.
>>
>> > Thanks.
>>
>> It's hard to debug your code when you don't show us your code,  so I
>> won't try.
>>
>> I can however supply a bit of working code:
>>
>>  >>> import tarfile
>>  >>> f=tarfile.open('f.tgz')
>>  >>> f.list()
>> -rw-r--r-- gherron/gherron        703 2009-03-14 15:59:46 t.py
>> -rw-r--r-- gherron/gherron        566 2009-05-15 09:13:08 note-bzr-hg
>> -rw-r--r-- gherron/gherron      11563 2009-04-24 09:33:12 .emacs
>>
>> So the module correctly reads the tar-gzip file, and finds its contents.
>>
>> Now, what is it you were trying to do?
>>
>> Gary Herron
>
> Okay so I managed to figure out the problem.
>
> Initially, I manually downloaded the TGZ above through Chrome and then
> opened it in WinRAR and extracted the files. Everything worked fine.
>
> Then, I ran my Python script which uses urlretrieve() to download the
> file and then tarfile is used to extract it. However, urlretrieve()
> was not downloading the full file because of a bug that was corrupting
> my URL.
>
> Sorry for the confusion, and thanks to everyone for the very prompt
> help!


Just to clarify:

A tar.gz is a gzipped tar archive. It's a normal tar file, processed
into a normal gzip file.

tar.bz2 is the same, but bzip2 instead of gzip.

So, if for some reason you want to do it yourself, just decompress the
data first, then handle it like a tar file.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Parsing Strings in Enclosed in Curly Braces

[Kay Schluehr]

> Since when is a list a number? Perhaps the help needs clarification,
> in line with the docs.

Everyone is supposed to use reduce() here ;)

Kay
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Swapping superclass from a module

[Arnaud Delobelle]

"Emanuele D'Arrigo"  writes:

> Hi everybody,
>
> let's assume I have a module with loads of classes inheriting from one
> class, from the same module, i.e.:
>
> ## myFile.py
> class SuperClass(object)
> class SubClass1(SuperClass)
> class SubClass2(SuperClass)
> class SubClass3(SuperClass)
>
> In a separate file I also have:
>
> ## myOtherFile.py
> class NewSuperClass(object)
>
> Now, let's also assume that myFile.py cannot be changed or it's
> impractical to do so. Is there a way to replace the SuperClass at
> runtime, so that when I instantiate one of the subclasses
> NewSuperClass is used instead of the original SuperClass provided by
> the first module module?
>
> That was the generic case. Would the solution change much if
> NewSuperClass was actually inheriting from SuperClass, effectively
> wedging itself between the SuperClass and the SubClasses?
>

Here is an attempt to do this.  Imagine there is a module 'modfoo.py'
like this:

--
class Base(object):
def bar(self):
print 'Base.bar'

class A(Base):
def bar(self): 
Base.bar(self)
print 'A.bar'

class B(Base):
def bar(self):
super(B, self).bar()
print 'B.bar'
--

The following script will patch the module so that all subclasses of
Base in module modfoo have their new superclass changed.  I called it
'patchfoo.py'.

--
import modfoo

# This is the new base
class Wedge(modfoo.Base):
def bar(self):
super(Wedge, self).bar()
print 'Wedge.bar'

# Insert Wedge into each subclass of modfoo.Base
for subclass in modfoo.Base.__subclasses__():
if subclass.__module__ != 'modfoo': continue
attrs = dict(item for item in subclass.__dict__.items() 
  if item[0][:2] != '__')
name = subclass.__name__
setattr(modfoo, name, type(name, (Wedge,), attrs))

# Replace modfoo.Base with Wedge
modfoo.Base = Wedge

# Tests
a = modfoo.A()
b = modfoo.B()

for obj in 'a' ,'b':
call = '%s.bar()' % (obj,)
print '-'*5, call
exec call
--

Now let's try it:

marigold:junk arno$ python patchfoo.py 
- a.bar()
Base.bar
Wedge.bar
A.bar
- b.bar()
Base.bar
Wedge.bar
B.bar

Of course, there are plenty of ways this could break.

-- 
Arnaud
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Options for creating a statistics screen on a Mac with Python?

[Arnaud Delobelle]

Phillip B Oldham  writes:

> On May 16, 2:15 pm, "Diez B. Roggisch"  wrote:
>> matplotlib should serve you well.
>
> Thanks, I'll check that out.
>
>> Or you use some web-based solution with a javascript-based chart library.
>
> I'd rather stay away from web-based solutions; since this is going to
> be running 24/7 and has a fixed a screen size. I'd prefer something
> that has few dependencies, if possible hooking into the OS X platform
> for graphics.

* You could create a pdf/png and pipe the output through to Preview.app
  which can display it in full screen.

* A simple and portable solution may be to use pygame

-- 
Arnaud
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to get Exif data from a jpeg file

[Daniel Fetchinson]

> I need to get the creation date from a jpeg file in Python.  Googling
> brought up a several references to apparently defunct modules.  The best
> way I have been able to find so far is something like this:
>
> from PIL import Image
> img = Image.open('img.jpg')
> exif_data = img._getexif()
> creation_date = exif_data[36867]
>
> Where 36867 is the exif tag for the creation date data (which I found by
> ooking at PIL.ExifTags.TAGS).  But this doesn't even seem to be
> documented in the PIL docs.  Is there a more natural way to do this?


Have you tried http://sourceforge.net/projects/exif-py/ ?

HTH,
Daniel

-- 
Psss, psss, put it down! - http://www.cafepress.com/putitdown
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Difference between list() and [] with dictionaries

[Aahz]

In article ,
Ned Deily   wrote:
>In article 
>,
> Sam Tregar  wrote:
>>
>> Can anyone explain why this creates a list containing a
>> dictionary:
>>   [{'a': 'b', 'foo': 'bar'}] 
>> But this creates a list of keys of the dictionary:
>>   list({ "a": "b", "foo": "bar" })
>
>The first example is a list, expressed as a list display literal, 
>containing one object: a dictionary, expressed as a dictionary display 
>literal.
>
>The second example is a call to the built-in function "list",  [...]

Actually, list() is not a function:

>>> list


Rather, ``list`` is an object (specifically a ``type`` object) with a
__call__() method.
-- 
Aahz (a...@pythoncraft.com)   <*> http://www.pythoncraft.com/

"In 1968 it took the computing power of 2 C-64's to fly a rocket to the moon.
Now, in 1998 it takes the Power of a Pentium 200 to run Microsoft Windows 98.
Something must have gone wrong."  --/bin/fortune
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python system with exemplary organization/coding style

[Aahz]

In article <2009051416013116807-tomfsess...@gmailcom>,
TomF   wrote:
>
>I'm looking for a medium-sized Python system with very good coding 
>style and good code organization, so I can learn from it.  I'm reading 
>various books on Python with advice on such things but I'd prefer to 
>see a real system.
>
>By medium-sized I mean 5-20 classes, 5-20 files, etc; a code base that 
>has some complexity but isn't overwhelming.

Actually, that's what I'd call "small".  "Medium" would be about 30-50
classes (Python mostly has multiple classes per file, unlike Java, so
number of files is usally a poor measure of complexity).  Something about
the right complexity level for what you're looking for can be found in
several parts of the Python standard library, including the thread,
threading, and Queue modules (although I haven't looked at the code
recently, so I can't vouch for its quality).

Another option would be dnspython, but that's getting a bit larger than
you're looking for.
-- 
Aahz (a...@pythoncraft.com)   <*> http://www.pythoncraft.com/

"In 1968 it took the computing power of 2 C-64's to fly a rocket to the moon.
Now, in 1998 it takes the Power of a Pentium 200 to run Microsoft Windows 98.
Something must have gone wrong."  --/bin/fortune
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Difference between list() and [] with dictionaries

[Ned Deily]

In article , a...@pythoncraft.com (Aahz) 
wrote:
> In article ,
> Ned Deily   wrote:
> >The second example is a call to the built-in function "list",  [...]
> Actually, list() is not a function: 
> >>> list
> 
> Rather, ``list`` is an object (specifically a ``type`` object) with a
> __call__() method.

Yes.  However, it is also true that "list" is listed under the Built-in 
Functions section of the Python Standard Library documentation and, wrt 
to explaining the OP's issue, quacks like a function here.

-- 
 Ned Deily,
 n...@acm.org

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Swapping superclass from a module

[Emanuele D'Arrigo]

On May 16, 8:17 pm, Arnaud Delobelle  wrote:
> # Insert Wedge into each subclass of modfoo.Base
> for subclass in modfoo.Base.__subclasses__():
>     if subclass.__module__ != 'modfoo': continue
>     attrs = dict(item for item in subclass.__dict__.items()
>                       if item[0][:2] != '__')
>     name = subclass.__name__
>     setattr(modfoo, name, type(name, (Wedge,), attrs))
>
> # Replace modfoo.Base with Wedge
> modfoo.Base = Wedge

That-is-neat! I'm impressed. It took me a while to understand it all:
I didn't know about __subclasses__, the way you fill the dictionary is
completely novel to me and the use of type(name, tuple, attributes)
was also completely novel to me. But eventually I got it and it's
quite neat. Thank you.

> Of course, there are plenty of ways this could break.

Uh-oh. Ok, you got me all excited and now I have to read the fine
prints. So, how can it break?

Ciao!

Manu
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python code-bloat tool-- warning n00b stuff...

[Robert Kern]

On 2009-05-16 12:13, anand j wrote:

Hi,
 I am looking for a bunch of rules or a tool that takes the code for
my python class and checks the amount of code bloat and points out where
i can improve. I am a n00b to python and built an application linking
wordnet and graph packages. but somehow have this nagging feeling my
code is too bloated with too many functions. might just be paranoia
, but worth an investigation i guess.

Sorry if this is a repeat/trivial question, I could not find any
comprehensive links on google or the mailing list archive that is within
my gmail. Currently, trying to download the other archives and index
using whoosh and try searching it.


Just use Google: "site:mail.python.org code bloat tool"

But basically, such a thing doesn't really exist. It's not really a quantifiable 
concept. You can use tools like pylint to enforce a simple policy (e.g. "no more 
than 50 methods per class"), but that's not really the same thing, nor is it 
something you can't do yourself with a little bit of grepping.


A slightly more interesting and nontrivial metric that you can apply is 
cyclomatic complexity. Basically, it is the number of independent code paths 
your each function may go down.


http://en.wikipedia.org/wiki/Cyclomatic_complexity
http://www.traceback.org/2008/03/31/measuring-cyclomatic-complexity-of-python-code/

This will give a not-too-unreasonable measure of how complicated each function 
is. I don't know if that's what you are getting at with the term "code bloat".


However, there is no substitute for experienced judgment. Show your code to 
other Pythonistas. What do they think? A quick question to an experienced 
programmer is usually much more efficient than downloading a tool and trying to 
interpret its results.


--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

--
http://mail.python.org/mailman/listinfo/python-list

Re: Swapping superclass from a module

[Arnaud Delobelle]

"Emanuele D'Arrigo"  writes:

> On May 16, 8:17 pm, Arnaud Delobelle  wrote:
>> # Insert Wedge into each subclass of modfoo.Base
>> for subclass in modfoo.Base.__subclasses__():
>>     if subclass.__module__ != 'modfoo': continue
>>     attrs = dict(item for item in subclass.__dict__.items()
>>                       if item[0][:2] != '__')
>>     name = subclass.__name__
>>     setattr(modfoo, name, type(name, (Wedge,), attrs))
>>
>> # Replace modfoo.Base with Wedge
>> modfoo.Base = Wedge
>
> That-is-neat! I'm impressed. It took me a while to understand it all:
> I didn't know about __subclasses__, the way you fill the dictionary is
> completely novel to me and the use of type(name, tuple, attributes)
> was also completely novel to me. But eventually I got it and it's
> quite neat. Thank you.
>
>> Of course, there are plenty of ways this could break.
>
> Uh-oh. Ok, you got me all excited and now I have to read the fine
> prints. So, how can it break?

I can think of 3 easy ways (and there may be more):

1. For simplicity I chose to ignore attributes that start with '__' but of
course in real life some of them would need to be included (__init__ &
co...).

2. It wouldn't work with multiple inheritance.

3. If the original classes are not just defined, but also used when the
module is loaded then this will cause problems.  E.g. imagine that the
file 'modfoo.py' in my example contains the additional line

AA = A

Then when A is patched, AA will remain bound to the original class A,
and there is not much that can be done about this.


1. and 2. are only problems because my code was proof-of-concept and can
be addressed without difficulty I think.  

3. is a more serious limitation, but if your module doesn't use the
classes (i.e. it just defines them), then the method might work OK I
guess.

-- 
Arnaud
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Parsing Strings in Enclosed in Curly Braces

[John Machin]

On May 17, 1:28 am, a...@pythoncraft.com (Aahz) wrote:
> In article 
> <180531ca-33aa-47b9-9c69-5b5973f6b...@v35g2000pro.googlegroups.com>,
> John Machin   wrote:
> >Neat trick. However, from 2.6.2:
>
>  help(sum)
> >Help on built-in function sum in module __builtin__:
>
> >sum(...)
> >    sum(sequence[, start]) -> value
>
> >    Returns the sum of a sequence of numbers (NOT strings) plus the
> >value
> >    of parameter 'start' (which defaults to 0).  When the sequence is
> >    empty, returns start.
>
> >Since when is a list a number? Perhaps the help needs clarification,
> >in line with the docs.
>
> The primary use-case for sum() is numbers, with a special exception to
> prohibit using strings.  Only strings are prohibited to allow using sum()
> with user-defined classes.  That makes it a little difficult to document

Non sequitur.

> precisely in a summary; unless you can come up with a specific better
> wording, the docs will probably stay as-is.  If you do come up with
> something better, please file it on bugs.python.org.

OK let's start with getting the docs right and then summarise that for
the help.
URI: http://docs.python.org/library/functions.html#sum
Contents:
"""
sum(iterable[, start])¶

Sums start and the items of an iterable from left to right and
returns the total. start defaults to 0. The iterable‘s items are
normally numbers, and are not allowed to be strings. The fast, correct
way to concatenate a sequence of strings is by calling ''.join
(sequence). Note that sum(range(n), m) is equivalent to reduce
(operator.add, range(n), m) To add floating point values with extended
precision, see math.fsum().

New in version 2.3
"""
Suggestions:
(1) fix what should be an apostrophe in "iterable's"
(2) s/sequence/iterable/g
(3) Either replace the sentence about "reduce" by "Note that sum
(iterable, start) is equivalent to reduce(operator.add, iterable,
start) except for the prohibition of strings" or explain why not or
what's so special about range(n).
(4) Add a full stop after the sentence about "reduce".

Help from 2.6.2:
"""
sum(sequence[, start]) -> value

Returns the sum of a sequence of numbers (NOT strings) plus the
value
of parameter 'start' (which defaults to 0).  When the sequence is
empty, returns start.
"""
Suggested replacement:
"""
sum(iterable[, start]) -> value

Returns the sum of the contents of the iterable plus the value
of parameter 'start' (which defaults to 0).  When the iterable is
empty, returns start. Strings are disallowed; use
''.join(iterable) to concatenate strings efficiently.
"""

Cheers,
John
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Concurrency Email List

[Aahz]

[posted and e-mailed]

On Sat, May 16, 2009, Pete wrote:
>
> python-concurre...@googlegroups.com is a new email list for discussion  
> of concurrency issues in python.  It arose out of Dave Beazley's class  
> on the subject last week: http://www.dabeaz.com/chicago/concurrent.html
>
> The list wills serve as a forum for discussion of such topics as:
>
> * threads
> * multiprocessing
> * asynchronous
> * coroutines
> * the GIL
>
> Hope to see you there!  Join at 
> http://groups.google.com/group/python-concurrency/

Is there some reason you chose not to create a list on python.org?  I'm
not joining the list because Google requires that you create a login.
-- 
Aahz (a...@pythoncraft.com)   <*> http://www.pythoncraft.com/

"In 1968 it took the computing power of 2 C-64's to fly a rocket to the moon.
Now, in 1998 it takes the Power of a Pentium 200 to run Microsoft Windows 98.
Something must have gone wrong."  --/bin/fortune
-- 
http://mail.python.org/mailman/listinfo/python-list

Context manager, atexit processing, and PEP 3143 DaemonContext.close

[Ben Finney]

Howdy all,

I'm slowly developing PEP 3143 and, in parallel, its reference
implementation, the ‘python-daemon’ library
http://pypi.python.org/pypi/python-daemon/>. Feedback continues to
be welcome from people wanting to make a program become a daemon; please
try it out and critique the specification and/or the implementation.

Today a flaw has been uncovered by Pavol Babinčák. In diagnosing an
issue, we doscovered that PEP 3143 currently leads, in some common
circumstances, to exiting a context manager twice.

The current specification of the `DaemonContext` class says:

=
close()
Return: None

Close the daemon context. This performs the following step:

* If the pidfile attribute is not None, exit its context
  manager.
=

and:

=
__exit__(exc_type, exc_value, exc_traceback)
Return:True or False as defined by the context manager
protocol

Call the instance's close() method, then return True if the
exception was handled or False if it was not.
=

but also:

=
open()
Return: None

Open the daemon context, turning the current program into a daemon
process. This performs the following steps:
[…]

* Register the close method to be called during Python's exit
  processing.
=


This leads to the situation where the `DaemonContext.close` method can
be called twice:

pidfile = FunkyLockFile()
daemon_context = daemon.DaemonContext(pidfile=pidfile)
with daemon_context:
main_processing()

According to the current specification, this will cause
`daemon_context.close()` to be called on exiting its context manager (at
the end of the `with` suite) *and* on program exit (via `atexit`
processing).

This will cause the `pidfile` context manager to be exited twice in
succession, which surely can't be good and is at least undefined AFAICT.


How should a context manager be implemented to handle situations like
this? One possible way is to raise an exception when trying to close a
not-open context manager. Another is to return immediately if the
attempt is made, making it safe to try closing a context manager
multiple times.

The “Register the close method to be called during Python's exit
processing” is there to ensure clean-up occurs even if the program
isn't using the DaemonContext as a context manager. Ideally, what I'd
like to do is *un*-register the `close` method after having already
closed it, so that exit processing *won't* call it. The `atexit` module,
though, doesn't appear to give me that option.

Ideas? How should this be addressed both Pythonically and respecting the
intent of PEP 3143?

-- 
 \   “For fast acting relief, try slowing down.” —Jane Wagner, via |
  `\   Lily Tomlin |
_o__)  |
Ben Finney
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Swapping superclass from a module

[Steven D'Aprano]

On Sat, 16 May 2009 09:55:39 -0700, Emanuele D'Arrigo wrote:

> Hi everybody,
> 
> let's assume I have a module with loads of classes inheriting from one
> class, from the same module, i.e.:
[...]
> Now, let's also assume that myFile.py cannot be changed or it's
> impractical to do so. Is there a way to replace the SuperClass at
> runtime, so that when I instantiate one of the subclasses NewSuperClass
> is used instead of the original SuperClass provided by the first module
> module?

That's called "monkey patching" or "duck punching".

http://en.wikipedia.org/wiki/Monkey_patch

http://wiki.zope.org/zope2/MonkeyPatch

http://everything2.com/title/monkey%2520patch



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

UDP reading on multiple sockets

[Thomas Vogel]

Hi all,

I'm currently have the problem that I try to read UDP messages from 
multiple sockets in parallel. So let's say I get UDP packets from the 
same IP on the ports 2000, 2001, 2002,...


Therefore I created the following class.
But if I try to instantiate it several times with different ports I 
experience that if I get a broadcast message all ports are waking up, 
the 1st socket is using the message and the others close itself as the 
buffer is empty.


Does anybody have an idea how to do this in a better way?
I just need non-blocking UDP-receiver that are listening on several 
ports in parallel and return me the incomming messages. Are there 
examples in the net that describe this problem?


class Receiver(asyncore.dispatcher, threading.Thread):
'''
This is the command receiver entity.
'''

def __init__(self, api, localIp, localPort):

asyncore.dispatcher.__init__(self)
threading.Thread.__init__(self)
self.create_socket(socket.AF_INET, socket.SOCK_DGRAM)
self.bind((localIp, localPort))
self.cv = threading.Condition()
self.data = []
self.sorbasApi = api

def handle_connect(self):
return

def writable(self):
return False

def handle_error(self):
self.logger.error('error')
return asyncore.dispatcher.handle_error(self)

def handle_read(self):
s = self.recv(4096)
self.cv.acquire()
self.data.append(s)
self.cv.notifyAll()
self.cv.release()

def handle_close(self):
self.close()

def run(self):
asyncore.loop()

def flush(self):
self.data = []

def readNextMsg(self, paramSet, timeout=1):
if len(self.data) == 0:
self.cv.acquire()
self.cv.wait(timeout)
self.cv.release()
if len(self.data) > 0:
buffer = array('B', self.data.pop(0)).tolist()
m = ReceiveMessage(self.sorbasApi, paramSet)
m.decode(buffer)
return m
else:
return None

def __del__(self):
self.close()

Thanks a lot in advance
Thomas
--
http://mail.python.org/mailman/listinfo/python-list

Re: UDP reading on multiple sockets

[Grant Edwards]

On 2009-05-17, Thomas Vogel  wrote:

> I'm currently have the problem that I try to read UDP messages from 
> multiple sockets in parallel. So let's say I get UDP packets from the 
> same IP on the ports 2000, 2001, 2002,...

Is there any reason you can't do it the easy way by using
select?

http://docs.python.org/library/select.html

-- 
Grant

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Context manager, atexit processing, and PEP 3143 DaemonContext.close

[Carl Banks]

On May 16, 5:50 pm, Ben Finney  wrote:
> Ideas? How should this be addressed both Pythonically and respecting the
> intent of PEP 3143?


There's already precedent for what to do in the Python library.

Python 2.5.2 (r252:60911, Jan  4 2009, 17:40:26)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> f = open('somefile')
>>> f.close()
>>> f.close()
>>> f.close()
>>> f.close()


Is a context manager really necessary here?  It's not like daemon
processes can reattach themselves to the terminal when they're done
being a daemon.  What's the use case for being able to partially clean
up before program exit?


Carl Banks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Context manager, atexit processing, and PEP 3143 DaemonContext.close

[Ben Finney]

Carl Banks  writes:

> There's already precedent for what to do in the Python library.
> 
> Python 2.5.2 (r252:60911, Jan  4 2009, 17:40:26)
> [GCC 4.3.2] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> f = open('somefile')
> >>> f.close()
> >>> f.close()
> >>> f.close()
> >>> f.close()

Yes, that's a precedent in favour of “silently return immediately when
closing an already-closed DaemonContext”. I'm not about to assume that
it's the *only* relevant precedent.

> Is a context manager really necessary here? It's not like daemon
> processes can reattach themselves to the terminal when they're done
> being a daemon. What's the use case for being able to partially clean
> up before program exit?

The use case is::

pidfile = MyChoiceOfLockFile()
daemon_context = daemon.DaemonContext(pidfile=pidfile)

with daemon_context:
do_main_processing()

which could easily become something like::

beans_resource = crunchy_init()

with beans_resource:

spam_resource = crispy_init()

pidfile = MyChoiceOfLockFile()
daemon_context = daemon.DaemonContext(pidfile=pidfile)

with daemon_context:
do_main_processing()

spam_resource.cleanup()

In other words, the DaemonContext has a specific job, and is easier to
use if it can be relied on to clean up its own stuff via a context
manager; but the larger program could easily have other things (in the
example above, both of `spam_resource` and `beans_resource`) it needs to
clean up apart from just the daemon context.

-- 
 \ “We reserve the right to serve refuse to anyone. ” —restaurant, |
  `\ Japan |
_o__)  |
Ben Finney
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Nimrod programming language

[Tomasz Rola]

On Mon, 11 May 2009, k...@fiber-space.de wrote:

> On 12 Mai, 02:10, Tomasz Rola  wrote:
> > On Mon, 11 May 2009, rump...@web.de wrote:
> > > > One question I ask myself upon seeing a new language is if it is 
> > > > possible
> > > > to program amb (amb=ambiguous) operator in it. This page gives a very
> > > > nice, "code first" explanation of amb and how it is supposed to work:
> >
> > > >http://www.randomhacks.net/articles/2005/10/11/amb-operator
> >
> > > Hm. I am not sure either. To me, it looks like a constraint solver by
> > > using brute force.
> >
> > Yes, kind of. From what I understand [1], it's goal is to simulate
> > nondeterministic "get right answer in one try", only this "one try"
> > requires browsing the solution space (because we have only "classic"
> > hardware). It is easy to use amb in wrong way, but I can see some cases
> > when I would like it.
> 
> In Python you can use a generator expression. Dan Piponis example can
> be stated as
> 
> g = ((i,j) for i in range(2,100) for j in range(2,i) if i*j == 481)
> 
> which doesn't require any call/cc hacks but is a deterministic "get
> right in one try". Once Nimrod has coroutines it's just a matter of
> sugaring them into comprehensions.

Yep, to be frank everything complicated can be achieved by means of a 
"for" loop (so said one friend of mine, and this is dirty, dirty way of 
thinking). Or putting generators in a row... but try to do this trick 
with ten of them and things start to look messy. Or ugly. Or both - and I 
don't like neither.

There is another example, below Don Piponis' one. I like the look of 
simplicity. I don't like ten or seven nested loops, that would have been 
used instead. Perhaps amb could be somehow simulated with recursion, but 
to use recursion in a language, I need to know it is safe and doesn't 
overflow on 50th call. So, either I use the language that allows me to 
express things in a way appealing to my taste, or I use something else and 
perhaps end up with nested loops.

Besides, having search space defined with amb-statements may allow me to 
try other ideas than brute-force search. Like, say, genetic algorithm 
under the hood. No change to program, just link with other library (import 
amb's definition from other path etc). I cannot imagine trying such trick 
with nested loops without writing new program (writing new loop(s)).

Generators don't give me nice looking alternative to amb. So, if the 
problem requires two or three nested loops, ok, I can go for it. If more - 
I want another way out. Just my personal opinion, not forcing anybody to 
follow.

However, the question I wanted to be answered, was how far Nimrod would 
allow me to go. I've got my answer and, well, for some near future I will 
just sit and see what happens. So, discussing amb's usefulness or if the 
same ca be done with whatever else is not very interesting, because of 
course everything can be done with loops - no need for continuations, 
coroutines, recursion, macros etc.

Regards
Tomasz Rola

--
** A C programmer asked whether computer had Buddha's nature.  **
** As the answer, master did "rm -rif" on the programmer's home**
** directory. And then the C programmer became enlightened...  **
** **
** Tomasz Rola  mailto:tomasz_r...@bigfoot.com **
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Context manager, atexit processing, and PEP 3143 DaemonContext.close

[Carl Banks]

On May 16, 8:20 pm, Ben Finney  wrote:
> Carl Banks  writes:
> > There's already precedent for what to do in the Python library.
>
> > Python 2.5.2 (r252:60911, Jan  4 2009, 17:40:26)
> > [GCC 4.3.2] on linux2
> > Type "help", "copyright", "credits" or "license" for more information.
> > >>> f = open('somefile')
> > >>> f.close()
> > >>> f.close()
> > >>> f.close()
> > >>> f.close()
>
> Yes, that's a precedent in favour of “silently return immediately when
> closing an already-closed DaemonContext”. I'm not about to assume that
> it's the *only* relevant precedent.

I don't think this is anything more than a trivial consideration,
which should warrant nothing more than the simplest solution possible,
which is to simply allow multiple clean up calls.

That one of Python's most fundamental types allows its clean-up
operation to be invoked safely on an already-cleaned up object is
precedent enough, given the triviality of the issue.

You're welcome to your own assessment of the problem's importance.


Carl Banks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: KeyboardInterrupt catch does not shut down the socketserver

[Gabriel Genellina]

En Sat, 16 May 2009 04:04:03 -0300, Igor Katson   
escribió:

Gabriel Genellina wrote:

En Fri, 15 May 2009 09:04:05 -0300, Igor Katson escribió:

Lawrence D'Oliveiro wrote:
In message , Igor  
Katson wrote:

Lawrence D'Oliveiro wrote:
In message ,  
Igor Katson wrote:



I have problems in getting a SocketServer to shutdown.

Shutdown implies closing the listening socket, doesn't it?


No (perhaps it should, but that is another issue). There is a
documentation bug; BaseServer.shutdown is documented as "Tells the
serve_forever() loop to stop and waits until it does." [1]
The docstring is much more explicit: """Stops the serve_forever loop.
Blocks until the loop has finished. This must be called while
serve_forever() is running in another thread, or it will deadlock."""

So, if you have a single-threaded server, *don't* use shutdown(). And,  
to orderly close the listening socket, use server_close() instead. Your


Hmm. Gabriel, could you please show the same for the threaded version?  
This one deadlocks:

[code removed]


The shutdown method should *only* be called while serve_forever is  
running. If called after server_forever exited, shutdown() blocks forever.



from SocketServer import TCPServer, BaseRequestHandler
from threading import Thread

server = TCPServer(('localhost',1234), BaseRequestHandler)
run_thread = Thread(target=server.serve_forever)
run_thread.start()
try:
print "press ^C to exit"
import time; time.sleep(30)
except KeyboardInterrupt:
print "^C detected"
pass
server.shutdown()
run_thread.join()
print "bye"


But, what are you after, exactly? I think I'd use the above code only in a  
GUI application with a background server.

There are other alternatives, like asyncore or Twisted.

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: Context manager, atexit processing, and PEP 3143 DaemonContext.close

[Ben Finney]

Carl Banks  writes:

> I don't think this is anything more than a trivial consideration,

Okay, thank you.

Anyone else?

-- 
 \ “All television is educational television. The question is: |
  `\   what is it teaching?” —Nicholas Johnson |
_o__)  |
Ben Finney
-- 
http://mail.python.org/mailman/listinfo/python-list

ANN: wxPython 2.8.10.1

[Robin Dunn]

Announcing
--

The 2.8.10.1 release of wxPython is now available for download at
http://wxpython.org/download.php.  This release fixes the problem with 
using Python 2.6's default manifest, and updates wxcairo to work with 
the latest PyCairo.  A summary of changes is listed below and also

at http://wxpython.org/recentchanges.php.

Source code is available as a tarball and a source RPM, as well as
binaries for Python 2.4, 2.5 and 2.6, for Windows and Mac, as well
some packages for various Linux distributions.



What is wxPython?
-

wxPython is a GUI toolkit for the Python programming language. It
allows Python programmers to create programs with a robust, highly
functional graphical user interface, simply and easily. It is
implemented as a Python extension module that wraps the GUI components
of the popular wxWidgets cross platform library, which is written in
C++.

wxPython is a cross-platform toolkit. This means that the same program
will usually run on multiple platforms without modifications.
Currently supported platforms are 32-bit and 64-bit Microsoft Windows,
most Linux or other Unix-like systems using GTK2, and Mac OS X 10.4+.
In most cases the native widgets are used on each platform to provide
a 100% native look and feel for the application.



Changes in 2.8.10.1
---

wx.grid.Grid:  Added methods CalcRowLabelsExposed,
CalcColLabelsExposed, CalcCellsExposed, DrawRowLabels, DrawRowLabel,
DrawColLabels, and DrawColLabel to the Grid class.

Added the wx.lib.mixins.gridlabelrenderer module.  It enables the use
of label renderers for Grids that work like the cell renderers do.  See
the demo for a simple sample.

Solved the manifests problem with Python 2.6 on Windows.  wxPython now
programatically creates its own activation context and loads a
manifest in that context that specifies the use of the themable common
controls on Windows XP and beyond.  This also means that the external
manifest files are no longer needed for the other versions of Python.

wx.Colour: Updated the wx.Colour typemaps and also the wx.NamedColour
constructor to optionally allow an alpha value to be passed in the
color string, using these syntaxes:  "#RRGGBBAA" or "ColourName:AA"

wx.lib.wxcairo:  Fixed a problem resulting from PyCairo changing the
layout of their C API structure in a non-binary compatible way.  The
new wx.lib.wxcairo is known to now work with PyCairo 1.6.4 and 1.8.4,
and new binaries for Windows are available online at
http://wxpython.org/cairo/

--
Robin Dunn
Software Craftsman
http://wxPython.org

--
http://mail.python.org/mailman/listinfo/python-list

Re: Swapping superclass from a module

[Terry Reedy]

Steven D'Aprano wrote:

On Sat, 16 May 2009 09:55:39 -0700, Emanuele D'Arrigo wrote:


Hi everybody,

let's assume I have a module with loads of classes inheriting from one
class, from the same module, i.e.:

[...]

Now, let's also assume that myFile.py cannot be changed or it's
impractical to do so. Is there a way to replace the SuperClass at
runtime, so that when I instantiate one of the subclasses NewSuperClass
is used instead of the original SuperClass provided by the first module
module?


That's called "monkey patching" or "duck punching".

http://en.wikipedia.org/wiki/Monkey_patch

http://wiki.zope.org/zope2/MonkeyPatch

http://everything2.com/title/monkey%2520patch


If the names of superclasses is resolved when classes are instantiated, 
the patching is easy.  If, as I would suspect, the names are resolved 
when the classes are created, before the module becomes available to the 
importing code, then much more careful and extensive patching would be 
required, if it is even possible.  (Objects in tuples cannot be 
replaced, and some attributes are not writable.)


tjr

--
http://mail.python.org/mailman/listinfo/python-list