Re: Creating class instance from module and class name

[Steven D'Aprano]

On Mon, 05 Oct 2009 23:32:27 -0700, gentlestone wrote:

> Suppose I've saved the class name and (don't know how) I've also saved
> the Class's module (package path or I don't know what's the name for XYZ
> "from X.Y.Z import ...). How can I construct a new class according to
> saved informations? If I don't know what Class it could be, only I have
> the saved Class name?


If you have the module *object*, and the name of the class, then you can 
do this:

theclass = getattr(module, "MyClass")

to get the class itself, and then call it as normal to instantiate it:

instance = theclass(args)

Classes are just like any other object in that regard.

If you have the *name* of the module, you can import it first to get the 
module object:

module = __import__('module_name')
theclass = getattr(module, "MyClass")
instance = theclass(args)


There may be some complications if you have a dotted package name, in 
which case the docs for __import__ are your friend :)



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Creating class instance from module and class name

[gentlestone]

On 6. Okt, 08:55 h., Steven D'Aprano
 wrote:
> On Mon, 05 Oct 2009 23:32:27 -0700, gentlestone wrote:
> > Suppose I've saved the class name and (don't know how) I've also saved
> > the Class's module (package path or I don't know what's the name for XYZ
> > "from X.Y.Z import ...). How can I construct a new class according to
> > saved informations? If I don't know what Class it could be, only I have
> > the saved Class name?
>
> If you have the module *object*, and the name of the class, then you can
> do this:
>
> theclass = getattr(module, "MyClass")
>
> to get the class itself, and then call it as normal to instantiate it:
>
> instance = theclass(args)
>
> Classes are just like any other object in that regard.
>
> If you have the *name* of the module, you can import it first to get the
> module object:
>
> module = __import__('module_name')
> theclass = getattr(module, "MyClass")
> instance = theclass(args)
>
> There may be some complications if you have a dotted package name, in
> which case the docs for __import__ are your friend :)
>
> --
> Steven

thx for help

one more question - __class__ is the way for getting the classname
from the class instance -
how can I get the module name?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Creating class instance from module and class name

[Chris Rebert]

On Tue, Oct 6, 2009 at 12:09 AM, gentlestone  wrote:
> On 6. Okt, 08:55 h., Steven D'Aprano
>  wrote:
>> On Mon, 05 Oct 2009 23:32:27 -0700, gentlestone wrote:
>> > Suppose I've saved the class name and (don't know how) I've also saved
>> > the Class's module (package path or I don't know what's the name for XYZ
>> > "from X.Y.Z import ...). How can I construct a new class according to
>> > saved informations? If I don't know what Class it could be, only I have
>> > the saved Class name?
>>
>> If you have the module *object*, and the name of the class, then you can
>> do this:
>>
>> theclass = getattr(module, "MyClass")
>>
>> to get the class itself, and then call it as normal to instantiate it:
>>
>> instance = theclass(args)
>>
>> Classes are just like any other object in that regard.
>>
>> If you have the *name* of the module, you can import it first to get the
>> module object:
>>
>> module = __import__('module_name')
>> theclass = getattr(module, "MyClass")
>> instance = theclass(args)
>>
>> There may be some complications if you have a dotted package name, in
>> which case the docs for __import__ are your friend :)
>>
>> --
>> Steven
>
> thx for help
>
> one more question - __class__ is the way for getting the classname
> from the class instance -

No, that'd be some_instance.__class__.__name__

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Haskell's new logo, and the idiocy of tech geekers

[Chris Withers]

Jack Diederich wrote:

It's Xah Lee, he's been trolling this and every other programing
language group for over 10 years (preferably all at once).  Let it go.


I don't suppose there's any chance of just blocking the idiot, is there?

Chris

--
Simplistix - Content Management, Batch Processing & Python Consulting
   - http://www.simplistix.co.uk
--
http://mail.python.org/mailman/listinfo/python-list

Re: organizing your scripts, with plenty of re-use

[Processor-Dev1l]

On Oct 5, 8:29 pm, Robert Kern  wrote:
> On 2009-10-05 12:42 PM, Buck wrote:
>
>
>
> >> With the package layout, you would just do:
>
> >>     from parrot.sleeping import sleeping_in_a_bed
> >>     from parrot.feeding.eating import eat_cracker
>
> >> This is really much more straightforward than you are making it out to be.
>
> > As in the OP, I need things to "Just Work" without installation
> > requirements.
> > The reason for this is that I'm in a large corporate environment
> > servicing many groups with their own custom environments.
>
> The more ad hoc hacks you use rather than the standard approaches, the harder 
> it
> is going to be for you to support those custom environments.
>
> > Your solution requires work and knowledge on the user's part,
>
> *All* solutions require work and knowledge. There is no free lunch. The
> advantage of standard Python packages is that they are understood the best and
> the most widely.
>
> > but Stef
> > seems to be actually solving the complexity problem. It may seem
> > trivial to you, but it's beyond many people's grasp and brings the
> > usability and reliability of the system way down.
>
> > Like Stef, I was unable to grasp how to properly use python packages
> > in my environment even after reading the documentation repeatedly over
> > the course of several months.
>
> I do believe that you and Stef are exceptions. The vast majority of Python 
> users
> seem to be able to grasp packages well enough.
Well, I started using python many years ago (when there were no
packages), so I am used to manage such messy files as described above.
But it is true that for building larger solution I always use
packages, it just looks better :).
So my solution:
Packages for job tasks and one big messy folder for my private usage
(system administration tasks, automation, etc).
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless enigma
>   that is made terrible by our own mad attempt to interpret it as though it 
> had
>   an underlying truth."
>    -- Umberto Eco

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Creating class instance from module and class name

[Steven D'Aprano]

On Tue, 06 Oct 2009 00:09:08 -0700, gentlestone wrote:

> one more question - __class__ is the way for getting the classname from
> the class instance -

Not quite. __class__ returns the class object, not the name. Given the 
class object, you ask for __name__ to get the name it was defined as[1]. 
For example:

>>> class MyClass:
... pass
...
>>> instance = MyClass()
>>> instance.__class__

>>> MyClass.__name__
'MyClass'
>>> instance.__class__.__name__
'MyClass'


> how can I get the module name?

>>> import math
>>> math.__name__
'math'

But if possible, you should pass around the actual module and class 
objects rather than just their names. The only time you need to use the 
*names* rather than the actual objects themselves is if you are getting 
the information from a source outside of Python, e.g. user input, or a 
text file, etc.

E.g. suppose you have imported the decimal module, and you want access to 
the Decimal class elsewhere. You can pass the names "decimal" and 
"Decimal" to some function, and use __import__() and getattr() to access 
them. Or you can do this:


>>> def example(mod, cls):
... print "Module is", mod
... print "Class is", cls
... return cls(0)  # or whatever you want to do with it...
...
>>>
>>> example(math, decimal.Decimal)  # pass the objects, not the names
Module is 
Class is 
Decimal("0")









[1] But not necessarily the name it has now.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange performance issue

[Steven D'Aprano]

On Mon, 05 Oct 2009 22:31:05 -0700, Dan Stromberg wrote:

> I'm rewriting 3 programs as one program - from Python with Tkinter to
> Python with pygtk, both on Windows XP.
> 
> My new version formats an SD card and preallocates some file space in
> about 3 minutes with "Optimize Performance" selected, and in about 30
> minutes with "Optimize for Quick Removal" selected.  Needless to say, I
> don't like the 27 minute penalty much.

I don't understand what that means. How are you formatting the SD card?

I'm guessing that Optimize for Quick Removal means that every write is 
immediately synced to disk. That will probably be slow, no matter what.


> But the old version of the program formats and preallocates in 3 minutes
> irrespective of whether the drive is in "optimize performance" or
> "optimize for quick removal".

I suspect that if there was no performance penalty in the old version, it 
was because you inadvertently were always using "Optimize Performance" no 
matter what.

BTW, if you want to pre-allocate some space, and you don't care what is 
in the file, you *may* be able to use file.truncate() to do so quickly. 
Despite the name, it doesn't just truncate files, it can also be used to 
extend them. (At least on POSIX systems.)


>>> f = open('empty', 'w')
>>> f.seek(1000)
>>> f.truncate()
>>> f.close()
>>> f = open('empty', 'r')
>>> len(f.read())
1000
>>> f.close()



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Creating class instance from module and class name

[Diez B. Roggisch]

Steven D'Aprano schrieb:

On Mon, 05 Oct 2009 23:32:27 -0700, gentlestone wrote:


Suppose I've saved the class name and (don't know how) I've also saved
the Class's module (package path or I don't know what's the name for XYZ
"from X.Y.Z import ...). How can I construct a new class according to
saved informations? If I don't know what Class it could be, only I have
the saved Class name?



If you have the module *object*, and the name of the class, then you can 
do this:


theclass = getattr(module, "MyClass")

to get the class itself, and then call it as normal to instantiate it:

instance = theclass(args)

Classes are just like any other object in that regard.

If you have the *name* of the module, you can import it first to get the 
module object:


module = __import__('module_name')


Not working for modules in packages, it will return the toplevel module 
only. I never remember if there is a convenience-function, what I 
usually do is this:


module = __import__(module_name)
for part in module_name.split(".")[1:]:
module = getattr(module, part)



Diez
--
http://mail.python.org/mailman/listinfo/python-list

Re: Creating class instance from module and class name

[Bruno Desthuilliers]

gentlestone a écrit :

(snip)


one more question - __class__ is the way for getting the classname
from the class instance -


obj.__class__ is a reference to the class object itself, not it's name.


how can I get the module name?


module_obj.__name__

And while we're at it: obj.__class__.__module__ might be of some 
interest too.


--
http://mail.python.org/mailman/listinfo/python-list

unittest.TestCase and functools.partial don't seem to mix

[Joel Smith]

Hi List,
I want to make some test case classes that can have some data passed in 
to modify the way they behave. I can't see a straightforward manner to 
pass data to an __init__() method of a class derived from 
unittest.TestCase, or to pass data to a test function within that 
class.  Being a C++ guy, I thought "If only Python had something 
equivalent to boost::bind, I'd be fine," and then I discovered 
functools.partial. I found a post showing how to create classes using 
partial, and I thought I was good to go. The post I found is here:


http://mail.python.org/pipermail/bangpypers/2008-December/000974.html

So I adapted that code to see if it worked in the context of unittest. 
When I run the code, it acts like the parameter I provided with partial 
isn't even there. Any ideas?


#!/usr/bin/python

import functools
import unittest

class GenericWindow:
  def __init__(self, name, width, height, color='white'):
 print('Name: %s, width: %d, height: %d, color: %s' % (name, width, 
height, color))


class TestGenericWindow(unittest.TestCase):
  def __init__(self, methodName, color):
 unittest.TestCase.__init__(self, methodName)
 print('color: %s' % color)
 self.color = color

  def testit():
 GenericWindow('foo', width=100, height=100, color=self.color)

def suite():
  s = unittest.Suite()
  BrownWindowTest = functools.partial(TestGenericWindow, color='brown')
  BlueWindowTest = functools.partial(TestGenericWindow, color='blue')
  GreenWindowTest = functools.partial(TestGenericWindow, color='green')
  s.addTest(unittest.makeSuite(BrownWindowTest))
  s.addTest(unittest.makeSuite(BlueWindowTest))
  s.addTest(unittest.makeSuite(GreenWindowTest))
  return s

if __name__ == '__main__': #unittest.main()
  unittest.main()

That code gives the following:

Traceback (most recent call last):
  File "./functools.partial.py", line 32, in 
unittest.main()
  File "/usr/lib/python2.6/unittest.py", line 816, in __init__
self.parseArgs(argv)
  File "/usr/lib/python2.6/unittest.py", line 837, in parseArgs
self.test = self.testLoader.loadTestsFromModule(self.module)
  File "/usr/lib/python2.6/unittest.py", line 559, in loadTestsFromModule
tests.append(self.loadTestsFromTestCase(obj))
  File "/usr/lib/python2.6/unittest.py", line 550, in loadTestsFromTestCase
return self.suiteClass(map(testCaseClass, testCaseNames))
TypeError: __init__() takes exactly 3 arguments (2 given)

Thanks for having a look,
Joel
--
http://mail.python.org/mailman/listinfo/python-list

WMI remote call in python script to create process on remote windows computer

[davidj411]

import win32com.client
computer = "server"
strUser = "server\user_name"
strPassword ="my_password"
objSWbemLocator = win32com.client.Dispatch
("WbemScripting.SWbemLocator")
objSWbemServices = objSWbemLocator.ConnectServer(computer, "root
\cimv2",strUser,strPassword)
objCreateProc = objSWbemServices.Get("Win32_Process")
ProcessID  = u"200"
objCreateProc.Create(u"cabarc n c:\\temp\\test123.PY.cab c:\\temp\
\dmj.new.log",u"c:\\temp",u' ',ProcessID )

error returned:
Traceback (most recent call last):
  File "", line 1, in 
  File "C:\path\remote.process.py", line 20, in 
objCreateProc.Create(u"cmd /c cabarc n c:\\temp\\test123.PY.cab c:\
\temp\\new.log",u"c:\\temp",u'',ProcessID )

what is causing this "int" error?

i have tried this with and without Unicode.
i'd like to do this without having a need to use Tim Golden's wmi
module.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why is this slower?

[nn]

On Oct 5, 12:46 pm, Joseph Reagle  wrote:
> I would think the commented code would be faster (fewer loops), but it is
> not (because of function calls).
>
>     #Average user_time = 5.9975 over 4 iterations
>     inSRC = set([bio.name for bio in bios.values()])
>     inEB = set([bio.name for bio in bios.values() if bio.eb_title])
>     inWP = set([bio.name for bio in bios.values() if bio.wp_title])
>     inBoth = inEB & inWP
>     missingEB = inSRC - inEB
>     missingWP = inSRC - inWP
>     missingBoth = missingEB & missingWP
>     avg_size_ratio = find_average(
>         [bio.wp_ratio for bio in bios.values() if bio.wp_wc and bio.eb_wc])
>     mdn_size_ratio = find_median(
>         [bio.wp_ratio for bio in bios.values() if bio.wp_wc and bio.eb_wc])
>     SRCfem = set([bio.name for bio in bios.values() if bio.gender
> == 'female'])
>     EBfem = set([bio.name for bio in bios.values() if bio.eb_gender
> == 'female'])
>     WPfem = set([bio.name for bio in bios.values() if bio.wp_gender
> == 'female'])
>     SRCmale = set([bio.name for bio in bios.values() if bio.gender
> == 'male'])
>     EBmale = set([bio.name for bio in bios.values() if bio.eb_gender
> == 'male'])
>     WPmale = set([bio.name for bio in bios.values() if bio.wp_gender
> == 'male'])
>     SRCun = set([bio.name for bio in bios.values() if bio.gender
> == 'unknown'])
>     EBun = set([bio.name for bio in bios.values() if bio.eb_gender
> == 'unknown'])
>     WPun = set([bio.name for bio in bios.values() if bio.wp_gender
> == 'unknown'])
>
>     ##Average user_time = 6.0025 over 4 iterations
>     #def set_amend(obj, bio):
>         #if obj == None:
>             #obj = set([])
>         #obj.add(bio.name)
>         #return obj
>
>     #inSRC = set([])
>     #inSRC = set([])
>     #inEB = set([])
>     #inWP = set([])
>     #SRCfem = set([])
>     #EBfem = set([])
>     #WPfem = set([])
>     #SRCmale = set([])
>     #EBmale = set([])
>     #WPmale = set([])
>     #SRCun = set([])
>     #EBun = set([])
>     #WPun = set([])
>
>     #for bio in bios.values():
>         ### use a function that takes set name (creates one) and conditional
>         #inSRC = set_amend(inSRC, bio)
>         #if bio.eb_title: inEB = set_amend(inEB, bio)
>         #if bio.wp_title: inWP = set_amend(inWP, bio)
>         #if bio.gender == 'female': SRCfem = set_amend(SRCfem, bio)
>         #if bio.eb_gender == 'female': EBfem = set_amend(EBfem, bio)
>         #if bio.wp_gender == 'female': WPfem = set_amend(WPfem,bio)
>         #if bio.gender == 'male': SRCmale = set_amend(SRCmale, bio)
>         #if bio.eb_gender == 'male': EBmale = set_amend(EBmale, bio)
>         #if bio.wp_gender == 'male': WPmale = set_amend(WPmale, bio)
>         #if bio.gender == 'unknown': SRCun = set_amend(SRCun, bio)
>         #if bio.eb_gender == 'unknown': EBun = set_amend(EBun, bio)
>         #if bio.wp_gender == 'unknown': WPun = set_amend(WPun, bio)
>     #inBoth = inEB & inWP
>     #missingEB = inSRC - inEB
>     #missingWP = inSRC - inWP
>     #missingBoth = missingEB & missingWP
>     #avg_size_ratio = find_average(
>         #[bio.wp_ratio for bio in bios.values() if bio.wp_wc and bio.eb_wc])
>     #mdn_size_ratio = find_median(
>         #[bio.wp_ratio for bio in bios.values() if bio.wp_wc and bio.eb_wc])

Not only are you doing many function calls but you are assigning 12
objects each time. Why not do this?

for bio in bios.values():
 inSRC.add(bio)
 if bio.eb_title: inEB.add(bio)
 if bio.wp_title: inWP.add(bio)
 if bio.gender == 'female': SRCfem.add(bio)
 if bio.eb_gender == 'female': EBfem.add(bio)
 if bio.wp_gender == 'female': WPfem.add(bio)
 if bio.gender == 'male': SRCmale.add(bio)
 if bio.eb_gender == 'male': EBmale.add(bio)
 if bio.wp_gender == 'male': WPmale.add(bio)
 if bio.gender == 'unknown': SRCun.add(bio)
 if bio.eb_gender == 'unknown': EBun.add(bio)
 if bio.wp_gender == 'unknown': WPun.add(bio)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Enormous Input and Output Test

[n00m]

> 50-80%% of users from the 1st page in ranklist are
> super-extra-brilliant

#5 there: http://www.spoj.pl/users/tourist/
This 14 y.o. schoolboy won IOI 2009, in this August,
and he's about to get into Guiness' book as the youngest
winner for all the history of international olympiads on
informatics.
He lives at 20 minutes of walking from my house, -- it's
South East Belarus, near to Chernobyl (mutants?).
Hardly I'm able to do even 10% of what he can do, though
I'm older by 3 times than him.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Enormous Input and Output Test

[n00m]

> This takes 5 second on my machine using a file with 1,000,000 random...

Surely it will fail to pass time limit too
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to sort a list of strings on a substring

[n00m]

Here you are:

LogList = [\
"inbound tcp office 192.168.0.125 inside 10.1.0.91 88",
"inbound tcp office 192.168.0.220 inside 10.1.0.31 2967",
"inbound udp lab 172.24.0.110 inside 10.1.0.6 161",
"inbound udp office 192.168.0.220 inside 10.1.0.13 53"]


LogList.sort(key=lambda x: x[x.index('1'):])

for item in LogList:
print item

===

inbound udp lab 172.24.0.110 inside 10.1.0.6 161
inbound tcp office 192.168.0.125 inside 10.1.0.91 88
inbound udp office 192.168.0.220 inside 10.1.0.13 53
inbound tcp office 192.168.0.220 inside 10.1.0.31 2967
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Enormous Input and Output Test

[n00m]

What happened to performance of ver.2.6.2 (vs ver.2.5.x)?

https://www.spoj.pl/forum/viewtopic.php?f=20&t=5949
-- 
http://mail.python.org/mailman/listinfo/python-list

Why is this slower?

[Joseph Reagle]

I would think the commented code would be faster (fewer loops), but it is
not (because of function calls).


#Average user_time = 5.9975 over 4 iterations
inSRC = set([bio.name for bio in bios.values()])
inEB = set([bio.name for bio in bios.values() if bio.eb_title])
inWP = set([bio.name for bio in bios.values() if bio.wp_title])
inBoth = inEB & inWP
missingEB = inSRC - inEB
missingWP = inSRC - inWP
missingBoth = missingEB & missingWP
avg_size_ratio = find_average(
[bio.wp_ratio for bio in bios.values() if bio.wp_wc and bio.eb_wc])
mdn_size_ratio = find_median(
[bio.wp_ratio for bio in bios.values() if bio.wp_wc and bio.eb_wc])
SRCfem = set([bio.name for bio in bios.values() if bio.gender
== 'female'])
EBfem = set([bio.name for bio in bios.values() if bio.eb_gender
== 'female'])
WPfem = set([bio.name for bio in bios.values() if bio.wp_gender
== 'female'])
SRCmale = set([bio.name for bio in bios.values() if bio.gender
== 'male'])
EBmale = set([bio.name for bio in bios.values() if bio.eb_gender
== 'male'])
WPmale = set([bio.name for bio in bios.values() if bio.wp_gender
== 'male'])
SRCun = set([bio.name for bio in bios.values() if bio.gender
== 'unknown'])
EBun = set([bio.name for bio in bios.values() if bio.eb_gender
== 'unknown'])
WPun = set([bio.name for bio in bios.values() if bio.wp_gender
== 'unknown'])

##Average user_time = 6.0025 over 4 iterations
#def set_amend(obj, bio):
#if obj == None:
#obj = set([])
#obj.add(bio.name)
#return obj

#inSRC = set([])
#inSRC = set([])
#inEB = set([])
#inWP = set([])
#SRCfem = set([])
#EBfem = set([])
#WPfem = set([])
#SRCmale = set([])
#EBmale = set([])
#WPmale = set([])
#SRCun = set([])
#EBun = set([])
#WPun = set([])

#for bio in bios.values():
### use a function that takes set name (creates one) and conditional
#inSRC = set_amend(inSRC, bio)
#if bio.eb_title: inEB = set_amend(inEB, bio)
#if bio.wp_title: inWP = set_amend(inWP, bio)
#if bio.gender == 'female': SRCfem = set_amend(SRCfem, bio)
#if bio.eb_gender == 'female': EBfem = set_amend(EBfem, bio)
#if bio.wp_gender == 'female': WPfem = set_amend(WPfem,bio)
#if bio.gender == 'male': SRCmale = set_amend(SRCmale, bio)
#if bio.eb_gender == 'male': EBmale = set_amend(EBmale, bio)
#if bio.wp_gender == 'male': WPmale = set_amend(WPmale, bio)
#if bio.gender == 'unknown': SRCun = set_amend(SRCun, bio)
#if bio.eb_gender == 'unknown': EBun = set_amend(EBun, bio)
#if bio.wp_gender == 'unknown': WPun = set_amend(WPun, bio)
#inBoth = inEB & inWP
#missingEB = inSRC - inEB
#missingWP = inSRC - inWP
#missingBoth = missingEB & missingWP
#avg_size_ratio = find_average(
#[bio.wp_ratio for bio in bios.values() if bio.wp_wc and bio.eb_wc])
#mdn_size_ratio = find_median(
#[bio.wp_ratio for bio in bios.values() if bio.wp_wc and bio.eb_wc])


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: mktime, how to handle dates before 01-01-1970 ?

[Ben Finney]

Stef Mientki  writes:

> I'ld expect times before 1-1-1970, simply to become negative numbers
> (I'm interested in the age of living people, so that would suffice).
>
> Is there a general solution, (other library) or would it be better to
> handle all dates in the Delphi format (number of days since 1-1-1900

There are many epochs that have been used in computing
http://en.wikipedia.org/wiki/Epoch_(reference_date)#Notable_epoch_dates_in_computing>,
all of which have their problems. Switching from the Unix epoch to some
other involves data conversion, and (as you point out) raises concerns
about library support.

If you're committed to changing the epoch anyway, I would recommend
using http://en.wikipedia.org/wiki/Astronomical_year_numbering>
(epoch at 4004 BCE) since it is widely used to unify dates referring to
human history.

-- 
 \   “The surest way to corrupt a youth is to instruct him to hold |
  `\   in higher esteem those who think alike than those who think |
_o__)   differently.” —Friedrich Nietzsche |
Ben Finney
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: mktime, how to handle dates before 01-01-1970 ?

[Ben Finney]

Ben Finney  writes:

> There are many epochs that have been used in computing
> http://en.wikipedia.org/wiki/Epoch_(reference_date)#Notable_epoch_dates_in_computing>,
> all of which have their problems. Switching from the Unix epoch to some
> other involves data conversion, and (as you point out) raises concerns
> about library support.

Hmm, none of this addressed your question about what *data type* to use.
For that, the answer is: use the standard-library ‘datetime’ module's
types (‘date’, ‘timestamp’, etc.) in your code for representing the
actual values.

The advice for date and time values is similar to the advice for text
values: convert input to ‘datetime’ types as soon as possible, and
render output as late as possible; do all processing on the ‘datetime’
types to ensure preservation of information.

-- 
 \“I filled my humidifier with wax. Now my room is all shiny.” |
  `\—Steven Wright |
_o__)  |
Ben Finney
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: WMI remote call in python script to create process on remote windows computer

[Tim Golden]

davidj411 wrote:

import win32com.client
computer = "server"
strUser = "server\user_name"
strPassword ="my_password"
objSWbemLocator = win32com.client.Dispatch
("WbemScripting.SWbemLocator")
objSWbemServices = objSWbemLocator.ConnectServer(computer, "root
\cimv2",strUser,strPassword)
objCreateProc = objSWbemServices.Get("Win32_Process")
ProcessID  = u"200"
objCreateProc.Create(u"cabarc n c:\\temp\\test123.PY.cab c:\\temp\
\dmj.new.log",u"c:\\temp",u' ',ProcessID )

error returned:
Traceback (most recent call last):
  File "", line 1, in 
  File "C:\path\remote.process.py", line 20, in 
objCreateProc.Create(u"cmd /c cabarc n c:\\temp\\test123.PY.cab c:\
\temp\\new.log",u"c:\\temp",u'',ProcessID )

what is causing this "int" error?


Not clear which "int" error you mean, but trying to help anyway...

You need to use raw strings (or use double-backslashes). That
may or may not be the cause of your problems, but it certainly
won't help. You may also run into theoretical restrictions of
what remote WMI processes can achieve (no desktop/windowstation etc.).



i have tried this with and without Unicode.
i'd like to do this without having a need to use Tim Golden's wmi
module.


To ask the obvious question: why? I'm not offended, just curious :)

TJG
--
http://mail.python.org/mailman/listinfo/python-list

Re: Skeletal animation

[Tim Wintle]

On Mon, 2009-10-05 at 18:36 +0200, Donn wrote:
> see: http://www.panda3d.org/wiki/index.php/Attaching_an_Object_to_a_Joint

+1 for Panda 3d.

Their Node graph is very intuitive for animating joints, and Panda's API
is very nice and clean.

TimW

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: unicode issue

[Gabriel Genellina]

En Thu, 01 Oct 2009 12:10:58 -0300, Walter Dörwald   
escribió:

On 01.10.09 16:09, Hyuga wrote:

On Sep 30, 3:34 am, gentlestone  wrote:



_MAP = {
# LATIN
u'À': 'A', u'Á': 'A', u'Â': 'A', u'Ã': 'A', u'Ä': 'A', u'Å': 'A',
u'Æ': 'AE', u'Ç':'C', [...long table...]
}

def downcode(name):
"""
>>> downcode(u"Žabovitá zmiešaná kaša")
u'Zabovita zmiesana kasa'
"""
for key, value in _MAP.iteritems():
name = name.replace(key, value)
return name


import unicodedata

def downcode(name):
   return unicodedata.normalize("NFD", name)\
  .encode("ascii", "ignore")\
  .decode("ascii")


This article [1] shows a mixed technique, decomposing characters when such  
info is available in the Unicode tables, and also allowing for a custom  
mapping when not.


[1] http://effbot.org/zone/unicode-convert.htm

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: epydoc - can I provide multiple dirs to parse

[Jean-Michel Pichavant]

Montaseri wrote:

Thank you

It looks like it is possible to feed multiple dirs,

Also, can you explain the --inheritance=STYLE for me. What does the 
author mean by values like "grouped", "listed", "included"

I could not find any good document explaining these...

Thanks
Medi
Sorry, I don't remember exactly. It is related somehow to how epydoc 
will manage inheritance, for example if C inherits from A, in C  
documention, epydoc can include the documentation of A methods like if  
they were defined in C, or it can just list the method names, or 
whatever ... just try the options on one file and you'll get a better 
view on this option.


JM


--
http://mail.python.org/mailman/listinfo/python-list

Re: Object Relational Mappers are evil (a meditation)

[Bruno Desthuilliers]

Carl Banks a écrit :

On Oct 5, 7:25 am, Aaron Watters  wrote:

This is a bit off topic except that many Python
programmers seem to be allergic to typing SQL.

RESOLVED:  Using ORMs leads lazy programmers
to make bad database designs.  It's better to
carefully design your database with no invisible
means of support and there is no reason to not
use SQL directly for this purpose.


Yeah sure, whatever.  I'm sure a good programmer could use sql
directly and produce a tighter, faster, better-performing application
than an ORM-solution, same as you could use C to produce a tighter,
faster, better-performing application than a pure Python one.


It's certainly way easier to write good SQL than to write a good C 
program, and even when using an ORM, it's sometimes worth "going down" 
to hand-written SQL queries for some more or less complex cases. But I 
guess this falls in the same category as recoding a couple core 
functions in C for improved performances.


Isn't WordPress written in PHP?  Are ORMs even possible in PHP? 


There are (alas) some attempts. Nothing close to SQLAlchemy nor even 
Django's - PHP is just not hi-level enough.



I can
almost rationalize use of direct sql if the alternative is some
hellspawn PHP ORM.


What _could_ possibly be done in PHP would at least be a higher-level 
representation of sql expressions, that would make it easier to 
dynamically build / modifie / combine queries. Working with "SQL as raw 
strings" is both tiresome and error-prone when it comes to dynamically 
generate complex queries. That part is IMHO *much* more important than 
mapping tuples to objects.

--
http://mail.python.org/mailman/listinfo/python-list

Re: Object Relational Mappers are evil (a meditation)

[Bruno Desthuilliers]

Diez B. Roggisch a écrit :

Aaron Watters schrieb:

(snip)

FOR EXAMPLE:  Consider blogging.  The most
successful blog software is WORDPRESS.  Here
is the WordPress data model:

http://blog.kapish.co.in/wp-content/uploads/2009/03/wp_2.7.png

(snip)

Now let's look at the Sakai Blogger tool data model

(snip)

confluence.sakaiproject.org/download/attachments/17072138/
entitymodel.pdf

I think your example is nonsense. Just comparing the two models based on 
"they both are for blogging" 


FWIW, sakai seems to be a wiki, not a blog. And I would add that not 
having a nice graphical *representation* of a schema doesn't imply the 
schema is wrong from a relational design POV.


--
http://mail.python.org/mailman/listinfo/python-list

Re: Is there a way to specify a superclass at runtime?

[Mick Krippendorf]

Chris Colbert schrieb:
> 
> SIMULATION = False
> 
> class SimController(object):
> "do sim stuff here"
> 
> class RealController(object):
> " do real stuff here"
> 
> class Controller(SuperKlass):
> pass
> 
> 
> so if SIMULATION == False I want to be able to instance a Controller
> object that inherits from RealController and vice-versa.

How about:

SIMULATION = True

class Controller(object):
def __new__(cls):
if SIMULATION:
return SimController()
else:
return RealController()

class SimController(object):
pass

class RealController(object):
pass

print Controller()


But my preferred solution (in case SIMULATION never changes during
runtime) would be:


class SimController(object):
pass

class RealController(object):
pass

if SIMULATION:
Controller = SimController
else:
Controller = RealController

print Controller()


Regards,
Mick.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: organizing your scripts, with plenty of re-use

[Gabriel Genellina]

En Mon, 05 Oct 2009 18:15:15 -0300, Rami Chowdhury  
 escribió:



On Mon, 05 Oct 2009 13:46:09 -0700, Buck  wrote:


Thanks. I think we're getting closer to the core of this.

To restate my problem more simply:

My core goal is to have my scripts in some sort of organization better
than a single directory, and still have plenty of re-use between them.
The only way I can see to implement this is to have 10+ lines of
unintelligible hard-coded boilerplate in every runnable script.
That doesn't seem reasonable or pythonic.


If there's a standard directory you expect them to be dropped into by  
your users (e.g. $HOME/scripts) then surely you could do something like:


import mypathmunge

at the top of every script, and then have a mypathmunge.py in  
site-packages that goes:


# mypathmunge.py
import sys, os
sys.path.insert(0, os.path.join(os.getenv('HOME'), 'scripts'))


Since Python 2.6 and up, you don't even need that. There exists a per-user  
site directory (~/.local/lib/python2.6/site-packages on Linux; under  
%APPDATA% on Windows) that is prepended to sys.path automatically; .pth  
files found there are honored too (see PEP 370 [1]).


So there is no need to set the PYTHONPATH variable, nor alter the standard  
site-packages directory, nor play tricks with sys.path - just install the  
modules/packages inside the user site directory (or any other directory  
named in a .pth file found there).


[1] http://www.python.org/dev/peps/pep-0370/

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: WMI remote call in python script to create process on remote windows computer

[Dave Angel]

davidj411 wrote:

import win32com.client
computer = "server"
strUser = "server\user_name"
strPassword ="my_password"
objSWbemLocator = win32com.client.Dispatch
("WbemScripting.SWbemLocator")
objSWbemServices = objSWbemLocator.ConnectServer(computer, "root
\cimv2",strUser,strPassword)
objCreateProc = objSWbemServices.Get("Win32_Process")
ProcessID  = u"200"
objCreateProc.Create(u"cabarc n c:\\temp\\test123.PY.cab c:\\temp\
\dmj.new.log",u"c:\\temp",u' ',ProcessID )

error returned:
Traceback (most recent call last):
  File "", line 1, in 
  File "C:\path\remote.process.py", line 20, in 
objCreateProc.Create(u"cmd /c cabarc n c:\\temp\\test123.PY.cab c:\
\temp\\new.log",u"c:\\temp",u'',ProcessID )

what is causing this "int" error?

i have tried this with and without Unicode.
i'd like to do this without having a need to use Tim Golden's wmi
module.

  
As Tim says, you need to double your backslashes.  You have several 
places where they are not properly escaped.  Alternatively, at least in 
all these cases, you could use the raw strings.


But the real problem with your message would seem to be that you retyped 
the material, with typos.  And didn't include the entire error message.  
Copy/paste are your friend.  And in case you don't know how to do that 
from a Windows console window, just ask.  Or try Right-click on the 
title bar for some ideas.


I don't have specific answers for you -- I don't currently do remote 
stuff this way.  But you'll get better answers if you try the above.


DaveA
--
http://mail.python.org/mailman/listinfo/python-list

Re: Strange performance issue

[Ulrich Eckhardt]

Dan Stromberg wrote:
> My new version formats an SD card and preallocates some file space in
> about 3 minutes with "Optimize Performance" selected, and in about 30
> minutes with "Optimize for Quick Removal" selected.  Needless to say, I
> don't like the 27 minute penalty much.

For performance, the driver will probably use delayed write operations,
buffering etc. For quick removal, it will probably make all operations
atomic, i.e. perform a write operation and not return until the data has
really reached the SD card.

> But the old version of the program formats and preallocates in 3 minutes
> irrespective of whether the drive is in "optimize performance" or
> "optimize for quick removal".

Somehow, I guess the new version does many small writes while the old one
doesn't. When optimizing for quick removal, those operations add up,
otherwise their overhead is negligible.

> one_meg = 'a'*2**20

That's one mib, not one meg. ;)

> file = open('remove-me', 'w')

Try adding the 'b' flag, too. I wouldn't expect it to affect the IO speed,
but it is one factor that adds up. Otherwise, I looked over your program
and couldn't find anything that would explain a problem. Just wondering,
what transfer speed do you get with the two versions? What is the
theoretical maximum for the SD card?

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: unittest.TestCase and functools.partial don't seem to mix

[Gabriel Genellina]

En Mon, 05 Oct 2009 23:12:02 -0300, Joel Smith   
escribió:


I want to make some test case classes that can have some data passed in  
to modify the way they behave. I can't see a straightforward manner to  
pass data to an __init__() method of a class derived from  
unittest.TestCase, or to pass data to a test function within that  
class.  Being a C++ guy, I thought "If only Python had something  
equivalent to boost::bind, I'd be fine," and then I discovered  
functools.partial. I found a post showing how to create classes using  
partial, and I thought I was good to go. The post I found is here:


http://mail.python.org/pipermail/bangpypers/2008-December/000974.html

So I adapted that code to see if it worked in the context of unittest.  
When I run the code, it acts like the parameter I provided with partial  
isn't even there. Any ideas?


The problem isn't with partial - which works fine. By using  
unittest.main() your're building the default test suite using the default  
rules - your customized suite isn't used at all. Try with:


if __name__ == '__main__':
unittest.TextTestRunner().run(suite())

Note that you don't *have* to use partial in this case, as you're building  
the suite yourself. Just create the TestCase instances manually:


suite = unittest.TestSuite([
  TestGenericWindow('testit', 'brown'),
  TestGenericWindow('testit', 'blue'),
  TestGenericWindow('testit', 'green')
  ])
unittest.TextTestRunner().run(suite)

You may get rid of the 'testit' repetitive argument if you omit it from  
__init__ -- the generic TestLoader won't like it, but it doesn't like it  
right now because of the color argument, so you don't lose anything.


Another approach, if you want to stay with the built-in testing  
infrastructure:


class TestGenericWindow(unittest.TestCase):

  def _testit(self, color):
 # actual test implementation
 GenericWindow('foo', width=100, height=100, color=color)

  def test_brown(self):
 return self._testit('brown') # provide desired parameters

  def test_blue(self):
 return self._testit('blue')

unittest.main() should still work with this TestCase as written, without a  
custom Suite/Loader.


--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: Q: sort's key and cmp parameters

[Bearophile]

Raymond Hettinger:

> Psychologically, the thing that I find to be interesting is
> that beginners and intermediate users seem to take to key
> functions more readily than old timers.  The key function seems
> to be an easy thing to teach (perhaps because that's the
> way Excel sorts and the way SQL sorts, or perhaps it is because
> problem statements seem to arise in the form of "sort by this,
> then by that" instead of "here's how two objects should be
> compared").
>
> In contrast, some people who have have had deep experience with
> cmp functions may tend to first think of cmp solutions and then
> have a harder time seeing solutions with key functions.  If you
> grew-up on C's qsort() like I did, then a key function may not
> be the first thing that pops into your head.

I love the 'key', it makes my code simpler and it's simpler to
understand. I am not opposed to the idea of keeping cmp, that in some
rare cases may be useful or more natural.

The problem is that if you allow to use the cmp, lot of programmers
will use it straight away, not even bothering to know what that
strange 'key' argument may be useful for. And they miss the how much
handy 'key' is.

I am having a very hard type (despite my implementation, explanations,
etc) explaining to D programmers why for a flexible general-purpose
function a key function argument is often better. They in the end have
added something like that in the std lib, but with a weird name like
SchwartzSort that's surely not the first standard sort they look
for... this is bad.

So a funny solution to this situation may be the following: to keep
only 'key' in the sort/sorted for few years, so the community of
Python programmers gets used to 'key' only, and then put 'cmp' back
in, for the special cases ;-)

Bye,
bearophile
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: 'Once' properties.

[Scott David Daniels]

Scott David Daniels wrote:

...
Look into metaclasses:

...

class Initialized(ClassBase):
@classmethod
def _init_class(class_):
class_.a, class_.b = 1, 2
super(Initialized, class_)._init_class()


Mea culpa:  Here super is _not_ a good idea, and I had tried that
and recoded, but cut and pasted the wrong code.  I just noticed
that I had done so this morning.

class Initialized(ClassBase):
@classmethod
def _init_class(class_):
class_.a, class_.b = 1, 2
ClassBase._init_class()

print Initialized.a, Initialized.b

Much better.  There is probably a way to get to the MRO, but for now,
this should do.

--Scott David Daniels
scott.dani...@acm.org
--
http://mail.python.org/mailman/listinfo/python-list

Re: Problem with subprocess module on Windows with open file in append mode

[MRAB]

Gabriel Genellina wrote:
En Sat, 03 Oct 2009 21:53:12 -0300, Andrew Savige 
 escribió:



When I run this little test program on Linux:

import subprocess
subprocess.call(["python","-V"], stderr=open("log.tmp","a"))

the file log.tmp is appended to each time I run it.
When I run it on Windows, however, the file log.tmp gets
overwritten each time I run it.

Though I can make it append on Windows like this:

import os
import subprocess
f = open("log.tmp", "a")
f.seek(0, os.SEEK_END)
subprocess.call(["python","-V"], stderr=f)

I don't understand why that should be necessary.

Is this a Python/subprocess bug on Windows?


No, it's an "implementation-defined behavior" of the underlying C 
library. From the C89 standard:


"""4.9.3 Files

[...] If a
file can support positioning requests (such as a disk file, as opposed
to a terminal), then a file position indicator associated with
the stream is positioned at the start (character number zero) of the
file, unless the file is opened with append mode in which case it is
implementation-defined whether the file position indicator is
positioned at the beginning or the end of the file."""

The Linux implementation choose to move the file pointer to the end in 
such cases, the Windows one choose to always keep it at the start. Both 
are correct according to the specification above. Python 2.X just 
inherits that behavior from C. If you want to write portable code, 
you'll have to use the longer form.



That's strange: whenever I've opened a file for append in Microsoft
Visual C/C++ (or Python or Delphi, for that matter) on Windows I've
assumed that the file pointer would be at the end of file, and that's
always proved to be the case!
--
http://mail.python.org/mailman/listinfo/python-list

Re: Module inspect Bug

[Tomas Zulberti]

On Oct 6, 1:36 am, "Gabriel Genellina"  wrote:
> En Mon, 05 Oct 2009 11:59:01 -0300, Tomas Zulberti   
> escribió:
>
> > Hi. I have a class that extends collections.MutableMapping. I am
> > checking if it is abstract, using the moduleinspect. But isabstract
> > returns a number different from zero insted of True or False. The
> > problem with that is that sometimes it returns False when the class
> > isn't an abstract.
>
> inspect.isabstract(collections.MutableMapping)
> > 1048576
> inspect.isabstract(os)
> > False
>
> > Its true that the condition nevertheless will be True on the if, but
> > the return value I think that should be boolean.
>
> It would be nice ifinspect.isabstract() returned True/False, but 1048576  
> is as good as any other true value, ok? You should not rely on specific  
> values, nor compare the result against True nor False directly.
>


> Anyway, given that no other isXXX function behaves that way, and  
> inspect.isgeneratorfunction() uses a similar construct but always returns  
> True/False, I've submitted a bug+patch:http://bugs.python.org/issue7069
>

Thanks for the help... I wasn't using that the returned value was a
boolean, but the other methods returned a boolean, so I found that was
strange...

Thank you a lot,
Tomas Zulberti
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Object Relational Mappers are evil (a meditation)

[Simon Forman]

On Tue, Oct 6, 2009 at 2:00 AM, Carl Banks  wrote:
> On Oct 5, 7:25 am, Aaron Watters  wrote:
>> This is a bit off topic except that many Python
>> programmers seem to be allergic to typing SQL.
>>
>> RESOLVED:  Using ORMs leads lazy programmers
>> to make bad database designs.  It's better to
>> carefully design your database with no invisible
>> means of support and there is no reason to not
>> use SQL directly for this purpose.
>
> Yeah sure, whatever.  I'm sure a good programmer could use sql
> directly and produce a tighter, faster, better-performing application
> than an ORM-solution, same as you could use C to produce a tighter,
> faster, better-performing application than a pure Python one.
>
> No thanks, you can write your databases in database assembly language
> if you want, I'll stick to my ORM, tyvm.
>
> Isn't WordPress written in PHP?  Are ORMs even possible in PHP?  I can
> almost rationalize use of direct sql if the alternative is some
> hellspawn PHP ORM.
>

I recently had the uncomfortable task of picking an ORM to use in a
php project for a start up.

There's something called RedBean http://www.redbeanphp.com/ that's
actually pretty effin sweet (IMHO).  Sweet enough that I'm considering
porting it to python.

It's more of a RAD tool than something you'd want to use in production.

As far as the OP rant goes, my $0.02:  bad programmers will write bad
code in any language, with any tool or system or environment they're
given.  If you want to avoid bad code there's (apparently) no
substitute for smrt programmers who are familiar with the tools
they're using, not just the syntax but the underlying conceptual
models as well.

That said, "I feel ya bro'"
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PIL : How to write array to image ???

[Scott David Daniels]

Mart. wrote:

On Oct 5, 5:14 pm, Martin  wrote:

On Oct 4, 10:16 pm, "Mart."  wrote:






On Oct 4, 9:47 am, Martin  wrote:

On Oct 3, 11:56 pm, Peter Otten <__pete...@web.de> wrote:

Martin wrote:

Dear group
I'm trying to use PIL to write an array (a NumPy array to be exact) to
an image.
Peace of cake, but it comes out looking strange.
I use the below mini code, that I wrote for the purpose. The print of
a looks like expected:
[[ 200.  200.  200. ...,0.0.0.]
 [ 200.  200.  200. ...,0.0.0.]
 [ 200.  200.  200. ...,0.0.0.]
 ...,
 [   0.0.0. ...,  200.  200.  200.]
 [   0.0.0. ...,  200.  200.  200.]
 [   0.0.0. ...,  200.  200.  200.]]
But the image looks nothing like that.
Please see the images on:
http://hvidberg.net/Martin/temp/quat_col.png
http://hvidberg.net/Martin/temp/quat_bw.png
or run the code to see them locally.
Please – what do I do wrong in the PIL part ???
:-? Martin
import numpy as np
from PIL import Image
from PIL import ImageOps
maxcol = 100
maxrow = 100
a = np.zeros((maxcol,maxrow),float)
for i in range(maxcol):
for j in range(maxrow):
if (i<(maxcol/2) and j<(maxrow/2)) or (i>=(maxcol/2) and j>=
(maxrow/2)):
a[i,j] = 200
else:
a[i,j] = 0
print a
pilImage = Image.fromarray(a,'RGB')
pilImage.save('quat_col.png')
pilImage = ImageOps.grayscale(pilImage)
pilImage.save('quat_bw.png')

The PIL seems to copy the array contents directly from memory without any
conversions or sanity check. In your example The float values determine the
gray value of 8 consecutive pixels.
If you want a[i,j] to become the color of the pixel (i, j) you have to use
an array with a memory layout that is compatible to the Image.
Here are a few examples:

import numpy
from PIL import Image
a = numpy.zeros((100, 100), numpy.uint8)
a[:50, :50] = a[50:, 50:] = 255
Image.fromarray(a).save("tmp1.png")
b = numpy.zeros((100, 100, 3), numpy.uint8)
b[:50, :50, :] = b[50:, 50:, :] = [255, 0, 0]
Image.fromarray(b).save("tmp2.png")
c = numpy.zeros((100, 100), numpy.uint32)
c[:50, :50] = c[50:, 50:] = 0xff808000
Image.fromarray(c, "RGBA").save("tmp3.png")

Peter

Thanks All - That helped a lot...
The working code ended with:
imga = np.zeros((imgL.shape[1],imgL.shape[0]),np.uint8)
for ro in range(imgL.shape[1]):
for co in range(imgL.shape[0]):
imga[ro,co] = imgL[ro,co]
Image.fromarray(imga).save('_a'+str(lev)+'.png')

Without knowing how big your image is (can't remember if you said!).
Perhaps rather than looping in the way you might in C for example, the
numpy where might be quicker if you have a big image. Just a
thought...
And a good thought too... 


I think what Martin is telling you is:

Look to numpy to continue working on the array first.

byte_store = imgL.astype(np.uint8)
Image.fromarray(byte_store).save('_a%s.png' % lev)

--Scott David Daniels
scott.dani...@acm.org
--
http://mail.python.org/mailman/listinfo/python-list

Re: Storing a C pointer in a Python class instance

[lallous]

"Carl Banks"  wrote in message 
news:d50bba1e-b272-4e39-8a58-377531278...@z4g2000prh.googlegroups.com...

On Sep 30, 5:24 am, "lallous"  wrote:

Hello

After using the PyCObject, I cannot pickle the class anymore.
Any simple solution to this problem? (or resorting to __reduce__ is the 
only

solution?)



You can't pickle a CObject, you'd have to create a custom type (one
that implements one of the pickling methods) for that.  Or arrange for
whatever object contains the CObject to pack and unpack it manually.

Out of curiosity, what kind of data you storing in this CObject?
Maybe we can help you choose a better way to handle it at the C level.




I am wrapping a C++ pointer with the python object. That way I can tell with 
which C++ object a given python class instance is associated.


The thing is when developing, I need to pickle but I don't need the C++ 
pointer, so I solved the problem with conditional compilation:

- testing: pickle allowed and "This" is stored in the py object
- production code: no need to pickle and "this" and "pyobject" are bound

Thanks,
Elias 


--
http://mail.python.org/mailman/listinfo/python-list

ANN: GUI2Exe 0.4.0

[Andrea Gavana]

Hi All,

I am happy to announce a new release of GUI2Exe (0.4.0).


What is it?
=

GUI2Exe is my first attempt to unify all the available "executable
builders" for Python in a single and simple to use graphical user
interface. At the moment the supported executable builders are:

- py2exe;
- py2app;
- PyInstaller;
- cx_Freeze;
- bbFreeze.


What's new?
=

Bug fixes and implementation of new features, namely:

* [py2exe-only]: Possibility to use UPX compression on dlls/exes while
compiling (PyInstaller has a built-in option for it);
* [py2exe-only]: Automatic generation of simple Inno Setup scripts;
* [py2exe-only]: Support for more keywords in the Target class (i.e.,
all distutils keywords are now supported);
* [py2exe-only]: Easy access to the most recent error log file via the
menu Builds => Examine error log file;
* Easy access to the distribution folder via the menu Builds => Open
distribution folder;
* [py2exe-only]: A new distribution folder "Explorer" dialog allows to
check which PYDs and DLLs are included, and to quickly exclude them
and then rebuild the script, with "undo" capabilities.


Project information
===

Project home page & downloads:
http://code.google.com/p/gui2exe

Bleeding edge SVN repository:
svn checkout http://gui2exe.googlecode.com/svn/trunk/ gui2exe-read-only

Project mailing list:
http://groups.google.com/group/gui2exe


As usual, feedback is more than welcome. Enjoy!


Andrea.

"Imagination Is The Only Weapon In The War Against Reality."
http://xoomer.alice.it/infinity77/
http://thedoomedcity.blogspot.com/
-- 
http://mail.python.org/mailman/listinfo/python-list

package import dangers

[Ethan Furman]

Greetings!

I'm working on a package with multiple modules (and possibly packages), 
and I would like to do it correctly.  :)


I have read of references to possible issues regarding a module being 
imported (and run) more than once, but I haven't been able to find 
actual examples of such failing code.


My google search was fruitless (although still educational !-), so if 
anyone could point me in the right direction I would greatly appreciate it.


~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list

Re: package import dangers

[Diez B. Roggisch]

Ethan Furman wrote:

> Greetings!
> 
> I'm working on a package with multiple modules (and possibly packages),
> and I would like to do it correctly.  :)
> 
> I have read of references to possible issues regarding a module being
> imported (and run) more than once, but I haven't been able to find
> actual examples of such failing code.
> 
> My google search was fruitless (although still educational !-), so if
> anyone could point me in the right direction I would greatly appreciate
> it.

The most common problem is that a file is used as module and as executable
at the same time.

Like this:

--- test.py ---

class Foo(object):
pass


if __name__ == "__main__":
   import test
   assert Foo is test.Foo

---

This will fail when executed from the commandline because the module is
known twice - once as "__main__", once as "test".

So keep your startup-scripts trivial, or don't ever import from them.

You might create similar situations when modifying sys.path to reach *into*
a package - but that would be sick to do anyway.

Other than that, I'm not aware of any issues.

Diez
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: module path?

[Aahz]

In article <0d6b75ee-37ab-4106-8dd0-ff7a1a995...@a7g2000yqo.googlegroups.com>,
akonsu   wrote:
>
>thanks! i did not know about dir() method.

dir() is a function, not a method.
-- 
Aahz (a...@pythoncraft.com)   <*> http://www.pythoncraft.com/

"Normal is what cuts off your sixth finger and your tail..."  --Siobhan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why is this slower?

[Joseph Reagle]

nn wrote:
> Not only are you doing many function calls but you are assigning 12
> objects each time. Why not do this?
> 
> for bio in bios.values():
>  inSRC.add(bio)

That obviously makes sense, but I was trying to get away from the verbosity
of:

inSRC = set([])
inSRC = set([])
inEB = set([])
inWP = set([])
SRCfem = set([])
EBfem = set([])
WPfem = set([])
SRCmale = set([])
EBmale = set([])
WPmale = set([])
SRCun = set([])
EBun = set([])
WPun = set([])
-- 
http://mail.python.org/mailman/listinfo/python-list

del an imported Class at EOF... why?

[Ryan]

Good day all!

I've just inherited a large amount of python code. After spending some
time pour through the code, I've noticed that the original developer
(who is no longer w/ the company) constantly deletes the imported
classes at the end of the .py file. Why would one want to do such a
thing?

Ryan

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: module path?

[Christian Heimes]

Dave Angel wrote:
> The property is called __file__
> 
> So in this case,filename = settings.__file__

Actually it's an attribute set by Python's import machine. Since the
__file__ attribute may contain a relative path it's a good idea to use
os.path.abspath(__file__).

Christian

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange performance issue

[Dan Stromberg]

Steven D'Aprano wrote:

On Mon, 05 Oct 2009 22:31:05 -0700, Dan Stromberg wrote:

  

I'm rewriting 3 programs as one program - from Python with Tkinter to
Python with pygtk, both on Windows XP.

My new version formats an SD card and preallocates some file space in
about 3 minutes with "Optimize Performance" selected, and in about 30
minutes with "Optimize for Quick Removal" selected.  Needless to say, I
don't like the 27 minute penalty much.



I don't understand what that means. How are you formatting the SD card?
  

The FormatEx function via ctypes.
I'm guessing that Optimize for Quick Removal means that every write is 
immediately synced to disk. That will probably be slow, no matter what.
  
Yes, presumably Optimize for Quick Removal is a write-through cache 
(synchronous) and "Optimize for Performance" is a write-back cache 
(asynchronous).
  

But the old version of the program formats and preallocates in 3 minutes
irrespective of whether the drive is in "optimize performance" or
"optimize for quick removal".



I suspect that if there was no performance penalty in the old version, it 
was because you inadvertently were always using "Optimize Performance" no 
matter what.
  
I'm pretty confident that unless Windows was lying to me, that I did 
some tests with Optimize for Performance and some tests without.
BTW, if you want to pre-allocate some space, and you don't care what is 
in the file, you *may* be able to use file.truncate() to do so quickly. 
Despite the name, it doesn't just truncate files, it can also be used to 
extend them. (At least on POSIX systems.)
  
On a POSIX system, I'd normally just seek and write a single null to get 
a file with holes.  But this is not only Windows, but FAT32 - and the 
data will later be read without going through the filesystem.  It seems 
prudent to actually write the blocks this time.



Thanks, and see my other message (to be sent momentarily) for an 
apparent solution.

f = open('empty', 'w')
f.seek(1000)
f.truncate()
f.close()
f = open('empty', 'r')
len(f.read())


1000
  

f.close()





  


--
http://mail.python.org/mailman/listinfo/python-list

Re: Why is this slower?

[Paul Rubin]

Joseph Reagle  writes:
> inSRC = set([bio.name for bio in bios.values()])

You should use:

 inSRC = set(bio.name for bio in bios.values())

without the square brackets.  That avoids creating an intermediate list.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: del an imported Class at EOF... why?

[Carl Banks]

On Oct 6, 10:56 am, Ryan  wrote:
> Good day all!
>
> I've just inherited a large amount of python code. After spending some
> time pour through the code, I've noticed that the original developer
> (who is no longer w/ the company) constantly deletes the imported
> classes at the end of the .py file. Why would one want to do such a
> thing?


Sometimes an object is used only temporarily by a modules, while it's
importing.

In such cases, there are two reasons you might delete the object.  If
it uses a lot of memory you could free up the memory for other
objects.  Also you might want to clean up the module's namespace.

I del objects for these reasons sparingly, usually only when the
object uses a whole lot of memory, or the namespace is very messy.

However, I'm guessing that the person who wrote your code is just
overly paranoid.



Carl Banks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange performance issue

[Dan Stromberg]

Ulrich Eckhardt wrote:

Dan Stromberg wrote:
  

My new version formats an SD card and preallocates some file space in
about 3 minutes with "Optimize Performance" selected, and in about 30
minutes with "Optimize for Quick Removal" selected.  Needless to say, I
don't like the 27 minute penalty much.



For performance, the driver will probably use delayed write operations,
buffering etc. For quick removal, it will probably make all operations
atomic, i.e. perform a write operation and not return until the data has
really reached the SD card.
  
The driver may delay writes, but I'd think it more likely that the 
filesystem or buffer cache would be doing so.

But the old version of the program formats and preallocates in 3 minutes
irrespective of whether the drive is in "optimize performance" or
"optimize for quick removal".



Somehow, I guess the new version does many small writes while the old one
doesn't. When optimizing for quick removal, those operations add up,
otherwise their overhead is negligible.
  

Nope, same block size.

one_meg = 'a'*2**20



That's one mib, not one meg. ;)

  

You're aware that a lot of people are ignoring the new units?

file = open('remove-me', 'w')



Try adding the 'b' flag, too. I wouldn't expect it to affect the IO speed,
but it is one factor that adds up.

This appears to have wholly accounted for the problem!  Thank you very much.

Adding a "b" to my open sped up the writes by a factor of about 15.

 Otherwise, I looked over your program
and couldn't find anything that would explain a problem. Just wondering,
what transfer speed do you get with the two versions? What is the
theoretical maximum for the SD card?
  
I don't know what the theoretical max write speeds of the USB bus, card 
reader and card are, but I was getting over 10x faster using the same 
card and card reader on an ancient Thinkpad running Ubuntu.  With the 
"b" specified, the XP system is now faster than the Thinkpad.


Interesting that the "b" would make such a performance difference.  
You'd think it'd just be, in C, looking for newlines and adding a 
carriage return after them into a buffer of potentially double the size, 
then writing as usual.  The data I've been writing so far -has- no newlines.


Thanks!

--
http://mail.python.org/mailman/listinfo/python-list

Python: Text file insert to MySQL

[Schedule]

Hello,

I am currenty using MySQL 5.1 community server and trying to import the data
of the comma delimited text file into the table using python 2.6 scripts. I
have installed Mysqldb 1.2.2.

follwoing is my script:




   1. import MySQLdb, csv, sys
   2. conn = MySQLdb.connect (host = "localhost",user = "usr", passwd
= "pass",db = "databasename")
   3. c = conn.cursor()
   4. csv_data=csv.reader(file("a.txt"))
   5. for row in csv_data:
   6.   print row
   7.   c.execute("INSERT INTO a (first, last) VALUES (%s, %s), row")
   8. c.commit()
   9. c.close()

import MySQLdb, csv, sys conn = MySQLdb.connect (host = "localhost",user =
"usr", passwd = "pass",db = "databasename") c = conn.cursor()
csv_data=csv.reader(file("a.txt")) for row in csv_data: print row
c.execute("INSERT INTO a (first, last) VALUES (%s, %s), row") c.commit()
c.close()

the contents of the text file eg. :
-
John,Smith
Danie,Thomas
Ronald,Rey


When I execute the statement I get the following error:

C:\Python26\lib\site-packages\MySQLdb\__init__.py:34: DeprecationWarning:
the sets module is deprecate
d
from sets import ImmutableSet
['John', 'Smith']
Traceback (most recent call last):
File "e:\Scripts\test.py", line 10, in 
c.execute("INSERT INTO a (first, last) VALUES (%s, %s), row")
File "C:\Python26\lib\site-packages\MySQLdb\cursors.py", line 166, in
execute
self.errorhandler(self, exc, value)
File "C:\Python26\lib\site-packages\MySQLdb\connections.py", line 35, in
defaulterrorhandler
raise errorclass, errorvalue
_mysql_exceptions.ProgrammingError: (1064, "You have an error in your SQL
syntax; check the manual tha
t corresponds to your MySQL server version for the right syntax to use near
'%s, %s), row' at line 1")


Any kind of help to get me going will be greatly appreciated.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Q: sort's key and cmp parameters

[Paul Rubin]

Bearophile  writes:
> The problem is that if you allow to use the cmp, lot of programmers
> will use it straight away, not even bothering to know what that
> strange 'key' argument may be useful for. And they miss the how much
> handy 'key' is.

Given how often we hear "consenting adults" as justification for any
number of gaps in Python error checking, the argument above is
singularly unpersuasive.

> I am having a very hard type (despite my implementation, explanations,
> etc) explaining to D programmers why for a flexible general-purpose
> function a key function argument is often better. They in the end have
> added something like that in the std lib, but with a weird name like
> SchwartzSort that's surely not the first standard sort they look
> for... this is bad.

"key" and the DSU (Schwartz) sorting pattern makes lots of sense in
scripting languages like Python that are primarily used on relatively
small data sets.  The best reason for writing something in a lower
level language like D is because you want to push the limits of your
availiable machine resources.  Therefore, the DSU strategy's extra
memory consumption will be what you want less of the time than it is
in Python.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Problem with subprocess module on Windows with open file in append mode

[Gabriel Genellina]

En Tue, 06 Oct 2009 11:24:23 -0300, MRAB   
escribió:



Gabriel Genellina wrote:
En Sat, 03 Oct 2009 21:53:12 -0300, Andrew Savige  
 escribió:



When I run this little test program on Linux:

import subprocess
subprocess.call(["python","-V"], stderr=open("log.tmp","a"))

the file log.tmp is appended to each time I run it.
When I run it on Windows, however, the file log.tmp gets
overwritten each time I run it.

Though I can make it append on Windows like this:

import os
import subprocess
f = open("log.tmp", "a")
f.seek(0, os.SEEK_END)
subprocess.call(["python","-V"], stderr=f)

I don't understand why that should be necessary.

Is this a Python/subprocess bug on Windows?
 No, it's an "implementation-defined behavior" of the underlying C  
library.

That's strange: whenever I've opened a file for append in Microsoft
Visual C/C++ (or Python or Delphi, for that matter) on Windows I've
assumed that the file pointer would be at the end of file, and that's
always proved to be the case!


Yes, true, I've used a small test program in C and that's indeed the case  
-- forget the "implementation-defined behavior", in any case, it works the  
same way on both platforms.


There is no problem from inside the same process. And the second example  
above shows that the file pointer *is* inherited by the child process when  
explicitely set. After experimenting a bit, looks like the file pointer is  
always at 0 right after opening the file, even for "a" or "a+" modes; it  
is moved at the end-of-file before every write.


I guess, on Windows, the file mode isn't properly recreated on the child  
process, defaulting to "w" for stdout or something like that. This may be  
a bug in the way the subprocess module handles its file arguments (I can  
see that subprocess calls functions like _get_osfhandle and  
DuplicateHandle, and maybe the original file mode is lost after all those  
operations).


--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: regex (?!..) problem

[Hans Mulder]

Stefan Behnel wrote:

Wolfgang Rohdewald wrote:

I want to match a string only if a word (C1 in this example) appears
at most once in it.


def match(s):
if s.count("C1") > 1:
return None
return s

If this doesn't fit your requirements, you may want to provide some more
details.


That will return a false value if s is the empty string.

How about:

def match(s):
return s.count("C1") <= 1

-- HansM
--
http://mail.python.org/mailman/listinfo/python-list

Problem Displaying Pics

[Victor Subervi]

Hi;
I have the following archaic code that worked just fine for another site. It
is called with the following url:

http://13gems.com/stxresort/cart/getpic1.py?id=1&x=1

#!/usr/local/bin/python
import cgitb; cgitb.enable()
import MySQLdb
import cgi
import sys,os
sys.path.append(os.getcwd())
from login import login
user, passwd, db, host = login()
form = cgi.FieldStorage()
picid = int(form['id'].value)
x = int(form['x'].value)
pics = {1:'pic1',2:'pic2',3:'pic3',4:'pic4',5:'pic5',6:'pic6'}
pic = pics[x]
print 'Content-Type: text/html'
db = MySQLdb.connect(host=host, user=user, passwd=passwd, db=db)
cursor= db.cursor()
sql = "select " + pic + " from products where id='" + str(picid) + "';"
cursor.execute(sql)
content = cursor.fetchall()[0][0].tostring()
cursor.close()
print 'Content-Type: image/jpeg'
print
print content

Now, the data is in the database:
[BLOB - 64.0 KB]

and I can get it to print out as a string. Why doesn't it show the image?
TIA,
Victor
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Problem Displaying Pics

[Rami Chowdhury]

On Tue, 06 Oct 2009 12:19:56 -0700, Victor Subervi  
 wrote:



Hi;
I have the following archaic code that worked just fine for another  
site. It

is called with the following url:

http://13gems.com/stxresort/cart/getpic1.py?id=1&x=1

#!/usr/local/bin/python
import cgitb; cgitb.enable()
import MySQLdb
import cgi
import sys,os
sys.path.append(os.getcwd())
from login import login
user, passwd, db, host = login()
form = cgi.FieldStorage()
picid = int(form['id'].value)
x = int(form['x'].value)
pics = {1:'pic1',2:'pic2',3:'pic3',4:'pic4',5:'pic5',6:'pic6'}
pic = pics[x]
print 'Content-Type: text/html'


^^^ I believe this is the problem line. It's telling the browser to expect  
HTML, not an image. Remove this, and the later Content-Type header that's  
being printed ("Content-Type: image/jpeg") should take effect.



--
Rami Chowdhury
"Never attribute to malice that which can be attributed to stupidity" --  
Hanlon's Razor

408-597-7068 (US) / 07875-841-046 (UK) / 0189-245544 (BD)
--
http://mail.python.org/mailman/listinfo/python-list

Re: Q: sort's key and cmp parameters

[Raymond Hettinger]

[bearophile]
> I love the 'key', it makes my code simpler and it's simpler to
> understand. I am not opposed to the idea of keeping cmp, that in some
> rare cases may be useful or more natural.
>
> The problem is that if you allow to use the cmp, lot of programmers
> will use it straight away, not even bothering to know what that
> strange 'key' argument may be useful for. And they miss the how much
> handy 'key' is.
...
> So a funny solution to this situation may be the following: to keep
> only 'key' in the sort/sorted for few years, so the community of
> Python programmers gets used to 'key' only, and then put 'cmp' back
> in, for the special cases ;-)

FWIW, sorting is only a small part of the picture.  The notion of
three-way cmp functions was removed throughout the language.  The
built-in cmp() function and the __cmp__ magic method and the
corresponding C slot were all excised.  None of them played nicely
with rich comparisons.  It was a case where having two-ways-to-do-it
was complicating everyone's lives.  AFAICT, very few people understood
exactly how the two interacted -- the under-the-hood C code for it
was complex, unattractive, and hard to understand.


> I am having a very hard type (despite my implementation,
> explanations, etc) explaining to D programmers why for a
> flexible general-purpose function a key function argument
> is often better.

Last week, I learned a new term, "Code Prion", that referred to
a code technique such as cmp-functions which cause the proteins
of the brain to become contagiously misfolded so that the victim
1) always prefers that technique, 2) easily infects others, and
3) is resistant to sterilization (using other techniques) ;-)

Once Kernighan and Ritchie put C's qsort() into the food supply,
we were doomed.

FWIW, I think the standard library's unittest module is also a
code prion -- there's no other explanation for its thriving
despite competition from the vastly superior syntax of py.test
or nose.


Raymond

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python: Text file insert to MySQL

[Gerhard Häring]

Schedule wrote:
> Hello,
> 
> I am currenty using MySQL 5.1 community server and trying to import the
> data of the comma delimited text file into the table using python 2.6
> scripts. I have installed Mysqldb 1.2.2.
> 
> follwoing is my script:
> [...]
>7.
>   c.execute("INSERT INTO a (first, last) VALUES (%s, %s), row")
> [...] 
> When I execute the statement I get the following error:
> 
> [...]
> _mysql_exceptions.ProgrammingError: (1064, "You have an error in your
> SQL syntax; check the manual tha
> t corresponds to your MySQL server version for the right syntax to use
> near '%s, %s), row' at line 1")

You misplaced the closing quote.

wrong:   c.execute("INSERT INTO a (first, last) VALUES (%s, %s), row")
correct: c.execute("INSERT INTO a (first, last) VALUES (%s, %s)", row)


-- Gerhard

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Q: sort's key and cmp parameters

[Paul Rubin]

Raymond Hettinger  writes:
> Once Kernighan and Ritchie put C's qsort() into the food supply,
> we were doomed.

It was already too late.  Knuth vol 3 came out in 1973(?) and its
sorting half is mostly about comparison sorting. 

> FWIW, I think the standard library's unittest module is also a
> code prion -- there's no other explanation for its thriving
> despite competition from the vastly superior syntax of py.test
> or nose.

unittest is sort of a clone of JUnit if I understand correctly.
py.test and nose are obscure because they're not in the stdlib.  I
have used unittest a little bit but I mostly use doctest these days.
I have the impression that py.test and/or nose are better in some ways
but I haven't bothered checking further.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Problem Displaying Pics

[Victor Subervi]

I remember going round and round on this issue before until I finally got it
right. I haven't changed the code. It worked before. I just tried your
update and it gave me the same result :( Any other ideas?
TIA,
V

On Tue, Oct 6, 2009 at 2:24 PM, Rami Chowdhury wrote:

> On Tue, 06 Oct 2009 12:19:56 -0700, Victor Subervi <
> victorsube...@gmail.com> wrote:
>
>  Hi;
>> I have the following archaic code that worked just fine for another site.
>> It
>> is called with the following url:
>>
>> http://13gems.com/stxresort/cart/getpic1.py?id=1&x=1
>>
>> #!/usr/local/bin/python
>> import cgitb; cgitb.enable()
>> import MySQLdb
>> import cgi
>> import sys,os
>> sys.path.append(os.getcwd())
>> from login import login
>> user, passwd, db, host = login()
>> form = cgi.FieldStorage()
>> picid = int(form['id'].value)
>> x = int(form['x'].value)
>> pics = {1:'pic1',2:'pic2',3:'pic3',4:'pic4',5:'pic5',6:'pic6'}
>> pic = pics[x]
>> print 'Content-Type: text/html'
>>
>
> ^^^ I believe this is the problem line. It's telling the browser to expect
> HTML, not an image. Remove this, and the later Content-Type header that's
> being printed ("Content-Type: image/jpeg") should take effect.
>
>
> --
> Rami Chowdhury
> "Never attribute to malice that which can be attributed to stupidity" --
> Hanlon's Razor
> 408-597-7068 (US) / 07875-841-046 (UK) / 0189-245544 (BD)
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Problem Displaying Pics

[Rami Chowdhury]

On Tue, 06 Oct 2009 12:37:17 -0700, Victor Subervi  
 wrote:


I remember going round and round on this issue before until I finally  
got it

right. I haven't changed the code. It worked before. I just tried your
update and it gave me the same result :( Any other ideas?
TIA,
V



I'm afraid changing the 'Content-type' header to be correct (which it's  
still not, by the way; you need to be printing '\r\n\r\n' after the  
content-type, to signal the end of the headers) is all I can think of.  
Apart from that, could it be a browser issue? When I went to the URL you  
provided in your first email, for instance, my browser (Opera 9.6 on  
Windows Vista) was expecting an image, as was Firefox 3.1 (again on  
Windows). However, neither seem to read valid data for the image, and both  
report it as 0x0.


Could you give us more information on what has changed? Have you upgraded  
the version of Python on the server? Have you changed the webserver  
serving your scripts? Have you changed the back-end database at all?




--
Rami Chowdhury
"Never attribute to malice that which can be attributed to stupidity" --  
Hanlon's Razor

408-597-7068 (US) / 07875-841-046 (UK) / 0189-245544 (BD)
--
http://mail.python.org/mailman/listinfo/python-list

Re: Q: sort's key and cmp parameters

[Bearophile]

Paul Rubin:

> bearophile:
>> I am having a very hard type (despite my implementation, explanations,
>> etc) explaining to D programmers why for a flexible general-purpose
>> function a key function argument is often better. They in the end have
>> added something like that in the std lib, but with a weird name like
>> SchwartzSort that's surely not the first standard sort they look
>> for... this is bad.

> "key" and the DSU (Schwartz) sorting pattern makes lots of sense in
> scripting languages like Python that are primarily used on relatively
> small data sets.  The best reason for writing something in a lower
> level language like D is because you want to push the limits of your
> availiable machine resources.  Therefore, the DSU strategy's extra
> memory consumption will be what you want less of the time than it is
> in Python.

I think you are quite wrong.

In D dynamic arrays are built-in, they are used almost as Python lists
(but they are single-typed, unless you create an array of variants),
among other things they have a built-in sort. Such sort is used for
small situations too. Even in a language like D *very* often what you
need is to sort a small amount of data in a very flexible way. In such
situations what you need isn't to squeeze the last byte or CPU cycle
out of the PC, but to have a handy and short syntax, a very flexible
sorting, and something that's surely not bug-prone. In such situation
having a 'key' argument is *better*. Such sort can be stable. That's
why the main sort/sorted routines in my dlibs accept an optional key
callable, and they are very handy. I can show you examples.

On the other hand once in a while in a D program you may need to sort
a very large amount of data, or you may need something faster (and
unstable, like a refined introsort based on a 2-pivot QS), or even
more special than the things allowed by the built in sort. In such
situation you can import a special sorting function from the std lib
and use it, such sort can have something equivalent to a cmp (but
lower level, it can accept a function template alias, to inline the
comparison operation).

So the idea is: in a multi-level language you very often have to sort
a limited amount of data in complex ways. Because in most programs a
quite large percentage of the code isn't performance-critical. In such
large parts of the code what you want is something very handy, safe
and flexible. So having a key function is positive. In the few spots
where you need high performance, you can import (or even implement
with your hands) and use a specialized sorting routine. D is a general
purpose language, it's not used like C in Python projects to write
performance-critical spots only.

This is also visible in other parts of the language, for example there
are built-in associative arrays. Their syntax is handy, and even if
they aren't high performance (they are sometimes slower than Python
dicts despite being O(1), but they never have a O(n^2) performance
profile, as may happen to Python dicts, so they are safer) you can use
them in a very quick way, even if you have to create a 10-items
associative array. The end result is that in D programs you can find
more hashes than in C++ code, where I think programmers use them only
where simpler solutions (like iterating on an array) aren't good
enough. (So D AAs have to be fast even in situations where you put 8
items inside them, like in Python dicts). This free usage of AAs makes
the code stronger too (because once in a while you may put 1000 items
in such AA, and it will keeps working efficiently, while a linear scan
in a list of 1000 items starts to become not efficient).

Bye,
bearophile
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: mktime, how to handle dates before 01-01-1970 ?

[Stef Mientki]

Stephen Hansen wrote:
On Mon, Oct 5, 2009 at 4:54 PM, Stef Mientki > wrote:


hello,

I want to handle datetime vars in a general way, so I use the
default time-format,
so I can use the standard cinversion procedures.


Personally, I love mx.DateTime; its the best date/time library around. 
But, Python's built in datetime isn't too bad and is similar and 
built-in if you don't want to use a third-party library.

thanks guys,
mx works a bit better  


But:

>>> birthday = mx.DateTime.Date(1960,3,3)
>>> birthday

>>> age = mx.DateTime.now() - birthday
>>> print "Age in days", age.days
18113.722499758693
>>> print "Age in years", age.days / 365
49.626636985640253

I really can't quite fathom why you'd want to use something so 
low-level as time.mktime... or just about anything in the time module :)

I didn't know anything better,
but (forgive me if I'm wrong) I find mx almost as low-level :
>>> mx.DateTime.strptime('01-01-53',"%d-%m-%y")

while all we human know ..

I agree it's better not to work with string dates, but that's the way 
how we human write things down ;-)

So I must use at least something like strptime.

cheers,
Stef

If you want to handle dates in a general way, I'd use one of the 
general-purpose date/time handling libraries. They're far more capable 
and easier to use.


HTH,

--S




--
http://mail.python.org/mailman/listinfo/python-list

Re: Problem Displaying Pics

[Victor Subervi]

The code in question is generated automatically from another script. I took
your idea of the \r\n\r\n and added triple quoting and now it prints out
this:

#!/usr/local/bin/python
import cgitb; cgitb.enable()
import MySQLdb
import cgi
import sys,os
sys.path.append(os.getcwd())
from login import login
user, passwd, db, host = login()
form = cgi.FieldStorage()
picid = int(form['id'].value)
x = int(form['x'].value)
pics = {1:'pic1',2:'pic2',3:'pic3',4:'pic4',5:'pic5',6:'pic6'}
pic = pics[x]
print '''Content-Type: text/html

'''
db = MySQLdb.connect(host=host, user=user, passwd=passwd, db=db)
cursor= db.cursor()
sql = "select " + pic + " from products where id='" + str(picid) + "';"
cursor.execute(sql)
content = cursor.fetchall()[0][0].tostring()
cursor.close()
print '''Content-Type: image/jpeg

'''
print
print content


To answer your questions, I have no idea what eNom has done to their
servers, which is where the other site was hosted (never went live), but am
in the process of building on DreamHost, who seems to be a FAR better
service than eNom, whom I actually had to report to the BBB and they never
fixed or acknowledged very obvious problems. At any rate, the above code
gets the browser to print out all the binary "garbage" that should translate
into an image (you can look:
http://13gems.com/stxresort/cart/getpic1.py?id=1&x=1 ). Any more ideas would
be helpful.
TIA,
V

On Tue, Oct 6, 2009 at 2:48 PM, Rami Chowdhury wrote:

> On Tue, 06 Oct 2009 12:37:17 -0700, Victor Subervi <
> victorsube...@gmail.com> wrote:
>
>  I remember going round and round on this issue before until I finally got
>> it
>> right. I haven't changed the code. It worked before. I just tried your
>> update and it gave me the same result :( Any other ideas?
>> TIA,
>> V
>>
>>
> I'm afraid changing the 'Content-type' header to be correct (which it's
> still not, by the way; you need to be printing '\r\n\r\n' after the
> content-type, to signal the end of the headers) is all I can think of. Apart
> from that, could it be a browser issue? When I went to the URL you provided
> in your first email, for instance, my browser (Opera 9.6 on Windows Vista)
> was expecting an image, as was Firefox 3.1 (again on Windows). However,
> neither seem to read valid data for the image, and both report it as 0x0.
>
> Could you give us more information on what has changed? Have you upgraded
> the version of Python on the server? Have you changed the webserver serving
> your scripts? Have you changed the back-end database at all?
>
>
>
>
> --
> Rami Chowdhury
> "Never attribute to malice that which can be attributed to stupidity" --
> Hanlon's Razor
> 408-597-7068 (US) / 07875-841-046 (UK) / 0189-245544 (BD)
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sftp login without password

[Joshua Kugler]

David wrote:
> transport.connect(username = username, pkey = mykey)
> 
> I get a "AuthenticationException: Authentication failed." exception.
> 
> 
> My ~/.ssh/id_rsa is correct because if, at console, I type
> 
> bags...@bagvapp:~$ sftp bags...@192.168.92.129
> Connecting to 192.168.92.129...
> sftp>
> 
> I get a successful login. How can I get an sftp login without using a
> password in python?
> I am using Ubuntu 9.04, python 2.6.2 and paramiko 1.7.5

What is the contents of 'mykey'?  Is it a file object? A string containing
the contents of ~/.ssh/id_rsa? The Paramiko docs indicate mykey should be a
subclass of PKey, probably RSAKey.

So, you'd need to do:

mykey = RSAKey.from_private_key_file('/path/to/private/key/id_rsa')
transport.connect(username = username, pkey = mykey)

Beyond that, I"m not sure.  Of course, I've never used Paramiko before.

j

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Re: unittest.TestCase and functools.partial don't seem to mix

[Joel Smith]

Gabriel Genellina wrote:
Note that you don't *have* to use partial in this case, as you're 
building the suite yourself. Just create the TestCase instances manually:


suite = unittest.TestSuite([
  TestGenericWindow('testit', 'brown'),
  TestGenericWindow('testit', 'blue'),
  TestGenericWindow('testit', 'green')
  ])
unittest.TextTestRunner().run(suite)


Perfect!  This is exactly what I needed.  For some reason, I didn't
understand that I could construct my TestCase objects directly... I
thought that I had to allow the unittest framework to construct them for
me.  This is straightforward, and does exactly what I need.
Many thanks,
Joel

--
http://mail.python.org/mailman/listinfo/python-list

Re: mktime, how to handle dates before 01-01-1970 ?

[Christian Heimes]

Ben Finney wrote:
> If you're committed to changing the epoch anyway, I would recommend
> using http://en.wikipedia.org/wiki/Astronomical_year_numbering>
> (epoch at 4004 BCE) since it is widely used to unify dates referring to
> human history.

I prefer JDN or MJD (http://en.wikipedia.org/wiki/JDN) for dates long
before or after the unix epoch. The conversion from JDN as float to a
datetime object is trivial.

Christian

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: mktime, how to handle dates before 01-01-1970 ?

[Stephen Hansen]

>
> I really can't quite fathom why you'd want to use something so low-level as
>> time.mktime... or just about anything in the time module :)
>>
> I didn't know anything better,
> but (forgive me if I'm wrong) I find mx almost as low-level :
> >>> mx.DateTime.strptime('01-01-53',"%d-%m-%y")
> 
> while all we human know ..
>
> I agree it's better not to work with string dates, but that's the way how
> we human write things down ;-)
> So I must use at least something like strptime.
>

mx.DateTime provides all the low-level tools if you must use them of course,
but it has high level tools too.

>>> mx.DateTime.Parser.DateFromString("01-01-1953")


Now if you're storing your years as 2-digits, yeah, it'll run into trouble
and you'll have to deal with issues related to that manually. It can't know
by default if you mean 01-01-53 as someone's birthdate in 1953, or some
future event in 2053.

It also parses the common "standardized" date representations (and faster if
its a known format, as mx.DateTime.Parser has to guess at the format until
it finds one that matches/works)-- ISO 8601 and the ARPA formats, for
example. The mx.DateTime.ISO and mx.DateTime.ARPA modules are for that.

Personally, while /users/ may write a date in a lazy way like "01-01-53",
I'd never store that and would avoid having them enter them directly. At the
UI level I'd validate and normalize it to a standard format before storing
it.

E.g. instead of users entering someone's birthday into a control, I'd use a
calendar control, and store it with (in wx terms) date.FormatISODate(). Or
use date.toString(ISODate) in QT terms (ish).

I might not display it in ISO format, but.. storing it in a standard format
makes processing easier. Faster to import into mx.DateTime, and then I just
display it as dt.Format("%x") so they see a date like they expect.

HTH,

--S
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: del an imported Class at EOF... why?

[Dave Angel]

Carl Banks wrote:

On Oct 6, 10:56 am, Ryan  wrote:
  

Good day all!

I've just inherited a large amount of python code. After spending some
time pour through the code, I've noticed that the original developer
(who is no longer w/ the company) constantly deletes the imported
classes at the end of the .py file. Why would one want to do such a
thing?




Sometimes an object is used only temporarily by a modules, while it's
importing.

In such cases, there are two reasons you might delete the object.  If
it uses a lot of memory you could free up the memory for other
objects.  Also you might want to clean up the module's namespace.

I del objects for these reasons sparingly, usually only when the
object uses a whole lot of memory, or the namespace is very messy.

However, I'm guessing that the person who wrote your code is just
overly paranoid.



Carl Banks

  
There are several things you might have meant by "the end of the .py 
file"   Presumably you mean in top-level code, and since you said module 
and not script, I'm presuming this is the code that runs outside of the 
if __name__ ==   logic.


An example would be very good.  I'd assume you meant something like:

//start module
from extern_module import MyClass

some definitions and classes

some common initialization code

if __name__ == "__main__":
 some testing code

del MyClass
//end module

In other words, these classes are being deleted before the module is 
made visible to other modules that imported it.  This seems to me just 
prudent management.  He/she is making sure that the importer of your 
code doesn't use your code as a longcut to get at MyClass.  He's forcing 
them to import them explicitly (from extern_module), not just get your 
copy.  Presumably these classes are not a part of your public interface, 
so they shouldn't be visible, except with a leading underscore.  
Naturally, if you try to use them directly in your own definitions and 
classes, you'll also have trouble.  So presumably these are classes that 
are used only in the initialization code, or as base classes for new 
classes defined in your code, or in default value expressions of your 
function definitions.


I tend to always use the plain   "import xmodule" form of statement, so 
just one symbol gets into my space.  And each use of a class from there 
is of the form
   
xmodule.MyClass


Clearly there you would not delete MyClass in your own code, though you 
could delete xmodule.


DaveA

--
http://mail.python.org/mailman/listinfo/python-list

Combining python and sqlite DB into a single, "executeable".

[tcumming123]

Any body got any ideas how to do the following...

I would like to be able to write an app in python that keeps it's persistent
data in a sqlite database file.

So far so good. The problem, is that I need the python app and the sqlite db
file to exist in the same disk file. This way the app to access the data and
the data are in the same file.

If there is a way to do this, I could (for example) have an address book
app, _AND_ the address book all in the same file. I could put the file on a
usb stick and move it to any other computer. Then click on the file and get
my address book, all without installing anything (even temporary files) on
the local computer. I could add new entries into my address book, copy the
"app+db" file to yet another computer... Note that this would _not_ extract
and/or install files into the local computer. Everything would always stay
within the one "app+db" file.

I suspect that one way to do it would be to have a generic boot-up script
that does some magic that will convince sqlite that the database is some
offset into the file. That way the first part of the file would remain
untouched by sqlite so the file could start with the usual, "#!/usr/bin/env
python" stuff. The actual app could be placed in a well known entry in the
db.

Of course the, "app" could not use any loadable modules that are not,
"common" (and/or keep loadable modules in the db somehow) allowing it to be
moved to most machines. BTW., "most machines" doesn't have to include Windoz
machines...

Any ideas?

tom
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Combining python and sqlite DB into a single, "executeable".

[Robert Kern]

On 2009-10-06 16:16 PM, tcumming...@gmail.com wrote:

Any body got any ideas how to do the following...

I would like to be able to write an app in python that keeps it's
persistent data in a sqlite database file.

So far so good. The problem, is that I need the python app and the
sqlite db file to exist in the same disk file. This way the app to
access the data and the data are in the same file.


Would having the app and the data file in the same directory satisfy your use 
case just as well? You could move the directory around just as well as you could 
move the file.


--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

--
http://mail.python.org/mailman/listinfo/python-list

Re: Combining python and sqlite DB into a single, "executeable".

[Che M]

On Oct 6, 5:31 pm, Robert Kern  wrote:
> On 2009-10-06 16:16 PM, tcumming...@gmail.com wrote:
>
> > Any body got any ideas how to do the following...
>
> > I would like to be able to write an app in python that keeps it's
> > persistent data in a sqlite database file.
>
> > So far so good. The problem, is that I need the python app and the
> > sqlite db file to exist in the same disk file. This way the app to
> > access the data and the data are in the same file.

An SQLite database is a single file.  But it can't be "in
the same file" as the .py file (I don't think).

What's your goal?  If you just want to distribute a single file
to your users, you can make an executable Python file and have
it create a fresh (empty) SQLite database once it runs for the
first time.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: del an imported Class at EOF... why?

[Steven D'Aprano]

On Tue, 06 Oct 2009 10:56:26 -0700, Ryan wrote:

> Good day all!
> 
> I've just inherited a large amount of python code. After spending some
> time pour through the code, I've noticed that the original developer
> (who is no longer w/ the company) constantly deletes the imported
> classes at the end of the .py file. Why would one want to do such a
> thing?


Too much lead paint in his milk as a small child? *grin*


Possibly out of a (misplaced?) sense of "keeping the namespace clean", or 
a desire to prevent people importing his module and then using the 
classes he imports from elsewhere.

Personally, I think it's just a quirk. There's nothing wrong with doing 
so, but nor is there anything right with it either. If he's worried about 
the presence of an alien class messing up his beautifully designed API, 
that's an aesthetic judgment, and can be (partially) managed by using 
__all__ (a global list of names). But in general, I would prefer to 
manage such namespace issues by saying:


import alienmodule 

class MyClass(alienmodule.AlienClass):
do_stuff()



rather than:


from alienmodule import AlienClass

class MyClass(AlienClass):
do_stuff()

del AlienClass





-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: package import dangers

[Steven D'Aprano]

On Tue, 06 Oct 2009 18:42:16 +0200, Diez B. Roggisch wrote:

> The most common problem is that a file is used as module and as
> executable at the same time.
> 
> Like this:
> 
> --- test.py ---
> 
> class Foo(object):
> pass
> 
> 
> if __name__ == "__main__":
>import test
>assert Foo is test.Foo
> 
> ---
> 
> This will fail when executed from the commandline because the module is
> known twice - once as "__main__", once as "test".


Why would a module need to import itself? Surely that's a very rare 
occurrence -- I think I've used it twice, in 12 years or so. I don't see 
why you need to disparage the idea of combining modules and scripts in 
the one file because of one subtle gotcha.




-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

bug with itertools.groupby?

[Kitlbast]

Hi there,

the code below on Python 2.5.2:

from itertools import groupby

info_list = [
{'profile': 'http://somesite.com/profile1', 'account': 61L},
{'profile': 'http://somesite.com/profile2', 'account': 64L},
{'profile': 'http://somesite.com/profile3', 'account': 61L},
]

grouped_by_account = groupby(info_list, lambda x: x['account'])
for acc, iter_info_items in grouped_by_account:
print 'grouped acc: ', acc

gives output:

grouped acc:  61
grouped acc:  64
grouped acc:  61

am I doing something wrong?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: bug with itertools.groupby?

[Diez B. Roggisch]

Kitlbast schrieb:

Hi there,

the code below on Python 2.5.2:

from itertools import groupby

info_list = [
{'profile': 'http://somesite.com/profile1', 'account': 61L},
{'profile': 'http://somesite.com/profile2', 'account': 64L},
{'profile': 'http://somesite.com/profile3', 'account': 61L},
]

grouped_by_account = groupby(info_list, lambda x: x['account'])
for acc, iter_info_items in grouped_by_account:
print 'grouped acc: ', acc

gives output:

grouped acc:  61
grouped acc:  64
grouped acc:  61

am I doing something wrong?


http://docs.python.org/library/itertools.html#itertools.groupby

"""
Generally, the iterable needs to already be sorted on the same key function.
"""

Diez
--
http://mail.python.org/mailman/listinfo/python-list

Re: bug with itertools.groupby?

[Rhodri James]

On Wed, 07 Oct 2009 00:06:43 +0100, Kitlbast   
wrote:



Hi there,

the code below on Python 2.5.2:

from itertools import groupby

info_list = [
{'profile': 'http://somesite.com/profile1', 'account': 61L},
{'profile': 'http://somesite.com/profile2', 'account': 64L},
{'profile': 'http://somesite.com/profile3', 'account': 61L},
]

grouped_by_account = groupby(info_list, lambda x: x['account'])
for acc, iter_info_items in grouped_by_account:
print 'grouped acc: ', acc

gives output:

grouped acc:  61
grouped acc:  64
grouped acc:  61

am I doing something wrong?


That depends on whether you expected groupby to sort your datastream
for you.  It doesn't.  It just collects up items in order until your
group key changes and then delivers that batch; it neither knows nor
cares that a previous group key has recurred.

If the output you want is more like:

grouped acc: 61
grouped acc: 64

then you're going to have to sort your info_list first.  That might
not be desirable, depending on just how long it is.  If you tell us
more about your specific use case, we may be able to give you more
specific advice.

--
Rhodri James *-* Wildebeest Herder to the Masses
--
http://mail.python.org/mailman/listinfo/python-list

Re: creating class objects inside methods

[Rhodri James]

On Sun, 04 Oct 2009 19:44:48 +0100, horos11  wrote:

[somehow managing to trim all other attributions: he's the innermost,
then me next]


> Thanks for the info, but a couple of points:

> 1. it wasn't meant to be production code, simply a way to teach
> python.

Speaking as someone who does teach Python, "Ew, no!"  If you start by
teaching people bad habits, every educator who comes along afterwards
will curse your name.  That includes teaching yourself.


No offense, but I disagree. By programming without regards to pre-
existing style or convention I learned far more than I otherwise would
have if I had simply mimicked someone else.

And I still think that unbridled assignment - especially assignment
that can change the operational semantics of surrounding terms, at a
distance no less - is a horrid thing.


[snip]

Thank you for providing a perfect example of my point.  How are you
getting on unlearning that assumption?

Pretty much every assignment in every language changes the operational
semantics of surrounding terms.  Most such changes are trivial, but for
an obvious example consider programs running finite state machines.
When you change the state variable, you change what the program will
do in the future, usually in fairly major ways that may not be visible
from wherever your assignment takes place.

I'm hardly an expert in the area, but that does seem to be the point
of a lot of object-oriented design.

--
Rhodri James *-* Wildebeest Herder to the Masses
--
http://mail.python.org/mailman/listinfo/python-list

Re: Q: sort's key and cmp parameters

[Steven D'Aprano]

On Tue, 06 Oct 2009 12:28:00 -0700, Raymond Hettinger wrote:

> [bearophile]
>> I love the 'key', it makes my code simpler and it's simpler to
>> understand. I am not opposed to the idea of keeping cmp, that in some
>> rare cases may be useful or more natural.
>>
>> The problem is that if you allow to use the cmp, lot of programmers
>> will use it straight away, not even bothering to know what that strange
>> 'key' argument may be useful for. And they miss the how much handy
>> 'key' is.
> ...
>> So a funny solution to this situation may be the following: to keep
>> only 'key' in the sort/sorted for few years, so the community of Python
>> programmers gets used to 'key' only, and then put 'cmp' back in, for
>> the special cases ;-)
> 
> FWIW, sorting is only a small part of the picture.  The notion of
> three-way cmp functions was removed throughout the language.  The
> built-in cmp() function and the __cmp__ magic method and the
> corresponding C slot were all excised.  None of them played nicely with
> rich comparisons.  It was a case where having two-ways-to-do-it was
> complicating everyone's lives.  AFAICT, very few people understood
> exactly how the two interacted -- the under-the-hood C code for it was
> complex, unattractive, and hard to understand.

There's no reason why a hypothetical comparison function for sort() needs 
to be a three-way comparison function. All you need to implement sorting 
is a function to implement less-than -- I believe that sort() and min() 
are already implemented in such a way as to only require the objects 
implements __lt__.

Hypothetically, sort(), min() and heapq.nsmallest() could take an 
optional less-than comparison function, and max() and heapq.nlargest() 
could use a greater-than comparison function.

Of course you can do this now with a cost of O(N) and a helper class:

class SortHelper:
def __init__(self, x):
self.x = x
def __lt__(self, other):
return self.x < other.x + 42  # or whatever...

# Decorate, Sort, Undecorate.
mylist = map(SortHelper, mylist)
mylist.sort()
mylist = [obj.x for obj in mylist]


If your list is huge, you can do the decoration in place:

for i, x in enumerate(mylist):
mylist[i] = SortHelper(x)
mylist.sort()
for i, obj in enumerate(mylist):
mylist[i] = obj.x



but since this is going to especially useful for really big lists, paying 
that extra O(N) cost twice will hurt.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: bug with itertools.groupby?

[Kitlbast]

Thanks guys!

Miss sorting when reading docs.. (

However, I just create simple "groupby":

def groupby(_list, key_func):
res = {}
for i in _list:
k = key_func(i)
if k not in res:
res[k] = [i]
else:
res[k].append(i)
return res

and it works 3 times faster then itertools.groupby for my example (for
tests I extend number of profiles)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: package import dangers

[Carl Banks]

On Oct 6, 3:56 pm, Steven D'Aprano
 wrote:
> On Tue, 06 Oct 2009 18:42:16 +0200, Diez B. Roggisch wrote:
> > The most common problem is that a file is used as module and as
> > executable at the same time.
>
> > Like this:
>
> > --- test.py ---
>
> > class Foo(object):
> >     pass
>
> > if __name__ == "__main__":
> >    import test
> >    assert Foo is test.Foo
>
> > ---
>
> > This will fail when executed from the commandline because the module is
> > known twice - once as "__main__", once as "test".
>
> Why would a module need to import itself? Surely that's a very rare
> occurrence -- I think I've used it twice, in 12 years or so. I don't see
> why you need to disparage the idea of combining modules and scripts in
> the one file because of one subtle gotcha.

I'm sorry, this can't reasonably be characterized as a "subtle
gotcha".  I totally disagree, it's not a gotcha but a major time-
killing head-scratcher, and it's too thoroughly convoluted to be
called subtle (subtle is like one tricky detail that messes up an
otherwise clean design, whereas this is like a dozen tricky details
the mess the whole thing up).

It's easily the most confusing thing commonly encountered in Python.
I've seen experts struggle to grasp the details.

Newbies and intermediate programmers should be advised never to do it,
use a file as either a script or a module, not both.  Expert
programmers who understand the issues--and lots of experts don't--can
feel free to venture into those waters warily.  I would say that's an
inferior solution than the method I advised in another thread that
uses a single script as an entry point and inputs modules.  But I'm
not going to tell an expert how to do it.

Average programmers, yes I will.  Too easy to mess up, too hard to
understand, and too little benefit, so don't do it.  File should be
either a module or script, not both.


Carl Banks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: bug with itertools.groupby?

[Raymond Hettinger]

On Oct 6, 4:06 pm, Kitlbast  wrote:
> Hi there,
>
> the code below on Python 2.5.2:
>
> from itertools import groupby
>
> info_list = [
>     {'profile': 'http://somesite.com/profile1', 'account': 61L},
>     {'profile': 'http://somesite.com/profile2', 'account': 64L},
>     {'profile': 'http://somesite.com/profile3', 'account': 61L},
> ]
>
> grouped_by_account = groupby(info_list, lambda x: x['account'])
> for acc, iter_info_items in grouped_by_account:
>     print 'grouped acc: ', acc
>
> gives output:
>
> grouped acc:  61
> grouped acc:  64
> grouped acc:  61
>
> am I doing something wrong?

Try another variant of groupby() that doesn't require the data to be
sorted:

   http://code.activestate.com/recipes/259173/


Raymond
-- 
http://mail.python.org/mailman/listinfo/python-list

ANN: ActivePython 2.6.3.7 (and PyPM) is now available

[Sridhar Ratnakumar]

I'm happy to announce that ActivePython 2.6.3.7 is now available for
download from:

http://www.activestate.com/activepython/

This is a patch-level release that updates ActivePython to core Python
2.6.3 along with the fixes for a couple of critical regressions that
instigated the work on 2.6.4.  See the release notes for full details:

http://docs.activestate.com/activepython/2.6/relnotes.html

Introducing PyPM


This release includes a new packaging tool by activestate called Python
Package Manager (PyPM). PyPM - currently in beta - is the package
management utility for ActivePython. It simplifies the task of locating,
installing, upgrading and removing Python modules. For full details,
see:

http://docs.activestate.com/activepython/2.6/pypm.html

Here's a sample command line output::

$ pypm install lxml
Get: [pypm.activestate.com] :repository-index:
Ready to perform these actions:
The following packages will be installed:
 lxml-2.2.2
Get: [pypm.activestate.com] lxml 2.2.2-1
Installing lxml-2.2.2
$ python
>>> import lxml.etree
>>>^D

$ pypm remove lxml
Ready to perform these actions:
The following packages will be removed:
 lxml-2.2.2
Removing lxml-2.2.2

$ pypm install pylons
Ready to perform these actions:
The following packages will be installed:
 pastescript-1.7.3 formencode-1.2.2 weberror-0.10.1 simplejson-2.0.9  
routes-1.11 nose-0.11.1 mako-0.2.5 past
edeploy-1.3.3 pylons-0.9.7 tempita-0.4 webtest-1.2 beaker-1.4.2  
webhelpers-0.6.4 paste-1.7.2 pygments-1.1.1

decorator-3.1.2 webob-0.9.6.1
Get: [pypm.activestate.com] formencode 1.2.2-1
Get: [pypm.activestate.com] nose 0.11.1-1
[...]
Get: [pypm.activestate.com] decorator 3.1.2-1
Get: [pypm.activestate.com] webob 0.9.6.1-1
Installing formencode-1.2.2
Installing weberror-0.10.1
[...]
Installing pygments-1.1.1
Fixing script /home/sridharr/.local/bin/pygmentize
Installing decorator-3.1.2


What is ActivePython?
-

ActivePython is ActiveState's binary distribution of Python. Builds for
Windows, Mac OS X, Linux, HP-UX and AIX are made freely available.

ActivePython includes the Python core and the many core extensions: zlib
and bzip2 for data compression, the Berkeley DB (bsddb) and SQLite
(sqlite3) database libraries, OpenSSL bindings for HTTPS support, the
Tix GUI widgets for Tkinter, ElementTree for XML processing, ctypes (on
supported platforms) for low-level library access, and others. The
Windows distribution ships with PyWin32 -- a suite of Windows tools
developed by Mark Hammond, including bindings to the Win32 API and
Windows COM. See this page for full details:

http://docs.activestate.com/activepython/2.6/whatsincluded.html

As well, ActivePython ships with a wealth of documentation for both new
and experienced Python programmers. In addition to the core Python docs,
ActivePython includes the "What's New in Python" series, "Dive into
Python", the Python FAQs & HOWTOs, and the Python Enhancement Proposals
(PEPs).

An online version of the docs can be found here:

http://docs.activestate.com/activepython/2.6/

We would welcome any and all feedback to:

activepython-feedb...@activestate.com

Please file bugs against ActivePython at:

http://bugs.activestate.com/query.cgi?set_product=ActivePython


On what platforms does ActivePython run?


ActivePython includes installers for the following platforms:

- Windows/x86
- Windows/x64 (aka "AMD64")
- Mac OS X
- Linux/x86
- Linux/x86_64 (aka "AMD64")
- Solaris/SPARC
- Solaris/x86
- HP-UX/PA-RISC
- AIX/PowerPC
- AIX/PowerPC 64-bit


Extra Bits
--

ActivePython releases also include the following:

- ActivePython26.chm: An MS compiled help collection of the full
  ActivePython documentation set. Linux users of applications such as
  xCHM might find this useful. This package is installed by default on
  Windows.

Extra bits are available from:

http://downloads.activestate.com/ActivePython/etc/

Thanks, and enjoy!

The Python Team

--
Sridhar Ratnakumar
sridharr at activestate.com



--
http://mail.python.org/mailman/listinfo/python-list

Re: Is there a way to specify a superclass at runtime?

[Rhodri James]

On Mon, 05 Oct 2009 14:49:42 +0100, Chris Colbert   
wrote:



I dont think so, because that would require logic outside of the
controller class to determine which controller to instantiate.

My whole purpose for Controller is to encapsulate this logic.

So, if the data should be simulated, then i just need to pass
-simulate True as a command line argument, and the controller takes
care of it...


I almost hate to point it out, but that command line argument is
"logic outside of the controller class to determine which controller
to instantiate."  It's also a darn sight easier to code.

--
Rhodri James *-* Wildebeest Herder to the Masses
--
http://mail.python.org/mailman/listinfo/python-list

Re: del an imported Class at EOF... why?

[alex23]

Steven D'Aprano  wrote:
> import alienmodule
>
> class MyClass(alienmodule.AlienClass):
>     do_stuff()
>
> rather than:
>
> from alienmodule import AlienClass
>
> class MyClass(AlienClass):
>     do_stuff()
>
> del AlienClass

The original developer may also have been unaware of the ability to
limit a * import through the use of __all__.

from alienmodule import AlienClass

__all__ = ['MyClass']

class MyClass(AlienClass):
do_stuff()

Of course, this is really only useful if you're doing "from module
import *" which is generally discouraged inside actual code.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: bug with itertools.groupby?

[Kitlbast]

On Oct 7, 3:04 am, Raymond Hettinger  wrote:
> On Oct 6, 4:06 pm, Kitlbast  wrote:
>
>
>
>
>
> > Hi there,
>
> > the code below on Python 2.5.2:
>
> > from itertools import groupby
>
> > info_list = [
> >     {'profile': 'http://somesite.com/profile1', 'account': 61L},
> >     {'profile': 'http://somesite.com/profile2', 'account': 64L},
> >     {'profile': 'http://somesite.com/profile3', 'account': 61L},
> > ]
>
> > grouped_by_account = groupby(info_list, lambda x: x['account'])
> > for acc, iter_info_items in grouped_by_account:
> >     print 'grouped acc: ', acc
>
> > gives output:
>
> > grouped acc:  61
> > grouped acc:  64
> > grouped acc:  61
>
> > am I doing something wrong?
>
> Try another variant of groupby() that doesn't require the data to be
> sorted:
>
>    http://code.activestate.com/recipes/259173/
>
> Raymond

I've checked few options of groupby() implementations

1. def groupby(_list, key_func):
res = {}
for i in _list:
k = key_func(i)
if k not in res:
res[k] = [i]
else:
res[k].append(i)
return res


2. def groupby(_list, key_func):
res = {}
[res.setdefault(key_func(i), []).append(i) for i in _list]
return res


second one with setdefault works little bit slower then (1), although
it use list comprehension
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: package import dangers

[Steven D'Aprano]

On Tue, 06 Oct 2009 17:01:41 -0700, Carl Banks wrote:

>> Why would a module need to import itself? Surely that's a very rare
>> occurrence -- I think I've used it twice, in 12 years or so. I don't
>> see why you need to disparage the idea of combining modules and scripts
>> in the one file because of one subtle gotcha.
> 
> I'm sorry, this can't reasonably be characterized as a "subtle gotcha". 
> I totally disagree, it's not a gotcha but a major time- killing
> head-scratcher, and it's too thoroughly convoluted to be called subtle
> (subtle is like one tricky detail that messes up an otherwise clean
> design, whereas this is like a dozen tricky details the mess the whole
> thing up).

Even if that were true, it's still rare for a module to import itself. If 
a major head-scratcher only bites you one time in a hundred combination 
module+scripts, that's hardly a reason to say don't write combos. It's a 
reason to not have scripts that import themselves, or a reason to learn 
how Python behaves in this case.

But I dispute it's a head-scratcher. You just need to think a bit about 
what's going on. (See below.)


> It's easily the most confusing thing commonly encountered in Python.

But it's not commonly encountered at all, in my opinion. I see no 
evidence for it being common.

I'll admit it might be surprising the first time you see it, but if you 
give it any thought it shouldn't be: when you run a module, you haven't 
imported it. Therefore it hasn't gone through the general import 
machinery. The import machinery needs to execute the code in a module, 
and it can't know that the module is already running. Therefore you get 
two independent executions of the code, which means the class accessible 
via the running code and the class accessible via the imported code will 
be different objects.

Fundamentally, it's no more mysterious than this:


>>> def factory():
... class K:
... pass
... return K
...
>>> factory() is factory()
False



> I've seen experts struggle to grasp the details.

Perhaps they're trying to hard and ignoring the simple things:

$ cat test.py
class Foo(object):
pass

if __name__ == "__main__":
import test
print Foo
print test.Foo

$ python test.py



All you have to do is look at the repr() of the class, and the answer is 
right there in your face.

Still too hard to grasp? Then make it really simple:

$ cat test2.py
print "hello"
if __name__ == "__main__":
import test2
$ python test2.py
hello
hello


I don't see how it could be more obvious what's going on. You run the 
script, and the print line is executed. Then the script tries to import a 
module (which just happens to be the same script running). Since the 
module hasn't gone through the import machinery yet, it gets loaded, and 
executed.

Simple and straight-forward and not difficult at all.



> Newbies and intermediate programmers should be advised never to do it,
> use a file as either a script or a module, not both.

There's nothing wrong with having modules be runnable as scripts. There 
are at least 93 modules in the std library that do it (as of version 
2.5). It's a basic Pythonic technique that is ideal for simple scripts.

Of course, once you have a script complicated enough that it needs to be 
broken up into multiple modules, you run into all sorts of complications, 
including circular imports. A major command line app might need hundreds 
of lines just dealing with the UI. It's fundamentally good advice to 
split the UI (the front end, the script) away from the backend (the 
modules) once you reach that level of complexity. Your earlier suggestion 
of having a single executable script to act as a front end for your 
multiple modules and packages is a good idea. But that's because of the 
difficulty of managing complicated applications, not because there's 
something fundamentally wrong with having an importable module also be 
runnable from the command line.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Combining python and sqlite DB into a single, "executeable".

[tcumming123]

Ya, I thought of that... However...

   - It would be nice to be able to execute it directly (i.e,. click on it).
   You can't, "execute" a directory.
   - It would be more work to send as an email attachment.
   - I thought it was a cool idea, and had hoped someone else had figured
   out how to do it.

In my case, I have a bunch of data to plot, and I'd like to be able to send
both the data and a program to view and manipulate the data. When the
recipient is done, they have the option of deleting the one file and there's
no mess. It got me thinking of lots of other things the general paradigm
would work for (i.e., my address book example).

tom

On Tue, Oct 6, 2009 at 11:31 AM, Robert Kern  wrote:

> On 2009-10-06 16:16 PM, tcumming...@gmail.com wrote:
>
>> Any body got any ideas how to do the following...
>>
>> I would like to be able to write an app in python that keeps it's
>> persistent data in a sqlite database file.
>>
>> So far so good. The problem, is that I need the python app and the
>> sqlite db file to exist in the same disk file. This way the app to
>> access the data and the data are in the same file.
>>
>
> Would having the app and the data file in the same directory satisfy your
> use case just as well? You could move the directory around just as well as
> you could move the file.
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless
> enigma
>  that is made terrible by our own mad attempt to interpret it as though it
> had
>  an underlying truth."
>  -- Umberto Eco
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Combining python and sqlite DB into a single, "executeable".

[Philip Semanchuk]

On Oct 6, 2009, at 7:28 PM, tcumming...@gmail.com wrote:


Ya, I thought of that... However...

  - It would be nice to be able to execute it directly (i.e,. click  
on it).

  You can't, "execute" a directory.
  - It would be more work to send as an email attachment.
  - I thought it was a cool idea, and had hoped someone else had  
figured

  out how to do it.

In my case, I have a bunch of data to plot, and I'd like to be able  
to send

both the data and a program to view and manipulate the data. When the
recipient is done, they have the option of deleting the one file and  
there's
no mess. It got me thinking of lots of other things the general  
paradigm

would work for (i.e., my address book example).


As someone else suggested, you could send a single .py file that  
creates a default database if it doesn't find one in the current  
directory (or ~/.your_app or a directory of your choosing). Does that  
give you most of what you want?




Cheers
Philip


On Tue, Oct 6, 2009 at 11:31 AM, Robert Kern   
wrote:



On 2009-10-06 16:16 PM, tcumming...@gmail.com wrote:


Any body got any ideas how to do the following...

I would like to be able to write an app in python that keeps it's
persistent data in a sqlite database file.

So far so good. The problem, is that I need the python app and the
sqlite db file to exist in the same disk file. This way the app to
access the data and the data are in the same file.



Would having the app and the data file in the same directory  
satisfy your
use case just as well? You could move the directory around just as  
well as

you could move the file.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma
that is made terrible by our own mad attempt to interpret it as  
though it

had
an underlying truth."
-- Umberto Eco

--
http://mail.python.org/mailman/listinfo/python-list


--
http://mail.python.org/mailman/listinfo/python-list


--
http://mail.python.org/mailman/listinfo/python-list

Re: package import dangers

[Dave Angel]

Steven D'Aprano wrote:

On Tue, 06 Oct 2009 18:42:16 +0200, Diez B. Roggisch wrote:

  

The most common problem is that a file is used as module and as
executable at the same time.

Like this:

--- test.py ---

class Foo(object):
pass


if __name__ == "__main__":
   import test
   assert Foo is test.Foo

---

This will fail when executed from the commandline because the module is
known twice - once as "__main__", once as "test".




Why would a module need to import itself? Surely that's a very rare 
occurrence -- I think I've used it twice, in 12 years or so. I don't see 
why you need to disparage the idea of combining modules and scripts in 
the one file because of one subtle gotcha.


  
I'm surprised to see you missed this.  A module doesn't generally import 
itself, but it's an easy mistake for a circular dependency to develop 
among modules.  modulea imports moduleb, which imports modulea again.   
This can cause problems in many cases, but two things make it worse.  
One is if an import isn't at the very beginning of the module, and even 
worse is when one of the modules involved is the original script.  You 
end up with two instances of the module, including separate copies of 
the global variables.  Lots of subtle bugs this way.


And there have been many threads right here, probably an average of once 
every two months, where the strange symptoms are ultimately caused by 
exactly this.


DaveA

--
http://mail.python.org/mailman/listinfo/python-list

Re: ANN: ActivePython 2.6.3.7 (and PyPM) is now available

[Robert H]

On Oct 6, 8:16 pm, "Sridhar Ratnakumar" 
wrote:
> I'm happy to announce that ActivePython 2.6.3.7 is now available for
> download from:
>
>      http://www.activestate.com/activepython/
>
> This is a patch-level release that updates ActivePython to core Python
> 2.6.3 along with the fixes for a couple of critical regressions that
> instigated the work on 2.6.4.  See the release notes for full details:
>
>      http://docs.activestate.com/activepython/2.6/relnotes.html
>
> Introducing PyPM
> 
>
> This release includes a new packaging tool by activestate called Python
> Package Manager (PyPM). PyPM - currently in beta - is the package
> management utility for ActivePython. It simplifies the task of locating,
> installing, upgrading and removing Python modules. For full details,
> see:
>
>      http://docs.activestate.com/activepython/2.6/pypm.html
>
> Here's a sample command line output::
>
>      $ pypm install lxml
>      Get: [pypm.activestate.com] :repository-index:
>      Ready to perform these actions:
>      The following packages will be installed:
>       lxml-2.2.2
>      Get: [pypm.activestate.com] lxml 2.2.2-1
>      Installing lxml-2.2.2
>      $ python
>      >>> import lxml.etree
>      >>>^D
>
>      $ pypm remove lxml
>      Ready to perform these actions:
>      The following packages will be removed:
>       lxml-2.2.2
>      Removing lxml-2.2.2
>
>      $ pypm install pylons
>      Ready to perform these actions:
>      The following packages will be installed:
>       pastescript-1.7.3 formencode-1.2.2 weberror-0.10.1 simplejson-2.0.9  
> routes-1.11 nose-0.11.1 mako-0.2.5 past
>      edeploy-1.3.3 pylons-0.9.7 tempita-0.4 webtest-1.2 beaker-1.4.2  
> webhelpers-0.6.4 paste-1.7.2 pygments-1.1.1
>      decorator-3.1.2 webob-0.9.6.1
>      Get: [pypm.activestate.com] formencode 1.2.2-1
>      Get: [pypm.activestate.com] nose 0.11.1-1
>      [...]
>      Get: [pypm.activestate.com] decorator 3.1.2-1
>      Get: [pypm.activestate.com] webob 0.9.6.1-1
>      Installing formencode-1.2.2
>      Installing weberror-0.10.1
>      [...]
>      Installing pygments-1.1.1
>      Fixing script /home/sridharr/.local/bin/pygmentize
>      Installing decorator-3.1.2
>
> What is ActivePython?
> -
>
> ActivePython is ActiveState's binary distribution of Python. Builds for
> Windows, Mac OS X, Linux, HP-UX and AIX are made freely available.
>
> ActivePython includes the Python core and the many core extensions: zlib
> and bzip2 for data compression, the Berkeley DB (bsddb) and SQLite
> (sqlite3) database libraries, OpenSSL bindings for HTTPS support, the
> Tix GUI widgets for Tkinter, ElementTree for XML processing, ctypes (on
> supported platforms) for low-level library access, and others. The
> Windows distribution ships with PyWin32 -- a suite of Windows tools
> developed by Mark Hammond, including bindings to the Win32 API and
> Windows COM. See this page for full details:
>
>      http://docs.activestate.com/activepython/2.6/whatsincluded.html
>
> As well, ActivePython ships with a wealth of documentation for both new
> and experienced Python programmers. In addition to the core Python docs,
> ActivePython includes the "What's New in Python" series, "Dive into
> Python", the Python FAQs & HOWTOs, and the Python Enhancement Proposals
> (PEPs).
>
> An online version of the docs can be found here:
>
>      http://docs.activestate.com/activepython/2.6/
>
> We would welcome any and all feedback to:
>
>      activepython-feedb...@activestate.com
>
> Please file bugs against ActivePython at:
>
>      http://bugs.activestate.com/query.cgi?set_product=ActivePython
>
> On what platforms does ActivePython run?
> 
>
> ActivePython includes installers for the following platforms:
>
> - Windows/x86
> - Windows/x64 (aka "AMD64")
> - Mac OS X
> - Linux/x86
> - Linux/x86_64 (aka "AMD64")
> - Solaris/SPARC
> - Solaris/x86
> - HP-UX/PA-RISC
> - AIX/PowerPC
> - AIX/PowerPC 64-bit
>
> Extra Bits
> --
>
> ActivePython releases also include the following:
>
> - ActivePython26.chm: An MS compiled help collection of the full
>    ActivePython documentation set. Linux users of applications such as
>    xCHM might find this useful. This package is installed by default on
>    Windows.
>
> Extra bits are available from:
>
>      http://downloads.activestate.com/ActivePython/etc/
>
> Thanks, and enjoy!
>
> The Python Team
>
> --
> Sridhar Ratnakumar
> sridharr at activestate.com

Very nice, I will try it out since I am in the process of learning
Python.

Bob
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: bug with itertools.groupby?

[Dave Angel]

Kitlbast wrote:

On Oct 7, 3:04 am, Raymond Hettinger  wrote:
  

On Oct 6, 4:06 pm, Kitlbast  wrote:







Hi there,
  
the code below on Python 2.5.2:
  
from itertools import groupby
  
info_list =

{'profile': 'http://somesite.com/profile1', 'account': 61L},
{'profile': 'http://somesite.com/profile2', 'account': 64L},
{'profile': 'http://somesite.com/profile3', 'account': 61L},
]
  
grouped_by_account =roupby(info_list, lambda x: x['account'])

for acc, iter_info_items in grouped_by_account:
print 'grouped acc: ', acc
  
gives output:
  
grouped acc:  61

grouped acc:  64
grouped acc:  61
  
am I doing something wrong?
  

Try another variant of groupby() that doesn't require the data to be
sorted:

   http://code.activestate.com/recipes/259173/

Raymond



I've checked few options of groupby() implementations

1. def groupby(_list, key_func):
res =}
for i in _list:
k =ey_func(i)
if k not in res:
res[k] =i]
else:
res[k].append(i)
return res


2. def groupby(_list, key_func):
res =}
[res.setdefault(key_func(i), []).append(i) for i in _list]
return res


second one with setdefault works little bit slower then (1), although
it use list comprehension

  

Or option 3:   use defaultdict


--
http://mail.python.org/mailman/listinfo/python-list

Multiprocessing.Queue deadlock

[Felix]

Hello,

I keep running into a deadlock in a fairly simple parallel script
using Multiprocessing.Queue for sending tasks and receiving results.
>From the documentation I cannot figure out what is happening and none
of the examples seem to cover quite what I am doing. The main code is

results = mp.Queue()
tasks = mp.JoinableQueue()
tasks.put( (0,0) )
procs = [ mp.Process(target=work, args=(tasks, results)) for i in range
(nprocs)]
for p in procs:
p.daemon = True
p.start()

tasks.join()
for i in range(nprocs): tasks.put('STOP')
for p in procs: p.join()
res=[]
while 1:
try:
res.append(res.get(False))
except Empty: break


The function 'work' both consumes tasks adding the results to the
output queue and adds new tasks to the input queue based on its
result.

def work(tasks, results):
for task in iter(tasks.get, 'STOP'):
res = calc(*task)
if res:
results.put(res)
tasks.put((task[0], res[1]))
tasks.put((res[0],task[1]))
   queue.task_done()

This program will hang while the main process joins the workers (after
all results are computed, i.e. after tasks.join() ). The workers have
finished function 'work', but have not terminated yet.

Calling results.cancel_join_thread as a last line in 'work' prevents
the deadlocks, as does terminating the workers directly. However I am
not sure why that would be needed and if it might not make me loose
results.

It seems to be the workers cannot finish pusing buffered results into
the output queue when calling 'results.join_thread' while terminating,
but why is that? I tried calling 'results.close()' before joining the
workers in the main process, but it does not make a difference.

Is there something I am understanding wrong about the interface? Is
there a much better way to do what I am trying to do above?

Thanks
  Felix
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: package import dangers

[Steven D'Aprano]

On Tue, 06 Oct 2009 21:44:35 -0400, Dave Angel wrote:

> I'm surprised to see you missed this.  A module doesn't generally import
> itself, but it's an easy mistake for a circular dependency to develop
> among modules.

Circular imports are always a difficulty. That has nothing to do with 
making modules executable as scripts.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Need feedback on subprocess-using function

[Nobody]

On Mon, 05 Oct 2009 02:29:38 -0700, ryles wrote:

>> If you want a "double-ended" slave process, you need to use polling or
>> non-blocking I/O or asynchronous I/O or multiple threads. I'm not aware of
>> any single solution which works on all platforms.
>>
>> The easiest way around this problem is to redirect stderr to a temporary
>> file and read in the file's contents in the close() method.
> 
> There is also Popen.communicate():

That doesn't help if you want to write data incrementally.

You could always lift the code from Popen._communicate(), which uses
threads for Windows and select() for POSIX.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Skeletal animation

[Nobody]

On Mon, 05 Oct 2009 15:25:17 -0700, Carl Banks wrote:

> There are two ways to do what you want.  The first way is to represent
> bones as OpenGL transformations.  Basically a joint deformation you'd
> represent as a translation + rotation.  To rotate a bone you'd simply
> update the rotation of that joint.  Then, when the proper
> transformation is in place, just draw the mesh attached to the bone.
> This can be done with simply with Python an OpenGL.  It's conceptually
> simple but the details can be nasty.  Also it doesn't look very good.

The main problem here is you have to have a separate mesh for each bone.
You can't make a bone affect part of a mesh; or rather, you can't make a
bone affect part of a polygon, i.e. you can't have one vertex attached to
one bone and another vertex of the same polygon attached to a different
bone. This limits it to rigid bodies (machines, robots, etc); you can't
use it for anything with a flexible skin.

> The second way is skinning.  You have a big mesh and each vertex on
> the mesh in influenced by any nearby bones.  This is more difficult
> because a calculation is required for each vertex, and you can't take
> advantage of OpenGL calcualtions(**), but it looks much better.  Also,
> there is no way to do skinning calculations fast enough in Python.
> You might be able to manage it with numpy.

A numpy solution for one-bone-per-vertex (i.e. no blending) is quite
straightforward:

bone_xforms = np.empty((n_bones,3,3), dtype=float)
base_vertices = np.empty((n_verts,3), dtype=float)
vertex_bones = np.empty((n_verts,), dtype=int)
...
xform_vertices = add.reduce(bone_xforms[vertex_bones] * b[:,newaxis,:], 
axis=2)

[This can be extended to blending a fixed number of bones per vertex,
but is inefficient if the number of bones varies.]

This will be much quicker than doing the arithmetic in Python, but may
still be noticeably slower than a typical C/C++ solution due to copying
the matrices (i.e. bone_xforms[vertex_bones] will duplicate each 3*3 array
to create a n_verts*3*3 array in memory).

OTOH, there might be ways to improve upon this; you could try asking
on the numpy mailing list.

-- 
http://mail.python.org/mailman/listinfo/python-list

ANN: M2Crypto 0.20.2

[Heikki Toivonen]

M2Crypto is the most complete Python wrapper for OpenSSL featuring RSA,
DSA, DH, HMACs, message digests, symmetric ciphers (including AES); SSL
functionality to implement clients and servers; HTTPS extensions to
Python's httplib, urllib, and xmlrpclib; unforgeable HMAC'ing
AuthCookies for web session management; FTP/TLS client and server;
S/MIME; ZServerSSL: A HTTPS server for Zope and ZSmime: An S/MIME
messenger for Zope. Smartcards supported with the Engine interface.

This is the 0.20.2 release. Download links and bug filing instructions
on the homepage at:

  http://chandlerproject.org/Projects/MeTooCrypto.

Changelog:
- (Re)Enable configuration and use with OpenSSL 0.9.7g and older by
disabling RSA PSS methods when using such old OpenSSL that don't support
it, thanks to Stef Walter

NOTE: If you are using OpenSSL that is newer than 0.9.7g there is no
need to update.

-- 
  Heikki Toivonen - http://heikkitoivonen.net
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Internationalized To:'s and Cc:'s

[Nobody]

On Tue, 06 Oct 2009 06:33:37 -0700, Maxim Kuleshov wrote:

> How should I correctly construct internationalized base64'ed MIME
> header?
> The problem is that 'real name' _should_ be encoded, but the email
> address - should not.
> For example, ?utf-8?bla-bla=?=  should be the correct
> format, and
> ?utf-8?bla-bla-bla-bla=?= - incorrect.

   from email.header import Header, decode_header
   h = Header()
   h.append("Real Name", 'utf-8')
   h.append(" ")
   h.encode()
 '=?utf-8?q?Real_Name?=  '
   decode_header(_)
 [('Real Name', 'utf-8'), ('', None)]

-- 
http://mail.python.org/mailman/listinfo/python-list

ANN: python-gnupg v0.2.2 released

[Vinay Sajip]

A new version of the Python module which wraps GnuPG has been
released.

What Changed?
=
This is a minor bug-fix release. See the project website (
http://code.google.com/p/python-gnupg/ ) for more information.

The changes were to the name of the distribution archive (now prefixed
with "python-") and support was added for GnuPG v2.0.x (tested on
Linux only).

The current version passes all tests on Windows (Python 2.4, 2.5, 2.6,
3.1, Jython 2.5.1) and Ubuntu (Python 2.4, 2.5, 2.6, 3.0, Jython
2.5.1).

What Does It Do?

The gnupg module allows Python programs to make use of the
functionality provided by the Gnu Privacy Guard (abbreviated GPG or
GnuPG). Using this module, Python programs can encrypt and decrypt
data, digitally sign documents and verify digital signatures, manage
(generate, list and delete) encryption keys, using proven Public Key
Infrastructure (PKI) encryption technology based on OpenPGP.

This module is expected to be used with Python versions >= 2.4, as it
makes use of the subprocess module which appeared in that version of
Python. This module is a newer version derived from earlier work by
Andrew Kuchling, Richard Jones and Steve Traugott.

A test suite using unittest is included with the source distribution.

Simple usage:

>>> import gnupg
>>> gpg = gnupg.GPG(gnupghome='/path/to/keyring/directory')
>>> gpg.list_keys()
[{
  ...
  'fingerprint': 'F819EE7705497D73E3CCEE65197D5DAC68F1AAB2',
  'keyid': '197D5DAC68F1AAB2',
  'length': '1024',
  'type': 'pub',
  'uids': ['', 'Gary Gross (A test user) ']},
 {
  ...
  'fingerprint': '37F24DD4B918CC264D4F31D60C5FEFA7A921FC4A',
  'keyid': '0C5FEFA7A921FC4A',
  'length': '1024',
  ...
  'uids': ['', 'Danny Davis (A test user) ']}]
>>> encrypted = gpg.encrypt("Hello, world!", ['0C5FEFA7A921FC4A'])
>>> str(encrypted)
'-BEGIN PGP MESSAGE-\nVersion: GnuPG v1.4.9 (GNU/Linux)\n
\nhQIOA/6NHMDTXUwcEAf
...
-END PGP MESSAGE-\n'
>>> decrypted = gpg.decrypt(str(encrypted), passphrase='secret')
>>> str(decrypted)
'Hello, world!'
>>> signed = gpg.sign("Goodbye, world!", passphrase='secret')
>>> verified = verified = gpg.verify(str(signed))
>>> print "Verified" if verified else "Not verified"
'Verified'

For more information, visit http://code.google.com/p/python-gnupg/ -
as always, your feedback is most welcome (especially bug reports,
patches and suggestions for improvement). Enjoy!

Cheers

Vinay Sajip
Red Dove Consultants Ltd.
-- 
http://mail.python.org/mailman/listinfo/python-list

Graphical nodes

[Dylan Palmboom]

Hi everyone
 
Please could someone tell me of any libraries that you could use to make use
of graphical nodes in python.
Each node would hold data and properties.
I am trying to achieve a similar effect, as in the application spoken about
below.
If you have seen an application called Nuke, by the foundry, you will see
what I mean. It uses graphical nodes
such as rectangles etc (similar to UML) to link operations on images
together with arrows etc.
I know that Nuke is made using the Qt toolkit, so maybe someone could tell
me what classes were probably used
to make this node based interface. Maybe the QGraphicsItem and
QGraphicsScene classes? It looks cool
so if anyone has any ideas on how to do something like this, it would be
great.
 
Thanks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: package import dangers

[Dave Angel]

Steven D'Aprano wrote:

On Tue, 06 Oct 2009 21:44:35 -0400, Dave Angel wrote:

  

I'm surprised to see you missed this.  A module doesn't generally import
itself, but it's an easy mistake for a circular dependency to develop
among modules.



Circular imports are always a difficulty. That has nothing to do with 
making modules executable as scripts.


  
I was mainly making the point that while self-importing would be rare, 
circular imports probably crop up fairly often.  Circular imports are 
(nearly always) a design flaw.  But until you made me think about it, I 
would have said that they are safe in CPython as long as all imports are 
at the top of the file.  And as long as the script isn't part of the 
dependency loop.  Thanks for the word "always" above;   in trying to 
refute it, I thought hard enough to realize you're right.  And what's 
better, realized it before hitting "SEND."


I would still say that the bugs caused in circular imports are 
relatively easy to spot, while the ones caused by importing the script 
can be quite painful to discover, if you aren't practiced at looking for 
them.


And my practice is to keep the two separate, only using a module as a 
script when testing that module.  The only time I've run into the 
problem of the dual loading of the script was in a simple program I 
copy-pasted from the wxPython demo code.  That demo had a common module 
(shell) which each individual demo imported.  But if you ran that common 
module as a script, it interactively let you choose which demo to import.


DaveA


--
http://mail.python.org/mailman/listinfo/python-list

Re: organizing your scripts, with plenty of re-use

[catafest]

I think the basically you need to write one python script name it
__init__.py
If you want this script may include functions to reading yours scripts
from folder.
Put this script in each folder and you use Margie solution .
This allow you to use import from each folder .
-- 
http://mail.python.org/mailman/listinfo/python-list