Re: regex (?!..) problem

[Carl Banks]

On Oct 4, 11:17 pm, Wolfgang Rohdewald  wrote:
> On Monday 05 October 2009, Carl Banks wrote:
>
> > What you're not realizing is that if a regexp search comes to a
> >  dead end, it won't simply return "no match".  Instead it'll throw
> >  away part of the match, and backtrack to a previously-matched
> >  variable-length subexpression, such as ".*?", and try again with a
> >  different length.
>
> well, that explains it. This is contrary to what the documentation
> says, though. Should I fill a bug 
> report?http://docs.python.org/library/re.html

If you're referring to the section where it explains greedy
qualifiers, it is not wrong per se.  re.match does exactly what the
documentation says: it matches as few characters as possible to the
non-greedy pattern.

However, since it's easy to misconstrue that if you don't know about
regexp backtracking, perhaps a little mention of backtracking is is
warranted.  IMO it's not a documentation bug, so if you want to file a
bug report I'd recommend filing as a wishlist item.

I will mention that my followup contained an error (which you didn't
quote).  I said greedy versus non-greedy doesn't affect the substring
matched.  That was wrong, it does affect the substring matched; what
it doesn't affect is whether there is a match found.


> Now back to my original problem: Would you have any idea how
> to solve it?
>
> count() is no solution in my case, I need re.search to either
> return None or a match.

Why do you have to use a regexp at all?

In Python we recommend using string operations and methods whenever
reasonable, and avoiding regexps unless you specifically need their
extra power.  String operations can easily do the examples you posted,
so I see no reason to use regexps.

Depending on what you want to do with the result, one of the following
functions should be close to what you need.  (I am using "word" to
refer to the string being matched against, "token" to be the thing you
don't want to appear more than once.)


def token_appears_once(word,token):
return word.count(token) == 1

def parts(word,token):
head,sep,tail = word.partition("C1")
if sep == "" or "C1" in tail:
return None
return (head,sep,tail)


If you really need a match object, you should do a search, and then
call the .count method on the matched substring to see if there is
more than one occurrence, like this:

def match_only_if_token_appears_once(pattern,wotd,token):
m = re.search(pattern,word)
if m.group(0).count("C1") != 1:
m = None
return m


Carl Banks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: regex (?!..) problem

[Wolfgang Rohdewald]

On Monday 05 October 2009, Stefan Behnel wrote:
> Wolfgang Rohdewald wrote:
> > I want to match a string only if a word (C1 in this example)
> > appears at most once in it.
> 
> def match(s):
> if s.count("C1") > 1:
> return None
> return s
> 
> If this doesn't fit your requirements, you may want to provide some
>  more details.

Well - the details are simple and already given: I need re.search
to either return None or a match. But I will try to state it
differently:

I have a string representing the results for a player of a board
game (Mah Jongg - not the solitaire but the real one, played by
4 players), and I have a list of scoring rules. Those rules
can be modified by the user, he can also add new rules. Mah Jongg
is played with very different rulesets worldwide.

The rules are written as regular expressions. Since what they
do varies greatly I do not want do treat some of them in a special
way. That would theoretically be possible but it would really
complificate things.

For each rule I simply need to check whether it applies or not.
I do that by calling re.search(rule, gamestring) and by checking
the result against None.

Here you can look at all rules I currently have.
http://websvn.kde.org/trunk/playground/games/kmj/src/predefined.py?view=markup
The rule I want to rewrite is called "Robbing the Kong". Of
course it is more complicated than my example with C1.

Here you can find the documentation for the gamestring:
http://websvn.kde.org/trunk/playground/games/doc/kmj/index.docbook?revision=1030476&view=markup
(get HTML files with "meinproc index.docbook") 

-- 
Wolfgang
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is pythonic version of scanf() or sscanf() planned?

[TerryP]

In the last 4 years, I have never missed functions like .*scanf() or
atoi().

It's probably a greeaaat thing that Python provides nether as built
ins (per se).
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Haskell's new logo, and the idiocy of tech geekers

[Adam Michalik]

Kenneth Tilton  writes:

> Hunh? He has done a nice job of collecting different logos and putting
> them all in one place where one can see them all just by
> scrolling. ie, it's a cool web page with added value available nowhere
> else.

But it's Xah Lee. He is evil and so are all his works. His website is
full of filth. I heard reading it also causes brain damage. Don't go
there. Remember, we warned you.

-- 
Adam Michalik
 vel Dodek Dodecki

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Enormous Input and Output Test

[n00m]

> but unlike us, he's routinely under 11s.  Crazy.

No wonder!
50-80%% of users from the 1st page in ranklist are
super-extra-brilliant (young students) programmers.
They are winners of numerous competitions, national
and international olympiads on informatics, etc.
Some of them are/were even true wunderkinders.
E.g. Tomek Czajka from Poland (now he lives and works,
in some university, in America)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Enormous Input and Output Test

[n00m]

Duncan Booth,

alas... still TLE:

2800839
2009-10-04 13:03:59
Q
Enormous Input and Output Test
time limit exceeded
-
88M
PYTH
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trouble sending / receiving compressed data (using zlib) as HTTP POST to server (in django)

[subeen]

Thanks for your response.
> How did you post the data? If you post binary data you should indicate
> this with a proper mime type, like application/octet-stream. Otherwise
> it might be interpreted as text which it isn't.
> --
I am trying to send the data using following code:
...
opener = urllib2.build_opener()

opener.addheaders = [

 ('User-Agent', 'python'),
 ('Content-Type', 'application/octet-stream'),
]
data = zlib.compress(data)
params = urllib.urlencode({'uid': uid, 'reqid': rid, 'data':
data})
usock = opener.open(url, params)
resp = usock.read()

usock.close()
...

regards,
Subeen.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Skeletal animation

[Hendrik van Rooyen]

On Monday, 5 October 2009 00:05:21 Manowar wrote:
> I am new to pyton and have asked this question several times the
> answer is always not sure.
> Here is my question sekeltal animation ( bone animation) is it
> possible with python? What i want to develop is an aquarium in
> realtime, skeletal animation, all the movements will be from
> programming., no keyframes  let me know if it is possible with python

Sure  - you can draw anything on a Tkinter canvas, and move it around.

Also look at pygame, and PIL

- Hendrik


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: regex (?!..) problem

[MRAB]

Wolfgang Rohdewald wrote:

Hi,

I want to match a string only if a word (C1 in this example) appears
at most once in it. This is what I tried:


re.match(r'(.*?C1)((?!.*C1))','C1b1b1b1 b3b3b3b3 C1C2C3').groups()

('C1b1b1b1 b3b3b3b3 C1', '')

re.match(r'(.*?C1)','C1b1b1b1 b3b3b3b3 C1C2C3').groups()

('C1',)

but this should not have matched. Why is the .*? behaving greedy
if followed by (?!.*C1)? I would have expected that re first 
evaluates (.*?C1) before proceeding at all.


I also tried:

re.search(r'(.*?C1(?!.*C1))','C1b1b1b1 b3b3b3b3 

C1C2C3C4').groups()
('C1b1b1b1 b3b3b3b3 C1',)

with the same problem.

How could this be done?


You're currently looking for one that's not followed by another; the
solution is to check first whether there are two:

>>> re.match(r'(?!.*?C1.*?C1)(.*?C1)','C1b1b1b1 b3b3b3b3 C1C2C3').groups()

Traceback (most recent call last):
  File "", line 1, in 
re.match(r'(?!.*?C1.*?C1)(.*?C1)','C1b1b1b1 b3b3b3b3 C1C2C3').groups()
AttributeError: 'NoneType' object has no attribute 'groups'
--
http://mail.python.org/mailman/listinfo/python-list

some site login problem help plz..

[james27]

hello..
im new to python.
i have some problem with mechanize.
before i was used mechanize with no problem.
but i couldn't success login with some site.
for several days i was looked for solution but failed.
my problem is , login is no probelm but can't retrieve html source code from
opened site.
actually only can read some small html code, such like below.



location.replace("http://www.naver.com";);



i want to retrive full html source code..but i can't . i was try with twill
and mechanize and urllib and so on.
i have no idea.. anyone can help me?

here is full source code.
and Thanks in advance!

# -*- coding: cp949 -*-
import sys,os
import mechanize, urllib
import cookielib
import re
import BeautifulSoup

params = urllib.urlencode({'url':'http://www.naver.com',
   'svctype':'',
   'viewtype':'',
   'postDataKey':'',
  
'encpw':'3a793b174d976d8a614467eb0466898230f39ca68a8ce2e9c866f9c303e7c96a17c0e9bfd02b958d88712f5799abc5d26d5b6e2dfa090e10e236f2afafb723d42d2a2aba6cc3f268e214a169086af782c22d0c440c876a242a4411860dd938c4051acce987',
   'encnm':'13774',
   'saveID':'0',
   'enctp':'1',
   'smart_level':'1',
   'id':'lbu142vj',
   'pw':'wbelryl',
   'x':'24',
   'y':'4'
   })
rq = mechanize.Request("http://nid.naver.com/nidlogin.login";, params)
rs = mechanize.urlopen(rq)
data = rs.read()   
print data 
rq = mechanize.Request("http://mail2.naver.com";)
rs = mechanize.urlopen(rq)
data = rs.read()
print data 
-- 
View this message in context: 
http://www.nabble.com/some-site-login-problem-help-plz..-tp25746497p25746497.html
Sent from the Python - python-list mailing list archive at Nabble.com.

-- 
http://mail.python.org/mailman/listinfo/python-list

Walking an object/reference graph

[Austin Bingham]

I'm looking for the proper way to "walk" a graph of python objects. My
specific task is to find all objects of a given type that are referred
to (transitively) by some starting object. My approach has been to
loop through the object's attributes, examining each, and then
recursing on them.

This approach has immediate problems in that I immediately hit
infinite recursion due to method-wrappers; I'm not sure *exactly*
what's going on, but it appears that method-wrappers refer to
method-wrappers ad infinitum. If I add code to avoid this issue,
others soon crop up, so before I wrote too much more code, I thought
I'd check with experts to see if there's a better way.

So, if I've explained what I'm looking for well enough, does anyone
know of a proper way to walk a graph of object references? I need to
support cycles in the graph, so keeping track of visited object IDs or
something will have to be employed. Also, I need to be able to follow
references stored in lists, dictionaries, etc...basically any way to
programatically get from one object to another should be followed by
the algorithm. I feel like I'm missing something simple, but perhaps
this is just a tricky problem for python. Any ideas or insight would
be great. Thanks!

Austin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: regex (?!..) problem

[Wolfgang Rohdewald]

On Monday 05 October 2009, Carl Banks wrote:
> Why do you have to use a regexp at all?

not one but many with arbitrary content.

please read my answer to Stefan. Take a look
at the regexes I am using:
http://websvn.kde.org/trunk/playground/games/kmj/src/predefined.py?view=markup

moreover they are saved in a data base. I would
not want to do that with python code.

Actually I already did that - I defined Python classes
which made it possible to write a rule as (example)

self.mjRules.append(Rule('Fourfold Plenty', 'PKong()*4 + Pair()', 
limits=1))

see 
http://websvn.kde.org/trunk/playground/games/kmj/src/rulesets.py?view=markup&pathrev=998607

this was then executed with eval. But eval seems unsafe
and some rules were simply not writable that way without
specialized hard wired python code. That is not what I
envision. Also regex turned out to have about the same
speed IIRC

-- 
Wolfgang
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: unicode issue

[gentlestone]

Thx for useful advices. They seems to be very clever.

Thx to dajngo users comunity, I've got a nice solution, how to avoid
unicode problems in doctests:

"""
>>> Osoba(meno = "Ľudmila".decode('utf-8'), priezvisko =
"Šafářová".decode('utf-8'))

"""

It is - do not use unicode string at all. Instead of it create a
unicode object by explicitly decoding a bytestring using the proper
codec.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: regex (?!..) problem

[Wolfgang Rohdewald]

On Monday 05 October 2009, MRAB wrote:
> You're currently looking for one that's not followed by another;
>  the solution is to check first whether there are two:
> 
>  >>> re.match(r'(?!.*?C1.*?C1)(.*?C1)','C1b1b1b1 b3b3b3b3
>  C1C2C3').groups()
> 
> Traceback (most recent call last):
>File "", line 1, in 
>  re.match(r'(?!.*?C1.*?C1)(.*?C1)','C1b1b1b1 b3b3b3b3
>  C1C2C3').groups() AttributeError: 'NoneType' object has no
>  attribute 'groups'

that is a nice solution!

now to make it more similar to my real world case
where C1 is actually part of the string:

same regex but using a group for C1 does not work - why?

>>> re.match(r'(?!.*?(C1).*?\1)(.*?\1)','C1b1b1b1 b3b3b3b3 C2C2C3').groups()
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: 'NoneType' object has no attribute 'groups'

>>> re.match(r'(?!.*?(?PC1).*?(?P=tile))(.*?(?P=tile))','C1B1B1B1 
>>> b3b3b3b3 C2C2C3').groups()
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: 'NoneType' object has no attribute 'groups'


-- 
Wolfgang
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: question about the GC implementation

[Francis Moreau]

On Oct 3, 11:58 pm, ryles  wrote:
>
>     Only objects with GC_TENTATIVELY_UNREACHABLE still set are
> candidates for collection.  If it's decided not to collect such an object
> (e.g., it has a __del__ method), its gc_refs is restored to GC_REACHABLE
> again.

Ok so it happens _only_ when the object has a __del__ method, right ?

Thanks
-- 
http://mail.python.org/mailman/listinfo/python-list

Path for compile kinterbasdb with mingw32 and python 2.6

[Alexandr N Zamaraev]

I write path for compile kinterbasdb with mingw32 and python 2.6 for rev
1038
See
http://sourceforge.net/tracker/?func=detail&aid=2872794&group_id=9913&atid=309913


--
http://mail.python.org/mailman/listinfo/python-list

Re: some site login problem help plz..

[Diez B. Roggisch]

james27 wrote:

> 
> hello..
> im new to python.
> i have some problem with mechanize.
> before i was used mechanize with no problem.
> but i couldn't success login with some site.
> for several days i was looked for solution but failed.
> my problem is , login is no probelm but can't retrieve html source code
> from opened site.
> actually only can read some small html code, such like below.
> 
> 
> 
> location.replace("http://www.naver.com";);
> 
> 
> 
> i want to retrive full html source code..but i can't . i was try with
> twill and mechanize and urllib and so on.
> i have no idea.. anyone can help me?

Your problem is that the site uses JavaScript to replace itself. Mechanize
can't do anything about that. You might have more luck with scripting a
browser. No idea if there are any special packages available for that
though.

Diez
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: regex (?!..) problem

[MRAB]

Wolfgang Rohdewald wrote:

On Monday 05 October 2009, MRAB wrote:

You're currently looking for one that's not followed by another;
 the solution is to check first whether there are two:

 >>> re.match(r'(?!.*?C1.*?C1)(.*?C1)','C1b1b1b1 b3b3b3b3
 C1C2C3').groups()

Traceback (most recent call last):
   File "", line 1, in 
 re.match(r'(?!.*?C1.*?C1)(.*?C1)','C1b1b1b1 b3b3b3b3
 C1C2C3').groups() AttributeError: 'NoneType' object has no
 attribute 'groups'


that is a nice solution!

now to make it more similar to my real world case
where C1 is actually part of the string:

same regex but using a group for C1 does not work - why?


re.match(r'(?!.*?(C1).*?\1)(.*?\1)','C1b1b1b1 b3b3b3b3 C2C2C3').groups()

Traceback (most recent call last):
  File "", line 1, in 
AttributeError: 'NoneType' object has no attribute 'groups'


re.match(r'(?!.*?(?PC1).*?(?P=tile))(.*?(?P=tile))','C1B1B1B1 b3b3b3b3 
C2C2C3').groups()

Traceback (most recent call last):
  File "", line 1, in 
AttributeError: 'NoneType' object has no attribute 'groups'


"(?!.*?(C1).*?\1)" will succeed only if ".*?(C1).*?\1" has failed, in
which case the group (group 1) will be undefined (no capture).
--
http://mail.python.org/mailman/listinfo/python-list

Re: Walking an object/reference graph

[MRAB]

Austin Bingham wrote:

I'm looking for the proper way to "walk" a graph of python objects. My
specific task is to find all objects of a given type that are referred
to (transitively) by some starting object. My approach has been to
loop through the object's attributes, examining each, and then
recursing on them.

This approach has immediate problems in that I immediately hit
infinite recursion due to method-wrappers; I'm not sure *exactly*
what's going on, but it appears that method-wrappers refer to
method-wrappers ad infinitum. If I add code to avoid this issue,
others soon crop up, so before I wrote too much more code, I thought
I'd check with experts to see if there's a better way.

So, if I've explained what I'm looking for well enough, does anyone
know of a proper way to walk a graph of object references? I need to
support cycles in the graph, so keeping track of visited object IDs or
something will have to be employed. Also, I need to be able to follow
references stored in lists, dictionaries, etc...basically any way to
programatically get from one object to another should be followed by
the algorithm. I feel like I'm missing something simple, but perhaps
this is just a tricky problem for python. Any ideas or insight would
be great. Thanks!


Keep a set of the objects you've visited (that will have to be a set of
the object ids because not all objects are hashable) and a
list/stack/queue of the objects you haven't checked yet, and then
iterate until the list/etc is empty.
--
http://mail.python.org/mailman/listinfo/python-list

Re: [Image-SIG] Some issue with easy_install and PIL/Imaging

[Fredrik Lundh]

The problem is that too many people arguing for eggs do this by
sending nastygrams, which doesn't really provide much motivation for
doing anything about it (I don't do asshole-driven development).  The
public review PIL got a couple a minutes ago matches some of the
private mail I've gotten:

   no egg - worst seen ever, remove it from pypi or provide an egg
(jensens, 2009-10-05, 0 points)



On Wed, Sep 30, 2009 at 6:24 PM, Chris Withers  wrote:
> Fredrik Lundh wrote:
>>
>> On Fri, Sep 11, 2009 at 3:49 PM, Chris Withers 
>> wrote:
>>>
>>> Klein Stéphane wrote:

 Resume :
 1. first question : why PIL package in "pypi" don't work ?
>>>
>>> Because Fred Lundh have his package distributions unfortunate names that
>>> setuptools doesn't like...
>>
>> It used to support this, but no longer does.  To me, that says more
>> about the state of setuptools than it does about the state of PIL,
>> which has been using the same naming convention for 15 years.
>
> Yep, but it is now in the minority, and consistency in package naming is
> always good.
>
> Would there be any problems for you in naming the distribution in a
> setuptools-friendly way from the next point release?
>
> cheers,
>
> Chris
>
> --
> Simplistix - Content Management, Batch Processing & Python Consulting
>           - http://www.simplistix.co.uk
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: regex (?!..) problem

[Wolfgang Rohdewald]

On Monday 05 October 2009, MRAB wrote:
> "(?!.*?(C1).*?\1)" will succeed only if ".*?(C1).*?\1" has failed,
>  in which case the group (group 1) will be undefined (no capture).

I see. 

I should have moved the (C1) out of this expression anyway:

>>> re.match(r'L(?P..)(?!.*?(?P=tile).*?(?P=tile))(.*?
(?P=tile))','LC1 C1B1B1B1 b3b3b3b3 C2C2C3').groups()
('C1', ' C1')

this solves my problem, thank you!


-- 
Wolfgang
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: epydoc - can I provide multiple dirs to parse

[Jean-Michel Pichavant]

Medi wrote:

Can I present multiple directories to epydoc to process. For example

epydoc -option -option -option dir_1 dir_2 dir_3

where dir_i is a directory containing some python src codes ?

I am currently running it on two directories with about 10M + 20 Meg
which takes a very long time (more than 40 minutes) and strace(1) is
showing mmap(2) ops back to backmight not be related but the
process gets wedged somewhere

Thanks
Medi
  

Here is an ugly python script we are using to build the doc.

#!/usr/bin/python
# coding: utf-8

if __name__ == '__main__':
   import sys
   try :
   import epydoc
   except ImportError:
   print "Error while importing epydoc"
   print "Make sure it is installed"
   from epydoc.cli import cli
   modules = ['dir_1','dir_2','dir_3']
   sys.argv = ["epydoc.py", "--html", "--no-frame", "--parse-only",
   "--exclude=.*scripts.bug.*",
   "--graph=all",
   #"--no-sourcecode",
   "--inheritance=grouped",
   "-v",
   "--output", 
"/home/labsys/public_html"

   ] + sys.argv[1:]
   sys.argv += modules
   cli()

--
http://mail.python.org/mailman/listinfo/python-list

Re: Opinions, please, on PEP 8 and local, 3rd party imports

[Jean-Michel Pichavant]

Philip Semanchuk wrote:

Hi all,
Our project uses some libraries that were written by 3rd parties (i.e. 
not us). These libraries fit into a single Python file and live in our 
source tree alongside other modules we've written. When our app is 
distributed, they'll be included in the installation. In other words, 
they're not prerequisites that we'll make the user install, and they 
won't live in site-packages.


For example, we use configobj from 
, and I think 
Beautiful Soup can be installed as a single file too, although we're 
not using that particular module.


PEP 8  says the following:

   Imports should be grouped in the following order:
   1. standard library imports
   2. related third party imports
   3. local application/library specific imports


I'm not sure in which category to place local, 3rd-party modules like 
configobj.


Clearly, they belong in category 3 since they're local.

Clearly, they also belong in category 2 since they're 3rd party 
modules, and explicitly labeling an imported module as "this is code 
we didn't write" is useful. But I've always reserved category 2  for 
external libraries (e.g. numpy, wxPython, Twisted, PIL, etc.).


Clearly, the best choice is category 2.5?

In your Pythonic opinion, should 3rd-party modules that live alongside 
homegrown code be listed in import category 2 or 3?


Thanks
Philip
--http://mail.python.org/mailman/listinfo/python-list

Since you provide a local file, this could be considered from a QA point 
of view as an uncontrolled version of the standard library. So it's no 
more standard, could be then considered as local.

I'm not sure it is *that* import though.

JM
--
http://mail.python.org/mailman/listinfo/python-list

Re: [Image-SIG] Some issue with easy_install and PIL/Imaging

[Chris Withers]

Fredrik Lundh wrote:

The problem is that too many people arguing for eggs do this by
sending nastygrams, which doesn't really provide much motivation for
doing anything about it (I don't do asshole-driven development). 


Indeed, I couldn't agree more, and I'm sorry you've been subjected to this.

My (hopefully more polite) request still stands though:
>> Would there be any problems for you in naming the distribution in a
>> setuptools-friendly way from the next point release?

> The

public review PIL got a couple a minutes ago matches some of the
private mail I've gotten:

   no egg - worst seen ever, remove it from pypi or provide an egg
(jensens, 2009-10-05, 0 points)


*sigh*

Interesting timing, myself and Doug Hellmann have been trying to 
persuade Martin von Lewis that while ratings are fine, the commenting 
system is likely to be abused:


https://sourceforge.net/tracker/?func=detail&atid=513503&aid=2866081&group_id=66150
https://sourceforge.net/tracker/?func=detail&atid=513503&aid=2872293&group_id=66150

I'll put in a request to have that comment removed...

Chris

--
Simplistix - Content Management, Batch Processing & Python Consulting
   - http://www.simplistix.co.uk
--
http://mail.python.org/mailman/listinfo/python-list

Re: can i use the browser to show the result of python

[catafest]

On Sep 25, 12:41 pm, Hacken  wrote:
> I have write some python script
>
> i want to use browser(IE or FF) to call it, an show the returns!
>
> how to?

Python script running under web browsers only like:
 python + website = django .

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Need feedback on subprocess-using function

[ryles]

On Oct 4, 9:46 pm, Nobody  wrote:
> On Sat, 03 Oct 2009 13:21:00 +, gb345 wrote:
> > I'm relatively new to Python, and I'm trying to get the hang of
> > using Python's subprocess module.  As an exercise, I wrote the Tac
> > class below, which can prints output to a file "in reverse order",
> > by piping it through the Unix tac utility.  (The idea is to delegate
> > the problem of managing the memory for an arbitrarily large task
> > to tac.)
> >         self.pipe = subprocess.Popen(['tac'], stdout=out,
> >                                      stdin=subprocess.PIPE,
> >                                      stderr=subprocess.PIPE)
> > This works OK, as far as I can tell, but I'm not sure that I've
> > dotted all the i's and crossed all the t's...  E.g., I had to add
> > the line "p.stdin.close()" to the close method when I when I ran
> > into sporadic deadlock at the p.stderr.read() statement.  Are there
> > other similar problems lurking in this code?
>
> Yep. If the process writes more than a buffer-full of data to stderr, it
> will deadlock. tac will block trying to write to stderr, and won't be
> reading its stdin, so your program will block trying to write to tac.
>
> This is why the POSIX popen() call only lets you attach a pipe to stdin or
> stdout but not both.
>
> If you want a "double-ended" slave process, you need to use polling or
> non-blocking I/O or asynchronous I/O or multiple threads. I'm not aware of
> any single solution which works on all platforms.
>
> The easiest way around this problem is to redirect stderr to a temporary
> file and read in the file's contents in the close() method.

There is also Popen.communicate():

http://docs.python.org/library/subprocess.html#subprocess.Popen.communicate
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Compile kinterbasdb with mingw32 and python 2.6 - DLL load failed

[Tonal]

I write path for compile kinterbasdb with mingw32 and python 2.6 for
rev
1038
See
http://sourceforge.net/tracker/?func=detail&aid=2872794&group_id=9913&atid=309913
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python shared lib

[ryles]

On Oct 4, 11:04 am, a...@pythoncraft.com (Aahz) wrote:
> I've got a dim memory that there's a reason for this -- you might try
> searching the python-dev archives and/or bugs.python.org.

Found mention of it in this discussion:

http://bugs.python.org/issue771998

"I disagree that --enable-shared should be the default. It is
difficult to maintain, as finding shared libraries is always
a PITA (in particular on Solaris). Its only real use is for
embedding, in even in the case of embedding, it is possible
to avoid using shared libraries."
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Delete all list entries of length unknown

[flebber]

On Oct 5, 3:05 pm, "Mark Tolonen"  wrote:
> "Chris Rebert"  wrote in message
>
> news:50697b2c0910042047i1cf2c1a3mc388bc74bab95...@mail.gmail.com...
>
> > Tuples are immutable (i.e. they cannot be modified after creation) and
> > are created using parentheses.
>
> Slight correction: tuples are created using commas.  Parentheses are only
> needed to disambiguate from other uses of comma:
>
> Python 2.6.2 (r262:71605, Apr 14 2009, 22:40:02) [MSC v.1500 32 bit (Intel)]
> on win32
> Type "help", "copyright", "credits" or "license" for more information.>>> a=1,
> >>> a
> (1,)
> >>> a=1,2,3
> >>> a
>
> (1, 2, 3)
>
> -Mark

Awesome that has cleared it up for me, plus a bit more thanks.
-- 
http://mail.python.org/mailman/listinfo/python-list

Conversion of npyscreen module to python 3 -- help!

[Nicholas]

Dear Pythoners,

Like a loyal python coder, I was attempting to convert a library to
python 3.  I have been stung in various ways by the new import
semantics - the tests that used to live happily within the module have
now had to be moved elsewhere. So be it.

But even though I have removed all the obvious things that are causing
it to break, and even refactored the whole file-scheme of the module
to make conversion more straight-forward, I'm still unable to get the
code to work.

I am clearly committing some horrible sin that works well in python 2
but which doesn't in python 3.  Despite reading the Release notes and
PEPs over and over, though, I can't see what is wrong.

If anyone could spare the time to have a quick look at the code at

http://code.google.com/p/npyscreen/

over and provide enlightenment, I would be extremely grateful.  (the
source code in the hg archive is more recent than the package for
download)

Best wishes,

Nicholas
Nicholas
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Q: sort's key and cmp parameters

[Paul Rubin]

Raymond Hettinger  writes:
> So, it looks like the relevant comparison values can be stored in
> nested lists:
> 
>list_of_lists = \
>  [[1, [3, [], []],
>   [7, [5, [], []], []]],
>   [19, [23, [], []],
>   []],
>  ]

Now you've copied all the data out of the original tree, which is even
worse than putting a class wrapper around it.  The comparison remains
(asymptotically) as expensive as before, too.

> Are there any of these trees that cannot be transformed
> (by a key function) into an equivalent nested list of lists
> and values so that the trees can be directly compared to one another
> using Python's normal built-in recursive comparison order for lists?

I think so.  Say you want to change the numeric comparisons so that
even numbers always sort before odd numbers, ie.
-4 < -2 < 0 < 2 < 4 < ... < -999 < ... -1 < 1 < 3 ...
I don't think there is a way to re-encode the numbers so they
can be compared with direct integer comparison.  Even if there is,
I think there are reasonable orderings on the trees themselves that
can't possibly be embedded in the standard python comparison order.
I might be able to construct an example using ordinal diagrams from
proof theory.
-- 
http://mail.python.org/mailman/listinfo/python-list

python multiprocessing proxy

[DrFalk3N]

I have a 2 processes:

the first process is manager.py and starts in backgroung:

from multiprocessing.managers import SyncManager, BaseProxy
from CompositeDict import *

class CompositeDictProxy(BaseProxy):

_exposed_ = ('addChild', 'setName')
def addChild(self, child):
return self._callmethod('addChild', [child])

def setName(self, name):
return self._callmethod('setName', [name])

class Manager(SyncManager):
def __init__(self):
super(Manager, self).__init__(address=('127.0.0.1',
5), authkey='abracadabra')

def start_Manager():
Manager().get_server().serve_forever()

if __name__=="__main__":
Manager.register('get_plant', CompositeDict,
proxytype=CompositeDictProxy)
start_Manager()

--

and the second is consumer.py supposed to use registered objects
defined into the manager:

from manager import *
import time
import random

class Consumer():

def __init__(self):
Manager.register('get_plant')

m = Manager()
m.connect()
plant = m.get_plant()
#plant.setName('alfa')
plant.addChild('beta')


if __name__=="__main__":
Consumer()

--

Running the manager in background, and than the consumer I get the
error message:
RuntimeError: maximum recursion depth exceeded,
when using addChild into the consumer, while I can correctly use
setName.


The detailed error message is:

Traceback (most recent call last):
  File "consumer.py", line 21, in 
Consumer()
  File "consumer.py", line 17, in __init__
plant.addChild('beta')
  File "", line 2, in addChild
  File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-
py2.5-linux-i686.egg/multiprocessing/managers.py", line 729, in
_callmethod
kind, result = conn.recv()
  File "/home/--/--/CompositeDict.py", line 99, in __getattr__
child = self.findChild(name)
  File "/home/--/--/CompositeDict.py", line 185, in findChild
for child in self.getAllChildren():
  File "/home/--/--/CompositeDict.py", line 167, in getAllChildren
l.extend(child.getAllChildren())
  File "/home/--/--/CompositeDict.py", line 165, in getAllChildren
for child in self._children:
  File "/home/--/--/CompositeDict.py", line 99, in __getattr__
child = self.findChild(name)
  File "/home/--/--/CompositeDict.py", line 185, in findChild
for child in self.getAllChildren():
  File "/--/--/prove/CompositeDict.py", line 165, in
getAllChildren
for child in self._children:
  ...
  File "/home/--/--/CompositeDict.py", line 99, in __getattr__
child = self.findChild(name)
  File "/home/--/--/CompositeDict.py", line 185, in findChild
for child in self.getAllChildren():
  RuntimeError: maximum recursion depth exceeded

Methods addChild and setName belongs to CompositeDict, I suppose to be
proxied.

What's wrong?

--

CompositeDict overwrites native __getattr__ method and is involved in
the error message. I suppose, in some way, it's not used the right one
__getattr__ method. If so how could I solve this problem??

Following used CompositeDict is taken from 
http://code.activestate.com/recipes/498249/
:

"""
A class which defines a composite object which can store
hieararchical dictionaries with names.

This class is same as a hiearchical dictionary, but it
provides methods to add/access/modify children by name,
like a Composite.

Created Anand B Pillai 

"""
__author__ = "Anand B Pillai"
__maintainer__ = "Anand B Pillai"
__version__ = "0.2"


def normalize(val):
""" Normalize a string so that it can be used as an attribute
to a Python object """

if val.find('-') != -1:
val = val.replace('-','_')

return val

def denormalize(val):
""" De-normalize a string """

if val.find('_') != -1:
val = val.replace('_','-')

return val

class SpecialDict(dict):
""" A dictionary type which allows direct attribute
access to its keys """

def __getattr__(self, name):

if name in self.__dict__:
return self.__dict__[name]
elif name in self:
return self.get(name)
else:
# Check for denormalized name
name = denormalize(name)
if name in self:
return self.get(name)
else:
raise AttributeError,'no attribute named %s' % name

def __setattr__(self, name, value):

if name in self.__dict__:
self.__dict__[name] = value
elif name in self:
self[name] = value
else:
# Check for denormalized name
name2 = denormalize(name)
if name2 in self:
self[name2] = value
else:
# New attribute
self[name] = value

class CompositeDict(Sp

conflict between various library to process EXIF data

[Bram Mertens]

Hi,

I posted this to [Image-SIG] a while ago but did not got a reply so I
assume it may not have been the appropriate list.  If this is again
not the right list please direct me to the proper list.

How to read and write EXIF data from images has been discussed several
times on the [Image-SIG] list before but I could not find a discussion
on the problem I've run into when using multiple of these.

In short: after adding a comment to the EXIF header using the
setComment(...) and writeMetadata(...) methods of pyeviv2
(http://tilloy.net/dev/pyexiv2/doc/release-0.1.2/pyexiv2.htm) the EXIF
header is altered in such a way that it is no longer possible to read
the EXIF header with EXIF.py.
Furthermore  imghdr does not even recognize it as a JPEG anymore.

>>> import imghdr
>>> type = imghdr.what('DSC02299.JPG')
>>> print type
jpeg
>>> type = imghdr.what('20070623-171634-DSC-H5.jpg')
>>> print type
None

Also the file command in Linux no longer recognizes the EXIF header:
before:
$ file DSC02299.JPG
DSC02299.JPG: JPEG image data, EXIF standard 2.21
after:
$ file 20070623-171634-DSC-H5.jpg
20070623-171634-DSC-H5.jpg: JPEG image data
(Note: part of the processing done is to rename the image based on the
date in the EXIF header)

I switched from EXIF.py to pyexiv2 because the first does not provide
a method to add a comment to the header.  Considering the fact that
EXIF.py appears to be no longer under active development I simply
replaced all my calls to EXIF.py with calls to the relevant pyexeiv2
methods.  However imghdr is part of the standard library so the fact
it will not recognize the image anymore makes me wonder if maybe
pyexiv2 is doing something "wrong" that might cause other problems
later on.

Can anybody provide insight into this?

Thanks in advance

Bram
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: some site login problem help plz..

[james27]

still looking for good solution.
anyway..thanks Diez :)

Diez B. Roggisch-2 wrote:
> 
> james27 wrote:
> 
>> 
>> hello..
>> im new to python.
>> i have some problem with mechanize.
>> before i was used mechanize with no problem.
>> but i couldn't success login with some site.
>> for several days i was looked for solution but failed.
>> my problem is , login is no probelm but can't retrieve html source code
>> from opened site.
>> actually only can read some small html code, such like below.
>> 
>> 
>> 
>> location.replace("http://www.naver.com";);
>> 
>> 
>> 
>> i want to retrive full html source code..but i can't . i was try with
>> twill and mechanize and urllib and so on.
>> i have no idea.. anyone can help me?
> 
> Your problem is that the site uses JavaScript to replace itself. Mechanize
> can't do anything about that. You might have more luck with scripting a
> browser. No idea if there are any special packages available for that
> though.
> 
> Diez
> -- 
> http://mail.python.org/mailman/listinfo/python-list
> 
> 

-- 
View this message in context: 
http://www.nabble.com/some-site-login-problem-help-plz..-tp25746497p25750229.html
Sent from the Python - python-list mailing list archive at Nabble.com.

-- 
http://mail.python.org/mailman/listinfo/python-list

Twisted PB vs JMS

[jacopo]

I looked at Twisted PB for an architecture with one client
distributing some processing to several servers. Now I came across JMS
and I have seen that using ActiveMQ with the Stomp protocol there
would be a good support for Python.
Surprising I couldn’t  find any article comparing the two
technologies. I wonder if they target different problems (it doesn’t
look to me).

Is anyone able to sketch advantages of one solution against the other?

Thanks,
Jacopo

-- 
http://mail.python.org/mailman/listinfo/python-list

Is there a way to specify a superclass at runtime?

[Chris Colbert]

I have an application that needs to run different depending on whether
the input data is being simulated, or provided from instrumentation.

I am trying to abstract this machinery in a single class called
Controller which I want to inherit from either SimController or
RealController based on whether a module level flag SIMULATION is set
to True or False.

so I have something like this:


SIMULATION = False

class SimController(object):
"do sim stuff here"

class RealController(object):
" do real stuff here"

class Controller(SuperKlass):
pass


so if SIMULATION == False I want to be able to instance a Controller
object that inherits from RealController and vice-versa.

I thought this might be possible with metaclasses, but I didnt find
anything useful in the docs or on google.

Thanks for any help!

Cheers,

Chris
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is there a way to specify a superclass at runtime?

[MRAB]

Chris Colbert wrote:

I have an application that needs to run different depending on whether
the input data is being simulated, or provided from instrumentation.

I am trying to abstract this machinery in a single class called
Controller which I want to inherit from either SimController or
RealController based on whether a module level flag SIMULATION is set
to True or False.

so I have something like this:


SIMULATION = False

class SimController(object):
"do sim stuff here"

class RealController(object):
" do real stuff here"

class Controller(SuperKlass):
pass


so if SIMULATION == False I want to be able to instance a Controller
object that inherits from RealController and vice-versa.

I thought this might be possible with metaclasses, but I didnt find
anything useful in the docs or on google.

Thanks for any help!


Why not just:

SIMULATION = False

class SimController(object):
"do sim stuff here"

class RealController(object):
" do real stuff here"

if SIMULATION:
SuperKlass = SimController
else:
SuperKlass = RealController

class Controller(SuperKlass):
pass
--
http://mail.python.org/mailman/listinfo/python-list

Re: Is there a way to specify a superclass at runtime?

[Richard Brodie]

"Chris Colbert"  wrote in message 
news:mailman.868.1254748945.2807.python-l...@python.org...

> I am trying to abstract this machinery in a single class called
> Controller which I want to inherit from either SimController or
> RealController based on whether a module level flag SIMULATION is set
> to True or False.

At first sight, that seems kind of odd. Wouldn't it be simpler to have
SimController and RealController inherit from Controller? 


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is there a way to specify a superclass at runtime?

[Chris Colbert]

because when i import this module, the classes will already be
determined by the intitial flag setting.

i.e.
SIMULATION = False

class SimController(object):
   def foo(self):
   print 'bar'

class RealController(object):
   def foo(self):
   print 'baz'

if SIMULATION:
   SuperKlass = SimController
else:
   SuperKlass = RealController

class Controller(SuperKlass):
   pass





In [2]: import testcontroller

In [3]: testcontroller.SIMULATION
Out[3]: False

In [4]: c = testcontroller.Controller()

In [5]: c.foo()
baz

In [6]: testcontroller.SIMULATION = True

In [7]: c = testcontroller.Controller()

In [8]: c.foo()
baz

On Mon, Oct 5, 2009 at 3:32 PM, MRAB  wrote:
> Chris Colbert wrote:
>>
>> I have an application that needs to run different depending on whether
>> the input data is being simulated, or provided from instrumentation.
>>
>> I am trying to abstract this machinery in a single class called
>> Controller which I want to inherit from either SimController or
>> RealController based on whether a module level flag SIMULATION is set
>> to True or False.
>>
>> so I have something like this:
>>
>>
>> SIMULATION = False
>>
>> class SimController(object):
>>    "do sim stuff here"
>>
>> class RealController(object):
>>    " do real stuff here"
>>
>> class Controller(SuperKlass):
>>    pass
>>
>>
>> so if SIMULATION == False I want to be able to instance a Controller
>> object that inherits from RealController and vice-versa.
>>
>> I thought this might be possible with metaclasses, but I didnt find
>> anything useful in the docs or on google.
>>
>> Thanks for any help!
>>
> Why not just:
>
> SIMULATION = False
>
> class SimController(object):
>    "do sim stuff here"
>
> class RealController(object):
>    " do real stuff here"
>
> if SIMULATION:
>    SuperKlass = SimController
> else:
>    SuperKlass = RealController
>
> class Controller(SuperKlass):
>    pass
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is there a way to specify a superclass at runtime?

[Chris Colbert]

I dont think so, because that would require logic outside of the
controller class to determine which controller to instantiate.

My whole purpose for Controller is to encapsulate this logic.

So, if the data should be simulated, then i just need to pass
-simulate True as a command line argument, and the controller takes
care of it...

On Mon, Oct 5, 2009 at 3:44 PM, Richard Brodie  wrote:
>
> "Chris Colbert"  wrote in message
> news:mailman.868.1254748945.2807.python-l...@python.org...
>
>> I am trying to abstract this machinery in a single class called
>> Controller which I want to inherit from either SimController or
>> RealController based on whether a module level flag SIMULATION is set
>> to True or False.
>
> At first sight, that seems kind of odd. Wouldn't it be simpler to have
> SimController and RealController inherit from Controller?
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is there a way to specify a superclass at runtime?

[Chris Colbert]

I suppose i can just move the SIMULATION flag to another module, and
then import it and check it before intstantiation.

So, the arg parser will have to set the flag before any other
processing begins...

On Mon, Oct 5, 2009 at 3:49 PM, Chris Colbert  wrote:
> I dont think so, because that would require logic outside of the
> controller class to determine which controller to instantiate.
>
> My whole purpose for Controller is to encapsulate this logic.
>
> So, if the data should be simulated, then i just need to pass
> -simulate True as a command line argument, and the controller takes
> care of it...
>
> On Mon, Oct 5, 2009 at 3:44 PM, Richard Brodie  wrote:
>>
>> "Chris Colbert"  wrote in message
>> news:mailman.868.1254748945.2807.python-l...@python.org...
>>
>>> I am trying to abstract this machinery in a single class called
>>> Controller which I want to inherit from either SimController or
>>> RealController based on whether a module level flag SIMULATION is set
>>> to True or False.
>>
>> At first sight, that seems kind of odd. Wouldn't it be simpler to have
>> SimController and RealController inherit from Controller?
>>
>>
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>>
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is there a way to specify a superclass at runtime?

[MRAB]

Chris Colbert wrote:

because when i import this module, the classes will already be
determined by the intitial flag setting.

i.e.
SIMULATION = False

class SimController(object):
   def foo(self):
   print 'bar'

class RealController(object):
   def foo(self):
   print 'baz'

if SIMULATION:
   SuperKlass = SimController
else:
   SuperKlass = RealController

class Controller(SuperKlass):
   pass





In [2]: import testcontroller

In [3]: testcontroller.SIMULATION
Out[3]: False

In [4]: c = testcontroller.Controller()

In [5]: c.foo()
baz

In [6]: testcontroller.SIMULATION = True

In [7]: c = testcontroller.Controller()

In [8]: c.foo()
baz


[snip]
You could put the Controller class inside a factory function:

def Controller():
if SIMULATION:
SuperKlass = SimController
else:
SuperKlass = RealController
class Controller(SuperKlass):
pass
return Controller()
--
http://mail.python.org/mailman/listinfo/python-list

SAX: Short tag's ...

[Thomas Lehmann]

Hi!

Is there a way to recognize short tags in a XML?
I'm implementing a SAX handler...

Problem: storing the XML code I would need this information
in the startElement ...

How can I handle this?


any text
-- 
http://mail.python.org/mailman/listinfo/python-list

Object Relational Mappers are evil (a meditation)

[Aaron Watters]

This is a bit off topic except that many Python
programmers seem to be allergic to typing SQL.

RESOLVED:  Using ORMs leads lazy programmers
to make bad database designs.  It's better to
carefully design your database with no invisible
means of support and there is no reason to not
use SQL directly for this purpose.

FOR EXAMPLE:  Consider blogging.  The most
successful blog software is WORDPRESS.  Here
is the WordPress data model:

http://blog.kapish.co.in/wp-content/uploads/2009/03/wp_2.7.png

Beautiful, isn't it?  It was designed by people who
thought about what they were doing and did it carefully.

Now let's look at the Sakai Blogger tool data model
(as reverse engineered by someone who had to
fix a bug -- there actually was no data design created
by the implementers):

confluence.sakaiproject.org/download/attachments/17072138/
entitymodel.pdf

How did the above happen?  I suspect someone opened
up Eclipse and started typing, relying on the Hibernate
ORM to handle all the database stuff automagically.  The
result is a massive headache for people like me.  Another
one.  I routinely open up the mysql prompt and start typing
"show tables", "describe table blah", "select * from blah limit 10"...
trying to figure out WTF Hibernate did with my data.

Occasionally I fantasize about making a non-trivial change
to one of these programs, but I strongly resist going further
than that because the ORM meatgrinder makes it somewhere
between extremely unpleasant and impossible to make any
non-trivial changes to a non-trivial program, especially after
it has been populated with data.

Ok I feel a little better now.  Maybe I should get back
to work...

   -- Aaron Watters
  http://aaron.oirt.rutgers.edu/myapp/docs/W1100_1200.wwiki

===

It has been said that democracy is the worst form of government except
all the others that have been tried.
Sir Winston Churchill
http://www.quotationspage.com/quote/364.html
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Object Relational Mappers are evil (a meditation)

[Bruno Desthuilliers]

Aaron Watters a écrit :

This is a bit off topic except that many Python
programmers seem to be allergic to typing SQL.


What I am mostly allergic to is manipulating sql queries as strings and 
resultsets as lists of tuples. I strongly prefer a higher level 
representation of both the queries and the resultsets. From this POV, 
SQLAlchemy or Django's ORM are better than the primitive scheme above - 
and don't have that much impact on db design.



RESOLVED:  Using ORMs leads lazy programmers
to make bad database designs.


to me, a "lazy" programmer is someone that is mostly concerned with not 
repeating itself. As such, "lazy" usually ends up with better designs 
(db or whatever) !-)




 It's better to
carefully design your database with no invisible
means of support and there is no reason to not
use SQL directly for this purpose.


I don't use SQL to *design* my databases, I use SQL (directly or 
indirectly) to create and mananage my schemas and datas.


For the design part, I usually start with a pencil and some paper.

(snip remaining rant).
--
http://mail.python.org/mailman/listinfo/python-list

Module inspect Bug

[Tomas Zulberti]

Hi. I have a class that extends collections.MutableMapping. I am
checking if it is abstract, using the module inspect. But isabstract
returns a number different from zero insted of True or False. The
problem with that is that sometimes it returns False when the class
isn't an abstract.

>>> inspect.isabstract(collections.MutableMapping)
1048576
>>> inspect.isabstract(os)
False

Its true that the condition nevertheless will be True on the if, but
the return value I think that should be boolean.

Thanks,
Tomas Zulberti

pd: Sorry for my bad English...
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: SAX: Short tag's ...

[Diez B. Roggisch]

Thomas Lehmann wrote:

> Hi!
> 
> Is there a way to recognize short tags in a XML?
> I'm implementing a SAX handler...
> 
> Problem: storing the XML code I would need this information
> in the startElement ...
> 
> How can I handle this?
> 
> 
> any text

I don't think there is a (standard) way, and it shouldn't matter (unless you
are actually talking about HTML, where sadly enough it does).

In XML,  is the exact same thing as . 

Or do you need to *write* XML with shortened tags?

Diez
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is there a way to specify a superclass at runtime?

[Chris Colbert]

that's a good idea.

Thanks!


On Mon, Oct 5, 2009 at 4:18 PM, MRAB  wrote:
> Chris Colbert wrote:
>>
>> because when i import this module, the classes will already be
>> determined by the intitial flag setting.
>>
>> i.e.
>> SIMULATION = False
>>
>> class SimController(object):
>>   def foo(self):
>>       print 'bar'
>>
>> class RealController(object):
>>   def foo(self):
>>       print 'baz'
>>
>> if SIMULATION:
>>   SuperKlass = SimController
>> else:
>>   SuperKlass = RealController
>>
>> class Controller(SuperKlass):
>>   pass
>>
>>
>>
>>
>>
>> In [2]: import testcontroller
>>
>> In [3]: testcontroller.SIMULATION
>> Out[3]: False
>>
>> In [4]: c = testcontroller.Controller()
>>
>> In [5]: c.foo()
>> baz
>>
>> In [6]: testcontroller.SIMULATION = True
>>
>> In [7]: c = testcontroller.Controller()
>>
>> In [8]: c.foo()
>> baz
>>
> [snip]
> You could put the Controller class inside a factory function:
>
> def Controller():
>        if SIMULATION:
>                SuperKlass = SimController
>        else:
>                SuperKlass = RealController
>        class Controller(SuperKlass):
>                pass
>        return Controller()
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Enormous Input and Output Test

[nn]

On Oct 4, 8:41 am, Duncan Booth  wrote:
> Jon Clements  wrote:
> > On Oct 4, 12:08 pm, n00m  wrote:
> >> Duncan Booth,
>
> >> alas... still TLE:
>
> >> 2800839
> >> 2009-10-04 13:03:59
> >> Q
> >> Enormous Input and Output Test
> >> time limit exceeded
> >> -
> >> 88M
> >> PYTH
>
> > Just to throw into the mix...
>
> > What about buffering? Does anyone know what the effective stdin buffer
> > is for Python? I mean, it really can't be the multiplying that's a
> > bottleneck. Not sure if it's possible, but can a new stdin be created
> > (possibly using os.fdopen) with a hefty buffer size?
>
> > I'm probably way off, but something to share.
>
> I did try a version where I just read the data in, split it up, and then
> wrote it out again. On my test file that took about 2 seconds compared with
> the 8 seconds it took the full code I posted, so while there may be scope
> for faster I/O (e.g. using mmap), any real speedup would have to be in the
> convert to int, multiply, convert back to str pipeline.

This takes 5 second on my machine using a file with 1,000,000 random
entries:

inf=open('tstf')
outf=open('tstof','w')
for i in xrange(int(inf.next(),10)):
n1,s,n2=inf.next().partition(' ')
print >>outf,int(n1,10)*int(n2,10)
outf.close()
inf.close()
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Skeletal animation

[Manowar]

On Oct 4, 8:38 pm, Carl Banks  wrote:
> On Oct 4, 5:16 pm, Manowar  wrote:
>
>
>
>
>
> > On Oct 4, 6:38 pm, TerryP  wrote:
>
> > > On Oct 4, 10:05 pm, Manowar  wrote:
>
> > > > I am new to pyton and have asked this question several times the
> > > > answer is always not sure.
> > > > Here is my question sekeltal animation ( bone animation) is it
> > > > possible with python? What i want to develop is an aquarium in
> > > > realtime, skeletal animation, all the movements will be from
> > > > programming., no keyframes  let me know if it is possible with python
>
> > > > Manowar
>
> > > Depending on how much code you want to write, you'll probably want to
> > > check out something like pygame or python-ogre
>
> > Yeah, i know about those but i just really need to know if it is
> > possible to do in python
> > if it is what about some tutorials on bone animation?
>
> It's possible.  I can't really recommend a tutorial because I don't
> know where you are coming from, and it is a very complex subject.
> What do know about 3D animation so far?  Have you done 3D animation in
> other languages?  Have you done 3D mesh modeling with a tool like 3DS
> Max or Blender?  What is your mathematical background (especially in
> linear algrebra)?  Please help us help you by providing us details on
> what you know already and what you want to do.  You won't get much
> help by asking simple yes/no questions.
>
> Carl Banks- Hide quoted text -
>
> - Show quoted text -


Let me tell you in more detail what i want maybe i am not explaining
it correctly ( i will look at those links) I have been using Blender
for years. I know animation but not an expert. If i make an aquarium i
want the fish to move on their own with their own A.I. (Self-
contained fish.) To make it clearer if i put a rock in it's path, the
fish will move around the rock and if a predator comes into the scene
the fish will scatter away at a high speed then return to normal when
the predator goes. I know this can be done in C++ I won't go into why
i am not using c++ just need to know if i can manipulate the bones in
realtime with python. can't find any info on that but someone must
know how to do it in python. Don't want keyframes just want true bone
animation

Manowar
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is there a way to specify a superclass at runtime?

[Jean-Michel Pichavant]

Chris Colbert wrote:

I dont think so, because that would require logic outside of the
controller class to determine which controller to instantiate.

My whole purpose for Controller is to encapsulate this logic.

So, if the data should be simulated, then i just need to pass
-simulate True as a command line argument, and the controller takes
care of it...

On Mon, Oct 5, 2009 at 3:44 PM, Richard Brodie  wrote:
  

"Chris Colbert"  wrote in message
news:mailman.868.1254748945.2807.python-l...@python.org...



I am trying to abstract this machinery in a single class called
Controller which I want to inherit from either SimController or
RealController based on whether a module level flag SIMULATION is set
to True or False.
  

At first sight, that seems kind of odd. Wouldn't it be simpler to have
SimController and RealController inherit from Controller?


--
http://mail.python.org/mailman/listinfo/python-list



Please don't top post.

Yet Richard's design is the way to go.

controller.py:

class Controller:
   def getInstance(simulation):
  if simulation:
 return SimController()
  else:
 return RealController()
   getInstance = staticmethod(getInstance)

class RealController(Controller):
   pass

class SimController(Controller):
   pass

myController = Controller.getInstance(simulation=True)


I personnally would define getInstance as a module function, but because 
you prefer to put all the logic in Controller... It doesn't really 
matter in the end.


Cheers,

Jean-Michel
--
http://mail.python.org/mailman/listinfo/python-list

Re: PIL : How to write array to image ???

[Martin]

On Oct 4, 10:16 pm, "Mart."  wrote:
> On Oct 4, 9:47 am, Martin  wrote:
>
>
>
> > On Oct 3, 11:56 pm, Peter Otten <__pete...@web.de> wrote:
>
> > > Martin wrote:
> > > > Dear group
>
> > > > I'm trying to use PIL to write an array (a NumPy array to be exact) to
> > > > an image.
> > > > Peace of cake, but it comes out looking strange.
>
> > > > I use the below mini code, that I wrote for the purpose. The print of
> > > > a looks like expected:
>
> > > > [[ 200.  200.  200. ...,    0.    0.    0.]
> > > >  [ 200.  200.  200. ...,    0.    0.    0.]
> > > >  [ 200.  200.  200. ...,    0.    0.    0.]
> > > >  ...,
> > > >  [   0.    0.    0. ...,  200.  200.  200.]
> > > >  [   0.    0.    0. ...,  200.  200.  200.]
> > > >  [   0.    0.    0. ...,  200.  200.  200.]]
>
> > > > But the image looks nothing like that.
>
> > > > Please see the images on:
> > > >http://hvidberg.net/Martin/temp/quat_col.png
> > > >http://hvidberg.net/Martin/temp/quat_bw.png
>
> > > > or run the code to see them locally.
>
> > > > Please – what do I do wrong in the PIL part ???
>
> > > > :-? Martin
>
> > > > import numpy as np
> > > > from PIL import Image
> > > > from PIL import ImageOps
>
> > > > maxcol = 100
> > > > maxrow = 100
>
> > > > a = np.zeros((maxcol,maxrow),float)
>
> > > > for i in range(maxcol):
> > > >     for j in range(maxrow):
> > > >         if (i<(maxcol/2) and j<(maxrow/2)) or (i>=(maxcol/2) and j>=
> > > > (maxrow/2)):
> > > >             a[i,j] = 200
> > > >         else:
> > > >             a[i,j] = 0
>
> > > > print a
>
> > > > pilImage = Image.fromarray(a,'RGB')
> > > > pilImage.save('quat_col.png')
> > > > pilImage = ImageOps.grayscale(pilImage)
> > > > pilImage.save('quat_bw.png')
>
> > > The PIL seems to copy the array contents directly from memory without any
> > > conversions or sanity check. In your example The float values determine 
> > > the
> > > gray value of 8 consecutive pixels.
>
> > > If you want a[i,j] to become the color of the pixel (i, j) you have to use
> > > an array with a memory layout that is compatible to the Image.
> > > Here are a few examples:
>
> > > >>> import numpy
> > > >>> from PIL import Image
> > > >>> a = numpy.zeros((100, 100), numpy.uint8)
> > > >>> a[:50, :50] = a[50:, 50:] = 255
> > > >>> Image.fromarray(a).save("tmp1.png")
> > > >>> b = numpy.zeros((100, 100, 3), numpy.uint8)
> > > >>> b[:50, :50, :] = b[50:, 50:, :] = [255, 0, 0]
> > > >>> Image.fromarray(b).save("tmp2.png")
> > > >>> c = numpy.zeros((100, 100), numpy.uint32)
> > > >>> c[:50, :50] = c[50:, 50:] = 0xff808000
> > > >>> Image.fromarray(c, "RGBA").save("tmp3.png")
>
> > > Peter
>
> > Thanks All - That helped a lot...
>
> > The working code ended with:
>
> >         imga = np.zeros((imgL.shape[1],imgL.shape[0]),np.uint8)
> >         for ro in range(imgL.shape[1]):
> >             for co in range(imgL.shape[0]):
> >                 imga[ro,co] = imgL[ro,co]
> >         Image.fromarray(imga).save('_a'+str(lev)+'.png')
>
> Without knowing how big your image is (can't remember if you said!).
> Perhaps rather than looping in the way you might in C for example, the
> numpy where might be quicker if you have a big image. Just a
> thought...

And a good thought too... I use to teach ansi C at Univ. many ears
ago, and it's still one of my favorits. But I prefere Python for this
type of work. Python have a much faster develop time, is much easier
to write (fewer linees, simpler syntax) and is much more 'hot' in
these days. Finally my favorit GIS and RS applications support Python
out-of-the-box.

:-) Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Walking an object/reference graph

[Simon Forman]

On Mon, Oct 5, 2009 at 3:59 AM, Austin Bingham  wrote:
> I'm looking for the proper way to "walk" a graph of python objects. My
> specific task is to find all objects of a given type that are referred
> to (transitively) by some starting object. My approach has been to
> loop through the object's attributes, examining each, and then
> recursing on them.
>
> This approach has immediate problems in that I immediately hit
> infinite recursion due to method-wrappers; I'm not sure *exactly*
> what's going on, but it appears that method-wrappers refer to
> method-wrappers ad infinitum. If I add code to avoid this issue,
> others soon crop up, so before I wrote too much more code, I thought
> I'd check with experts to see if there's a better way.
>
> So, if I've explained what I'm looking for well enough, does anyone
> know of a proper way to walk a graph of object references? I need to
> support cycles in the graph, so keeping track of visited object IDs or
> something will have to be employed. Also, I need to be able to follow
> references stored in lists, dictionaries, etc...basically any way to
> programatically get from one object to another should be followed by
> the algorithm. I feel like I'm missing something simple, but perhaps
> this is just a tricky problem for python. Any ideas or insight would
> be great. Thanks!
>
> Austin
> --
> http://mail.python.org/mailman/listinfo/python-list
>

Look in the pickle module.  It has to do this to serialize arbitrary
python objects I think.

HTH,
~Simon
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Skeletal animation

[Donn]

On Monday 05 October 2009 18:09:46 Manowar wrote:
> just need to know if i can manipulate the bones in
> realtime with python. 
If you are looking for some kind of "from awesome import Bones", then I really 
don't know of anything. You should look at the various 3D toolkits (like Panda 
3D, python-gl etc.) and use what they offer and extend it.

see: http://www.panda3d.org/wiki/index.php/Attaching_an_Object_to_a_Joint

\d
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why is this slower?

[Stefan Behnel]

Joseph Reagle wrote:
> I would think the commented code would be faster (fewer loops), but it is
> not (because of function calls).

You just answered your own question.


> inSRC = set([bio.name for bio in bios.values()])

Note that this is actually very inefficient as it first creates a list of
values, then iterates over them to create a new list, and then creates a
set from that result. Py3 has set comprehensions, but in Py2.5+, you might
want to try .itervalues() and a generator expression instead.

Stefan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: organizing your scripts, with plenty of re-use

[Buck]

>
> With the package layout, you would just do:
>
>    from parrot.sleeping import sleeping_in_a_bed
>    from parrot.feeding.eating import eat_cracker
>
> This is really much more straightforward than you are making it out to be.
>

As in the OP, I need things to "Just Work" without installation
requirements.
The reason for this is that I'm in a large corporate environment
servicing many groups with their own custom environments.

Your solution requires work and knowledge on the user's part, but Stef
seems to be actually solving the complexity problem. It may seem
trivial to you, but it's beyond many people's grasp and brings the
usability and reliability of the system way down.

Like Stef, I was unable to grasp how to properly use python packages
in my environment even after reading the documentation repeatedly over
the course of several months.

The purpose of this thread is to discover and discuss how to use
packages in a user-space (as opposed to python-installation)
environment.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pool module -- does map pre-assign tasks to processes?

[Aahz]

In article ,
Luca   wrote:
>I would like to use the Pool module, but my tasks take sometimes
>unpredictably different time to complete.  The simplest way to write
>the code would be to put all task descriptions in an array, then call
>
>p = Pool(8)
>p.map(f, a)
>
>But I don't want to preassign process 0 with elements a[0], a[8], a
>[16],  process 1 with elements a[1], a[9], a[17], and so forth.
>Rather, I would like all tasks to be put into a queue, and I would
>like the processes to each time grab the next task to be done, and do
>it.  This would ensure fairly equal loading.
>
>My question is: does the map() method of Pool pre-assign which task
>gets done by each process, or is this done at runtime, on a get-first-
>task-to-be-done basis?

You can probably get what you want by setting chunksize to 1.
-- 
Aahz (a...@pythoncraft.com)   <*> http://www.pythoncraft.com/

"Normal is what cuts off your sixth finger and your tail..."  --Siobhan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: epydoc - can I provide multiple dirs to parse

[Montaseri]

Thank you

It looks like it is possible to feed multiple dirs,

Also, can you explain the --inheritance=STYLE for me. What does the author
mean by values like "grouped", "listed", "included"
I could not find any good document explaining these...

Thanks
Medi

On Mon, Oct 5, 2009 at 2:09 AM, Jean-Michel Pichavant <
jeanmic...@sequans.com> wrote:

> Medi wrote:
>
>> Can I present multiple directories to epydoc to process. For example
>>
>> epydoc -option -option -option dir_1 dir_2 dir_3
>>
>> where dir_i is a directory containing some python src codes ?
>>
>> I am currently running it on two directories with about 10M + 20 Meg
>> which takes a very long time (more than 40 minutes) and strace(1) is
>> showing mmap(2) ops back to backmight not be related but the
>> process gets wedged somewhere
>>
>> Thanks
>> Medi
>>
>>
> Here is an ugly python script we are using to build the doc.
>
> #!/usr/bin/python
> # coding: utf-8
>
> if __name__ == '__main__':
>   import sys
>   try :
>   import epydoc
>   except ImportError:
>   print "Error while importing epydoc"
>   print "Make sure it is installed"
>   from epydoc.cli import cli
>   modules = ['dir_1','dir_2','dir_3']
>   sys.argv = ["epydoc.py", "--html", "--no-frame", "--parse-only",
>   "--exclude=.*scripts.bug.*",
>   "--graph=all",
>   #"--no-sourcecode",
>   "--inheritance=grouped",
>   "-v",
>   "--output",
> "/home/labsys/public_html"
>   ] + sys.argv[1:]
>   sys.argv += modules
>   cli()
>
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: organizing your scripts, with plenty of re-use

[Robert Kern]

On 2009-10-05 12:42 PM, Buck wrote:


With the package layout, you would just do:

from parrot.sleeping import sleeping_in_a_bed
from parrot.feeding.eating import eat_cracker

This is really much more straightforward than you are making it out to be.



As in the OP, I need things to "Just Work" without installation
requirements.
The reason for this is that I'm in a large corporate environment
servicing many groups with their own custom environments.


The more ad hoc hacks you use rather than the standard approaches, the harder it 
is going to be for you to support those custom environments.



Your solution requires work and knowledge on the user's part,


*All* solutions require work and knowledge. There is no free lunch. The 
advantage of standard Python packages is that they are understood the best and 
the most widely.



but Stef
seems to be actually solving the complexity problem. It may seem
trivial to you, but it's beyond many people's grasp and brings the
usability and reliability of the system way down.

Like Stef, I was unable to grasp how to properly use python packages
in my environment even after reading the documentation repeatedly over
the course of several months.


I do believe that you and Stef are exceptions. The vast majority of Python users 
seem to be able to grasp packages well enough.


--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

--
http://mail.python.org/mailman/listinfo/python-list

Re: organizing your scripts, with plenty of re-use

[Buck]

On Oct 5, 11:29 am, Robert Kern  wrote:
> On 2009-10-05 12:42 PM, Buck wrote:
>
>
>
> >> With the package layout, you would just do:
>
> >>     from parrot.sleeping import sleeping_in_a_bed
> >>     from parrot.feeding.eating import eat_cracker
>
> >> This is really much more straightforward than you are making it out to be.
>
> > As in the OP, I need things to "Just Work" without installation
> > requirements.
> > The reason for this is that I'm in a large corporate environment
> > servicing many groups with their own custom environments.
>
> The more ad hoc hacks you use rather than the standard approaches, the harder 
> it
> is going to be for you to support those custom environments.

I too would prefer a standard approach but there doesn't seem to be an
acceptable one.

> I do believe that you and Stef are exceptions. The vast majority of Python 
> users
> seem to be able to grasp packages well enough.

You're failing to differentiate between python programmer and a
system's users. I understand packages well enough, but I need to
reduce the users' requirements down to simply running a command. I
don't see a way to do that as of now without a large amount of
boilerplate code in every script.

I've considered installing the thing to the PYTHONPATH as most people
suggest, but this has two drawbacks:
  * Extremely hard to push thru my IT department. Possibly impossible.
  * Local checkouts of scripts use the main installation, rather than
the local, possibly revised package code. This necessitates the
boilerplate that installation to the PYTHONPATH was supposed to avoid.
  * We can work around the previous point by requiring a user-owned
dev installation of Python, but this raises the bar to entry past most
of my co-developers threshold. They are more comfortable with tcsh and
perl...

I think the issue here is that the current python-package system works
well enough for the core python devs but leaves normal python
developers without much options beyond "all scripts in one directory"
or "tons of boilerplate everywhere".
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: epydoc - can I provide multiple dirs to parse

[Medi]

On Oct 5, 2:09 am, Jean-Michel Pichavant 
wrote:
> Medi wrote:
> > Can I present multiple directories toepydocto process. For example
>
> >epydoc-option -option -option dir_1 dir_2 dir_3
>
> > where dir_i is a directory containing some python src codes ?
>
> > I am currently running it on two directories with about 10M + 20 Meg
> > which takes a very long time (more than 40 minutes) and strace(1) is
> > showing mmap(2) ops back to backmight not be related but the
> > process gets wedged somewhere
>
> > Thanks
> > Medi
>
> Here is an ugly python script we are using to build the doc.
>
> #!/usr/bin/python
> # coding: utf-8
>
> if __name__ == '__main__':
>         import sys
>         try :
>                 importepydoc
>         except ImportError:
>                 print "Error while importingepydoc"
>                 print "Make sure it is installed"
>         fromepydoc.cli import cli
>         modules = ['dir_1','dir_2','dir_3']
>         sys.argv = ["epydoc.py", "--html", "--no-frame", "--parse-only",
>                                         "--exclude=.*scripts.bug.*",
>                                         "--graph=all",
>                                         #"--no-sourcecode",
>                                         "--inheritance=grouped",
>                                         "-v",
>                                         "--output",
> "/home/labsys/public_html"
>                                         ] + sys.argv[1:]
>         sys.argv += modules
>         cli()

Looks like multiple dir should work...
Also, what does the author mean by --inheritance=STYLE where style =
{ grouped, listed, included } . The help text does not explain this
well.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: organizing your scripts, with plenty of re-use

[Robert Kern]

On 2009-10-05 13:48 PM, Buck wrote:

On Oct 5, 11:29 am, Robert Kern  wrote:

On 2009-10-05 12:42 PM, Buck wrote:




With the package layout, you would just do:



 from parrot.sleeping import sleeping_in_a_bed
 from parrot.feeding.eating import eat_cracker



This is really much more straightforward than you are making it out to be.



As in the OP, I need things to "Just Work" without installation
requirements.
The reason for this is that I'm in a large corporate environment
servicing many groups with their own custom environments.


The more ad hoc hacks you use rather than the standard approaches, the harder it
is going to be for you to support those custom environments.


I too would prefer a standard approach but there doesn't seem to be an
acceptable one.


I do believe that you and Stef are exceptions. The vast majority of Python users
seem to be able to grasp packages well enough.


You're failing to differentiate between python programmer and a
system's users. I understand packages well enough, but I need to
reduce the users' requirements down to simply running a command. I
don't see a way to do that as of now without a large amount of
boilerplate code in every script.


I would like to see an example of such boilerplate. I do not understand why 
packages would require more than any other organization scheme.



I've considered installing the thing to the PYTHONPATH as most people
suggest, but this has two drawbacks:
   * Extremely hard to push thru my IT department. Possibly impossible.
   * Local checkouts of scripts use the main installation, rather than
the local, possibly revised package code. This necessitates the
boilerplate that installation to the PYTHONPATH was supposed to avoid.
   * We can work around the previous point by requiring a user-owned
dev installation of Python, but this raises the bar to entry past most
of my co-developers threshold. They are more comfortable with tcsh and
perl...


Are you sure that you are not referring to site-packages/ when you say 
"PYTHONPATH"?

PYTHONPATH an environment variable that the user can set. He can add whatever 
directories he wants to that environment variable. The user can make a directory 
for his checkouts:


  $ mkdir ~/LocalToolCheckouts
  $ cd ~/LocalToolCheckouts
  $ cvs ...

Now add that directory to the front of his PYTHONPATH:

  $ export PYTHONPATH=~/LocalToolCheckouts/:$PYTHONPATH

Now everything works fine. Packages in ~/LocalToolCheckouts will get picked up 
before anything else. This is a simple no-installation way to use the normal 
Python package mechanism that works well if you don't actually need to build 
anything.



I think the issue here is that the current python-package system works
well enough for the core python devs but leaves normal python
developers without much options beyond "all scripts in one directory"
or "tons of boilerplate everywhere".


The "vast majority" I am talking about *are* the normal Python developers.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

--
http://mail.python.org/mailman/listinfo/python-list

Re: Object Relational Mappers are evil (a meditation)

[Diez B. Roggisch]

Aaron Watters schrieb:

This is a bit off topic except that many Python
programmers seem to be allergic to typing SQL.

RESOLVED:  Using ORMs leads lazy programmers
to make bad database designs.  It's better to
carefully design your database with no invisible
means of support and there is no reason to not
use SQL directly for this purpose.

FOR EXAMPLE:  Consider blogging.  The most
successful blog software is WORDPRESS.  Here
is the WordPress data model:

http://blog.kapish.co.in/wp-content/uploads/2009/03/wp_2.7.png

Beautiful, isn't it?  It was designed by people who
thought about what they were doing and did it carefully.

Now let's look at the Sakai Blogger tool data model
(as reverse engineered by someone who had to
fix a bug -- there actually was no data design created
by the implementers):

confluence.sakaiproject.org/download/attachments/17072138/
entitymodel.pdf

How did the above happen?  I suspect someone opened
up Eclipse and started typing, relying on the Hibernate
ORM to handle all the database stuff automagically.  The
result is a massive headache for people like me.  Another
one.  I routinely open up the mysql prompt and start typing
"show tables", "describe table blah", "select * from blah limit 10"...
trying to figure out WTF Hibernate did with my data.


I think your example is nonsense. Just comparing the two models based on 
"they both are for blogging" is like comparing a cessna to a 747 - yes, 
both are flying, but that's pretty much what they have in common.


It is pretty clear that sakai's data-model caters to a very 
sophisticated *user*-model, with roles and permissions, and whatnot. 
Plus other features. So it appears to be more in the CMS-league.


Now it's obviously easy to argue that this isn't needed for a simple 
blog. Nonetheless, it doesn't make a point about ORM. With any ORM I 
know (both in Java and Python) you could design the simple and straight 
model WP uses.


And I've seen my fair share of convoluted, trashy pure-SQL-based DBs as 
well.


So I think to make your point you need some more convincing arguments.

Diez
--
http://mail.python.org/mailman/listinfo/python-list

Re: Q: sort's key and cmp parameters

[Raymond Hettinger]

> > So, it looks like the relevant comparison values can be stored in
> > nested lists:
>
> >list_of_lists = \
> >  [[1, [3, [], []],
> >   [7, [5, [], []], []]],
> >   [19, [23, [], []],
> >   []],
> >  ]
>
> Now you've copied all the data out of the original tree, which is even
> worse than putting a class wrapper around it.  The comparison remains
> (asymptotically) as expensive as before, too.

FWIW, you could also just flatten it to:  [(1,3,7,5), (19,23)].

The point is that the tree comparisons you presented have a
predetermined
order of values to compare, so they either be recast a flattened list
of comparison values or the tree itself can be cast in a list of lists
form.
Either way, the O(n log n) step of doing the actual comparisons runs
much
faster than if you coded a recursive cmp function written in pure
python.


>  Say you want to change the numeric comparisons so that
> even numbers always sort before odd numbers, ie.
> -4 < -2 < 0 < 2 < 4 < ... < -999 < ... -1 < 1 < 3 ...

This is too easy:

>>> s = [-2, 4, 2, -1, -3, 1, -4, 0, 3]
>>> s.sort()
>>> s.sort(key=lambda x: x%2)
>>> s
[-4, -2, 0, 2, 4, -3, -1, 1, 3]

The use cases you've presented so far aren't convincing,
but I'm sure that if you keep making-up random, weird use cases,
there will be one where a cmp function is a better approach.


> I think there are reasonable orderings on the trees themselves that
> can't possibly be embedded in the standard python comparison order.
> I might be able to construct an example using ordinal diagrams from
> proof theory.

As long as the values of a tree get compared in a predetermined order,
there will always be a flattened list equivalent that works faster
using a key function.  If you're going to concoct something isn't
easily transformable to a key function, I think your best bet is to
create a comparison where the trees have an interaction other than
comparing values at identical node positions; otherwise, the tree
shape hasn't made the problem any more difficult than sorting a list
of successive values.  The concocted comparison function needs to
exploit some aspect of the tree shape than can't be precomputed
in a key function (perhaps involving some complex interaction between
the two compared items like a tree merge or somesuch).

Instead of trying to create a hard comparison from first principles,
there may already be some research on the subject in the SQL world.
It seems to me that SQL's ORDER BY clauses are essentially just
key functions, so maybe there are known cases where a query cannot
be sorted with just the tools provided by SQL.


Raymond

P.S.  I accept than you hate sorting in Py3.x.  There's no need to
convince me of that.  I was just curious about your one real-world
use case and whether I could find a straight-forward key function
that would get the job done.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PIL : How to write array to image ???

[Mart.]

On Oct 5, 5:14 pm, Martin  wrote:
> On Oct 4, 10:16 pm, "Mart."  wrote:
>
>
>
>
>
> > On Oct 4, 9:47 am, Martin  wrote:
>
> > > On Oct 3, 11:56 pm, Peter Otten <__pete...@web.de> wrote:
>
> > > > Martin wrote:
> > > > > Dear group
>
> > > > > I'm trying to use PIL to write an array (a NumPy array to be exact) to
> > > > > an image.
> > > > > Peace of cake, but it comes out looking strange.
>
> > > > > I use the below mini code, that I wrote for the purpose. The print of
> > > > > a looks like expected:
>
> > > > > [[ 200.  200.  200. ...,    0.    0.    0.]
> > > > >  [ 200.  200.  200. ...,    0.    0.    0.]
> > > > >  [ 200.  200.  200. ...,    0.    0.    0.]
> > > > >  ...,
> > > > >  [   0.    0.    0. ...,  200.  200.  200.]
> > > > >  [   0.    0.    0. ...,  200.  200.  200.]
> > > > >  [   0.    0.    0. ...,  200.  200.  200.]]
>
> > > > > But the image looks nothing like that.
>
> > > > > Please see the images on:
> > > > >http://hvidberg.net/Martin/temp/quat_col.png
> > > > >http://hvidberg.net/Martin/temp/quat_bw.png
>
> > > > > or run the code to see them locally.
>
> > > > > Please – what do I do wrong in the PIL part ???
>
> > > > > :-? Martin
>
> > > > > import numpy as np
> > > > > from PIL import Image
> > > > > from PIL import ImageOps
>
> > > > > maxcol = 100
> > > > > maxrow = 100
>
> > > > > a = np.zeros((maxcol,maxrow),float)
>
> > > > > for i in range(maxcol):
> > > > >     for j in range(maxrow):
> > > > >         if (i<(maxcol/2) and j<(maxrow/2)) or (i>=(maxcol/2) and j>=
> > > > > (maxrow/2)):
> > > > >             a[i,j] = 200
> > > > >         else:
> > > > >             a[i,j] = 0
>
> > > > > print a
>
> > > > > pilImage = Image.fromarray(a,'RGB')
> > > > > pilImage.save('quat_col.png')
> > > > > pilImage = ImageOps.grayscale(pilImage)
> > > > > pilImage.save('quat_bw.png')
>
> > > > The PIL seems to copy the array contents directly from memory without 
> > > > any
> > > > conversions or sanity check. In your example The float values determine 
> > > > the
> > > > gray value of 8 consecutive pixels.
>
> > > > If you want a[i,j] to become the color of the pixel (i, j) you have to 
> > > > use
> > > > an array with a memory layout that is compatible to the Image.
> > > > Here are a few examples:
>
> > > > >>> import numpy
> > > > >>> from PIL import Image
> > > > >>> a = numpy.zeros((100, 100), numpy.uint8)
> > > > >>> a[:50, :50] = a[50:, 50:] = 255
> > > > >>> Image.fromarray(a).save("tmp1.png")
> > > > >>> b = numpy.zeros((100, 100, 3), numpy.uint8)
> > > > >>> b[:50, :50, :] = b[50:, 50:, :] = [255, 0, 0]
> > > > >>> Image.fromarray(b).save("tmp2.png")
> > > > >>> c = numpy.zeros((100, 100), numpy.uint32)
> > > > >>> c[:50, :50] = c[50:, 50:] = 0xff808000
> > > > >>> Image.fromarray(c, "RGBA").save("tmp3.png")
>
> > > > Peter
>
> > > Thanks All - That helped a lot...
>
> > > The working code ended with:
>
> > >         imga = np.zeros((imgL.shape[1],imgL.shape[0]),np.uint8)
> > >         for ro in range(imgL.shape[1]):
> > >             for co in range(imgL.shape[0]):
> > >                 imga[ro,co] = imgL[ro,co]
> > >         Image.fromarray(imga).save('_a'+str(lev)+'.png')
>
> > Without knowing how big your image is (can't remember if you said!).
> > Perhaps rather than looping in the way you might in C for example, the
> > numpy where might be quicker if you have a big image. Just a
> > thought...
>
> And a good thought too... I use to teach ansi C at Univ. many ears
> ago, and it's still one of my favorits. But I prefere Python for this
> type of work. Python have a much faster develop time, is much easier
> to write (fewer linees, simpler syntax) and is much more 'hot' in
> these days. Finally my favorit GIS and RS applications support Python
> out-of-the-box.
>
> :-) Martin

I wasn't advocating doing it in c just that you could speed up your
program considerably by not looping and using some of the numpy array
options.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: organizing your scripts, with plenty of re-use

[Margie]

On Oct 5, 12:34 pm, Robert Kern  wrote:
> On 2009-10-05 13:48 PM, Buck wrote:
>
>
>
> > On Oct 5, 11:29 am, Robert Kern  wrote:
> >> On 2009-10-05 12:42 PM, Buck wrote:
>
>  With the package layout, you would just do:
>
>       from parrot.sleeping import sleeping_in_a_bed
>       from parrot.feeding.eating import eat_cracker
>
>  This is really much more straightforward than you are making it out to 
>  be.
>
> >>> As in the OP, I need things to "Just Work" without installation
> >>> requirements.
> >>> The reason for this is that I'm in a large corporate environment
> >>> servicing many groups with their own custom environments.
>
> >> The more ad hoc hacks you use rather than the standard approaches, the 
> >> harder it
> >> is going to be for you to support those custom environments.
>
> > I too would prefer a standard approach but there doesn't seem to be an
> > acceptable one.
>
> >> I do believe that you and Stef are exceptions. The vast majority of Python 
> >> users
> >> seem to be able to grasp packages well enough.
>
> > You're failing to differentiate between python programmer and a
> > system's users. I understand packages well enough, but I need to
> > reduce the users' requirements down to simply running a command. I
> > don't see a way to do that as of now without a large amount of
> > boilerplate code in every script.
>
> I would like to see an example of such boilerplate. I do not understand why
> packages would require more than any other organization scheme.
>
> > I've considered installing the thing to the PYTHONPATH as most people
> > suggest, but this has two drawbacks:
> >    * Extremely hard to push thru my IT department. Possibly impossible.
> >    * Local checkouts of scripts use the main installation, rather than
> > the local, possibly revised package code. This necessitates the
> > boilerplate that installation to the PYTHONPATH was supposed to avoid.
> >    * We can work around the previous point by requiring a user-owned
> > dev installation of Python, but this raises the bar to entry past most
> > of my co-developers threshold. They are more comfortable with tcsh and
> > perl...
>
> Are you sure that you are not referring to site-packages/ when you say 
> "PYTHONPATH"?
>
> PYTHONPATH an environment variable that the user can set. He can add whatever
> directories he wants to that environment variable. The user can make a 
> directory
> for his checkouts:
>
>    $ mkdir ~/LocalToolCheckouts
>    $ cd ~/LocalToolCheckouts
>    $ cvs ...
>
> Now add that directory to the front of his PYTHONPATH:
>
>    $ export PYTHONPATH=~/LocalToolCheckouts/:$PYTHONPATH
>
> Now everything works fine. Packages in ~/LocalToolCheckouts will get picked up
> before anything else. This is a simple no-installation way to use the normal
> Python package mechanism that works well if you don't actually need to build
> anything.
>
> > I think the issue here is that the current python-package system works
> > well enough for the core python devs but leaves normal python
> > developers without much options beyond "all scripts in one directory"
> > or "tons of boilerplate everywhere".
>
> The "vast majority" I am talking about *are* the normal Python developers.
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless enigma
>   that is made terrible by our own mad attempt to interpret it as though it 
> had
>   an underlying truth."
>    -- Umberto Eco

I think that Buck's issue with your above example is that he doesn't
want his users to have to type
   $ export PYTHONPATH=~/LocalToolCheckouts/:$PYTHONPATH

For example, say that the developer does this:
   $ mkdir ~/LocalToolCheckouts
   $ cd ~/LocalToolCheckouts
   $ cvs ...

Which results in the following directory structure:

~/LocalToolCheckouts
+-- scripts/
+-- myscript.py
+-- parrot/
+-- __init__.py
+-- feeding/
+-- __init__.py
+-- eating.py
+-- drinking.py


I think he is looking for a way for users to be able to use scripts/
myscript.py (which imports parrot) without having to change their
PYTHON path with something like this:

   $ export PYTHONPATH=~/LocalToolCheckouts/:$PYTHONPATH

I'm guessing that Buck has users that are running out of a cvs
repository.  Although many would say those users are now "developers",
they really are not.  They probably don't even know they are running
from a cvs repository.  They in fact may think of it as their own
personal installation, and all they know is that they have scripts
directory and that that scripts directory has some scripts they want
to run.

As Buck said, it can often be very difficult to get things properly
and quickly installed in a large corporate environment, and providing
the user with a way to check out a cvs repository can be very quick
and easy.  The problem is that once the user has access to that cvs
repository, it is difficult to tell them "hey, every time you r

Re: organizing your scripts, with plenty of re-use

[Buck]

Thanks. I think we're getting closer to the core of this.

To restate my problem more simply:

My core goal is to have my scripts in some sort of organization better
than a single directory, and still have plenty of re-use between them.
The only way I can see to implement this is to have 10+ lines of
unintelligible hard-coded boilerplate in every runnable script.
That doesn't seem reasonable or pythonic.


On Oct 5, 12:34 pm, Robert Kern  wrote:
> I would like to see an example of such boilerplate. I do not understand why
> packages would require more than any other organization scheme.

This example is from the 2007 post I referenced in my OP. I'm pretty
sure he meant 'dirname' rather than 'basename', and even then it
doesn't quite work.

http://mail.python.org/pipermail/python-3000/2007-April/006814.html
  import os,sys
  sys.path.insert(1, os.path.basename(os.path.basename(__file__)))


This is from a co-worker trying to address this topic:
  import os, sys
  binpath = binpath or os.path.dirname(os.path.realpath(sys.argv[0]))
  libpath = os.path.join(binpath, 'lib')

  verinfo = sys.version_info
  pythonver = 'python%d.%d' % (verinfo[0], verinfo[1])
  sys.path.append(os.path.join(libpath, pythonver, 'site-packages'))
  sys.path.append(libpath)


This is my personal code:

  from sys import path
  from os.path import abspath, islink, realpath, dirname, normpath,
join
  f = __file__
  #continue working even if the script is symlinked and then compiled
  if f.endswith(".pyc"): f = f[:-1]
  if islink(f): f = realpath(f)
  here = abspath(dirname(f))
  libpath = join(here, "..", "lib")
  libpath = normpath(libpath)
  path.insert(1, libpath)


>$ export PYTHONPATH=~/LocalToolCheckouts/:$PYTHONPATH
> This is a simple no-installation way to use the normal
> Python package mechanism that works well if you don't actually need to build
> anything.

This seems simple to you, but my users are electrical engineers and
know just enough UNIX commands to get by. Most are afraid of Python.
Half of them will assume the script is borked when they see a
"ImportError: No module named foo". Another 20% will then read the
README and
set their environment wrong (setenv PYTHONPATH foo). The rest will get
it to work after half an hour but never use it again because it was
too complicated. I could fix the error message to tell them exactly
what to do, but at that point I might as well re-write the above
boilerplate code.

I'm overstating my case here for emphasis, but it's essentially true.
--Buck
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: organizing your scripts, with plenty of re-use

[Rami Chowdhury]

On Mon, 05 Oct 2009 13:46:09 -0700, Buck  wrote:


Thanks. I think we're getting closer to the core of this.

To restate my problem more simply:

My core goal is to have my scripts in some sort of organization better
than a single directory, and still have plenty of re-use between them.
The only way I can see to implement this is to have 10+ lines of
unintelligible hard-coded boilerplate in every runnable script.
That doesn't seem reasonable or pythonic.



Perhaps I've simply not followed this thread closely enough but could you  
let us know a little bit more about how you intend / expect the scripts to  
be used?


If there's a standard directory you expect them to be dropped into by your  
users (e.g. $HOME/scripts) then surely you could do something like:


import mypathmunge

at the top of every script, and then have a mypathmunge.py in  
site-packages that goes:


# mypathmunge.py
import sys, os
sys.path.insert(0, os.path.join(os.getenv('HOME'), 'scripts'))

?



On Oct 5, 12:34 pm, Robert Kern  wrote:
I would like to see an example of such boilerplate. I do not understand  
why

packages would require more than any other organization scheme.


This example is from the 2007 post I referenced in my OP. I'm pretty
sure he meant 'dirname' rather than 'basename', and even then it
doesn't quite work.

http://mail.python.org/pipermail/python-3000/2007-April/006814.html
  import os,sys
  sys.path.insert(1, os.path.basename(os.path.basename(__file__)))


This is from a co-worker trying to address this topic:
  import os, sys
  binpath = binpath or os.path.dirname(os.path.realpath(sys.argv[0]))
  libpath = os.path.join(binpath, 'lib')

  verinfo = sys.version_info
  pythonver = 'python%d.%d' % (verinfo[0], verinfo[1])
  sys.path.append(os.path.join(libpath, pythonver, 'site-packages'))
  sys.path.append(libpath)


This is my personal code:

  from sys import path
  from os.path import abspath, islink, realpath, dirname, normpath,
join
  f = __file__
  #continue working even if the script is symlinked and then compiled
  if f.endswith(".pyc"): f = f[:-1]
  if islink(f): f = realpath(f)
  here = abspath(dirname(f))
  libpath = join(here, "..", "lib")
  libpath = normpath(libpath)
  path.insert(1, libpath)



   $ export PYTHONPATH=~/LocalToolCheckouts/:$PYTHONPATH
This is a simple no-installation way to use the normal
Python package mechanism that works well if you don't actually need to  
build

anything.


This seems simple to you, but my users are electrical engineers and
know just enough UNIX commands to get by. Most are afraid of Python.
Half of them will assume the script is borked when they see a
"ImportError: No module named foo". Another 20% will then read the
README and
set their environment wrong (setenv PYTHONPATH foo). The rest will get
it to work after half an hour but never use it again because it was
too complicated. I could fix the error message to tell them exactly
what to do, but at that point I might as well re-write the above
boilerplate code.

I'm overstating my case here for emphasis, but it's essentially true.
--Buck




--
Rami Chowdhury
"Never attribute to malice that which can be attributed to stupidity" --  
Hanlon's Razor

408-597-7068 (US) / 07875-841-046 (UK) / 0189-245544 (BD)
--
http://mail.python.org/mailman/listinfo/python-list

Re: organizing your scripts, with plenty of re-use

[Robert Kern]

On 2009-10-05 15:33 PM, Margie wrote:


I think he is looking for a way for users to be able to use scripts/
myscript.py (which imports parrot) without having to change their
PYTHON path with something like this:

$ export PYTHONPATH=~/LocalToolCheckouts/:$PYTHONPATH

I'm guessing that Buck has users that are running out of a cvs
repository.  Although many would say those users are now "developers",
they really are not.


Since he called them "co-developers", I was operating under the assumption that 
they were, in fact, developers.



They probably don't even know they are running
from a cvs repository.  They in fact may think of it as their own
personal installation, and all they know is that they have scripts
directory and that that scripts directory has some scripts they want
to run.

As Buck said, it can often be very difficult to get things properly
and quickly installed in a large corporate environment, and providing
the user with a way to check out a cvs repository can be very quick
and easy.  The problem is that once the user has access to that cvs
repository, it is difficult to tell them "hey, every time you run from
it, you need to execute this special command to set up your PYTHONPATH
environment variable."


No, you tell them: "Add the line

  source ~/LocalToolCheckouts/tool-env.csh

to your ~/.tcshrc and start a new shell."

They each do this once, and it works for all of your scripts with no 
boilerplate. You can add features to this shell script, too, like adding the 
script directory to their $PATH.


Or you can write a simple "installation" script, which doesn't really install 
anything, just adds to their ~/.tcshrc files like so.


Or if this still is beyond your users, put a module into your scripts directory 
that does the prepends ~/LocalToolCheckouts/ to sys.path . The only 
"boilerplate" you need is one line in each script that imports this module.


But this is all stuff you have to do whether you organize your code into 
packages or not. The organization of the code really has no effect on the 
installation problems the OP faces.


--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

--
http://mail.python.org/mailman/listinfo/python-list

Tabular data package

[Elaine Angelino]

Hi there,

We are writing to announce the release of "Tabular", a package of Python
modules for working with tabular data.

Tabular is a package of Python modules for working with tabular data. Its
main object is the tabarray class, a data structure for holding and
manipulating tabular data. By putting data into a tabarray object, you’ll
get a representation of the data that is more flexible and powerful than a
native Python representation. More specifically, tabarray provides:

-- ultra-fast filtering, selection, and numerical analysis methods, using
convenient Matlab-style matrix operation syntax
-- spreadsheet-style operations, including row & column operations, 'sort',
'replace', 'aggregate', 'pivot', and 'join'
-- flexible load and save methods for a variety of file formats, including
delimited text (CSV), binary, and HTML
-- helpful inference algorithms for determining formatting parameters and
data types of input files
-- support for hierarchical groupings of columns, both as data structures
and file formats

You can download Tabular from PyPI
(http://pypi.python.org/pypi/tabular/)
or alternatively clone our hg repository from bitbucket (
http://bitbucket.org/elaine/tabular/ ).
We also have posted tutorial-style Sphinx documentation (
http://www.parsemydata.com/tabular/).

The tabarray object is based on the record
arrayobject
from the Numerical Python package (
NumPy ), and Tabular is built to interface well
with NumPy in general.  Our intended audience is two-fold: (1) Python users
who, though they may not be familiar with NumPy, are in need of a way to
work with tabular data, and (2) NumPy users who would like to do
spreadsheet-style operations on top of their more "numerical" work.

We hope that some of you find Tabular useful!

Best,

Elaine and Dan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: numpy f2py question

[George Trojan]

sturlamolden wrote:

On 2 Okt, 22:41, George Trojan  wrote:

I have a problem with numpy's vectorize class and f2py wrapped old
FORTRAN code. I found that the function _get_nargs() in
site-packages/numpy/lib/function_base.py tries to find the number of
arguments for a function from an error message generated when the
function is invoked with no arguments. However, with numpy 1.3.0 the
error message is different that the code expects. Is there any known
solution? I don';t know where the message is coming from, a cursory
check of numpy files did not yield any hits. A ctypes issue?
I did find an  unanswered seven weeks old related 
postinghttp://article.gmane.org/gmane.comp.python.numeric.general/32168/matc...
though I don't think this is gfortran related. Mine is 4.1.2.


The wrappers generated by f2py just calls PyArg_ParseTupleAndKeywords
and return NULL if it fails. The error message from
PyArg_ParseTupleAndKeywords does not fit the regex used by _get_nargs:

re.compile(r'.*? takes exactly (?P\d+) argument(s|) \((?
P\d+) given\)')

It should have said

re.compile(r'.*? takes (exactly|at least) (?P\d+) argument(s|)
\((?P\d+) given\)')

It is not just an f2py issue. It affects any function that uses
keyword arguments. I'll file this as a bug on NumPy trac.

S.M.






That's not enough. f2py generated library ftest.so providing wrapper 
around Fortran function


integer function f2(a, b)
integer, intent(in) :: a, b

run from

try:
   ftest.f2()
except Exception, e:
   print e

produces traceback
Required argument 'a' (pos 1) not found
so the proposed fix will miss this case.

I tracked the issue to changes (from 2.5.2 to 2.6.2) of Python/getargs.c 
code. Still not sure what the proper fix should be.


George
--
http://mail.python.org/mailman/listinfo/python-list

Re: Skeletal animation

[Carl Banks]

On Oct 5, 9:09 am, Manowar  wrote:
> On Oct 4, 8:38 pm, Carl Banks  wrote:
>
>
>
>
>
> > On Oct 4, 5:16 pm, Manowar  wrote:
>
> > > On Oct 4, 6:38 pm, TerryP  wrote:
>
> > > > On Oct 4, 10:05 pm, Manowar  wrote:
>
> > > > > I am new to pyton and have asked this question several times the
> > > > > answer is always not sure.
> > > > > Here is my question sekeltal animation ( bone animation) is it
> > > > > possible with python? What i want to develop is an aquarium in
> > > > > realtime, skeletal animation, all the movements will be from
> > > > > programming., no keyframes  let me know if it is possible with python
>
> > > > > Manowar
>
> > > > Depending on how much code you want to write, you'll probably want to
> > > > check out something like pygame or python-ogre
>
> > > Yeah, i know about those but i just really need to know if it is
> > > possible to do in python
> > > if it is what about some tutorials on bone animation?
>
> > It's possible.  I can't really recommend a tutorial because I don't
> > know where you are coming from, and it is a very complex subject.
> > What do know about 3D animation so far?  Have you done 3D animation in
> > other languages?  Have you done 3D mesh modeling with a tool like 3DS
> > Max or Blender?  What is your mathematical background (especially in
> > linear algrebra)?  Please help us help you by providing us details on
> > what you know already and what you want to do.  You won't get much
> > help by asking simple yes/no questions.
>
> > Carl Banks- Hide quoted text -
>
> > - Show quoted text -
>
> Let me tell you in more detail what i want maybe i am not explaining
> it correctly ( i will look at those links) I have been using Blender
> for years. I know animation but not an expert. If i make an aquarium i
> want the fish to move on their own with their own A.I. (Self-
> contained fish.) To make it clearer if i put a rock in it's path, the
> fish will move around the rock and if a predator comes into the scene
> the fish will scatter away at a high speed then return to normal when
> the predator goes. I know this can be done in C++ I won't go into why
> i am not using c++ just need to know if i can manipulate the bones in
> realtime with python. can't find any info on that but someone must
> know how to do it in python. Don't want keyframes just want true bone
> animation

Don't worry, I totally understand what you mean by "not using
keyframes".   Now that I'm a little better appraised of what you know
already now I can help better.  (See, if you were a newbie with no
programming experience who is asking "i wanna make fishies can mv
themslvs plz hlp" my advice would be a bit different)

There are two ways to do what you want.  The first way is to represent
bones as OpenGL transformations.  Basically a joint deformation you'd
represent as a translation + rotation.  To rotate a bone you'd simply
update the rotation of that joint.  Then, when the proper
transformation is in place, just draw the mesh attached to the bone.
This can be done with simply with Python an OpenGL.  It's conceptually
simple but the details can be nasty.  Also it doesn't look very good.

The second way is skinning.  You have a big mesh and each vertex on
the mesh in influenced by any nearby bones.  This is more difficult
because a calculation is required for each vertex, and you can't take
advantage of OpenGL calcualtions(**), but it looks much better.  Also,
there is no way to do skinning calculations fast enough in Python.
You might be able to manage it with numpy.  However, what I do is to
use a C++ library called cal3d.  Yes, I had to write a wrapper, but
once I did that I could call everything from Python.  cal3d provides
keyframe animation, but it allows you to manipulate bones by hand if
you want.  (Given a bone you can call bone.setRotation.)

There's a third way, hardward skinning, but I think that's a fairly
recent development and you'd have to use a vertex shader.  Pretty
tough, but probably also the future.

Maybe harder than the actual drawing functions is to export a Blender
model in a format you can parse.  (There's a Blender cal3d exporter
out there, so if you were to use cal3d you could use that.)

So, tutorials.  Well I assume you are aware the PyOpenGL is a
relatively thin wrapper around C OpenGL, so you should be able to
follow a C OpenGL tutorial in Python as well.  Google for "skeletal
animation in OpenGL", you should find plenty, as it's a common, basic
technique.  Skinning is a more advanced so there is probably not as
many tutorials on that.  For exporting models, I think I would suggest
the studying the cal3d exporter.  Even if you don't use cal3d, it
should help you learn what you need to do to export skeletal models.
If you want to try hardware skinning you might look at lighthouse
opengl tutorials to learn GLSL first, then look for something on
hardware skinning.  That's hard as hell, though.


Carl Banks
-- 
http://mail.python.org/mailman/listinfo/python-list

How to sort a list of strings on a substring

[Scott]

I create a list of logs called LogList. Here is a sample:

LogList =
["inbound tcp office 192.168.0.125 inside 10.1.0.91 88",
"inbound tcp office 192.168.0.220 inside 10.1.0.31 2967",
"inbound udp lab 172.24.0.110 inside 10.1.0.6 161",
"inbound udp office 192.168.0.220 inside 10.1.0.13 53"]

I want to sort the list on index 3 of each string - the first IP
Address.

I only need strings with similar, first IP's to be together. I don't
need all of the IP's to be in order. For example:
either:
SortedList =
["inbound udp lab 172.24.0.110 inside 10.1.0.6 161",
"inbound tcp office 192.168.0.220 inside 10.1.0.31 2967",
"inbound udp office 192.168.0.220 inside 10.1.0.13 53",
"inbound tcp office 192.168.0.125 inside 10.1.0.91 88"]
-or-
SortedList =
["inbound tcp office 192.168.0.220 inside 10.1.0.31 2967",
"inbound udp office 192.168.0.220 inside 10.1.0.13 53",
"inbound udp lab 172.24.0.110 inside 10.1.0.6 161",
"inbound tcp office 192.168.0.125 inside 10.1.0.91 88"]
-or-
etc.

would be fine.

I'm reading a lot on sort, sorted, cmp, etc. but I'm just not getting
how to use an element of a string as a "key" within a list of strings.
I'm using Python 2.6.2.

Thanks

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to sort a list of strings on a substring

[MRAB]

Scott wrote:

I create a list of logs called LogList. Here is a sample:

LogList =
["inbound tcp office 192.168.0.125 inside 10.1.0.91 88",
"inbound tcp office 192.168.0.220 inside 10.1.0.31 2967",
"inbound udp lab 172.24.0.110 inside 10.1.0.6 161",
"inbound udp office 192.168.0.220 inside 10.1.0.13 53"]

I want to sort the list on index 3 of each string - the first IP
Address.

I only need strings with similar, first IP's to be together. I don't
need all of the IP's to be in order. For example:
either:
SortedList =
["inbound udp lab 172.24.0.110 inside 10.1.0.6 161",
"inbound tcp office 192.168.0.220 inside 10.1.0.31 2967",
"inbound udp office 192.168.0.220 inside 10.1.0.13 53",
"inbound tcp office 192.168.0.125 inside 10.1.0.91 88"]
-or-
SortedList =
["inbound tcp office 192.168.0.220 inside 10.1.0.31 2967",
"inbound udp office 192.168.0.220 inside 10.1.0.13 53",
"inbound udp lab 172.24.0.110 inside 10.1.0.6 161",
"inbound tcp office 192.168.0.125 inside 10.1.0.91 88"]
-or-
etc.

would be fine.

I'm reading a lot on sort, sorted, cmp, etc. but I'm just not getting
how to use an element of a string as a "key" within a list of strings.
I'm using Python 2.6.2.


Forget about cmp, just use the 'key' argument of the list's 'sort'
method or the 'sorted' function (the latter is better if you want to
keep the original list). The 'key' argument expects a function (anything
callable, actually) that accepts a single argument (the item) and
returns a value to be used as the key, and the items will be sorted
according to that key. In this case you want the items sorted by the
fourth 'word', so split the item into words and return the one at index
3:

def key_word(item):
return item.split()[3]

SortedList = sorted(LogList, key=key_word)

If the function is short and simple enough, lambda is often used instead
of a named function:

SortedList = sorted(LogList, key=lambda item: item.split()[3])

--
http://mail.python.org/mailman/listinfo/python-list

Re: organizing your scripts, with plenty of re-use

[Carl Banks]

On Oct 5, 1:46 pm, Buck  wrote:
> Thanks. I think we're getting closer to the core of this.
>
> To restate my problem more simply:
>
> My core goal is to have my scripts in some sort of organization better
> than a single directory, and still have plenty of re-use between them.
> The only way I can see to implement this is to have 10+ lines of
> unintelligible hard-coded boilerplate in every runnable script.
> That doesn't seem reasonable or pythonic.

Well, ignoring Python packaging conventions isn't reasonable or
Pythonic either, but oh well.

Here's what you should do: Create ONE script that is a single entry
point to the everything.  The script would accept a subcommand, set up
the Python path, and invoke the appropirate Python module (NOT script)
corresponding to that subcommand, say by calling a main() function
you've defined within.

Users would invoke the appropirate tool using the appropriate
subcommand (similar to tools like Subversion and GIT).

If you really can't use subcommands, make symlinks to the ONE script,
and have the ONE script check what name it was invoked with.

Notice the key idea in all of this: ONE script.  When you design it
that a file can be used either as a script or as a module, you are
asking for trouble.


Carl Banks
-- 
http://mail.python.org/mailman/listinfo/python-list

mktime, how to handle dates before 01-01-1970 ?

[Stef Mientki]

hello,

I want to handle datetime vars in a general way, so I use the default 
time-format,

so I can use the standard cinversion procedures.
Time format is the time difference since 1-1-1970.

So how do I handle dates before 1-1-1970 ?

I'ld expect times before 1-1-1970, simply to become negative numbers
(I'm interested in the age of living people, so that would suffice).

Is there a general solution, (other library)
or would it be better to handle all dates in the Delphi format (number 
of days since  1-1-1900


thanks,
Stef Mientki
--
http://mail.python.org/mailman/listinfo/python-list

Re: Q: sort's key and cmp parameters

[Paul Rubin]

Raymond Hettinger  writes:
> FWIW, you could also just flatten it to:  [(1,3,7,5), (19,23)].
> 
> The point is that the tree comparisons you presented have a
> predetermined order of values to compare, so they either be recast a
> flattened list of comparison values or the tree itself can be cast
> in a list of lists form.  Either way, the O(n log n) step of doing
> the actual comparisons runs much faster than if you coded a
> recursive cmp function written in pure python.

Possibly so, but asymptotically it's the same, and anyway 

> >  Say you want to change the numeric comparisons so that
> > even numbers always sort before odd numbers, ie.
> > -4 < -2 < 0 < 2 < 4 < ... < -999 < ... -1 < 1 < 3 ...
> 
> This is too easy:
> 
> >>> s = [-2, 4, 2, -1, -3, 1, -4, 0, 3]
> >>> s.sort()
> >>> s.sort(key=lambda x: x%2)
> >>> s
> [-4, -2, 0, 2, 4, -3, -1, 1, 3]

I don't think so:

>>> s = [0, 2, -2, 3, 1, -1, 4, -4, -3]
>>> s.sort(key=lambda x:x%2)
>>> s
[0, 2, -2, 4, -4, 3, 1, -1, -3]

s.sort(key=lambda x:(x%2,x)) might work but why are you so opposed
to being able to write one's natural thought pattern if that pattern
happens to be a comparison function?  Who wants to concoct these
ad-hoc encodings for every ordering?

> As long as the values of a tree get compared in a predetermined order,
> there will always be a flattened list equivalent that works faster
> using a key function.  If you're going to concoct something isn't
> easily transformable to a key function, I think your best bet is to
> create a comparison where the trees have an interaction other than
> comparing values at identical node positions; 

How about this: gen_e, gen_pi, gen_sqrt2, etc. all generate
representations of various real numbers as possibly infinite sequences
of digits:

   def gen_pi():
  yield 3
  yield 1
  yield 4
  # ... compute infinite stream one digit at a time

Comparison is obvious if (hmmm) you can guarantee that you're
sorting unique elements and that the sort function won't compare an
element with itself (otherwise I guess you could cut off comparison at
a million digits or whatever):

   def cmp(gen1, gen2):
 for d,e in izip(gen1(), gen2()):
c = cmp(d,e)
if c: return c

I don't see any clean way to handle that with key=, though of course
maybe I'm missing something.  

> P.S.  I accept than you hate sorting in Py3.x.  There's no need to
> convince me of that.  I was just curious about your one real-world
> use case and whether I could find a straight-forward key function
> that would get the job done.

I don't hate sorting in py3.x.  I understand the concept that py3.x
introduces incompatibilities to make things better.  What I don't like
is the approach of breaking stuff gratutiously with no benefit.  If
the positional arg for cmp is a kludge, why not switch it to a keyword
arg instead of eliminating it?  

I don't subscribe to the cult of the real-world use case.  I'm not
smart enough to anticipate every way anyone might ever want to use
code that I write.  So I try to identify the obvious "basis vectors"
of sensible functionality, implement those, and make sure that the
entire space of combinations works the way it should, without worrying
about whether anyone will care about any specific combination.  There
is no real-world use case for the constant assignment

 a = 0x36b20b32c927eaf68bb40d87d251049db98da0c03a0e8c791f04ee486ec2f7ee

which I'm sure nobody has ever seen before (I just made it from
os.urandom).  But if Python somehow failed to parse that number (and
only that number), that would be considered an unacceptable bug,
because a straightforward combination of primitives had failed to work
as it should.

Any book on sorting spends most of its time on comparison sorting,
which is the primitive operation.  key= is very useful but it is
a derived operation, not the other way around.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: mktime, how to handle dates before 01-01-1970 ?

[Gabriel Genellina]

En Mon, 05 Oct 2009 20:54:18 -0300, Stef Mientki   
escribió:


I want to handle datetime vars in a general way, so I use the default  
time-format,

so I can use the standard cinversion procedures.
Time format is the time difference since 1-1-1970.

So how do I handle dates before 1-1-1970 ?


Use the datetime module and manage the dates as actual datetime objects,  
not as formatted strings.


py> import datetime
py> birthdate = datetime.datetime(1914, 7, 23)
py> birthdate
datetime.datetime(1914, 7, 23, 0, 0)
py> now = datetime.datetime.now()
py> now - birthdate
datetime.timedelta(34773, 77146, 765000)
py> _.days // 365
95


py> datetime.datetime(1931, 8, 23)
datetime.datetime(1931, 8, 23, 0, 0)
py> datetime.datetime(1851, 6, 12)
datetime.datetime(1851, 6, 12, 0, 0)
py> datetime.datetime(1492, 10, 12)
datetime.datetime(1492, 10, 12, 0, 0)
py> a = datetime.datetime(1492, 10, 12)
py> b = datetime.datetime(2009, 10, 5)
py> (b - a) // 365
datetime.timedelta(517, 27932, 54794)
py> b = datetime.datetime.now()
py> _.days
34773
py> _.days // 365
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: 'int' object has no attribute 'days'
py> now - birthdate
datetime.timedelta(34773, 77146, 765000)


I'ld expect times before 1-1-1970, simply to become negative numbers
(I'm interested in the age of living people, so that would suffice).

Is there a general solution, (other library)
or would it be better to handle all dates in the Delphi format (number  
of days since  1-1-1900


py> birthdate.toordinal()
698912

(number of days since 1 AD, approximately)

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: mktime, how to handle dates before 01-01-1970 ?

[Stephen Hansen]

On Mon, Oct 5, 2009 at 4:54 PM, Stef Mientki  wrote:

> hello,
>
> I want to handle datetime vars in a general way, so I use the default
> time-format,
> so I can use the standard cinversion procedures.
>

Personally, I love mx.DateTime; its the best date/time library around. But,
Python's built in datetime isn't too bad and is similar and built-in if you
don't want to use a third-party library.

But:

>>> birthday = mx.DateTime.Date(1960,3,3)
>>> birthday

>>> age = mx.DateTime.now() - birthday
>>> print "Age in days", age.days
18113.722499758693
>>> print "Age in years", age.days / 365
49.626636985640253

I really can't quite fathom why you'd want to use something so low-level as
time.mktime... or just about anything in the time module :) If you want to
handle dates in a general way, I'd use one of the general-purpose date/time
handling libraries. They're far more capable and easier to use.

HTH,

--S
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Q: sort's key and cmp parameters

[Raymond Hettinger]

> > >  Say you want to change the numeric comparisons so that
> > > even numbers always sort before odd numbers, ie.
> > >     -4 < -2 < 0 < 2 < 4 < ... < -999 < ... -1 < 1 < 3 ...
>
> > This is too easy:
>
> >     >>> s = [-2, 4, 2, -1, -3, 1, -4, 0, 3]
> >     >>> s.sort()
> >     >>> s.sort(key=lambda x: x%2)
> >     >>> s
> >     [-4, -2, 0, 2, 4, -3, -1, 1, 3]
>
> I don't think so:
>
>     >>> s = [0, 2, -2, 3, 1, -1, 4, -4, -3]
>     >>> s.sort(key=lambda x:x%2)
>     >>> s
>     [0, 2, -2, 4, -4, 3, 1, -1, -3]

There were two sorts in my post and only one in yours.
That's why you didn't get the same answer.


> s.sort(key=lambda x:(x%2,x)) might work but why are you so opposed
> to being able to write one's natural thought pattern if that pattern
> happens to be a comparison function?  Who wants to concoct these
> ad-hoc encodings for every ordering?

Your (x%2, x) idea seems like a good way to do it in a single pass.
Also, it seems to be a natural translation of the problem statement,
"even numbers always sort before odd numbers", into a primary key
and secondary sort key.


> How about this: gen_e, gen_pi, gen_sqrt2, etc. all generate
> representations of various real numbers as possibly infinite sequences
> of digits:
>
>    def gen_pi():
>       yield 3
>       yield 1
>       yield 4
>       # ... compute infinite stream one digit at a time
>
> Comparison is obvious if (hmmm) you can guarantee that you're
> sorting unique elements and that the sort function won't compare an
> element with itself (otherwise I guess you could cut off comparison at
> a million digits or whatever):
>
>    def cmp(gen1, gen2):
>      for d,e in izip(gen1(), gen2()):
>         c = cmp(d,e)
>         if c: return c
>
> I don't see any clean way to handle that with key=, though of course
> maybe I'm missing something.  

Right.  I would be forced to use a Cmp2Key wrapper for this one.

If sorting lazily evaluated infinite sequences is a common use
case for you, then you're going to love Haskell ;-)


> I don't hate sorting in py3.x.

That's good to hear.


>  I understand the concept that py3.x
> introduces incompatibilities to make things better.  What I don't like
> is the approach of breaking stuff gratutiously with no benefit.  If
> the positional arg for cmp is a kludge, why not switch it to a keyword
> arg instead of eliminating it?

When Guido made the final call, I'm sure he was balancing
a number of goals including language simplification and
one-way-to-do-it.  I'm also sure that sorting lazily
evaluated infinite sequences was not on his radar screen :-)


>  key= is very useful but it is
> a derived operation, not the other way around.

As you showed with infinite sequence example, it may well
be the case that cmp is more general and can more readily
handle an exotic case.

Psychologically, the thing that I find to be interesting is
that beginners and intermediate users seem to take to key
functions more readily than old timers.  The key function seems
to be an easy thing to teach (perhaps because that's the
way Excel sorts and the way SQL sorts, or perhaps it is because
problem statements seem to arise in the form of "sort by this,
then by that" instead of "here's how two objects should be
compared").

In contrast, some people who have have had deep experience with
cmp functions may tend to first think of cmp solutions and then
have a harder time seeing solutions with key functions.  If you
grew-up on C's qsort() like I did, then a key function may not
be the first thing that pops into your head.

One other interesting data point:  Python uses key functions
in min(), max(), heapq.nsmallest(), heapq.nlargest, and
itertools.groupby().  Those functions never supported a
cmp function argument, nor has one ever been requested.


Raymond
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: conflict between various library to process EXIF data

[Gabriel Genellina]

En Mon, 05 Oct 2009 09:27:34 -0300, Bram Mertens
 escribió:


In short: after adding a comment to the EXIF header using the
setComment(...) and writeMetadata(...) methods of pyeviv2
(http://tilloy.net/dev/pyexiv2/doc/release-0.1.2/pyexiv2.htm) the EXIF
header is altered in such a way that it is no longer possible to read
the EXIF header with EXIF.py.
Furthermore  imghdr does not even recognize it as a JPEG anymore. [...]
maybe
pyexiv2 is doing something "wrong" that might cause other problems
later on.


It seems so - don't be so shy and report the problem to the author :)

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: Conversion of npyscreen module to python 3 -- help!

[Gabriel Genellina]

En Mon, 05 Oct 2009 06:54:06 -0300, Nicholas 
escribió:

[...] I am clearly committing some horrible sin that works well in  
python 2

but which doesn't in python 3.  Despite reading the Release notes and
PEPs over and over, though, I can't see what is wrong.

If anyone could spare the time to have a quick look at the code at

http://code.google.com/p/npyscreen/

over and provide enlightenment, I would be extremely grateful.


The package is too large for just a "quick look". What's precisely your
problem? Try to isolate it; remove all unnecesary modules and directories,
try to keep the minimum code that still reproduces the problem.
You may solve it yourself in the process.

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: a questions about thread-safety of boolean variables

[Gabriel Genellina]

En Wed, 30 Sep 2009 06:14:50 -0300, Charlie Dickens
 escribió:


What about dictionaries? Reading values, adding new ones and the most
important: changing an existing value - can it corrupt the state of the
dictionary or that it is guaranteed that if I try to read the value of  
this

key, I can only get the old one or the new one, but not something weird
instead (because a different thread changed the value while this thread  
was

trying to read it).


See http://effbot.org/zone/thread-synchronization.htm

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: SQLite or files?

[AggieDan04]

On Sep 17, 9:10 am, J Kenneth King  wrote:
> ici  writes:
> > I likeshelvefor saving small amounts of data, user preferences,
> > recent files etc.
> >http://docs.python.org/library/shelve.html
>
> I like it too, but I hear the great powers that be are going to
> deprecate it.

If you want the convenience of shelve without the limitations of dbm,
you can do:


"""Implementation of Python shelves using SQLite."""

from __future__ import division

import UserDict
import pickle
import sqlite3

def to_db_type(value):
"""
If value's type is supported natively in SQLite, return value.
Otherwise, return a pickled representation.
"""
if value is None or isinstance(value, (int, long, float,
basestring)):
return value
else:
return buffer(pickle.dumps(value))

def from_db_type(value):
"""
Converts a value from the database to a Python object.
"""
if isinstance(value, buffer):
return pickle.loads(value)
else:
return value

class SQLiteShelf(UserDict.DictMixin):
"""
Shelf implementation using an SQLite3 database.
"""
def __init__(self, filename):
self._database = sqlite3.connect(filename)
self._database.execute("CREATE TABLE IF NOT EXISTS Shelf "
   "(Key TEXT PRIMARY KEY NOT NULL, Value
BLOB)")
self._open = True
def __del__(self):
self.close()
def __getitem__(self, key):
row = self._database.execute("SELECT Value FROM Shelf WHERE
Key=?",
 [key]).fetchone()
if row:
return from_db_type(row[0])
else:
raise KeyError(key)
def __setitem__(self, key, value):
self._database.execute("INSERT OR REPLACE INTO Shelf VALUES
(?, ?)",
   [key, to_db_type(value)])
def __delitem__(self, key):
self._database.execute("DELETE FROM Shelf WHERE Key=?", [key])
def keys(self):
"""Return a list of keys in the shelf."""
return [row[0] for row in
self._database.execute("SELECT Key FROM Shelf")]
def close(self):
"""Commit changes and close the file."""
if self._database is not None:
self._database.commit()
self._database.close()
self._database = None
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: ARP code

[Gabriel Genellina]

En Tue, 29 Sep 2009 17:01:14 -0300, Nattapong Limprungpattanakit
 escribió:


I have a problem can not run.


Which problem?
If you get an exception, copy and paste it including the whole traceback.


How to use and detail ?



import py_net_libs


If your problem is related to a specific library, it's usually better to
directly ask the author.


class dissect:
def __init__(self,pkt,offset):
[...]


You posted only a class definition - how do you *use* it?

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: ARP code

[Gabriel Genellina]

En Tue, 29 Sep 2009 17:01:14 -0300, Nattapong Limprungpattanakit
 escribió:


I have a problem can not run.


Which problem?
If you get an exception, copy and paste it including the whole traceback.


How to use and detail ?



import py_net_libs


If your problem is related to a specific library, it's usually better to
directly ask the author.


class dissect:
def __init__(self,pkt,offset):
[...]


You posted only a class definition - how do you *use* it?

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: Q: sort's key and cmp parameters

[Paul Rubin]

Raymond Hettinger  writes:
> There were two sorts in my post and only one in yours.
> That's why you didn't get the same answer.

Whoops, missed that.

> When Guido made the final call, I'm sure he was balancing
> a number of goals including language simplification and
> one-way-to-do-it.  I'm also sure that sorting lazily
> evaluated infinite sequences was not on his radar screen :-)

I can't help wondering if there will be feature requests for a cmp
argument for the rest of eternity until it eventually makes its way
back in.

> Psychologically, the thing that I find to be interesting is
> that beginners and intermediate users seem to take to key
> functions more readily than old timers.

I think key is preferable in at least 90% of the cases.  I also think
a general purpose language should be able to handle 100% of the cases,
so if key is all you have, there can be a gap to fill.

> In contrast, some people who have have had deep experience with
> cmp functions may tend to first think of cmp solutions and then
> have a harder time seeing solutions with key functions.  If you
> grew-up on C's qsort() like I did, then a key function may not
> be the first thing that pops into your head.

That could be.  

> One other interesting data point:  Python uses key functions
> in min(), max(), heapq.nsmallest(), heapq.nlargest, and
> itertools.groupby().  Those functions never supported a
> cmp function argument, nor has one ever been requested.

Interesting.  Maybe someday...
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to sort a list of strings on a substring

[Steven D'Aprano]

On Mon, 05 Oct 2009 15:45:58 -0700, n00m wrote:

> Here you are:
> 
> LogList = [\
> "inbound tcp office 192.168.0.125 inside 10.1.0.91 88", "inbound tcp
> office 192.168.0.220 inside 10.1.0.31 2967", "inbound udp lab
> 172.24.0.110 inside 10.1.0.6 161", "inbound udp office 192.168.0.220
> inside 10.1.0.13 53"]
> 
> 
> LogList.sort(key=lambda x: x[x.index('1'):])


No, that's incorrect. Try it with this data and you will see it fails:


LogList = [
"inbound tcp office1 192.168.0.125 inside 10.1.0.91 88",
"inbound tcp office2 192.168.0.220 inside 10.1.0.31 2967",
"inbound udp lab1 172.24.0.110 inside 10.1.0.6 161",
"inbound udp office2 192.168.0.220 inside 10.1.0.13 53",
"inbound udp lab2 172.24.0.121 inside 10.1.0.6 161",
"inbound udp webby 220.96.0.2 inside 20.2.0.9 54",
]


Worse, if you delete the last item ("webby"), the code silently does the 
wrong thing. Code that crashes is bad, but code that silently does the 
wrong thing is a nightmare. Your test succeeded by accident -- it was a 
fluke of the data that you failed to see both failure modes.

The question asked was how to sort the list according to item 3 of the 
strings, *not* how to sort the list according to the first character '1'. 
The way to solve this correctly is by extracting item 3 and sorting on 
that, not by searching for the first character '1'. That is a hack[1] 
that just happened to work for the specific test data you tried it on.




[1] Hack in the bad sense, not in the good sense.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

'Once' properties.

[menomnon]

Does python have a ‘once’ (per class) feature?

‘Once’, as I’ve know it is in Eiffel.  May be in Java don’t.

The first time you instantiate a given class into an object it
constructs, say, a dictionary containing static information.  In my
case static is information that may change once a week at the most and
there’s no need to be refreshing this data during a single running of
the program (currently maybe 30 minutes).

So you instantiate the same class into a second object, but instead of
going to the databases again and recreating the same dictionary a
second time, you get a pointer or reference to the one already created
in the first object – copies into the second object that is.  And the
dictionary, no matter how many instances of the object you make, is
always the same one from the first object.

So, as we put it, once per class and not object.

Saves on both time and space.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: 'Once' properties.

[Chris Rebert]

On Mon, Oct 5, 2009 at 7:56 PM, menomnon  wrote:
> Does python have a ‘once’ (per class) feature?

In Python, `class` is an executable statement, so you can put whatever
code you want in the class body (along with your method definitions)
and it will be run exactly once, at the time the class is defined
(that is, when the `class` statement is encountered by the
interpreter). In this way, you could create class variables containing
such static information.

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to refer to class name and function name in a python program?

[Gabriel Genellina]

En Mon, 28 Sep 2009 22:54:55 -0300, Peng Yu  escribió:

On Sun, Sep 20, 2009 at 12:43 PM, Benjamin Kaplan
 wrote:

On Sun, Sep 20, 2009 at 12:43 PM, Peng Yu  wrote:

On Sun, Sep 20, 2009 at 11:32 AM, Vijayendra Bapte
 wrote:



class A:

   @echo
   def __repr__(self):
   pass


What does @echo mean?


It's a decorator, which wraps the function with another function


I looked at the table of content of python tutorial at
http://docs.python.org/tutorial/

But I don't see which section discuss this concept. If it is there,
would you please let me know which section I should read. Or this
concept is discussed somewhere else?


It isn't menctioned in the tutorial - look in the Glossary, or the  
original PEP [1]
There's a good introduction by Bruce Eckel at [2], and the decorator  
module by M. Simionato is a must [3]


[1] http://www.python.org/dev/peps/pep-0318/
[2] http://www.artima.com/weblogs/viewpost.jsp?thread=240808
[3] http://pypi.python.org/pypi/decorator

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: Object Relational Mappers are evil (a meditation)

[Steve Holden]

Diez B. Roggisch wrote:
> Aaron Watters schrieb:
>> This is a bit off topic except that many Python
>> programmers seem to be allergic to typing SQL.
>>
>> RESOLVED:  Using ORMs leads lazy programmers
>> to make bad database designs.  It's better to
>> carefully design your database with no invisible
>> means of support and there is no reason to not
>> use SQL directly for this purpose.
>>
>> FOR EXAMPLE:  Consider blogging.  The most
>> successful blog software is WORDPRESS.  Here
>> is the WordPress data model:
>>
>> http://blog.kapish.co.in/wp-content/uploads/2009/03/wp_2.7.png
>>
>> Beautiful, isn't it?  It was designed by people who
>> thought about what they were doing and did it carefully.
>>
>> Now let's look at the Sakai Blogger tool data model
>> (as reverse engineered by someone who had to
>> fix a bug -- there actually was no data design created
>> by the implementers):
>>
>> confluence.sakaiproject.org/download/attachments/17072138/
>> entitymodel.pdf
>>
>> How did the above happen?  I suspect someone opened
>> up Eclipse and started typing, relying on the Hibernate
>> ORM to handle all the database stuff automagically.  The
>> result is a massive headache for people like me.  Another
>> one.  I routinely open up the mysql prompt and start typing
>> "show tables", "describe table blah", "select * from blah limit 10"...
>> trying to figure out WTF Hibernate did with my data.
> 
> I think your example is nonsense. Just comparing the two models based on
> "they both are for blogging" is like comparing a cessna to a 747 - yes,
> both are flying, but that's pretty much what they have in common.
> 
> It is pretty clear that sakai's data-model caters to a very
> sophisticated *user*-model, with roles and permissions, and whatnot.
> Plus other features. So it appears to be more in the CMS-league.
> 
It's also pretty clear that the person who developed the sakai map had
nothing to guide them, making it difficult to divine any structural
regularity that may be present in the design. The entity-relationship
diagram doesn't even show the cardinality of the relationships, making
it much less useful than it otherwise might be.

As a result the whole thing looks like a dog's breakfast.

> Now it's obviously easy to argue that this isn't needed for a simple
> blog. Nonetheless, it doesn't make a point about ORM. With any ORM I
> know (both in Java and Python) you could design the simple and straight
> model WP uses.
> 
> And I've seen my fair share of convoluted, trashy pure-SQL-based DBs as
> well.
> 
Yup, me too. Many such designs make mistakes like using multiple columns
(or, even worse, comma-separated values) instead of many-to-many
relationships. Sad how such elegant tools are so badly abused so often,
isn't it?

> So I think to make your point you need some more convincing arguments.
> 
It seems to me that the biggest sin in databases is a failure to use
rigorous design techniques. If somebody doesn't understand relational
theory then they will probably not design database representations that
are easy to work with. If they do understand then the designs they
produce will operate just as easily with object relational mappers as
they do with direct SQL.

I suspect Aaron was really complaining about ignorance rather than
laziness (though the two do sometimes go together, and the former is
often but by no means invariably caused by the latter). But he'd have to
confirm or deny that.

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/
Watch PyCon on video now!  http://pycon.blip.tv/

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: organizing your scripts, with plenty of re-use

[Steven D'Aprano]

On Mon, 05 Oct 2009 16:46:12 -0700, Carl Banks wrote:

> Notice the key idea in all of this: ONE script.  When you design it that
> a file can be used either as a script or as a module, you are asking for
> trouble.

I agree with everything you said in your post *except* that final 
comment. The basic idea of modules usable as scripts is a fine, reliable 
one, provided all the modules each script calls are visible in the 
PYTHONPATH. (You still have problems with recursive imports, but that can 
happen with any module, not just scripts.)

The Original Poster is confusing installation difficulties with code 
organization -- his problem is that users have special requirements for 
installation, and he's trying to work around those requirements by 
organizing his code differently.

As far as I can see, whether or not he uses a package, he will still have 
the same problem with installation, namely, that his users aren't 
developers, plus for non-technical reasons he can't provide an installer 
and has to have users check code out of CVS. Using a package will solve 
his internal code organization problems, and a simple setup script that 
modifies the user's .tcshrc to include the appropriate PYTHONPATH will 
solve his other problem. The solution is to work with the language, 
instead of fighting the language.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Object Relational Mappers are evil (a meditation)

[Paul Rubin]

Steve Holden  writes:
> It seems to me that the biggest sin in databases is a failure to use
> rigorous design techniques. If somebody doesn't understand relational
> theory

Where does one find out about this?  I've somehow managed to avoid
it for an awfully long time, but it begins to look useful.  Thanks.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: SQLite or files?

[Gabriel Genellina]

En Mon, 05 Oct 2009 23:08:59 -0300, AggieDan04   
escribió:

On Sep 17, 9:10 am, J Kenneth King  wrote:

ici  writes:



> I likeshelvefor saving small amounts of data, user preferences,
> recent files etc.
>http://docs.python.org/library/shelve.html

I like it too, but I hear the great powers that be are going to
deprecate it.


If you want the convenience of shelve without the limitations of dbm,
you can do:

"""Implementation of Python shelves using SQLite."""


See also http://bugs.python.org/issue3783 "dbm.sqlite proof of concept"

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: Object Relational Mappers are evil (a meditation)

[Ben Finney]

Paul Rubin  writes:

> Steve Holden  writes:
> > It seems to me that the biggest sin in databases is a failure to use
> > rigorous design techniques. If somebody doesn't understand relational
> > theory
>
> Where does one find out about this? I've somehow managed to avoid it
> for an awfully long time, but it begins to look useful. Thanks.

Wikipedia's article http://en.wikipedia.org/wiki/Relational_model>
and at the original Wiki http://c2.com/cgi/wiki?RelationalModel> are
good introductions to the concepts of the topic.

The works of C. J. Date, especially (in my view) “Database In Depth”
http://en.wikipedia.org/wiki/Special:BookSources/0596100124> and
the further reading recommended at the Wikipedia article
http://en.wikipedia.org/wiki/Relational_model#Further_reading>
http://en.wikipedia.org/wiki/Special:BookSources/0321197844>
http://en.wikipedia.org/wiki/Special:BookSources/0201709287>, are
solid groundings in the topic for programming practitioners.

In learning about the relational model, you'll inevitably learn that SQL
is rather flawed in its implementation of that model. Nevertheless, it
turns out to be the best (largely because most widespread) language for
describing a relational database, *if* used with knowledge of the
relational model to avoid those implementation flaws.

-- 
 \ “As scarce as truth is, the supply has always been in excess of |
  `\   the demand.” —Josh Billings |
_o__)  |
Ben Finney
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to sort a list of strings on a substring

[n00m]

> No, that's incorrect. Try it with this data and you will see it fails:

Of course, you are right, but I think the topic-starter is smart
enough
to understand that I suggested only a hint, a sketch, a sample of how
to use "key=" with "lambda", not a ready-to-apply solution.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Q: sort's key and cmp parameters

[Steven D'Aprano]

On Mon, 05 Oct 2009 19:34:27 -0700, Paul Rubin wrote:

> Raymond Hettinger  writes:

>> When Guido made the final call, I'm sure he was balancing a number of
>> goals including language simplification and one-way-to-do-it.  I'm also
>> sure that sorting lazily evaluated infinite sequences was not on his
>> radar screen :-)
>
> I can't help wondering if there will be feature requests for a cmp
> argument for the rest of eternity until it eventually makes its way back
> in.

When I learned that cmp was being dropped, I was horrified. That lasted 
for about fifteen minutes of time actually using key :)

Occasionally I come across sorting problems where it isn't obvious how to 
write the key function, but I've never come across one where it wasn't 
possible at all. I no longer miss cmp.


>> Psychologically, the thing that I find to be interesting is that
>> beginners and intermediate users seem to take to key functions more
>> readily than old timers.
> 
> I think key is preferable in at least 90% of the cases.  I also think a
> general purpose language should be able to handle 100% of the cases, so
> if key is all you have, there can be a gap to fill.

I don't think it's possible to enumerate 100% of the cases, let alone 
support them. 

Here's one thing I've sometimes wanted to do: sort multiple lists at 
once, according to the contents of one of them, with a single call. You 
can get the same result with a key function and sorting each list 
separately, but if you've got a lot of large lists, that becomes 
expensive.

Python's sort is a memory-sort -- you can't sort a list too big to fit 
into memory. Sorting 400GB of data using a single function call is not 
something I see as fundamental to a general purpose language (although of 
course you should be able to write a disk-sort utility using Python).

Nor does sort() support the case where the value of the comparisons 
differs according to where the elements are in the list, e.g. given:

[a, b, c, d, a, b]

the result of comparing a and b at the start of the list is different 
from comparing them at the end of the list. I don't think there's an 
actual use-case for this feature, and cmp would have just as much trouble 
solving this (pretend) problem as key does, but I don't think languages 
need to supply this as a fundamental operation. If you need it, write 
your own sort function :)

Which of course would be a solution to your problem. The source code for 
timsort is available, and stable. You could fork it, re-enable the 
support for comparison functions, and put it on the Cheeseshop. If there 
is sufficient interest, maybe you'll get cmp support re-added to the 
built-in in a version or two.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to sort a list of strings on a substring

[Steven D'Aprano]

On Mon, 05 Oct 2009 20:33:51 -0700, n00m wrote:

>> No, that's incorrect. Try it with this data and you will see it fails:
> 
> Of course, you are right, but I think the topic-starter is smart enough
> to understand that I suggested only a hint, a sketch, a sample of how to
> use "key=" with "lambda", not a ready-to-apply solution.

Oh please. That's a ridiculous excuse. Your post started with "Here you 
are" -- the implication is that you thought it *was* a solution, not a 
hint. A hint would be something like "Write a key function, perhaps using 
lambda, and pass it to the sort() method using the key parameter."

There's no shame at writing buggy code. There's not a person here who has 
never made a silly mistake, and most of us have done so in public too. 
Some real clangers too. What matters is how folks respond to having the 
their mistakes pointed out, and whether they learn from it.




-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to sort a list of strings on a substring

[n00m]

English language is not my mother toung,
so I can't grasp many subtle nuances of it.
Maybe "here you are" means to me quite a
different thing than to you.
-- 
http://mail.python.org/mailman/listinfo/python-list