Re: spawning pyhon apps...

[Jason Scheirer]

On Jan 9, 3:43 pm, "bruce"  wrote:
> hi jason
>
> forgive me... but in your sample:
>         my_popenobjects = [subprocess.Popen("foo.py", "--filename=file
>         %i.txt"%x) for x in xrange(10)]
> are you spawning 'foo.py' 10 times? that can't be right!
> so just what is "foo.py" used for? what am i missing...
>
> it looks like the my_popenobjects array is iterated through to check the
> statuscode. is the statuscode the value that would be returned from a child
> python script via something like "return(2)"
>
> i've seen mention of os.waitpid(..) does this play into waiting for child
> processes to complete, or determine if they've terminated??
>
> thanks
>
> -Original Message-
> From: python-list-bounces+bedouglas=earthlink@python.org
>
> [mailto:python-list-bounces+bedouglas=earthlink@python.org]on Behalf
> Of Jason Scheirer
> Sent: Friday, January 09, 2009 3:19 PM
> To: python-l...@python.org
> Subject: Re: spawning pyhon apps...
>
> On Jan 9, 2:47 pm, "bruce"  wrote:
> > hi...
>
> > toying with an idea.. trying to figure out a good/best way to spawn
> multiple
> > python scripts from a parent python app. i'm trying to figure out how to
> > determine when all child apps have completed, or to possibly determine if
> > any of the child processes have died/halted..
>
> > parent app
> >  spawn child1
> >  spawn child2
> >  spawn child3
> >  .
> >  .
> >  .
> >  spawn childn
>
> > do i iterate through a os.waitpid(pid) for each pid of the child processes
> i
> > create?
>
> > is there another approach? code samples/tutorial...??
>
> > i've seen various approaches via google, but not just what i'm looking
> for..
>
> > thanks
>
> Investigate the subprocess module, you probably want Popen objects.
> You can do a poll loop like
>
> my_popenobjects = [subprocess.Popen("foo.py", "--filename=file
> %i.txt"%x) for x in xrange(10)]
>
> while any(popenobject.returncode is None for popenobject in
> my_popenobjects):
>   time.sleep(0.25)
>
> If your tasks are more like function calls and less like shell
> scripts, then investigate writing your python as an importable module
> and use the multiprocessing module, which will do threading/
> subprocessing for you.
> --http://mail.python.org/mailman/listinfo/python-list
>
>

Correction: statuscode is wrong. It's returncode.

Foo.py is the hypothetical Python script you want to run in a
subprocess. In this example, I have an external shell script named
foo.py, and I am indeed spawning 10 copies of it, each with a second
argument that varies (foo.py --filename=file0.txt, foo.py --
filename=file1.txt, ... foo.py --filename=file9.txt). You don't need
os.waitpid() with a Popen object, there is a Popen.wait() method you
can call which will accomplish the exact same thing. I'm polling the
Popen.returncode for each process' return code (which is the numeric
code a process returns when it finishes, like sys.exit(x) or return,
or gives None if it's not done yet. What this sample is doing is
opening 10 copies of the script and running them in parallel, if you
want to run it is serial then you can do a simple for loop and .wait()
on each:

for cmd in ('a', 'b', 'c'):
  sp = subprocess.Popen(cmd)
  sp.wait()
  print "Command %r completed with status %i" % (cmd, sp.returncode)

I'm still not 100% sure what you're trying to accomplish. What is the
exact problem you are wishing to solve?
--
http://mail.python.org/mailman/listinfo/python-list

Re: Why can't I store a DLL in a module library Zip file?

[Martin v. Löwis]

> It's almost like shared libraries are disallowed from the module
> library Zip format.

Correct. That's a limitation of the Windows operating system, which can
load DLLs only from plain, regular files. Python can load .py files from
zip files because it implements the loading itself. For DLL loading, it
must use the OS routines, which just don't look into zipfiles.

Regards,
Martin
--
http://mail.python.org/mailman/listinfo/python-list

RE: spawning pyhon apps...

[bruce]

thanks jason

or i could also, simply iterate through a loop of the names of the "child
processes" i want to spawn, using the names in the subprocess.popen, and
then proceeding with the rest of your example..

question though... if i have a child app that's hanging.. how do i kill it.

or is this getting into the aspect of using interprocess pipes, where if the
parent isn't getting a 'live ping' via the pipe back from the child after a
certain amount of time... it could kill the child...

thoughts/comments...

thanks


-Original Message-
From: python-list-bounces+bedouglas=earthlink@python.org
[mailto:python-list-bounces+bedouglas=earthlink@python.org]on Behalf
Of Jason Scheirer
Sent: Friday, January 09, 2009 3:59 PM
To: python-list@python.org
Subject: Re: spawning pyhon apps...


On Jan 9, 3:43 pm, "bruce"  wrote:
> hi jason
>
> forgive me... but in your sample:
>         my_popenobjects = [subprocess.Popen("foo.py", "--filename=file
>         %i.txt"%x) for x in xrange(10)]
> are you spawning 'foo.py' 10 times? that can't be right!
> so just what is "foo.py" used for? what am i missing...
>
> it looks like the my_popenobjects array is iterated through to check the
> statuscode. is the statuscode the value that would be returned from a
child
> python script via something like "return(2)"
>
> i've seen mention of os.waitpid(..) does this play into waiting for child
> processes to complete, or determine if they've terminated??
>
> thanks
>
> -Original Message-
> From: python-list-bounces+bedouglas=earthlink@python.org
>
> [mailto:python-list-bounces+bedouglas=earthlink@python.org]on Behalf
> Of Jason Scheirer
> Sent: Friday, January 09, 2009 3:19 PM
> To: python-l...@python.org
> Subject: Re: spawning pyhon apps...
>
> On Jan 9, 2:47 pm, "bruce"  wrote:
> > hi...
>
> > toying with an idea.. trying to figure out a good/best way to spawn
> multiple
> > python scripts from a parent python app. i'm trying to figure out how to
> > determine when all child apps have completed, or to possibly determine
if
> > any of the child processes have died/halted..
>
> > parent app
> >  spawn child1
> >  spawn child2
> >  spawn child3
> >  .
> >  .
> >  .
> >  spawn childn
>
> > do i iterate through a os.waitpid(pid) for each pid of the child
processes
> i
> > create?
>
> > is there another approach? code samples/tutorial...??
>
> > i've seen various approaches via google, but not just what i'm looking
> for..
>
> > thanks
>
> Investigate the subprocess module, you probably want Popen objects.
> You can do a poll loop like
>
> my_popenobjects = [subprocess.Popen("foo.py", "--filename=file
> %i.txt"%x) for x in xrange(10)]
>
> while any(popenobject.returncode is None for popenobject in
> my_popenobjects):
>   time.sleep(0.25)
>
> If your tasks are more like function calls and less like shell
> scripts, then investigate writing your python as an importable module
> and use the multiprocessing module, which will do threading/
> subprocessing for you.
> --http://mail.python.org/mailman/listinfo/python-list
>
>

Correction: statuscode is wrong. It's returncode.

Foo.py is the hypothetical Python script you want to run in a
subprocess. In this example, I have an external shell script named
foo.py, and I am indeed spawning 10 copies of it, each with a second
argument that varies (foo.py --filename=file0.txt, foo.py --
filename=file1.txt, ... foo.py --filename=file9.txt). You don't need
os.waitpid() with a Popen object, there is a Popen.wait() method you
can call which will accomplish the exact same thing. I'm polling the
Popen.returncode for each process' return code (which is the numeric
code a process returns when it finishes, like sys.exit(x) or return,
or gives None if it's not done yet. What this sample is doing is
opening 10 copies of the script and running them in parallel, if you
want to run it is serial then you can do a simple for loop and .wait()
on each:

for cmd in ('a', 'b', 'c'):
  sp = subprocess.Popen(cmd)
  sp.wait()
  print "Command %r completed with status %i" % (cmd, sp.returncode)

I'm still not 100% sure what you're trying to accomplish. What is the
exact problem you are wishing to solve?
--
http://mail.python.org/mailman/listinfo/python-list

--
http://mail.python.org/mailman/listinfo/python-list

Re: Problem with -3 switch

[John Machin]

On Jan 10, 6:58 am, Carl Banks  wrote:
> On Jan 9, 12:36 pm, "J. Cliff Dyer"  wrote:
>
>
>
>
>
> > On Fri, 2009-01-09 at 13:13 -0500, Steve Holden wrote:
> > > Aivar Annamaa wrote:
> > > >> As was recently pointed out in a nearly identical thread, the -3
> > > >> switch only points out problems that the 2to3 converter tool can't
> > > >> automatically fix. Changing print to print() on the other hand is
> > > >> easily fixed by 2to3.
>
> > > >> Cheers,
> > > >> Chris
>
> > > > I see.
> > > > So i gotta keep my own discipline with print() then :)
>
> > > Only if you don't want to run your 2.x code through 2to3 before you use
> > > it as Python 3.x code.
>
> > > regards
> > >  Steve
>
> > And mind you, if you follow that route, you are programming in a
> > mightily crippled language.
>
> How do you figure?
>
> I expect that it'd be a PITA in some cases to use the transitional
> dialect (like getting all your Us in place), but that doesn't mean the
> language is crippled.

What is this "transitional dialect"? What does "getting all your Us in
place" mean?

Steve & Cliff are talking about the rather small subset of Python that
is not only valid syntax in both 2.x and 3.x but also has the same
meaning in 2.x and 3.x.

>
> > It's about as bad as trying to write
> > cross-browser CSS.  Don't put yourself through that pain if you don't
> > have to.
>
> Have you tried doing that, or are you imagining how it will be?  I'm
> curious about people's actual experiences.

I maintain two packages, xlrd which supports 2.1 to 2.6, and xlwt
which supports 2.3 to 2.6. I've done suck-it-and-see trials on being
able to support 3.x as well from the same codebase, and it's turned
out reasonably well. xlrd already had a module called timemachine
which caters for version- dependent stuff. Extending this to 3.x was
more a voyage of discovery than a PITA. timemachine.py is "crippled"
in Cliff's sense, in that because I'm the principal user I need to
make it robust and idiot-proof, so it has been written under the
following constraints:
(1) not one copy of timemachine.py for 2.1, one for 2.2, one for
2.3, ... etc; just one copy, period.
(2) means that it must use syntax that's valid in all supported
versions
(3) must be able to be processed by 2to3 without causing a commotion
(4) the original version and the 2to3 output must have the same effect
when imported by 3.x.

So one ends up with code like:
   glued = BYTES_NULL.join(list_of_pieces_of_a_binary_file)
which is supported by timemachine definitions like
BYTES_NULL = bytes(0) # 3.x ... note b'' is not valid in 2.x
BYTES_NULL = '' # 2.x

BYTES_NULL.join() may be ugly, but it's not crippled, it's fully
functional, and it would be very easy to find and change in the future
in two possible scenarios (1) drop 2.x support (2) change codebase to
be mostly 3.x, support 2.x by a (mythical, hoped-for) 3to2 mechanism.

> Problem is, a lot of people use the "bang at it with a hammer till it
> works" approach to programming, and really have no shame when it comes
> to engaging in questionable practices like relying on accidental side
> effects, rather than taking the time to try to program robustly.  I
> expect people with that style of programming will have many more
> issues with the transition.

Those with many more issues are likely to be those who don't have
adequate tests and/or can't debug their way out of a wet paper bag --
could well be we're talking about the same bunch :-)

Cheers,
John
--
http://mail.python.org/mailman/listinfo/python-list

Re: spawning pyhon apps...

[Jason Scheirer]

On Jan 9, 4:07 pm, "bruce"  wrote:
> thanks jason
>
> or i could also, simply iterate through a loop of the names of the "child
> processes" i want to spawn, using the names in the subprocess.popen, and
> then proceeding with the rest of your example..
>
> question though... if i have a child app that's hanging.. how do i kill it.
>
> or is this getting into the aspect of using interprocess pipes, where if the
> parent isn't getting a 'live ping' via the pipe back from the child after a
> certain amount of time... it could kill the child...
>
> thoughts/comments...
>
> thanks
>
> -Original Message-
> From: python-list-bounces+bedouglas=earthlink@python.org
>
> [mailto:python-list-bounces+bedouglas=earthlink@python.org]on Behalf
> Of Jason Scheirer
> Sent: Friday, January 09, 2009 3:59 PM
> To: python-l...@python.org
> Subject: Re: spawning pyhon apps...
>
> On Jan 9, 3:43 pm, "bruce"  wrote:
> > hi jason
>
> > forgive me... but in your sample:
> >         my_popenobjects = [subprocess.Popen("foo.py", "--filename=file
> >         %i.txt"%x) for x in xrange(10)]
> > are you spawning 'foo.py' 10 times? that can't be right!
> > so just what is "foo.py" used for? what am i missing...
>
> > it looks like the my_popenobjects array is iterated through to check the
> > statuscode. is the statuscode the value that would be returned from a
> child
> > python script via something like "return(2)"
>
> > i've seen mention of os.waitpid(..) does this play into waiting for child
> > processes to complete, or determine if they've terminated??
>
> > thanks
>
> > -Original Message-
> > From: python-list-bounces+bedouglas=earthlink@python.org
>
> > [mailto:python-list-bounces+bedouglas=earthlink@python.org]on Behalf
> > Of Jason Scheirer
> > Sent: Friday, January 09, 2009 3:19 PM
> > To: python-l...@python.org
> > Subject: Re: spawning pyhon apps...
>
> > On Jan 9, 2:47 pm, "bruce"  wrote:
> > > hi...
>
> > > toying with an idea.. trying to figure out a good/best way to spawn
> > multiple
> > > python scripts from a parent python app. i'm trying to figure out how to
> > > determine when all child apps have completed, or to possibly determine
> if
> > > any of the child processes have died/halted..
>
> > > parent app
> > >  spawn child1
> > >  spawn child2
> > >  spawn child3
> > >  .
> > >  .
> > >  .
> > >  spawn childn
>
> > > do i iterate through a os.waitpid(pid) for each pid of the child
> processes
> > i
> > > create?
>
> > > is there another approach? code samples/tutorial...??
>
> > > i've seen various approaches via google, but not just what i'm looking
> > for..
>
> > > thanks
>
> > Investigate the subprocess module, you probably want Popen objects.
> > You can do a poll loop like
>
> > my_popenobjects = [subprocess.Popen("foo.py", "--filename=file
> > %i.txt"%x) for x in xrange(10)]
>
> > while any(popenobject.returncode is None for popenobject in
> > my_popenobjects):
> >   time.sleep(0.25)
>
> > If your tasks are more like function calls and less like shell
> > scripts, then investigate writing your python as an importable module
> > and use the multiprocessing module, which will do threading/
> > subprocessing for you.
> > --http://mail.python.org/mailman/listinfo/python-list
>
> Correction: statuscode is wrong. It's returncode.
>
> Foo.py is the hypothetical Python script you want to run in a
> subprocess. In this example, I have an external shell script named
> foo.py, and I am indeed spawning 10 copies of it, each with a second
> argument that varies (foo.py --filename=file0.txt, foo.py --
> filename=file1.txt, ... foo.py --filename=file9.txt). You don't need
> os.waitpid() with a Popen object, there is a Popen.wait() method you
> can call which will accomplish the exact same thing. I'm polling the
> Popen.returncode for each process' return code (which is the numeric
> code a process returns when it finishes, like sys.exit(x) or return,
> or gives None if it's not done yet. What this sample is doing is
> opening 10 copies of the script and running them in parallel, if you
> want to run it is serial then you can do a simple for loop and .wait()
> on each:
>
> for cmd in ('a', 'b', 'c'):
>   sp = subprocess.Popen(cmd)
>   sp.wait()
>   print "Command %r completed with status %i" % (cmd, sp.returncode)
>
> I'm still not 100% sure what you're trying to accomplish. What is the
> exact problem you are wishing to solve?
> --http://mail.python.org/mailman/listinfo/python-list
>
>

Yes, so to open your processes you can loop over a list of commands
and create new subprocess.Popen(cmd) objects for each.

Everything is explained:

http://docs.python.org/library/subprocess.html#popen-objects

You can do .terminate() to kill a process that may be hanging, you can
get the .stdout and poll on its .read() to see if it's still putting
anything out to the console.
--
http://mail.python.org/mailman/listinfo/python-list

Re: why cannot assign to function call

[Joe Strout]

Mark Wooding wrote:


As an aside, I don't notice anywhere near as much confusion in Lisp and
Scheme groups, which might be surprising since Lisp and Scheme have
precisely the same data model, argument passing convention, and
assignment semantics, as Python has.


Nor is there anywhere near as much confusion in the REALbasic community 
(with which I'm most familiar), which also has the same semantics for 
reference types (which in RB is everything except numbers, colors, and 
Boolean).


Apparently there is occasionally a little confusion in the Java 
community, but it rarely reaches the sillyness proportions seen here:




  * The Lisp and Scheme communities are smaller.  This is certainly
true.  But it wouldn't explain what appears to be a disproportionate
level of confusion on the topic among Python beginners.

  * Individuals in the Lisp and Scheme communities are cleverer and/or
more widely experienced.  One might make an argument that this is
true and a result of the relative community sizes -- basically a
result of self-selection.  But instead I'll reject this as an
explanation.  It's arrogant and unproven.

  * The Lisp and Scheme communities make a concerted effort to explain
their data model clearly and precisely.  They accept that it's
actually quite complicated and, rather than pretend that it isn't,
explain the complexity and the benefits it brings that make the
complexity worthwhile.  I think this is the likely one.


That's a nice way of putting it.  I might go a step further and say that 
there is a small but vocal portion of the Python community that insists 
on confounding the issue by claiming that Python's assignment and 
parameter-passing conventions are different from other languages. 
(Which, of course, requires one's head to be firmly in the sand with 
regard to the basic fact that Python variables contain references, not 
objects.)



But it's not just assignment that deals with references.  It's argument
passing and storage of compound data as well.  (See PLR 3.1.)

They expect that assignment copies stuff, because that's what assignment
does.  Everywhere that I can think of -- except C++, which leaves
assignment semantics in hands of the programmer.  What they're confused
about is what, precisely, it is that gets copied.  And that, really, is
a result of an inadequate understanding of the data model.


I have nothing to add to this.  It just seem well worth quoting.  :)


I agree that most of the time, when one is using large (memory)
composite "objects", and one needs to pass, or access them by
different names, one will often use references to do so in order to
avoid expensive copies or to get desired "shared" behavior.  But (with
the exception of C arrays [*1]), doing so requires some special syntax
in all the languages I mentioned (AFAIK).


Ummm... you mentioned C, C++, `Python, Java, REALbasic, .NET'.


No, actually, that was me.  rurpy's list was something like C, FORTRAN, 
Perl, and VBA.



Well, C we've dealt with.  C++ is weird.  Python we all know, and is the
main subject of the argument.  REALbasic I don't know at all, but BASICs
traditionally represent data fairly directly (rather than via
references) so will largely be like C.


Not REALbasic.  It's a very modern language with semantics pretty much 
identical to Java.  Simple types (numbers, colors, Booleans) are stored 
directly; all other types (including strings, objects, and arrays) are 
stored on the heap and accessed via references.



.NET isn't a language at all:
rather, it's a virtual machine, runtime system, class library and family
of languages each of which may have idiosyncratic semantics.


But they don't, AFAIK -- they all have the same semantics; only the 
surface syntax differs.  And those semantics are the same as REALbasic 
and Java.


See  for some side-by-side 
comparisons.



Which leaves Java.  Java divides the world into `primitive' and
`reference' types (4.1).  The former are represented directly; the
latter have a pointer to the true data as immediate representation.


Right -- a very common pattern among modern languages.


So it still seems to me that this is a likely explanation to why there
is frequent misunderstanding of Python's assignments, and why
responding to such misunderstandings with, "Python's assignments are
the same as other languages'", is at best not helpful.


That's why I'm not just saying that assignment is the same.  I'm also
saying that the data model is most definitely not the same as C.


Technically true, in that pointers in C require some special syntax, but 
the common idiom is to hide this away by defining a new type:


 typedef Foo* FooPtr;

Now, for any code using the "FooPtr" type, the data model is the same as 
Python (or as Java, RB, .NET, etc., again for code that's using only 
reference types).


Best,
- Joe



--
http:/

Re: why cannot assign to function call

[rurpy]

Joe Strout wrote:
> ru...@yahoo.com wrote:
>
>>> I never claimed that you *couldn't* have copy semantics in C; you can do
>>> almost anything you want in C (or C++).  But the *normal* usage of an
>>> array is via a pointer, in which case the semantics are exactly the same
>>> as in Python, Java, REALbasic, .NET, etc.
>>
>> Arrays are the only datatype in C that don't use
>> copy-like assignment.  Everything else does.
>
> No, arrays are just one reference type; pointers are another (and in
> most ways, these are the same thing).

Pointers are passed and assigned by value, just as
other types (disputedly except arrays) are.
One can then use that pointer to manually effect
pass-(the-value-pointed-to)-by-reference, or sharing,
etc.

> When dealing with objects in C++,
> one routinely handles them with pointers, so that's the use case which
> is analogous to Python -- all the value types can be ignored for the
> sake of comparison.  (C is not an OOP language, of course, but even
> there, all but the most trivial of structs are usually allocated on the
> heap and passed around via pointers, just like in C++, Java, .NET, RB,
> and Python.)

In C (you have to explicitly ask for a reference
(pointer) to something (other than arrays) if you
want to use/pass a reference to something.
If you simply use the name, you get by-value semantics.

>>> Ah.  OK then, I guess I missed you're point.  You're absolutely right;
>>> many languages have both reference types and value types.  Python is a
>>> bit unusual in that it has only reference types.  I would have picked a
>>> different example to illustrate that, but it's true nonetheless.
>>
>> If one accepts that there are a "lot" of people
>> who post in here that clearly are surprised by
>> Python's assignment semantics, and further appear
>> to expect assignment to have copy-like semantics,
>> then where is that expectation coming from?
>
> I think it comes from people stumbling across posts in this forum
> claiming that Python has unusual assignment semantics.  I wish people
> would stop saying that, as it causes a lot of confusion.
>
>> How would anyone develop that expectation if (from
>> a different post in this thread), "[Python's] idea
>> of assignment is the same as anyone else's."
>
> I can think of two ways:
>
> 1. They're new to programming in general, and would have had the same
> expectation for any other language.  OR,

IIRC, Someone posted here that his experience was
that 12-year old kids (presumably without programming
experience) had no problem with Python and references
when described as "names given to an object".  (From
memory, can't locate the post right now.)

> 2. They already understand some other language, and then they come here
> and read wild claims that Python's assignment and parameter-passing
> semantics are different from other languages.  Duped by this claim, they
>   conclude that, if it's unlike other languages, then Python must have
> copy semantics.

I have seen no evidence of that.  If it were true I
would expect at least some posts to refer to reading
those "wild claims".

>> If you maintain that reference-like assignment
>> is very common and something every programmer is
>> accustomed to, then where are they getting the
>> copy-like assignment expectations from?
>
> Reference-like assignment IS very common (see
> ).  So, see above.
>
>> I agree that most of the time, when one is using
>> large (memory) composite "objects", and one needs
>> to pass, or access them by different names, one will
>> often use references to do so in order to avoid
>> expensive copies or to get desired "shared" behavior.
>
> Right.
>
>> But (with the exception of C arrays [*1]), doing so
>> requires some special syntax in all the languages I
>> mentioned (AFAIK).
>
> Whether you consider it "special" or not, pointers are extremely common
> in C.  Even more so in C++, which is the closest thing to an OOP
> language in the list of moldy languages you mentioned.

Non-special is "b = a".

> You also mentioned VBA -- if that's anything like VB, it does NOT
> require any special syntax; a variable is a reference type if its
> declared type is a class or string, and a simple type if it's anything
> else (just like in .NET, Java, and REALbasic).

It (mercifully) been a long time since I've used
VB but the VB code I posted does run, and does exhibit
by-value assignment behavior.  My faint recollection
is that if you want to assign a reference you cannot
write, "b = a" but must instead write "set b = a".
"b = a" assigns by value.

In Perl it is definitely true that you different syntax:
@a = (1,2,3)
@b = @a   # Copy
$b = \...@a  # Reference

C is the same way for everything (including pointers)
except arrays:

struct {...} foo;
foo a, b *bp;
b = a;  # Copy
bp = &a; # Reference

>> So it still seems to me that this is a likely
>> explanation to why there is frequent misunderstanding
>> of Python's assignments, and why respo

Re: why cannot assign to function call

[Benjamin Kaplan]

On Fri, Jan 9, 2009 at 7:11 PM, Joe Strout  wrote:

> Mark Wooding wrote:
>
>
>  .NET isn't a language at all:
>> rather, it's a virtual machine, runtime system, class library and family
>> of languages each of which may have idiosyncratic semantics.
>>
>
> But they don't, AFAIK -- they all have the same semantics; only the surface
> syntax differs.  And those semantics are the same as REALbasic and Java.
>

They do in fact differ. You can have Visual C++ compile to CLI, in which
case you have the C++ assignment in .NET, you can use VisualBasic, in which
you can specify pass-by-value or -reference when you pass arguments to
methods, and you have Visual C#, which copies Java's model and is therefore
the same as Python.


>
> See  for some side-by-side
> comparisons.
>
--
http://mail.python.org/mailman/listinfo/python-list

Re: spawning pyhon apps...

[Mark Wooding]

bruce  wrote:

> toying with an idea.. trying to figure out a good/best way to spawn
> multiple python scripts from a parent python app. i'm trying to figure
> out how to determine when all child apps have completed, or to
> possibly determine if any of the child processes have died/halted..

You don't say what platform you're using, but you mention os.waitpid so
I'll randomly assume it's Unix-like.

> do i iterate through a os.waitpid(pid) for each pid of the child
> processes i create?

That will technically work, and give you the information you wanted,
though not necessarily in the most timely fashion.  It'll block on each
process in turn, waiting for its exit status -- so it'll finish as soon
as all the children are dead, but if (say) the fifth process dies first,
you won't find out until the first four have also passed on.

If you don't have anything better for your program to do, and you're
really on Unix, you can call

  kid, status = os.waitpid(0, 0)

to wait for something to happen to any of your process's children; the
kid is the process-id of the child being reported and the status is what
happened to it.

If you do have other things for your process to be doing, then your best
bet is to establish a signal handler for SIGCHLD and installing a
handler of the form

  import signal as S
  import os as OS
  import errno as E

  ## Children sometimes die.  It's sad.
  def sigchld(sig, frame):
try:
  while True:
kid, status = OS.waitpid(0, OS.WNOHANG)
if kid == 0:
  break
## Handle death of KID here.
except OSError, err:
  if err.errno != E.ECHILD:
raise

  ### ...

  ## Establish handler.
  S.signal(S.SIGCHLD, sigchld)

should work.

If you're running on Windows then these tricks won't work.  As a grim
hack, you could start a thread per child process and have each thread
wait for its own child (sending the exit status through a queue or
something).  I'm afraid I don't know Windows well enough to offer
slicker solutions; maybe someone else can help.

-- [mdw]
--
http://mail.python.org/mailman/listinfo/python-list

Re: BadZipfile "file is not a zip file"

[John Machin]

On Jan 10, 9:52 am, webcomm  wrote:
> On Jan 9, 5:21 pm, John Machin  wrote:
>
> > Thanks. Would you mind spending a few minutes more on this so that we
> > can see if it's a problem that can be fixed easily, like the one that
> > Chris Mellon reported?
>
> Don't mind at all.  I'm now working with a zip file with some dummy
> data I downloaded from the web service.  You'll notice it's a smaller
> archive than the one I was working with when I ran zip_susser.py, but
> it has the same problem (whatever the problem is).

You mean it produces the same symptom. The zipfile.py has several
paths to the symptom i.e. the uninformative "bad zipfile" exception;
we don't know which path, yet. That's why Martin was suggesting that
you debug the sucker; that's why I'm trying to do it for you by remote
control. It is not impossible for a file with dummy data to have been
handcrafted or otherwise produced by a process different to that used
for a real-data file. Please run v2 of the gadget on the real-data zip
and report the results.

> It's the one I
> uploaded tohttp://webcomm.webfactional.com/htdocs/data.zip
>
> Here's what I get when I run zip_susser_v2.py...
>
> archive size is 1092
> FileHeader at 0
> CentralDir at 844
> EndArchive at 894
> using posEndArchive = 894
> endArchive: ('PK\x05\x06', 0, 0, 1, 1, 50, 844, 0)
>                         signature : 'PK\x05\x06'
>                     this_disk_num : 0
>              central_dir_disk_num : 0
> central_dir_this_disk_num_entries : 1
>   central_dir_overall_num_entries : 1
>                  central_dir_size : 50
>                central_dir_offset : 844
>                      comment_size : 0
>
> expected_comment_size: 0
> actual_comment_size: 176
> comment is all spaces: False
> comment is all '\0': True
> comment (first 100 bytes):
> '\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
> \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0­0\x00
> \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0­0\x00
> \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0­0\x00
> \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0­0\x00
> \x00\x00\x00\x00\x00\x00\x00'
>
> Not sure if you've seen this 
> thread...http://groups.google.com/group/comp.lang.python/browse_thread/thread/...

Yeah, I've seen it ... (sigh) ... pax Steve Holden, but *please* stick
with one thread ...


--
http://mail.python.org/mailman/listinfo/python-list

Common path all PyObjects take on destruction?

[Ryan Stutsman]

I've added a field to all PyObjects in the interpreter which is of type
PyObject*.  Most of the time this pointer is NULL but occassionally I
want to track some information in there.  The problem I'm running into
is that I can add a reference to a PyObject inside any of my PyObjects,
but it seems that there isn't any one path that all objects follow on
destruction so that I can later Py_DECREF that reference.

Eventually most of the types seem to call PyObject_Free, but this gets
called with void* and it seems it isn't always the case that PyObject*s
are passed in.

Where is the best place to implement something like this?  It really
won't work to implement this in the destructor of each of the individual
types because other types added later won't know to DECREF this field on
destruction.

Any hints would be appreciated.  Hopefully I'm just missing something
simple.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python 2.6 fails on compiling > Bug report

[googler . 1 . webmaster]

hi, thanks for your help.

May the diff file is wrong? on my system it doesn't work.
Well, set the -enable-universalsdk= path occurs that error.

Hmm. that is really a disaster, isn't it :( hm.
--
http://mail.python.org/mailman/listinfo/python-list

how to remove 'FFFD' character

[webcomm]

Does anyone know a way to remove the 'FFFD' character with python?

You can see the browser output I'm dealing with here:
http://webcomm.webfactional.com/htdocs/fffd.JPG
I deleted a big chunk out of the middle of that JPG to protect
sensitive data.

I don't know what the character encoding of this data is and don't
know what the 'FFFD' represents.  I guess it is something that can't
be represented in whatever this particular encoding is, or maybe it is
something corrupt that can't be represented in any encoding.  I just
want to scrub it out.  I tried this...

clean = txt.encode('ascii','ignore')

...but the 'FFFD' still comes through.  Other ideas?

Thanks,
Ryan
--
http://mail.python.org/mailman/listinfo/python-list

Re: why cannot assign to function call

[Steven D'Aprano]

On Fri, 09 Jan 2009 21:03:39 +, Mark Wooding wrote:

> Steven D'Aprano  wrote:
> 
>> Python doesn't do the same thing as C. It actually passes the same
>> value to the function, without copying it.
>> 
>> Why oh why do you keep insisting that Python is no different from C?
> 
> I'm beginning to think that you're not bothing to read what I'm writing,

Er, perhaps you missed that I was replying to Joe Strout.

Joe wrote:

"The reference itself is passed in, not the variable (or 
expression) that held or generated the reference in the calling code.

This is no different from, in C, passing an integer:"

I'm pretty sure I clearly quoted Joe in my post. Perhaps you missed it. 
In any case, I'm happy for you to contribute (it's a public forum), but 
how do you conclude I'm not reading *your* posts because I disagree with 
Joe's claim that Python is just like C?




-- 
Steven
--
http://mail.python.org/mailman/listinfo/python-list

Re: why cannot assign to function call

[Grant Edwards]

On 2009-01-10, Joe Strout  wrote:
> Mark Wooding wrote:
>
>> As an aside, I don't notice anywhere near as much confusion in Lisp and
>> Scheme groups, which might be surprising since Lisp and Scheme have
>> precisely the same data model, argument passing convention, and
>> assignment semantics, as Python has.
>
> Nor is there anywhere near as much confusion in the REALbasic
> community (with which I'm most familiar), which also has the
> same semantics for reference types (which in RB is everything
> except numbers, colors, and Boolean).

It's not that there's a lot of confusion, it's just that we
spend a lot of time talking about it.  Programming in Python is
so much more productive that we've got a lot more spare time
that people who use other languages.

-- 
Grant

--
http://mail.python.org/mailman/listinfo/python-list

Re: Implementing file reading in C/Python

[John Machin]

On Jan 9, 9:56 pm, mk  wrote:

> The factor of 30 indeed does not seem right -- I have done somewhat
> similar stuff (calculating Levenshtein distance [edit distance] on words
> read from very large files), coded the same algorithm in pure Python and
> C++ (using linked lists in C++) and Python version was 2.5 times slower.

Levenshtein distance using linked lists? That's novel. Care to
divulge?

And if C++ is using linked lists and Python isn't, it's not really the
same algorithm, is it?

Cheers,
John
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python 2.6 fails on compiling > Bug report

[Ned Deily]

In article 
,
 googler.1.webmas...@spamgourmet.com wrote:
> May the diff file is wrong? on my system it doesn't work.
> Well, set the -enable-universalsdk= path occurs that error.

Sorry, there were a few line wrap-arounds in the diff file due to long 
lines but the changes for the four significant lines should be easy to 
do manually.

Note that the build I made this way has not been tested on ppc64 and has 
had minimal testing so far on x64_64.

-- 
 Ned Deily,
 n...@acm.org

--
http://mail.python.org/mailman/listinfo/python-list

Re: Python 2.6 fails on compiling > Bug report

[googler . 1 . webmaster]

Hi! :)

>Sorry, there were a few line wrap-arounds in the diff file due to long
>lines but the changes for the four significant lines should be easy to
>do manually.


Hadn't done that before but I found a ressource how to read that
syntax.
x86_64 would be enough. i test that again. Thank you. :)
--
http://mail.python.org/mailman/listinfo/python-list

download timeout vs. socket timeout

[p.]

i'm using urllib2 in python 2.4

wondering how people typically deal with the case in which a download
is too slow. setting the socket timeout only covers those cases where
there is no response in the socket for whatever the timeout period is.
what if, however, i'm getting bits back but want simply to bail out if
the total time to download takes too long?

i'm trying to avoid creating a whole other thread if possible?
--
http://mail.python.org/mailman/listinfo/python-list

Re: download timeout vs. socket timeout

[Gabriel Genellina]

En Sat, 10 Jan 2009 00:07:15 -0200, p.  escribió:


i'm using urllib2 in python 2.4

wondering how people typically deal with the case in which a download
is too slow. setting the socket timeout only covers those cases where
there is no response in the socket for whatever the timeout period is.
what if, however, i'm getting bits back but want simply to bail out if
the total time to download takes too long?

i'm trying to avoid creating a whole other thread if possible?


You may use signal.alarm if it is available on your platform.
On Windows, using a separate thread for reading is the easiest option -  
unless you plan to download hundreds of files simultaneously.


--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: download timeout vs. socket timeout

[MRAB]

p. wrote:

i'm using urllib2 in python 2.4

wondering how people typically deal with the case in which a download
is too slow. setting the socket timeout only covers those cases where
there is no response in the socket for whatever the timeout period is.
what if, however, i'm getting bits back but want simply to bail out if
the total time to download takes too long?

i'm trying to avoid creating a whole other thread if possible?

Don't try to download all the data in one go, but do it a chunk at a 
time. If you calculate that it would take too long to complete, then stop.

--
http://mail.python.org/mailman/listinfo/python-list

Detecting open files and forcing closure

[Andrew Robert]

Hi Everyone,

We have a process that does a copy of a share from one location to another.

This usually works fine but can occasionally bomb if a file is opened by 
a user somewhere.


Is there a way to code detection of open files and force a close?

The files in question are typically PDF files if that matters.

Any pointers you can provide on this would be greatly appreciated.
--
http://mail.python.org/mailman/listinfo/python-list

Re: why cannot assign to function call

[Rhodri James]

On Sat, 10 Jan 2009 00:21:27 -,  wrote:


Joe Strout wrote:

ru...@yahoo.com wrote:


[snip]


Pointers are passed and assigned by value, just as
other types (disputedly except arrays) are.
One can then use that pointer to manually effect
pass-(the-value-pointed-to)-by-reference, or sharing,
etc.


[snip]


In C (you have to explicitly ask for a reference
(pointer) to something (other than arrays) if you
want to use/pass a reference to something.
If you simply use the name, you get by-value semantics.



How would anyone develop that expectation if (from
a different post in this thread), "[Python's] idea
of assignment is the same as anyone else's."


I can think of two ways:

1. They're new to programming in general, and would have had the same
expectation for any other language.  OR,


IIRC, Someone posted here that his experience was
that 12-year old kids (presumably without programming
experience) had no problem with Python and references
when described as "names given to an object".  (From
memory, can't locate the post right now.)


'Twas I.  It was a rebuttal to your point that Python's assignment,
parameter passing and data model is somehow inherently more difficult
to wrap your brain around than that of other languages.  It isn't; if
anything it seems to be easier.


2. They already understand some other language, and then they come here
and read wild claims that Python's assignment and parameter-passing
semantics are different from other languages.  Duped by this claim, they
  conclude that, if it's unlike other languages, then Python must have
copy semantics.


I have seen no evidence of that.  If it were true I
would expect at least some posts to refer to reading
those "wild claims".


3. They conflate assignment, parameter passing and the data model, bring in
preconceptions of their own from other languages, and get caught out by
them.  It's quite easy to do, even within a language.  Look at the number
of times you had to say "except arrays" about C above.


In Perl it is definitely true that you different syntax:
@a = (1,2,3)
@b = @a   # Copy
$b = \...@a  # Reference


Perl has different syntax for everything.  It also has its own
peculiarities of assignment and data model that make it a less than
glowing example of comprehensibility.  List flattening, for instance,
is amazingly useful in many common Perl idioms, but really not what
you expect during assignment.


C is the same way for everything (including pointers)
except arrays:


Oh look, there's that exception again.


It may be one can make a technical case that assignment
is the same, but the reason I posted the code and results,
was because the behavior of "=" in them produces clearly
different results that "=" in Python.  If you want to
then say to someone, "ignore those different results,
'=' works exactly the same", all I can say is good luck.
I will look forward to continuing to see weekly questions
on this list. :-)


It would help if you were using "=" on things that were conceptually
the same across the languages, instead of things that only looked
similar.  Which is where the data model comes in, as has already been
explained at length, several times now.

--
Rhodri James *-* Wildebeeste Herder to the Masses
--
http://mail.python.org/mailman/listinfo/python-list

Re: why cannot assign to function call

[Mark Wooding]

Joe Strout  wrote:

> No, actually, that was me.  rurpy's list was something like C, FORTRAN, 
> Perl, and VBA.

My mistake -- I failed to read the quoting properly.  Apologies to all.

We still dealt with C.

Fortran (to give it its modern spelling) has a similar data model to C,
but passes arguments by reference, as described in my Epic Argument
Passing Article; I believe that arbitrary expressions may be used as
arguments, though I'm unsure as to the semantics of modifying parameters
bound to the resulting temporary argument locations.

I must confess to being ignorant of VBA.  My excuse is that I avoid
Windows systems as much as practical, and VBA doesn't have a significant
uptake on other systems.

Perl has a very strange data model indeed, and it's hard to get a proper
handle on it without getting into implementation details: unlike Python,
Perl is largely defined /by/ its implementation.

Perl has `references', which are proper (scalar) values through which
one may read and modify other values; i.e., they're what I called
`locatives' elsewhere.  Perl provides syntactic sugar, through its
`prototypes' which will convert an actual argument which designates
(e.g.) a list or hash into a reference to that list or hash; prototypes
provide other syntactic shortcuts too, though they have no fundamental
semantic effect.  In order to add to the confusion, Perl also provides
`typeglobs', which are a reification of toplevel variable bindings.

Perl's argument passing is fundamentally by /reference/.  Given the
function

  sub swap { ($_[0], $_[1]) = ($_[1], $_[0]) }

after setting $a = 1, $b = 2, and calling swap $a, $b, we find that $a
has the value 2 and $b is 1.

What's going on here is that a `location' in Perl is an explicit SV, AV
or HV object (for `scalar-', `array-' and `hash-value' respectively.
Calling a subroutine involves marking a position on a stack, pushing a
number of SVs, and then executing the subroutine's code, which receives
the items between the stack pointer and mark in the @_ array.  In the
case of argument expressions which designate SVs, those SVs are pushed
directly, and are therefore made available via @_.  Arguments which are
arrays or hashes are flattened: their components are pushed onto the
stack.  (This use of the stack corresponds to what the Perl manual
refers to as `list context'.)

> Not REALbasic.  It's a very modern language with semantics pretty much
> identical to Java.

Very well; thanks for the correction.  I might have a look at this at
some point.

> > .NET isn't a language at all: rather, it's a virtual machine,
> > runtime system, class library and family of languages each of which
> > may have idiosyncratic semantics.
> 
> But they don't, AFAIK -- they all have the same semantics; only the 
> surface syntax differs.  And those semantics are the same as REALbasic 
> and Java.

There's a .NET implementation of C++, which obviously brings all of
C++'s low-level data model (and it's user-definable assignment and
copying).  C#'s data model is more complex than Java's because it
provides mutable compound `value types', i.e., types whose immediate
representations consist of the raw contents of the object rather than a
reference.  The mutability allows one to distinguish this IR from a
reference IR.  C# is additionally complicated by its automatic boxing
and unboxing rules: an object of value type may under some circumstances
be `boxed', appearing as an object of reference type, and obeying the
reference-type semantics.

> Technically true, in that pointers in C require some special syntax, but 
> the common idiom is to hide this away by defining a new type:
> 
>   typedef Foo* FooPtr;
> 
> Now, for any code using the "FooPtr" type, the data model is the same
> as Python (or as Java, RB, .NET, etc., again for code that's using
> only reference types).

This is a syntactic transformation rather than a change to the data
model, though.

-- [mdw]
--
http://mail.python.org/mailman/listinfo/python-list

Re: why cannot assign to function call

[Mark Wooding]

Steven D'Aprano  wrote:

> Er, perhaps you missed that I was replying to Joe Strout.

Yes, evidently.  My apologies for the mix up!

-- [mdw], who obviously should put the keyboard down now.
--
http://mail.python.org/mailman/listinfo/python-list

Re: why cannot assign to function call

[rurpy]

Mark Wooding wrote:
> ru...@yahoo.com  wrote:
>
>> If one accepts that there are a "lot" of people who post in here that
>> clearly are surprised by Python's assignment semantics,
>
> But one should not accept that.  One might accept that there are many
> who post who claim that they are surprised by Python's assignment
> semantics.  Such people are wrong, however, since what they are
> surprised by is Python's data model, and one reason that they are
> surprised by Python's data model is because it's not actually explained
> very well.

Agreed.  I think the docs, especially those that develop
the conceptual model of how Python work at runtime, could
use some major attention.

>> and further appear to expect assignment to have copy-like semantics,
>
> This expectation is justified and, indeed, satisfied.  Python does, most
> definitely, copy on assignment.  What it copies is references, however.
> This might be clearer if the data model were explained better.
> ...
>> then where is that expectation coming from?
>
>> How would anyone develop that expectation if (from a different post in
>> this thread), "[Python's] idea of assignment is the same as anyone
>> else's."
>
> Because they've fundamentally misunderstood the data model.  The very
> fact that their confusion is ascribed to the wrong thing is strongly
> indicative of this.
>
>> If you maintain that reference-like assignment is very common and
>> something every programmer is accustomed to, then where are they
>> getting the copy-like assignment expectations from?
>
> But it's not just assignment that deals with references.  It's argument
> passing and storage of compound data as well.  (See PLR 3.1.)
>
> They expect that assignment copies stuff, because that's what assignment
> does.  Everywhere that I can think of -- except C++, which leaves
> assignment semantics in hands of the programmer.  What they're confused
> about is what, precisely, it is that gets copied.  And that, really, is
> a result of an inadequate understanding of the data model.
...
>> So it still seems to me that this is a likely explanation to why there
>> is frequent misunderstanding of Python's assignments, and why
>> responding to such misunderstandings with, "Python's assignments are
>> the same as other languages'", is at best not helpful.
>
> That's why I'm not just saying that assignment is the same.  I'm also
> saying that the data model is most definitely not the same as C.

I would be willing to bet that most of the confused
posters do not distinguish between assignment operation
(AO) and data model (DM).  Their conceptual box is
labeled "assignment behavior" and includes both AO and
DM.  They expect that AO+DM in Python will produce the
same results in Python as they are used to in the other
languages they've used.  That the discrepancy comes from
the DM part rather than the AO part is pretty irrelevant
to them given that world view.

So responding to the cry, "Python assignment is bizarre!"
with an indignant, "No, it is the same as other common
languages", is talking with different vocabularies, unless
it's also accompanied with all the other information presented
in this thread about how Python treats all names as references
(which *is* different that some other languages).

I notice there is not even an FAQ on the subject of assignment
despite the frequency which which people ask about it.

>> I have often wished that C handled arrays the same way it does
>> structs.  I am sure that the pointer-array pseudo-equivalence seemed
>> like a very clever idea at the time but I wonder if Dennis Richie ever
>> had second thoughts about it.
>
> Probably.  But I think the idea was actually inherited from BCPL, via B.
> In BCPL, memory is divided into words.  Depending on which operator you
> use, you can treat a particular word as an integer, a floating-point
> number, or a pointer.  An array in BCPL is represented as a pointer to
> its first element -- always -- and you index it by an expression of the
> form p!i (a notation inherited by BBC BASIC and Haskell of all things).
> The array->pointer decay is a compromise position between the BCPL
> notion of array-as-pointer and the desire to allocate such things with
> automatic storage duration and have sizeof and so on work properly.

Interesting tidbit, thanks.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Detecting open files and forcing closure

[MRAB]

Andrew Robert wrote:

Hi Everyone,

We have a process that does a copy of a share from one location to
another.

This usually works fine but can occasionally bomb if a file is opened
by a user somewhere.

Is there a way to code detection of open files and force a close?

The files in question are typically PDF files if that matters.

Any pointers you can provide on this would be greatly appreciated.


If the file is open then an exception will be raised. You could catch
it, sleep a while, and then retry, or continue with the other files and
then come back to it and retry. It might take more than one retry.
Anyway, it's a bad idea to force a close, even if that's possible.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Detecting open files and forcing closure

[Andrew Robert]

MRAB wrote:

Andrew Robert wrote:

Hi Everyone,

We have a process that does a copy of a share from one location to
another.

This usually works fine but can occasionally bomb if a file is opened
by a user somewhere.

Is there a way to code detection of open files and force a close?

The files in question are typically PDF files if that matters.

Any pointers you can provide on this would be greatly appreciated.


If the file is open then an exception will be raised. You could catch
it, sleep a while, and then retry, or continue with the other files and
then come back to it and retry. It might take more than one retry.
Anyway, it's a bad idea to force a close, even if that's possible.


The usual scenario is that a user will leave a PDF open and then go home 
for the evening. They are simply viewing and not modifying the file. 
When the XCOPY executes, it signals a failure and subsequent scheduler 
job abend.


What I need to do is detect if files are open for viewing and force a 
close before the copy operation is attempted.


Sleeping and retrying the copy is not an option because the user will 
likely leave it open all night.


Is there a way to detect the open files and close them out?
--
http://mail.python.org/mailman/listinfo/python-list

Re: Jython 2.5 Beta1 Released!

[Matthew Nuzum]

On Fri, Jan 9, 2009 at 3:37 PM, Frank Wierzbicki  wrote:
> On behalf of the Jython development team, I'm pleased to announce that
> Jython 2.5b1 is available for download:
> http://downloads.sourceforge.net/jython/jython_installer-2.5b1.jar.
> See the installation instructions here:
> http://www.jython.org/Project/installation.html.
>
> Jython 2.5 Beta1 continues a code cooling period where the number of
> new features should significantly slow as we concentrate on
> solidifying Jython 2.5 for an eventual release.  I would guess that we
> will put out about two more betas before we start pushing out release
> candidates, hopefully in February.
>
> This is a beta release so be careful.

Congrats, I'm watching with eager anticipation! I'm stunned at how
well this is working and the progress you're making. Keep up the great
work.

-- 
Matthew Nuzum
newz2000 on freenode, skype, linkedin, identi.ca and twitter
--
http://mail.python.org/mailman/listinfo/python-list

Re: why cannot assign to function call

[rurpy]

Rhodri James wrote:
> On Sat, 10 Jan 2009 00:21:27 -,  wrote:
>> IIRC, Someone posted here that his experience was
>> that 12-year old kids (presumably without programming
>> experience) had no problem with Python and references
>> when described as "names given to an object".  (From
>> memory, can't locate the post right now.)
>
> 'Twas I.  It was a rebuttal to your point that Python's assignment,
> parameter passing and data model is somehow inherently more difficult
> to wrap your brain around than that of other languages.  It isn't; if
> anything it seems to be easier.

That was and is not my point.  My point was that there
seems to be an observably significant number of people
who are surprised by the way Python assignment works.
I hypothesized that this was due to their experience with
other languages, *not* that Python is somehow "inherently
more difficult to wrap your brain around", and that the
response, "Python assignments are the same as in those
other languages" is insufficient.
--
http://mail.python.org/mailman/listinfo/python-list

Mocking `from foo import *` functions

[Silfheed]

So I'm in the current testing situation:

sender.py:
-
def sendEmails():
   return "I send emails"

alerter.py:
-
from sender import *
def DoStuffAndSendEmails():
  doStuff()
  sendEmails()

I'm trying to write a test fn that will test DoStuffAndSendEmails()
(as well as it's kin) without actually sending any emails out.  I
could go through alter alerter so that it does `import sender` and
then find and replace fn() with sender.fn() so I can just create a
mock fn fakeSendEmails() and and do something like sender.sendEmails =
fakeSendEmails, but I'd rather not.

Anyone know how to test alerter.py with out altering the file?

Thanks!
--
http://mail.python.org/mailman/listinfo/python-list

Re: Mocking `from foo import *` functions

[MRAB]

Silfheed wrote:

So I'm in the current testing situation:

sender.py:
-
def sendEmails():
   return "I send emails"

alerter.py:
-
from sender import *
def DoStuffAndSendEmails():
  doStuff()
  sendEmails()

I'm trying to write a test fn that will test DoStuffAndSendEmails()
(as well as it's kin) without actually sending any emails out.  I
could go through alter alerter so that it does `import sender` and
then find and replace fn() with sender.fn() so I can just create a
mock fn fakeSendEmails() and and do something like sender.sendEmails =
fakeSendEmails, but I'd rather not.

Anyone know how to test alerter.py with out altering the file?



You could alter sender.py. :-)

Actually, you could have:

alerter.py:
-
TEST = False
if TEST:
from mock_sender import *
else:
from sender import *


so you're changing alerter.py in only one place, or read the value of 
TEST from a config file.

--
http://mail.python.org/mailman/listinfo/python-list

Re: Common path all PyObjects take on destruction?

[Carl Banks]

On Jan 9, 6:39 pm, Ryan Stutsman  wrote:
> I've added a field to all PyObjects in the interpreter which is of type
> PyObject*.  Most of the time this pointer is NULL but occassionally I
> want to track some information in there.  The problem I'm running into
> is that I can add a reference to a PyObject inside any of my PyObjects,
> but it seems that there isn't any one path that all objects follow on
> destruction so that I can later Py_DECREF that reference.
>
> Eventually most of the types seem to call PyObject_Free, but this gets
> called with void* and it seems it isn't always the case that PyObject*s
> are passed in.
>
> Where is the best place to implement something like this?  It really
> won't work to implement this in the destructor of each of the individual
> types because other types added later won't know to DECREF this field on
> destruction.
>
> Any hints would be appreciated.  Hopefully I'm just missing something
> simple.


I believe no object in CPython ever gets deleted except by the
Py_DECREF macro (Py_XDECREF and Py_CLEAR both expand Py_DECREF).  So
there is your common reference point.  Let's look at the macro, shall
we?


#define Py_DECREF(op)   \
if (_Py_DEC_REFTOTAL  _Py_REF_DEBUG_COMMA   \
--((PyObject*)(op))->ob_refcnt != 0)\
_Py_CHECK_REFCNT(op)\
else\
_Py_Dealloc((PyObject *)(op))


So, if the reference count goes down to zero, Py_DECREF calls
_Py_Dealloc to delete the object.  _Py_Dealloc is the common point you
want.


Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list

Re: Mocking `from foo import *` functions

[Rob Williscroft]

Silfheed wrote in news:c73b304b-f601-4bb5-89c1-3ee667eeb7d9
@l37g2000vba.googlegroups.com in comp.lang.python:

> So I'm in the current testing situation:
> 
> sender.py:
> -
> def sendEmails():
>return "I send emails"
> 
> alerter.py:
> -
> from sender import *
> def DoStuffAndSendEmails():
>   doStuff()
>   sendEmails()
> 
> I'm trying to write a test fn that will test DoStuffAndSendEmails()
> (as well as it's kin) without actually sending any emails out.  I
> could go through alter alerter so that it does `import sender` and
> then find and replace fn() with sender.fn() so I can just create a
> mock fn fakeSendEmails() and and do something like sender.sendEmails =
> fakeSendEmails, but I'd rather not.
> 
> Anyone know how to test alerter.py with out altering the file?

Yes you alter the module *after* you have imported it.

In your test script do:

  def mock_sendEmails():
pass # or some test code maybe

  # setup ...

  import alerter
  
  # now patch the module
  
  alerter.sendEmails = mock_sendEmails 

  # run the test ...

  DoStuffAndSendEmails()

Because python is dynamic alerter.DoStuffAndSendEmails will call 
the sendEmails in the alerter module that has been replaced with
mock_sendEmails from the test script.

Rob.
-- 
http://www.victim-prime.dsl.pipex.com/
--
http://mail.python.org/mailman/listinfo/python-list

Re: "python -3" not working as expected

[Benjamin]

On Jan 8, 11:35 pm, John Machin  wrote:
> On Jan 9, 1:56 pm, Benjamin  wrote:
>
> > On Jan 8, 4:21 pm, Thorsten Kampe  wrote:
>
> > > * Terry Reedy (Thu, 08 Jan 2009 17:04:04 -0500)
> > > > Since you are, I believe, at least the second person to report being bit
> > > > by this confusion, please open an issue at bugs.python.org and suggest a
> > > > couple of revised sentences that you think are more informative.
>
> > > Will do tomorrow. The revised sentence could be in the line of "warn
> > > about Python 3.x incompatibilities that cannot trivially be fixed by
> > > 2to3.py".
>
> > Actually, don't bother now; I've fixed it up in the trunk.
>
> Would you mind giving a pointer to where or what your fix is? The
> reason for asking is that Thorsten's suggestion is ambiguous: warn
> about some? all? 3.x problems that can't be trivially fixed by 2to3?
> Can't be "all"; there are in fact a number of problems that can't be
> trivially fixed by 2to3 and can't be detected by running 2.6 with the
> -3 option.

I added "and cannot by trivially fixed by 2to3".

>
> These include
> (a) problems that cause a reasonably informative exception in 3.x
> right at the point where the problem exists
> (b) problems where the behaviour has changed but no exception is
> raised, and your code lurches off down the wrong path, and you need to
> work backwards from subsequent exception(s) and/or failing test case
> (s) to pin-point the problem.
>
> I'll use string constants to provide an example of each type. When
> faced with "abcd", 2to3 has no way of telling whether that should be
> str ("abcd") or bytes (b"abcd"). In the vast majority of cases, to
> guess it should be str is correct, so there is no change to the source
> file, and a warning would  almostly always be noise.
>
> Example of problem (a): chunks is a list of slices of bytes read from
> a binary file.
> In 2.x you write
> glued = ''.join(chunks)
> In 3.0 you get this:>>> chunks = [b'x', b'y']
> >>> ''.join(chunks)
>
> Traceback (most recent call last):
>   File "", line 1, in 
> TypeError: sequence item 0: expected str instance, bytes found
>
> Example of problem (b): some_bytes has been read from a file that was
> opened in 'rb' mode and contains the 4 ASCII bytes 'abcd'
> # 2.x simulation>> some_bytes == "abcd"
>
> True
> # 3.0 simulation>>> type(some_bytes)
> 
> >>> type("abcd")
> 
> >>> some_bytes == "abcd"
>
> False # because the types are not comparable for equality.
>
> Another type (b) example is the (majority-guessed) 2to3 change from [c]
> StringIO.StringIO to io.StringIO ... if you really should feed some
> library an io.BytesIO instance instead, it can travel quite a distance
> before blowing up.

Yes, bytes/str is an excellent example of where the third part of the
porting helpers. We'll need good documentation. Unfortunately, as you
note below, this isn't exactly the case yet.

>
> Perhaps some of this info could be put 
> intohttp://docs.python.org/dev/py3k/whatsnew/3.0.html#porting-to-python-3-0
> ... or maybe a separate HOWTO or wiki chapter could be set up for
> porting to 3.x, including topics like:
> (1) maintaining one set of source files (when you are maintaining a
> package that must run on e.g. 2.1 through 3.x)
> (2) the possibility of a 3to2 kit for those who have the 2.1 to 3.x
> support-range issue but would like to have the one set of source
> looking like 3.x code instead of the ugliness of version-conditional
> stuff like
> BYTES_NULL = bytes(0) # 3.x
> or
> BYTES_NULL = '' # 2.x
> and (3) [getting in early] what about a %to{} kit (or a {}to% kit!)?
>
> Cheers,
> John

--
http://mail.python.org/mailman/listinfo/python-list

Re: "python -3" not working as expected

[John Machin]

On Jan 10, 2:55 pm, Benjamin  wrote:
> On Jan 8, 11:35 pm, John Machin  wrote:

> > > Actually, don't bother now; I've fixed it up in the trunk.
>
> > Would you mind giving a pointer to where or what your fix is? The
> > reason for asking is that Thorsten's suggestion is ambiguous: warn
> > about some? all? 3.x problems that can't be trivially fixed by 2to3?
> > Can't be "all"; there are in fact a number of problems that can't be
> > trivially fixed by 2to3 and can't be detected by running 2.6 with the
> > -3 option.
>
> I added "and cannot by trivially fixed by 2to3".

That's what I was afraid of. Please consider changing it so that it
does not give the impression that together -3 and 2to3 cover all the
bases.

> Yes, bytes/str is an excellent example of where the third part of the
> porting helpers.

I don't understand that. What is/are "the third part of the porting
helpers"? Are there words missing from the end?


> We'll need good documentation. Unfortunately, as you
> note below, this isn't exactly the case yet.

So is there a plot to remedy this? Where do we sign up?

> > Perhaps some of this info could be put 
> > intohttp://docs.python.org/dev/py3k/whatsnew/3.0.html#porting-to-python-3-0
> > ... or maybe a separate HOWTO or wiki chapter could be set up for
> > porting to 3.x, including topics like:
> > (1) maintaining one set of source files (when you are maintaining a
> > package that must run on e.g. 2.1 through 3.x)
> > (2) the possibility of a 3to2 kit for those who have the 2.1 to 3.x
> > support-range issue but would like to have the one set of source
> > looking like 3.x code instead of the ugliness of version-conditional
> > stuff like
> > BYTES_NULL = bytes(0) # 3.x
> > or
> > BYTES_NULL = '' # 2.x
> > and (3) [getting in early] what about a %to{} kit (or a {}to% kit!)?


--
http://mail.python.org/mailman/listinfo/python-list

Re: Detecting open files and forcing closure

[Rhodri James]

On Sat, 10 Jan 2009 03:06:03 -, Andrew Robert   
wrote:


The usual scenario is that a user will leave a PDF open and then go home  
for the evening. They are simply viewing and not modifying the file.  
When the XCOPY executes, it signals a failure and subsequent scheduler  
job abend.


What I need to do is detect if files are open for viewing and force a  
close before the copy operation is attempted.


Sleeping and retrying the copy is not an option because the user will  
likely leave it open all night.


Is there a way to detect the open files and close them out?


I can't think of one off the top of my head, but this approach will cause
more anguish than you would believe possible in any case.  Imagine if the
file was open for write; you've either just corrupted it or lost the most
recent work if you force it closed.

It sounds like you're trying to implement a backup strategy.  If you are,
I'd suggest your problem is XCOPY -- you really need something more
combat capable instead.

--
Rhodri James *-* Wildebeeste Herder to the Masses
--
http://mail.python.org/mailman/listinfo/python-list

Re: Mocking `from foo import *` functions

[hsoft]

On Jan 10, 4:21 am, Silfheed  wrote:
> So I'm in the current testing situation:
>
> sender.py:
> -
> def sendEmails():
>    return "I send emails"
>
> alerter.py:
> -
> from sender import *
> def DoStuffAndSendEmails():
>   doStuff()
>   sendEmails()
>
> I'm trying to write a test fn that will test DoStuffAndSendEmails()
> (as well as it's kin) without actually sending any emails out.  I
> could go through alter alerter so that it does `import sender` and
> then find and replace fn() with sender.fn() so I can just create a
> mock fn fakeSendEmails() and and do something like sender.sendEmails =
> fakeSendEmails, but I'd rather not.
>
> Anyone know how to test alerter.py with out altering the file?
>
> Thanks!

Don't ever put testing flag in your non-code, that's very ugly and
creates a weird circular dependency between your real code and your
test code. It will give you headaches later.

To answer to Rob: yeah, sure that would work, but I always thought
mocking the imported function didn't feel right. The test then depends
on the import method of the tested module. If you later change your
mind and decide to use "import sender" and then "sender.sendEmails()",
you have to change your test code.

In my code, I have a custom TestCase class which has a method for
dealing with this stuff, here's the code relevant to your problem:

class TestCase(unittest.TestCase):
cls_tested_module = None

def run(self, result=None):
self._mocked = []
unittest.TestCase.run(self, result)
# We use reversed() so the original value is put back, even if
we mock twice.
for target, attrname, old_value in reversed(self._mocked):
setattr(target, attrname, old_value)

def mock(self, target, attrname, replace_with):
''' Replaces 'target' attribute 'attrname' with 'replace_with'
and put it back to normal at
tearDown.

The very nice thing about mock() is that it will scan
self.cls_tested_module for the
mock target and mock it as well. This is to fix the "from"
imports problem (Where even
if you mock(os, 'path'), if the tested module imported it
with "from os import path",
the mock will not work).
'''
oldvalue = getattr(target, attrname)
self._mocked.append((target, attrname, oldvalue))
setattr(target, attrname, replace_with)
if (self.cls_tested_module is not None) and
(self.cls_tested_module is not target):
for key, value in self.cls_tested_module.__dict__.iteritems
():
if value is oldvalue:
self.mock(self.cls_tested_module, key,
replace_with)

When you use it, set cls_tested_module (at the class level) to
"sender" (not the string, the module instance)
--
http://mail.python.org/mailman/listinfo/python-list

Re: Free Download - Microsoft Windows 7 Beta

[M�ta-MCI (MVP)]

Hi!

I downloaded W7 two days ago, directly on MSDN (Microsoft), and not on a 
bizarre, unknown site, and doubtful.


Python 2.6 run OK.
But the problem (well known) with Python 2.6.1 is always present.

@-salutations
--
Michel Claveau

--
http://mail.python.org/mailman/listinfo/python-list

Re: multiprocessing and SIGINT

[Gabriel Genellina]

En Fri, 09 Jan 2009 03:04:58 -0200, akineko  escribió:


(2) test program with multiprocessing
Both processes receives SIGINT.
OS apparently distributes the SIGINT event to processes associated
with the terminal.


Yes, to avoid that, the child process has to detach itself from the  
terminal. I'd expect the multiprocessing module to do that for us - but it  
doesn't. See http://www.onlamp.com/python/pythoncook2/solution.csp?day=1



(3) signal handler
I realized that I could assign a signal handler specific to a process
by placing it to a worker method.

def worker():
   # this process ignores SIGINT
   signal.signal(signal.SIGINT, signal.SIG_IGN)
   ... the rest ...


Looks fine...


(4) terminating the spawned process
I needed to send a shutdown message to the process via a communication
between two processes.
You can use Process.terminate() to brutally kill the process but that
is a last resort.


What about sending a signal, like SIGTERM?

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: Implementing file reading in C/Python

[Rhamphoryncus]

On Jan 9, 2:14 pm, Marc 'BlackJack' Rintsch  wrote:
> On Fri, 09 Jan 2009 15:34:17 +, MRAB wrote:
> > Marc 'BlackJack' Rintsch wrote:
>
> >> def iter_max_values(blocks, block_count):
> >>     for i, block in enumerate(blocks):
> >>         histogram = defaultdict(int)
> >>         for byte in block:
> >>             histogram[byte] += 1
>
> >>         yield max((count, byte)
> >>                   for value, count in histogram.iteritems())[1]
>
> > [snip]
> > Would it be faster if histogram was a list initialised to [0] * 256?
>
> Don't know.  Then for every byte in the 2 GiB we have to call `ord()`.  
> Maybe the speedup from the list compensates this, maybe not.
>
> I think that we have to to something with *every* byte of that really
> large file *at Python level* is the main problem here.  In C that's just
> some primitive numbers.  Python has all the object overhead.

struct's B format might help here.  Also, struct.unpack_from could
probably be combined with mmap to avoid copying the input.  Not to
mention that the 0..256 ints are all saved and won't be allocated/
deallocated.
--
http://mail.python.org/mailman/listinfo/python-list

Re: BadZipfile "file is not a zip file"

[Steven D'Aprano]

On Thu, 08 Jan 2009 16:47:39 -0800, webcomm wrote:

> The error...
...
> BadZipfile: File is not a zip file
> 
> When I look at data.zip in Windows, it appears to be a valid zip file. 
> I am able to uncompress it in Windows XP, and can also uncompress it
> with 7-Zip.  It looks like zipfile is not able to read a "table of
> contents" in the zip file.  That's not a concept I'm familiar with.

No, ZipFile can read table of contents:

Help on method printdir in module zipfile:

printdir(self) unbound zipfile.ZipFile method
Print a table of contents for the zip file.



In my experience, zip files originating from Windows sometimes have 
garbage at the end of the file. WinZip just ignores the garbage, but 
other tools sometimes don't -- if I recall correctly, Linux unzip 
successfully unzips the file but then complains that the file was 
corrupt. It's possible that you're running into a similar problem.


> data.zip is created in this script...
> 
> decoded = base64.b64decode(datum)
> f = open('data.zip', 'wb')
> f.write(decoded)
> f.close()
> file = zipfile.ZipFile('data.zip', "r")
> 
> datum is a base64 encoded zip file.  Again, I am able to open data.zip
> as if it's a valid zip file.  Maybe there is something wrong with the
> approach I've taken to writing the data to data.zip?  I'm not sure if it
> matters, but the zipped data is Unicode.


The full signature of ZipFile is:

ZipFile(file, mode="r", compression=ZIP_STORED, allowZip64=True)

Try passing compression=zipfile.ZIP_DEFLATED and/or allowZip64=False and 
see if that makes any difference.

The zip format does support alternative compression methods, it's 
possible that this particular file uses a different sort of compression 
which Python doesn't deal with.


> What would cause a zip file to not have a table of contents?

What makes you think it doesn't have one?


-- 
Steven
--
http://mail.python.org/mailman/listinfo/python-list

Re: ulimit stack size and python threads

[Martin v. Löwis]

> Always crashing because I asked the OS to please not allow a process
> to grow too big is what I call overloading the meaning of ulimit -s.

Please trust that there is no explicit code in the Python interpreter
that tests whether the stack size is 4GB, and then produces an explicit
crash.

> It's quite surprising. Not to mention the poor error message.

AFAICT, you didn't even *report* yet what the error message is,
so I don't know whether it is poor.

Regards,
Martin

--
http://mail.python.org/mailman/listinfo/python-list

Re: ulimit stack size and python threads

[Martin v. Löwis]

> I see. I should be blaming the default behavior of pthreads. 

You shouldn't blame anybody. Instead, you should sit down and study
the problem in detail, until you fully understand it. Then you should
start contributing fixes. Never ever should you spread blame.

Regards,
Martin
--
http://mail.python.org/mailman/listinfo/python-list

Encrypted Logging in python

[koranthala]

   I was wondering if there is a mechanism to encrypt logging
automatically in python.
   The issue is as follows:
(a) An application (after py2exe) will go as executable and there
is no need for the user to know that it is written in python. If an
exception occurs and it is logged, then the user can understand it is
written in python.
(b) A security threat. If an exception occurs, the code is seen by
the user - and possibly be misused.

   Base64 encoding somewhat helps - which is supported by logging
module - but even that is not very secure. If there can be an option -
wherein we send in the password and the logging is encrypted - it
might be better.
   I would have loved to provide the code, but I am completely tied up
at the moment and wont be able to help for another month.

--
http://mail.python.org/mailman/listinfo/python-list

Re: threading in PyQt vs threading in standard library

[Phil Thompson]

On Fri, 9 Jan 2009 15:15:28 +0800, "Steven Woody" 
wrote:
> Hi,
> 
> I am considering using PyQt for GUI programs, and I notices that both
> of them include threading supports, so which one should I pick up?
> Similar also applies 'socket'.

I'd recommend using the PyQt versions of both.

A significant advantage of PyQt's sockets is that they are integrated with
the event loop. This means you don't need threads to handle networking.

Phil
--
http://mail.python.org/mailman/listinfo/python-list

Re: BadZipfile "file is not a zip file"

[Martin v. Löwis]

> What would cause a zip file to not have a table of contents?

AFAICT, _EndRecData is failing to find the "end of zipfile" structure in
the file. You might want debug through it to see where it looks, and how
it decides that this structure is not present in the file. Towards
22 bytes before the end of the file, the bytes PK\005\006 should appear.
If they don't appear, you don't have a zipfile. If they appear, but
elsewhere towards the end of the file, there might be a bug in the
zip file module (or, more likely, the zip file uses an optional zip
feature which the module doesn't implement).

Regards,
Martin
--
http://mail.python.org/mailman/listinfo/python-list

Re: Replying to list messages

[Steven D'Aprano]

On Thu, 08 Jan 2009 10:42:13 -0800, Paul McNett wrote:

> Ben Finney wrote:
>> Paul McNett  writes:
>>> But arguing about this here isn't going to change anything: opinions
>>> differ just like tabs/spaces and bottom-post/top-post.
>> 
>> In cases like this, one side can simply be wrong :-)
>> 
>> Best of luck getting your programs behaving as you want them to!
> 
> BTW, I agree with you that in an ideal, pure world mailing lists
> wouldn't munge the reply-to field, but when 80% of the people use email
> clients that don't support reply-list, the practical thing to do as a
> list admin that wants to avoid having to explain over and over again
> that "your client software is broken" is to simply swallow some pride
> and munge the reply-to. Now, 99% of the users are happy, and the
> remaining 1% are elite enough to understand how to get around any
> problems this caused.

The only problem this leads to is people who hit send without checking 
where they are sending to are likely to be embarrassed when they send 
personal posts to the entire mailing list.

As far as I'm concerned, hitting send without looking to see who you are 
sending to is akin to turning into a busy road without looking to see if 
there are any cars coming: any accident that happens is YOUR fault, not 
the fault of the road, car, mail client or mailing list.

It boggles my brain that [insert sweeping generalisation here] the people 
who are most vehement about blaming the mailing list software are usually 
also the least understanding when (l)users click "OK" to dialogs without 
reading what the dialog says first.



-- 
Steven
--
http://mail.python.org/mailman/listinfo/python-list

Re: BadZipfile "file is not a zip file"

[Carl Banks]

On Jan 9, 2:16 am, Steven D'Aprano  wrote:
> On Thu, 08 Jan 2009 16:47:39 -0800, webcomm wrote:
> > The error...
> ...
> > BadZipfile: File is not a zip file
>
> > When I look at data.zip in Windows, it appears to be a valid zip file.
> > I am able to uncompress it in Windows XP, and can also uncompress it
> > with 7-Zip.  It looks like zipfile is not able to read a "table of
> > contents" in the zip file.  That's not a concept I'm familiar with.
>
> No, ZipFile can read table of contents:
>
>     Help on method printdir in module zipfile:
>
>     printdir(self) unbound zipfile.ZipFile method
>         Print a table of contents for the zip file.
>
> In my experience, zip files originating from Windows sometimes have
> garbage at the end of the file. WinZip just ignores the garbage, but
> other tools sometimes don't -- if I recall correctly, Linux unzip
> successfully unzips the file but then complains that the file was
> corrupt. It's possible that you're running into a similar problem.


The zipfile format is kind of brain dead, you can't tell where the end
of the file is supposed to be by looking at the header.  If the end of
file hasn't yet been reached there could be more data.  To make
matters worse, somehow zip files came to have text comments simply
appended to the end of them.  (Probably this was for the benefit of
people who would cat them to the terminal.)

Anyway, if you see something that doesn't adhere to the zipfile
format, you don't have any foolproof way to know if it's because the
file is corrupted or if it's just an appended comment.

Most zipfile readers use a heuristic to distinguish.  Python's zipfile
module just assumes it's corrupted.

The following post from a while back gives a solution that tries to
snip the comment off so that zipfile module can handle it.  It might
help you out.

http://groups.google.com/group/comp.lang.python/msg/c2008e48368c6543


Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list

Re: drive a desktop app from python?

[James Stroud]

Tim Arnold wrote:
Hi, I don't even know what to google for on this one. I need to drive a 
commercial desktop app (on windows xp) since the app doesn't have a batch 
interface.  It's intended to analyze one file at a time and display a 
report.


I can get the thing to write out the report an html browser, but I have 
thousands of files I need it to analyze every night.


Is there any lib or recipe(s) for doing something like this via python?


You are a little thin on details here. My only advice at this point is 
to check out os.system, which is the simplest option. From there you can 
move to the popen2 module for greater control, but if you are talking 
about a gui app, you may run into hang-ups.


A quick recipe would be the following pretend application that 
recursively descends from the current directory calling the utility 
named "dosapp" on every file it finds that ends with "jpg" (what 
"dosapp" does is left to your imagination):




import os

def doit(suffix, adir, filenames):
  for afile in filenames:
if afile.endswith(suffix):
  pathname = os.path.join(adir, afile)
  os.system('dosapp %s' % pathname)


top = '.'
suffix = '.jpg'

os.path.walk(top, doit, suffix)


Read the docs on os.path.walk, of course.

James


--
James Stroud
UCLA-DOE Institute for Genomics and Proteomics
Box 951570
Los Angeles, CA 90095

http://www.jamesstroud.com
--
http://mail.python.org/mailman/listinfo/python-list

Re: Best practice in organize classes into modules

[Bruno Desthuilliers]

Steven Woody a écrit :

On Fri, Jan 9, 2009 at 1:02 PM, James Mills
 wrote:

On Fri, Jan 9, 2009 at 2:57 PM, Steven Woody  wrote:

In C++/Java, people usually put one class into one file.  What's the
suggestion on this topic in Python?  I so much interesting this
especially when exception classes also involved.

Normally i group related functionality into the one module.


Will that lead to too large source file size? 


When the case happens, then you can safely refactor the module into a 
package with submodules, and use the package's __init__.py to make it a 
facade for the submodules so the refactoring is transparent for client code.


But given Python's expressivity and metaprogramming features, it's 
usually easy to avoid cruft and boilerplate and keep the code short.



Is there a
recommendation on max lines of a python source?


Not really - use your own judgement, mostly. As far as I'm concerned, I 
start worrying about this when a module grows bigger than 1Kloc, but I 
seldom have this problem.


--
http://mail.python.org/mailman/listinfo/python-list

Re: Best practice in organize classes into modules

[Steven Woody]

On Fri, Jan 9, 2009 at 4:54 PM, Bruno Desthuilliers
 wrote:
> Steven Woody a écrit :
>>
>> On Fri, Jan 9, 2009 at 1:02 PM, James Mills
>>  wrote:
>>>
>>> On Fri, Jan 9, 2009 at 2:57 PM, Steven Woody 
>>> wrote:

 In C++/Java, people usually put one class into one file.  What's the
 suggestion on this topic in Python?  I so much interesting this
 especially when exception classes also involved.
>>>
>>> Normally i group related functionality into the one module.
>>
>> Will that lead to too large source file size?
>
> When the case happens, then you can safely refactor the module into a
> package with submodules, and use the package's __init__.py to make it a
> facade for the submodules so the refactoring is transparent for client code.

really a smart idea.  Did you mean putting some 'import statement in
__init__.py and use the old module name as the new package name?
Thanks.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Implementing file reading in C/Python

[Marc 'BlackJack' Rintsch]

On Fri, 09 Jan 2009 04:04:41 +0100, Johannes Bauer wrote:

> I've first tried Python. Please don't beat me, it's slow as hell and
> probably a horrible solution:
> 
> #!/usr/bin/python
> import sys
> import os
> 
> f = open(sys.argv[1], "r")

Mode should be 'rb'.

> filesize = os.stat(sys.argv[1])[6]

`os.path.getsize()` is a little bit more readable.

> width = 1024
> height = 1024
> pixels = width * height
> blocksize = filesize / width / height
> 
> print("Filesize   : %d" % (filesize)) print("Image size : %dx%d"
> % (width, height)) print("Bytes per Pixel: %d" % (blocksize))

Why parentheses around ``print``\s "argument"?  In Python <3 ``print`` is 
a statement and not a function.

> picture = { }
> havepixels = 0
> while True:
>   data = f.read(blocksize)
>   if len(data) <= 0: break

if data:
break

is enough.

>   datamap = { }
>   for i in range(len(data)):
>   datamap[ord(data[i])] = datamap.get(data[i], 0) + 1

Here you are creating a list full of integers to use them as index into 
`data` (twice) instead of iterating directly over the elements in 
`data`.  And you are calling `ord()` for *every* byte in the file 
although you just need it for one value in each block.  If it's possible 
to write the raw PGM format this conversion wouldn't be necessary at all.

For the `datamap` a `collections.defaultdict()` might be faster.

>   maxchr = None
>   maxcnt = None
>   for (char, count) in datamap.items():
>   if (maxcnt is None) or (count > maxcnt):
>   maxcnt = count
>   maxchr = char

Untested:

maxchr = max((i, c) for c, i in datamap.iteritems())[1]

>   most = maxchr

Why?

>   posx = havepixels % width
>   posy = havepixels / width

posx, posy = divmod(havepixels, width)

Don't know if this is faster though.

>   havepixels += 1
>   if (havepixels % 1024) == 0:
>   print("Progresss %s: %.1f%%" % (sys.argv[1], 100.0 * 
havepixels /
>   pixels))
> 
>   picture[(posx, posy)] = most

Why are you using a dictionary as "2d array"?  In the C code you simply 
write the values sequentially, why can't you just use a flat list and 
append here?

Ciao,
Marc 'BlackJack' Rintsch
--
http://mail.python.org/mailman/listinfo/python-list

Re: Default __nonzero__ impl doesn't throw a TypeError exception

[Bruno Desthuilliers]

Bruno Desthuilliers a écrit :

Sergey Kishchenko a écrit :

(snip)

#prints "Ouch!"
f=Foo()
if f:
print "Ouch!"

So, default __nonzero__ impl is to return True.


Yes. It's clearly documented FWIW.


To be more exact: there's no "default __nonzero__". The boolean value of 
an object is eval'd this way:


If the object is None (special cased by the interpreter AFAICT), it is 
false.
Else if the object implements __nonzero__, it has the boolean value 
returned by __nonzero__.
Else if the object implements __len__, it has the boolean value of its 
length.

Else if is true.


I think, this
behaviour conflicts with 'Explicit is better than implicit'


Why so ? It *is* explicit that the default for an object is to have a 
true value in a boolean context.


I meant "explicit because documented", of course.



and
'Practicality beats purity'


Quite on the contrary. From a practical POV, the default truth values of 
Python objects are most of the time what you practically want in a 
boolean context - that is, any non-None object, non-empty sequence and 
non-zero numeric objects are true. __nonzero__ is here for the *very 
few* corner cases where this is not the sensible default.




statements. I think, throwing a TypeError
exception would be better.  It will result in more explicit code with
fewer errors.



As a last note wrt/ explicitness and practicality: Implementing your 
proposition, one would have to either put each and every boolean 
expression in a try/except block or "explicitly" define __nonzero__ for 
each and any class - most of the time (about 99.% I'd say) 
implementing it as to return True - and expect that *everybody* does so.


So yes, from a "purity" POV, it might look "more explicit". But this 
would certainly not be practical at all.


OTHO, since boolean algebra only knows two values, it's clear that what 
is not false is by definition true. So having a default value (true) and 
a way to override it (__len__ and __nonzero__) is perhaps less "pure", 
but really as explicit and much more practical.


My 2 cents...
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python 2.6 fails on compiling > Bug report

[googler . 1 . webmaster]

hm... any ideas?
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python 2.6 fails on compiling > Bug report

[David Cournapeau]

On Fri, Jan 9, 2009 at 6:18 PM,   wrote:
> hm... any ideas?

Posting the config.log file would be a first step to give more information,

David
--
http://mail.python.org/mailman/listinfo/python-list

Re: Implementing file reading in C/Python

[James Mills]

On Fri, Jan 9, 2009 at 7:15 PM, Marc 'BlackJack' Rintsch  wrote:
>> print("Filesize   : %d" % (filesize)) print("Image size : %dx%d"
>> % (width, height)) print("Bytes per Pixel: %d" % (blocksize))
>
> Why parentheses around ``print``\s "argument"?  In Python <3 ``print`` is
> a statement and not a function.

Not true as of 2.6+ and 3.0+

print is now a function.

cheers
James
--
http://mail.python.org/mailman/listinfo/python-list

Re: BadZipfile "file is not a zip file"

[John Machin]

On Jan 9, 7:16 pm, Steven D'Aprano  wrote:
> On Thu, 08 Jan 2009 16:47:39 -0800, webcomm wrote:
> > The error...
> ...
> > BadZipfile: File is not a zip file
>
> > When I look at data.zip in Windows, it appears to be a valid zip file.
> > I am able to uncompress it in Windows XP, and can also uncompress it
> > with 7-Zip.  It looks like zipfile is not able to read a "table of
> > contents" in the zip file.  That's not a concept I'm familiar with.
>
> No, ZipFile can read table of contents:
>
>     Help on method printdir in module zipfile:
>
>     printdir(self) unbound zipfile.ZipFile method
>         Print a table of contents for the zip file.
>
> In my experience, zip files originating from Windows sometimes have
> garbage at the end of the file. WinZip just ignores the garbage, but
> other tools sometimes don't -- if I recall correctly, Linux unzip
> successfully unzips the file but then complains that the file was
> corrupt. It's possible that you're running into a similar problem.
>
> > data.zip is created in this script...
>
> >     decoded = base64.b64decode(datum)
> >     f = open('data.zip', 'wb')
> >     f.write(decoded)
> >     f.close()
> >     file = zipfile.ZipFile('data.zip', "r")
>
> > datum is a base64 encoded zip file.  Again, I am able to open data.zip
> > as if it's a valid zip file.  Maybe there is something wrong with the
> > approach I've taken to writing the data to data.zip?  I'm not sure if it
> > matters, but the zipped data is Unicode.
>
> The full signature of ZipFile is:
>
> ZipFile(file, mode="r", compression=ZIP_STORED, allowZip64=True)
>
> Try passing compression=zipfile.ZIP_DEFLATED and/or allowZip64=False and
> see if that makes any difference.

"compression" is irrelevant when reading. The compression method used
is stored on a per-file basis, not on a per-archive basis, and it
hasn't got anywhere near per-file details when that exception is
raised. "allowZip64" has not been used either.

>
> The zip format does support alternative compression methods, it's
> possible that this particular file uses a different sort of compression
> which Python doesn't deal with.
>
> > What would cause a zip file to not have a table of contents?
>
> What makes you think it doesn't have one?
>
> --
> Steven

--
http://mail.python.org/mailman/listinfo/python-list

Re: drive a desktop app from python?

[Ant]

On Jan 8, 9:06 pm, "Tim Arnold"  wrote:
> Is there any lib or recipe(s) for doing something like this via python?

Look into the PyWin32 extension module. It gives access to Windows
internals including the COM interface. You'll need to do some research
into how to automate the GUI you're using from other sources (there
are often VBA examples on the web that can be modified).

http://pywin32.sourceforge.net/

I've successfully used this in the past for automating MS Excel.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Implementing file reading in C/Python

[Marc 'BlackJack' Rintsch]

On Fri, 09 Jan 2009 19:33:53 +1000, James Mills wrote:

> On Fri, Jan 9, 2009 at 7:15 PM, Marc 'BlackJack' Rintsch
>  wrote:

>> Why parentheses around ``print``\s "argument"?  In Python <3 ``print``
>> is a statement and not a function.
> 
> Not true as of 2.6+ and 3.0+
> 
> print is now a function.

Please read again what I wrote.

Ciao,
Marc 'BlackJack' Rintsch
--
http://mail.python.org/mailman/listinfo/python-list

Re: Implementing file reading in C/Python

[James Mills]

On Fri, Jan 9, 2009 at 7:41 PM, Marc 'BlackJack' Rintsch  wrote:
> Please read again what I wrote.

Lol I thought "<3" was a smiley! :)

Sorry!

cheers
James
--
http://mail.python.org/mailman/listinfo/python-list

Re: Implementing file reading in C/Python

[Marc 'BlackJack' Rintsch]

On Fri, 09 Jan 2009 04:04:41 +0100, Johannes Bauer wrote:

>   datamap = { }
>   for i in range(len(data)):
>   datamap[ord(data[i])] = datamap.get(data[i], 0) + 1

Here is an error by the way:  You call `ord()` just on the left side of 
the ``=``, so all keys in the dictionary are mapped to ones after the 
loop which gives a pretty boring PGM.  :-)

Ciao,
Marc 'BlackJack' Rintsch
--
http://mail.python.org/mailman/listinfo/python-list

Re: why cannot assign to function call

[Steven D'Aprano]

On Thu, 08 Jan 2009 18:33:50 +, Mark Wooding wrote:

> [Steven's message hasn't reached my server, so I'll reply to it here.
> Sorry if this is confusing.]
> 
> Aaron Brady  wrote:
>> On Jan 8, 1:45 am, Steven D'Aprano
>>  wrote:
>> > On Wed, 07 Jan 2009 10:17:55 +, Mark Wooding wrote:
>> >
>> > > The `they're just objects' model is very simple, but gets tied up
>> > > in knots explaining things.  The `it's all references' model is
>> > > only a little more complicated, but explains everything.
>> >
>> > But it *over* explains, because it implies things that "everybody
>> > knows" about references in other languages that aren't true for
>> > Python.
> 
> I addressed this elsewhere.  Summary: `pass-by-reference' is a different
> thing to `all you manipulate are references': 

You know, I've written a fair bit of Python code over the years, and I've 
never manipulated a reference *once*. Ints, strings, floats, lists, 
tuples... but references? Never.

I'm pretty sure that no other pure-Python coder has manipulated 
references either. They've manipulated objects. Whatever the VM does 
under the hood is another story.

If you insist on talking about implementations, then at least get it 
right: in Python, like every other programming language, all you do is 
flip bits. It's *all* bit flipping.

That's why we should try to keep the different layers of explanation 
separate, without conflating them. Python programmers don't actually flip 
bits, and neither do they manipulate references. Python programmers don't 
have access to bits, or references. What they have access to is objects.

(Of course, there are ways to get under the hood if you really want to.)


> Python does pass-by-value,
> but the things it passes -- by value -- are references.

If you're going to misuse pass-by-value to describe what Python does, 
*everything* is pass-by-value "where the value is foo", for some foo. You 
can't have anything but pass-by-value with current computer technology, 
because computers don't actually move arguments, they only copy bytes. So 
pass-by-value becomes a meaningless term, because it describes every 
computer language imaginable, including hypothetical ones using calling 
conventions not yet invented, and therefore explains nothing.


> (The `pass-by-*' notions are confusingly named anyway.  Pass-by-name
> doesn't actually involve names at all.)

You might find them confusing, but I don't. What I find confusing is that 
people insist on misusing terminology invented for describing one type of 
behaviour in order to use it for a completely different type of behaviour 
just because of certain similarities under the hood.



>> > Of course it's not literally true that "everybody knows" that you can
>> > use references to implement a swap(x, y) procedure. But people coming
>> > from a C or Pascal background tend to assume that everything is like
>> > C/Pascal, and there are a lot of them. If C was a rare, unfamiliar
>> > language, my opposition to using the term "reference" would be a lot
>> > milder. Maybe in another five years?
> 
> I agree with the comment about Pascal, but C is actually pretty similar
> to Python here.  C only does pass-by-value.

Except for arrays.


> If you want a function to
> modify your variable, you have to pass a pointer value which points to
> it.

Yes, because the variable is copied before the function sees it. So if 
you pass a struct, and modify one of the struct's fields, the caller 
doesn't see the change.

Now try that with Python, and you'll see completely different behaviour. 
(You'll have to use something *like* a struct, because Python doesn't 
have them. Try an object with attributes.)

In other words... C is call-by-value, and (according to you) Python is 
call-by-value, but they behaviour differently.


> Python has no pointer values, so you need a different hack.  The
> hack usually involves lists.  (Though it's easier in the main to return
> compound data objects like tuples.  I don't suppose that a proposal for
> true multiple return values would go down well here.  No, didn't think
> so...)

Out of curiosity, what makes Python returning tuples less "true" than 
"true multiple return values", and what can you do with TMRVs that you 
can't do with tuples?


>> > Okay, the abstraction has leaked again... are the paperweights
>> > references to the objects, or the names we've bound objects to? I'm
>> > confused...
> 
> They're the references to the objects.  You don't bind names to objects.

Amazing. It sure feels like it to me.

x = 23

There's a name, and an object, and I've bound the name to the object so I 
can refer to the object 23 by the name x.


>  You bind names slots in which you store references.

I'm pretty sure I don't. I'd have noticed.



You may have missed my last question:

>> > How do we deal with anonymous objects in your model?



> What I am pretty sure of is that references are going to have to enter
> the picture at some point, because othe

Re: BadZipfile "file is not a zip file"

[Steven D'Aprano]

On Fri, 09 Jan 2009 00:46:27 -0800, Carl Banks wrote:

> On Jan 9, 2:16 am, Steven D'Aprano  cybersource.com.au> wrote:
>> On Thu, 08 Jan 2009 16:47:39 -0800, webcomm wrote:
>> > The error...
>> ...
>> > BadZipfile: File is not a zip file
>>
>> > When I look at data.zip in Windows, it appears to be a valid zip
>> > file. I am able to uncompress it in Windows XP, and can also
>> > uncompress it with 7-Zip.  It looks like zipfile is not able to read
>> > a "table of contents" in the zip file.  That's not a concept I'm
>> > familiar with.
>>
>> No, ZipFile can read table of contents:
>>
>>     Help on method printdir in module zipfile:
>>
>>     printdir(self) unbound zipfile.ZipFile method
>>         Print a table of contents for the zip file.
>>
>> In my experience, zip files originating from Windows sometimes have
>> garbage at the end of the file. WinZip just ignores the garbage, but
>> other tools sometimes don't -- if I recall correctly, Linux unzip
>> successfully unzips the file but then complains that the file was
>> corrupt. It's possible that you're running into a similar problem.
> 
> 
> The zipfile format is kind of brain dead, you can't tell where the end
> of the file is supposed to be by looking at the header.  If the end of
> file hasn't yet been reached there could be more data.  To make matters
> worse, somehow zip files came to have text comments simply appended to
> the end of them.  (Probably this was for the benefit of people who would
> cat them to the terminal.)
> 
> Anyway, if you see something that doesn't adhere to the zipfile format,
> you don't have any foolproof way to know if it's because the file is
> corrupted or if it's just an appended comment.

Yes, this has lead to a nice little attack vector, using hostile Java 
classes inside JAR files (a variant of ZIP).

http://www.infoworld.com/article/08/08/01/
A_photo_that_can_steal_your_online_credentials_1.html

or http://snipurl.com/9oh0e



-- 
Steven
--
http://mail.python.org/mailman/listinfo/python-list

Fatal Python error: ceval: tstate mix-up

[Laszlo Nagy]

After upgrading my system, a program started to throw this error, and 
make a core dump:


Fatal Python error: ceval: tstate mix-up

Kernel log says:

Jan  9 05:06:49 shopzeus kernel: pid 89184 (python), uid 1024: exited on 
signal 6 (core dumped)



I found out that this can happen only when executing Python code without 
holding the GIL. My program is written entriely in Python, but uses some 
third part libraries:


py25-json
py25-psycopg2

Here is Python version:

Python 2.5.2 (r252:60911, Nov 17 2008, 22:19:20)
[GCC 4.2.1 20070719  [FreeBSD]] on freebsd7

(OS is FreeBSD 7.0 amd64 stable on dual quad core xeon 5420, it that 
matters...)


I have the code dump, but I do not know how to debug that. What should I do?

Thanks,

  Laszlo


--
http://mail.python.org/mailman/listinfo/python-list

Re: Implementing file reading in C/Python

[Steven D'Aprano]

On Fri, 09 Jan 2009 19:33:53 +1000, James Mills wrote:

> On Fri, Jan 9, 2009 at 7:15 PM, Marc 'BlackJack' Rintsch
>  wrote:
>>> print("Filesize   : %d" % (filesize)) print("Image size :
>>> %dx%d" % (width, height)) print("Bytes per Pixel: %d" % (blocksize))
>>
>> Why parentheses around ``print``\s "argument"?  In Python <3 ``print``
>> is a statement and not a function.
> 
> Not true as of 2.6+ and 3.0+
> 
> print is now a function.


Not so. print is still a statement in 2.6.

$ python2.6
Python 2.6.1 (r261:67515, Dec 24 2008, 00:33:13)
[GCC 4.1.2 20070502 (Red Hat 4.1.2-12)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 
>>> print 23
23





-- 
Steven
--
http://mail.python.org/mailman/listinfo/python-list

Re: Implementing file reading in C/Python

[Steven D'Aprano]

On Fri, 09 Jan 2009 09:15:20 +, Marc 'BlackJack' Rintsch wrote:

>> picture = { }
>> havepixels = 0
>> while True:
>>  data = f.read(blocksize)
>>  if len(data) <= 0: break
> 
> if data:
> break
> 
> is enough.


You've reversed the sense of the test. The OP exits the loop when data is 
empty, you exit the loop when it *isn't* empty.



-- 
Steven
--
http://mail.python.org/mailman/listinfo/python-list

Re: Encrypted Logging in python

[Steven D'Aprano]

On Fri, 09 Jan 2009 00:21:09 -0800, koranthala wrote:

> I was wondering if there is a mechanism to encrypt logging automatically
> in python.
>The issue is as follows:
> (a) An application (after py2exe) will go as executable and there
> is no need for the user to know that it is written in python. If an
> exception occurs and it is logged, then the user can understand it is
> written in python.
> (b) A security threat. If an exception occurs, the code is seen by
> the user - and possibly be misused.

Security by obscurity is not security. If your application isn't secure 
against people who know what language is written in, then it isn't secure.




-- 
Steven
--
http://mail.python.org/mailman/listinfo/python-list

Re: Implementing file reading in C/Python

[Marc 'BlackJack' Rintsch]

On Fri, 09 Jan 2009 04:04:41 +0100, Johannes Bauer wrote:

> As this was horribly slow (20 Minutes for a 2GB file) I coded the whole
> thing in C also:

Yours took ~37 minutes for 2 GiB here.  This "just" ~15 minutes:

#!/usr/bin/env python
from __future__ import division, with_statement
import os
import sys
from collections import defaultdict
from functools import partial
from itertools import imap


def iter_max_values(blocks, block_count):
for i, block in enumerate(blocks):
histogram = defaultdict(int)
for byte in block:
histogram[byte] += 1

yield max((count, byte)
  for value, count in histogram.iteritems())[1]

if i % 1024 == 0:
print 'Progresss: %.1f%%' % (100 * i / block_count)


def write_pgm(filename, width, height, pixel_values):
with open(filename, 'w') as pgm_file:
pgm_file.write('P2\n'
'# CREATOR: Crappyass Python Script\n'
'%d %d\n'
'255\n' % (width, height))
pgm_file.writelines('%d\n' % value for value in pixel_values)


def main():
filename = sys.argv[1]
filesize = os.path.getsize(filename)

width = 1024
height = 1024
pixels = width * height
blocksize = filesize // width // height

print 'Filesize   : %d' % filesize
print 'Image size : %dx%d' % (width, height)
print 'Bytes per Pixel: %d' % blocksize

with open(filename, 'rb') as data_file:
blocks = iter(partial(data_file.read, blocksize), '')
pixel_values = imap(ord, iter_max_values(blocks, pixels))
write_pgm(filename + '.pgm', width, height, pixel_values)


if __name__ == '__main__':
main()


Ciao,
Marc 'BlackJack' Rintsch
--
http://mail.python.org/mailman/listinfo/python-list

Re: Best practice in organize classes into modules

[Bruno Desthuilliers]

Steven Woody a écrit :

On Fri, Jan 9, 2009 at 4:54 PM, Bruno Desthuilliers
 wrote:

Steven Woody a écrit :

On Fri, Jan 9, 2009 at 1:02 PM, James Mills
 wrote:

On Fri, Jan 9, 2009 at 2:57 PM, Steven Woody 
wrote:

In C++/Java, people usually put one class into one file.  What's the
suggestion on this topic in Python?  I so much interesting this
especially when exception classes also involved.

Normally i group related functionality into the one module.

Will that lead to too large source file size?

When the case happens, then you can safely refactor the module into a
package with submodules, and use the package's __init__.py to make it a
facade for the submodules so the refactoring is transparent for client code.


really a smart idea.


Mostly a common Python idiom.


 Did you mean putting some 'import statement in
__init__.py and use the old module name as the new package name?


Yes, you guessed.

--
http://mail.python.org/mailman/listinfo/python-list

Re: Implementing file reading in C/Python

[Steve Holden]

Marc 'BlackJack' Rintsch wrote:
> On Fri, 09 Jan 2009 04:04:41 +0100, Johannes Bauer wrote:
[...]
>> print("Filesize   : %d" % (filesize)) print("Image size : %dx%d"
>> % (width, height)) print("Bytes per Pixel: %d" % (blocksize))
> 
> Why parentheses around ``print``\s "argument"?  In Python <3 ``print`` is 
> a statement and not a function.
> 
Portability?

regards
 Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

--
http://mail.python.org/mailman/listinfo/python-list

Re: BadZipfile "file is not a zip file"

[John Machin]

On Jan 9, 7:46 pm, Carl Banks  wrote:
> On Jan 9, 2:16 am, Steven D'Aprano 
>
>
>
>
> cybersource.com.au> wrote:
> > On Thu, 08 Jan 2009 16:47:39 -0800, webcomm wrote:
> > > The error...
> > ...
> > > BadZipfile: File is not a zip file
>
> > > When I look at data.zip in Windows, it appears to be a valid zip file.
> > > I am able to uncompress it in Windows XP, and can also uncompress it
> > > with 7-Zip.  It looks like zipfile is not able to read a "table of
> > > contents" in the zip file.  That's not a concept I'm familiar with.
>
> > No, ZipFile can read table of contents:
>
> >     Help on method printdir in module zipfile:
>
> >     printdir(self) unbound zipfile.ZipFile method
> >         Print a table of contents for the zip file.
>
> > In my experience, zip files originating from Windows sometimes have
> > garbage at the end of the file. WinZip just ignores the garbage, but
> > other tools sometimes don't -- if I recall correctly, Linux unzip
> > successfully unzips the file but then complains that the file was
> > corrupt. It's possible that you're running into a similar problem.
>
> The zipfile format is kind of brain dead, you can't tell where the end
> of the file is supposed to be by looking at the header.  If the end of
> file hasn't yet been reached there could be more data.  To make
> matters worse, somehow zip files came to have text comments simply
> appended to the end of them.  (Probably this was for the benefit of
> people who would cat them to the terminal.)
>
> Anyway, if you see something that doesn't adhere to the zipfile
> format, you don't have any foolproof way to know if it's because the
> file is corrupted or if it's just an appended comment.
>
> Most zipfile readers use a heuristic to distinguish.  Python's zipfile
> module just assumes it's corrupted.
>
> The following post from a while back gives a solution that tries to
> snip the comment off so that zipfile module can handle it.  It might
> help you out.
>
> http://groups.google.com/group/comp.lang.python/msg/c2008e48368c6543

And here's a little gadget that might help the diagnostic effort; it
shows the archive size and the position of all the "magic" PKnn
markers. In a "normal" uncommented archive, EndArchive_pos + 22 ==
archive_size.
8<---
# usage: python zip_susser.py name_of_archive.zip
import sys
grimoire = [
("FileHeader",  "PK\003\004"), # magic number for file
header
("CentralDir",  "PK\001\002"), # magic number for central
directory
("EndArchive",  "PK\005\006"), # magic number for end of
archive record
("EndArchive64","PK\x06\x06"), # magic token for Zip64
header
("EndArchive64Locator", "PK\x06\x07"), # magic token for locator
header
]
f = open(sys.argv[1], 'rb')
buff = f.read()
f.close()
blen = len(buff)
print "archive size is", blen
for magic_name, magic in grimoire:
pos = 0
while pos < blen:
pos = buff.find(magic, pos)
if pos < 0:
break
print "%s at %d" % (magic_name, pos)
pos += 4
8<---

HTH,
John
--
http://mail.python.org/mailman/listinfo/python-list

Re: Implementing file reading in C/Python

[Steve Holden]

Steven D'Aprano wrote:
> On Fri, 09 Jan 2009 19:33:53 +1000, James Mills wrote:
> 
>> On Fri, Jan 9, 2009 at 7:15 PM, Marc 'BlackJack' Rintsch
>>  wrote:
 print("Filesize   : %d" % (filesize)) print("Image size :
 %dx%d" % (width, height)) print("Bytes per Pixel: %d" % (blocksize))
>>> Why parentheses around ``print``\s "argument"?  In Python <3 ``print``
>>> is a statement and not a function.
>> Not true as of 2.6+ and 3.0+
>>
>> print is now a function.
> 
> 
> Not so. print is still a statement in 2.6.
> 
> $ python2.6
> Python 2.6.1 (r261:67515, Dec 24 2008, 00:33:13)
> [GCC 4.1.2 20070502 (Red Hat 4.1.2-12)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
 print 23
> 23
> 
C:\Users\sholden>\python26\python
Python 2.6 (r26:66721, Oct  2 2008, 11:35:03) [MSC v.1500 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> print

>>>

OK, I confess I missed out

>>> from __future__ import print_function

regards
 Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

--
http://mail.python.org/mailman/listinfo/python-list

Line completion with custom commands

[gu]

Hi, my Python program can be launched with a range of different options 
(or subcommands) like:


$ myProgram doSomething
$ myProgram doSomethingElse
$ myProgram nowDoSomethingDifferent

I want it to use auto-completion with so that if i type "myProgram d" it 
returns "myProgram doSomething" and if i type "myProgram n" it renders 
"myProgram nowDoSomethingDifferent". This is similar to the average use 
of the module rlcompleter, but it does not pick possible completion 
options from the filesystem (or from history) but from a custom set of 
strings (that correspond to the available options for my program)


Any idea on how to implement this?

I'm aware of the variable PYTHONSTARTUP (that should point to a file I 
don't know how to write).


As a working example, django-admin (from the django package) has the 
same exact feature i'm looking for

--
http://mail.python.org/mailman/listinfo/python-list

Re: [Python-Dev] compiling python2.5 on linux under wine

[Luke Kenneth Casson Leighton]

On Thu, Jan 8, 2009 at 9:07 PM, "Martin v. Löwis"  wrote:
>>  i'd just ... much rather be completely independent of proprietary
>> software when it comes to building free software.
>
> I guess my question is then: why do you want to use Windows in the
> first place?

 ha ha :)  the same question was asked when i started the nt domains
reverse-engineering for samba, in 1996.  the answer is: i don't.  but
there are a lot of users and developers who feel that they don't have
a choice.  or haven't been given one.

 so if it's possible for me, as one of the "under 1% of computer users
i.e. linux" to compile stuff that will work on the "over 95% of
computers used by everyone else i.e. windows" _and_ i get to stick to
free software principles, that's gotta be good.

 take pywebkit-gtk as an example.

 the first-level (and some of the second-level) dependencies for
pywebkit-gtk are roughly as follows:

 * libstdc++
 * cairo, pango, gdk, fontconfig, gtk
 * libxml2 (which is dodgy)
 * libxslt1 (which is so dodgy and dependent on incompatible versions
of libxml2 it can't be compiled on win32)
 * libicu38
 * libcurl
 * libssl
 * webkit
 * python2.5
 * python-gobect
 * python-gtk

 that's a *big* ing list that comes in at a whopping 40mb of
_binaries_.  webkit itself comes in at 10mb alone.

 libicu38 fails _miserably_ to cross-compile with mingw32.  i was damn
lucky to have beaten it into submission: it took two days and i
couldn't run any of the tests, but actually managed to get at least
some .libs, .dlls and .a's out of the mess.

  libxslt1 and libxml2 have compile errors in mutually incompatible
versions on win32, plus, unfortunately, the versions that _do_ compile
correctly (really old versions like libxslt-1.12 + libxml2-18 or
something) are not the ones that can be used on webkit!

 i had to get the source code for gcc (4.4) because when linking
webkit against the MSVC-compiled libicu38 gcc actually segfaulted (!).
 and that was tracked down to exception handling across process /
thread boundaries in libstdc++-6 which had only literally been
fixed/patched a few days before i started the monster-compile-process.

 i tried hunting down python-gobject and python-gtk for win32, but
there is a dependency needed before you get to that: python25.lib.
as i mentioned previously i tried hunting down a .lib for python25 but
of course that would be useless unless i also have a libtool-compiled
.a so there wasn't any point.

 so, all the hard work that i did cross-compiling up webkit for win32
was completely wasted because python itself could not be compiled on
linux for a win32 platform.

hence my interest in making sure that it can be.

_then_ i can go back and revisit the monster compile process and
finally come up with the goods, on win32, on the gobject-based
DOM-model manipulation stuff i've added to pywebkit-gtk.  i've got
linux covered, i've got macosx covered.  win32 is the last one.

l.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Implementing file reading in C/Python

[mk]

Johannes Bauer wrote:


Which takes about 40 seconds. I want the niceness of Python but a little
more speed than I'm getting (I'd settle for factor 2 or 3 slower, but
factor 30 is just too much).


This probably doesn't contribute much, but have you tried using Python 
profiler? You might have *something* wrong that eats up a lot of time in 
the code.


The factor of 30 indeed does not seem right -- I have done somewhat 
similar stuff (calculating Levenshtein distance [edit distance] on words 
read from very large files), coded the same algorithm in pure Python and 
C++ (using linked lists in C++) and Python version was 2.5 times slower.


Regards,
mk


--
http://mail.python.org/mailman/listinfo/python-list

Re: Line completion with custom commands

[Steve Holden]

gu wrote:
> 
> Hi, my Python program can be launched with a range of different options
> (or subcommands) like:
> 
> $ myProgram doSomething
> $ myProgram doSomethingElse
> $ myProgram nowDoSomethingDifferent
> 
> I want it to use auto-completion with so that if i type "myProgram d" it
> returns "myProgram doSomething" and if i type "myProgram n" it renders
> "myProgram nowDoSomethingDifferent". This is similar to the average use
> of the module rlcompleter, but it does not pick possible completion
> options from the filesystem (or from history) but from a custom set of
> strings (that correspond to the available options for my program)
> 
> Any idea on how to implement this?
> 
> I'm aware of the variable PYTHONSTARTUP (that should point to a file I
> don't know how to write).
> 
> As a working example, django-admin (from the django package) has the
> same exact feature i'm looking for

The issue here is that Python doesn't get control until afer you've hit
RETURN on the command line. so nothing you can do in your program or
interpreter setup will have any effect on how the command shell behaves.

regards
 Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

--
http://mail.python.org/mailman/listinfo/python-list

Re: figuring week of the day....

[Tim Chase]

Tim Chase wrote:

tekion wrote:

Is there a module where you could figure week of the day, like where
it starts and end. I need to do this for a whole year. Thanks.


the monthcalendar() call returns the whole month's calendar which 
may be more what you want for the big-picture.


And if you want a whole year's worth, you can get pretty close with:

  import itertools as i
  import calendar as c
  for month in range(1,13):
for week in c.monthcalendar(2009, month):
  print repr(w)

You don't detail how you want the month-boundaries to behave, so 
this gives "calendar"'s default behavior of filling in zeros on 
month-boundaries, so November through the 1st week in Dec 2009 
comes back as


  ...
  [0, 0, 0, 0, 0, 0, 1],
  [2, 3, 4, 5, 6, 7, 8],
  [9, 10, 11, 12, 13, 14, 15],
  [16, 17, 18, 19, 20, 21, 22],
  [23, 24, 25, 26, 27, 28, 29],
  [30, 0, 0, 0, 0, 0, 0],
  [0, 1, 2, 3, 4, 5, 6],
  ...

rather than

  ...
  [26, 27, 28, 29, 30, 31, 1],
  [2, 3, 4, 5, 6, 7, 8],
  [9, 10, 11, 12, 13, 14, 15],
  [16, 17, 18, 19, 20, 21, 22],
  [23, 24, 25, 26, 27, 28, 29],
  [30, 1, 2, 3, 4, 5, 6],
  ...

-tkc



--
http://mail.python.org/mailman/listinfo/python-list

Re: Line completion with custom commands

[gu]

Steve Holden wrote:

gu wrote:

Hi, my Python program can be launched with a range of different options
(or subcommands) like:

$ myProgram doSomething
$ myProgram doSomethingElse
$ myProgram nowDoSomethingDifferent

I want it to use auto-completion with so that if i type "myProgram d" it
returns "myProgram doSomething" and if i type "myProgram n" it renders
"myProgram nowDoSomethingDifferent". This is similar to the average use
of the module rlcompleter, but it does not pick possible completion
options from the filesystem (or from history) but from a custom set of
strings (that correspond to the available options for my program)

Any idea on how to implement this?

I'm aware of the variable PYTHONSTARTUP (that should point to a file I
don't know how to write).

As a working example, django-admin (from the django package) has the
same exact feature i'm looking for


The issue here is that Python doesn't get control until afer you've hit
RETURN on the command line. so nothing you can do in your program or
interpreter setup will have any effect on how the command shell behaves.

regards
 Steve


I see, but how does django-admin work, then?

--
http://mail.python.org/mailman/listinfo/python-list

Re: Line completion with custom commands

[Marco Mariani]

gu wrote:


I see, but how does django-admin work, then?


from bash:

complete -W "doSomething doSomethingElse doSomethingDifferent" myProgram
--
http://mail.python.org/mailman/listinfo/python-list

Re: why cannot assign to function call

[Aaron Brady]

On Jan 9, 4:01 am, Steven D'Aprano  wrote:
> On Thu, 08 Jan 2009 18:33:50 +, Mark Wooding wrote:
> > [Steven's message hasn't reached my server, so I'll reply to it here.
> > Sorry if this is confusing.]
>
> > Aaron Brady  wrote:
> >> On Jan 8, 1:45 am, Steven D'Aprano
> >>  wrote:
> >> > On Wed, 07 Jan 2009 10:17:55 +, Mark Wooding wrote:
>
> >> > > The `they're just objects' model is very simple, but gets tied up
> >> > > in knots explaining things.  The `it's all references' model is
> >> > > only a little more complicated, but explains everything.
>
> >> > But it *over* explains, because it implies things that "everybody
> >> > knows" about references in other languages that aren't true for
> >> > Python.
>
> > I addressed this elsewhere.  Summary: `pass-by-reference' is a different
> > thing to `all you manipulate are references':
>
> You know, I've written a fair bit of Python code over the years, and I've
> never manipulated a reference *once*. Ints, strings, floats, lists,
> tuples... but references? Never.
snip
> That's why we should try to keep the different layers of explanation
> separate, without conflating them. Python programmers don't actually flip
> bits, and neither do they manipulate references. Python programmers don't
> have access to bits, or references. What they have access to is objects.
snip
> >> > How do we deal with anonymous objects in your model?
> > What I am pretty sure of is that references are going to have to enter
> > the picture at some point, because other models get too complicated.
>
> Well, I dare say that at *some* point all models are insufficient. The
> map is not the territory, and there's always something that gets left
> out. But I think your model with strings is more complicated: robots,
> sticky Blu-Tack, string that you can't touch or see, and so forth.
> Compared to that, TARDIS technology enabling objects to be in two places
> at once is remarkably straightforward. Despite it being physically
> unrealistic, it's logically simple.
>
> --
> Steven

Possible compromise.  You can think of functions as mutation-only.
You pass the object, and it gets a new (additional) name.  The old
name doesn't go in.  

Regardless, IMO, references don't add any explanatory power; they just
make you feel cocky that you know what they are.

M: If 'fun()' returned a reference, you would be able to assign to it.
m: You can't assign to it.
C: It doesn't return a reference.

-- Why can't I assign to a function call?
-- Python variables are references only.
-- Ok, why can't I assign to a function call?
-- [Explanation Steven is trying to give.]

In other words, Mark and Joe, cut to the chase.  You're shirking.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Line completion with custom commands

[Steve Holden]

gu wrote:
> Steve Holden wrote:
>> gu wrote:
>>> Hi, my Python program can be launched with a range of different options
>>> (or subcommands) like:
>>>
>>> $ myProgram doSomething
>>> $ myProgram doSomethingElse
>>> $ myProgram nowDoSomethingDifferent
>>>
>>> I want it to use auto-completion with so that if i type "myProgram d" it
>>> returns "myProgram doSomething" and if i type "myProgram n" it renders
>>> "myProgram nowDoSomethingDifferent". This is similar to the average use
>>> of the module rlcompleter, but it does not pick possible completion
>>> options from the filesystem (or from history) but from a custom set of
>>> strings (that correspond to the available options for my program)
>>>
>>> Any idea on how to implement this?
>>>
>>> I'm aware of the variable PYTHONSTARTUP (that should point to a file I
>>> don't know how to write).
>>>
>>> As a working example, django-admin (from the django package) has the
>>> same exact feature i'm looking for
>>
>> The issue here is that Python doesn't get control until afer you've hit
>> RETURN on the command line. so nothing you can do in your program or
>> interpreter setup will have any effect on how the command shell behaves.
>>
>> regards
>>  Steve
> 
> I see, but how does django-admin work, then?
> 
Probably the issue you has was not knowing the name of the feature. When
I asked Google to tell me about django-admin command completion it said:

  http://code.djangoproject.com/ticket/1240

as the first hit. As you will see, this is done by configuring the shell
to know about the specific commands it has to deal with. Enjoy!

regards
 Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

--
http://mail.python.org/mailman/listinfo/python-list

Re: Line completion with custom commands

[gu]

Marco Mariani wrote:

gu wrote:


I see, but how does django-admin work, then?


from bash:

complete -W "doSomething doSomethingElse doSomethingDifferent" myProgram


This worked like a charm, thank you so much. Is this available for bash 
only or any shell?

--
http://mail.python.org/mailman/listinfo/python-list

Re: [Help] The pywinauto Can't select the MDI's menu using the MenuItems() which return [].

[Korobase]

2008/12/22 Simon Brunning 

> 2008/12/21  :
> > The code below opens the Choose Font dialog on my Spanish Windows
> version:
> >
> > py> from pywinauto.application import Application
> > py> app = Application.start("Notepad.exe")
>
> Notepad's menus are build with MFC. Word's menus are not. Trust me,
> give it up. For automating Word, COM (with Mark Hammond's excellent
> Python/COM bridge) is the only way to go.


Yeah.
But the GuiLib program also don't work.
You can dowload sample from http://www.beyondata.com/
and use the pywinauto to test it.
I want to know how to do the GuiLib's automation.


>
>
> --
> Cheers,
> Simon B.
> si...@brunningonline.net
> --
> http://mail.python.org/mailman/listinfo/python-list
>



-- 
"OpenBookProject"-开放图书计划邮件列表
详情: http://groups.google.com/group/OpenBookProject
维基: http://wiki.woodpecker.org.cn/
--
http://mail.python.org/mailman/listinfo/python-list

Re: drive a desktop app from python?

[zugn...@gmail.com]

> Is there any lib or recipe(s) for doing something like this via python?
>

I've used pywinauto to do something similar, and found it good.

http://pywinauto.openqa.org/


--
http://mail.python.org/mailman/listinfo/python-list

Re: Unexpected scientific notation

[Mark Dickinson]

On Jan 8, 1:00 am, Paul McNett  wrote:
> It displays '3E+1' instead of '30.0'.
>
> As I can't reproduce I'm looking for an idea brainstorm of what could be 
> causing
> this. What would be choosing to display such a normal number in scientific 
> notation?
>
> Ideas?

[I thought I replied to this earlier, but the post isn't showing up.
So here it is again.]

I suspect it's your use of the Decimal normalize() method that's
causing
this.  Trailing zeros on Decimal instances are significant, so
Decimal('30.0'), Decimal('30') and Decimal('3E+1') are considered
distinct (though they all have the same value).  The normalize method
strips all trailing zeros, turning Decimal('30.0') into Decimal('3E
+1').

One way to get around this is to add 0 after normalizing: this will
make sure that scientific notation is used only for very large
or small numbers, as usual.

Python 2.7a0 (trunk:68298:68318, Jan  6 2009, 10:39:14)
[GCC 4.0.1 (Apple Inc. build 5490)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from decimal import Decimal
>>> 0 + Decimal('3E1')
Decimal('30')
>>> Decimal('0.0') + Decimal('3E1')
Decimal('30.0')

Adding 0 also has the side-effect of turning a negative zero
into a positive zero, but I suspect that this isn't going to
worry you much.  :)

You might also want to look at the Decimal.quantize method.

Mark
--
http://mail.python.org/mailman/listinfo/python-list

Re: Fatal Python error: ceval: tstate mix-up

[Laszlo Nagy]

I could start "gdb python python.core" but don't know what it means.
Unfortunately, there are no debugging symbols.


%gdb /usr/local/bin/python python.core
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...(no debugging
symbols found)...
Core was generated by `python'.
Program terminated with signal 6, Aborted.
Reading symbols from /lib/libutil.so.7...(no debugging symbols
found)...done.
Loaded symbols for /lib/libutil.so.7
Reading symbols from /lib/libm.so.5...(no debugging symbols found)...done.
Loaded symbols for /lib/libm.so.5
Reading symbols from /lib/libthr.so.3...(no debugging symbols found)...done.
Loaded symbols for /lib/libthr.so.3
Reading symbols from /lib/libc.so.7...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.7
Reading symbols from /usr/local/lib/python2.5/lib-dynload/time.so...(no
debugging symbols found)...done.
Loaded symbols for /usr/local/lib/python2.5/lib-dynload/time.so
Reading symbols from
/usr/local/lib/python2.5/lib-dynload/collections.so...(no debugging
symbols found)...done.
Loaded symbols for /usr/local/lib/python2.5/lib-dynload/collections.so
Reading symbols from
/usr/local/lib/python2.5/lib-dynload/datetime.so...(no debugging symbols
found)...done.
Loaded symbols for /usr/local/lib/python2.5/lib-dynload/datetime.so
Reading symbols from
/usr/local/lib/python2.5/lib-dynload/cStringIO.so...(no debugging
symbols found)...done.
Loaded symbols for /usr/local/lib/python2.5/lib-dynload/cStringIO.so
Reading symbols from /usr/local/lib/python2.5/lib-dynload/strop.so...(no
debugging symbols found)...done.
Loaded symbols for /usr/local/lib/python2.5/lib-dynload/strop.so
Reading symbols from
/usr/local/lib/python2.5/lib-dynload/operator.so...(no debugging symbols
found)...done.
Loaded symbols for /usr/local/lib/python2.5/lib-dynload/operator.so
Reading symbols from
/usr/local/lib/python2.5/lib-dynload/_struct.so...(no debugging symbols
found)...done.
Loaded symbols for /usr/local/lib/python2.5/lib-dynload/_struct.so
Reading symbols from
/usr/local/lib/python2.5/lib-dynload/binascii.so...(no debugging symbols
found)...done.
Loaded symbols for /usr/local/lib/python2.5/lib-dynload/binascii.so
Reading symbols from
/usr/local/lib/python2.5/site-packages/sgmlop.so...(no debugging symbols
found)...done.
Loaded symbols for /usr/local/lib/python2.5/site-packages/sgmlop.so
Reading symbols from
/usr/local/lib/python2.5/lib-dynload/pyexpat.so...(no debugging symbols
found)...done.
Loaded symbols for /usr/local/lib/python2.5/lib-dynload/pyexpat.so
Reading symbols from
/usr/local/lib/python2.5/lib-dynload/_socket.so...(no debugging symbols
found)...done.
Loaded symbols for /usr/local/lib/python2.5/lib-dynload/_socket.so
Reading symbols from /usr/local/lib/python2.5/lib-dynload/_ssl.so...(no
debugging symbols found)...done.
Loaded symbols for /usr/local/lib/python2.5/lib-dynload/_ssl.so
Reading symbols from /usr/lib/libssl.so.5...(no debugging symbols
found)...done.
Loaded symbols for /usr/lib/libssl.so.5
Reading symbols from /lib/libcrypto.so.5...(no debugging symbols
found)...done.
Loaded symbols for /lib/libcrypto.so.5
Reading symbols from /usr/local/lib/python2.5/lib-dynload/math.so...(no
debugging symbols found)...done.
Loaded symbols for /usr/local/lib/python2.5/lib-dynload/math.so
Reading symbols from
/usr/local/lib/python2.5/lib-dynload/_random.so...(no debugging symbols
found)...done.
Loaded symbols for /usr/local/lib/python2.5/lib-dynload/_random.so
Reading symbols from /usr/local/lib/python2.5/lib-dynload/fcntl.so...(no
debugging symbols found)...done.
Loaded symbols for /usr/local/lib/python2.5/lib-dynload/fcntl.so
Reading symbols from
/usr/local/lib/python2.5/lib-dynload/select.so...(no debugging symbols
found)...done.
Loaded symbols for /usr/local/lib/python2.5/lib-dynload/select.so
Reading symbols from
/usr/local/lib/python2.5/site-packages/OpenSSL/rand.so...(no debugging
symbols found)...done.
Loaded symbols for /usr/local/lib/python2.5/site-packages/OpenSSL/rand.so
Reading symbols from
/usr/local/lib/python2.5/site-packages/OpenSSL/crypto.so...(no debugging
symbols found)...done.
Loaded symbols for /usr/local/lib/python2.5/site-packages/OpenSSL/crypto.so
Reading symbols from
/usr/local/lib/python2.5/site-packages/OpenSSL/SSL.so...(no debugging
symbols found)...done.
Loaded symbols for /usr/local/lib/python2.5/site-packages/OpenSSL/SSL.so
Reading symbols from
/usr/local/lib/python2.5/lib-dynload/cPickle.so...(no debugging symbols
found)...done.
Loaded symbols for /usr/local/lib/python2.5/lib-dynload/cPickle.so
Reading symbols from
/usr/local/lib/python2.5/lib-dynload/_weakref.so...(no debugging symbols
found)...done.
Loaded 

pySerial - accessing GSM module failed

[Dave Dave]

Hello all,
I'm newbie in the serial buissness and I beed some Help. I'm a student at my
last year and I got final assaignment.
My goal is to comunicate with SIM free, GSM Module through computer. I want
to simulate SIM card by receiving and transferring data from my code.

In order to understand how to comunicate with the GSM module I took an
existing module which gut RS232 connector and connected it to the computer.
I ran my python code:
class MySerial():
 def __init__ (self,port,baundrate,)
  self.s = serial.Serial(port,baundreate,)

 def read(self,size=1):
   txt = self.s.read(size)

def write(self,text=''):txt = self.s.write(text)

def main():
  obj = MySerial('COM1')
  obj.write('ATI')
   sleep(1)
  obj.read(10)

the code run and I send the comman 'ATI' which ask the GSM Module if
everything is OK.
the read line returs 'ATI'. echo to my write function.

Whenever I run this comman ('ATI') through hyperterminal I gut the result :
232 OK

Does any one got an Idea why my code isn't getting the same answer from the
GSM Module?
thanks
dave
--
http://mail.python.org/mailman/listinfo/python-list

Re: Implementing file reading in C/Python

[Johannes Bauer]

Marc 'BlackJack' Rintsch schrieb:

>> f = open(sys.argv[1], "r")
> 
> Mode should be 'rb'.

Check.

>> filesize = os.stat(sys.argv[1])[6]
> 
> `os.path.getsize()` is a little bit more readable.

Check.

>> print("Filesize   : %d" % (filesize)) print("Image size : %dx%d"
>> % (width, height)) print("Bytes per Pixel: %d" % (blocksize))
> 
> Why parentheses around ``print``\s "argument"?  In Python <3 ``print`` is 
> a statement and not a function.

I write all new code to work under Python3.0. Actually I develop on
Python 3.0 but the code is currently deployed onto 2.6.

>> picture = { }
>> havepixels = 0
>> while True:
>>  data = f.read(blocksize)
>>  if len(data) <= 0: break
> 
> if data:
> break
> 
> is enough.
> 
>>  datamap = { }
>>  for i in range(len(data)):
>>  datamap[ord(data[i])] = datamap.get(data[i], 0) + 1
> 
> Here you are creating a list full of integers to use them as index into 
> `data` (twice) instead of iterating directly over the elements in 
> `data`.  And you are calling `ord()` for *every* byte in the file 
> although you just need it for one value in each block.  If it's possible 
> to write the raw PGM format this conversion wouldn't be necessary at all.

OK, those two are just stupid, you're right. I changed it to:

datamap = { }
for i in data:
datamap[i] = datamap.get(i, 0) + 1

array = sorted([(b, a) for (a, b) in datamap.items()], reverse=True)
most = ord(array[0][1])
pic.write("%d\n" % (most))


> For the `datamap` a `collections.defaultdict()` might be faster.

Tried that, not much of a change.

>>  maxchr = None
>>  maxcnt = None
>>  for (char, count) in datamap.items():
>>  if (maxcnt is None) or (count > maxcnt):
>>  maxcnt = count
>>  maxchr = char
> 
> Untested:
> 
> maxchr = max((i, c) for c, i in datamap.iteritems())[1]

This is nice, I use it - the sort thing was a workaround anyways.

>>  most = maxchr
> 
> Why?

I don't really know anymore :-\

>>  posx = havepixels % width
>>  posy = havepixels / width
> 
> posx, posy = divmod(havepixels, width)

That's a nice one.

> Why are you using a dictionary as "2d array"?  In the C code you simply 
> write the values sequentially, why can't you just use a flat list and 
> append here?

Yup, I changed the Python code to behave the same way the C code did -
however overall it's not much of an improvement: Takes about 15 minutes
to execute (still factor 23).

Thanks for all your pointers!

Kind regards,
Johannes

-- 
"Meine Gegenklage gegen dich lautet dann auf bewusste Verlogenheit,
verlästerung von Gott, Bibel und mir und bewusster Blasphemie."
 -- Prophet und Visionär Hans Joss aka HJP in de.sci.physik
 <48d8bf1d$0$7510$54022...@news.sunrise.ch>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Implementing file reading in C/Python

[Johannes Bauer]

James Mills schrieb:

> What does this little tool do anyway ?
> It's very interesting the images it creates
> out of files. What is this called ?

It has no particular name. I was toying around with the Princeton Cold
Boot Attack (http://citp.princeton.edu/memory/). In particular I was
interested in how much memory is erased when I would (on my system)
enable the slow POST (which counts through all RAM three times).

I downloaded the provided utitilities, dumped my system memory via PXE
boot onto another system after resetting it hard in the middle of a
running Linux session. I did sync, though. Praise all journaling
filesystems.

As a 2GB file is not really of much use for telling where something is
and where isn't, I thought of that picture coloring. In a 1024x1024
picture a pixel is 2048 bytes with 2GB of RAM, so exactly half a page.
This is sufficiently high resolution to detect what's in there.

> I'm curious :) I haven't had much tiem to
> optimize it yet - I'll try to when I get home from work.

Thanks for your effort, I appreciate it... hope my work leads to some
meaningful results. Currently it looks (*cough* if there aren't bugs in
my picture code) as if my PC would reset the whole RAM anyways, although
I do not have any ECC. Strange.

Kind regards,
Johannes

-- 
"Meine Gegenklage gegen dich lautet dann auf bewusste Verlogenheit,
verlästerung von Gott, Bibel und mir und bewusster Blasphemie."
 -- Prophet und Visionär Hans Joss aka HJP in de.sci.physik
 <48d8bf1d$0$7510$54022...@news.sunrise.ch>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Implementing file reading in C/Python

[Johannes Bauer]

Marc 'BlackJack' Rintsch schrieb:
> On Fri, 09 Jan 2009 04:04:41 +0100, Johannes Bauer wrote:
> 
>> As this was horribly slow (20 Minutes for a 2GB file) I coded the whole
>> thing in C also:
> 
> Yours took ~37 minutes for 2 GiB here.  This "just" ~15 minutes:

Ah, ok... when implementing your suggestions int he other post, I did
not get such a drastic performance increase. I really will have a look
at it and try to locate where I'm wasting the time.

Thanks a lot,
Kind regards,
Johannes

-- 
"Meine Gegenklage gegen dich lautet dann auf bewusste Verlogenheit,
verlästerung von Gott, Bibel und mir und bewusster Blasphemie."
 -- Prophet und Visionär Hans Joss aka HJP in de.sci.physik
 <48d8bf1d$0$7510$54022...@news.sunrise.ch>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Implementing file reading in C/Python

[Johannes Bauer]

mk schrieb:
> Johannes Bauer wrote:
> 
>> Which takes about 40 seconds. I want the niceness of Python but a little
>> more speed than I'm getting (I'd settle for factor 2 or 3 slower, but
>> factor 30 is just too much).
> 
> This probably doesn't contribute much, but have you tried using Python
> profiler? You might have *something* wrong that eats up a lot of time in
> the code.

No - and I've not known there was a profiler yet have found anything
meaningful (there seems to be an profiling C interface, but that won't
get me anywhere). Is that a seperate tool or something? Could you
provide a link?

> The factor of 30 indeed does not seem right -- I have done somewhat
> similar stuff (calculating Levenshtein distance [edit distance] on words
> read from very large files), coded the same algorithm in pure Python and
> C++ (using linked lists in C++) and Python version was 2.5 times slower.

Yup, that was about what I had expected (and what I could well live
with, it's a tradeoff).

Thanks,
Kind regards,
Johannes

-- 
"Meine Gegenklage gegen dich lautet dann auf bewusste Verlogenheit,
verlästerung von Gott, Bibel und mir und bewusster Blasphemie."
 -- Prophet und Visionär Hans Joss aka HJP in de.sci.physik
 <48d8bf1d$0$7510$54022...@news.sunrise.ch>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Force exception on attribute write access only one object

[Thomas Guettler]

Hi Peter and others,

your idea was good, but it does not work with Django ORM Models:

Traceback (most recent call last):
  File "/localhome/modw/django/core/handlers/base.py", line 87, in get_response
response = callback(request, *callback_args, **callback_kwargs)
  File "/localhome/modw/foo/views/filter.py", line 473, in add
return edit(request, 'add')
  File "/localhome/modw/foo/views/filter.py", line 493, in edit
filter=form.save()
  File "/localhome/modw/foo/views/filter.py", line 457, in save
action=form.save()
  File "/localhome/modw/django/forms/models.py", line 315, in save
if self.instance.pk is None:
  File "/localhome/modw/django/db/models/base.py", line 292, in _get_pk_val
return getattr(self, meta.pk.attname)
AttributeError: 'MyAction' object has no attribute 'filter_action_ptr_id'


Peter Otten schrieb:
> Thomas Guettler wrote:
> 
>> for debugging I want to raise an exception if an attribute is
>> changed on an object. Since it is only for debugging I don't want
>> to change the integer attribute to a property.
> 
> Why?
>  
>> This should raise an exception:
>>
>> myobj.foo=1
>>
>> Background:
>> Somewhere this value gets changed. But I don't now where.
> 
> If you change your mind:
> 
> class A(object):
> def __init__(self): 
> self.foo = 42
> 
> a = A()
> b = A()
> 
> class B(A):
> @property
> def foo(self):
> return self.__dict__["foo"]
> 
> b.__class__ = B
> 
> a.foo = "whatever"
> print b.foo
> b.foo = "whatever"
> 
> Peter


-- 
Thomas Guettler, http://www.thomas-guettler.de/
E-Mail: guettli (*) thomas-guettler + de
--
http://mail.python.org/mailman/listinfo/python-list

Re: Encrypted Logging in python

[koranthala]

On Jan 9, 3:16 pm, Steven D'Aprano  wrote:
> On Fri, 09 Jan 2009 00:21:09 -0800, koranthala wrote:
> > I was wondering if there is a mechanism to encrypt logging automatically
> > in python.
> >    The issue is as follows:
> >     (a) An application (after py2exe) will go as executable and there
> > is no need for the user to know that it is written in python. If an
> > exception occurs and it is logged, then the user can understand it is
> > written in python.
> >     (b) A security threat. If an exception occurs, the code is seen by
> > the user - and possibly be misused.
>
> Security by obscurity is not security. If your application isn't secure
> against people who know what language is written in, then it isn't secure.
>
> --
> Steven

I understand that completely.
My point is that even though I can try to make the application
completely secure - I can never be sure of that. Especially if your
company is a very small one - and might not be able to have the best
programmers around. So, another layer of security - even security
through obscurity - can give that bit extra time in which the bugs in
the system can be ironed out.

Also, what I am asking is a generic option in logging - which can help
the adoption of the logging framework in even closed source systems.
It is not just about security - just that a closed source company
might be much more comfortable in using the system if crypt is there.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Fatal Python error: ceval: tstate mix-up

[Laszlo Nagy]

Meanwhile I'm trying to turn off threads in that program one by one. I 
just got this new type of error:


Fatal Python error: PyThreadState_Delete: invalid tstate



--
http://mail.python.org/mailman/listinfo/python-list

win32gui

[Gandalf]

Hi, everyone
I'm searching the win32gui lib to find a way to get the text under the
user cursor.
so far I managed to find only the controller ID which under the cursor
this way

cursorID = win32gui.WindowFromPoint(win32gui.GetCursorPos())

their is function called GetWindowText I tried to use but it doesn't
work in most of the time

someone know a way to do this?

thank you!
--
http://mail.python.org/mailman/listinfo/python-list

Re: win32gui

[TheSeeker]

On Jan 9, 7:34 am, Gandalf  wrote:
> Hi, everyone
> I'm searching the win32gui lib to find a way to get the text under the
> user cursor.
> so far I managed to find only the controller ID which under the cursor
> this way
>
> cursorID = win32gui.WindowFromPoint(win32gui.GetCursorPos())
>
> their is function called GetWindowText I tried to use but it doesn't
> work in most of the time
>
> someone know a way to do this?
>
> thank you!

One app that might be of help with your endeavors is called AutoIt
(http://www.autoitscript.com/autoit3/)

This suite of apps has a nice window browser, which shows most (if mot
all) information available from a Windows window. Sometimes not all
text in a window is available, depending on the window. It even comes
with a COM component, so it can be scripted from Python...

No affiliation, other than a contented user.

Duane
--
http://mail.python.org/mailman/listinfo/python-list

Re: Encrypted Logging in python

[pruebauno]

On Jan 9, 8:02 am, koranth...@gmail.com wrote:
> Also, what I am asking is a generic option in logging - which can help
> the adoption of the logging framework in even closed source systems.
> It is not just about security - just that a closed source company
> might be much more comfortable in using the system if crypt is there.

Python is an open source project. Many people that read this list
don't like closed source code too much and are not willing to invest
time to work in features like this. You might get lucky and somebody
that is interested in the topic might give you some tips.
--
http://mail.python.org/mailman/listinfo/python-list

Python Apache Handler

[Scooter]

Does anyone have any good examples, or links thereto for using python
as an Apache handler? And I should qualify all of this by saying I'm a
python newbie, and while having experience with Apache, I've never
done anything outside whats "in the box" .

What I'm looking for is how one might use Python not from the CGI/
presentation side but more on the backend...i.e. for each page Apache
serves up examine the request and update some headers, or add a cookie
to the response. Or possibly use Python for writing a custom Apache
logger. I've searched the web but typically end up with pages for
mod_python and writing CGI scripts.

And if you feel this is better posted in an Apache group vs. here, let
me apologize up front.

Thanks
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python Imaging Library and textmate

[Philip Semanchuk]

On Jan 9, 2009, at 2:49 AM, bilgin arslan wrote:


Hello,
I am a beginner in python and I am trying to create image files that  
contain

lines from a text file.
I am trying to do this with PIL, which seems like a suitable tool. I  
have a

copy of TextMate(1.5.8) and I run Macosx 10.5.6

The problem is, even though I installed PIL correctly (I get no  
errors from
"import Image" in IDLE or in terminal), I can't get TextMate to  
import the

module.
I keep getting "ImportError: No module named Image" error.
I tried reloading TextMate's bundles but that did not work either.


I use TextMate but I don't know what you mean when you say you can't  
get TM to import a module. Can you give a little more context?



There was a similar thread in this list with textmate but I can't  
understand

how that issue was solved in the end.


link?



bye
Philip
--
http://mail.python.org/mailman/listinfo/python-list