Re: Function name unchanged in error message

[Peter Otten]

Gabriel Genellina wrote:

> En Fri, 29 Jan 2010 13:09:40 -0300, Michele Simionato
>  escribió:
> 
>> On Jan 29, 2:30 pm, andrew cooke  wrote:
>>> Is there any way to change the name of the function in an error
>>> message?  In the example below I'd like the error to refer to bar(),
>>> for example (the motivation is related function decorators - I'd like
>>> the wrapper function to give the same name)
>>
>> Use the decorator module which does the right thing:
>> http://pypi.python.org/pypi/decorator
> 
> The decorator module is a very fine addition to anyone's tool set -- but
> in this case it is enough to use the wraps() function from the functools
> standard module.

I don't know about the decorator module, but functools.wraps() doesn't 
affect the error message:

>>> from functools import wraps
>>> def f(): pass
...
>>> @wraps(f)
... def g(): pass
...
>>> g.__name__
'f'
>>> try: g(42)
... except TypeError as e:
... print e
...
g() takes no arguments (1 given)

Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Sleep timer but still responsive?

[Dave Angel]

JohnnyFive wrote:

On Jan 29, 9:33 am, Andreas Tawn  wrote:
  

On Jan 28, 4:55 pm, "Gabriel Genellina" 
wrote:
  

Please provide more details. What do you want your program to do


while
  

sleeping? What kind of actions do you want a response to?
Do you have a GUI? A curses-based interfase?

--

Gabriel Genellina


My app is purely console based. I just don't want the console to lock
up (on Windows using time.sleep(x) causes the console to become
unresponsive until the timer is done), and I want people to be able to
CTRL+C to stop the script if need be (which can't be done if it's
unresponsive!).
  
Thanks.
  

How about this? Responds to ctrl+c, but still sleeps.

import time

def responsiveSleep(n):
while n > 0:
time.sleep(1)
n -=

Cheers,

Drea



Thanks for the ideas! Maybe it's just my computer, but using your
solution still causes the prompt to become unresponsive during the
sleeps.

I am using 2.6.4 btw. It's not a major deal though, I just thought
there had to be a way to do this fairly easily.

  

You responded to my message off-list, so I have to paste it here.

"""
Dave,

Thanks for the response.

Here's my scenario. I have a program that checks an ftp site every hour for
updated files, downloads them, and then sleeps again for another hour when
done.

It's 100% console.

When I run the app, no matter how I try, I can't get the window to act
normal during the sleep cycle. I've tried putting the sleep in another
thread and have the main thread.join(), but it all has the same behavior:
the console window behaves like it is crashing.

Using the small sleep() increments also causes the same thing, the becomes
undraggable for periods of time, and again acts like the program has crashed
(even though it's just asleep).

I am running a version of it right now, and i've managed to minimize it, but
it won't maximize so I can ctrl+c it. All it's doing is exactly what Andreas
recommended.

This is not user-friendly behavior!

I've also tried having the main thread sit at a raw_input, and have another
thread have the timer, but there was odd behavior when I wanted the program
to continue what it was doing, even if the user hadn't pressed "enter" to
get passed the command prompt.

 """


You probably need to tell us your complete environment. I know you're 
running 2.6.4, but don't know which version of Windows, so I'll guess 
XP.  You're running in a cmd.exe  console.


There must be something else going on in your system, since a console 
does not become unresponsive during a sleep.  The python program is 
sleeping, but the console is very much alive;  it's a separate process.  
So dragging,  minimizing, restoring and maximizing is unaffected.  Try 
the following simple script from a cmd console:


import time
print
print
print "going to sleep"
print "Use Ctrl-C to get my attention, and end the program",
time.sleep(30)
print "done"


On XP SP3, running Python 2.6.4, this ignores regular keystrokes, but 
responds nicely to control C.  And it can be dragged around, resized, 
minimized, etc. with no problem.



If you get different behavior, tell us more precisely how your 
environment differs from my guesses.  Once we've solved Ctrl-C, drag the 
console, minimize the console, then maybe you're going to request 
"respond to other keystrokes".  It can all be done, but only with more 
careful wording than "behaves like its crashing."


Regards, DaveA

--
http://mail.python.org/mailman/listinfo/python-list

Re: Funny behaviour with __future__ and doctest between 2.6 and 3.1

[Mattsteel]

On 29 Gen, 17:30, Peter Otten <__pete...@web.de> wrote:
> I think you can work around the problem. The following should pass in Python
> 2.6 and 3.1:
>
> '''>>> concat('hello','world') == 'hello world'
> True
> '''

I see. Thank for the concern.
M.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Some C-API functions clear the error indicator?

[Antoine Pitrou]

Le Fri, 29 Jan 2010 22:25:14 +0100, Austin Bingham a écrit :
> Maybe I'm not following what you're saying. In my case, I already know
> that an exception has been thrown. In the course of processing that
> exception, I call another function which, for whatever reason and even
> when it succeeds, clears the exception indicators.

If you know an error occurred and need to retain it somewhere, just use 
PyErr_Fetch() and PyErr_Restore().

Regards

Antoine.


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: C API: module cleanup function

[Mr.M]

Stefan Behnel ha scritto:

Gabriel already pointed you to the module cleanup support in Py3, which can
be used to provide reload capabilities to your module.

In Py2, there are at least some ways to free resources when terminating the
interpreter. See the "atexit" module and the Py_AtExit() function:

http://docs.python.org/library/atexit.html
http://docs.python.org/c-api/sys.html

Note that both have their specific limitations, though, as you can see from
the docs.

Also note that it might help you to take a look at Cython, a Python-to-C
compiler for writing fast C extensions. It has an option for generating
module-level cleanup code automatically, and generally simplifies writing
binary extension modules quite a bit.

Stefan


Thank you very much Stefan for your reply, I'll study the sources you 
have pointed me to.


Could I allocate my resources in a "static" object (without publishing 
the type of that object so that I can't instantiate another one) linked 
to my module?
This way, when I stop the interpreter, the object will be destroyed 
calling its destructor.


There's something I'm missing?

Thank you, Luca.
--
http://mail.python.org/mailman/listinfo/python-list

Utility to screenscrape sites using javascript ?

[KB]

Hi there,

I have a service I subscribe to that uses javascript to stream news.
Ideally I would like to use python to parse the information for me.
Note there is an option to take a static snapshot of the current
stream but that is still done via Javascript. (I can reference the
snapshot with a unique URL though, so I can pass that to a parser as
long as it can "resolve" the javascript and get at the content)

I had a quick look at Windmill but it doesn't appear to be what I am
looking for. Does anyone else have any experience in screenscraping
sites that utilise javascript? Can you share how you did it and
perhaps some sample code if possible?

Thanks a bunch!
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Microsoft Office Word and Python (Win XP)

[peskar.m...@hotmail.com]

21 days has passed and still noone is willing to help :-(
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: C API: module cleanup function

[Stefan Behnel]

Mr.M, 30.01.2010 14:24:
> Could I allocate my resources in a "static" object linked to my module?

Sure, but you will have to let CPython know about it so that it can see
that the reference is only held by the module that it is cleaning up.
Otherwise it can't collect the reference. This works easily in Py3 but not
in Py2.

In Cython, you would just write

cdef ResourceKeeper resource_keeper = ResourceKeeper()

at the module level and let Cython generate the cleanup code for it. In C,
you have to implement an atexit function and decref the resource object
either there or in the module cleanup function of Py3.


> (without publishing
> the type of that object so that I can't instantiate another one)

Note that there is the type() builtin function which returns the type given
an instance. So you can't hide the type.

Stefan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Microsoft Office Word and Python (Win XP)

[Alf P. Steinbach]

* peskar.m...@hotmail.com:

21 days has passed and still noone is willing to help :-(


Did you see the reply from Marco Nawin?

If you don't see that reply, dated (a bit less than) 2 hours after your original 
posting on the 9th, I can repost it here.


If you have any follow-up questions just post them.



Cheers & hth.,

- Alf
--
http://mail.python.org/mailman/listinfo/python-list

Re: Microsoft Office Word and Python (Win XP)

[Steve Holden]

peskar.m...@hotmail.com wrote:
> 21 days has passed and still noone is willing to help :-(

Y'know, the Internet isn't a magic lantern. Perhaps you need to ask your
question differently, or provide more information. Maybe there's
something *you* could do to help you get closer to the answer to *your*
question.

Read the question at the time, don't remember having much to contribute
right then. But you lucked out today: it's Saturday morning and I am at
home in my family room, and it's snowing outside. Hey, at least you do
know already that people will respond to your questions.

Look for a (second-hand?) copy of Andy Robinson's and Mark Hammond's
"Python Programming on Win32", which taught me a lot about COM I have
thankfully since been able to forget. Go to Microsoft channels for
details of the Office object models, and good luck with that (the
quality and quantity of that used to be something of a movable feast,
though I imagine things have improved since).

You might also ask yourself whether IronPython and the .NET interfaces
aren't a better and more modern way of controlling Microsoft products.
The .NET documentation is pretty well organized and complete, I believe.

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
PyCon is coming! Atlanta, Feb 2010  http://us.pycon.org/
Holden Web LLC http://www.holdenweb.com/
UPCOMING EVENTS:http://holdenweb.eventbrite.com/

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: C API: module cleanup function

[Mr.M]

Stefan Behnel ha scritto:

Note that there is the type() builtin function which returns the type given
an instance. So you can't hide the type.


Argh! Yes, you are right.

So I'd better have a look to Cython, right?

L-
--
http://mail.python.org/mailman/listinfo/python-list

list.extend([]) Question

[Dan Brown]

Why does extending a list with the empty list result in None?  It
seems very counterintuitive to me, at least --- I expected ['a'].extend
([]) to result in ['a'], not None.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: list.extend([]) Question

[Alf P. Steinbach]

* Dan Brown:

Why does extending a list with the empty list result in None?  It
seems very counterintuitive to me, at least --- I expected ['a'].extend
([]) to result in ['a'], not None.


It does.

'extend' is an operation that /modifies/ the array.

It just returns None as its expression result, in the same way as e.g. the 
Python 3.x 'print' (another pure "doer" operation).


  >>> L = ['a']
  >>> L
  ['a']
  >>> L2 = L.extend( [] )
  >>> L2
  >>> L2 is None
  True
  >>> L
  ['a']
  >>> _


Cheers & hth.,

- Alf
--
http://mail.python.org/mailman/listinfo/python-list

Re: list.extend([]) Question

[Andre Engels]

On Sat, Jan 30, 2010 at 4:32 PM, Dan Brown  wrote:
> Why does extending a list with the empty list result in None?  It
> seems very counterintuitive to me, at least --- I expected ['a'].extend
> ([]) to result in ['a'], not None.

Extend is a method of the list. The list itself is changed, it does
not return itself:

>>> A = [1,2]
>>> B = [3,4]
>>> C = A.extend(B)
>>> C
>>> C is None
True
>>> A
[1, 2, 3, 4]


Thus, nothing special about extend([]), this is the general behaviour of extend

-- 
André Engels, andreeng...@gmail.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: list.extend([]) Question

[Dan Brown]

On Jan 30, 8:38 am, "Alf P. Steinbach"  wrote:
> It does.
>
> 'extend' is an operation that /modifies/ the array.
>
> It just returns None as its expression result, in the same way as e.g. the
> Python 3.x 'print' (another pure "doer" operation).
>
>    >>> L = ['a']
>    >>> L
>    ['a']
>    >>> L2 = L.extend( [] )
>    >>> L2
>    >>> L2 is None
>    True
>    >>> L
>    ['a']
>    >>> _
>
> Cheers & hth.,
>
> - Alf

Aha.  Well, I feel a bit silly for not thinking to try it that way.
Thanks!
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: list.extend([]) Question

[Steve Holden]

Dan Brown wrote:
> Why does extending a list with the empty list result in None?  It
> seems very counterintuitive to me, at least --- I expected ['a'].extend
> ([]) to result in ['a'], not None.

How very inconvenient of Python! What it actually does is create an
anonymous list containing only the element 'a', and leave it unchanged
by extending it with an empty list. Since there is no longer any
reference to the list it has become garbage.

Contrast that with:

>>> lst = ['a']
>>> lst.extend([])
>>> lst
['a']
>>> lst.append([])
>>> lst
['a', []]
>>> lst.extend(['1'])
>>> lst
['a', [], '1']
>>>

As you can see by the absence of output, both the .extend() and
.append() list methods return None. They mutate the list instance upon
which they are called.

In your example you were expecting the methods to return the mutated
list. They don't.

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
PyCon is coming! Atlanta, Feb 2010  http://us.pycon.org/
Holden Web LLC http://www.holdenweb.com/
UPCOMING EVENTS:http://holdenweb.eventbrite.com/

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Default path for files

[Nobody]

On Sun, 24 Jan 2010 15:08:15 +, Rotwang wrote:

> Hi all, can anybody tell me whether there's a way to change the default
> location for files to be opened by open()? I'd like to be able to create
> files somewhere other than my Python folder without having to write the
> full path in the filename every time. Sorry if this is a stupid question,
> I don't know much about programming.

If you pass a relative pathname to open() (or any other function which
expects a filename), it will be interpreted relative to the current
directory.

Given that the current directory always seems to be the Python directory,
and you refer to it as a "folder", I'm guessing that you're running Python
on Windows via a shortcut in the Start Menu or on the desktop.

In which case, the ideal solution is probably to open the Properties
dialog for the shortcut and change the "Start in" field to your
"My Documents" directory (or some subdirectory of it).

Python itself won't care which directory it starts in.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: myths about python 3

[Nobody]

On Wed, 27 Jan 2010 12:56:10 -0800, John Nagle wrote:

> Arguably, Python 3 has been rejected by the market.

Arguably, Python 3 has not yet been accepted by the market.

Part of it is down to a catch-22: applications won't use Python 3 if the
libraries on which they depend don't support it, and support for Python 3
by libraries will be influenced by the perceived demand.

OTOH, it's safe to assume that there will remain areas where Python 2 is
preferred. Primarily Unix scripting, where most data is byte strings with
the encoding either unknown or irrelevant. That alone will ensure that
Python 2 is alive and well even as Python 4 is released. Even if
python.org doesn't support Python 2, it's a safe bet that e.g. ActiveState
will.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: myths about python 3

[Kevin Walzer]

On 1/30/10 11:29 AM, Nobody wrote:


Arguably, Python 3 has not yet been accepted by the market.

Part of it is down to a catch-22: applications won't use Python 3 if the
libraries on which they depend don't support it, and support for Python 3
by libraries will be influenced by the perceived demand.


This is part of my reason for not yet moving to Python 3--several 
libraries that I will need do not currently support Python 3.


--
Kevin Walzer
Code by Kevin
http://www.codebykevin.com
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python and Ruby

[Nobody]

On Wed, 27 Jan 2010 15:29:05 -0800, Jonathan Gardner wrote:

> There's a lot of "magic" in Ruby as well. For instance, function calls are
> made without parentheses.

That's also true for most functional languages, e.g. Haskell and ML, as
well as e.g. Tcl and most shells. Why require "f(x)" or "(f x)" if "f x"
will suffice?

> Python is much, much cleaner. I don't know how anyone can honestly say
> Ruby is cleaner than Python.

I'm not familiar with Ruby, but most languages are cleaner than Python
once you get beyond the "10-minute introduction" stage.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Keyboard input

[Nobody]

On Fri, 29 Jan 2010 16:53:43 +, Mr.SpOOn wrote:

> I'm using the raw_input method and it works fine, but I noted that I can't
> use backspace and when I accidentally press shift space (because I need to
> input uppercase letters separated by a space) it writes some strange
> characters.
> 
> So, is there another way to get keyboard input?

sys.stdin.readline()

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python and Ruby

[tanix]

In article , Nobody 
 wrote:
>On Wed, 27 Jan 2010 15:29:05 -0800, Jonathan Gardner wrote:
>
>> There's a lot of "magic" in Ruby as well. For instance, function calls are
>> made without parentheses.
>
>That's also true for most functional languages, e.g. Haskell and ML, as
>well as e.g. Tcl and most shells. Why require "f(x)" or "(f x)" if "f x"
>will suffice?
>
>> Python is much, much cleaner. I don't know how anyone can honestly say
>> Ruby is cleaner than Python.
>
>I'm not familiar with Ruby, but most languages are cleaner than Python
>once you get beyond the "10-minute introduction" stage.

I'd have to agree. The only ones that beat Python in that department
are Javascript and PHP. Plus CSS and HTML if you can call those languages.

The very idea of using a number of blanks to identify your block level
is as insane as it gets. First of all, combinations of blanks and tabs,
depending on how your ide is setup to expand tabs, may get you bugs,
you'd never imagine in your wild dreams.

Braces is the most reliable way to identify blocks.

Sane compilers ignore blanks altogether.



--
Programmer's Goldmine collections:

http://preciseinfo.org

Tens of thousands of code examples and expert discussions on
C++, MFC, VC, ATL, STL, templates, Java, Python, Javascript, PHP,
organized by major topics of language, tools, methods, techniques.

All collections are fully searchable down to specific chapter.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Utility to screenscrape sites using javascript ?

[Nobody]

On Sat, 30 Jan 2010 06:21:01 -0800, KB wrote:

> I have a service I subscribe to that uses javascript to stream news.
> Ideally I would like to use python to parse the information for me. Note
> there is an option to take a static snapshot of the current stream but
> that is still done via Javascript. (I can reference the snapshot with a
> unique URL though, so I can pass that to a parser as long as it can
> "resolve" the javascript and get at the content)
> 
> I had a quick look at Windmill but it doesn't appear to be what I am
> looking for. Does anyone else have any experience in screenscraping sites
> that utilise javascript? Can you share how you did it and perhaps some
> sample code if possible?

There's a Python interface to SpiderMonkey (Mozilla's JavaScript
interpreter):

http://pypi.python.org/pypi/python-spidermonkey

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python and Ruby

[Ed Keith]

--- On Sat, 1/30/10, Nobody  wrote:

> From: Nobody 
> > Python is much, much cleaner. I don't know how anyone
> can honestly say
> > Ruby is cleaner than Python.
> 
> I'm not familiar with Ruby, but most languages are cleaner
> than Python
> once you get beyond the "10-minute introduction" stage.

You need to be clear about what you mean by "clean". Is Python scoping "clean"? 
I suspect lots of people would argue either side.

-EdK

Ed Keith
e_...@yahoo.com

Blog: edkeith.blogspot.com



  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: myths about python 3

[Christian Heimes]

Blog wrote:
> WTF? Where'd you hear about version 2.8? FRI, 2.7 is and will be THE 
> LAST version of the 2.x series - "the" End-Of-Life for Python 2

Where do you get your information from? Your answer is the first that
clearly marks the end of lifetime for the 2.x series. I didn't know that
and I'm a Python core dev as well as a PSF member ... *scnr*

Christian
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: myths about python 3

[Blog]

On 1/30/2010 11:47 PM, Christian Heimes wrote:

Blog wrote:

WTF? Where'd you hear about version 2.8? FRI, 2.7 is and will be THE
LAST version of the 2.x series - "the" End-Of-Life for Python 2


Where do you get your information from? Your answer is the first that
clearly marks the end of lifetime for the 2.x series. I didn't know that
and I'm a Python core dev as well as a PSF member ... *scnr*

Christian


Well, here's what the Python 2.7 alpha 2 (January 9th, 2010) release 
note says:


"Python 2.7 is scheduled to be the last major version in the 2.x series 
before it moves into 5 years of bugfix-only mode. This release contains 
many of the features that were first released in Python 3.1..."


Ref: http://www.python.org/download/releases/2.7/


Here are some more resources:

"Python 2 nears end of life"

"Python 2.7, expected to be the last major version of the 2.x series of 
the dynamic language, was released as a second alpha earlier month by 
the Python Software Foundation, with the final release set for June.


When 2.7 is released, the 2.x line will move into five years of a bug 
fix-only mode."


http://www.computerworlduk.com/technology/development/software/news/index.cfm?newsid=18331


"2.7's the end of the line. There was some discussion around the release 
of 2.6 as to how far the 2.x series should go, and the conclusion which 
came out of it was that 2.7 is it. Beyond this, the world is 3.x (which, 
given the time it takes OS distributors to catch up -- most are still on 
2.5 -- is about right with the projected time frame for most projects to 
port)."


http://www.reddit.com/r/Python/comments/aoloc/python_27_alpha_2_has_been_released/


"Let’s also account for the fact that, as of this writing, Python 2.7 
(scheduled for next year) is *intended* to be the End of Life release of 
the Python 2.x syntax – Python 3 being the next evolutionary step."


http://jessenoller.com/2009/12/04/pythons-moratorium-lets-think-about-this/

Of course, this isn't written in stone - there may very well be another 
version after 2.7. However, at this instant, it does look like 2.7 will 
be the veritable last version.


Best regards.

--
http://mail.python.org/mailman/listinfo/python-list

Re: Threading issue with SQLite

[John Nagle]

Jonathan Gardner wrote:

On Jan 29, 8:37 am, Alan Harris-Reid 
wrote:

Questions...
1.  Is there a large overhead in opening a new SQLite connection for
each thread (ie. within each method)?



Suggestion: Use something like SQLAlchemy to manage you DB
interactions. One day, you'll move away from SQLite, and you'll be
glad you were programming at a higher level. SQLAlchemy also does
things like connection pooling and such for you.


   Generally, if you get to the point where you're concerned about
concurrency performance issues with SQLite, it's time to upgrade
to a more serious database.  Especially if you're doing much updating.
SQLite can do multiple SELECT operations in parallel, but the entire
database is locked for all operations that write.

John Nagle
--
http://mail.python.org/mailman/listinfo/python-list

Re: myths about python 3

[Blog]

On 1/30/2010 10:06 AM, Ben Finney wrote:

Blog  writes:


(Debian does ship with 2.5, but the next major release "sid' is due
out in Q2)


Sid is the perpetual development playground (“unstable”), never released
as a suite, but a proving ground for packages to determine their fitness
for going to the next level of testing.

The next-to-be-released suite is Squeeze (currently “testing”), which
has Python 2.5 (the default ‘python’) and Python 2.6.



Oops! My bad! I actually meant Squeeze.

Thanks for catching the "typo".

--
http://mail.python.org/mailman/listinfo/python-list

unencountered error in FFT python

[uche]

Hi,
I have the following FFT python code and it doesn't seem to compile
correctly. To run it, please create a file called output.csv with
1,2,3,4,5,6,7,8. simply run the main function. I get an error such as
the following:

x[a], x[b] = x[(a)] + W[(n % N)] * x[(b)], x[(a)] - W[(n % N)] * x
[(b)]
TypeError: list indices must be integers, not float

How can I fixe this problem ? I have tried puttin int on all of the
variables, but I don't think that is the intension of the person who
wore the original code.


#!/usr/bin/python
"""
FFT using Cooley-Tukey algorithm where N = 2^n.  The choice of
e^{-j2\pi/N} or e^{j2\pi/N} is made by 'sign=-1' or 'sign=1'
respectively.  Since I prefer Engineering convention, I chose
'sign=-1' as the default.

FFT is performed as follows:
1. bit-reverse the array.
2. partition the data into group of m = 2, 4, 8, ..., N data points.
3. for each group with m data points,
1. divide into upper half (section A) and lower half (section B),
each containing m/2 data points.
2. divide unit circle by m.
3. apply "butterfly" operation
|a| = |1  w||a| or  a, b = a+w*b, a-w*b
|b|   |1 -w||b|
where a and b are data points of section A and B starting from
the top of each section, and w is data points along the unit
circle starting from z = 1+0j.
FFT ends after applying "butterfly" operation on the entire data array
as whole, when m = N.
"""


def main():

array = []
array2 = []

import os
import time

os.path.exists("input.csv")

fin=open('input.csv', 'r')

for line in fin:#read the line from the file

array=line.split(',')

for a in range(len(array)): #convert into integers

array2.append((array[a]))

Ti=time.time()
print (fft(array2))
Tf=time.time()
print (("It took"),Tf-Ti,("seconds to calculate an FFT of
size"),len(array))
#end timer

"""
Find 2^n that is equal to or greater than.
"""

def nextpow2(i):
n = 2
while n < i: n = n * 2
return n

"""
Return bit-reversed list, whose length is assumed to be 2^n:
eg. 0111 <--> 1110 for N=2^4.
"""
def bitrev(x):

N, x = len(x), x[:]
if N != nextpow2(N): raise ValueError ('N is not power of 2')
for i in range(N):
k, b, a = 0, N>>1, 1
while b >= a:
if b & i: k = k | a
if a & i: k = k | b
b, a = b>>1, a<<1
if i < k: x[i], x[k] = (x[k],x[i])
return x

def fft(x, sign=-1):
from cmath import pi, exp
N, W = len(x), []
for i in range(N):  # exp(-j...) is default
W.append(exp(sign * 2j * pi * i / N))
x = bitrev(x)
m = 2
while m <= N:
for s in range(0,N,m):
for i in range (int(m/2)):
n = i * N / m
a, b = s + i, s + i + m/2
x[a], x[b] = x[(a)] + W[(n % N)] * x[(b)], x[(a)] - W
[(n % N)] * x[(b)]
m = m * 2
return x
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: myths about python 3

[Martin v. Loewis]

Christian Heimes wrote:
> Blog wrote:
>> WTF? Where'd you hear about version 2.8? FRI, 2.7 is and will be THE 
>> LAST version of the 2.x series - "the" End-Of-Life for Python 2
> 
> Where do you get your information from? 

It was discussed repeatedly on python-dev, last time when the release
announcements for the 2.7 alpha releases were discussed. See

http://www.python.org/download/releases/2.7/

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: unencountered error in FFT python

[Stefan Behnel]

uche, 30.01.2010 19:33:
> I have the following FFT python code

You didn't seriously implement an FFT in plain Python code, did you? FFTs
are amongst the first thing that come to my mind when I try to imagine what
I'd use NumPy for. (and I *never* used it!)

Stefan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Default path for files

[Aahz]

In article ,
Rotwang   wrote:
>
>Hi all, can anybody tell me whether there's a way to change the default 
>location for files to be opened by open()? I'd like to be able to create 
>files somewhere other than my Python folder without having to write the 
>full path in the filename every time. Sorry if this is a stupid 
>question, I don't know much about programming.

from os.path import join
BASE = '/path/to/root'
f = open(join(BASE, filename))

Trust me, you'll be much happier with this.
-- 
Aahz (a...@pythoncraft.com)   <*> http://www.pythoncraft.com/

import antigravity
-- 
http://mail.python.org/mailman/listinfo/python-list

www.assembly.learn.net.in

[groups_ads12]

http://www.assembly.learn.net.in/ 

assembly


assembly of god


assemble


assembly jobs


assembly language


assembly work


assembler


electronic assembly


pcb assembly


assemblies


crib assembly


cable assembly


christian assembly


contract assembly


furniture assembly


hub assembly


the assembly


assembly code


assembly instructions


assembly language programming


assembly programming


document assembly


smt assembly


x86 assembly


axle assembly


computer assembly


electronics assembly


urban assembly


wheel bearing assembly


assembly magazine


assembly services


bearing assembly


mechanical assembly


arm assembly


assembly required


assembly table


assembly tools


car assembly


medical assembly


mips assembly


wheel assembly


ikea assembly


assembly program


assembly service


assembly worker


product assembly


assembly instruction


assembly language step by step


assembly systems


assembly testing


c# assembly


central assembly


distributor assembly

Re: unencountered error in FFT python

[MRAB]

uche wrote:

Hi,
I have the following FFT python code and it doesn't seem to compile
correctly. To run it, please create a file called output.csv with
1,2,3,4,5,6,7,8. simply run the main function. I get an error such as
the following:

x[a], x[b] = x[(a)] + W[(n % N)] * x[(b)], x[(a)] - W[(n % N)] * x
[(b)]
TypeError: list indices must be integers, not float

How can I fixe this problem ? I have tried puttin int on all of the
variables, but I don't think that is the intension of the person who
wore the original code.


Which version of Python are you using? In Python 3 the division operator
'/' returns a float, whereas in Python 2 it returns an int if both
operands are int. In Python 3 the int division operator is '//', which
is also accepted in recent versions of Python 2.

[snip]


os.path.exists("input.csv")

fin=open('input.csv', 'r')

for line in fin:#read the line from the file

array=line.split(',')


These lines should be indented more:


for a in range(len(array)): #convert into integers

array2.append((array[a]))


 array2.append(int(array[a]))

[snip]

--
http://mail.python.org/mailman/listinfo/python-list

Re: Python and Ruby

[Steven D'Aprano]

On Sat, 30 Jan 2010 16:58:34 +, tanix wrote:

> In article , Nobody
>  wrote:
>>On Wed, 27 Jan 2010 15:29:05 -0800, Jonathan Gardner wrote:
>>
>>> There's a lot of "magic" in Ruby as well. For instance, function calls
>>> are made without parentheses.
>>
>>That's also true for most functional languages, e.g. Haskell and ML, as
>>well as e.g. Tcl and most shells. Why require "f(x)" or "(f x)" if "f x"
>>will suffice?
>>
>>> Python is much, much cleaner. I don't know how anyone can honestly say
>>> Ruby is cleaner than Python.
>>
>>I'm not familiar with Ruby, but most languages are cleaner than Python
>>once you get beyond the "10-minute introduction" stage.
> 
> I'd have to agree. The only ones that beat Python in that department are
> Javascript and PHP. Plus CSS and HTML if you can call those languages.

Your sentence makes no sense. You agree that "most" languages are cleaner 
than Python, and then in the very next sentence, out of the hundreds if 
not thousands of languages invented, you can only list TWO that are 
better than Python -- and those are Javascript and PHP!!!

That's like saying "most things are harder than diamond -- the only 
things that beat diamond are jelly and talc".


> The very idea of using a number of blanks to identify your block level
> is as insane as it gets.

Not at all. People do it all the time. The very idea of expecting people 
to count nested braces to identify block level is what is crazy, which is 
why in languages with braces people still indent the blocks.


> First of all, combinations of blanks and tabs,
> depending on how your ide is setup to expand tabs, may get you bugs,
> you'd never imagine in your wild dreams.

Not really. The bugs are quite simple, and generally easy to fix. To 
describe them as unimaginable is stupid.

>>> for x in [1, 2, 3]:
... print x
... print x+1
  File "", line 3
print x+1
^
IndentationError: unexpected indent


If you can't imagine getting an IndentationError from making an 
indentation error, there's something wrong with you.

In any case, if your IDE mixes tabs and spaces, your IDE is broken and 
you should fix your tools rather than blame the language.


> Braces is the most reliable way to identify blocks.

Nonsense. For the compiler, both are equally reliable, and for the human 
reader, indents beat braces easily.


> Sane compilers ignore blanks altogether.

Really? So a "sane compiler" sees no difference between:

for x in mylist:

and

forxinmylist:


I'm glad I don't have to program using a compiler you consider "sane".




-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: unencountered error in FFT python

[uche]

On Jan 30, 2:08 pm, MRAB  wrote:
> uche wrote:
> > Hi,
> > I have the following FFT python code and it doesn't seem to compile
> > correctly. To run it, please create a file called output.csv with
> > 1,2,3,4,5,6,7,8. simply run the main function. I get an error such as
> > the following:
>
> >     x[a], x[b] = x[(a)] + W[(n % N)] * x[(b)], x[(a)] - W[(n % N)] * x
> > [(b)]
> > TypeError: list indices must be integers, not float
>
> > How can I fixe this problem ? I have tried puttin int on all of the
> > variables, but I don't think that is the intension of the person who
> > wore the original code.
>
> Which version of Python are you using? In Python 3 the division operator
> '/' returns a float, whereas in Python 2 it returns an int if both
> operands are int. In Python 3 the int division operator is '//', which
> is also accepted in recent versions of Python 2.
>
> [snip]
>
> >     os.path.exists("input.csv")
>
> >     fin=open('input.csv', 'r')
>
> >     for line in fin:    #read the line from the file
>
> >         array=line.split(',')
>
> These lines should be indented more:
>
> >     for a in range(len(array)): #convert into integers
>
> >         array2.append((array[a]))
>
>           array2.append(int(array[a]))
>
> [snip]

Thanks, I just got this code from a site and trying to compile it.
it's just a nightmare!
I got another problem after changing / to // . Yes, I am using 3.1.

W.append(exp(sign * 2j * pi * i // N))
TypeError: can't take floor of complex number.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: unencountered error in FFT python

[Stefan Behnel]

Stefan Behnel, 30.01.2010 19:52:
> uche, 30.01.2010 19:33:
>> I have the following FFT python code
> 
> You didn't seriously implement an FFT in plain Python code, did you?

Sorry, no, you didn't. Should have read your post a little closer.


> FFTs
> are amongst the first thing that come to my mind when I try to imagine what
> I'd use NumPy for. (and I *never* used it!)

On second thought, I'd probably use FFTW for that, and there seem to be
Python bindings for it:

http://developer.berlios.de/projects/pyfftw/

Stefan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Utility to screenscrape sites using javascript ?

[KB]

> On Sat, 30 Jan 2010 06:21:01 -0800, KB wrote:
> > I have a service I subscribe to that uses javascript to stream news.

> There's a Python interface to SpiderMonkey (Mozilla's JavaScript
> interpreter):
>
> http://pypi.python.org/pypi/python-spidermonkey

Thanks! I don't see a documentation page, but how would one use this?
Would you download the HTML using urllib2/mechanize, then parse for
the .js script and use spider-monkey to "execute" the script and the
output is passed back to python?

TIA.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: unencountered error in FFT python

[Stefan Behnel]

uche, 30.01.2010 20:18:
> I got another problem after changing / to // . Yes, I am using 3.1.
> 
> W.append(exp(sign * 2j * pi * i // N))
> TypeError: can't take floor of complex number.

Don't change it everywhere, just where it deals with integers. In the
above, "/" is perfectly right.

Stefan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: unencountered error in FFT python

[uche]

On Jan 30, 2:21 pm, Stefan Behnel  wrote:
> Stefan Behnel, 30.01.2010 19:52:
>
> > uche, 30.01.2010 19:33:
> >> I have the following FFT python code
>
> > You didn't seriously implement an FFT in plain Python code, did you?
>
> Sorry, no, you didn't. Should have read your post a little closer.
>
> > FFTs
> > are amongst the first thing that come to my mind when I try to imagine what
> > I'd use NumPy for. (and I *never* used it!)
>
> On second thought, I'd probably use FFTW for that, and there seem to be
> Python bindings for it:
>
> http://developer.berlios.de/projects/pyfftw/
>
> Stefan

Thanks for the suggestions and site. I just wanted to know what the
heck is going wrong with this code. Please take a look at the code ,
thanks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python and Ruby

[Stephen Hansen]

On Sat, Jan 30, 2010 at 8:58 AM, tanix  wrote:

> The very idea of using a number of blanks to identify your block level
> is as insane as it gets. First of all, combinations of blanks and tabs,
> depending on how your ide is setup to expand tabs, may get you bugs,
> you'd never imagine in your wild dreams.
>
> Braces is the most reliable way to identify blocks.
>
> Sane compilers ignore blanks altogether.
>

God forbid the compiler use the same syntactical structure as the human eye
in reading code and assigning meaning.

I mean, everyone knows that in C, no one ever has any issues with
indentation levels causing human confusion and misunderstanding between what
they think the structure of the code is and what the compiler interprets it
as.

Also, we all know very well that no modern IDE's or text editors let you
manage tabs and spaces in a consistent fashion, and Python doesn't error out
when you use inconsistent or nonsensical indentation. It just lets you
create anonymous blocks of code when you indent too far in for no reason at
all.

--S

P.S. Yes, my response is juvenile and snarky. However, I am weary of the
imbecilic OMG WITESPACE IS RONG AND STUPID response. If you don't like
whitespace being structurally significant, so be it-- aesthetics are
subjective. If its not for you, its not for you. Python isn't for everyone.
"Its insane", "Its stupid", "Its causes all kinds of confusion", "Its causes
all kinds of hard to find bugs" is just complete, utter, unsupportable
nonsense.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: unencountered error in FFT python

[uche]

On Jan 30, 2:30 pm, uche  wrote:
> On Jan 30, 2:21 pm, Stefan Behnel  wrote:
>
>
>
>
>
> > Stefan Behnel, 30.01.2010 19:52:
>
> > > uche, 30.01.2010 19:33:
> > >> I have the following FFT python code
>
> > > You didn't seriously implement an FFT in plain Python code, did you?
>
> > Sorry, no, you didn't. Should have read your post a little closer.
>
> > > FFTs
> > > are amongst the first thing that come to my mind when I try to imagine 
> > > what
> > > I'd use NumPy for. (and I *never* used it!)
>
> > On second thought, I'd probably use FFTW for that, and there seem to be
> > Python bindings for it:
>
> >http://developer.berlios.de/projects/pyfftw/
>
> > Stefan
>
> Thanks for the suggestions and site. I just wanted to know what the
> heck is going wrong with this code. Please take a look at the code ,
> thanks- Hide quoted text -
>
> - Show quoted text -

Thanks Stephan. You are the best!

Another issue:
x[a], x[b] = x[(a)] + W[(n % N)] * x[(b)], x[(a)] - W[(n % N)] * x
[(b)]
TypeError: can't multiply sequence by non-int of type 'complex'





-- 
http://mail.python.org/mailman/listinfo/python-list

Slow down while creating a big list and iterating over it

[marc magrans de abril]

Dear colleagues,

I was doing a small program to classify log files for a cluster of
PCs, I just wanted to simplify a quite repetitive task in order to
find errors and so.

My first naive implementation was something like:
patterns = []
while(logs):
pattern = logs[0]
new_logs = [l for l in logs if dist(pattern,l)>THERESHOLD]
entry = (len(logs)-len(new_logs),pattern)
patterns.append(entry)
logs = new_logs

Where dist(...) is the levenshtein distance (i.e. edit distance) and
logs is something like 1.5M logs (700 MB file). I thought that python
will be an easy choice although not really fast..

I was not surprised when the first iteration of the while loop was
taking ~10min. I thought "not bad, let's how much it takes". However,
it seemed that the second iteration never finished.

My surprise was big when I added a print instead of the list
comprehension:
new_logs=[]
for count,l in enumerate(logs):
   print count
   if dist(pattern,l)>THERESHOLD:
  new_logs.append(l)

The surprise was that the displayed counter was running ~10 times
slower on the second iteration of the while loop.

I am a little lost. Anyone knows the reson of this behavior?  How
should I write a program that deals with large data sets in python?

Thanks a lot!
marc magrans de abril
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Slow down while creating a big list and iterating over it

[Alf P. Steinbach]

* marc magrans de abril:

Dear colleagues,

I was doing a small program to classify log files for a cluster of
PCs, I just wanted to simplify a quite repetitive task in order to
find errors and so.

My first naive implementation was something like:
patterns = []
while(logs):
pattern = logs[0]
new_logs = [l for l in logs if dist(pattern,l)>THERESHOLD]
entry = (len(logs)-len(new_logs),pattern)
patterns.append(entry)
logs = new_logs

Where dist(...) is the levenshtein distance (i.e. edit distance) and
logs is something like 1.5M logs (700 MB file). I thought that python
will be an easy choice although not really fast..

I was not surprised when the first iteration of the while loop was
taking ~10min. I thought "not bad, let's how much it takes". However,
it seemed that the second iteration never finished.

My surprise was big when I added a print instead of the list
comprehension:
new_logs=[]
for count,l in enumerate(logs):
   print count
   if dist(pattern,l)>THERESHOLD:
  new_logs.append(l)

The surprise was that the displayed counter was running ~10 times
slower on the second iteration of the while loop.

I am a little lost. Anyone knows the reson of this behavior?


It's on line 42 of your program. :-) That is, it's in the dist function. 
Evidently it doesn't like a more complex 'pattern'.




How should I write a program that deals with large data sets in python?


As in any other language. Try to avoid repeating the same computations. Try to 
make the data fit the computational task.



Cheers & hth.,

- Alf
--
http://mail.python.org/mailman/listinfo/python-list

PEP 3147 - new .pyc format

[John Roth]

PEP 3147 has just been posted, proposing that, beginning in release
3.2 (and possibly 2.7) compiled .pyc and .pyo files be placed in a
directory with a .pyr extension. The reason is so that compiled
versions of a program can coexist, which isn't possible now.

Frankly, I think this is a really good idea, although I've got a few
comments.

1. Apple's MAC OS X should be mentioned, since 10.5 (and presumably
10.6) ship with both Python release 2.3 and 2.5 installed.

2. I think the proposed logic is too complex. If this is installed in
3.2, then that release should simply store its .pyc file in the .pyr
directory, without the need for either a command line switch or an
environment variable (both are mentioned in the PEP.)

3. Tool support. There are tools that look for the .pyc files; these
need to be upgraded somehow. The ones that ship with Python should, of
course, be fixed with the PEP, but there are others.

4. I'm in favor of putting the source in the .pyr directory as well,
but that's got a couple more issues. One is tool support, which is
likely to be worse for source, and the other is some kind of algorithm
for identifying which source goes with which object.

Summary: I like it, but I think it needs a bit more work.

John Roth
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Function name unchanged in error message

[andrew cooke]

On Jan 29, 11:22 am, Peter Otten <__pete...@web.de> wrote:
> The name is looked up in the code object. As that is immutable you have to
> make a new one:
[details snipped]

thanks very much!  sorry i didn't reply earlier - been travelling.

(also, thanks to any other replies - i'm just reading through at the
moment and this is the first one i've got to that will help me solve
it, but i don't mean to exclude anything later...!)

andrew
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Function name unchanged in error message

[andrew cooke]

On Jan 29, 5:37 pm, "Gabriel Genellina" 
wrote:
> The decorator module is a very fine addition to anyone's tool set -- but  
> in this case it is enough to use the wraps() function from the functools  
> standard module.

ah, thanks!  i thought something like this existed in the standard
lib, but couldn't find it.

andrew
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Function name unchanged in error message

[andrew cooke]

On Jan 29, 11:50 am, exar...@twistedmatrix.com wrote:
> new.function and new.code will let you construct new objects with
> different values (and copying over whichever existing attributes you
> want to preserve).

unfortunately new is deprecated and dropped from 3.  i can't see how
the same functionality is available in the types module for 3 - am i
missing something obvious?

http://docs.python.org/library/new.html
http://docs.python.org/3.1/library/types.html#module-types

thanks,
andrew
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Function name unchanged in error message

[andrew cooke]

On Jan 30, 7:17 pm, andrew cooke  wrote:
> On Jan 29, 5:37 pm, "Gabriel Genellina" 
> wrote:
>
> > The decorator module is a very fine addition to anyone's tool set -- but  
> > in this case it is enough to use the wraps() function from the functools  
> > standard module.
>
> ah, thanks!  i thought something like this existed in the standard
> lib, but couldn't find it.
>
> andrew

ah, sorry, peter's code uses types, so i assume that's the way to go
(i was hoping that there was something a bit simpler - i don't like
the fact that the code in peter's code has a fixed list of special
names).

andrew
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP 3147 - new .pyc format

[Mensanator]

On Jan 30, 4:14 pm, John Roth  wrote:
> PEP 3147 has just been posted, proposing that, beginning in release
> 3.2 (and possibly 2.7) compiled .pyc and .pyo files be placed in a
> directory with a .pyr extension. The reason is so that compiled
> versions of a program can coexist, which isn't possible now.
>
> Frankly, I think this is a really good idea, although I've got a few
> comments.
>
> 1. Apple's MAC OS X should be mentioned, since 10.5 (and presumably
> 10.6) ship with both Python release 2.3 and 2.5 installed.

Mac OSX 10.6 has 2.6 installed.

>
> 2. I think the proposed logic is too complex. If this is installed in
> 3.2, then that release should simply store its .pyc file in the .pyr
> directory, without the need for either a command line switch or an
> environment variable (both are mentioned in the PEP.)
>
> 3. Tool support. There are tools that look for the .pyc files; these
> need to be upgraded somehow. The ones that ship with Python should, of
> course, be fixed with the PEP, but there are others.
>
> 4. I'm in favor of putting the source in the .pyr directory as well,
> but that's got a couple more issues. One is tool support, which is
> likely to be worse for source, and the other is some kind of algorithm
> for identifying which source goes with which object.
>
> Summary: I like it, but I think it needs a bit more work.
>
> John Roth

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP 3147 - new .pyc format

[MRAB]

John Roth wrote:

PEP 3147 has just been posted, proposing that, beginning in release
3.2 (and possibly 2.7) compiled .pyc and .pyo files be placed in a
directory with a .pyr extension. The reason is so that compiled
versions of a program can coexist, which isn't possible now.

Frankly, I think this is a really good idea, although I've got a few
comments.

1. Apple's MAC OS X should be mentioned, since 10.5 (and presumably
10.6) ship with both Python release 2.3 and 2.5 installed.

2. I think the proposed logic is too complex. If this is installed in
3.2, then that release should simply store its .pyc file in the .pyr
directory, without the need for either a command line switch or an
environment variable (both are mentioned in the PEP.)

3. Tool support. There are tools that look for the .pyc files; these
need to be upgraded somehow. The ones that ship with Python should, of
course, be fixed with the PEP, but there are others.

4. I'm in favor of putting the source in the .pyr directory as well,
but that's got a couple more issues. One is tool support, which is
likely to be worse for source, and the other is some kind of algorithm
for identifying which source goes with which object.

Summary: I like it, but I think it needs a bit more work.


The PEP has a .pyr directory for each .py file:

foo.py
foo.pyr/
f2b30a0d.pyc # Python 2.5
f2d10a0d.pyc # Python 2.6
f2d10a0d.pyo # Python 2.6 -O
f2d20a0d.pyc # Python 2.6 -U
0c4f0a0d.pyc # Python 3.1

Other possibilities are:

1. A single pyr directory:

foo.py
pyr/
foo.f2b30a0d.pyc # Python 2.5
foo.f2d10a0d.pyc # Python 2.6
foo.f2d10a0d.pyo # Python 2.6 -O
foo.f2d20a0d.pyc # Python 2.6 -U
foo.0c4f0a0d.pyc # Python 3.1

2. A .pyr directory for each version of Python:

foo.py
f2b30a0d.pyr/ # Python 2.5
foo.pyc
f2d10a0d.pyr/ # Python 2.6/Python 2.6 -O
foo.pyc
foo.pyo
f2d20a0d.pyr/ # Python 2.6 -U
foo.pyc
0c4f0a0d.pyr/ # Python 3.1
foo.pyc
--
http://mail.python.org/mailman/listinfo/python-list

Still too slow

[elsa]

Hello again,

Thanks for the tips r.e random.ranint(). This improved matters
somewhat, however my program is still too slow. If anyone has any
further tips on how to speed it up, they would be much appreciated!

So, I'm calling evolve(L,limit) from the interactive prompt. L is
initally [[100],['NA']]. Ideally limit would be 10^7.

Here is my program:

import random
n=100

def evolve(L,limit):
global n
while nhttp://mail.python.org/mailman/listinfo/python-list

Re: Slow down while creating a big list and iterating over it

[MRAB]

Alf P. Steinbach wrote:

* marc magrans de abril:

Dear colleagues,

I was doing a small program to classify log files for a cluster of
PCs, I just wanted to simplify a quite repetitive task in order to
find errors and so.

My first naive implementation was something like:
patterns = []
while(logs):
pattern = logs[0]
new_logs = [l for l in logs if dist(pattern,l)>THERESHOLD]
entry = (len(logs)-len(new_logs),pattern)
patterns.append(entry)
logs = new_logs

Where dist(...) is the levenshtein distance (i.e. edit distance) and
logs is something like 1.5M logs (700 MB file). I thought that python
will be an easy choice although not really fast..

I was not surprised when the first iteration of the while loop was
taking ~10min. I thought "not bad, let's how much it takes". However,
it seemed that the second iteration never finished.

My surprise was big when I added a print instead of the list
comprehension:
new_logs=[]
for count,l in enumerate(logs):
   print count
   if dist(pattern,l)>THERESHOLD:
  new_logs.append(l)

The surprise was that the displayed counter was running ~10 times
slower on the second iteration of the while loop.

I am a little lost. Anyone knows the reson of this behavior?


It's on line 42 of your program. :-) That is, it's in the dist function. 
Evidently it doesn't like a more complex 'pattern'.



Find out which pattern is being used on the second iteration and then
try it on the first iteration. Is it just as slow? If so, then it's down
to the length/complexity of that pattern - a much longer/more complex
pattern might take much longer when computing the distance.


How should I write a program that deals with large data sets in python?


As in any other language. Try to avoid repeating the same computations. 
Try to make the data fit the computational task.



True. Basically, you're computing the distance between every pair of
logs!
--
http://mail.python.org/mailman/listinfo/python-list

Re: Function name unchanged in error message

[andrew cooke]

On Jan 29, 1:09 pm, Michele Simionato 
wrote:
> On Jan 29, 2:30 pm, andrew cooke  wrote:
>
> > Is there any way to change the name of the function in an error
> > message?  In the example below I'd like the error to refer to bar(),
> > for example (the motivation is related function decorators - I'd like
> > the wrapper function to give the same name)
>
> Use the decorator module which does the right 
> thing:http://pypi.python.org/pypi/decorator

curiously, decorator doesn't have this issue, because the way it
defines decorators uses *args.  so the error i gave cannot occur at
the level of the decorator - the extra arg is passed to the wrapped
function, and so the error message is correct because it is generated
by the inner function.

i need to look at my code; this might be the simplest solution of all.

thanks,
andrew
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Still too slow

[Philip Semanchuk]

On Jan 30, 2010, at 6:08 PM, elsa wrote:


Hello again,

Thanks for the tips r.e random.ranint(). This improved matters
somewhat, however my program is still too slow. If anyone has any
further tips on how to speed it up, they would be much appreciated!

So, I'm calling evolve(L,limit) from the interactive prompt. L is
initally [[100],['NA']]. Ideally limit would be 10^7.

Here is my program:

import random
n=100

def evolve(L,limit):
global n
while n

Hi Elsa,
I didn't follow the earlier discussion on this, so pardon me if I'm  
repeating what others have said.


First, have you profiled the code? That will help you find bottlenecks.

I haven't profiled it myself so I am shooting from the hip, but a  
couple of obvious speedups in event() jump out at me. I'd rewrite it  
like this:


def event():
choice = random.random()
if choice <= .3:
return 'b'
elif choice <= .4:
return 'd'
elif choice <= .5:
return 'm'

return None

Returning immediately when you've found your choice will save  
evaluating a couple of elifs -- not a big help, but it is a little. It  
comes at the cost having multiple exits to the function which is  
something I  prefer to avoid, but this function is small enough that  
having multiple exits won't get confusing.


Second, returning the Python object None rather than the string "None"  
allows you to change evolve() from this string comparison:

   if evnt!="None":

to this comparison to the Python singleton None:
   if evnt != None:

or this, which is even simpler (although it may not be faster):
   if evnt:

Hope this helps
Philip

--
http://mail.python.org/mailman/listinfo/python-list

Re: PEP 3147 - new .pyc format

[John Bokma]

MRAB  writes:

> The PEP has a .pyr directory for each .py file:
>
> foo.py
> foo.pyr/
> f2b30a0d.pyc # Python 2.5
> f2d10a0d.pyc # Python 2.6
> f2d10a0d.pyo # Python 2.6 -O
> f2d20a0d.pyc # Python 2.6 -U
> 0c4f0a0d.pyc # Python 3.1

wow: so much for human readable file names :-(

-- 
John Bokma   j3b

Hacking & Hiking in Mexico -  http://johnbokma.com/
http://castleamber.com/ - Perl & Python Development
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP 3147 - new .pyc format

[Alf P. Steinbach]

* John Bokma:

MRAB  writes:


The PEP has a .pyr directory for each .py file:

foo.py
foo.pyr/
f2b30a0d.pyc # Python 2.5
f2d10a0d.pyc # Python 2.6
f2d10a0d.pyo # Python 2.6 -O
f2d20a0d.pyc # Python 2.6 -U
0c4f0a0d.pyc # Python 3.1


wow: so much for human readable file names :-(


I agree.

Human readable filenames would be much better.


Cheers,

- Alf
--
http://mail.python.org/mailman/listinfo/python-list

Re: PEP 3147 - new .pyc format

[Carl Banks]

On Jan 30, 2:14 pm, John Roth  wrote:
> PEP 3147 has just been posted, proposing that, beginning in release
> 3.2 (and possibly 2.7) compiled .pyc and .pyo files be placed in a
> directory with a .pyr extension. The reason is so that compiled
> versions of a program can coexist, which isn't possible now.
>
> Frankly, I think this is a really good idea, although I've got a few
> comments.


-1

I think it's a terrible, drastic approach to a minor problem.  I'm not
sure why the simple approach of just appending a number (perhaps the
major-minor version, or a serial number) to the filename wouldn't
work, like this:

foo.pyc25

All I can think of is they are concerned with the typically minor
expense of listing the directory (to see if there's already .pyc??
file present).  This operation can be reasonably cached; when scanning
a directory listing it need only record all occurrencs of .pyc?? and
mark those modules as subject to version-specific .pyc files.  Anyway,
I'd expect the proposed -R switch would only be used in special cases
(like installation time) when a minor inefficiency would be tolerable.



> 1. Apple's MAC OS X should be mentioned, since 10.5 (and presumably
> 10.6) ship with both Python release 2.3 and 2.5 installed.
>
> 2. I think the proposed logic is too complex. If this is installed in
> 3.2, then that release should simply store its .pyc file in the .pyr
> directory, without the need for either a command line switch or an
> environment variable (both are mentioned in the PEP.)

This is utterly unacceptable.  Versioned *.pyc files should only be
optionally requested by people who have to deal multiple versions,
such as distro package maintainers.  For my projects I don't give a
flying F about versioned *.pyc and I don't want my project directory
cluttered with a million subdirectories.  (It would be a bit more
tolerable if my directory was merely cluttered with *.pyc?? files, but
I'd still rather Python didn't do that unless asked.)


> 3. Tool support. There are tools that look for the .pyc files; these
> need to be upgraded somehow. The ones that ship with Python should, of
> course, be fixed with the PEP, but there are others.

How will this affect tools like Py2exe?  Now you have a bunch of
identically-named *.pyc files.


> 4. I'm in favor of putting the source in the .pyr directory as well,
> but that's got a couple more issues. One is tool support, which is
> likely to be worse for source, and the other is some kind of algorithm
> for identifying which source goes with which object.

Now this just too much.  I didn't like the suggestion that I should be
forced to put up with dozens of subdirectories, now you want me to
force me to put the source files into the subdirectories as well?
That would be a deal-breaker.  Thankfully it is too ridiculous to ever
happen.


> Summary: I like it, but I think it needs a bit more work.

I hope it's replaced with something less drastic.


Carl Banks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP 3147 - new .pyc format

[MRAB]

Alf P. Steinbach wrote:

* John Bokma:

MRAB  writes:


The PEP has a .pyr directory for each .py file:

foo.py
foo.pyr/
f2b30a0d.pyc # Python 2.5
f2d10a0d.pyc # Python 2.6
f2d10a0d.pyo # Python 2.6 -O
f2d20a0d.pyc # Python 2.6 -U
0c4f0a0d.pyc # Python 3.1


wow: so much for human readable file names :-(


I agree.

Human readable filenames would be much better.


The names are the magic numbers. It's all in the PEP.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Still too slow

[John Posner]

On 1/30/2010 6:08 PM, elsa wrote:

Hello again,

Thanks for the tips r.e random.ranint(). This improved matters
somewhat, however my program is still too slow. If anyone has any
further tips on how to speed it up, they would be much appreciated!

So, I'm calling evolve(L,limit) from the interactive prompt. L is
initally [[100],['NA']]. Ideally limit would be 10^7.

Here is my program:

import random
n=100

def evolve(L,limit):
global n
while n

Elsa,

1. You changed the subject line from "For loop searching takes too 
long!" to "Still too slow". This causes newsreader programs to start a 
new discussion thread, which makes life difficult for people who need to 
refer back to previous messages. Please don't change the subject line 
any more.


2. You provided a very clear description of your original question:

  Now, what I need to do is randomly choose one myList[i], however the
  distribution of my random choice needs to be proportional to the
  values of myList[i][0].

This description should be the doc string for the chooseInd() function 
-- for example:


  def chooseInd(L,n):
  """
  randomly choose one L[i], so that the distribution
  of choices is proportional to the values of L[i][0]
  """

It is not clear (to me, anyway) what the other functions are trying to 
accomplish. So please add a doc string to each of these functions, with 
descriptions of similar clarity. This will help us a lot. And it will 
help you, too, if you return to the code after a few days/weeks/months 
of directing your attention elsewhere.


3. Please provide a *complete* transcript of an interactive session that 
exercises this code.


Tx,
John
--
http://mail.python.org/mailman/listinfo/python-list

Re: PEP 3147 - new .pyc format

[John Bokma]

MRAB  writes:

> Alf P. Steinbach wrote:
>> * John Bokma:
>>> MRAB  writes:
>>>
 The PEP has a .pyr directory for each .py file:

 foo.py
 foo.pyr/
 f2b30a0d.pyc # Python 2.5
 f2d10a0d.pyc # Python 2.6
 f2d10a0d.pyo # Python 2.6 -O
 f2d20a0d.pyc # Python 2.6 -U
 0c4f0a0d.pyc # Python 3.1
>>>
>>> wow: so much for human readable file names :-(
>>
>> I agree.
>>
>> Human readable filenames would be much better.
>>
> The names are the magic numbers. It's all in the PEP.

Naming files using magic numbers is really beyond me. The fact that the
above needs comments to explain what's what already shows to me that
there's a problem with this naming scheme. What if for one reason or
another I want to delete all pyc files for Python 2.5? Where do I look
up the magic number?

-- 
John Bokma   j3b

Hacking & Hiking in Mexico -  http://johnbokma.com/
http://castleamber.com/ - Perl & Python Development
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP 3147 - new .pyc format

[Neil Hodgson]

John Roth:

> 4. I'm in favor of putting the source in the .pyr directory as well,
> but that's got a couple more issues. One is tool support, which is
> likely to be worse for source, and the other is some kind of algorithm
> for identifying which source goes with which object.

   Many tools work recursively except for hidden directories so would
return both the source in the repository as well as the original source.
If you want to do this then the repository directory should be hidden by
starting with ".".

   Neil
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP 3147 - new .pyc format

[MRAB]

John Bokma wrote:

MRAB  writes:


Alf P. Steinbach wrote:

* John Bokma:

MRAB  writes:


The PEP has a .pyr directory for each .py file:

foo.py
foo.pyr/
f2b30a0d.pyc # Python 2.5
f2d10a0d.pyc # Python 2.6
f2d10a0d.pyo # Python 2.6 -O
f2d20a0d.pyc # Python 2.6 -U
0c4f0a0d.pyc # Python 3.1

wow: so much for human readable file names :-(

I agree.

Human readable filenames would be much better.


The names are the magic numbers. It's all in the PEP.


Naming files using magic numbers is really beyond me. The fact that the
above needs comments to explain what's what already shows to me that
there's a problem with this naming scheme. What if for one reason or
another I want to delete all pyc files for Python 2.5? Where do I look
up the magic number?


True. You might also want to note that "Python 2.6 -U" appears to have a
different magic number from "Python 2.6" and "Python 2.6 -O".

I don't know whether they always change for each new version.
--
http://mail.python.org/mailman/listinfo/python-list

Re: PEP 3147 - new .pyc format

[Steven D'Aprano]

On Sat, 30 Jan 2010 14:14:54 -0800, John Roth wrote:

> PEP 3147 has just been posted, proposing that, beginning in release 3.2
> (and possibly 2.7) compiled .pyc and .pyo files be placed in a directory
> with a .pyr extension. The reason is so that compiled versions of a
> program can coexist, which isn't possible now.


http://www.python.org/dev/peps/pep-3147/


Reading through the PEP, I went from an instinctive "oh no, that sounds 
horrible" reaction to "hmmm, well, that doesn't sound too bad". I don't 
think I need this, but I could live with it.

Firstly, it does sound like there is a genuine need for a solution to the 
problem of multiple Python versions. This is not the first PEP trying to 
solve it, so even if you personally don't see the need, we can assume 
that others do.

Secondly, the current behaviour will remain unchanged. Python will 
compile spam.py to spam.pyc (or spam.pyo with the -O switch) by default. 
If you don't need to support multiple versions, you don't need to care 
about this PEP. I like this aspect of the PEP very much. I would argue 
that any solution MUST support the status quo for those who don't care 
about multiple versions.

To get the new behaviour, you have to explicitly ask for it. You ask for 
it by calling python with the -R switch, by setting an environment 
variable, or explicitly providing the extra spam/.pyc files.

Thirdly, the magic file names aren't quite as magic as they appear at 
first glance. They represent the hexified magic number of the version of 
Python. More about the magic number here:

http://nedbatchelder.com/blog/200804/the_structure_of_pyc_files.html

Unfortunately the magic number doesn't seem to be documented anywhere I 
can find other than in the source code (import.c). The PEP gives some 
examples:

f2b30a0d.pyc # Python 2.5
f2d10a0d.pyc # Python 2.6
f2d10a0d.pyo # Python 2.6 -O
f2d20a0d.pyc # Python 2.6 -U
0c4f0a0d.pyc # Python 3.1

but how can one map magic numbers to versions, short of reading import.c? 
I propose that sys grow an object sys.magic which is the hexlified magic 
number.


> 2. I think the proposed logic is too complex. If this is installed in
> 3.2, then that release should simply store its .pyc file in the .pyr
> directory, without the need for either a command line switch or an
> environment variable (both are mentioned in the PEP.)

I disagree. Making the new behaviour optional is an advantage, even if it 
leads to extra complexity. It is pointless forcing .pyc files to be in a 
subdirectory if you don't need multiple versions.


> 3. Tool support. There are tools that look for the .pyc files; these
> need to be upgraded somehow. The ones that ship with Python should, of
> course, be fixed with the PEP, but there are others.

Third party tools will be the responsibility of the third parties.


> 4. I'm in favor of putting the source in the .pyr directory as well, but
> that's got a couple more issues. One is tool support, which is likely to
> be worse for source, and the other is some kind of algorithm for
> identifying which source goes with which object.

It certain does.

What's the advantage of forcing .py files to live inside a directory with 
the same name?

Modules:
mymodule.py  =>  mymodule/mymodule.py

Packages:
mypackage/__init__.py  =>  mypackage/__init__/__init__.py
mypackage/spam.py  =>  mypackage/spam/spam.py


Seems like a pointless and annoying extra layer to me.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Still too slow

[Alf P. Steinbach]

* elsa:


Thanks for the tips r.e random.ranint(). This improved matters
somewhat, however my program is still too slow. If anyone has any
further tips on how to speed it up, they would be much appreciated!

So, I'm calling evolve(L,limit) from the interactive prompt. L is
initally [[100],['NA']].


This would produce an unhandled exception since your code tries to compute the 
sum of the values in the list. You can't add strings and numbers.




Ideally limit would be 10^7.

Here is my program:

import random
n=100

def evolve(L,limit):
global n
while n

This is the problematic part of your program, because it uses time linear in the 
length of L.


Consider if you know the sum s_low of the lower half of the array, and the sum 
s_high of the higher half of the array. Then you can choose the lower half with 
probability s_low/(s_low+s_high), otherwise the higher half. Then repeat this 
recursively for the half chosen, until you get down to a single array element.


This requires keeping track of the sums of halfs of the array, e.g. in a tree 
like structure or a set of arrays of diminishing lengths. Generally it will use 
some constant factor times twice the storage that you're now using. But it 
reduces the time for choosing an index from O(n) to O(log n).


For example, if you have a logical structure like

   *
  / \
 *   1
/ \
   98  1

then at the top level * you choose the left branch with probability 99/(99+1) = 
99/100 = 0.99. At the second level * you choose the right branch with 
probability 1/(98+1) = 1/99. The combined probability of choosing that lower 1 
is then 0.99*(1/99) = 0.01, as it should be.




def event():
choice = random.random()
if choice <= .3:
event='b'
elif choice <= .4:
event ='d'
elif choice<= .5:
event = 'm'
else:
event = 'None'
return event


Note that you can double the speed of your program by removing the 'None' value. 
Adjust the limits you're checking against so as to retain the same probabilities 
of the choices. Further constant factor speedup is possible by simply returning 
a function to Do Stuff instead of returning a character saying what to do.





def action(event, L, index):
global n
if event == 'b':
L[index][0]+=1
n +=1
elif event == 'd':
L[index][0]-=1
n -=1
elif event == 'm':
L.append([1,index])
n +=1


thanks in advance,


An observation about design: you have a global n (the current total sum) and an 
array L that you're passing around everywhere, so that it's effectively also a 
global. This indicates that they should ideally instead be attributes of an 
object, with your routines above as methods. However in Python this may cause 
some overhead, so, perhaps first make it Work (and then make a Copy). :-)



Cheers & hth.,

- Alf
--
http://mail.python.org/mailman/listinfo/python-list

Re: Still too slow

[Alf P. Steinbach]

* Alf P. Steinbach:

* elsa:


Thanks for the tips r.e random.ranint(). This improved matters
somewhat, however my program is still too slow. If anyone has any
further tips on how to speed it up, they would be much appreciated!

So, I'm calling evolve(L,limit) from the interactive prompt. L is
initally [[100],['NA']].


This would produce an unhandled exception since your code tries to 
compute the sum of the values in the list. You can't add strings and 
numbers.




Ideally limit would be 10^7.

Here is my program:

import random
n=100

def evolve(L,limit):
global n
while n

This is the problematic part of your program, because it uses time 
linear in the length of L.


Consider if you know the sum s_low of the lower half of the array, and 
the sum s_high of the higher half of the array. Then you can choose the 
lower half with probability s_low/(s_low+s_high), otherwise the higher 
half. Then repeat this recursively for the half chosen, until you get 
down to a single array element.


This requires keeping track of the sums of halfs of the array, e.g. in a 
tree like structure or a set of arrays of diminishing lengths. Generally 
it will use some constant factor times twice the storage that you're now 
using. But it reduces the time for choosing an index from O(n) to O(log n).


For example, if you have a logical structure like

   *
  / \
 *   1
/ \
   98  1

then at the top level * you choose the left branch with probability 
99/(99+1) = 99/100 = 0.99. At the second level * you choose the right 
branch with probability 1/(98+1) = 1/99. The combined probability of 
choosing that lower 1 is then 0.99*(1/99) = 0.01, as it should be.


Oh, forgot to add: since your array's first item will be the one that's far most 
often chosen a "blind" sort of literal implementation of the above will perhaps 
slow down the program instead of speeding it up, because it only speeds up those 
searches that progress beyond the first item. But anyway, I'm guessing that 
checking the first item first, as a special case, will really improve things. 
Only when it turns out that it's not the first item, apply binary search.




def event():
choice = random.random()
if choice <= .3:
event='b'
elif choice <= .4:
event ='d'
elif choice<= .5:
event = 'm'
else:
event = 'None'
return event


Note that you can double the speed of your program by removing the 


/s/double/increase/g


'None' value. Adjust the limits you're checking against so as to retain 
the same probabilities of the choices. Further constant factor speedup 
is possible by simply returning a function to Do Stuff instead of 
returning a character saying what to do.





def action(event, L, index):
global n
if event == 'b':
L[index][0]+=1
n +=1
elif event == 'd':
L[index][0]-=1
n -=1
elif event == 'm':
L.append([1,index])
n +=1


thanks in advance,


An observation about design: you have a global n (the current total sum) 
and an array L that you're passing around everywhere, so that it's 
effectively also a global. This indicates that they should ideally 
instead be attributes of an object, with your routines above as methods. 
However in Python this may cause some overhead, so, perhaps first make 
it Work (and then make a Copy). :-)


Cheers & hth.,

- Alf
--
http://mail.python.org/mailman/listinfo/python-list

Re: Still too slow

[elsa]

Hi John and others,

sorry about my etiquette errors. As you can tell I'm a newbie, and
appreciate all the help I can get. I'm trying to master this thing
with only the net and a couple of books as tutors.

Here is what I'm running at the interactive prompt:

>>> import myBDM
>>> L=[[100,'NA']]
>>> myBDM.evolve(L,10)

Ideally, I'd like to bump it up to myBDM.evolve(L,1000). L keeps
track of the number of subpopulations, how many individuals are in
each subpopulation, and the parent subpopulation each subpopulation
arose from.

Here is my code again:


import random
n=100

def evolve(L,limit):
"""
evolves the population until the population size reaches limit, by
choosing an individual of a particular subpopulation
type, then randomly performing
a birth, death, or mutation on this individual
"""
global n
while nhttp://mail.python.org/mailman/listinfo/python-list

Re: Microsoft Office Word and Python (Win XP)

[Dave Angel]

peskar.m...@hotmail.com wrote:

21 days has passed and still noone is willing to help :-(

  

ch /willing/able/

I wouldn't say no-one, even then, since there were at least 10 messages 
in the thread on the 19th and 20th.  Presumably they weren't all from you.


If you were doing this to text files, I would have been happy to help.  
But interacting with MS Word isn't easy, and I don't know how.  If 
you're a beginner as you stated, you need to pick an easier first task.


If you're determined to do it, best advice I could give is to head over 
to the python-Win32 mailing list.  At least they're used to dealing with 
COM, and other MS-specific stuff.


DaveA

--
http://mail.python.org/mailman/listinfo/python-list

Re: PEP 3147 - new .pyc format

[Ben Finney]

Steven D'Aprano  writes:

> Unfortunately the magic number doesn't seem to be documented anywhere
> I can find other than in the source code (import.c). The PEP gives
> some examples:
>
> f2b30a0d.pyc # Python 2.5
> f2d10a0d.pyc # Python 2.6
> f2d10a0d.pyo # Python 2.6 -O
> f2d20a0d.pyc # Python 2.6 -U
> 0c4f0a0d.pyc # Python 3.1
>
> but how can one map magic numbers to versions, short of reading
> import.c? 

Mapping magic numbers to versions is infeasible and will be incomplete:
Any mapping that exists in (say) Python 3.1 can't know in advance what
the magic number will be for Python 4.5.

The more important mapping is from version to magic number.

> I propose that sys grow an object sys.magic which is the hexlified magic 
> number.

The ‘imp’ module already has this::

>>> import sys
>>> import imp
>>> sys.version
'2.5.4 (r254:67916, Jan 24 2010, 16:09:54) \n[GCC 4.4.2]'
>>> imp.get_magic().encode('hex')
'b3f20d0a'

Unfortunately, I think the examples in the PEP have mangled the magic
number into little-endian byte ordering.

-- 
 \ “I got up the other day, and everything in my apartment has |
  `\   been stolen and replaced with an exact replica.” —Steven Wright |
_o__)  |
Ben Finney
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Still too slow

[MRAB]

elsa wrote:

Hi John and others,

sorry about my etiquette errors. As you can tell I'm a newbie, and
appreciate all the help I can get. I'm trying to master this thing
with only the net and a couple of books as tutors.

Here is what I'm running at the interactive prompt:


[snip]


def event():
"""
randomly chooses a birth (prob .3), death (prob .1) or mutation
(prob .1) event
"""
choice = random.random()
if choice <= .3:
event='b'
elif choice <= .4:
event ='d'
elif choice<= .5:
event = 'm'
else:
event = 'None'
return event


[snip]
Here's my line of thought:

Currently, 0 <= choice < 1.0.

If you multiplied the random number by 10 then the probability
thresholds could all be ints, and if you're using ints then 'choice'
could be an index into a list:

event_choices = ['b', 'b', 'b', 'd', 'm'] + ['None'] * 5

which then suggests using random.choice() instead:

def event():
"""
	randomly chooses a birth (prob .3), death (prob .1) or mutation (prob 
.1) event

"""
return random.choice(event_choices)

and you could put the event code inline instead of in a function, which
saves the cost of a function call.

Incidentally, for a probability of 0.3 the test should be "choice <
0.3", etc, not "choice <= 0.3", etc, although I suspect that you won't
notice any difference in the results! :-)
--
http://mail.python.org/mailman/listinfo/python-list

Re: PEP 3147 - new .pyc format

[Paul Rubin]

Ben Finney  writes:
>> 0c4f0a0d.pyc # Python 3.1
> Mapping magic numbers to versions is infeasible and will be incomplete:
> Any mapping that exists in (say) Python 3.1 can't know in advance what
> the magic number will be for Python 4.5.

But why do the filenames have magic numbers instead of version numbers?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP 3147 - new .pyc format

[Steven D'Aprano]

On Sat, 30 Jan 2010 18:40:42 -0800, Paul Rubin wrote:

> Ben Finney  writes:
>>> 0c4f0a0d.pyc # Python 3.1
>> Mapping magic numbers to versions is infeasible and will be incomplete:
>> Any mapping that exists in (say) Python 3.1 can't know in advance what
>> the magic number will be for Python 4.5.
> 
> But why do the filenames have magic numbers instead of version numbers?

The magic number changes with each incompatible change in the byte code 
format, which is not the same as each release. Selected values taken from 
import.c:

   Python 2.5a0: 62071
   Python 2.5a0: 62081 (ast-branch)
   Python 2.5a0: 62091 (with)
   Python 2.5a0: 62092 (changed WITH_CLEANUP opcode)
   Python 2.5b3: 62101 (fix wrong code: for x, in ...)
   Python 2.5b3: 62111 (fix wrong code: x += yield)
   Python 2.5c1: 62121 (fix wrong lnotab with for loops and
storing constants that should have been
removed)
   Python 2.5c2: 62131 (fix wrong code: for x, in ... in 
listcomp/genexp)


http://svn.python.org/view/python/trunk/Python/import.c?view=markup

The relationship between byte code magic number and release version 
number is not one-to-one. We could have, for the sake of the argument, 
releases 3.2.3 through 3.5.0 (say) all having the same byte codes. What 
version number should the .pyc file show?


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Still too slow

[Dave Angel]

John Posner wrote:
On 
1/30/2010 6:08 PM, elsa wrote:

Hello again,

Thanks for the tips r.e random.ranint(). This improved matters
somewhat, however my program is still too slow. If anyone has any
further tips on how to speed it up, they would be much appreciated!

So, I'm calling evolve(L,limit) from the interactive prompt. L is
initally [[100],['NA']]. Ideally limit would be 10^7.

Here is my program:

import random
n=100

def evolve(L,limit):
global n
while n

Elsa,

1. You changed the subject line from "For loop searching takes too 
long!" to "Still too slow". This causes newsreader programs to start a 
new discussion thread, which makes life difficult for people who need 
to refer back to previous messages. Please don't change the subject 
line any more.


2. You provided a very clear description of your original question:

  Now, what I need to do is randomly choose one myList[i], however the
  distribution of my random choice needs to be proportional to the
  values of myList[i][0].

This description should be the doc string for the chooseInd() function 
-- for example:


  def chooseInd(L,n):
  """
  randomly choose one L[i], so that the distribution
  of choices is proportional to the values of L[i][0]
  """

It is not clear (to me, anyway) what the other functions are trying to 
accomplish. So please add a doc string to each of these functions, 
with descriptions of similar clarity. This will help us a lot. And it 
will help you, too, if you return to the code after a few 
days/weeks/months of directing your attention elsewhere.


3. Please provide a *complete* transcript of an interactive session 
that exercises this code.


Tx,
John



Remarks aimed at elsa, not John.

You used tabs in your source code, which don't survive mail very well.  
So indentation is different in my quoted copy than presumably in your 
original.  No bug there, but I much prefer having my editor expand all 
tabs into spaces (4 colums)


You have a typo in the initialization of L.  Try pasting it from a 
working session, instead of retyping it.  Presumably you wanted

  L= [[100,'NA']].rather than [[100],['NA']].
The latter value throws an exception in the code.

You could start simply by changing the values in the event() function.  
No point in returning None half the time when that'll accomplish 
nothing.  So just double the fractions you're checking against, or scale 
the random value.


The pattern produced is interesting.  The only count that gets very big 
is L[0][0]. ("Them what has, gets")   Lots of the other counts reach 
zero, and once they do, they won't ever change.


The algorithm is O(N*N), and it already takes a long time for 10**4.  So 
10**7 would be enormous.


If the improvement needed were minor, you should profile it, figure 
where all the time is spent, and optimize that.  But I think that's 
hopeless with this data structure.


At a guess, I'd think you should build a single list that grows to size 
'limit', where you start with 100 items of value "NA".  No counts 
needed, because they're all implicitly count of 1.  Then write your code 
to manipulate that list.  When you finish, construct the list you really 
want, by using something like the groupby() function.


DaveA

--
http://mail.python.org/mailman/listinfo/python-list

Re: Function name unchanged in error message

[Steven D'Aprano]

On Sat, 30 Jan 2010 14:26:43 -0800, andrew cooke wrote:

> On Jan 29, 11:50 am, exar...@twistedmatrix.com wrote:
>> new.function and new.code will let you construct new objects with
>> different values (and copying over whichever existing attributes you
>> want to preserve).
> 
> unfortunately new is deprecated and dropped from 3.  i can't see how the
> same functionality is available in the types module for 3 - am i missing
> something obvious?

You have to get the constructor from an existing object. type(obj) will 
always return the type object, which you can use as a constructor.


>>> newfunction = type(lambda: None)
>>> newcode = type((lambda: None).__code__)


Possibly even easier:

>>> import types
>>> types.FunctionType is newfunction
True
>>> types.CodeType is newcode
True

So that's two ways to get a constructor. Now all we need is to learn how 
they work:


>>> help(newfunction)

Help on class function in module builtins:

class function(object)
 |  function(code, globals[, name[, argdefs[, closure]]])
 |
 |  Create a function object from a code object and a dictionary.
 [...]


>>> help(newcode)

Help on class code in module builtins:

class code(object)
 |  code(argcount, kwonlyargcount, nlocals, stacksize, flags, codestring,
 |constants, names, varnames, filename, name, firstlineno,
 |lnotab[, freevars[, cellvars]])
 |
 |  Create a code object.  Not for the faint of heart.
 [...]


Not for the faint of heart indeed! The best way of using this is to copy 
the parameters from an existing code object, one created by using def. 
Look for attributes of f.__code__ starting with "co_".

Don't forget compile as well, which may help.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP 3147 - new .pyc format

[Alf P. Steinbach]

* Steven D'Aprano:

On Sat, 30 Jan 2010 18:40:42 -0800, Paul Rubin wrote:


Ben Finney  writes:

0c4f0a0d.pyc # Python 3.1

Mapping magic numbers to versions is infeasible and will be incomplete:
Any mapping that exists in (say) Python 3.1 can't know in advance what
the magic number will be for Python 4.5.

But why do the filenames have magic numbers instead of version numbers?


The magic number changes with each incompatible change in the byte code 
format, which is not the same as each release. Selected values taken from 
import.c:


   Python 2.5a0: 62071
   Python 2.5a0: 62081 (ast-branch)
   Python 2.5a0: 62091 (with)
   Python 2.5a0: 62092 (changed WITH_CLEANUP opcode)
   Python 2.5b3: 62101 (fix wrong code: for x, in ...)
   Python 2.5b3: 62111 (fix wrong code: x += yield)
   Python 2.5c1: 62121 (fix wrong lnotab with for loops and
storing constants that should have been
removed)
   Python 2.5c2: 62131 (fix wrong code: for x, in ... in 
listcomp/genexp)



http://svn.python.org/view/python/trunk/Python/import.c?view=markup

The relationship between byte code magic number and release version 
number is not one-to-one. We could have, for the sake of the argument, 
releases 3.2.3 through 3.5.0 (say) all having the same byte codes. What 
version number should the .pyc file show?


I don't know enough about Python yet to comment on your question, but, just an 
idea: how about a human readable filename /with/ some bytecode version id (that 
added id could be the magic number)?


I think that combo could serve both the human and machine needs, so to speak. 
:-)


Cheers,

- Alf
--
http://mail.python.org/mailman/listinfo/python-list

Re: PEP 3147 - new .pyc format

[Steven D'Aprano]

On Sun, 31 Jan 2010 04:44:18 +0100, Alf P. Steinbach wrote:

>> The relationship between byte code magic number and release version
>> number is not one-to-one. We could have, for the sake of the argument,
>> releases 3.2.3 through 3.5.0 (say) all having the same byte codes. What
>> version number should the .pyc file show?
> 
> I don't know enough about Python yet to comment on your question, but,
> just an idea: how about a human readable filename /with/ some bytecode
> version id (that added id could be the magic number)?

Sorry, that still doesn't work. Consider the hypothetical given above. 
For simplicity, I'm going to drop the micro point versions, so let's say 
that releases 3.2 through 3.5 all use the same byte-code. (In reality, 
you'll be likely looking at version numbers like 3.2.1 rather than just 
3.2.) Now suppose you have 3.2 and 3.5 both installed, and you look 
inside your $PYTHONPATH and see this:

spam.py
spam.pyr/
3.2-f2e70a0d.pyc


It would be fairly easy to have the import machinery clever enough to 
ignore the version number prefix, so that Python 3.5 correctly uses 
3.2-f2e70a0d.pyc. That part is easy.

(I trust nobody is going to suggest that Python should create multiple 
redundant, identical, copies of the .pyc files. That would be just lame.)

But now consider the human reader. You want human-readable file names for 
the benefit of the human reader. How is the human reader supposed to know 
that Python 3.5 is going to use 3.2-f2e70a0d.pyc? 

Suppose I'm running Python 3.5, and have a troubling bug, and I think "I 
know, maybe there's some sort of problem with the compiled byte code, I 
should delete it". I go looking for spam.pyr/3.5-*.pyc and don't find 
anything. 

Now I have two obvious choices: 

(1) Start worrying about why Python 3.5 isn't writing .pyc files. Is my 
installation broken? Have I set the PYTHONDONTWRITEBYTECODE environment 
variable? Do I not have write permission to the folder? WTF is going on? 
Confusion will reign.

(2) Learn that the 3.2- prefix is meaningless, and ignore it.

Since you have to ignore the version number prefix anyway, let's not lay 
a trap for people by putting it there. You need to know the magic number 
to do anything sensible with the .pyc files other than delete them, so 
let's not pretend otherwise.

If you don't wish to spend time looking up the magic number, the solution 
is simple: hit it with a sledgehammer. Why do you care *which* 
specific .pyc file is being used anyway?

rm -r spam.pyr/

or for the paranoid:

rm spam.pyr/*.py[co]

(Obviously there will be people who care about the specific .pyc file 
being deleted. Those people can't use a sledgehammer, they need one of 
those little tack-hammers with the narrow head. But 99% of the time, a 
sledgehammer is fine.)

Frankly, unless you're a core developer or you're hacking byte-code, you 
nearly always won't care about the .pyc files. You want the compiler to 
manage them. And for the few times you do care, it isn't hard to find out:

>>> import binascii, imp
>>> binascii.hexlify(imp.get_magic())
b'4f0c0d0a'

Stick that in your "snippets" folder (you do have a snippets folder, 
don't you?) and, as they say Down Under, "She'll be right mate".


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list