Fast 2d brown/pink noise generation?
Dear NG, I'm looking for a fast way to produce 2d-noise images with 1/f or 1/f^2 spectrum. I currently generate the noise via inverse FFT, but since I need lots of data (~10^7 for a monte carlo simulation) it needs to be really fast. Does someone know a faster way than my approach? - Dimensionality is between 20x20 and 100x100 - The spectrum doesn't need to be exactly pink/brown, an approximation is fine. - Implementation in either matlab or scientific python (LAPACK anyway) Thanks a lot for hints, literature, algorithms! Mathias -- http://mail.python.org/mailman/listinfo/python-list
Fast 2d brown/pink noise generation?
Dear NG, I'm looking for a fast way to produce 2d-noise images with 1/f or 1/f^2 spectrum. I currently generate the noise via inverse FFT, but since I need lots of data (~10^7 for a monte carlo simulation) it needs to be really fast. Does someone know a faster way than my approach? - Dimensionality is between 20x20 and 100x100 - The spectrum doesn't need to be exactly pink/brown, an approximation is fine. - Implementation in either matlab or scientific python (LAPACK anyway) Thanks a lot for hints, literature, algorithms! Mathias -- http://mail.python.org/mailman/listinfo/python-list
Parallelization with Python: which, where, how?
Dear NG, I have a (pretty much) "emberassingly parallel" problem and look for the right toolbox to parallelize it over a cluster of homogenous linux workstations. I don't need automatic loop-parallelization or the like since I prefer to prepare the work packets "by hand". I simply need - to specify a list of clients - a means of sending a work packet to a free client and receiving the result (hopefully automatically without need to login to each one) - optionally a timeout mechanism if a client doesn't respond - optionally help for debugging of remote clients So far I've seen scipy's COW (cluster of workstation) package, but couldn't find documentation or even examples for it (and the small example in the code crashes...). I've noticed PYRO as well, but didn't look too far yet. Can someone recommend a parallelization approach? Are there examples or documentation? Has someone got experience with stability and efficiency? Thanks a lot, Mathias -- http://mail.python.org/mailman/listinfo/python-list
Q: Scheduling in scipy.cow
Dear NG, can somebody tell me how the work packages are scheduled to the workers? From the code it seems to me like a a static distribution, ie each worker gets the same amount of work, not taking into account if a faster worker already finished all work packages. Thanks, Mathias PS: Is there a better NG for scipy related questions, espescially scipy.cow? -- http://mail.python.org/mailman/listinfo/python-list
Segmentation faults using threads
Dear ng, I use the thread module (not threading) for a client/server app where I distribute large amounts of pickled data over ssh tunnels. Now I get regular Segmentation Faults during high load episodes. I use a semaphore to have pickle/unpickle run nonthreaded, but I still get frequent nondeterministic segmentation faults. Since there is no traceback after a sf, I have no clue what exactly happened, and debugging a multithreaded app is no fun anyway :( Can someone recommend me how to get extra info during such a crash? Or any other opinion on where the problem might lie? Thanks a lot, Mathias -- http://mail.python.org/mailman/listinfo/python-list
Re: Segmentation faults using threads
> Hi, it would be helpful if you posted a minimalistic code snippet > which showed the problem you describe. > > Daniel I wish I could! If I knew exactly where the effect takes place I could probably circumvent it. All I know know is that it happens under high load and with a lot of waitstates I can reduce the propability of crashing. So there must be some race condition somewhere I think. Is there a way to analyze where the crash took place? I guess I can have a core dumped and somehow analyze it, but that's probably very hard to do. Would a profiler work which records the function call structure? Does someone have experience with threading in python - are there non-threadsafe functions I should know about? Thanks, Mathias -- http://mail.python.org/mailman/listinfo/python-list
Re: Segmentation faults using threads
John Nagle wrote: > Daniel Nogradi wrote: >>> I use the thread module (not threading) for a client/server app where I >>> distribute large amounts of pickled data over ssh tunnels. > > What module are you using for SSH? > > What's in your program that isn't pure Python? > The problem is probably in some non-Python component; you shouldn't > be able to force a memory protection error from within Python code. > > Also note that the "marshal" module may be unsafe. > > John Nagle I'm using os.popen2() to pipe into an ssh session via stdin/stdout. That's probably not the elegant way... Other modules: scipy 0.3.2 (with Numeric 24.2) and python 2.4 Does pickle/cPickle count as part of the marshal module? Mathias -- http://mail.python.org/mailman/listinfo/python-list
Re: Segmentation faults using threads
PS: setting sys.setcheckinterval(1) reduces the probablilty of a failure as well, but definetely at a performance cost. -- http://mail.python.org/mailman/listinfo/python-list
Re: Segmentation faults using threads
>
> What module are you using for SSH?
>
> What's in your program that isn't pure Python?
> The problem is probably in some non-Python component; you shouldn't
> be able to force a memory protection error from within Python code.
>
It looks like the error could be in scipy/Numeric, when a large array's
type is changed, like this:
>>> from scipy import *
>>> a=zeros(1,'b') #100 MiB
>>> b=a.copy().astype('d') #800 MiB, ok
>>> a=zeros(10,'b')#1GiB
>>> b=a.copy().astype('d') #8GiB, fails with sf
Segmentation fault
if I use zeros directly for allocation of the doubles it works as expected:
>>> from scipy import *
>>> a=zeros(10,'d')#8GiB, fails with python exception
Traceback (most recent call last):
File "", line 1, in ?
MemoryError: can't allocate memory for array
>>>
I use python 2.4, but my scipy and Numeric aren't quite up-to-date:
scipy version 0.3.2, Numeric v 24.2
--
http://mail.python.org/mailman/listinfo/python-list
How to get rid of 3.7.2 (32-bit) in favour of 3.7.2 (64-bit)
Hello, I'm as new to this mailinglist as I'm to Win10 (actually les than 24 hour). Just installed both on my new Thinkpad. Well, accidently I also just installed Python 3.7.2 (32-bit). Now I have trouble to get rid of it (in favour of the 64-bit version). I tried to "uninstall the App" more6 times now by Window :: ?settings? :: Apps :: Select "Python 3.7.2 (32-bit)" Klick Uninstall ... Admin-Password ... The "Python 3.7.2 (32-bit) Setup" sayes "Uninstall was successful" An hint me to this mailinglist if I encounter problems. Well, here I am. Thus my Question are: Does I try to uninstall the wrong way (New to Windows)? Does the Package got some pitfalls stiking back? What can I do to get rid of the "wrong" package? Thank You for Your Answer in ahead. Sincerely Mat -- Mathias Feiler - Universität Hohenheim Stabsstelle für Datenschutz Raum 04.35/-150 Schloss, äußerer Osthof-Süd Schwerzstraße 46 | 70599 Stuttgart Tel. +49 711 459 23949 | Fax +49 711 459 22432 -- https://mail.python.org/mailman/listinfo/python-list
Listening for a certain keypress in the background
I have created a Python program that is running on my desktop. How can I make it listen for a certain keypress (say, Windows-key + space) in a controlled fashion even when it is not the program having focus? I need to do this running GNOME under Mandrake GN/Linux 10. /Mathias -- http://mail.python.org/mailman/listinfo/python-list
Listening for keypress in the background
I have created a small Python program that is running on my desktop. How can I make it listen for a certain keypress (say, Windows-key + space) in a controlled fashion even when it is not the program having focus? I need to do this running under GNOME in Mandrake GN/Linux 10. /Mathias -- http://mail.python.org/mailman/listinfo/python-list
Re: Listening for keypress in the background
Peter Hansen <[EMAIL PROTECTED]> writes: >> How can I make it listen for a certain keypress (say, Windows-key + >> space) in a controlled fashion even when it is not the program having >> focus? >> >> I need to do this running under GNOME in Mandrake GN/Linux 10. > > Search Google for "python keylogger". Thanks, good idea! The relevant pages I found only link to Windows-specific keyloggers though. I just got another idea on a solution for my main problem (invoking my sleeping app) and that would be to let my window manager execute a small "invoker" program that would communicate with the sleeping app. The question is, which is the easiest way to communicate with a running Python program? Signals? TCP/IP (not really, right?) Pipes? -- http://mail.python.org/mailman/listinfo/python-list
Re: Listening for keypress in the background
Mathias Dahl <[EMAIL PROTECTED]> writes: > Peter Hansen <[EMAIL PROTECTED]> writes: > >>> How can I make it listen for a certain keypress (say, Windows-key + >>> space) in a controlled fashion even when it is not the program having >>> focus? >>> >>> I need to do this running under GNOME in Mandrake GN/Linux 10. >> >> Search Google for "python keylogger". > > Thanks, good idea! The relevant pages I found only link to > Windows-specific keyloggers though. > > I just got another idea on a solution for my main problem (invoking my > sleeping app) and that would be to let my window manager execute a > small "invoker" program that would communicate with the sleeping > app. The question is, which is the easiest way to communicate with a > running Python program? Signals? TCP/IP (not really, right?) Pipes? I solved it using named pipes (mkfifo, open and read) plus a simple echo x > pipe to invoke. -- http://mail.python.org/mailman/listinfo/python-list
Re: Connecting to Firebird database using Kinterbasdb+Python
Maurice LING wrote: > I've been using FB1.5 and access the database using Kinterbasdb + > Python. My connection is established using kinterbasdb.connect() method > and the parameters host, dns, database, user, password are all defaulted > to 'None'. > > On my own machine running Mac OSX 10.3, I can connect using the following: > host = 'localhost' > database = '' > user = '' > password = '' > > At the same time, I can also connect if I set host=None on my machine. > > However, I cannot use 'localhost' on a shared Linux machine (not > allowed. Don't ask why, system admin's mandate. And the Linux machine is > without inetd). So when I set host=None, I get this error: Just for my understanding: if you start your script on the machine hosting the DB you're able to connect. If you start it on a different machine, you are not able to connect without giving a hostname? Whats wrong with offering the the name (or IP-address) of your MacOS-box instead of 'localhost' to the script? Mathias -- http://mail.python.org/mailman/listinfo/python-list
Re: Connecting to Firebird database using Kinterbasdb+Python
Maurice LING wrote: > What I am trying to do is "port" a workable program from my own machine > (Mac OSX) to a larger machine (Linux). So, the DB and the program are > also on the same Linux machine. > > On the Linux machine, I cannot use localhost, so I set host parameter in > kinterbasdb.connect() method to None, which is the default kinterbasdb > uses. On Linux machine, I get the error as posted. On my Mac OSX > machine, initiating host parameter to 'localhost' or None has no > difference, it works. How do you connect to the Linux-DB using tools like isql? Check you firebird config (most likely in /etc/firebird2/firebird.conf) for the parameters "RemoteServicePort" and "RemoteBindAddress". And at least try "netstat -alp | grep fbserv" (as root) to check if the firebird is running properly. If you're running the classic server you can simply follow Grig's advice and connect "directly" to the db files. Mathias -- http://mail.python.org/mailman/listinfo/python-list
Web-Forms
Hi, I need to access some information from a web site which are only accessible through a form. Thus for each bucket of data you have to fill out the form, submit it and wait for an answer. Very easy - if you don't have to check some hundred times. Of course this site requires cookies, it is not directly accessible by URL and so on. All that nice stuff used to make a web site more professional;) But now the question: how can this be solved by using Python? Mathias -- http://mail.python.org/mailman/listinfo/python-list
Re: Web-Forms
Michele Simionato wrote: > Use twill:http://www.idyll.org/~t/www-tools/twill.html I'll do so. Twill is great, anyway thanks for all other responses too. Mathias -- http://mail.python.org/mailman/listinfo/python-list
Invoking GUI for app running in background with a keypress
I am creating a small app called PyQe (http://klibb.com/cgi-bin/wiki.pl/PyQe) to launch commands and programs quickly. I works more or less as I want it now and I have managed to make my window manager (Metacity) under Mandrake GNU/Linux start my program so that it can be started easily with just a keypress. Now, the only annoyance I have is that when I have not started the program for a while, the OS seems to not have Python or the program in "the cache" (or whatever, what I mean is that if I have started the program "recently" it starts fast the next time) anymore, which means that the program, even though quite small, takes about a second to start. This is too slow to feel good given the nature of the program (a quick launcher). I have tried making it start faster by calling python with the -S switch and by compiling my program to a .pyc file. It has not helped much. So, I was wondering if I could have my program running in the background and instead capture a certain keystroke (the same one I have my window manager to capture now) to make the GUI appear. How does one go about doing this? I found a small program written in C (xbindkeys) that can do this and understand that it probably involves a lot of "low-level" stuff in X which feels a bit "scary" :). Any clues of doing this "easily" in Python + some module? /Mathias -- http://mail.python.org/mailman/listinfo/python-list
Re: Invoking GUI for app running in background with a keypress
Jeremy Moles <[EMAIL PROTECTED]> writes: > If you want to get crazy you can poll() on one of the evdev nodes > (/dev/input/event*) and behave accordingly. I do this in a C application > we use to do the exact same thing you're talking about. > > Each successful read from the device returns a 16-byte input_event > struct (or similar, I'm going from memory here) that represents a key > action. > > A google search returned this: > > http://svn.navi.cx/misc/trunk/python/evdev/evdev.py Thanks for the pointer. That sounds crazy indeed! Maybe a bit too crazy (low-level) for me... :) I tested the code very briefly: $ python evdev.py /dev/input/event0 Traceback (most recent call last): File "evdev.py", line 549, in ? demo() File "evdev.py", line 35, in demo dev = Device(sys.argv[1]) File "evdev.py", line 91, in __init__ self.fd = os.open(filename, os.O_RDWR | os.O_NONBLOCK) OSError: [Errno 13] Permission denied: '/dev/input/event0' Loggin in (su:ing) as root solves that problem but I'm not sure I want to require the user being root to be able to run my program. Anyway, thanks again for the hint! /Mathias -- http://mail.python.org/mailman/listinfo/python-list
64 bit Python
Hi, one of my colleagues got some trouble with a program handling large amounts of data. I figured out that a 32 bit application on HP-UX cannot address more than 1 GB of memory. In fact (I think due to the overhead of memory management done by python) a python application cannot use much more than 500 MB of "real" data. For this reason I've created a 64 bit version of python 2.3.5. I've tested a simple C program before to make sure its able to address the whole available memory (8 GB main memory and about 12 GB swap). The simple example in C works, but the python script didn't. I was able to create only 2 GB of python objects. The application occupied approximately 2.2 GB of memory. After that, python failed with a memory error. Is there any internal restriction on the size of the heap? Mathias -- http://mail.python.org/mailman/listinfo/python-list
Re: 64 bit Python
Ivan Voras wrote: > Mathias Waack wrote: >> amounts of data. I figured out that a 32 bit application on HP-UX >> cannot address more than 1 GB of memory. In fact (I think due to >> the overhead of memory management done by python) a python >> application cannot use much more than 500 MB of "real" data. For >> this reason > > I don't thik this is likely. Don't know about HP-UX but on some > platforms, FreeBSD for example, there is a soft memory-cap for > applications. By default, a single application on FreeBSD cannot > use more than 512MB of memory, period. The limit can be modified by > root (probably involves rebooting). As I stated I wrote a simple C-program before. The c-program was able to allocate a bit more than 900MB in 32 bit mode. My python script allocates a bunch of strings each of 1024 characters and writes it in a cStringIO. And it fails after writing 512K of strings. Don't know how python restricts the heap size - but I'm fairly sure its not a restriction of the OS. But thats not the point, I don't want to talk about one or two megabytes - I'm just missing some GB of heap;) Mathias -- http://mail.python.org/mailman/listinfo/python-list
Re: 64 bit Python
Mike C. Fletcher wrote: > Mathias Waack wrote: > ... > >>My python script allocates a bunch of strings each of 1024 >>characters and writes it in a cStringIO. And it fails after writing >>512K of strings. Don't know how python restricts the heap size - >>but I'm fairly sure its not a restriction of the OS. >> >> > Does cStringIO require contiguous blocks of memory? I'm wondering > if maybe there's something going on where you just can't allocate > more than > a GB of contiguous space? That was the problem. I've used cStringIO to reduce the overhead by the python memory management. But now I know - that was a bad idea in this case. > What happens if you just store the > strings in > a list (or use StringIO instead of cStringIO)? It consumed 10 GB really fast, another GB really slow, some hundred MB much slower and then it died;) Mathias -- http://mail.python.org/mailman/listinfo/python-list
Re: StringIO objects sharing a buffer
Thomas Lotze wrote: > Is there a way for multiple StringIO objects to share a buffer of > data, or do I have to give up on subclassing StringIO for this > purpose? (An alternative would be a tokenizer class that has a > StringIO instead of being one and do the file pointer housekeeping > in there.) I'm not sure if I understand the problem right, but there is a rule of thumb: Prefer delegation to inheritance. If you can use delegation, avoid inheritance. Esp. on python, where (from the point of view of usage) there is no distinction between delegation or inheritance. Mathias -- http://mail.python.org/mailman/listinfo/python-list
Re: Multiple initialization methods?
alex wrote: > it is possible to define multiple initialization methods so that > the method is used that fits? > > I am thinking of something like this: > > def __init__(self, par1, par2): > self.init(par1, par2); > > def __init__(self, par1): > self.init(par1, None) > > def init(self, par1, par2): > ... > ... > > So if the call is with one parameter only the second class is > executed (calling the 'init' method with the second parameter set > to 'None' or whatever. But this example does not work. > > How to get it work? You have to do the method dispatching by yourself. For variable length parameter list you could choose this solution: def __init__(self, *args): if len(args) == 1: self.__setup1(*args) elif len(args) == 2: self.__setup2(*args) ... def __setup1(self, arg1): print "setup1" def __setup2(self, arg1, arg2): print "setup2" Or for different parameter lists which may have the same length my suggestion would be: def __init__(self, **kw): self.__dict__.update(kw) Mathias PS: anyone working on a patch for multimethod dispatching for python? -- http://mail.python.org/mailman/listinfo/python-list
Re: How to wrap a class's methods?
Grant Edwards wrote:
> I want to subclass an IMAP connection so that most of the
> methods raise an exception if the returned status isn't 'OK'.
> This works, but there's got to be a way to do it that doesn't
> involve so much duplication:
>
> class MyImap4_ssl(imaplib.IMAP4_SSL):
>
> def login(*args):
> s,r = imaplib.IMAP4_SSL.login(*args)
> if s!='OK':
> raise NotOK((s,r))
> return r
>
> def list(*args):
> s,r = imaplib.IMAP4_SSL.list(*args)
> if s!='OK':
> raise NotOK((s,r))
> return r
>
> def search(*args):
> s,r = imaplib.IMAP4_SSL.search(*args)
> if s!='OK':
> raise NotOK((s,r))
> return r
>
> [and so on for another dozen methods]
How about using a delegator:
class Wrapper:
funcs = ("login", "list", "search")
def __init__(self, classobj):
self.__wrapped = classobj()
def __getattr__(self, attr):
if attr in Wrapper.funcs:
def f(*args):
f1 = getattr(self.__wrapped, attr)
s,r = f1(args)
if s != 'OK': raise NotOk((s,r))
return r
return f
# raise some exception here
imap = Wrapper(imaplib.IMAP4_SSL)
If you wrap all methods you can ignore the if-test. Instead of the
class object you can pass instances to the wrapper if you need
special arguments for initialization.
I don't like subclassing;)
Mathias
PS: note that we're wrapping the instance's methods, not the class's
methods!
--
http://mail.python.org/mailman/listinfo/python-list
Re: Start new process by function ?
George Sakkis wrote: > Is it possible to start a new process by specifying a function call > (in similar function to thread targets) instead of having to write > the function in a separate script and call it through os.system or > os.spawn* ? That is, something like > > def foo(): pass > os.spawn(foo) Just starting is easy: def spawn(f): if os.fork() == 0: f() sys.exit(0) But I'd expect you would like to get some return values from f... Mathias -- http://mail.python.org/mailman/listinfo/python-list
Re: What Next After Python Basics
On Sunday, July 13, 2014 10:16:47 PM UTC+2, Orochi wrote: > Hi, > I am beginner in Python > I have Completed Basic Python Course from Codecademy.com . > Now that I have a hands on the basics what is the next thing I should do. > I mean should I learn more or start a small Project(Any Ideas are Welcomed) > or any Other suggestions. > Where to Start? > > Thank You, > Orochi > I started up going through a few problems on projecteuler.net, and then casually escalated to look through codegolf.stackexchange.com for fun code to study, and even some problem to mimic. So, there's *an* idea. (mind you, might not be the best of ideas). If you're up for it, there's even rosettacode.org, with crazy code which I found fun to study. But mostly, just code. Anything. For my studies, I get assignments to go through large bodies of text and sort them for some criteria, and while I'm given tools to do it, I try to make my own tools to get the job done. That is to say, if there's something in your day that you use in some way, and it's fairly simple, or its essential functionality can be copied, then that's something to copy. I apologize for the raving suggestions. Mathias H. M. -- https://mail.python.org/mailman/listinfo/python-list
Re: How to test if object is an integer?
On Sat, Oct 15, 2011 at 1:44 AM, MrPink wrote: > > Is there a function in Python that can be used to test if the value in > a string is an integer? I had to make one up for myself and it looks > like this: > > def isInt(s): > try: > i = int(s) > return True > except ValueError: > return False According to [1], there're more Exceptions to test for: try: int(s) return True except (TypeError, ValueError, OverflowError): # int conversion failed return False [1] http://jaynes.colorado.edu/PythonIdioms.html, idiom "Catch errors rather than avoiding them to avoid cluttering your code with special cases" -Mathias -- http://mail.python.org/mailman/listinfo/python-list
protect logger from shutdown syslogd
Hi
I have a script which calls getLogger to set up a logging service.
The standard logger description used both the consoleHandler and the
syslogHandler.
It seems that if syslogd is shut down, getLogger fails, and the script
won't be able to log at all.
is there a way to make it immune against this and automatically continue
the consoleHandler. Sure, we could amend the .ini file
to remove the syslogHandler from the logger definition when this is
known, but on a cron-initiated script this is not feasible.
below is the logger definition:
> [loggers]
> keys=root,mylogger
>
> [handlers]
> keys=consoleHandler,syslogHandler,rootHandler
>
> [formatters]
> keys=simpleFormatter,syslogFormatter,rootFormatter
>
> [logger_root]
> level=NOTSET
> handlers=rootHandler
>
> [logger_mylogger]
> level=INFO
> handlers=consoleHandler,syslogHandler
> #handlers=syslogHandler
> #handlers=consoleHandler
> qualname=mylogger
> propagate=0
>
> [handler_rootHandler]
> class=StreamHandler
> level=DEBUG
> formatter=rootFormatter
> args=(sys.stdout,)
>
> [handler_consoleHandler]
> class=StreamHandler
> level=DEBUG
> formatter=simpleFormatter
> args=(sys.stdout,)
>
> [handler_syslogHandler]
> class=handlers.SysLogHandler
> formatter=syslogFormatter
> level=INFO
> #args=(('localhost', handlers.SYSLOG_UDP_PORT),
> handlers.SysLogHandler.LOG_USER)
> args=('/dev/log', handlers.SysLogHandler.LOG_USER)
thanks
--
http://mail.python.org/mailman/listinfo/python-list
encoding for colon-separated hex?
is there a built-in encoding (for encode/decode methods) that as colon-separated hex (01:02:03:04...)? 'hex' seems to encode as '01020304' and while one can postprocess that to insert the colons, if a single operation exists, I'd rather use that. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to get database metadata information (i.e. existing tables and columns in tables)
Chris Brat wrote:
> Is it possible to retrieve details about the database, specifically a
> list of the tables in the database; and then to retrieve the columns
> and their types for the tables?
Yes.
> Is this dependant on the database?
Yes.
Real databases usually store meta-data (list of tables, list of columns,
list of indexes aso...) in system tables which can be queried in the same
manner as common tables or view. Just read your database handbook...
HTH
Mathias
--
http://mail.python.org/mailman/listinfo/python-list
Re: 10GB XML Blows out Memory, Suggestions?
[EMAIL PROTECTED] wrote: > I wrote a program that takes an XML file into memory using Minidom. I > found out that the XML document is 10gb. > > I clearly need SAX or something else? More memory;) Maybe you should have a look at pulldom, a combination of sax and dom: it reads your document in a sax-like manner and expands only selected sub-trees. > Any suggestions on what that something else is? Is it hard to convert > the code from DOM to SAX? Assuming a good design of course not. Esp. if you only need some selected parts of the document SAX should be your choice. Mathias -- http://mail.python.org/mailman/listinfo/python-list
Illegal instruction or undefined symbol from import
Hi,
I've embedded python into a legacy application. It works - most of the time.
In some special situations the app crashes executing the "import random".
There are two different situations:
1. the sources compiled with gcc 4.1.2 crash with illegal instruction error:
(running my application)
Python 2.3.5 (#1, Jun 9 2006, 11:49:02)
[GCC 4.1.2 20060604 (prerelease) (Debian 4.1.1-2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
dlopen("/usr/lib/python2.3/lib-dynload/readline.so", 2);
import readline # dynamically loaded
from /usr/lib/python2.3/lib-dynload/readline.so
>>> import random
import random # from /usr/lib/python2.3/random.py
# can't create /usr/lib/python2.3/random.pyc
dlopen("/usr/lib/python2.3/lib-dynload/math.so", 2);
import math # dynamically loaded from /usr/lib/python2.3/lib-dynload/math.so
Illegal instruction
Running python itself works.
2. the sources compiled with 4.0.3 give me an undefined symbol error:
>>> import random
# /usr/lib/python2.3/random.pyc matches /usr/lib/python2.3/random.py
import random # precompiled from /usr/lib/python2.3/random.pyc
dlopen("/usr/lib/python2.3/lib-dynload/math.so", 2);
Traceback (most recent call last):
File "", line 1, in ?
File "/usr/lib/python2.3/random.py", line 42, in ?
from math import log as _log, exp as _exp, pi as _pi, e as _e
ImportError: /usr/lib/python2.3/lib-dynload/math.so: undefined symbol:
PyFPE_jbuf
>>>
It gives the same traceback in both python itself and the embedded version.
My main problem is the first error. I've found some older postings
describing this behaviour and pointing at a compiler error
(http://mail.python.org/pipermail/python-dev/2003-May/035386.html). But I'm
not able to verify this error with gcc 4.1.2.
Google finds some postings describing the same error - but it looks like
nobody ever got an answer:( Would be nice to have more success...
Regards
Mathias
--
http://mail.python.org/mailman/listinfo/python-list
Re: Illegal instruction or undefined symbol from import
Continuing my monologe;)
Mathias Waack wrote:
> I've embedded python into a legacy application. It works - most of the
> time. In some special situations the app crashes executing the "import
> random". There are two different situations:
>
> 1. the sources compiled with gcc 4.1.2 crash with illegal instruction
> error:
>
> (running my application)
>
> Python 2.3.5 (#1, Jun 9 2006, 11:49:02)
> [GCC 4.1.2 20060604 (prerelease) (Debian 4.1.1-2)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> dlopen("/usr/lib/python2.3/lib-dynload/readline.so", 2);
> import readline # dynamically loaded
> from /usr/lib/python2.3/lib-dynload/readline.so
>>>> import random
> import random # from /usr/lib/python2.3/random.py
> # can't create /usr/lib/python2.3/random.pyc
> dlopen("/usr/lib/python2.3/lib-dynload/math.so", 2);
> import math # dynamically loaded from
> /usr/lib/python2.3/lib-dynload/math.so Illegal instruction
>
> Running python itself works.
The error occurred in calls to math.log(). I've patched mathmodule.c and
call log1p(x-1) instead of log. It works as a first workaround.
BTW, does anybody know why the c-lib offers both log and log1p?
Mathias
--
http://mail.python.org/mailman/listinfo/python-list
Re: Illegal instruction or undefined symbol from import
Richard Brodie wrote: > "Mathias Waack" <[EMAIL PROTECTED]> wrote in message > news:[EMAIL PROTECTED] > >> BTW, does anybody know why the c-lib offers both log and log1p? > > So you can get a sensible answer computing log(1 + 10 ^ -30). Ok, that make sense to me. > There's > a lot of somewhat obscure mathematical stuff that got into the standard > C lib. How often do you need Bessel functions? Maybe each day. What is a Bessel function?;) Mathias -- http://mail.python.org/mailman/listinfo/python-list
a few extensions for the itertools
I wrote a few functions which IMHO are missing in python(s itertools). You can download them here: http://sourceforge.net/project/showfiles.php?group_id=165721&package_id=212104 A short description to all the functions: icmp(iterable1, iterable2) -> integer Return negative if iterable1 < iterable2, zero if iterable1 == iterable1, positive if iterable1 > iterable1. isum(iterable, start=0) -> value Returns the sum of the elements of a iterable plus the value of parameter 'start'. When the iterable is empty, returns start. iproduct(iterable, start=0) -> value Returns the product of the elements of a iterable times the value of parameter 'start'. When the iterable is empty, returns start. forall(predicate, iterable, default=True) -> bool Returns True, when for all elements x in iterable predicate(x) is True. When the iterable is empty, returns default. forany(predicate, iterable, default=False) -> bool Returns True, when for any element x in iterable predicate(x) is True. When the iterable is empty, returns default. take(n,iterable) -> iterator returns a iterator over the first n elements of the iterator drop(n,iterable) -> iterable drops the first n elemetns of iterable and return a iterator over the rest heads(iterable) -> iterator over all heads example: for head in heads([1,2,3,4,5,6,7,8,9]): print head output: [] [1] [1, 2] [1, 2, 3] [1, 2, 3, 4] [1, 2, 3, 4, 5] [1, 2, 3, 4, 5, 6] [1, 2, 3, 4, 5, 6, 7] [1, 2, 3, 4, 5, 6, 7, 8] [1, 2, 3, 4, 5, 6, 7, 8, 9] tails(iterable) -> iterator over all tails example: for tail in tails([1,2,3,4,5,6,7,8,9]): print tail output: [1, 2, 3, 4, 5, 6, 7, 8, 9] [2, 3, 4, 5, 6, 7, 8, 9] [3, 4, 5, 6, 7, 8, 9] [4, 5, 6, 7, 8, 9] [5, 6, 7, 8, 9] [6, 7, 8, 9] [7, 8, 9] [8, 9] [9] [] fcain(funct,*functs) -> function(...,***) fcain(f1,f2,...,fn)(*args,*kwargs) equals f1(f2(...fn(*args,*kwargs))) -- http://mail.python.org/mailman/listinfo/python-list
my small hashlib - using pythons hash-functions
Hi.
I wrote a small hashlib for C. Because I'm new to hashes I looked at pythons
implementation and
reused *some* of the code... or more the mathematical "hash-function", not
really the code.
In particular I looked at pythons hash and lookup functions, so I came up with
this (see the code
underneath).
So, can this code be considered as derived and do I have to put my code under
the GPL? I'd like to
publish it under something less restrictive, like a BSD style license. But if
GPL is the only way,
then GPL it is. :)
-panzi
#define PERTURB_SHIFT 5
static ENTRY_T * lookup(CONTAINER_T * table, hash_const_str_t key,
int (*loop_cond)(ENTRY_T * ptr, hash_const_str_t key)) {
register ENTRY_T * entries = table->entries;
register hash_size_t size= table->size;
register hash_size_t pos = hash(key) % size;
register hash_size_t perturb = pos;
register ENTRY_T * ptr = entries + pos;
while(loop_cond(ptr,key)) {
pos = ((pos << 2) + pos + perturb + 1) % size;
perturb >>= PERTURB_SHIFT;
ptr = entries + pos;
#ifdef HASH_COUNT_COLLISIONS
++ table->collisions;
#endif
}
return ptr;
}
hash_t hash(register hash_const_str_t str) {
/* python 2.5's hash function: */
register hash_size_t len;
register hash_t h;
assert(str);
len = strlen(str);
h = *str << 7;
while (*str)
h = (103 * h) ^ *str ++;
h ^= len;
return h;
}
--
http://mail.python.org/mailman/listinfo/python-list
Re: my small hashlib - using pythons hash-functions
Paul Rubin wrote: > Mathias Panzenboeck <[EMAIL PROTECTED]> writes: >> So, can this code be considered as derived and do I have to put my >> code under the GPL? I'd like to publish it under something less >> restrictive, like a BSD style license. But if GPL is the only way, >> then GPL it is. :) > > Python is not GPL'd but has its own license. You can make the derived > work GPL if you want, but it is not required. You'll have to check > its terms to see whether you can specifically use the BSD license. > Simplest might be to just stay with the Python license. Yes, right. It's PSF. But the question is: *IS* this derived work? I mean, it's not copied code. It's the same hashing-logic, which I learned by watching pythons code. -panzi -- http://mail.python.org/mailman/listinfo/python-list
Re: my small hashlib - using pythons hash-functions
Fredrik Lundh wrote:
> Mathias Panzenboeck wrote:
>
>> But the question is: *IS* this derived work? I mean, it's not copied
>> code.
>> It's the same hashing-logic, which I learned by watching pythons code.
>
> given that it's only a few lines of code, and there's hardly any other
> way to write those lines if you want to implement the same algorithm,
> I'd say it falls under "fair use". crediting the Python developers in
> the source code should be good enough.
>
>
>
ic. I'll write a mail to the python developers then.
The code in python looks like this (see code underneath), just if someone
what's to know.
I only used the logic of the parts I understand. ;)
Objects/stringobject.c:
static long
string_hash(PyStringObject *a)
{
register Py_ssize_t len;
register unsigned char *p;
register long x;
if (a->ob_shash != -1)
return a->ob_shash;
len = a->ob_size;
p = (unsigned char *) a->ob_sval;
x = *p << 7;
while (--len >= 0)
x = (103*x) ^ *p++;
x ^= a->ob_size;
if (x == -1)
x = -2;
a->ob_shash = x;
return x;
}
Objects/dictobject.c:
static dictentry *
lookdict_string(dictobject *mp, PyObject *key, register long hash)
{
register size_t i;
register size_t perturb;
register dictentry *freeslot;
register size_t mask = (size_t)mp->ma_mask;
dictentry *ep0 = mp->ma_table;
register dictentry *ep;
/* Make sure this function doesn't have to handle non-string keys,
including subclasses of str; e.g., one reason to subclass
strings is to override __eq__, and for speed we don't cater to
that here. */
if (!PyString_CheckExact(key)) {
#ifdef SHOW_CONVERSION_COUNTS
++converted;
#endif
mp->ma_lookup = lookdict;
return lookdict(mp, key, hash);
}
i = hash & mask;
ep = &ep0[i];
if (ep->me_key == NULL || ep->me_key == key)
return ep;
if (ep->me_key == dummy)
freeslot = ep;
else {
if (ep->me_hash == hash && _PyString_Eq(ep->me_key, key))
return ep;
freeslot = NULL;
}
/* In the loop, me_key == dummy is by far (factor of 100s) the
least likely outcome, so test for that last. */
for (perturb = hash; ; perturb >>= PERTURB_SHIFT) {
i = (i << 2) + i + perturb + 1;
ep = &ep0[i & mask];
if (ep->me_key == NULL)
return freeslot == NULL ? ep : freeslot;
if (ep->me_key == key
|| (ep->me_hash == hash
&& ep->me_key != dummy
&& _PyString_Eq(ep->me_key, key)))
return ep;
if (ep->me_key == dummy && freeslot == NULL)
freeslot = ep;
}
}
--
http://mail.python.org/mailman/listinfo/python-list
Re: Generator question
Robert Kern wrote: > Timothy Wu wrote: >> Hi, >> >> Using generator recursively is not doing what I expect: >> >> def test_gen(x): >> yield x >> x = x - 1 >> if x != 0: >> test_gen(x) > > The only thing that the last line does is *create* a new generator object. You > need to actually iterate over it and yield its values. E.g. > > > In [2]: def test_gen(x): >...: yield x >...: x -= 1 >...: if x != 0: >...: for y in test_gen(x): >...: yield y >...: >...: > > In [3]: list(test_gen(3)) > Out[3]: [3, 2, 1] > > why not this? def test_gen(x): for y in xrange(x,0,-1): yield y or even better/shorter: test_gen = lambda x: xrange(x,0,-1) -- http://mail.python.org/mailman/listinfo/python-list
Re: type(foo) == function ?
Chris Mellon wrote: > On 11/29/06, Tom Plunket <[EMAIL PROTECTED]> wrote: >> I'd like to figure out if a given parameter is a function or not. >> >> E.g. >> >> >>> type(1) >> >> >>> type(1) == int >> True >> >> implies: >> >> >>> def foo(): >> ... pass >> ... >> >>> type(foo) >> >> >>> type(foo) == function >> Traceback (most recent call last): >> File "", line 1, in ? >> NameError: name 'function' is not defined >> >> Is there a way I can know if 'foo' is a function? >> > def foo(): > ... pass > ... from inspect import isfunction isfunction(foo) > True > > > But you probably want the callable() builtin instead. > This builtin will be removed in python 3.0! >> thanks, >> -tom! >> -- >> http://mail.python.org/mailman/listinfo/python-list >> -- http://mail.python.org/mailman/listinfo/python-list
Re: merits of Lisp vs Python
Mark Tarver wrote: > How do you compare Python to Lisp? What specific advantages do you > think that one has over the other? > > Note I'm not a Python person and I have no axes to grind here. This is > just a question for my general education. > > Mark > I do not know much about Lisp. What I know is: Python is a imperative, object oriented dynamic language with duck typing, List is a declarative, functional dynamic language -> those two languages have different scopes. For more Information: http://en.wikipedia.org/wiki/Functional_programming http://en.wikipedia.org/wiki/Object-oriented_programming http://en.wikipedia.org/wiki/Dynamic_programming_language http://en.wikipedia.org/wiki/Lisp_programming_language http://en.wikipedia.org/wiki/Python_%28programming_language%29 -- http://mail.python.org/mailman/listinfo/python-list
Re: merits of Lisp vs Python
Rob Thorpe wrote: > Mathias Panzenboeck wrote: >> Mark Tarver wrote: >>> How do you compare Python to Lisp? What specific advantages do you >>> think that one has over the other? >>> >>> Note I'm not a Python person and I have no axes to grind here. This is >>> just a question for my general education. >>> >>> Mark >>> >> I do not know much about Lisp. What I know is: >> Python is a imperative, object oriented dynamic language with duck typing, > > Yes, but Python also supports the functional style to some extent. > I currently visit a course about functional programming at the university of technology vienna: python implements only a small subset of things needed to be called a functional language (list comprehension). but yes, for a imperativ oop language python is very close to functional. >> List is a declarative, >> functional dynamic language > > Lisp is only a functional language in that it support both functional > and imperative programming styles. Duck typing is almost identical to > latent typing in Lisp. > And, Common Lisp at least is object orientated. > >> -> those two languages have different scopes. > > Their scope is actually very similar. Learn about lisp and you will > soon discover their similarity. > ic -- http://mail.python.org/mailman/listinfo/python-list
Re: call of __del__ non-deterministic in python 2.4 (cpython)?
Anthony Baxter wrote:
> On 12/13/06, Holger Joukl <[EMAIL PROTECTED]> wrote:
>> I did read this but didn't think it applied to my situation. I'm quite
>> sure that the refcount of the local variable is 1 before the local scope
>> is left.
>> So let me rephrase the question: Even if I can make sure that non of the
>> problematic situtions apply, might it _still_ happen that __del__ gets
>> called
>> after some other code has already been "entered"?
>
> You shouldn't rely on __del__ being called exactly when you expect it,
> particularly in a threaded application. Make explicit cleanup calls,
> instead.
a nice way to do such things in python 2.5 is using the with statement. this
ensures to call a
__exit__ (cleanup) function after a __enter__ function was called, no matter if
there is a exception
or something other in between.
with open("foo.txt","w") as fp:
fp.write("foo")
translates to:
mgr = open("foo.txt","w")
# well, in the case of files, __enter__ just returns self
fp = mgr.__enter__()
try:
fp.write("foo")
finally:
mgr.__exit__() # this calls fp.close()
this of course is a simplified translation of the with statement.
see more details here: http://www.python.org/dev/peps/pep-0343/
with is also cool for using it in combination with mutexes and similar stuff.
it would also be possible to write a transaction manager which calls commit()
or abort()
automatically, so you can just write:
with new_transaction():
# if do_stuff() returns, commit() will be called
# if it raises an exception abort() will be called
do_stuff()
But if you can't use python 2.5, there is no other way then:
try:
do_stuff()
finally:
cleanup()
--
http://mail.python.org/mailman/listinfo/python-list
Re: merits of Lisp vs Python
Bruno Desthuilliers wrote: > Mathias Panzenboeck a écrit : >> Rob Thorpe wrote: >> >>> Mathias Panzenboeck wrote: >>> >>>> Mark Tarver wrote: >>>> >>>>> How do you compare Python to Lisp? What specific advantages do you >>>>> think that one has over the other? >>>>> >>>>> Note I'm not a Python person and I have no axes to grind here. >>>>> This is >>>>> just a question for my general education. >>>>> >>>>> Mark >>>>> >>>> >>>> I do not know much about Lisp. What I know is: >>>> Python is a imperative, object oriented dynamic language with duck >>>> typing, >>> >>> Yes, but Python also supports the functional style to some extent. >>> >> >> >> I currently visit a course about functional programming at the >> university of technology vienna: >> python implements only a small subset of things needed to be called a >> functional language (list >> comprehension). > > Python has functions as first-class objects (you can pass functions as > arguments to functions, return functions from functions, and bind > functions to identifiers), and that's the only thing you need to use a > functional approach. You mean like function pointers in C and C++? I think this should be possible in assembler, too. I thought functional languages have to be declarative? The boost C++ library has even lambdas! -- http://mail.python.org/mailman/listinfo/python-list
Re: Quake 3 and the Python interpreter
Neil Toronto wrote: > > 2) Is there any way to restrict Python modules from accessing files > outside of a sandbox? > As far as I heard, there is no really working way. But maybe that has changed in python 2.5? Or it will in 3.0? I dunno. -- http://mail.python.org/mailman/listinfo/python-list
Re: Wow, Python much faster than MatLab
A other great thing: With rpy you have R bindings for python. So you have the power of R and the easy syntax and big standard lib of python! :) -- http://mail.python.org/mailman/listinfo/python-list
Re: Yield
Fredrik Lundh schrieb: > Danny Colligan wrote: > >> Carsten mentioned that generators are more memory-efficient to use when >> dealing with large numbers of objects. Is this the main advantage of >> using generators? Also, in what other novel ways are generators used >> that are clearly superior to alternatives? > > the main advantage is that it lets you decouple the generation of data > from the use of data; instead of inlining calculations, or using a pre- > defined operation on their result (e.g printing them or adding them to a > container), you can simply yield them, and let the user use whatever > mechanism she wants to process them. i.e. instead of > > for item in range: > calculate result > print result > > you'll write > > def generate(): > for item in range: > yield result > > and can then use > > for item in generate(): > print item > > or > >list(process(s) for s in generate()) > > or > >sys.stdout.writelines(generate()) > > or > >sum(generate()) > > etc, without having to change the generator. > > you can also do various tricks with "endless" generators, such as the > following pi digit generator, based on an algorithm by Python's grand- > father Lambert Meertens: > > def pi(): > k, a, b, a1, b1 = 2, 4, 1, 12, 4 > while 1: > # Next approximation > p, q, k = k*k, 2*k+1, k+1 > a, b, a1, b1 = a1, b1, p*a+q*a1, p*b+q*b1 > # Yield common digits > d, d1 = a/b, a1/b1 > while d == d1: > yield str(d) > a, a1 = 10*(a%b), 10*(a1%b1) > d, d1 = a/b, a1/b1 > > import sys > sys.stdout.writelines(pi()) > > > or fibonacci: def fib(): a, b = 0, 1 while True: yield a a, b = b, a+b now get the first 10 ones: >>> from itertools import * >>> list(islice(fib(),10)) [0, 1, 1, 2, 3, 5, 8, 13, 21, 34] or primes: def primes(): yield 1 yield 2 p = 3 psofar = [] smallerPrimes = lambda: takewhile(lambda x: x+x <= p,psofar) while True: if all(p % n != 0 for n in smallerPrimes()): psofar.append(p) yield p p += 2 etc. -- http://mail.python.org/mailman/listinfo/python-list
Re: Bragging about Python
Steve Howell schrieb: > --- "[EMAIL PROTECTED]" > <[EMAIL PROTECTED]> wrote: > >> Is there a resource somewhere on the net that can be >> used to quickly >> and effectively show Python's strengths to >> non-Python programmers? >> Small examples that will make them go "Wow, that >> _is_ neat"? >> > > 15 small programs here: > > http://wiki.python.org/moin/SimplePrograms > IMHO a few python goodies are missing there. e.g. generator functions: # generic fibonacci without upper bound: def fib(): parent_rabbits, baby_rabbits = 1, 1 while True: yield baby_rabbits parent_rabbits, baby_rabbits = baby_rabbits, parent_rabbits + baby_rabbits # only calculate and print the first 100 fibonacci numbers: from itertools import islice for baby_rabbits in islice(fib(),100): print 'This generation has %d rabbits' % baby_rabbits -- http://mail.python.org/mailman/listinfo/python-list
Running Python with XAMPP
Hello everyone!
I just installed Python 2.5 and i want to use Python to build websites.
I could load mod_python successfully with Apache but i fail to let
the .py-files to be executed!
In /htdocs/python i got my test file:
[code=python.py]
from mod_python import apache
def index(req):
req.content_type = "text/html"
req.write("")
req.write("Python is running with mod_python...")
return apache.OK
[/code]
And then i got the following code in my httpd.conf:
[code]
AddHandler mod_python .py
pythonHandler index
pythonDebug On
[/code]
But when i type "http://localhost/python/python.py"; it won't execute!
My browser just shows me the source code. :-(
Please, can anyone tell me what i have to do?
Thanks in advance!
~ Mathias
--
http://mail.python.org/mailman/listinfo/python-list
Compiling python extension on amd64 for 32 bit
After switching my development environment to 64 bit I've got a problem with
a python extension for a 32 bit application. Compiling the app under Linux
results in the following error:
g++ -m32 -Wall -g -O2 -I. -Idb -DPYTHON=25 -o mappy.o -c mappy.cpp
In file included from /usr/include/python2.5/Python.h:57,
from mappy.cpp:29:
/usr/include/python2.5/pyport.h:732:2: error: #error "LONG_BIT definition
appears wrong for platform (bad gcc/glibc config?)."
My gcc is:
Using built-in specs.
Target: x86_64-suse-linux
Configured with: ../configure --enable-threads=posix --prefix=/usr
--with-local-prefix=/usr/local --infodir=/usr/share/info
--mandir=/usr/share/man
--libdir=/usr/lib64 --libexecdir=/usr/lib64
--enable-languages=c,c++,objc,fortran,obj-c++,java,ada
--enable-checking=release
--with-gxx-include-dir=/usr/include/c++/4.1.2
--enable-ssp --disable-libssp
--disable-libgcj --with-slibdir=/lib64
--with-system-zlib --enable-shared --enable-__cxa_atexit
--enable-libstdcxx-allocator=new --program-suffix=-4.1
--enable-version-specific-runtime-libs --without-system-libunwind
--with-cpu=generic
--host=x86_64-suse-linux
Thread model: posix
gcc version 4.1.2 20061115 (prerelease) (SUSE Linux)
So I've checked this magic LONG_BIT define:
#:/tmp cat t.c
#include
#include
int main()
{
printf("%d\n",sizeof(long));
printf("%d\n",LONG_BIT);
return 0;
}
#:/tmp gcc t.c
#:/tmp ./a.out
8
64
#:/tmp gcc -m32 t.c
#:/tmp ./a.out
4
32
Ok, thats fine. So why is python complaining? Or even more interesting, what
do I have to do to compile the code?
Mathias
--
http://mail.python.org/mailman/listinfo/python-list
Re: Compiling python extension on amd64 for 32 bit
Andrew MacIntyre wrote:
> Mathias Waack wrote:
>> After switching my development environment to 64 bit I've got a
>> problem with a python extension for a 32 bit application.
>
> {...}
>
>> Ok, thats fine. So why is python complaining? Or even more
>> interesting, what do I have to do to compile the code?
>
> Is the Python your toolchain is referencing 32 or 64 bit? Based on
> what I can see in pyport.h, I'd guess that you're finding a 64 bit
> Python (ie SIZEOF_LONG == 8).
There is no such thing as "the Python". A biarch environment offers
both the 32 bit and the 64 bit versions of each library. And unique
headers, because its assumed that the headers are independent of the
architecture. Because of the -m32 Flag the pyport.h is used within a
32 bit env.
Mathias
--
http://mail.python.org/mailman/listinfo/python-list
optparse option prefix
We've integrated python into a legacy application. Everything works fine (of course because its python;). There's only one small problem: the application reads the commandline and consumes all arguments prefixed with a '-' sign. Thus its not possible to call a python module from the commandline with a parameter list containing options prefixed by '-' or '--' signs. Thats not a major problem, but it prevents us from using th optparse module. Is there a way to change to prefix, so one could use a '+' (for instance) to mark command line options for optparse? Regards Mathias -- http://mail.python.org/mailman/listinfo/python-list
Re: optparse option prefix
Diez B. Roggisch wrote: >> (3) Create a filter module that reads sys.argv, replaces leading "+" >> signs with "-" signs, and then stuffs it back into sys.argv before >> optparse gets to see it. > > That's not even necessary, the optparser will work on a passed argument > list. No need to alter sys.argv. Sounds nice, I'll do so. Thanks! Mathias -- http://mail.python.org/mailman/listinfo/python-list
weakrefs and bound methods
Hi. I have a problem with weak refs and bound methods. The best explanation for the problem is a short bit of code. I have the three classes Wrapper, Foo and Bar: import weakref class Wrapper(object): def __init__(self,x): self.x = weakref.ref(x) def __call__(self,*args,**kwargs): x = self.x() if x is None: print "lost reference" else: return x(*args,**kwargs) class Foo(object): def __init__(self): self._methods = set() self._methods.add(Wrapper(self._foo)) def _foo(self): print "_foo" def callMethods(self): for method in self._methods: method() def __del__(self): print "del Foo" class Bar(object): def __init__(self): self._methods = set() self._methods.add(self._foo) def _foo(self): print "_foo" def callMethods(self): for method in self._methods: method() def __del__(self): print "del Bar" Now look what happens when I do this: >>> f=Foo() >>> f.callMethods() lost reference >>> del f del Foo >>> b=Bar() >>> b.callMethods() _foo >>> del b >>> Foo looses the reference to its method but Bar on the other hand has a refloop and never gets deleted. Does anyone know what happens here? How can I write such classes without refloop and without lost reference? I'm using: Python 2.5.1 (r251:54863, May 2 2007, 16:56:35) [GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2 -panzi -- http://mail.python.org/mailman/listinfo/python-list
Re: weakrefs and bound methods
When I change the class Wrapper to following, the class Foo works: class Wrapper(object): def __init__(self,x): self.func_name = x.func_name self.x = weakref.ref(x.im_self) def __call__(self,*args,**kwargs): x = self.x() if x is None: print "lost reference" else: return getattr(x,self.func_name)(*args,**kwargs) But that's ugly. Any better idea? -- http://mail.python.org/mailman/listinfo/python-list
Re: weakrefs and bound methods
Marc 'BlackJack' Rintsch wrote: > On Sun, 07 Oct 2007 16:51:33 +0200, Mathias Panzenboeck wrote: > >> import weakref >> >> class Wrapper(object): >> def __init__(self,x): >> self.x = weakref.ref(x) >> >> def __call__(self,*args,**kwargs): >> x = self.x() >> if x is None: >> print "lost reference" >> else: >> return x(*args,**kwargs) >> >> class Foo(object): >> def __init__(self): >> self._methods = set() >> self._methods.add(Wrapper(self._foo)) >> >> def _foo(self): >> print "_foo" >> >> def callMethods(self): >> for method in self._methods: >> method() >> >> def __del__(self): >> print "del Foo" >> >> class Bar(object): >> def __init__(self): >> self._methods = set() >> self._methods.add(self._foo) >> >> def _foo(self): >> print "_foo" >> >> def callMethods(self): >> for method in self._methods: >> method() >> >> def __del__(self): >> print "del Bar" >> >> >> >> Now look what happens when I do this: >> >>>>> f=Foo() >>>>> f.callMethods() >> lost reference >>>>> del f >> del Foo >>>>> b=Bar() >>>>> b.callMethods() >> _foo >>>>> del b >>>>> >> Foo looses the reference to its method but Bar on the other hand has a >> refloop and >> never gets deleted. > > ``del b`` just deletes the name `b`. It does not delete the object. > There's still the name `_` bound to it in the interactive interpreter. > `_` stays bound to the last non-`None` result in the interpreter. > Why is it then that f (instance of Foo) really gets deleted? (__del__ is called) > Drop all those `__del__()` methods as they prevent the garbage collector > from collecting "cycles". > > Ciao, > Marc 'BlackJack' Rintsch I only inserted them so I can see if the objects are really freed. How can I see that without a __del__ method? Thanks so far, panzi -- http://mail.python.org/mailman/listinfo/python-list
Re: weakrefs and bound methods
Marc 'BlackJack' Rintsch wrote: > ``del b`` just deletes the name `b`. It does not delete the object. > There's still the name `_` bound to it in the interactive interpreter. > `_` stays bound to the last non-`None` result in the interpreter. > Actually I have the opposite problem. The reference (to the bound method) gets lost but it shouldn't! -panzi -- http://mail.python.org/mailman/listinfo/python-list
Re: weakrefs and bound methods
Alex Martelli wrote: > Mathias Panzenboeck <[EMAIL PROTECTED]> wrote: > >> Marc 'BlackJack' Rintsch wrote: >>> ``del b`` just deletes the name `b`. It does not delete the object. >>> There's still the name `_` bound to it in the interactive interpreter. >>> `_` stays bound to the last non-`None` result in the interpreter. >> Actually I have the opposite problem. The reference (to the bound method) >> gets lost but it shouldn't! > > weakrefs to bound methods require some subtlety, see > <http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/81253> (or what > I believe is the better treatment of this recipe in the printed edition > of the Python Cookbook -- of course, being the latter's editor, I'm > biased;-). > > > Alex Thank you, that really helped. :) -panzi -- http://mail.python.org/mailman/listinfo/python-list
Re: weakrefs and bound methods
Bruno Desthuilliers wrote:
> Mathias Panzenboeck a écrit :
>
> About the lost weakref problem: in Python, methods are just tiny
> wrappers around the object, class and function created at lookup time
> (yes, on *each* lookup) (and WWAI, they are by the function object
> itself, which implements the descriptor protocol).
>
>> When I change the class Wrapper to following, the class Foo works:
>>
>> class Wrapper(object):
>> def __init__(self,x):
>> self.func_name = x.func_name
>> self.x = weakref.ref(x.im_self)
>
>> def __call__(self,*args,**kwargs):
>> x = self.x()
>> if x is None:
>> print "lost reference"
>> else:
>> return getattr(x,self.func_name)(*args,**kwargs)
>
> Anyway, the whole thing looks quite baroque to me - and I'm known for my
> immoderate appetite for brain-melting features
>
> If I understand correctly, you're doing all this to avoid circular
> references while keeping track of a list of methods to call ?
>
> If yes, then you're thru much pain for nothing. Mostly because there are
> other much more simpler solutions. The first one is obviously to use
> names and getattr. In it's most naïve implementation, this would look like:
>
> class Foo(object):
> def __init__(self):
> self._methods = set()
> self._methods.add('_foo')
>
> def _foo(self):
> print "_foo"
>
> def callMethods(self):
> for name in self._methods:
> meth = getattr(self, name, None)
> if callable(meth):
> meth()
>
>
>
>
> Now if _methods is supposed to have the same content class-wise (which
> is the case in your exemple), no need to have it as an instance method:
>
> class Foo(object):
> _methods = set('_foo')
>
> def _foo(self):
> print "_foo"
>
> def callMethods(self):
> for name in self._methods:
> meth = getattr(self, name, None)
> if callable(meth):
> meth()
>
> We could go further, like using a decorator to mark methods to add to
> _methods, and a metaclass to set the whole thing up, but this would
> probably be overkill. Unless there are other requirements, I'd
> personnaly stop here.
>
> Oh, and yes, while we're at it : Python usually knows how to deal with
> circular references
>
Well yes and no. It does with the enabled gc but I like refcounting much more
because
of the predictable behaviour. And when there are no refloops, refcounting makes
no
problems. However, Alex Martelli already showed a great way to do what I need,
see:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/81253
I need it for a little UI lib. Well lib is too much. Just a small wrapper around
blenders unusable API for building GUIs for your scripts. It's ugly as hell.
I made very few classes to wrap up that crap. Every input widget in my UI lib
has an
action. You can register action listeners like this:
def actionListener(value):
print "new value:",value
widget.addActionListener(actionListener)
When an widget itself want's to listen for it's action and does this in
__init__:
self.addActionListener(sefl.__actionListener)
there would be a ref loop. Therefore I do this now:
self.addActionListener(WeakListener(sefl.__actionListener))
And WeakListener I implemented as follows:
class WeakListener(object):
__slots__ = ('__weakref__', '_func','_obj')
def __init__(self,listener):
self._func = listener.im_func
self._obj = weakref.ref(listener.im_self)
def __call__(self,*args,**kwargs):
obj = self._obj()
if obj is None:
print "*** lost reference to %s's self object ***" % \
self._func.func_name
else:
self._func(obj,*args,**kwargs)
Could be better/more generic, but that's all I need.
To implement the action listener by overloading a action method is IMHO a bad
idea.
Because that would kill the listener of the base class. Explicitly calling the
base
classes action method is ugly and error prone. And when I have to register
action
listeners from "outside" the widget, too.
But thanks for your reply. :)
-panzi
--
http://mail.python.org/mailman/listinfo/python-list
Re: wxPython and Tkinter
Hello, take a look here [1]. I think its a got place to start. br Mathias [1] http://sebsauvage.net/python/gui/#import 2007/11/29, whatazor <[EMAIL PROTECTED]>: > > Hi all, > I migrate some code from tkinter to wxpython. I need the equivalent > Tkinter method Tkinter.Tk.after > in wxPython, but I'm not able to find it. It exist or there are other > trick to emulate it? > > thank you > w > -- > http://mail.python.org/mailman/listinfo/python-list > -- http://mail.python.org/mailman/listinfo/python-list
Re: Process Id
Which OS Windows or Linux? br Mathias 2007/12/3, Navid Parvini <[EMAIL PROTECTED]>: > > Dear All, > > Would you please help me to find a way to get the list of all processes id > on the machine along with their parent processes. > > Thank you in advance. > > > -- > Never miss a thing. Make Yahoo your > homepage.<http://us.rd.yahoo.com/evt=51438/*http://www.yahoo.com/r/hs> > > > -- > http://mail.python.org/mailman/listinfo/python-list > -- http://mail.python.org/mailman/listinfo/python-list
SIGILL importing random
Hi all, I've embedded python into an older application and created some extensions to interact with the old data structures. Everythings works like a charm - beside a simple "import random". During import the whole process crashes with a SIGILL. I've found some older mails describing just the same problem: http://mail.python.org/pipermail/python-dev/2003-May/035386.html Unfortunately I'm not able to read x86 assembler code:( I'm using gcc version 4.0.3 (Debian 4.0.3-1) and python: Python 2.3.5 (#2, Mar 6 2006, 10:12:24) [GCC 4.0.3 20060304 (prerelease) (Debian 4.0.2-10)] on linux2 Everything out of the box from Debian unstable. Anybody with any idea about this? Mathias -- http://mail.python.org/mailman/listinfo/python-list
Re: Append a new value to dict
Pat wrote:
I know it's not "fair" to compare language features, but it seems to me
(a Python newbie) that appending a new key/value to a dict in Python is
awfully cumbersome.
In Python, this is the best code I could come up with for adding a new
key, value to a dict
mytable.setdefault( k, [] ).append( v )
In Perl, the code looks like this:
$h{ $key } = $value ;
There's a huge difference here:
In your Python example you're using a list. In the Perl example you're
using a scalar value.
Is there a better/easier way to code this in Python than the
obtuse/arcane setdefault code?
When just assigning a new key-value-pair there's no problem in Python.
(Just refer to the answers before.) When I switched from Perl to Python
however I commonly ran into this problem:
>>> counter = {}
>>> counter['A'] = 1
>>> counter['A'] += 1
>>> counter['A']
2
Ok - assigning a key-value-pair works fine. Incrementing works as well.
>>> counter['B'] += 1
Traceback (most recent call last):
File "", line 1, in
KeyError: 'B'
However incrementing a non-existing key throws an exception. So you
either have to use a workaround:
>>> try:
... counter['B'] += 1
... except KeyError:
... counter['B'] = 1
Since this looks ugly somebody invented the setdefault method:
>>> counter['B'] = counter.setdefault('B',0) + 1
And this works with lists/arrays as well. When there's no list yet
setdefault will create an empty list and append the first value.
Otherwise it will just append.
Greetings from Vienna,
mathias
--
http://mail.python.org/mailman/listinfo/python-list
Getting stdout using ctypes.
Hello all. I have a simple application (C++) that relies on shared libraries. It works fine in console mode. Lot of job is done into the shared library, so there is some calls to 'std::cout' to inform the user in it. Now, I would like to wrap everything into a GUI, remove the application and call directly everything from Python using ctypes. (I still need the console application to launch it manually if needed). I've made a simple library to test ctypes and everything works fine except that I don't know how to get stout in order to redirect it somewhere (dialog box or so). I've looked for some help into the mailing list history and found nothing useful (until now). Do someone has any suggestion? Mathias -- http://mail.python.org/mailman/listinfo/python-list
Re: AJAX Widget Framework
Laszlo Nagy wrote: > I'm looking for an open source, AJAX based widget/windowing framework. > Here is what I need: > > - end user opens up a browser, points it to a URL, logs in > - on the server site, sits my application, creating a new session for > each user that is logged in > - on the server site, I create windows(frames), put widgets on them, > write event handlers etc. Just like with wx or pygtk. > - However, windows are created in the user's browser, and events are > triggered by Javascript, and sent back to server through AJAX. > - the server side would be - of course - written in Python. > > I was looking these projects: > > http://www.uize.com/ > http://pyjs.org/ > > There are many frameworks listed here which I did not check: > http://internetmindmap.com/javascript_frameworks. I have no idea which > has Python support, and probably there are only a few that worth looking > at. I wonder if you could tell me which are the pros and contras for > these frameworks. If there is a comparison available on the NET, > pointing me to the right URL would help a lot. > > The bottom line... > > My main goal is to develop enterprise class OLTP database applications. > I'll need grid widget to display data in tabular format, and I'll use > events heavily for data manipulation, live search, tooltips etc. I'm > familiar with wxWidgets, pygtk and other toolkits, but not with AJAX. I > have never used a system like that. Of course I don't know which is the right way for you. I'm very happy with pyjamas. It allows you the create client applications in the same manner as standalone toolkits like wxWidgets. The message passing between client and server is simple done by exchanging json strings (its ajax based of course, but this stuff works silently in the background). On the server side there are many python toolkits, I prefer cherrypy, others are django and web.py. Hope this helps you. Mathias -- http://mail.python.org/mailman/listinfo/python-list
Extension type inherit from Python int
Hello, I am trying to write an extension type that inherits from Pythons int. Based on the documentation, <https://docs.python.org/3/extending/newtypes_tutorial.html#subclassing-other-types> this should be possible, but I checked Pythons implementation and the struct defining a PyLongObject uses a flexible array to store the limbs to support arbitrary large numbers. So having a PyLongObject as first member of my own types struct doesn't work because parts of the ob_digit array would be overwritten by members defined in the subclass of the new type that inherits from PyLongObject. Has anyone already done something like that? Is there some trick to work around that issue, or is it simply not possible to inherit from PyLongObject? Thank you, Mathias -- https://mail.python.org/mailman/listinfo/python-list
