Re: do something in time interval

2008-10-06 Thread Mathieu Prevot
2008/10/6 Petr Jakeš <[EMAIL PROTECTED]>:
>> I am not an expert, but why not to use time.sleep(5)?
>> If you are using wxPython, you may also try wx.Timer, in which you could
>> set its interval.
>>
>
> Thanks for your reply.
> During the 5s period my script has to do some stuff instead of sleeping.
> Thats why it runs in the loop and once in 5s period it has to trigger some
> other stuff(function, method, action) to do.
> Petr Jakes

You might want to use threads so you have a very simple code like this:

data data0

thread1:
while True:
  time.sleep(5)
  doSomething(data0)

thread2:
while True:
  time.sleep(0.01)
 doAnotherthing(data0)

thread1 and thread2 can run in parallel; if thread2 take more time,
thread1 won't be impaired.

Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


Re: Can this be done in py?

2008-11-07 Thread Mathieu Prevot
2008/11/7 Robert Singer <[EMAIL PROTECTED]>:
> Now, don't get me wrong if this is a trivial question, or even an
> apsurd one. I'm new to python, so my mileage may vary.
>
> I have several exe files, console applications that pretty much run on
> the principle:
>  first.exe
> Enter file name: start.dat
>  
>
>  second.exe
> Enter file name: filename.dat
>  
>
> ... you get the point.
>
> These are old exe programs, not mine (I can't recompile them or modify
> them in any way), but which work fine and I still regularly use them.
> However, I would like to automate this process (since they are not
> just two, and since this process of entering always the same data is
> tiresome.
>
> Is there a way this could be done with a python script ?
> A script which starts the .exe file, enters some predefined data,
> waits for an exe to finish, then again, starts the second .exe file,
> ... ?

Of course. See the subprocess module for this.

Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


Popen: NameError: name 'PIPE' is not defined

2008-05-24 Thread Mathieu Prevot
Hi

I import subprocess and use Popen, but PIPE is not defined. I used
2.5.1, 2.5.2, Python 2.6a3+ (trunk:63576, May 24 2008, 12:13:40), it's
always the same. What am I missing ?

Thanks
Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


Re: Popen: NameError: name 'PIPE' is not defined

2008-05-24 Thread Mathieu Prevot
2008/5/24 Diez B. Roggisch <[EMAIL PROTECTED]>:
> Mathieu Prevot schrieb:
>>
>> Hi
>>
>> I import subprocess and use Popen, but PIPE is not defined. I used
>> 2.5.1, 2.5.2, Python 2.6a3+ (trunk:63576, May 24 2008, 12:13:40), it's
>> always the same. What am I missing ?
>
> Without showing code, it's hard to know. A guess is: if you use
>
> import subprocess
>
>
> then use
>
>
> subprocess.PIPE
>
> Or if using
>
> from subprocess import Popen
>
> make sure to do
>
> from subprocess import Popen, PIPE
>
> Diez

Indeed... thank you !

Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


PIPE stderr

2008-05-24 Thread Mathieu Prevot
Hi again,

I don't have the same ouptput between 1) and 2) ... what's happening ?

1)
wget http://www.youtube.com/v/A8bwZf3vXjg -O /dev/null 2> file

2)
k = Popen (["wget", "http://www.youtube.com/v/A8bwZf3vXjg";, "-O", "/dev/null"],
stderr=PIPE)
print k.stderr

In the case 2) I got:

open file '', mode 'rb' at 0x5a608>

and I want what I get in file from 1) ... how to do this ?

Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


Re: Flash Decoder

2008-05-28 Thread Mathieu Prevot
2008/5/28 Diez B. Roggisch <[EMAIL PROTECTED]>:
> Ankit wrote:
>
>> Hi everyone,i wanted to build a flash decoder using python can
>> somebody tell me which library to use and what steps should i follow
>> to make a flash(video) decoder?By a decoder i mean that i need to
>> display all the pixel values of each frame.Waiting for your replies.
>
> Check out libffmpeg. It should be accessible using python by various means,
> including gstreamer or ctypes.
>
> This is however a rather advanced topic.
>
> Diez

I think you might want make an ffmpeg/libffmpeg python wrap. It could
be really useful, and faster than a pure python decoder. Even for fun.

Cheers
Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


Re: Flash Decoder

2008-05-28 Thread Mathieu Prevot
2008/5/28 Ankit <[EMAIL PROTECTED]>:
> Thanks for replying guys but could you be a little more specific like
> in terms of steps i shd follow to make the decoder and also how is
> ffmpeg/libffmpeg going to help..

You can start by getting familiar with ffmpeg [1] by playing with it
and glance at its libraries (I call them libffmpeg, but they have
their specific name) allow you to do. Keep in mind that libraries
contain frequently used functions and are written so they can be
reused, shared by other/news softwares eg. the one you want. Start by
opening a flash video eg one .flv [2] file from youtube with a first
python script that use the right component(s) of libffmpeg. Python
allows you to use C libraries.

Mathieu

[1] http://ffmpeg.mplayerhq.hu/documentation.html
[2] http://en.wikipedia.org/wiki/Flv
--
http://mail.python.org/mailman/listinfo/python-list


Re: Flash Decoder

2008-05-28 Thread Mathieu Prevot
2008/5/28 ankit anand <[EMAIL PROTECTED]>:
> hmm i am more interested in .swf format and more specifically i would like
> to have all the pixel values of all the frames can i do that using this
> library?

Not with ffmpeg. You can check out the code from Gnash [1] or Swfdec
[2] or start you swf decoder from scratch using Adobe specification
[3] on the swf format.

Happy coding !
Mathieu

[1] http://en.wikipedia.org/wiki/Gnash
[2] http://en.wikipedia.org/wiki/Swfdec
[3] http://www.adobe.com/devnet/swf/
--
http://mail.python.org/mailman/listinfo/python-list


Re: gcc error in Mac OS X

2008-06-05 Thread Mathieu Prevot
2008/6/5 Zhaojie Boulder <[EMAIL PROTECTED]>:
> Hello,
> I am new to Mac and used python in linux before. What I am trying to do is
> to install "Ipython" and "PyCogent" in Mac OS X.
> For PyCogent, after entering the package path, I typed "python setup.py
> install". The results are as follows:
> Didn't find Pyrex - will compile from .c files
> running install
> running build
> running build_py
> running build_ext
> building 'cogent.align._compare' extension
> gcc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd
> -fno-common -dynamic -DNDEBUG -g -Os -Wall -Wstrict-prototypes -DMACOSX
> -I/usr/include/ffi -DENABLE_DTRACE -arch i386 -arch ppc -pipe
> -I/Users/zhaojie/Downloads/PyCogent-1.0.1/include
> -I/System/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5
> -c cogent/align/_compare.c -o
> build/temp.macosx-10.5-i386-2.5/cogent/align/_compare.o -w
> unable to execute gcc: No such file or directory
> error: command 'gcc' failed with exit status 1
> After google, I installed Xcode,but it did not help. Also, the Xcode folder
> is not within "applications" folder, but a separate one parallel with
> "applications". Dragging Xcode folder into the applications folder did not
> make a difference, either.
> Hope someone familiar with Mac can help me out.

Normally gcc is installed in /usr/bin. If you type "which gcc", it
should return "/usr/bin/gcc". Xcode is an IDE that use gcc or another
compiler eg. icc the Intel compiler, hence if you move Xcode in the
application folfer it won't change anything. You install Xcode by
double clicking on a .pkg file from your Leopard DVD.

You also can download gcc from the site and bootstrap/compile it.

Cheers
Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


Re: gcc error in Mac OS X

2008-06-05 Thread Mathieu Prevot
2008/6/6 Mathieu Prevot <[EMAIL PROTECTED]>:
> 2008/6/5 Zhaojie Boulder <[EMAIL PROTECTED]>:
>> Hello,
>> I am new to Mac and used python in linux before. What I am trying to do is
>> to install "Ipython" and "PyCogent" in Mac OS X.
>> For PyCogent, after entering the package path, I typed "python setup.py
>> install". The results are as follows:
>> Didn't find Pyrex - will compile from .c files
>> running install
>> running build
>> running build_py
>> running build_ext
>> building 'cogent.align._compare' extension
>> gcc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd
>> -fno-common -dynamic -DNDEBUG -g -Os -Wall -Wstrict-prototypes -DMACOSX
>> -I/usr/include/ffi -DENABLE_DTRACE -arch i386 -arch ppc -pipe
>> -I/Users/zhaojie/Downloads/PyCogent-1.0.1/include
>> -I/System/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5
>> -c cogent/align/_compare.c -o
>> build/temp.macosx-10.5-i386-2.5/cogent/align/_compare.o -w
>> unable to execute gcc: No such file or directory
>> error: command 'gcc' failed with exit status 1
>> After google, I installed Xcode,but it did not help. Also, the Xcode folder
>> is not within "applications" folder, but a separate one parallel with
>> "applications". Dragging Xcode folder into the applications folder did not
>> make a difference, either.
>> Hope someone familiar with Mac can help me out.
>
> Normally gcc is installed in /usr/bin. If you type "which gcc", it
> should return "/usr/bin/gcc". Xcode is an IDE that use gcc or another
> compiler eg. icc the Intel compiler, hence if you move Xcode in the
> application folfer it won't change anything. You install Xcode by
> double clicking on a .pkg file from your Leopard DVD.
>
> You also can download gcc from the site and bootstrap/compile it.

Note that you don't need to install Xcode in the "right place", it is
normally installed in /Developer, and installs gcc and other files in
/usr/bin /usr/share etc.

In general, try "locate prog" to try to find where is prog, and check
if the binary prog's path is in your PATH. eg. you can compile and
install a gnu tool by ./configure --prefix=$HOME && make && make
install, and the binaries will be installed in ~/bin/. If you don't
have ~/bin/ in your PATH, it won't execute. Adding 'export
PATH="$PATH:$HOME/bin:"' in your .bashrc will solve the problem (or
'setenv PATH "$PATH:$HOME/bin:' in your .tcshrc)

Using --prefix=$HOME allow you to run and install your binaries and
libraries. You can do this for a group of users without root access,
so you can share, or preserve your OS bin/lib/include/share easily, or
allow alternative use of several versions of libraries/binaries etc.

Other possibilities are the chroot and the sophisticated FreeBSD
jails. Google for them to know more.

Hope that helps,
Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


configure fails

2008-06-05 Thread Mathieu Prevot
Hi,

I have the following error on a OSX.5 OS with CC=icc and using the
python-svn files:

checking size of wchar_t... configure: error: cannot compute sizeof (wchar_t)

I would like to help so we can compile python with icc/OSX.

Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


python compilation on macosx with icc

2008-06-13 Thread Mathieu Prevot
Hi,

when I run configure, it fails at:

checking for wchar.h... yes
checking for wchar_t... yes
checking size of wchar_t... configure: error: cannot compute sizeof (wchar_t)

what can I do ?

Thanks,
Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


Problem with subprocess.Popen wget within a thread

2008-07-06 Thread Mathieu Prevot
Hi

it seems the script (A) finishes before the downloading ends, and the
(B) version doesn't (wanted behavior) ... this is unexpected. What
happens ?


(A) 
class vid(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
def download(self):
self.cmd = 'wget
ftp://ftp.freebsd.org/pub/FreeBSD/releases/i386/ISO-IMAGES/7.0/7.0-RELEASE-i386-bootonly.iso'
self.child = subprocess.Popen(self.cmd.split())
def run(self):
self.download()

def main():
w = vid()
w.start()
w.join()

(B) 
class vid(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
def download(self):
self.cmd = 'wget
ftp://ftp.freebsd.org/pub/FreeBSD/releases/i386/ISO-IMAGES/7.0/7.0-RELEASE-i386-bootonly.iso'
self.child = subprocess.Popen(self.cmd.split(), 
stderr=subprocess.PIPE)
def run(self):
self.download()
self.child.stderr.readlines()

def main():
w = vid()
w.start()
w.join()

--
http://mail.python.org/mailman/listinfo/python-list


Re: Problem with subprocess.Popen wget within a thread

2008-07-07 Thread Mathieu Prevot
2008/7/6 Sebastian lunar Wiesner <[EMAIL PROTECTED]>:
> Mathieu Prevot <[EMAIL PROTECTED]>:
>
>> it seems the script (A) finishes before the downloading ends, and the
>> (B) version doesn't (wanted behavior) ... this is unexpected. What
>> happens ?
>
> "readlines" blocks, until the pipe is closed, which usually happens, if the
> process dies.
>
> On the other hand, spawned processes are usually asynchronous, you have to
> explicitly _wait_ for them.  And you're not waiting for it in example A.
>
> Anyway, the _proper_ way to wait for a child process is ... guess what ...
> the "wait" method of the Popen object ;)

Thanks :)
Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


do a sed / awk filter with python tools (at least as fast)

2008-07-07 Thread Mathieu Prevot
Hi,

I use in a bourne shell script the following filter:

sed '/watch?v=/! d;s/.*v=//;s/\(.\{11\}\).*/\1/' \
| sort | uniq | awk 'ORS=" "{print $1}'

that give me all sets of 11 characters that follows the "watch?v="
motif. I would like to do it in python on stdout from a
subprocess.Popen instance, using python tools rather than sed awk etc.
How can I do this ? Can I expect something as fast ?

Thanks,
Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


Re: do a sed / awk filter with python tools (at least as fast)

2008-07-07 Thread Mathieu Prevot
2008/7/7 Peter Otten <[EMAIL PROTECTED]>:
> Mathieu Prevot wrote:
>
>> I use in a bourne shell script the following filter:
>>
>> sed '/watch?v=/! d;s/.*v=//;s/\(.\{11\}\).*/\1/' \
>> | sort | uniq | awk 'ORS=" "{print $1}'
>>
>> that give me all sets of 11 characters that follows the "watch?v="
>> motif. I would like to do it in python on stdout from a
>> subprocess.Popen instance, using python tools rather than sed awk etc.
>> How can I do this ? Can I expect something as fast ?
>
> You should either do it in Python , e. g.:
>
> def process(lines):
>candidates = (line.rstrip().partition("/watch?v=") for line in lines)
>matches = (c[:11] for a, b, c in candidates if len(c) >= 11)
>print " ".join(sorted(set(matches)))
>
> if __name__ == "__main__":
>import sys
>process(sys.stdin)
>
> or invoke your shell script via subprocess.Popen(). Invoking a python script
> via subprocess doesn't make sense IMHO.

:) Thanks.
Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


Tricky problem: loss of terminal control after python script running (depending on internal parameters)

2008-07-08 Thread Mathieu Prevot
Hi there,

I have a python script that runs subprocesses (task =  ffmpeg for
convertion of videos to images) from threads [1]. I want to run at
most 4 (default) tasks at the same time, and I usually ask the python
script to process 10 files for testing. I'll run it on thousands of
files for primetime.
It seems that when I ask to process < 4 files everything is ok, but if
I ask for > 4 files to process, I loose terminal features (no more
newlines, etc).
What's happening ? What is wrong in my script ?

Thanks,
Mathieu

[1]:
,--  thread -- subprocess
script --|-- thread -- subprocess
|--  thread -- subprocess
'--  thread -- subprocess


youff.py
Description: Binary data
--
http://mail.python.org/mailman/listinfo/python-list

error with configure (svn 64857)

2008-07-10 Thread Mathieu Prevot
Hi,

I have the following error when I run configure:

checking size of wchar_t... configure: error: cannot compute sizeof (wchar_t)

what can I do ?

Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


Re: error with configure (svn 64857)

2008-07-10 Thread Mathieu Prevot
2008/7/10 "Martin v. Löwis" <[EMAIL PROTECTED]>:
>> I have the following error when I run configure:
>>
>> checking size of wchar_t... configure: error: cannot compute sizeof (wchar_t)
>>
>> what can I do ?
>
> Study config.log for the source of the problem.

Thank you Martin. How can I remove -lgcc_s and use the Intel equivalent ?

configure:21939: checking for wchar_t
configure:21970: icc -c -g -O2  conftest.c >&5
conftest.c(123): warning #279: controlling expression is constant
  if ((ac__type_new_ *) 0)
  ^

configure:21976: $? = 0
configure:21991: result: yes
configure:21998: checking size of wchar_t
configure:22306: icc -o conftest -g -O2   conftest.c  >&5
ld: library not found for -lgcc_s
configure:22309: $? = 1
configure: program exited with status 1

Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


Re: error with configure (svn 64857)

2008-07-11 Thread Mathieu Prevot
2008/7/11 "Martin v. Löwis" <[EMAIL PROTECTED]>:
>> Thank you Martin. How can I remove -lgcc_s and use the Intel equivalent ?
>
> I'm not so sure that there is anything wrong in configure. configure
> doesn't pass -lgcc_s to icc; instead, icc is making this up on its
> own. So I would guess you need to get libgcc_s onto you system in a
> way that the linker finds it. Else you need to read the icc
> documentation (but I'm fairly sure that icc is *required* to link
> with libgcc_s, for interoperability with gcc-compiled binaries).
>
> I'm somewhat puzzled that the wchar_t test is the one where it
> crashes; this test comes fairly late, and configure has run multiple
> compiler invocations before that.
>
> Can you build any binaries at all with your icc installation?

Yes of course, I successfuly built and installed nmap, wget, so I
thought there was something in the python configure process.

If didn't investigate everything but I solved the problem by adding
the "-static-libgcc" option:

CFLAGS="-w -static-intel -static-libgcc"

I think one should commit changes so configure can manage this. I can
I want to help for further diagnosis or improvement. Also it seems
-Wall will be deprecated, and usually we use -w:

   -w  Control diagnostics, where  is one of the following:

 0 -- Display errors (same as -w)

 1 -- Display warnings and errors (default)

 2 -- Display remarks, warnings, and errors


Cheers,
Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


Re: error with configure (svn 64857)

2008-07-11 Thread Mathieu Prevot
2008/7/11 WDC <[EMAIL PROTECTED]>:
> On Jul 10, 6:57 pm, "Mathieu Prevot" <[EMAIL PROTECTED]> wrote:
>> 2008/7/10 "Martin v. Löwis" <[EMAIL PROTECTED]>:
>>
>> >> I have the following error when I run configure:
>>
>> >> checking size of wchar_t... configure: error: cannot compute sizeof 
>> >> (wchar_t)
>>
>> >> what can I do ?
>>
>> > Study config.log for the source of the problem.
>>
>> Thank you Martin. How can I remove -lgcc_s and use the Intel equivalent ?
>>
>> configure:21939: checking for wchar_t
>> configure:21970: icc -c -g -O2  conftest.c >&5
>> conftest.c(123): warning #279: controlling expression is constant
>>   if ((ac__type_new_ *) 0)
>>   ^
>>
>> configure:21976: $? = 0
>> configure:21991: result: yes
>> configure:21998: checking size of wchar_t
>> configure:22306: icc -o conftest -g -O2   conftest.c  >&5
>> ld: library not found for -lgcc_s
>> configure:22309: $? = 1
>> configure: program exited with status 1
>>
>> Mathieu
>
> I think he wanted YOU to find the problem. The best way to learn is to
> figure it out yourself! Good luck.

Yes WDC I know, but for some problems 1) we can gain much efficiency
with getting help from guys that are familiar with the environment and
 2) we also allow the committers to know a problem and improve the
sources. That's the way an open source group works IMHO :)

Now that the problem is solved for me, it is for all MacOSX+Intel icc guys.

Cheers,
Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


kill thread

2008-08-07 Thread Mathieu Prevot
Hi,

I have a threading.Thread class with a "for i in range(1,50)" loop
within. When it runs and I do ^C, I have the error [1] as many as
loops. I would like to catch this exception (and if possible do some
cleanup like in C pthreads) so the program finishes cleanly. Where and
how can I do this ? in __run__ ? __init__ ? a try/except stuff ?

Thanks,
Mathieu

[1]:
^CException in thread Thread-1:
Traceback (most recent call last):
  File 
"/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/threading.py",
line 486, in __bootstrap_inner
self.run()
  File "./youfetch.py", line 148, in run
self.getids()
  File "./youfetch.py", line 145, in getids
self.ids.append(self.getidsatpage(i))
  File "./youfetch.py", line 138, in getidsatpage
self.child = subprocess.Popen(cmd.split(),stdout=subprocess.PIPE)
  File 
"/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/subprocess.py",
line 594, in __init__
errread, errwrite)
  File 
"/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/subprocess.py",
line 1011, in _execute_child
self.pid = os.fork()
KeyboardInterrupt
--
http://mail.python.org/mailman/listinfo/python-list


Re: kill thread

2008-08-08 Thread Mathieu Prevot
2008/8/8 Miki <[EMAIL PROTECTED]>:
> Hello,
>
>> I have a threading.Thread class with a "for i in range(1,50)" loop
>> within. When it runs and I do ^C, I have the error [1] as many as
>> loops. I would like to catch this exception (and if possible do some
>> cleanup like in C pthreads) so the program finishes cleanly. Where and
>> how can I do this ? in __run__ ? __init__ ? a try/except stuff ?
> You can have a try/except KeyboardException around the thread code.
>
> HTH,
> --
> Miki

Of course, but I don't know where. I placed this inside loop, within
called functions from the loop, I still have the problem.

Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


Re: kill thread

2008-08-09 Thread Mathieu Prevot
2008/8/8  <[EMAIL PROTECTED]>:
> On 8 Ago, 10:03, "Mathieu Prevot" <[EMAIL PROTECTED]> wrote:
>> 2008/8/8 Miki <[EMAIL PROTECTED]>:
>>
>> > Hello,
>>
>> >> I have a threading.Thread class with a "for i in range(1,50)" loop
>> >> within. When it runs and I do ^C, I have the error [1] as many as
>> >> loops. I would like to catch this exception (and if possible do some
>> >> cleanup like in C pthreads) so the program finishes cleanly. Where and
>> >> how can I do this ? in __run__ ? __init__ ? a try/except stuff ?
>> > You can have a try/except KeyboardException around the thread code.
>>
>> > HTH,
>> > --
>> > Miki
>>
>> Of course, but I don't know where. I placed this inside loop, within
>> called functions from the loop, I still have the problem.
>>
>> Mathieu
>
> Try this:
>
>  loop_completed = True
>  for i in range(1,50):
>  try:
> # your code here
>  except KeyboardException:
> loop_completed = False
> break # this breaks the loop
>  # end loop
>  if loop_completed:
> # code to be executed in case of normal completion
>  else:
> # code to be executed in case of interruption
>  # code to be executed in both cases

Thanks for answers. My code sheme was the following: main() starts 2
thread trees (threads of threads of ...) and some of these have "for"
loops. These loops needed to be as you recommended:

for ... :
  try:
# instructions
  except KeyboardInterrupt:
# cleaning instructions
break

The problem with atexit.register is that is doesn't work in case of
system signals catches (http://tinyurl.com/6kdaba)

Thanks,
Mathieu

_
def main():
query1 = Thread1
batch1 = Thread2
while True:
try:
#some code for updating / synchronize / etc threads
except KeyboardInterrupt:
try:
query1.terminate()
batch1.terminate()
except:
pass
finally:
break
_


I used also from http://sebulba.wikispaces.com/recipe+thread2
the following new Thread class:

_
import threading, inspect, ctypes

def _async_raise(tid, exctype):
"""raises the exception, performs cleanup if needed"""
if not inspect.isclass(exctype):
raise TypeError("Only types can be raised (not instances)")
res = ctypes.pythonapi.PyThreadState_SetAsyncExc(tid,
ctypes.py_object(exctype))
if res == 0:
raise ValueError("invalid thread id")
elif res != 1:
# """if it returns a number greater than one, you're in trouble,
# and you should call it again with exc=NULL to revert the effect"""
ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, 0)
raise SystemError("PyThreadState_SetAsyncExc failed")

class Thread(threading.Thread):
def _get_my_tid(self):
"""determines this (self's) thread id"""
if not self.isAlive():
raise threading.ThreadError("the thread is not active")

# do we have it cached?
if hasattr(self, "_thread_id"):
return self._thread_id

# no, look for it in the _active dict
for tid, tobj in threading._active.items():
if tobj is self:
self._thread_id = tid
return tid

raise AssertionError("could not determine the thread's id")

def raise_exc(self, exctype):
"""raises the given exception type in the context of this thread"""
_async_raise(self._get_my_tid(), exctype)

def terminate(self):
"""raises SystemExit in the context of the given thread, which should
cause the thread to exit silently (unless caught)"""
self.raise_exc(SystemExit)
_
--
http://mail.python.org/mailman/listinfo/python-list


download limit

2008-08-10 Thread Mathieu Prevot
Hi,

I have a multithreaded script that  mainly creates several wget
processes to download files. I would like to check/see and eventually
limit the bandwidth of the pool of processes. One way to do this is to
change the number of wget instances, but it's a workaround.

What do you recommend to do the following in python:
1) know the bitrate at the script scale
2) control, and limit or not this bitrate

Thanks,
Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


Re: threading

2008-08-14 Thread Mathieu Prevot
2008/8/13 Parimala <[EMAIL PROTECTED]>:
> Hello,
>
>I am using python2.5.1 version to run my test scripts. I want to use
> 'threading' module in my tests. As a startup program, I had run the
> following one.
>
> import threading
> import sys
> import time
>
> def hello():
>   i=0
>   try:
> while i<10:
>   print "hi"
>   time.sleep(1)
>   i+=1
>   except KeyboardInterrupt:
> print 'KeyboardInterrupt'
> raise KeyboardInterrupt
>
> try:
>   thread=threading.Thread(target=hello,args=())
>   thread.start()
> except KeyboardInterrupt:
>   print 'KeyboardInterrupt'
>   raise KeyboardInterrupt
>
> once program starts, problem is..
> I am not able to abort the thread using (CTRL+C) KeyboardInterrupt.  While
> running if I press CTRL+C, it won't generate any exception until the end of
> the execution. Once the execution gets over,  it will give "Exception
> exceptions.KeyboardInterrupt in  'C:\python25\lib\threading.py'> ignored" this message and exits.
>
> I had gone through some documents, it says if a thread is joined with
> .join() method then we can't stop that process until it releases the lock
> what it acquired. But in the above program I didn't use .join() method but
> still I am not able to abort the thread.
>
> Could you please suggest me how can I abort the thread at any point in time
> using CTRL+C.

Hi,

a terminate method is given here:
http://sebulba.wikispaces.com/recipe+thread2

so you can terminate the thread by:

(...)
t.start()
(...)

while True:
  try:
#some code
  except KeyboardInterrupt:
t.terminate()
break

Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


python 64bits on macosx.5 /leopard

2008-08-14 Thread Mathieu Prevot
Hi,

in order to run django on apache, I need mod_python in apache, and it
runs in 64 on leopard, so I need to run python in 64 bits.
I tryed ./configure --enable-framework OPT="-arch x86_64" but make fails:

===
gcc  -u _PyMac_Error Python.framework/Versions/2.6/Python -o python.exe \
Modules/python.o \
 -ldl
ld: warning in Modules/python.o, file is not of required architecture
===

file Python.framework/Versions/2.6/Python gives me:

===
Python.framework/Versions/2.6/Python: Mach-O dynamically linked shared
library i386
===

Which method do you recommend ?

Thanks
Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


subprocess seems to "detach" / ignore wait()

2008-08-20 Thread Mathieu Prevot
Hi there,

it seems that  child.wait() is ignored when

  print "Server running [PID %s]"%(child.pid)
  fpid.write(child.pid)

are between the process creation child = Popen(cmd.split(),
stderr=flog) and child.wait().
It seems to be a bug, doesn't it ?

Mathieu

(I'm running x11vnv with args in the cmd string on FreeBSD 8.0/CURRENT)

flog = open(logfile, 'w')
fpid = open(pidfile, 'w')
try:
  child = Popen(cmd.split(), stderr=flog)
  print "Server running [PID %s]"%(child.pid)
  fpid.write(child.pid)
  child.wait()
except KeyboardInterrupt:
  print "INT sent to vnc server"
finally:
  fpid.close()
  flog.close()
  os.remove(pidfile)
  os.remove(logfile)
  sys.exit(0)
--
http://mail.python.org/mailman/listinfo/python-list


Re: subprocess seems to "detach" / ignore wait()

2008-08-20 Thread Mathieu Prevot
2008/8/20 Gabriel Genellina <[EMAIL PROTECTED]>:
> En Wed, 20 Aug 2008 12:22:16 -0300, Wojtek Walczak <[EMAIL PROTECTED]> 
> escribió:
>
>> On Wed, 20 Aug 2008 15:09:11 +0200, Mathieu Prevot wrote:
>>
>>>   child = Popen(cmd.split(), stderr=flog)
>>>   print "Server running [PID %s]"%(child.pid)
>>>   fpid.write(child.pid)
>>
>> I think that the problem here is that fpid.write() fails silently
>> (probably TypeError), because it takes string as its first argument,
>> not integer.
>
> Exactly, but it doesn't fail "silently" (that would be a bug). The exception 
> is raised, but due to the finally clause ending in sys.exit(0), it has no 
> chance of being handled.
> This is the original code, for reference:
>
> flog = open(logfile, 'w')
> fpid = open(pidfile, 'w')
> try:
>  child = Popen(cmd.split(), stderr=flog)
>  print "Server running [PID %s]"%(child.pid)
>  fpid.write(child.pid)
>  child.wait()
> except KeyboardInterrupt:
>  print "INT sent to vnc server"
> finally:
>  fpid.close()
>  flog.close()
>  os.remove(pidfile)
>  os.remove(logfile)
>  sys.exit(0)
>
> --
> Gabriel Genellina


Indeed, I got TypeError: argument 1 must be string or read-only
character buffer, not int
and Wojtek's code works. So what is the right thing to do so my script
returns 1 or 0 depending on its state and success ?

Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


Re: subprocess seems to "detach" / ignore wait()

2008-08-20 Thread Mathieu Prevot
2008/8/21 Mathieu Prevot <[EMAIL PROTECTED]>:
> 2008/8/20 Gabriel Genellina <[EMAIL PROTECTED]>:
>> En Wed, 20 Aug 2008 12:22:16 -0300, Wojtek Walczak <[EMAIL PROTECTED]> 
>> escribió:
>>
>>> On Wed, 20 Aug 2008 15:09:11 +0200, Mathieu Prevot wrote:
>>>
>>>>   child = Popen(cmd.split(), stderr=flog)
>>>>   print "Server running [PID %s]"%(child.pid)
>>>>   fpid.write(child.pid)
>>>
>>> I think that the problem here is that fpid.write() fails silently
>>> (probably TypeError), because it takes string as its first argument,
>>> not integer.
>>
>> Exactly, but it doesn't fail "silently" (that would be a bug). The exception 
>> is raised, but due to the finally clause ending in sys.exit(0), it has no 
>> chance of being handled.
>> This is the original code, for reference:
>>
>> flog = open(logfile, 'w')
>> fpid = open(pidfile, 'w')
>> try:
>>  child = Popen(cmd.split(), stderr=flog)
>>  print "Server running [PID %s]"%(child.pid)
>>  fpid.write(child.pid)
>>  child.wait()
>> except KeyboardInterrupt:
>>  print "INT sent to vnc server"
>> finally:
>>  fpid.close()
>>  flog.close()
>>  os.remove(pidfile)
>>  os.remove(logfile)
>>  sys.exit(0)
>>
>> --
>> Gabriel Genellina
>
>
> Indeed, I got TypeError: argument 1 must be string or read-only
> character buffer, not int
> and Wojtek's code works. So what is the right thing to do so my script
> returns 1 or 0 depending on its state and success ?

PS: BTW how can I detach my process ie have an equivalent to
`myscript.py&` from the python script ?

Thanks,
Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


Re: subprocess seems to "detach" / ignore wait()

2008-08-21 Thread Mathieu Prevot
2008/8/21 Gabriel Genellina <[EMAIL PROTECTED]>:
> En Thu, 21 Aug 2008 02:46:06 -0300, Mathieu Prevot <[EMAIL PROTECTED]> 
> escribió:
>
>>> So what is the right thing to do so my script
>>> returns 1 or 0 depending on its state and success ?
>
> I use something like this:
>
> def main(argv):
>  try:
>try:
>  do_things()
>  return 0
>finally:
>  do_cleanup()
>  except:
>log_exception()
>return 1
>
> if __name__=='__main__':
>  import sys
>  sys.exit(main(sys.argv))
>
>> PS: BTW how can I detach my process ie have an equivalent to
>> `myscript.py&` from the python script ?
>
> There are a few recipes in the Python CookBook at 
> http://code.activestate.com/recipes/langs/python/

The return from main()... it was my thought too.
Thank you Gabriel :)

Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


use str as variable name

2008-09-04 Thread Mathieu Prevot
Hi,

I have a program that take a word as argument, and I would like to
link this word to a class variable.

eg.
class foo():
  width = 10
  height = 20

a=foo()
arg='height'
a.__argname__= new_value

rather than :

if arg == 'height':
  a.height = new_value
elif arg == 'width';
  a.width = new_value

Can I do this with python ? How ?

Thanks,
Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


Re: use str as variable name

2008-09-04 Thread Mathieu Prevot
2008/9/4 Chris Rebert <[EMAIL PROTECTED]>:
> On Thu, Sep 4, 2008 at 12:25 AM, Mathieu Prevot
> <[EMAIL PROTECTED]> wrote:
>> Hi,
>>
>> I have a program that take a word as argument, and I would like to
>> link this word to a class variable.
>>
>> eg.
>> class foo():
>
> You should subclass 'object', so that should be:
>class Foo(object):
>
>>  width = 10
>>  height = 20
>>
>> a=foo()
>> arg='height'
>> a.__argname__= new_value
>
> You're looking for the setattr() built-in function. In this exact case:
>setattr(a, arg, new_value)
>
> This is probably covered in the Python tutorial, please read it.
>
> Regards,
> Chris

Indeed.

I'll use:
a.__setattr__(height, new_value)

Thanks to all
Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


Re: use str as variable name

2008-09-04 Thread Mathieu Prevot
2008/9/4 Fredrik Lundh <[EMAIL PROTECTED]>:
> Bruno Desthuilliers wrote:
>
>> You wouldn't write something like 2.__add__(3), would you ?
>
> Don't give the "it's only OO if I write obj.method(args)" crowd more bad
> ideas, please ;-)
>
> (...as Bruno implies, setattr(), len() et al can be and should be viewed as
> generic functions.  A specific Python implementation may use custom code to
> implement behaviour for a given object; behaviour that's more efficient than
> a full Python-level method call.  For example, in CPython, len(L) is about
> twice as fast as L.__len__() for built-in sequences.)

Got it. Thanks :)
Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


path slashes cleaning

2008-09-04 Thread Mathieu Prevot
Hi,

for scripts that take arguments, I would like to remove the trailing
slash if it's present.

Is there something else than:

a='/usr/local/lib/'
if a[-1] == '/':
  a = list(a)
  a.pop()
  ''.join(a)

Thanks,
Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


Re: path slashes cleaning

2008-09-04 Thread Mathieu Prevot
2008/9/4 Mathieu Prevot <[EMAIL PROTECTED]>:
> Hi,
>
> for scripts that take arguments, I would like to remove the trailing
> slash if it's present.
>
> Is there something else than:
>
> a='/usr/local/lib/'
> if a[-1] == '/':
>  a = list(a)
>  a.pop()
>  ''.join(a)

A dummy

a.rstrip('/')

Sorry for the noise
Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


Canonical way to build Python 2.6/svn on MacIntel/MacOSX10.5 with icc 32bits 10.1.014

2008-09-06 Thread Mathieu Prevot
Hi,

I would like to build Python (svn) on Macosx 10.5 with icc in
/opt/intel/cc (32 bit). Can you help me to determine the right way to
do this ?

I got a prototype with:

export CC=icc
export CXX=icpc
export CFLAGS="-w"
./configure --with-framework-name=PythonIntel

1) It seems to be unsufficient. Does someone have links/mail
archives/patches/scripts ?

2) Python needs gettext for _locale package, but I fail to compile it with icc.

3) How can I build an autonomous and sufficient distribution (a dmg
file with everything in it) ?

Thanks,
Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


Failure in building ctypes with intel icc

2008-09-06 Thread Mathieu Prevot
Hi,

I got this error in compilation, and I don't know how to solve it:

*** WARNING: renaming "_ctypes" since importing it failed:
dlopen(build/lib.macosx-10.3-i386-2.6/_ctypes.so, 2): Symbol not
found: ___builtin_dwarf_cfa
  Referenced from:
/Users/mathieuprevot/svn/python/build/lib.macosx-10.3-i386-2.6/_ctypes.so
  Expected in: dynamic lookup

Any hints ?

thanks,
Mathieu
--
http://mail.python.org/mailman/listinfo/python-list


Re: how to build a MacOS universal python package including external dependencies

2008-09-24 Thread Mathieu Prevot
2008/9/24 Jaime Huerta Cepas <[EMAIL PROTECTED]>:
> Hi all,
>
> I have developed a set python libraries that provide several scientific
> methods to analyse and visualize certain type of biological data.  This
> libraries are based on many external python modules, such as python-mysql
> python-sip or python-qt4. I use GNU/linux to develop my tools and I found no
> problems into installing all dependencies, however it does not seem to be
> that easy on MacOS. I am sure that all the dependencies (qt4, pyqt4 ,
> mysqldb, scipy, numpy) are cross platform, but when you are trying to
> publish your tool in an academic journal, most referees (many of them using
> MacOS) expect some kind of straightforward installation process for the
> tool.
>
> I wonder if there would be a way in which I could compile all the
> dependencies and libraries in a MacOs system and then building a static
> universal binary that I can distribute. I guess  it should be possible, but
> I am not sure how difficult it might be, and whether all dependencies (qt4
> is huge) can be packaged together.

IMHO this is too complex to commit. Macport is a way to do what you
want, but packages may not be up to date enough. Maybe the easiest and
simplest way for you to do this is to write a script that will
download, compile and install everything.

The script should work like:

sudo all_in_one_script.py

and then wait for jobs to be done. Your script will need to know if a
package was sucessfully installed and then continue or take steps and
say it. For a complex set of dependencies, I recommend you to write
Makefiles.

For instance, in pseudo-code:
if /usr/local/lib/libfoo.dylib doesn't exist
  download foo
  install foo

if python-module foo doesn't exist
  download foo
  python foo/setup.py install

etc

HTH
Mathieu
--
http://mail.python.org/mailman/listinfo/python-list