[Python-Dev] PEP 8: mention bare 'except:'

2006-12-22 Thread A.M. Kuchling
http://www.python.org/sf/411881 is a bug about removing use
of 'except:' in stdlib code.  In many cases the intent is to catch
one particular exception such as ImportError or AttributeError,
but catching all exceptions can disguise other problems.

Should PEP 8 mention this issue?  Here's some proposed text for
discussion:

 - When catching exceptions, mention specific exceptions
   whenever possible instead of using a bare 'except:' clause.

   For example, use::

   try:
   import platform_specific_module
   except ImportError:
   platform_specific_module = None 

   A bare 'except:' clause will catch SystemExit and KeyboardInterrupt
   exceptions, making it harder to interrupt a program with Control-C,
   and can disguise other problems.  If you want to catch all
   exceptions that signal program errors, use 'except StandardError:'.

   A good rule of thumb is that you should only use 'except:' if the
   exception handler will be printing out or logging the traceback; at
   least the user will be aware that an error has occurred.

--amk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 8: mention bare 'except:'

2006-12-22 Thread Barry Warsaw
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Dec 22, 2006, at 8:05 AM, A.M. Kuchling wrote:

>A good rule of thumb is that you should only use 'except:' if the
>exception handler will be printing out or logging the traceback; at
>least the user will be aware that an error has occurred.

Another aspect of bare except (IME) is that they are used when you  
want to do some work, but continue to propagate the exception upwards  
via a bare raise.  E.g. rolling back a transaction, or as you  
mention, logging the error and continuing.

- -Barry

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRYv7znEjvBPtnXfVAQKjYAP/fxLKXLYQvA/CAfYKOVhBkeIUdqJ7qyxH
fv6PXuOM7RwS4FuwtmmbHOw9omyO0Kv+iddx/MD+2hJN9TeeLveGSyr4kEDG5cYD
APhy8KW18bKhtwAiZbxzV4VQ3Q22IQah/nHEmzUfaRYAl8Dms9FjwIPIrlEY2yaS
TY+MNrUum2Y=
=CZuk
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] multiple interpreters and extension modules

2006-12-22 Thread Jeremy Kloth
[[ This may be somewhat c.l.p.-ish but I feel that this crossed into CPython 
development enough to merit posting here ]]

I have received a bug report for 4Suite that involves a PyObject_IsInstance 
check failing for what appears to be the correct class, that is, the class 
names match.  With some investigating, I have found that the true problem is 
with multiple interpreters.  The reason for this is that each sub-interpreter 
has a "new" copy of any pure Python module. The following issues are also 
true for modules that have been reloaded, but I think that is common 
knowledge.  I mention it only for completeness.

Here is where it crosses into CPython development.

Now extension modules do not get reloaded into each sub-interpreter, but a 
copy of the module's dict from when it was first loaded.  This particular 
extension, when initialized, loads the class in question from the pure Python 
module and stores in in a global variable for quick access. I know I can 
change this specific case to reload the class each time and for this exact 
case, it is most likely what I'll do.  However, we have other extensions that 
make use of this same technique and their performance would suffer greatly by 
doing the lookup for each use of the class.

Also performance isn't the only issue here as we also have C types that 
inherit from a Python class and those types will obviously fail an 
isinstance() check in Python code run in different interpreters as the base 
class is "frozen" at the first import.

So I guess the questions are:
1) is subclassing Python classes in C as a static type supported? Even if they 
would be declared on the heap, they would be bound to the first loaded Python 
class.

2) would it be worthwhile to have an extension loading function similar to 
DllMain() for Windows' DLLs which is called multiple times with a flag 
stating the reason for the call? For reference, see:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/dllmain.asp
I envision the reasons as: 'module init', 'module fini', 'interpreter 
init', 'interpreter fini'. I see no current need for special treatment of 
threads as they share the same interpreter space.

As to question 2, I'm in the process of prototyping that approach to see if it 
is feasible in our code and if so, (and people find it useful) I'll attempt 
to write a PEP for that feature.

My current thoughts on that are instead of the current init entry 
point in C modules, a new entry point is defined, and if found, used in its 
place. That way, this proposal would be optional for extension writers and 
existing modules would continue to work without change.

Feedback appreciated.

-- 
Jeremy Kloth
http://4suite.org/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Possible platforms to drop in 2.6

2006-12-22 Thread Brett Cannon
I originally posted this list to python-3000 since I figured we could
be more aggressive with Py3K, but Guido said I should move it over
here and just be aggressive in 2.6.

So, here are the platforms I figured we should drop:

* AtheOS
* BeOS
* FreeBSD 2 (maybe more?)
* IRIX
* NeXT
* OS/2
* UnixWare

I had SunoS 5 but Ronald Oussoren said that is actually Solaris
through version 9, so I took that off.

Several people have questioned AtheOS, but considering the site for
the OS has not been updated since 2002 and it was a niche OS to begin
with I doubt we really need the support.

There is a fork of AtheOS called Syllable (http://www.syllable.org/)
that is still being developed, but someone from that group should
probably step forward and offer to help maintain the code if we are to
keep it.

Same goes with BeOS and Zeta (http://www.zeta-os.com/cms/news.php;
there is a drop-down in the right-hand nav column to change the
language to English for us non-German speakers).

And I listed FreeBSD 2 as a drop since FreeBSD 3 seemed to have been
released in 1999.  But if most users have upgraded by now (release 6
is the most current) then we could consider dropping 3 as well.

-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 8: mention bare 'except:'

2006-12-22 Thread Brett Cannon
On 12/22/06, A.M. Kuchling <[EMAIL PROTECTED]> wrote:
> http://www.python.org/sf/411881 is a bug about removing use
> of 'except:' in stdlib code.  In many cases the intent is to catch
> one particular exception such as ImportError or AttributeError,
> but catching all exceptions can disguise othter problems.
>
> Should PEP 8 mention this issue?

Yes.

-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Possible platforms to drop in 2.6

2006-12-22 Thread skip

Brett> So, here are the platforms I figured we should drop:

...
Brett> * OS/2

I'm pretty sure Andrew MacIntyre is still maintaining the OS/2+EMX port:

http://members.pcug.org.au/~andymac/python.html

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] multiple interpreters and extension modules

2006-12-22 Thread Josiah Carlson

Jeremy Kloth <[EMAIL PROTECTED]> wrote:
> [[ This may be somewhat c.l.p.-ish but I feel that this crossed into CPython 
> development enough to merit posting here ]]
> 
> I have received a bug report for 4Suite that involves a PyObject_IsInstance 
> check failing for what appears to be the correct class, that is, the class 
> names match.  With some investigating, I have found that the true problem is 
> with multiple interpreters.  The reason for this is that each sub-interpreter 
> has a "new" copy of any pure Python module. The following issues are also 
> true for modules that have been reloaded, but I think that is common 
> knowledge.  I mention it only for completeness.

If I remember correctly, Python allows you to use multiple interpreters
in the same process, but it makes no guarantees as to their correctness
when running.

See this post for further discussion on the issue:
http://mail.python.org/pipermail/python-list/2004-January/244343.html

You can also search for 'multiple python interpreters single process' in
google without quotes to hear people lament over the (generally broken)
multiple Python interpreter support.


 - Josiah

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Pydoc Improvements / Rewrite

2006-12-22 Thread Ron Adam


I started to discuss this on the new python-ideas list because I thought it may 
need a bit more completion before bringing it here.  But It was suggested that 
it really does belong here as it is something that would be nice to have in 
python 2.6.  So I'm reposting the introduction here.

The still very rough source files can be downloaded from:

http://ronadam.com/dl/_pydoc.zip

There is still much to do, but I think having some experienced feed back on 
where it should go is important.

Cheers,
Ron Adam

ps.. Please disregard the website for now, it's purpose was to share with 
family 
and friends, although I will probably share the python source to the ledger 
program there and other python toys and experiments (for fun) in the near 
future.



[Introduction From python-ideas list]

Improving pydoc has been suggested before by me and others.  I've been working
on a version that is probably 80% done and would like to get feed back at this
point to determine if I'm approaching this in the best way.

Basically pydoc.py consists of over 2,000 lines of python code and is not well
organized inside which means (my thoughts) it will pretty much stay the way it
is and not be as useful as it could be.

Currently pydoc.py has the following uses.

  * It is imported and used as pythons console help function.
  * It can be used to generate help text files.
  * It can open a tkinter search index and from that launch a web server and
a browser to veiw a html help page.
  * It can be used to generate static html help files.


It looks (to me) like splitting it into two modules would be good.  One module
for just the text help and introspection functions, and the other for the html
server and html output stuff.

[It was suggested on python-ideas that making it a package may be good.]


1. pyhelp.py - Pythons help function from inside the console, and running it
directly would open an interactive text help session.

2. _pydoc.py - Python html help browser.  This starts an html server and opens a
web page with a modules/package index.  The html page headers would contain the
current Python version info and the following inputs fields.

  * get field - directly bring up help on an object/module/or package.
  * Search field - returns a search results index.
  * Modules button - modules index page
  * Keywords button - keywords index page
  * Help button - pydoc Help instructions, version, and credits info.

Note: The leading underscore "_pydoc.py" is to keep it from clashing with the
current pydoc version.  It probably should be something else.


An additional feature is clicking on a filename opens up a numbered source
listing. Which is nice for source code browsing and for referencing specific
lines of code in python list discussions.  ;-)

The colors, fonts and general appearance can be changed by editing the style
sheet.  The output is readable as plain (outline form) text if the style sheet
is ignored by the browser.

_pydoc.py imports pyhelp and uses it to do the introspection work and extends
classes in pyhelp to produce html output.  I've tried to make pyhelp.py useful
in a general way so that it can more easily be used as a base that other output
formats can be built from. That's something that can't be done presently.

These improvements to pydoc mean you can browse pythons library dynamically
without ever leaving the web browser. Currently you switch back and forth
between the browser and a tkinter index window.  Something I found to be
annoying enough to discourage me from using pydoc.


The version I'm working on is split up into eight python files, each addressing
a particular function of pydoc. That was to help me organize things better.
These will be recombined into fewer files.  Some parts of it could be moved to
other modules if they seem more generally useful.  For example, the console text
pager could be used in many other console applications.

Things that still need to be done are adding the object resolution order output
back in.  And adding inter-page html links back in.  And a few other things I
just haven't gotten to yet.  I got a bit burned out on this project a while
back, and then moved to a new house.. etc.. etc..  But I'm starting to have more
time, and with the current discussion s turning on to refining pythons library
this seems like it would be a useful tool in that effort.

Any comments on this general approach?  Any suggestions, questions, or comments?

I'll post a link to the zipped files in the next day or two and announce it
here. I need to look into a glitch on my computer first that's probably a
windows path/name problem. I don't think it's anything that needs to be fixed in
the files but I want to be sure.

Cheers,
 Ron Adam

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python

Re: [Python-Dev] multiple interpreters and extension modules

2006-12-22 Thread Jeremy Kloth
On Friday 22 December 2006 5:02 pm, Josiah Carlson wrote:
> Jeremy Kloth <[EMAIL PROTECTED]> wrote:
> > [[ This may be somewhat c.l.p.-ish but I feel that this crossed into
> > CPython development enough to merit posting here ]]
> >
> > I have received a bug report for 4Suite that involves a
> > PyObject_IsInstance check failing for what appears to be the correct
> > class, that is, the class names match.  With some investigating, I have
> > found that the true problem is with multiple interpreters.  The reason
> > for this is that each sub-interpreter has a "new" copy of any pure Python
> > module. The following issues are also true for modules that have been
> > reloaded, but I think that is common knowledge.  I mention it only for
> > completeness.
>
> If I remember correctly, Python allows you to use multiple interpreters
> in the same process, but it makes no guarantees as to their correctness
> when running.
>
> See this post for further discussion on the issue:
> http://mail.python.org/pipermail/python-list/2004-January/244343.html
>
> You can also search for 'multiple python interpreters single process' in
> google without quotes to hear people lament over the (generally broken)
> multiple Python interpreter support.

The problem here is that it is mod_python using the multiple interpreters.  We 
have no control over that.  What I'm proposing is fixing the extension module 
support for multiple interpreters with the bonus of adding extension module 
finalization which I've seen brought up here before.

Fixing this does require support by the extension module author, but if that 
author doesn't feel the need to work in mod_python (if, of course, they load 
module level constants), that is their loss.

Is 4Suite that different in its use of hybrid Python and C extensions?  There 
is lots of back and forth between the two layers and performance is critical.  
I really don't feel like recoding thousands of lines of Python code into C 
just to get 4Suite to work in mod_python without error.

-- 
Jeremy Kloth
http://4suite.org/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] multiple interpreters and extension modules

2006-12-22 Thread Guido van Rossum
I think you understand exactly what is happening. It is happening for
good reasons. Rather than asking for a change in semantics, I
recommend that you deal with it, either in your Python code, or in
your extension. It's not likely to change.

--Guido

On 12/22/06, Jeremy Kloth <[EMAIL PROTECTED]> wrote:
> [[ This may be somewhat c.l.p.-ish but I feel that this crossed into CPython
> development enough to merit posting here ]]
>
> I have received a bug report for 4Suite that involves a PyObject_IsInstance
> check failing for what appears to be the correct class, that is, the class
> names match.  With some investigating, I have found that the true problem is
> with multiple interpreters.  The reason for this is that each sub-interpreter
> has a "new" copy of any pure Python module. The following issues are also
> true for modules that have been reloaded, but I think that is common
> knowledge.  I mention it only for completeness.
>
> Here is where it crosses into CPython development.
>
> Now extension modules do not get reloaded into each sub-interpreter, but a
> copy of the module's dict from when it was first loaded.  This particular
> extension, when initialized, loads the class in question from the pure Python
> module and stores in in a global variable for quick access. I know I can
> change this specific case to reload the class each time and for this exact
> case, it is most likely what I'll do.  However, we have other extensions that
> make use of this same technique and their performance would suffer greatly by
> doing the lookup for each use of the class.
>
> Also performance isn't the only issue here as we also have C types that
> inherit from a Python class and those types will obviously fail an
> isinstance() check in Python code run in different interpreters as the base
> class is "frozen" at the first import.
>
> So I guess the questions are:
> 1) is subclassing Python classes in C as a static type supported? Even if they
> would be declared on the heap, they would be bound to the first loaded Python
> class.
>
> 2) would it be worthwhile to have an extension loading function similar to
> DllMain() for Windows' DLLs which is called multiple times with a flag
> stating the reason for the call? For reference, see:
> http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/dllmain.asp
> I envision the reasons as: 'module init', 'module fini', 'interpreter
> init', 'interpreter fini'. I see no current need for special treatment of
> threads as they share the same interpreter space.
>
> As to question 2, I'm in the process of prototyping that approach to see if it
> is feasible in our code and if so, (and people find it useful) I'll attempt
> to write a PEP for that feature.
>
> My current thoughts on that are instead of the current init entry
> point in C modules, a new entry point is defined, and if found, used in its
> place. That way, this proposal would be optional for extension writers and
> existing modules would continue to work without change.
>
> Feedback appreciated.
>
> --
> Jeremy Kloth
> http://4suite.org/
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] multiple interpreters and extension modules

2006-12-22 Thread Bob Ippolito
On 12/23/06, Jeremy Kloth <[EMAIL PROTECTED]> wrote:
> On Friday 22 December 2006 5:02 pm, Josiah Carlson wrote:
> > Jeremy Kloth <[EMAIL PROTECTED]> wrote:
> > > [[ This may be somewhat c.l.p.-ish but I feel that this crossed into
> > > CPython development enough to merit posting here ]]
> > >
> > > I have received a bug report for 4Suite that involves a
> > > PyObject_IsInstance check failing for what appears to be the correct
> > > class, that is, the class names match.  With some investigating, I have
> > > found that the true problem is with multiple interpreters.  The reason
> > > for this is that each sub-interpreter has a "new" copy of any pure Python
> > > module. The following issues are also true for modules that have been
> > > reloaded, but I think that is common knowledge.  I mention it only for
> > > completeness.
> >
> > If I remember correctly, Python allows you to use multiple interpreters
> > in the same process, but it makes no guarantees as to their correctness
> > when running.
> >
> > See this post for further discussion on the issue:
> > http://mail.python.org/pipermail/python-list/2004-January/244343.html
> >
> > You can also search for 'multiple python interpreters single process' in
> > google without quotes to hear people lament over the (generally broken)
> > multiple Python interpreter support.
>
> The problem here is that it is mod_python using the multiple interpreters.  We
> have no control over that.  What I'm proposing is fixing the extension module
> support for multiple interpreters with the bonus of adding extension module
> finalization which I've seen brought up here before.
>
> Fixing this does require support by the extension module author, but if that
> author doesn't feel the need to work in mod_python (if, of course, they load
> module level constants), that is their loss.
>
> Is 4Suite that different in its use of hybrid Python and C extensions?  There
> is lots of back and forth between the two layers and performance is critical.
> I really don't feel like recoding thousands of lines of Python code into C
> just to get 4Suite to work in mod_python without error.

It's a whole lot more practical to just stop using mod_python and go
for one of the other ways of exposing Python code to the internet. I
bet you can get the same or better performance out of another solution
anyway, and you'd save deployment headaches.

-bob
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] multiple interpreters and extension modules

2006-12-22 Thread Jeremy Kloth
On Friday 22 December 2006 7:54 pm, Bob Ippolito wrote:
> It's a whole lot more practical to just stop using mod_python and go
> for one of the other ways of exposing Python code to the internet. I
> bet you can get the same or better performance out of another solution
> anyway, and you'd save deployment headaches.

I have no control over end-users choice of Python/webserver integration, I 
just end up making it possible to run our software in the environment of 
*their* choice.

If it is the opinion that it is mod_python that is broken, I'd gladly point 
the users to the location stating that fact/belief.  It would make my life 
easier.

-- 
Jeremy Kloth
http://4suite.org/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] multiple interpreters and extension modules

2006-12-22 Thread Jeremy Kloth
On Friday 22 December 2006 7:16 pm, Guido van Rossum wrote:
> I think you understand exactly what is happening. It is happening for
> good reasons. Rather than asking for a change in semantics, I
> recommend that you deal with it, either in your Python code, or in
> your extension. It's not likely to change.

I don't believe I was asking for a change in semantics, rather an additional, 
optional interface for extension module writers.

> > 2) would it be worthwhile to have an extension loading function similar
> > to DllMain() for Windows' DLLs which is called multiple times with a flag
> > stating the reason for the call? For reference, see:
> > http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/
> >base/dllmain.asp I envision the reasons as: 'module init', 'module fini',
> > 'interpreter init', 'interpreter fini'. I see no current need for special
> > treatment of threads as they share the same interpreter space.
> >
> > As to question 2, I'm in the process of prototyping that approach to see
> > if it is feasible in our code and if so, (and people find it useful) I'll
> > attempt to write a PEP for that feature.

I'll add here that it has been brought up here before that extension module 
finalization is a feature that would be appreciated.  With that, it is not 
that far to add support for initialization/finalization for each interpreter. 
That is, of course, using a DllMain-like solution.

> > My current thoughts on that are instead of the current init
> > entry point in C modules, a new entry point is defined, and if found,
> > used in its place. That way, this proposal would be optional for
> > extension writers and existing modules would continue to work without
> > change.

With that approach in mind, I will be making changes so 4Suite will work in a 
production mod_python deployment (where the aforementioned error occurred). 
When that works, I'll come back with a proper PEP *and* patches against 
Python SVN to support its use. I hope no one was thinking I wanted someone 
else to do the work.

-- 
Jeremy Kloth
http://4suite.org/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] multiple interpreters and extension modules

2006-12-22 Thread Bob Ippolito
On 12/23/06, Jeremy Kloth <[EMAIL PROTECTED]> wrote:
> On Friday 22 December 2006 7:54 pm, Bob Ippolito wrote:
> > It's a whole lot more practical to just stop using mod_python and go
> > for one of the other ways of exposing Python code to the internet. I
> > bet you can get the same or better performance out of another solution
> > anyway, and you'd save deployment headaches.
>
> I have no control over end-users choice of Python/webserver integration, I
> just end up making it possible to run our software in the environment of
> *their* choice.
>
> If it is the opinion that it is mod_python that is broken, I'd gladly point
> the users to the location stating that fact/belief.  It would make my life
> easier.

Well, it clearly is broken wrt pure python modules and objects that
persist across requests. I believe that it's also broken with any
extension that uses the PyGILState API due to the way it interacts
with multiple interpreters.

I stopped using mod_python years ago due to the sorts of issues that
you're bringing up here (plus problems compiling, deploying, RAM
bloat, etc.). I don't have any recent experience or references that I
can point you to, but I can definitely say that I have had many good
experiences with the WSGI based solutions (and Twisted, but that's a
different game).

I would at least advise your user that there are several perfectly
good ways to make Python speak HTTP, and mod_python is the only one
with this issue.

-bob
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Non-blocking (asynchronous) timer without thread?

2006-12-22 Thread Evgeniy Khramtsov
The question to python core developers:
Is there any plans to implement non-blocking timer like a 
threading.Timer() but without thread?
Some interpreted languages (like Tcl or Erlang) have such functionality, 
so I think it would be a great
feature in Python :)

The main goal is to prevent threads overhead and problems with race 
conditions and deadlocks.

Thanks in advance!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Non-blocking (asynchronous) timer without thread?

2006-12-22 Thread Mike Klaas
On 12/22/06, Evgeniy Khramtsov <[EMAIL PROTECTED]> wrote:
> The question to python core developers:
> Is there any plans to implement non-blocking timer like a
> threading.Timer() but without thread?
> Some interpreted languages (like Tcl or Erlang) have such functionality,
> so I think it would be a great
> feature in Python :)
>
> The main goal is to prevent threads overhead and problems with race
> conditions and deadlocks.

I'm not sure how having python execute code at an arbitrary time would
_reduce_ race conditions and/or deadlocks.  And if you want to make it
safe by executing code that shares no variables or resources, then it
is no less safe to use threads, due to the GIL.

If you can write you application in an event-driven way, Twisted might
be able to do what you are looking for.

cheers,
-Mike
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Non-blocking (asynchronous) timer without thread?

2006-12-22 Thread Josiah Carlson

Evgeniy Khramtsov <[EMAIL PROTECTED]> wrote:
> 
> The question to python core developers:
> Is there any plans to implement non-blocking timer like a 
> threading.Timer() but without thread?
> Some interpreted languages (like Tcl or Erlang) have such functionality, 
> so I think it would be a great
> feature in Python :)
> 
> The main goal is to prevent threads overhead and problems with race 
> conditions and deadlocks.

How would you propose a non-threaded timer work?  Would it involve the
writing of a scheduler and the execution of scheduled tasks in a
scheduler?  If so, you can implement it today (I would suggest using the
heapq module, and/or the pair heap I posted to this list a few weeks ago),
or you can use a previously existing mainloop for handling events
(wx.App().MainLoop() works well, you just need to write a wx.EvtHandler
subclass).  Heck, you could even use Twisted's event loop to handle it.

If you were really crazy, you could even use exceptions to signal that
an event was ready...

import time
import sys
_scheduled = []

def hook(*args):
if _scheduled and time.time() >= _scheduled[0][0]:
raise Exception("Scheduled Event Pending!")

sys.settrace(hook)

Aside from using a library that does it for you, or hooking into native
message hooks (I'm sure that Windows has a "notify this process in X
seconds" message), I personally don't see a way of making what you want
happen work in Python, or even *how* you would signal to the only thread
in your application, "hey, there is this other task that should happen"
that is substantially different from a periodic "if time.time() >=
scheduled" check.


 - Josiah

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Non-blocking (asynchronous) timer without thread?

2006-12-22 Thread Evgeniy Khramtsov
Mike Klaas пишет:

> I'm not sure how having python execute code at an arbitrary time would
> _reduce_ race conditions and/or deadlocks. And if you want to make it
> safe by executing code that shares no variables or resources, then it
> is no less safe to use threads, due to the GIL.
>
Ok. And what about a huge thread overhead? Just try to start 10-50k 
threading timers :)

> If you can write you application in an event-driven way, Twisted might
> be able to do what you are looking for.

I don't like an idea of Twisted: you want the banana, but get the whole 
gorilla as well :)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [Python-checkins] r53110 - pyt hon/trunk/Lib/mailbox.py

2006-12-22 Thread Fred L. Drake, Jr.
On Thursday 21 December 2006 15:23, Martin v. Löwis wrote:
 > Now it seem that introducing them has the unfortunate side effect
 > that people think they *have* to use them, and that doing so gives
 > better code...

Speaking strictly for myself:  I don't think I *have* to use them, but I do 
prefer to use them because I don't like magic constants that affect what a 
function does in code; I'd rather have a named constant for readability's 
sake.  Maybe I just can't keep enough in my head, but I don't find I actually 
use seek() enough to remember what the numeric values mean with checking the 
reference documentation.


  -Fred

-- 
Fred L. Drake, Jr.   
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Non-blocking (asynchronous) timer without thread?

2006-12-22 Thread Evgeniy Khramtsov
Josiah Carlson пишет:

>writing of a scheduler and the execution of scheduled tasks in a
>scheduler?  If so, you can implement it today (I would suggest using the
>heapq module, and/or the pair heap I posted to this list a few weeks ago),
>or you can use a previously existing mainloop for handling events
>(wx.App().MainLoop() works well, you just need to write a wx.EvtHandler
>subclass).
>
I did a dirty hack in the asyncore.py module, and now my asyncore objects
have methods for timers (self.start_timer, self.cancel_timer, etc.)
But I think that timer scheduling is a job for a python interpreter and 
it must be hidden from an end programmer.

Regards.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Non-blocking (asynchronous) timer without thread?

2006-12-22 Thread Bob Ippolito
On 12/23/06, Evgeniy Khramtsov <[EMAIL PROTECTED]> wrote:
> Mike Klaas пишет:
>
> > I'm not sure how having python execute code at an arbitrary time would
> > _reduce_ race conditions and/or deadlocks. And if you want to make it
> > safe by executing code that shares no variables or resources, then it
> > is no less safe to use threads, due to the GIL.
> >
> Ok. And what about a huge thread overhead? Just try to start 10-50k
> threading timers :)
>
> > If you can write you application in an event-driven way, Twisted might
> > be able to do what you are looking for.
>
> I don't like an idea of Twisted: you want the banana, but get the whole
> gorilla as well :)

Well you simply can't do what you propose without writing code in the
style of Twisted or with interpreter modifications or evil stack
slicing such as with stackless or greenlet. If you aren't willing to
choose any of those then you'll have to live without that
functionality or use another language (though I can't think of any
usable ones that actually safely do what you're asking). It should be
relatively efficient to do what you want with a thread pool (one
thread that manages all of the timers, and worker threads to execute
the timer callbacks).

FWIW, Erlang doesn't have that functionality. You can wait on messages
with a timeout, but there are no interrupts. You do have cheap and
isolated processes instead of expensive shared state threads, though.
Writing Erlang/OTP code is actually a lot closer to writing Twisted
style code than it is to other styles of concurrency (that you'd find
in Python). It's just that Erlang/OTP has better support for
concurrency oriented programming than Python does (across the board;
syntax, interpreter, convention and libraries).

-bob
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Non-blocking (asynchronous) timer without thread?

2006-12-22 Thread Evgeniy Khramtsov
Bob Ippolito пишет:

> Well you simply can't do what you propose without writing code in the
> style of Twisted or with interpreter modifications or evil stack
> slicing such as with stackless or greenlet. If you aren't willing to
> choose any of those then you'll have to live without that
> functionality or use another language (though I can't think of any
> usable ones that actually safely do what you're asking). It should be
> relatively efficient to do what you want with a thread pool (one
> thread that manages all of the timers, and worker threads to execute
> the timer callbacks).
>
> FWIW, Erlang doesn't have that functionality. You can wait on messages
> with a timeout, but there are no interrupts. You do have cheap and
> isolated processes instead of expensive shared state threads, though.
> Writing Erlang/OTP code is actually a lot closer to writing Twisted
> style code than it is to other styles of concurrency (that you'd find
> in Python). It's just that Erlang/OTP has better support for
> concurrency oriented programming than Python does (across the board;
> syntax, interpreter, convention and libraries).

Ok. I see your point. It looks like Python was not the right choice for 
my task :(

Thanks.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com