"Richard Oudkerk" <[EMAIL PROTECTED]> wrote: > I am not sure how sensible the idea is, but I have had a first stab at > writing a module processing.py which is a near clone of threading.py > but uses processes and sockets for communication. (It is one way of > avoiding the GIL.)
On non-windows platforms, you should check on unix domain sockets, I've found they can run a couple times faster than standard sockets on the local machine. And if you are using fork or a variant of subprocess to start processes on linux or Windows, you should consider using pipes, they can be competitive with sockets (though using a bunch on Windows can be a pain). > I have tested it on unix and windows and it seem to work pretty well. > (Getting round the lack of os.fork on windows is a bit awkward.) > There is also another module dummy_processing.py which has the same > api but is just a wrapper round threading.py. > > Queues, Locks, RLocks, Conditions, Semaphores and some other shared > objects are implemented. > > People are welcome to try out the tests in test_processing.py > contained in the zipfile. More information is included in the README > file. > > As a quick example, the code [snip] Looks interesting. Maybe it would become clearer with docs (I hope you've written some). Right now there is a difference, and it is basically that there are tokens and proxies, which could confuse some users. Presumably with this library you have created, you have also written a fast object encoder/decoder (like marshal or pickle). If it isn't any faster than cPickle or marshal, then users may bypass the module and opt for fork/etc. + XML-RPC; which works pretty well and gets them multi-machine calling, milti-language interoperability, and some other goodies, though it is a bit slow in terms of communication. - Josiah _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
