Russel Winder wrote:
Steven,
On Sun, 2012-05-13 at 10:22 +1000, Steven D'Aprano wrote:
carlo locci wrote:
Hello All,
I've started to study python a couple of month ago(and I truly love it :)),
however I'm having some problems understanding how to modify a sequential
script and make it multithreaded (I think it's because I'm not used to
think in that way),
No, that's because multithreading and parallel processing is hard.
Shared memory multithreading may be hard due to locks, semaphores,
monitors, etc., but concurrency and parallelism need not be hard.
No hard compared to what?
Using processes and message passing, using dataflow, actors or CSP,
parallelism and concurrency is far more straightforward. Not easy,
agreed, but then programming isn't easy.
My argument is that once you move beyond the one-operation-after-another
programming model, almost any parallel processing problem is harder than the
equivalent sequential version, inherently due to the parallelism. Except
perhaps for "embarrassingly parallel" problems, parallelism adds complexity
even if your framework abstracts away most of the tedious detail like semaphores.
http://en.wikipedia.org/wiki/Embarrassingly_parallel
Once you move beyond sequential execution, you have to think about issues that
don't apply to sequential programs: how to divide the task up between
processes/threads/actors/whatever, how to manage their synchronization,
resource starvation (e.g. deadlocks, livelocks), etc.
We have linear minds and it doesn't take that many real-time parallel tasks to
overwhelm the human brain. I'm not saying that people can't reason in
parallel, because we clearly can and do, but it's inherently harder than
sequential reasoning.
The GIL in Python is a bad thing for parallelism. Using the
multiprocessing package or concurrent.futures gets over the problem.
Well sort of, these processes are a bit heavyweight compared to what can
be achieved on the JVM or with Erlang.
Python doesn't have a GIL. Some Python implementations do, most obviously
CPython, the reference implementation. But Jython and IronPython don't. If the
GIL is a problem for your program, consider running it on Jython or IronPython.
--
Steven
_______________________________________________
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
