Re: [Cython] cython.parallel tasks, single, master, critical, barriers
Meta: I've been meaning to respond to this thread, but can't find the time. What's the time-frame for implementing this? If it's hypothetical at the moment and just is a question of getting things spec-ed, one could perhaps look at discussing it at the next Cython workshop, or perhaps a Skype call with the three of us as some point... Regarding the tasks: One of my biggest problems with Python is the lack of an elegant syntax for anonymous functions. But since Python has that problem, I feel it is not necesarrily something we should fix (by using the with statements to create tasks). Sometimes Pythonic-ness is more important than elegance (for Cython). In general I'm happy as long as there's a chance of getting things to work in pure Python mode as well (with serial execution). So if, e.g., with statements creating tasks have the same effect when running the same code (serially) in pure Python, I'm less opposed (didn't look at it in detail). Dag Sverre On 10/19/2011 09:53 PM, mark florisson wrote: On 19 October 2011 06:01, Robert Bradshaw wrote: On Fri, Oct 14, 2011 at 1:07 PM, mark florisson wrote: On 14 October 2011 19:31, Robert Bradshaw wrote: On Wed, Oct 12, 2011 at 7:55 AM, mark florisson wrote: I ultimately feel things like that is more important than 100% coverage of the OpenMP standard. Of course, OpenMP is a lot lower-hanging fruit. +1 Prange handles the (corse-grained) SIMD case nicely, and a task/futures model based on closures would I think flesh this out to the next level of generality (and complexity). Futures are definitely nice. I suppose I think really like "inline futures", i.e. openmp tasks. I realize that futures may look more pythonic. However, as mentioned previously, I also see issues with that. When you submit a task then you expect a future object, which you might want to pass around. But we don't have the GIL for that. I personally feel that futures is something that should be done by a library (such as concurrent.futures in python 3.2), and inline tasks by a language. It also means I have to write an entire function or closure for perhaps only a few lines of code. I might also want to submit other functions that are not closures, or I might want to reuse my closures that are used for tasks and for something else. So what if my tasks contain more parallel constructs? e.g. what if I have a task closure that I return from my function that generates more tasks itself? Would you just execute them sequentially outside of the parallel construct, or would you simply disallow that? Also, do you restrict future "objects" to only the parallel section? Another problem is that you can only wait on tasks of your direct children. So what if I get access to my parent's future object (assuming you allow tasks to generate tasks), and then want the result of my parent? Or what if I store these future objects in an array or list and access them arbitrarily? You will only know at runtime which task to wait on, and openmp only has a static, lexical taskwait. I suppose my point is that without either a drastic rewrite (e.g., use pthreads instead of openmp) or quite a bit of contraints, I am unsure how futures would work here. Perhaps you guys have some concrete syntax and semantics proposals? It feels to me that OpenMP tasks took a different model of parallelism and tried to force them into the OpenMP model/constraints, and so it'd be even more difficult to fit them into a nice pythonic interface. Perhaps to make progress on this front we need to have a concrete example to look at. I'm also wondering if the standard threading module (perhaps with overlay support) used with nogil functions would be sufficient--locking is required for handling the queues, etc. so the fact that the GIL is involved is not a big deal. It is possible that this won't scale to as small of work units, but the overhead should be minimal once your work unit is a sufficient size (which is probably quite small) and it's already implemented and well documented/used. It's all definitely possible with normal threads, but the thing you lose is convenience and conciseness. For big problems the programmer might sum up the courage and effort to implement it, but typically you will just stick to a serial version. This is really where OpenMP is powerful, you can take a simple sequential piece of code and make it parallel with minimal effort and without having to restructure, rethink and rewrite your algorithms. That is a very good point. Something like concurrent.futures is definitely nice, but most people cannot afford to mandate python 3.2 for their users. The most classical examples I can think of for tasks are 1) independent code sections, i.e. two or more pieces of code that don't depend on each other which you want to execute in parallel 2) traversal of some kind of custom data structure, like a tree or a linked list 3) some kind of other producer/consumer model e.g. using with task syn
Re: [Cython] cython.parallel tasks, single, master, critical, barriers
On 20 October 2011 09:42, Dag Sverre Seljebotn wrote: > Meta: I've been meaning to respond to this thread, but can't find the time. > What's the time-frame for implementing this? If it's hypothetical at the > moment and just is a question of getting things spec-ed, one could perhaps > look at discussing it at the next Cython workshop, or perhaps a Skype call > with the three of us as some point... For me this is just about getting this spec-ed, so that when someone finds the time, we don't need to discuss it for weeks first. And the implementor won't necessarily have to support everything at once, e.g. just critical sections or barriers alone would be nice. Is there any plan for a new workshop then? Because if it's in two years I think we could be more time-efficient :) > Regarding the tasks: One of my biggest problems with Python is the lack of > an elegant syntax for anonymous functions. But since Python has that > problem, I feel it is not necesarrily something we should fix (by using the > with statements to create tasks). Sometimes Pythonic-ness is more important > than elegance (for Cython). I agree it's not something we should fix, I just think tasks are most useful in inline blocks and not in separate functions or closures. Although it could certainly work, I think it restricts more, leads to more verbose code and possibly questionable semantics, and on top of that it would be a pain to implement (although that should not be used as a persuasive argument). I'm not saying there is no elegant way other than with blocks, I'm just saying that I think closures are not the right thing for it. > In general I'm happy as long as there's a chance of getting things to work > in pure Python mode as well (with serial execution). So if, e.g., with > statements creating tasks have the same effect when running the same code > (serially) in pure Python, I'm less opposed (didn't look at it in detail). Yes, it would have the same effect. The thing with tasks (and OpenMP constructs in general) is that usually if your compiler ignores all your pragmas, your code just runs serially in the same way. The same would be true for the tasks in with blocks. > Dag Sverre > > On 10/19/2011 09:53 PM, mark florisson wrote: >> >> On 19 October 2011 06:01, Robert Bradshaw >> wrote: >>> >>> On Fri, Oct 14, 2011 at 1:07 PM, mark florisson >>> wrote: On 14 October 2011 19:31, Robert Bradshaw wrote: > > On Wed, Oct 12, 2011 at 7:55 AM, mark florisson > wrote: I ultimately feel things like that is more important than 100% coverage of the OpenMP standard. Of course, OpenMP is a lot lower-hanging fruit. >>> >>> +1 Prange handles the (corse-grained) SIMD case nicely, and a >>> task/futures model based on closures would I think flesh this out to >>> the next level of generality (and complexity). >> >> Futures are definitely nice. I suppose I think really like "inline >> futures", i.e. openmp tasks. I realize that futures may look more >> pythonic. However, as mentioned previously, I also see issues with >> that. When you submit a task then you expect a future object, which >> you might want to pass around. But we don't have the GIL for that. I >> personally feel that futures is something that should be done by a >> library (such as concurrent.futures in python 3.2), and inline tasks >> by a language. It also means I have to write an entire function or >> closure for perhaps only a few lines of code. >> >> I might also want to submit other functions that are not closures, or >> I might want to reuse my closures that are used for tasks and for >> something else. So what if my tasks contain more parallel constructs? >> e.g. what if I have a task closure that I return from my function that >> generates more tasks itself? Would you just execute them sequentially >> outside of the parallel construct, or would you simply disallow that? >> Also, do you restrict future "objects" to only the parallel section? >> >> Another problem is that you can only wait on tasks of your direct >> children. So what if I get access to my parent's future object >> (assuming you allow tasks to generate tasks), and then want the result >> of my parent? >> Or what if I store these future objects in an array or list and access >> them arbitrarily? You will only know at runtime which task to wait on, >> and openmp only has a static, lexical taskwait. >> >> I suppose my point is that without either a drastic rewrite (e.g., use >> pthreads instead of openmp) or quite a bit of contraints, I am unsure >> how futures would work here. Perhaps you guys have some concrete >> syntax and semantics proposals? > > It feels to me that OpenMP tasks took a different model of parallelism > and tried to force them into the OpenMP model/constraints, and so it'd
Re: [Cython] cython.parallel tasks, single, master, critical, barriers
On 10/20/2011 11:13 AM, mark florisson wrote: On 20 October 2011 09:42, Dag Sverre Seljebotn wrote: Meta: I've been meaning to respond to this thread, but can't find the time. What's the time-frame for implementing this? If it's hypothetical at the moment and just is a question of getting things spec-ed, one could perhaps look at discussing it at the next Cython workshop, or perhaps a Skype call with the three of us as some point... For me this is just about getting this spec-ed, so that when someone finds the time, we don't need to discuss it for weeks first. And the implementor won't necessarily have to support everything at once, e.g. just critical sections or barriers alone would be nice. Is there any plan for a new workshop then? Because if it's in two years I think we could be more time-efficient :) At least in William's grant there's plans for 2-3 Cython workshops, so hopefully there's funding for one next year if we want to. We should ask him before planning anything though. Regarding the tasks: One of my biggest problems with Python is the lack of an elegant syntax for anonymous functions. But since Python has that problem, I feel it is not necesarrily something we should fix (by using the with statements to create tasks). Sometimes Pythonic-ness is more important than elegance (for Cython). I agree it's not something we should fix, I just think tasks are most useful in inline blocks and not in separate functions or closures. Although it could certainly work, I think it restricts more, leads to more verbose code and possibly questionable semantics, and on top of that it would be a pain to implement (although that should not be used as a persuasive argument). I'm not saying there is no elegant way other than with blocks, I'm just saying that I think closures are not the right thing for it. In general I'm happy as long as there's a chance of getting things to work in pure Python mode as well (with serial execution). So if, e.g., with statements creating tasks have the same effect when running the same code (serially) in pure Python, I'm less opposed (didn't look at it in detail). Yes, it would have the same effect. The thing with tasks (and OpenMP constructs in general) is that usually if your compiler ignores all your pragmas, your code just runs serially in the same way. The same would be true for the tasks in with blocks. Short note: I like the vision of Konrad Hinsen: http://www.euroscipy.org/talk/2011 The core idea is that the "task-ness" of a block of code is orthogonal to the place you actually write it. That is, a block of code may often either be fit for execution as a task, or not, depending on how heavy it is (= values of arguments it takes in, not its contents). He introduces the "async" expression to drive this point through. I think "with task" is fine if used in this way, if you simply call a function (which itself doesn't know whether it is a task or not). But once you start to implement an entire function within the with-statement there's a code-smell. Anyway, it's growing on me. But I think his "async" expression is more Pythonic in the way that it forces you away from making your code smell. We could simply have async(func)(arg, arg2, somekwarg=4) (He also says "functional-style programming is better for parallization than threads+locks", which I can kind of agree with but nobody tried to make an efficient immutable array implementation suitable for numerical computation yet to my knowledge... that's an interesting MSc or PhD-topic, but I already have one :-) ) (Look at me, going along discussing when I really shouldn't -- see you later.) Dag Sverre ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] cython.parallel tasks, single, master, critical, barriers
On 20 October 2011 10:35, Dag Sverre Seljebotn wrote: > On 10/20/2011 11:13 AM, mark florisson wrote: >> >> On 20 October 2011 09:42, Dag Sverre Seljebotn >> wrote: >>> >>> Meta: I've been meaning to respond to this thread, but can't find the >>> time. >>> What's the time-frame for implementing this? If it's hypothetical at the >>> moment and just is a question of getting things spec-ed, one could >>> perhaps >>> look at discussing it at the next Cython workshop, or perhaps a Skype >>> call >>> with the three of us as some point... >> >> For me this is just about getting this spec-ed, so that when someone >> finds the time, we don't need to discuss it for weeks first. And the >> implementor won't necessarily have to support everything at once, e.g. >> just critical sections or barriers alone would be nice. >> >> Is there any plan for a new workshop then? Because if it's in two >> years I think we could be more time-efficient :) > > At least in William's grant there's plans for 2-3 Cython workshops, so > hopefully there's funding for one next year if we want to. We should ask him > before planning anything though. > >>> Regarding the tasks: One of my biggest problems with Python is the lack >>> of >>> an elegant syntax for anonymous functions. But since Python has that >>> problem, I feel it is not necesarrily something we should fix (by using >>> the >>> with statements to create tasks). Sometimes Pythonic-ness is more >>> important >>> than elegance (for Cython). >> >> I agree it's not something we should fix, I just think tasks are most >> useful in inline blocks and not in separate functions or closures. >> Although it could certainly work, I think it restricts more, leads to >> more verbose code and possibly questionable semantics, and on top of >> that it would be a pain to implement (although that should not be used >> as a persuasive argument). I'm not saying there is no elegant way >> other than with blocks, I'm just saying that I think closures are not >> the right thing for it. >> >>> In general I'm happy as long as there's a chance of getting things to >>> work >>> in pure Python mode as well (with serial execution). So if, e.g., with >>> statements creating tasks have the same effect when running the same code >>> (serially) in pure Python, I'm less opposed (didn't look at it in >>> detail). >> >> Yes, it would have the same effect. The thing with tasks (and OpenMP >> constructs in general) is that usually if your compiler ignores all >> your pragmas, your code just runs serially in the same way. The same >> would be true for the tasks in with blocks. > > Short note: I like the vision of Konrad Hinsen: > > http://www.euroscipy.org/talk/2011 > > The core idea is that the "task-ness" of a block of code is orthogonal to > the place you actually write it. That is, a block of code may often either > be fit for execution as a task, or not, depending on how heavy it is (= > values of arguments it takes in, not its contents). > > He introduces the "async" expression to drive this point through. > > I think "with task" is fine if used in this way, if you simply call a > function (which itself doesn't know whether it is a task or not). But once > you start to implement an entire function within the with-statement there's > a code-smell. Definitely, do you'd just call the function from the task. > Anyway, it's growing on me. But I think his "async" expression is more > Pythonic in the way that it forces you away from making your code smell. > > We could simply have > > async(func)(arg, arg2, somekwarg=4) > That looks good. The question is, does this constitute an expression or a statement? If it's an expression, then you expect a meaningful return value, which means you're going to have to wait for the task to complete. That would be fine if you submit multiple tasks in one expression, from the slides: max(async expr1, async expr2) or even [async expr for ... in ...] I must say, it does look really elegant and it doesn't leave the user to question when the task is executed (and if you need a taskwait directive to wait for your variables to become defined). What I don't see is how to do the producer consumer trick, unless you regard using the result of async as a taskwait, and not using it as not having a taskwait, e.g. async func(...) # generate a task and don't wait for it result = async func(...) # generate a task and wait for it. The latter is not useful unless you have multiple expressions in one statement, so we should also allow result1, result2 = async func(data=a), async func(data=b). I think you would need special support for the expression form to work in multiple places, e.g. as a start you could allow it as function arguments, tuple expressions and possibly a nogil form of list comprehensions. The statement form is a lot more simple, and as a start synchronization must simply be done through barriers. If you want to change additional data through mechanisms other than imme
Re: [Cython] ImportError: DLL load failed: The specified module could not be found.
Dear Alexander and cython list, the same question, different approach a) I compiled and installed my python module with these two commands C:\Temporal\source\Cython_ext> python OK_setup_windows.py build_ext C:\Temporal\source\Cython_ext> python OK_setup_windows.py install b) Testing my module in IPython - In [5]: cd c:\ c:\ In [6]: import okriging_py as ok Traceback (most recent call last): File "", line 1, in ImportError: DLL load failed: The specified module could not be found. In [7]: cd C:\Python27\Lib\site-packages C:\Python27\Lib\site-packages In [8]: import okriging_py as ok In [9]: - as you can see the module works if we call it from the source directory. my question are: a) where is the problem b) how to distribute my module without this (possible system configuration) error my OS is windows 7 (probably with win xp compatibility) sorry about my ignorance (I am more Linux Debian user...) Regards Adrian On 19/10/2011 12:45 PM, Alexander T. Berghage wrote: Adrian I'm a little unclear on the big picture here. Are you trying to distribute a module (a .pyd / .dll) that you or someone else can import from a .py script, or are you looking to compile a .exe that runs your cython code on execution? Just interpreting the error you're describing (ImportError: DLL load failed: could not be found), the dynamic linker couldn't find a library it needed. Most likely this is either a symptom of missing dependencies or a path problem. Here's my suggestions for diagnosing and fixing the problem: Missing Dependencies: One very simple way to confirm that all the dependencies of your cython module are available is to point the dependency walker utility[1] at it, and look for missing DLLs. Directory Structure: Is the .pyd file you built from your cython module in the PYTHONPATH (or your current working directory? If it's not, there's your issue. [1] http://www.dependencywalker.com/ Hope that helps! Best, -Alex ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
