Re: [Cython] prange CEP updated
On 5 April 2011 22:29, Dag Sverre Seljebotn wrote: > I've done a pretty major revision to the prange CEP, bringing in a lot of > the feedback. > > Thread-private variables are now split in two cases: > > i) The safe cases, which really require very little technical knowledge -> > automatically inferred > > ii) As an advanced feature, unsafe cases that requires some knowledge of > threading -> must be explicitly declared > > I think this split simplifies things a great deal. Can't we obsolete the declaration entirely by assigning to variables that need to have firstprivate behaviour inside the with parallel block? Basically in the same way the scratch space is used. The only problem with that is that it won't be lastprivate, so the value will be undefined after the parallel block (but not after the worksharing loop). cdef int myvariable with nogil, parallel: myvariable = 2 for i in prange(...): use myvariable maybe assign to myvariable # myvariable is well-defined here # myvariable is not well-defined here If you still desperately want lastprivate behaviour you can simply assign myvariable to another variable in the loop body. > I'm rather excited over this now; this could turn out to be a really > user-friendly and safe feature that would not only allow us to support > OpenMP-like threading, but be more convenient to use in a range of common > cases. > > http://wiki.cython.org/enhancements/prange > > Dag Sverre > > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel > > ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] prange CEP updated
On 04/11/2011 10:45 AM, mark florisson wrote: On 5 April 2011 22:29, Dag Sverre Seljebotn wrote: I've done a pretty major revision to the prange CEP, bringing in a lot of the feedback. Thread-private variables are now split in two cases: i) The safe cases, which really require very little technical knowledge -> automatically inferred ii) As an advanced feature, unsafe cases that requires some knowledge of threading -> must be explicitly declared I think this split simplifies things a great deal. Can't we obsolete the declaration entirely by assigning to variables that need to have firstprivate behaviour inside the with parallel block? Basically in the same way the scratch space is used. The only problem with that is that it won't be lastprivate, so the value will be undefined after the parallel block (but not after the worksharing loop). cdef int myvariable with nogil, parallel: myvariable = 2 for i in prange(...): use myvariable maybe assign to myvariable # myvariable is well-defined here # myvariable is not well-defined here If you still desperately want lastprivate behaviour you can simply assign myvariable to another variable in the loop body. I don't care about lastprivate, I don't think that is an issue, as you say. My problem with this is that it means going into an area where possibly tricky things are implicit rather than explicit. I also see this as a rather special case that will be seldomly used, and implicit behaviour is more difficult to justify because of that. (The other instance of thread-local variables I feel is still explicit: You use prange instead of range, which means that you declare that values created in the iteration does not leak to the next iteration. The rest is just optimization from there.) As Robert said in his recent talk: A lot of languages are easy to write. The advantage of Python is that it is easy to *read*. That's what I feel is wrong with the proposal above: An assignment to a variable changes the semantics of it. Granted, it happens in a way so that it will almost always be correct, but I feel that reading the code, I'd spend some extra cycles to go "ah, so this variable is thread-local and therefore its values survive across a loop iteration". If I even knew about the feature in the first place. In seeing "threadprivate" spelled out, it is either obvious what it means, or obvious that I should look up the docs. There's *a lot* of things that can be made implicit in a programming language; Python/Cython simply usually leans towards the explicit side. Oh, and we may want to support writable shared variables (and flush) eventually too, and the above doesn't easily differentiate there? That's just my opinion, I'm happy to be overruled here. Dag Sverre ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] prange CEP updated
On 11 April 2011 11:10, Dag Sverre Seljebotn wrote: > On 04/11/2011 10:45 AM, mark florisson wrote: >> >> On 5 April 2011 22:29, Dag Sverre Seljebotn >> wrote: >>> >>> I've done a pretty major revision to the prange CEP, bringing in a lot of >>> the feedback. >>> >>> Thread-private variables are now split in two cases: >>> >>> i) The safe cases, which really require very little technical knowledge >>> -> >>> automatically inferred >>> >>> ii) As an advanced feature, unsafe cases that requires some knowledge of >>> threading -> must be explicitly declared >>> >>> I think this split simplifies things a great deal. >> >> Can't we obsolete the declaration entirely by assigning to variables >> that need to have firstprivate behaviour inside the with parallel >> block? Basically in the same way the scratch space is used. The only >> problem with that is that it won't be lastprivate, so the value will >> be undefined after the parallel block (but not after the worksharing >> loop). >> >> cdef int myvariable >> >> with nogil, parallel: >> myvariable = 2 >> for i in prange(...): >> use myvariable >> maybe assign to myvariable >> >> # myvariable is well-defined here >> >> # myvariable is not well-defined here >> >> If you still desperately want lastprivate behaviour you can simply >> assign myvariable to another variable in the loop body. > > I don't care about lastprivate, I don't think that is an issue, as you say. > > My problem with this is that it means going into an area where possibly > tricky things are implicit rather than explicit. I also see this as a rather > special case that will be seldomly used, and implicit behaviour is more > difficult to justify because of that. Indeed, I actually considered if we should support firstprivate at all, as it's really about "being firstprivate and lastprivate". Without any declaration, you can have firstprivate or lastprivate, but not both :) So I agree that supporting such a (probably) uncommon case is better left explicit. On the other hand it seems silly to have support for such a weird case. > (The other instance of thread-local variables I feel is still explicit: You > use prange instead of range, which means that you declare that values > created in the iteration does not leak to the next iteration. The rest is > just optimization from there.) > > As Robert said in his recent talk: A lot of languages are easy to write. The > advantage of Python is that it is easy to *read*. That's what I feel is > wrong with the proposal above: An assignment to a variable changes the > semantics of it. Granted, it happens in a way so that it will almost always > be correct, but I feel that reading the code, I'd spend some extra cycles to > go "ah, so this variable is thread-local and therefore its values survive > across a loop iteration". > > If I even knew about the feature in the first place. In seeing > "threadprivate" spelled out, it is either obvious what it means, or obvious > that I should look up the docs. > > There's *a lot* of things that can be made implicit in a programming > language; Python/Cython simply usually leans towards the explicit side. > > Oh, and we may want to support writable shared variables (and flush) > eventually too, and the above doesn't easily differentiate there? Right, everything is implicit. So I guess it'll be good to introduce it anyway as you say, so we can later declare stuff shared with similar syntax. I suppose that's the point where I'm convinced. > That's just my opinion, I'm happy to be overruled here. > > Dag Sverre > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] prange CEP updated
On 04/11/2011 11:41 AM, mark florisson wrote: On 11 April 2011 11:10, Dag Sverre Seljebotn wrote: On 04/11/2011 10:45 AM, mark florisson wrote: On 5 April 2011 22:29, Dag Sverre Seljebotn wrote: I've done a pretty major revision to the prange CEP, bringing in a lot of the feedback. Thread-private variables are now split in two cases: i) The safe cases, which really require very little technical knowledge -> automatically inferred ii) As an advanced feature, unsafe cases that requires some knowledge of threading ->must be explicitly declared I think this split simplifies things a great deal. Can't we obsolete the declaration entirely by assigning to variables that need to have firstprivate behaviour inside the with parallel block? Basically in the same way the scratch space is used. The only problem with that is that it won't be lastprivate, so the value will be undefined after the parallel block (but not after the worksharing loop). cdef int myvariable with nogil, parallel: myvariable = 2 for i in prange(...): use myvariable maybe assign to myvariable # myvariable is well-defined here # myvariable is not well-defined here If you still desperately want lastprivate behaviour you can simply assign myvariable to another variable in the loop body. I don't care about lastprivate, I don't think that is an issue, as you say. My problem with this is that it means going into an area where possibly tricky things are implicit rather than explicit. I also see this as a rather special case that will be seldomly used, and implicit behaviour is more difficult to justify because of that. Indeed, I actually considered if we should support firstprivate at all, as it's really about "being firstprivate and lastprivate". Without any declaration, you can have firstprivate or lastprivate, but not both :) So I agree that supporting such a (probably) uncommon case is better left explicit. On the other hand it seems silly to have support for such a weird case. Well, I actually need to do the per-thread cache thing I described in the CEP in my own codes, so it's not *that* special; it'd be nice to support it. OTOH I *could* work around it by having an array of scalars cdef int[:] old_ell = int[:numthreads]() ... if old_ell[threadid()] != ell: ... So I guess, it's at least on the bottom of list of priorities in that CEP. Dag Sverre ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] prange CEP updated
On 11 April 2011 12:08, Dag Sverre Seljebotn wrote: > On 04/11/2011 11:41 AM, mark florisson wrote: >> >> On 11 April 2011 11:10, Dag Sverre Seljebotn >> wrote: >>> >>> On 04/11/2011 10:45 AM, mark florisson wrote: On 5 April 2011 22:29, Dag Sverre Seljebotn wrote: > > I've done a pretty major revision to the prange CEP, bringing in a lot > of > the feedback. > > Thread-private variables are now split in two cases: > > i) The safe cases, which really require very little technical > knowledge > -> > automatically inferred > > ii) As an advanced feature, unsafe cases that requires some knowledge > of > threading -> must be explicitly declared > > I think this split simplifies things a great deal. Can't we obsolete the declaration entirely by assigning to variables that need to have firstprivate behaviour inside the with parallel block? Basically in the same way the scratch space is used. The only problem with that is that it won't be lastprivate, so the value will be undefined after the parallel block (but not after the worksharing loop). cdef int myvariable with nogil, parallel: myvariable = 2 for i in prange(...): use myvariable maybe assign to myvariable # myvariable is well-defined here # myvariable is not well-defined here If you still desperately want lastprivate behaviour you can simply assign myvariable to another variable in the loop body. >>> >>> I don't care about lastprivate, I don't think that is an issue, as you >>> say. >>> >>> My problem with this is that it means going into an area where possibly >>> tricky things are implicit rather than explicit. I also see this as a >>> rather >>> special case that will be seldomly used, and implicit behaviour is more >>> difficult to justify because of that. >> >> Indeed, I actually considered if we should support firstprivate at >> all, as it's really about "being firstprivate and lastprivate". >> Without any declaration, you can have firstprivate or lastprivate, but >> not both :) So I agree that supporting such a (probably) uncommon case >> is better left explicit. On the other hand it seems silly to have >> support for such a weird case. > > Well, I actually need to do the per-thread cache thing I described in the > CEP in my own codes, so it's not *that* special; it'd be nice to support it. You need 'old_ell' and 'alpha' after the loop? > OTOH I *could* work around it by having an array of scalars > > cdef int[:] old_ell = int[:numthreads]() > > ... > if old_ell[threadid()] != ell: ... > > > So I guess, it's at least on the bottom of list of priorities in that CEP. > > Dag Sverre > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] Test runner
Can we select tests in the tests directory selectively? I see the -T or --ticket option, but it doens't seem to find the test tagged with # ticket: . I can select unit tests using python runtests.py Cython.SubPackage.Tests.SomeTest, but I can't seem to do the same thing for tests in the tests directory. Running the entire suite takes rather long. ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Test runner
mark florisson, 11.04.2011 12:26: Can we select tests in the tests directory selectively? I see the -T or --ticket option, but it doens't seem to find the test tagged with # ticket:. I can select unit tests using python runtests.py Cython.SubPackage.Tests.SomeTest, but I can't seem to do the same thing for tests in the tests directory. Running the entire suite takes rather long. You can still select them by name using a regex, e.g. runtests.py 'run\.empty_builtin_constructors' Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Test runner
On 11 April 2011 12:45, Stefan Behnel wrote: > mark florisson, 11.04.2011 12:26: >> >> Can we select tests in the tests directory selectively? I see the -T >> or --ticket option, but it doens't seem to find the test tagged with # >> ticket:. >> >> I can select unit tests using python runtests.py >> Cython.SubPackage.Tests.SomeTest, but I can't seem to do the same >> thing for tests in the tests directory. Running the entire suite takes >> rather long. > > You can still select them by name using a regex, e.g. > > runtests.py 'run\.empty_builtin_constructors' > > Stefan > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel > Great, thanks! I'll update the hackerguide wiki. ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Test runner
On 11 April 2011 12:53, mark florisson wrote: > On 11 April 2011 12:45, Stefan Behnel wrote: >> mark florisson, 11.04.2011 12:26: >>> >>> Can we select tests in the tests directory selectively? I see the -T >>> or --ticket option, but it doens't seem to find the test tagged with # >>> ticket:. >>> >>> I can select unit tests using python runtests.py >>> Cython.SubPackage.Tests.SomeTest, but I can't seem to do the same >>> thing for tests in the tests directory. Running the entire suite takes >>> rather long. >> >> You can still select them by name using a regex, e.g. >> >> runtests.py 'run\.empty_builtin_constructors' >> >> Stefan >> ___ >> cython-devel mailing list >> cython-devel@python.org >> http://mail.python.org/mailman/listinfo/cython-devel >> > > Great, thanks! I'll update the hackerguide wiki. > I see now that it is briefly mentioned there, apologies. ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] prange CEP updated
On 04/11/2011 12:14 PM, mark florisson wrote: On 11 April 2011 12:08, Dag Sverre Seljebotn wrote: On 04/11/2011 11:41 AM, mark florisson wrote: On 11 April 2011 11:10, Dag Sverre Seljebotn wrote: On 04/11/2011 10:45 AM, mark florisson wrote: On 5 April 2011 22:29, Dag Sverre Seljebotn wrote: I've done a pretty major revision to the prange CEP, bringing in a lot of the feedback. Thread-private variables are now split in two cases: i) The safe cases, which really require very little technical knowledge -> automatically inferred ii) As an advanced feature, unsafe cases that requires some knowledge of threading -> must be explicitly declared I think this split simplifies things a great deal. Can't we obsolete the declaration entirely by assigning to variables that need to have firstprivate behaviour inside the with parallel block? Basically in the same way the scratch space is used. The only problem with that is that it won't be lastprivate, so the value will be undefined after the parallel block (but not after the worksharing loop). cdef int myvariable with nogil, parallel: myvariable = 2 for i in prange(...): use myvariable maybe assign to myvariable # myvariable is well-defined here # myvariable is not well-defined here If you still desperately want lastprivate behaviour you can simply assign myvariable to another variable in the loop body. I don't care about lastprivate, I don't think that is an issue, as you say. My problem with this is that it means going into an area where possibly tricky things are implicit rather than explicit. I also see this as a rather special case that will be seldomly used, and implicit behaviour is more difficult to justify because of that. Indeed, I actually considered if we should support firstprivate at all, as it's really about "being firstprivate and lastprivate". Without any declaration, you can have firstprivate or lastprivate, but not both :) So I agree that supporting such a (probably) uncommon case is better left explicit. On the other hand it seems silly to have support for such a weird case. Well, I actually need to do the per-thread cache thing I described in the CEP in my own codes, so it's not *that* special; it'd be nice to support it. You need 'old_ell' and 'alpha' after the loop? No...but I need the values to not be blanked out at the beginning of each loop iteration! Note that in the CEP, the implicitly thread-local variables are *not available* before the first assignment in the loop. That is, code such as this is NOT allowed: cdef double x ... for i in prange(10): print x x = f(x) We raise a compiler error in such cases if we can: The code above is violating the contract that the order of execution of loop bodies should not matter. In cases where we can't raise an error (because we didn't bother or because it is not possible with a proof), we still initialize the variables to invalid values (NaN for double) at the beginning of the for-loop just to be sure the contract is satisfied. This was added to answer Stefan's objection to new types of implicit scopes (and I agree with his concern). Dag Sverre ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] prange CEP updated
On 04/11/2011 01:02 PM, Dag Sverre Seljebotn wrote: On 04/11/2011 12:14 PM, mark florisson wrote: On 11 April 2011 12:08, Dag Sverre Seljebotn wrote: On 04/11/2011 11:41 AM, mark florisson wrote: On 11 April 2011 11:10, Dag Sverre Seljebotn wrote: On 04/11/2011 10:45 AM, mark florisson wrote: On 5 April 2011 22:29, Dag Sverre Seljebotn wrote: I've done a pretty major revision to the prange CEP, bringing in a lot of the feedback. Thread-private variables are now split in two cases: i) The safe cases, which really require very little technical knowledge -> automatically inferred ii) As an advanced feature, unsafe cases that requires some knowledge of threading -> must be explicitly declared I think this split simplifies things a great deal. Can't we obsolete the declaration entirely by assigning to variables that need to have firstprivate behaviour inside the with parallel block? Basically in the same way the scratch space is used. The only problem with that is that it won't be lastprivate, so the value will be undefined after the parallel block (but not after the worksharing loop). cdef int myvariable with nogil, parallel: myvariable = 2 for i in prange(...): use myvariable maybe assign to myvariable # myvariable is well-defined here # myvariable is not well-defined here If you still desperately want lastprivate behaviour you can simply assign myvariable to another variable in the loop body. I don't care about lastprivate, I don't think that is an issue, as you say. My problem with this is that it means going into an area where possibly tricky things are implicit rather than explicit. I also see this as a rather special case that will be seldomly used, and implicit behaviour is more difficult to justify because of that. Indeed, I actually considered if we should support firstprivate at all, as it's really about "being firstprivate and lastprivate". Without any declaration, you can have firstprivate or lastprivate, but not both :) So I agree that supporting such a (probably) uncommon case is better left explicit. On the other hand it seems silly to have support for such a weird case. Well, I actually need to do the per-thread cache thing I described in the CEP in my own codes, so it's not *that* special; it'd be nice to support it. You need 'old_ell' and 'alpha' after the loop? No...but I need the values to not be blanked out at the beginning of each loop iteration! Sorry, I now realize that re-reading your email I may have misunderstood you. Anyway, no, I don't need lastprivate at all anywhere. Dag Sverre Note that in the CEP, the implicitly thread-local variables are *not available* before the first assignment in the loop. That is, code such as this is NOT allowed: cdef double x ... for i in prange(10): print x x = f(x) We raise a compiler error in such cases if we can: The code above is violating the contract that the order of execution of loop bodies should not matter. In cases where we can't raise an error (because we didn't bother or because it is not possible with a proof), we still initialize the variables to invalid values (NaN for double) at the beginning of the for-loop just to be sure the contract is satisfied. This was added to answer Stefan's objection to new types of implicit scopes (and I agree with his concern). Dag Sverre ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] prange CEP updated
On 11 April 2011 13:03, Dag Sverre Seljebotn wrote: > On 04/11/2011 01:02 PM, Dag Sverre Seljebotn wrote: >> >> On 04/11/2011 12:14 PM, mark florisson wrote: >>> >>> On 11 April 2011 12:08, Dag Sverre >>> Seljebotn wrote: On 04/11/2011 11:41 AM, mark florisson wrote: > > On 11 April 2011 11:10, Dag Sverre > Seljebotn > wrote: >> >> On 04/11/2011 10:45 AM, mark florisson wrote: >>> >>> On 5 April 2011 22:29, Dag Sverre >>> Seljebotn >>> wrote: I've done a pretty major revision to the prange CEP, bringing in a lot of the feedback. Thread-private variables are now split in two cases: i) The safe cases, which really require very little technical knowledge -> automatically inferred ii) As an advanced feature, unsafe cases that requires some knowledge of threading -> must be explicitly declared I think this split simplifies things a great deal. >>> >>> Can't we obsolete the declaration entirely by assigning to variables >>> that need to have firstprivate behaviour inside the with parallel >>> block? Basically in the same way the scratch space is used. The only >>> problem with that is that it won't be lastprivate, so the value will >>> be undefined after the parallel block (but not after the worksharing >>> loop). >>> >>> cdef int myvariable >>> >>> with nogil, parallel: >>> myvariable = 2 >>> for i in prange(...): >>> use myvariable >>> maybe assign to myvariable >>> >>> # myvariable is well-defined here >>> >>> # myvariable is not well-defined here >>> >>> If you still desperately want lastprivate behaviour you can simply >>> assign myvariable to another variable in the loop body. >> >> I don't care about lastprivate, I don't think that is an issue, as you >> say. >> >> My problem with this is that it means going into an area where >> possibly >> tricky things are implicit rather than explicit. I also see this as a >> rather >> special case that will be seldomly used, and implicit behaviour is >> more >> difficult to justify because of that. > > Indeed, I actually considered if we should support firstprivate at > all, as it's really about "being firstprivate and lastprivate". > Without any declaration, you can have firstprivate or lastprivate, but > not both :) So I agree that supporting such a (probably) uncommon case > is better left explicit. On the other hand it seems silly to have > support for such a weird case. Well, I actually need to do the per-thread cache thing I described in the CEP in my own codes, so it's not *that* special; it'd be nice to support it. >>> >>> You need 'old_ell' and 'alpha' after the loop? >> >> >> No...but I need the values to not be blanked out at the beginning of >> each loop iteration! > > Sorry, I now realize that re-reading your email I may have misunderstood > you. Anyway, no, I don't need lastprivate at all anywhere. Right, so basically you can rewrite your example by introducing the parallel block (which doesn't add an indentation level as you're already using nogil) and assigning to your variables that need to be firstprivate there. The only thing you miss out on is lastprivate behaviour. So basically, the question is, do we want explicit syntax for such a rare case (firstprivate + lastprivate)? I must say, I found your previous argument of future shared declarations persuasive enough to introduce explicit syntax. > Dag Sverre > >> >> Note that in the CEP, the implicitly thread-local variables are *not >> available* before the first assignment in the loop. That is, code such >> as this is NOT allowed: >> >> cdef double x >> ... >> for i in prange(10): >> print x >> x = f(x) >> >> We raise a compiler error in such cases if we can: The code above is >> violating the contract that the order of execution of loop bodies should >> not matter. >> >> In cases where we can't raise an error (because we didn't bother or >> because it is not possible with a proof), we still initialize the >> variables to invalid values (NaN for double) at the beginning of the >> for-loop just to be sure the contract is satisfied. >> >> This was added to answer Stefan's objection to new types of implicit >> scopes (and I agree with his concern). >> >> Dag Sverre > > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] prange CEP updated
On 04/11/2011 01:12 PM, mark florisson wrote: On 11 April 2011 13:03, Dag Sverre Seljebotn wrote: On 04/11/2011 01:02 PM, Dag Sverre Seljebotn wrote: On 04/11/2011 12:14 PM, mark florisson wrote: On 11 April 2011 12:08, Dag Sverre Seljebotn wrote: On 04/11/2011 11:41 AM, mark florisson wrote: On 11 April 2011 11:10, Dag Sverre Seljebotn wrote: On 04/11/2011 10:45 AM, mark florisson wrote: On 5 April 2011 22:29, Dag Sverre Seljebotn wrote: I've done a pretty major revision to the prange CEP, bringing in a lot of the feedback. Thread-private variables are now split in two cases: i) The safe cases, which really require very little technical knowledge -> automatically inferred ii) As an advanced feature, unsafe cases that requires some knowledge of threading -> must be explicitly declared I think this split simplifies things a great deal. Can't we obsolete the declaration entirely by assigning to variables that need to have firstprivate behaviour inside the with parallel block? Basically in the same way the scratch space is used. The only problem with that is that it won't be lastprivate, so the value will be undefined after the parallel block (but not after the worksharing loop). cdef int myvariable with nogil, parallel: myvariable = 2 for i in prange(...): use myvariable maybe assign to myvariable # myvariable is well-defined here # myvariable is not well-defined here If you still desperately want lastprivate behaviour you can simply assign myvariable to another variable in the loop body. I don't care about lastprivate, I don't think that is an issue, as you say. My problem with this is that it means going into an area where possibly tricky things are implicit rather than explicit. I also see this as a rather special case that will be seldomly used, and implicit behaviour is more difficult to justify because of that. Indeed, I actually considered if we should support firstprivate at all, as it's really about "being firstprivate and lastprivate". Without any declaration, you can have firstprivate or lastprivate, but not both :) So I agree that supporting such a (probably) uncommon case is better left explicit. On the other hand it seems silly to have support for such a weird case. Well, I actually need to do the per-thread cache thing I described in the CEP in my own codes, so it's not *that* special; it'd be nice to support it. You need 'old_ell' and 'alpha' after the loop? No...but I need the values to not be blanked out at the beginning of each loop iteration! Sorry, I now realize that re-reading your email I may have misunderstood you. Anyway, no, I don't need lastprivate at all anywhere. Right, so basically you can rewrite your example by introducing the parallel block (which doesn't add an indentation level as you're already using nogil) and assigning to your variables that need to be firstprivate there. The only thing you miss out on is lastprivate behaviour. So basically, the question is, do we want explicit syntax for such a rare case (firstprivate + lastprivate)? OK, we're on the same page here. I must say, I found your previous argument of future shared declarations persuasive enough to introduce explicit syntax. OK, lets leave it at this then, we don't have to agree for the same reasons :-) Dag Sverre ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] speed.pypy.org
Hi, I'm currently discussing with Maciej Fijalkowski (PyPy) how to get Cython running on speed.pypy.org (that's what I wrote "cythonrun" for). If it works out well, we may have it up in a couple of days. I would expect that Cython won't be a big winner in this game, given that it will only compile plain untyped Python code. It's also going to fail entirely in some of the benchmarks. But I think it's worth having it up there, simply as a way for us to see where we are performance-wise and to get quick (nightly) feed-back about optimisations we try. The benchmark suite is also a nice set of real-world Python code that will allow us to find compliance issues. Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] speed.pypy.org
2011/4/11 Stefan Behnel : > Hi, > > I'm currently discussing with Maciej Fijalkowski (PyPy) how to get Cython > running on speed.pypy.org (that's what I wrote "cythonrun" for). If it works > out well, we may have it up in a couple of days. > > I would expect that Cython won't be a big winner in this game, given that it > will only compile plain untyped Python code. It's also going to fail > entirely in some of the benchmarks. But I think it's worth having it up > there, simply as a way for us to see where we are performance-wise and to > get quick (nightly) feed-back about optimisations we try. The benchmark > suite is also a nice set of real-world Python code that will allow us to > find compliance issues. > > Stefan Cool, that would be nice! -- vitja. ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] "Cython's Users Guide"
2011/4/11 William Stein > Hi, > > I'm teaching Cython in my Sage course yet again, and noticed that > again there are some very confusing aspects of the Cython > documentation organization, which could probably be improved by a few > simple changes. > > 1. At http://cython.org/ there is a big link in the middle of the > page labeled "Cython Users Guide" which goes to > http://docs.cython.org/. However, http://docs.cython.org/ is *not* > the users guide -- it is "Cython’s Documentation". In fact, the > Users Guide is Chapter 3 of the documentation. > > 2. Looking at http://docs.cython.org, we see that Chapter 2 is > "Tutorials". But then looking down to Chapter 3 we see that it is > "Cython Users Guide". Of course, that's what one is after having just > clicked a link called "Cython Users Guide". So we click on "Cython > Users Guide" again. > > 3. We arrive at a page that again has "Tutorial" as Chapter 2. For > some reason this makes me feel even more confused. > > Recommend changes: > > 1. Change the link on the main page from "Cython Users Guide" to > "Documentation" or put a direct link into the Users Guide, or have > two links. > > 2. At http://docs.cython.org/ rename the "Cython Users Guide" to > "Users Guide", since it is obviously the Cython Users Guide at this > point and "Cython documentation" is in the upper left of the page > everywhere. > > 3. Possibly rename the tutorial in chapter 2 of the users guide to > something like "First Steps" or "Basic Tutorial" or something. > Yeah, that's something that we discussed in the past workshop in Munich (BTW, many thanks for providing the means for making this happen!). The basic idea is to completely remove the Chapter 3 (Cython Users Guide) by moving its parts to either Chapter 2 (Tutorials), or either to Chapter 4 (Reference Guide). During the meeting we agreed that the doc repository should be moved (and has been moved indeed) into the source repo, so that modifications that affect to code and docs can be put in the same commit/branch. Also, the wiki has a lot of information that can be better consolidated and integrated into the User's Guide. In fact, I already started some job in this direction and created a couple of pull requests during the workshop (that they have been already integrated). I plan to continue this job, but unfortunately I'm pretty busy lately, so I don't think I can progress a lot in the next weeks, so if anybody is interested in joining the effort for improving Cython's documentation, she will be very welcome indeed! -- Francesc Alted ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] GSoC Proposal - Reimplement C modules in CPython's standard library in Cython.
Arthur de Souza Ribeiro, 08.04.2011 02:43: 2011/4/7 Robert Bradshaw What I'd like to see is an implementation of a single simple but not entirely trivial (e.g. not math) module, passing regression tests with comprable if not better speed than the current C version (though I think it'd probably make sense to start out with the Python version and optimize that). E.g. http://docs.python.org/library/json.html looks like a good candidate. That should only take 8 hours or so, maybe two days at most, given your background. I'm not expecting anything before the application deadline, but if you could whip something like this out in the next week to point to that would help your application out immensely. In fact, one of the Python foundation's requirements is that students submit a patch before being accepted, and this would knock out that requirement and give you a chance to prove yourself. Create an account on https://github.com and commit your code into a new repository there. I will start the implementation of json module right now. I created my github account and as soon as I have code implemented I will send repository link. Any news on this? We're currently discussing which of the Cython-related projects to mentor. It's likely that not all of our projects will get accepted, so if you could get us a good initial idea about your work, we'd have a stronger incentive to value yours over the others. Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
