Re: [Cython] OpenMP thread private variable not recognized (bug report + discussion)

2014-08-12 Thread Sturla Molden
Cython does not do an error here:

- i is recognized as private
- r is recognized as reduction
- w is (correctly) recognized as shared

If you need thread local storage, use threading.local()

I agree that scoped cdefs would be an advantage.

Personally I prefer to avoid OpenMP and just use Python threads and an
internal function (closure) or an internal class. If you start to use
OpenMP, Apple's libdispatch ("GCD"), Intel TBB, or Intel clikplus, you will
soon discover that they are all variations over the same theme: a thread
pool and a closure. Whether you call it a parallel block in OpenMP or an
anonymous block in GCD, it is fundamentally a closure. That's all there is.
You can easily do this with Python threads: Python, unlike C, supports
closures or internal classes directly in the language, and does not need
special extensions like C. Python threads and OpenMP threads will scale
equally well (they are all native OS threads, scheduled in the same way),
and there will be no scoping problems. The sooner you discover you do not
need Cython's prange, the less pain it will cause.

Sturla

___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel


Re: [Cython] OpenMP thread private variable not recognized (bug report + discussion)

2014-08-12 Thread Brett Calcott
On Tue, Aug 12, 2014 at 10:26 AM, Sturla Molden 
wrote:

> The sooner you discover you do not
> need Cython's prange, the less pain it will cause.
>
>
For someone who has bumbled around trying to use prange & openmp on the mac
(but successfully used python threading), this sounds great. Is there an
example of this somewhere that you can point us to?

Thanks
Brett
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel


Re: [Cython] aritmetic with arrays in Cython

2014-08-12 Thread Sturla Molden
But using Eigen will taint the output with Eigen's license, since the Eigen
library is statically linked. OTOH, Cilkplus is just a compiler extension
for C and C++. 

AFAIK, it is currently available for Intel C++ and Clang (also by Intel)
and GCC 4.9. On MSVC I believe it requires Intel Parallel Studio. The main
obstacle to its adoption today is MSVC and GCC 4.8. Cilkplus is also being
evaluated for becoming parts of the future C and C++ standards.

Another thing to observe is that Eigen depends on the C++ compiler to elide
temporary arrays. Currently it only/mostly happens for fixed size arrays.
Clikplus arrays does not suffer from this. You just index pointers like
typed memory views (though the indexing syntax is slightly different), and
it compiles to efficient machine code just like Fortran 90 array code. 

Sturla

Stefan Behnel  wrote:
> Ian Henriksen schrieb am 12.08.2014 um 04:34:
>> On Sun, Aug 10, 2014 at 12:41 PM, Sturla Molden wrote:
>>> Ian Henriksen wrote:
 Maybe I should clarify a little about why eigen is a good place to start.
 According to >>> href="http://eigen.tuxfamily.org/dox/TopicLazyEvaluation.html";>
>>> http://eigen.tuxfamily.org/dox/TopicLazyEvaluation.html
 it
 already takes care of things like the elimination of temporary variables
 and common subexpression reduction at compile time. This should make it
 possible to compile array expressions in Cython without having to
 re-implement those sorts of optimizations. Ideally we would just have to
 map memory view operations to corresponding equivalents from eigen. It's
 not yet clear to me how to do things with arbitrary-dimensional arrays or
 broadcasting, but, given some more time, a solution may present itself.
 -Ian
>>> 
>>> cilkplus is what you want, not Eigen.
>>> 
>>> But if you are serious about number crunching, learn Fortran 95.
>> 
>> Cilk Plus would also work really nicely for this. Thanks for the suggestion.
>> Fortran is a really great language for this sort of thing, but I don't
>> think I'm ready to tackle the difficulties of using it as a backend for
>> array arithmetic in Cython. It would be a really great feature to have
>> later on though.
> 
> That clarifies a bit of the design then: The syntax support should be
> somewhat generic, with specialised (sets of) node implementations as
> backends that generate code for different libraries/compilers/languages.
> 
> It's ok to start only with Eigen, though. We have working example code for
> it and everything else has either a much higher entry level for the
> implementation or a much lower general availability of the required tools.
> 
> For the syntax/type support, a look at the array expressions branch might
> also be helpful, although I doubt that there really is all that much to do
> on that front.
> 
> Stefan

___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel


Re: [Cython] OpenMP thread private variable not recognized (bug report + discussion)

2014-08-12 Thread Sturla Molden
Brett Calcott 
wrote:

> For someone who has bumbled around trying to use prange & openmp on the 
> mac
> (but successfully used python threading), this sounds great. Is there an
> example of this somewhere that you can point us to?

No, but I could make one :)

ipython notebook?


Sturla

___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel


Re: [Cython] OpenMP thread private variable not recognized (bug report + discussion)

2014-08-12 Thread Brett Calcott
That would be awesome (I love ipython notebook).


On Tue, Aug 12, 2014 at 11:20 AM, Sturla Molden 
wrote:

> Brett Calcott 
> wrote:
>
> > For someone who has bumbled around trying to use prange & openmp on
> the mac
> > (but successfully used python threading), this sounds great. Is there an
> > example of this somewhere that you can point us to?
>
> No, but I could make one :)
>
> ipython notebook?
>
>
> Sturla
>
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel


Re: [Cython] aritmetic with arrays in Cython

2014-08-12 Thread Stefan Behnel
Sturla Molden schrieb am 12.08.2014 um 17:18:
> But using Eigen will taint the output with Eigen's license

Which is ok for many users, definitely those who only run their own code
locally. And the others can eventually use a different backend.


> OTOH, Cilkplus is just a compiler extension for C and C++. 

Which isn't available in all compilers. But anyway, if you write the code,
I have no problem with Cilkplus becoming the first backend to support.

Stefan

___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel


[Cython] [Re] OpenMP thread private variable not recognized (bug report + discussion)

2014-08-12 Thread Leon Bottou
On Tue, 12 Aug 2014 14:26:31, Sturla Molden wrote:
> Cython does not do an error here:[...
> - i is recognized as private
> - r is recognized as reduction
> - w is (correctly) recognized as shared

Not according to the documentation.
http://docs.cython.org/src/userguide/parallelism.html documentation for
cython.parallel.parallel says "A contained prange will be a worksharing loop
that is not parallel, so any variable assigned to in the parallel section is
also private to the prange. Variables that are private in the parallel block
are unavailable after the parallel block.".  Variable w is such a variable.

Furthermore, if cython is correct, why does GCC report an error on the
cython generated C code?  

My point here is that there is a bug because (a) cython does not behave as
documented, and (b) it generates invalid C code despite not reporting an
error.

> Personally I prefer to avoid OpenMP and just use Python threads and an
> internal function (closure) or an internal class. If you start to use
OpenMP,
> Apple's libdispatch ("GCD"), Intel TBB, or Intel clikplus, you will soon
discover
> that they are all variations over the same theme: a thread pool and a
closure.

I am making heavy uses of OpenBlas which also uses OpenMP.
Using the same queue manager prevents lots of CPU provisioning problem.
Using multiple queue managers in the same code does not work as well because
they are not aware of what the other one is doing.


- L.




___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel


Re: [Cython] aritmetic with arrays in Cython

2014-08-12 Thread Matěj Laitl
On Tuesday 12 of August 2014 15:18:12 Sturla Molden wrote:
> But using Eigen will taint the output with Eigen's license, since the Eigen
> library is statically linked.

There is no such thing as "Eigen library". Eigen is fully implemented in 
header files. Cython would just generate C++ code that uses Eigen API.

Also, Eigen is now licensed under MPL2, which is file-level and permissive one 
[1]. I don't understand what you mean by 'taint' above and how this could 
restrict someone license-wise.

Regards,
Matěj

[1] http://eigen.tuxfamily.org/index.php?title=Licensing_FAQ
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel


Re: [Cython] aritmetic with arrays in Cython

2014-08-12 Thread Matěj Laitl
On Tuesday 12 of August 2014 15:18:12 Sturla Molden wrote:
> Another thing to observe is that Eigen depends on the C++ compiler to elide
> temporary arrays.

Either I don't understand you, or you don't understand Eigen. Eigen overloads 
operator=() to circumvent need for temporary arrays. It is *not* an 
optimisation on the compiler part, the code is explicitly written to do so, 
where it is beneficial for performance [1].

Matěj

[1] http://eigen.tuxfamily.org/dox/TopicLazyEvaluation.html
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel


[Cython] Cython 0.21 beta 1 released

2014-08-12 Thread Stefan Behnel
Hi all,

given the overwhelming lack of negative feedback on the latest alpha
release, and the recent changes that made it into the master branch, I
think it's time for a first beta release.

http://www.cython.org/release/Cython-0.21b1.tar.gz

http://www.cython.org/release/Cython-0.21b1.tar.gz.asc

This release features faster calls to C implemented Python functions and
methods in simple (non-kwargs) cases, so you might see a net win if your
code calls a lot into previously unoptimised builtins or other native
extension modules (including Cython modules). There's also a bit of a
speedup for Python method calls in general, in addition to the long list of
improvements that were already in the last alpha.

Complete changelog:

https://github.com/cython/cython/blob/ebafd54916e1fe976a282e77b4d59577c793bfca/CHANGES.rst

Please give it a try with your code. Any regressions that you find now have
a chance of being fixed before the final release. Positive success stories
are also appreciated.

Have fun,

Stefan

___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel


Re: [Cython] [Re] OpenMP thread private variable not recognized (bug report + discussion)

2014-08-12 Thread Sturla Molden
"Leon Bottou"  wrote:

> I am making heavy uses of OpenBlas which also uses OpenMP.
> Using the same queue manager prevents lots of CPU provisioning problem.
> Using multiple queue managers in the same code does not work as well because
> they are not aware of what the other one is doing.

Normally OpenBLAS is built without OpenMP. Also, OpenMP is not fork safe
(cf. multiprocessing) but OpenBLAS' own threadpool is. So it is recommended
to build OpenBLAS without OpenMP dependency.

That is: If you build OpenBLAS with OpenMP, numpy.dot will hang if used
together with multiprocessing.


Sturla

___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel


Re: [Cython] [Re] OpenMP thread private variable not recognized (bug report + discussion)

2014-08-12 Thread Dave Hirschfeld
Sturla Molden  writes:

> 
> "Leon Bottou"  wrote:
> 
> > I am making heavy uses of OpenBlas which also uses OpenMP.
> > Using the same queue manager prevents lots of CPU provisioning problem.
> > Using multiple queue managers in the same code does not work as well 
because
> > they are not aware of what the other one is doing.
> 
> Normally OpenBLAS is built without OpenMP. Also, OpenMP is not fork safe
> (cf. multiprocessing) but OpenBLAS' own threadpool is. So it is recommended
> to build OpenBLAS without OpenMP dependency.
> 
> That is: If you build OpenBLAS with OpenMP, numpy.dot will hang if used
> together with multiprocessing.
> 
> Sturla
> 
> 

Just wanting to clarify that it's only the GNU OpenMP implementation that 
isn't fork-safe? AFAIK the intel OpenMP runtime is and will at some stage be 
available in the master branch of clang.

-Dave

___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel