Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread Paul Anton Letnes
In the language wars, I have one question. Why is Fortran not being considered? Fortran already implements many of the features that we want in NumPy: - slicing and similar operations, at least some of the fancy indexing kind - element-wise array operations and function calls - array bounds-checki

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread Stéfan van der Walt
On Feb 19, 2012 4:14 PM, "Sturla Molden" wrote: > > Den 20.02.2012 00:39, skrev Nathaniel Smith: > > But there's an order-of-magnitude difference in compile times between > > most real-world C projects and most real-world C++ projects. It might > > not be a deal-breaker and it might not apply for

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread Sturla Molden
Den 20.02.2012 00:39, skrev Nathaniel Smith: > But there's an order-of-magnitude difference in compile times between > most real-world C projects and most real-world C++ projects. It might > not be a deal-breaker and it might not apply for subset of C++ you're > planning to use, but AFAICT that'

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread Charles R Harris
On Sun, Feb 19, 2012 at 4:42 PM, Nathaniel Smith wrote: > On Sun, Feb 19, 2012 at 1:42 PM, Neal Becker wrote: > > On Fedora linux I use ccache, which is completely transparant and makes > a huge > > difference in build times. > > ccache is fabulous (and it's fabulous for C too), but it only help

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread Nathaniel Smith
On Sun, Feb 19, 2012 at 1:42 PM, Neal Becker wrote: > On Fedora linux I use ccache, which is completely transparant and makes a huge > difference in build times. ccache is fabulous (and it's fabulous for C too), but it only helps when 'make' has screwed up and decided to rebuild some file that di

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread Nathaniel Smith
On Sun, Feb 19, 2012 at 7:13 PM, Mark Wiebe wrote: > On Sun, Feb 19, 2012 at 5:25 AM, Nathaniel Smith wrote: >> Precompiled headers can help some, but require complex and highly >> non-portable build-system support. (E.g., gcc's precompiled header >> constraints are here: >> http://gcc.gnu.org/on

Re: [Numpy-discussion] Scipy Cython refactor

2012-02-19 Thread Mark Wiebe
On Sun, Feb 19, 2012 at 7:35 AM, Pauli Virtanen wrote: > 19.02.2012 05:38, Travis Oliphant kirjoitti: > [clip] > >>> Sure. This list actually deserves a long writeup about that. > >>> First, there wasn't a "Cython-refactor" of NumPy. There was a > >>> Cython-refactor of SciPy. I'm not sure o

Re: [Numpy-discussion] NumPy in PyPy ?

2012-02-19 Thread xavier.gn...@gmail.com
I'm trying to promote the usage of python and scientific python modules at work. I fully agree with the fact that numpy is only the entrance point to scientific python. Without at least scipy and matplotlib, it is hopeless to forget about matlab. Speed : In my usecases, numpy is decently fast.

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread Mark Wiebe
On Sun, Feb 19, 2012 at 4:03 AM, David Cournapeau wrote: > On Sun, Feb 19, 2012 at 9:28 AM, Mark Wiebe wrote: > > > Is there anyone who uses a blue gene or small device which needs > up-to-date > > numpy support, that I could talk to directly? We really need a list of > > supported platforms on t

Re: [Numpy-discussion] Scipy Cython refactor

2012-02-19 Thread Matthew Brett
Hi, On Sun, Feb 19, 2012 at 7:35 AM, Pauli Virtanen wrote: > 19.02.2012 05:38, Travis Oliphant kirjoitti: > [clip] Sure.  This list actually deserves a long writeup about that. First, there wasn't a "Cython-refactor" of NumPy.   There was a Cython-refactor of SciPy.   I'm not sure

Re: [Numpy-discussion] NumPy in PyPy ?

2012-02-19 Thread Travis Oliphant
I have written up a summary of my views here: http://technicaldiscovery.blogspot.com/2011/10/thoughts-on-porting-numpy-to-pypy.html -Travis On Feb 19, 2012, at 9:45 AM, xavier.gn...@gmail.com wrote: > Hi, > > I'm trying to understand what's going on with : > http://morepypy.blogspot.com/2012

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread Mark Wiebe
On Sun, Feb 19, 2012 at 5:25 AM, Nathaniel Smith wrote: > On Sun, Feb 19, 2012 at 9:16 AM, David Cournapeau > wrote: > > On Sun, Feb 19, 2012 at 8:08 AM, Mark Wiebe wrote: > >> Is there a specific > >> target platform/compiler combination you're thinking of where we can do > >> tests on this? I

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread Nathaniel Smith
On Sun, Feb 19, 2012 at 4:13 PM, xavier.gn...@gmail.com wrote: > I'm no sure. If you want to be able to write A=B+C+D; with decent > performances, I think you have to use a lib based on expression templates. > It would be great if C++ compilers could automatically optimize out > spurious copies in

Re: [Numpy-discussion] How a transition to C++ could work

2012-02-19 Thread Stéfan van der Walt
On Feb 19, 2012 2:41 AM, "Mark Wiebe" wrote: > > This is the role I see good coding standards and consistent code review playing. Programmers who don't know how to write good C++ code can be taught. There are also good books to read, like "C++ Coding Standards," "Effective C++", and others that ca

Re: [Numpy-discussion] How a transition to C++ could work

2012-02-19 Thread Ralf Gommers
On Sun, Feb 19, 2012 at 4:53 PM, Sturla Molden wrote: > Den 19.02.2012 16:45, skrev Adam Klein: > > > > Just to add, with respect to acceptable compilation times, a judicious > > choice of C++ features is critical. > > > > I use Python to avoid recompiling my code all the time. I don't > recompil

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread xavier.gn...@gmail.com
On 02/19/2012 04:48 PM, Sturla Molden wrote: > Den 19.02.2012 10:28, skrev Mark Wiebe: >> Particular styles of using templates can cause this, yes. To properly >> do this kind of advanced C++ library work, it's important to think >> about the big-O notation behavior of your template instantiations,

Re: [Numpy-discussion] How a transition to C++ could work

2012-02-19 Thread Sturla Molden
Den 19.02.2012 16:45, skrev Adam Klein: > > Just to add, with respect to acceptable compilation times, a judicious > choice of C++ features is critical. > I use Python to avoid recompiling my code all the time. I don't recompile NumPy every time I use it. (I know you are thinking about developm

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread Sturla Molden
Den 19.02.2012 10:28, skrev Mark Wiebe: > > Particular styles of using templates can cause this, yes. To properly > do this kind of advanced C++ library work, it's important to think > about the big-O notation behavior of your template instantiations, not > just the big-O notation of run-time. C

Re: [Numpy-discussion] How a transition to C++ could work

2012-02-19 Thread Adam Klein
On Feb 19, 2012, at 10:38 AM, Sturla Molden wrote: Den 19.02.2012 10:52, skrev Mark Wiebe: C++ removes some of this advantage -- now there is extra code generated by > the compiler to handle constructors, destructors, operators etc which can > make a material difference to fast inner loops. S

[Numpy-discussion] NumPy in PyPy ?

2012-02-19 Thread xavier.gn...@gmail.com
Hi, I'm trying to understand what's going on with : http://morepypy.blogspot.com/2012/01/numpypy-status-update.html What's your opinion on such a numpy rewrite?? Thanks, Xavier ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.sc

Re: [Numpy-discussion] How a transition to C++ could work

2012-02-19 Thread Sturla Molden
Den 19.02.2012 10:52, skrev Mark Wiebe: C++ removes some of this advantage -- now there is extra code generated by the compiler to handle constructors, destructors, operators etc which can make a material difference to fast inner loops. So you end up just writing "C-s

[Numpy-discussion] Scipy Cython refactor

2012-02-19 Thread Pauli Virtanen
19.02.2012 05:38, Travis Oliphant kirjoitti: [clip] >>> Sure. This list actually deserves a long writeup about that. >>> First, there wasn't a "Cython-refactor" of NumPy. There was a >>> Cython-refactor of SciPy. I'm not sure of it's current status. >>> I'm still very supportive of that so

Re: [Numpy-discussion] How a transition to C++ could work

2012-02-19 Thread Sturla Molden
Den 19.02.2012 11:30, skrev Christopher Jordan-Squire: > > Can this possibly be extended to the following: How will Mark's > (extensive) experience about performance and long-term consequences of > design decisions be communicated to future developers? We not only > want new numpy developers, we wa

Re: [Numpy-discussion] How a transition to C++ could work

2012-02-19 Thread Adam Klein
On Feb 19, 2012, at 2:18 AM, Mark Wiebe wrote: The suggestion of transitioning the NumPy core code from C to C++ has sparked a vigorous debate, and I thought I'd start a new thread to give my perspective on some of the issues raised, and describe how such a transition could occur. First, I'd lik

Re: [Numpy-discussion] How a transition to C++ could work

2012-02-19 Thread Sturla Molden
Den 19. feb. 2012 kl. 09:51 skrev Stéfan van der Walt : > > OK, so let's talk specifics: how do you dynamically grab a function pointer > to a compiled C++ library, a la ctypes? Feel free to point me to > StackOverflow or elsewhere. > You declare the function with the signature extern "C".

[Numpy-discussion] Flag this message Re: [Matplotlib-users] Discussion with Guido van Rossum and (hopefully) core python-dev on scientific Python and Python3

2012-02-19 Thread Michiel de Hoon
> While a number of scientific Python packages are already available for > Python 3 (either in released form or in their master git branches), > it's fair to say that there hasn't been a major transition of the > scientific community to Python3. Since there is no more development > being done on t

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread Neal Becker
Sturla Molden wrote: > > Den 18. feb. 2012 kl. 01:58 skrev Charles R Harris > : > >> >> >> On Fri, Feb 17, 2012 at 4:44 PM, David Cournapeau wrote: >> I don't think c++ has any significant advantage over c for high performance >> libraries. I am not convinced by the number of people argument

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread Neal Becker
Nathaniel Smith wrote: > On Sun, Feb 19, 2012 at 9:16 AM, David Cournapeau wrote: >> On Sun, Feb 19, 2012 at 8:08 AM, Mark Wiebe wrote: >>> Is there a specific >>> target platform/compiler combination you're thinking of where we can do >>> tests on this? I don't believe the compile times are as

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread Neal Becker
Sturla Molden wrote: > Den 19.02.2012 01:12, skrev Nathaniel Smith: >> >> I don't oppose it, but I admit I'm not really clear on what the >> supposed advantages would be. Everyone seems to agree that >>-- Only a carefully-chosen subset of C++ features should be used >>-- But this subset wo

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread Nathaniel Smith
On Sun, Feb 19, 2012 at 9:16 AM, David Cournapeau wrote: > On Sun, Feb 19, 2012 at 8:08 AM, Mark Wiebe wrote: >> Is there a specific >> target platform/compiler combination you're thinking of where we can do >> tests on this? I don't believe the compile times are as bad as many people >> suspect,

Re: [Numpy-discussion] How a transition to C++ could work

2012-02-19 Thread Mark Wiebe
On Sun, Feb 19, 2012 at 4:30 AM, Christopher Jordan-Squire wrote: > On Sun, Feb 19, 2012 at 2:14 AM, David Cournapeau > wrote: > > On Sun, Feb 19, 2012 at 9:52 AM, Mark Wiebe wrote: > >> On Sun, Feb 19, 2012 at 3:10 AM, Ben Walsh > wrote: > >>> > >>> > >>> > >>> > Date: Sun, 19 Feb 2012 01:18:2

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread David Warde-Farley
On 2012-02-19, at 12:47 AM, Benjamin Root wrote: > Dude, have you seen the .c files in numpy/core? They are already read-only > for pretty much everybody but Mark. I've managed to patch several of them without incident, and I do not do a lot of programming in C. It could be simpler, but it's no

Re: [Numpy-discussion] How a transition to C++ could work

2012-02-19 Thread Mark Wiebe
On Sun, Feb 19, 2012 at 4:14 AM, David Cournapeau wrote: > On Sun, Feb 19, 2012 at 9:52 AM, Mark Wiebe wrote: > > On Sun, Feb 19, 2012 at 3:10 AM, Ben Walsh > wrote: > >> > >> > >> > >> > Date: Sun, 19 Feb 2012 01:18:20 -0600 > >> > From: Mark Wiebe > >> > Subject: [Numpy-discussion] How a tran

Re: [Numpy-discussion] How a transition to C++ could work

2012-02-19 Thread Christopher Jordan-Squire
On Sun, Feb 19, 2012 at 2:14 AM, David Cournapeau wrote: > On Sun, Feb 19, 2012 at 9:52 AM, Mark Wiebe wrote: >> On Sun, Feb 19, 2012 at 3:10 AM, Ben Walsh wrote: >>> >>> >>> >>> > Date: Sun, 19 Feb 2012 01:18:20 -0600 >>> > From: Mark Wiebe >>> > Subject: [Numpy-discussion] How a transition to

Re: [Numpy-discussion] How a transition to C++ could work

2012-02-19 Thread Mark Wiebe
On Sun, Feb 19, 2012 at 3:45 AM, David Cournapeau wrote: > On Sun, Feb 19, 2012 at 9:19 AM, Mark Wiebe wrote: > > On Sun, Feb 19, 2012 at 2:56 AM, David Cournapeau > > wrote: > >> > >> Hi Mark, > >> > >> thank you for joining this discussion. > >> > >> On Sun, Feb 19, 2012 at 7:18 AM, Mark Wiebe

Re: [Numpy-discussion] How a transition to C++ could work

2012-02-19 Thread David Cournapeau
On Sun, Feb 19, 2012 at 9:52 AM, Mark Wiebe wrote: > On Sun, Feb 19, 2012 at 3:10 AM, Ben Walsh wrote: >> >> >> >> > Date: Sun, 19 Feb 2012 01:18:20 -0600 >> > From: Mark Wiebe >> > Subject: [Numpy-discussion] How a transition to C++ could work >> > To: Discussion of Numerical Python >> > Messa

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread Ralf Gommers
On Sun, Feb 19, 2012 at 10:28 AM, Mark Wiebe wrote: > On Sun, Feb 19, 2012 at 3:16 AM, David Cournapeau wrote: > >> On Sun, Feb 19, 2012 at 8:08 AM, Mark Wiebe wrote: >> > On Sat, Feb 18, 2012 at 4:24 PM, David Cournapeau >> > wrote: >> >> >> >> On Sat, Feb 18, 2012 at 9:40 PM, Charles R Harris

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread David Cournapeau
On Sun, Feb 19, 2012 at 9:28 AM, Mark Wiebe wrote: > Is there anyone who uses a blue gene or small device which needs up-to-date > numpy support, that I could talk to directly? We really need a list of > supported platforms on the numpy wiki we can refer to when discussing this > stuff, it all se

Re: [Numpy-discussion] How a transition to C++ could work

2012-02-19 Thread Mark Wiebe
On Sun, Feb 19, 2012 at 3:10 AM, Ben Walsh wrote: > > > > Date: Sun, 19 Feb 2012 01:18:20 -0600 > > From: Mark Wiebe > > Subject: [Numpy-discussion] How a transition to C++ could work > > To: Discussion of Numerical Python > > Message-ID: > >kdurpzktgui516oqtqd4vazm746hmpqgpfx...@mail.g

Re: [Numpy-discussion] How a transition to C++ could work

2012-02-19 Thread David Cournapeau
On Sun, Feb 19, 2012 at 9:19 AM, Mark Wiebe wrote: > On Sun, Feb 19, 2012 at 2:56 AM, David Cournapeau > wrote: >> >> Hi Mark, >> >> thank you for joining this discussion. >> >> On Sun, Feb 19, 2012 at 7:18 AM, Mark Wiebe wrote: >> > The suggestion of transitioning the NumPy core code from C to

Re: [Numpy-discussion] How a transition to C++ could work

2012-02-19 Thread Mark Wiebe
On Sun, Feb 19, 2012 at 3:10 AM, Matthew Brett wrote: > > As you probably saw, I think the proposal was indeed to use Cython to > provide the higher-level parts of the core, while refactoring the rest > of the C code underneath it. Obviously one could also refactor the C > into C++, so the propo

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread Mark Wiebe
On Sun, Feb 19, 2012 at 3:16 AM, David Cournapeau wrote: > On Sun, Feb 19, 2012 at 8:08 AM, Mark Wiebe wrote: > > On Sat, Feb 18, 2012 at 4:24 PM, David Cournapeau > > wrote: > >> > >> On Sat, Feb 18, 2012 at 9:40 PM, Charles R Harris > >> wrote: > >> > >> > > >> > Well, we already have code ob

Re: [Numpy-discussion] How a transition to C++ could work

2012-02-19 Thread Mark Wiebe
On Sun, Feb 19, 2012 at 2:56 AM, David Cournapeau wrote: > Hi Mark, > > thank you for joining this discussion. > > On Sun, Feb 19, 2012 at 7:18 AM, Mark Wiebe wrote: > > The suggestion of transitioning the NumPy core code from C to C++ has > > sparked a vigorous debate, and I thought I'd start a

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread David Cournapeau
On Sun, Feb 19, 2012 at 8:08 AM, Mark Wiebe wrote: > On Sat, Feb 18, 2012 at 4:24 PM, David Cournapeau > wrote: >> >> On Sat, Feb 18, 2012 at 9:40 PM, Charles R Harris >> wrote: >> >> > >> > Well, we already have code obfuscation (DOUBLE_your_pleasure, >> > FLOAT_your_boat), so we might as well

Re: [Numpy-discussion] How a transition to C++ could work

2012-02-19 Thread Ben Walsh
> Date: Sun, 19 Feb 2012 01:18:20 -0600 > From: Mark Wiebe > Subject: [Numpy-discussion] How a transition to C++ could work > To: Discussion of Numerical Python > Message-ID: > > Content-Type: text/plain; charset="utf-8" > > The suggestion of transitioning the NumPy core code from C to C

Re: [Numpy-discussion] How a transition to C++ could work

2012-02-19 Thread Matthew Brett
Hi, On Sun, Feb 19, 2012 at 12:49 AM, Mark Wiebe wrote: > On Sun, Feb 19, 2012 at 2:32 AM, Matthew Brett > wrote: >> >> Hi, >> >> Thanks for this - it's very helpful. >> >> On Sat, Feb 18, 2012 at 11:18 PM, Mark Wiebe wrote: >> > The suggestion of transitioning the NumPy core code from C to C++

Re: [Numpy-discussion] How a transition to C++ could work

2012-02-19 Thread Mark Wiebe
On Sun, Feb 19, 2012 at 2:51 AM, Stéfan van der Walt wrote: > > On Feb 19, 2012 12:34 AM, "Mark Wiebe" wrote: > > > > I'm speaking from personal experience having dealt with these types of > issues extensively before. If people have more detailed examples of > problems, possibly links to discussi

Re: [Numpy-discussion] How a transition to C++ could work

2012-02-19 Thread David Cournapeau
Hi Mark, thank you for joining this discussion. On Sun, Feb 19, 2012 at 7:18 AM, Mark Wiebe wrote: > The suggestion of transitioning the NumPy core code from C to C++ has > sparked a vigorous debate, and I thought I'd start a new thread to give my > perspective on some of the issues raised, and

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread Stéfan van der Walt
On Feb 19, 2012 12:09 AM, "Mark Wiebe" wrote: > > These standard library issues were definitely valid 10 years ago, but all the major C++ compilers have great C++98 support now. Is there a specific target platform/compiler combination you're thinking of where we can do tests on this? I don't belie

Re: [Numpy-discussion] How a transition to C++ could work

2012-02-19 Thread Stéfan van der Walt
On Feb 19, 2012 12:34 AM, "Mark Wiebe" wrote: > > I'm speaking from personal experience having dealt with these types of issues extensively before. If people have more detailed examples of problems, possibly links to discussions where one of these problems has occurred, that would be helpful. This

Re: [Numpy-discussion] How a transition to C++ could work

2012-02-19 Thread Mark Wiebe
On Sun, Feb 19, 2012 at 2:32 AM, Matthew Brett wrote: > Hi, > > Thanks for this - it's very helpful. > > On Sat, Feb 18, 2012 at 11:18 PM, Mark Wiebe wrote: > > The suggestion of transitioning the NumPy core code from C to C++ has > > sparked a vigorous debate, and I thought I'd start a new threa

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread Ralf Gommers
On Sun, Feb 19, 2012 at 6:47 AM, Benjamin Root wrote: > > All kidding aside, is your concern that when Mark starts this that no one > will be able to contribute until he is done? I can tell you right now that > won't be the case as I will be trying to flesh out issues with datetime64 > with him.

Re: [Numpy-discussion] change the mask state of one element in a masked array

2012-02-19 Thread Chao YUE
thanks. 2012/2/18 Eric Firing > On 02/18/2012 05:52 AM, Chao YUE wrote: > > Dear all, > > > > I built a new empty masked array: > > > > In [91]: a=np.ma.empty((2,5)) > > Of course this only makes sense if you are going to immediately populate > the array. > > > > > In [92]: a > > Out[92]: > > ma

Re: [Numpy-discussion] How a transition to C++ could work

2012-02-19 Thread Mark Wiebe
On Sun, Feb 19, 2012 at 2:24 AM, Stéfan van der Walt wrote: > Hey, Mark > > On Feb 18, 2012 11:18 PM, "Mark Wiebe" wrote: > > My experience has been that providing a C API from a C++ library is no > harder than providing a C API from a C library. > > Interfacing to compiled C++ libs have been tri

Re: [Numpy-discussion] How a transition to C++ could work

2012-02-19 Thread Matthew Brett
Hi, Thanks for this - it's very helpful. On Sat, Feb 18, 2012 at 11:18 PM, Mark Wiebe wrote: > The suggestion of transitioning the NumPy core code from C to C++ has > sparked a vigorous debate, and I thought I'd start a new thread to give my > perspective on some of the issues raised, and descri

Re: [Numpy-discussion] How a transition to C++ could work

2012-02-19 Thread Stéfan van der Walt
Hey, Mark On Feb 18, 2012 11:18 PM, "Mark Wiebe" wrote: > My experience has been that providing a C API from a C++ library is no harder than providing a C API from a C library. Interfacing to compiled C++ libs have been tricky, so can this concern be dismissed so easily? (Some examples that came

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread Mark Wiebe
On Sat, Feb 18, 2012 at 4:24 PM, David Cournapeau wrote: > On Sat, Feb 18, 2012 at 9:40 PM, Charles R Harris > wrote: > > > > > Well, we already have code obfuscation (DOUBLE_your_pleasure, > > FLOAT_your_boat), so we might as well let the compiler handle it. > > Yes, those are not great, but on