Re: [Numpy-discussion] Latest Array-Interface PEP

[Neal Becker]

Travis Oliphant wrote:

> 
> I'm attaching my latest extended buffer-protocol PEP that is trying to
> get the array interface into Python.  Basically, it is a translation of
> the numpy header files into something as simple as possible that can
> still be used to describe a complicated block of memory to another user.
> 
> My purpose is to get feedback and criticisms from this community before
> display before the larger Python community.
> 
> -Travis

I'm wondering if having the buffer object specify the view is the right
choice.  I think the best choice is to separate the design into:

buffer: provides an interface to memory
array: provides a view of memory as an array of whatever dimensions

1. buffer may or may not map to contiguous memory.
2. multiple views of the same memory can be shared.  These different views
could represent different slicings.

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Latest Array-Interface PEP

[Neal Becker]

Travis Oliphant wrote:

> Neal Becker wrote:
>> Travis Oliphant wrote:
>>
>>   
>>> I'm attaching my latest extended buffer-protocol PEP that is trying to
>>> get the array interface into Python.  Basically, it is a translation of
>>> the numpy header files into something as simple as possible that can
>>> still be used to describe a complicated block of memory to another user.
>>>
>>> My purpose is to get feedback and criticisms from this community before
>>> display before the larger Python community.
>>>
>>> -Travis
>>> 
>>
>> I'm wondering if having the buffer object specify the view is the right
>> choice.  I think the best choice is to separate the design into:
>>
>> buffer: provides an interface to memory
>> array: provides a view of memory as an array of whatever dimensions
>>
>> 1. buffer may or may not map to contiguous memory.
>> 2. multiple views of the same memory can be shared.  These different
>> views could represent different slicings.
>>   
> 
> I don't understand your concerns, could you please help clarify?
> 
> I'm not doing anything with the buffer object at all.  I'm only using
> the buffer "protocol" (i.e. extending the set of function pointers
> pointed to by tp_as_buffer in the type-object).
> 
> -Travis

Several extensions to Python utilize the buffer protocol to share
the location of a data-buffer that is really an N-dimensional
array.  However, there is no standard way to exchange the
additional N-dimensional array information so that the data-buffer
is interpreted correctly.
 
I am questioning if this is the best concept.  It says that the data-buffer
will carry the information about it's interpretation as an N-dimensional
array.

I'm thinking that a buffer is just an interface to memory, and that the
interpretation as an array of n-dimensions, for example, is best left to
the application.  I might want to at one time view the data as
n-dimensional, but at another time as 1-dimensional, for example.


___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Latest Array-Interface PEP

[Neal Becker]

I believe we are converging, and this is pretty much the same design as I
advocated.  It is similar to boost::ublas.

Storage is one concept.

Interpretation of the storage is another concept.

Numpy is a combination of a storage and interpretation.

Storage could be dense or sparse.  Allocated in various ways. Sparse can be
implemented in different ways.

Interpretation can be 1-d, 2-d.  Zero-based, non-zero based.  Also there is
question of ownership (slices).



___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Latest Array-Interface PEP

[Neal Becker]

Travis Oliphant wrote:

> Neal Becker wrote:
>> I believe we are converging, and this is pretty much the same design as I
>> advocated.  It is similar to boost::ublas.
>>   
> I'm grateful to hear that.   It is nice when ideas come from several
> different corners.
>> Storage is one concept.
>>
>> Interpretation of the storage is another concept.
>>
>> Numpy is a combination of a storage and interpretation.
>>
>> Storage could be dense or sparse.  Allocated in various ways. Sparse can
>> be implemented in different ways.
>>
>> Interpretation can be 1-d, 2-d.  Zero-based, non-zero based.  Also there
>> is question of ownership (slices).
>>   
> 
> How do we extend the buffer interface then?  Do we have one API that
> allows sharing of storage and another that handles sharing of
> interpretation?
> 
> How much detail should be in the interface regarding storage detail.
> Is there a possibility of having at least a few storage models
> "shareable" so that memory can be shared by others that view the data in
> the same way?
> 

How about:

1. A memory concept, of which buffer is an example.
2. A view concept.
3. A variety of common concrete types composing 1+2.

So then, how do we use buffer in this scheme?  I'm thinking that buffer
isn't really the best thing to build on - but within this scheme buffer is
a kind of memory (assuming it provides/could_be_made_to_provide the
required interface).  The view is not part of buffer, (as was proposed) but
a separate piece.

Still, I agree that we want a commonly used array object that includes both
the memory and the view.  I propose that we build it out of these more
generic pieces, but also provide commonly used compositions of these
pieces.  I think this satisfies the desire for a self-describing array
component, while allowing more flexibility and serving a wider usage.

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [matplotlib-devel] Unifying numpy, scipy, and matplotlib docstring formats

[Neal Becker]

I have never used matlab, but a lot of my colleagues do.  Can anyone give me
some good references that I could show them to explain the advantages of
python over matlab?

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] matlab vs. python question

[Neal Becker]

I'm interested in this comparison (not in starting yet another flame fest). 
I actually know nothing about matlab, but almost all my peers use it.  One
of the things I recall reading on this subject is that matlab doesn't
support OO style programming.  I happened to look on the matlab vendor's
website, and found that it does have classes.  OTOH, I've seen at least
some matlab code, and never saw anyone use these features.

I'm perfectly happy with python myself, but I wonder if anyone has good
arguments for why to prefer python over matlab?

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] matlab vs. python question

[Neal Becker]

Sturla Molden wrote:

> On 4/26/2007 2:19 PM, Steve Lianoglou wrote:
> 
>>> Beside proper programing paradigm Python easily scales to large-
>>> scale number crunching: You can run large-matrices calculations
>>> with about 1/2 to 1/4 of memory consumption comparing to Matlab.
>> 
>> Is that really true? (The large-matrix number crunching, not the
>> proper programming paradigm ;-)
>> 
>> By no scientific means of evaluation, I was under the impression that
>> the opposite was true to a smaller degree.
> 
> Matlab have pass-by-value semantics, so you have to copy your data in
> and copy your data out for every function call. You can achieve the same
> result in Python by pickling and unpickling arguments and return values,
> e.g. using this function decorator:
> 
> 
> import cPickle as pickle
> 
> def Matlab_Semantics(f):
> 
> '''
> Emulates Matlab's pass-by-value semantics,
> objects are serialized in and serialized out.
> 
> Example:  @Matlab_Semantics
>   def foo(bar):
>   pass
> '''
> 
> func = f
> return wrapper
> 
> def wrapper(*args,**kwargs):
>args_in = pickle.loads(pickle.dumps(args))
>kwargs_in = {}
>for k in kwargs:
>   kwargs_in[k] = pickle.loads(pickle.dumps(kwargs[k]))
>args_out = func(*args_in,**kwargs_in)
>args_out = pickle.loads(pickle.dumps(args_out))
>return args_out
> 
> Imagine using this horrible semantics in several layers of function
> calls. That is exactly what Matlab does. Granted, Matlab optimizes
> function calls by using copy-on-write, so it will be efficient in some
> cases, excessive cycles og copy-in and copy-out is usually what you get.
> 
> 
That's interesting.  How did you find this information?

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] scipy for centos4.4?

[Neal Becker]

Anyone know where to find usable rpms from scipy on centos4.4?

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] numpy.ndarrays as C++ arrays (wrapped with boost)

[Neal Becker]

Travis E. Oliphant wrote:

> 
>> nd to copy hundres of MB around unnecessarily.
>>
>> I think it is a real shame that boost currently doesn't properly support
>> numpy out of the box, although numpy has long obsoleted both numarray and
>> Numeric (which is both buggy and completely unsupported). All the more so
>> since writing multimedial or scientific extensions (in which numpy's
>> array interface is very natural to figure prominently) would seem such an
>> ideal use for boost.python, as soon as complex classes or compound
>> structures that need to efficiently support several (primitive) datatypes
>> are involved, boost.python could really play its strenghts compared to
>> Fortran/C based extensions.
>>
>>   
> I think it could be that boost.python is waiting for the extended buffer
> interface which is coming in Python 3.0 and Python 2.6.  This would
> really be ideal for wrapping external code in a form that plays well
> with other libraries.
> 
> 
> -Travis O.

I've spent a lot of time on this issue as well.  There have been a few
efforts, and at present I've followed my own path.

My interest is in exposing algorithms that are written in generic c++ style
to python.

The requirement here is to find containers (vectors, n-dim arrays) that are
friendly to the generic c++ side and can be used from python.  My opinion
is that Numeric and all it's descendants aren't what I want on the c++
interface side.  Also, I've stumbled over trying to grok ndarrayobject.h,
and I haven't had much success finding docs.

What I've done instead is to basically write all that I need from numpy
myself.  I've usually used ublas vector and matrix to do this.  I've also
used boost::multi_array at times (and found it quite good), and fixed_array
from stlsoft.  I implemented all the arithmetic I need and many functions
that operate on vectors (mostly I'm interested in vectors to represent
signals - not so much higher dimen arrays).

As far as views of arrays, ref counting, etc.  I have not worried much about
it.  I thought it would be a very elegant idea, but in practice I don't
really need it.  The most common thing I'd do with a view is to operate on
a slice.  Python supports this via __setitem__.  For example:
u[4:10:3] += 2
works.  There is no need for python to hold a reference to a vector slice to
do this.

Probably the biggest problem I've encountered is that there is not any
perfect c++ array container.  For 1-dimen, std::vector is pretty good - and
the interface is reasonable.  For more dimen, there doesn't seem to be any
perfect solution or general agreement on interface (or semantics).

One of my favorite ideas related to this.  I've gotten a lot of mileage out
of moving from the pair-of-iterator interface featured by stl to the
boost::range.  I believe it would be useful to consider a multi-dimen
extension of this idea.  Perhaps this could present some unifying interface
to different underlying array libraries.  For example, maybe something
like:

template
void F (in_t const& in) {
  typename row_iterator::type r = row_begin (in);
  for (; r != row_end (in); ++r) {
typename col_iterator::type c = col_begin (r);
...

The idea is that even though multi_array and ublas::matrix present very
different interfaces, they can be adapted to a 2-dimen range abstraction. 
Anyway, that's a different issue.

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] numpy.ndarrays as C++ arrays (wrapped with boost)

[Neal Becker]

Travis E. Oliphant wrote:

> 
>> nd to copy hundres of MB around unnecessarily.
>>
>> I think it is a real shame that boost currently doesn't properly support
>> numpy out of the box, although numpy has long obsoleted both numarray and
>> Numeric (which is both buggy and completely unsupported). All the more so
>> since writing multimedial or scientific extensions (in which numpy's
>> array interface is very natural to figure prominently) would seem such an
>> ideal use for boost.python, as soon as complex classes or compound
>> structures that need to efficiently support several (primitive) datatypes
>> are involved, boost.python could really play its strenghts compared to
>> Fortran/C based extensions.
>>
>>   
> I think it could be that boost.python is waiting for the extended buffer
> interface which is coming in Python 3.0 and Python 2.6.  This would
> really be ideal for wrapping external code in a form that plays well
> with other libraries.
> 
> 
> -Travis O.

I should also mention a couple of other efforts.  IIRC, blitz++ is very
close to what I wanted.  It has views of arrays with ref counting.  I did
make some simple demo interface of blitz++ to python.  I don't recall why I
abandoned this approach, but I think it's because blitz++ has reached it's
end-of-life.

There are several promising efforts on the horizon.  There is mtl4, which
despite much promise has so far not delivered.  There is glas.  Also, there
is D, which has strong native array support and PyD.  This latter approach
seems very interesting, but is also very immature.

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] More complex data types

[Neal Becker]

I'm thinking (again) about using numpy for signal processing applications. 
One issue is that there are more data types that are commonly used in
signal processing that are not available in numpy (or python). 
Specifically, it is frequently required to convert floating point
algorithms into integer algorithms.  numpy is fine for arrays of integers
(of various sizes), but it is also very useful to have arrays of
complex.  While numpy has complex, it doesn't have
complex  Has anyone thought about this?

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] More complex data types

[Neal Becker]

Charles R Harris wrote:

> On 10/5/07, Neal Becker <[EMAIL PROTECTED]> wrote:
>>
>> I'm thinking (again) about using numpy for signal processing
>> applications. One issue is that there are more data types that are
>> commonly used in signal processing that are not available in numpy (or
>> python). Specifically, it is frequently required to convert floating
>> point
>> algorithms into integer algorithms.  numpy is fine for arrays of integers
>> (of various sizes), but it is also very useful to have arrays of
>> complex.  While numpy has complex, it doesn't
>> have
>> complex  Has anyone thought about this?
> 
> 
> A bit. Multiplication begins to be a problem, though. Would you also want
> fixed point multiplication with scaling, a la PPC with altivec? What about
> division? So on and so forth. I think something like this would best be
> implemented in a specialized signal processing package but I am not sure
> of the best way to do it.
> 

I'd keep things as simple as possible.  No fixed point/scaling.  It's simple
enough to explictly rescale things as you wish.

That is (using c++ syntax):
complex a, b;
complex c = a * b;
complex d = d >> 4;

Complicating life is interoperability (conversion) of types.

I've used this concept for some years with c++/python - but not with numpy. 
It's pretty trivial to make a complex type as a C extension to python. 
Adding this to numpy would be really useful.

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] More complex data types

[Neal Becker]

Charles R Harris wrote:

> On 10/5/07, Neal Becker <[EMAIL PROTECTED]> wrote:
>>
>> Charles R Harris wrote:
>>
>> > On 10/5/07, Neal Becker <[EMAIL PROTECTED]> wrote:
>> >>
>> >> I'm thinking (again) about using numpy for signal processing
>> >> applications. One issue is that there are more data types that are
>> >> commonly used in signal processing that are not available in numpy (or
>> >> python). Specifically, it is frequently required to convert floating
>> >> point
>> >> algorithms into integer algorithms.  numpy is fine for arrays of
>> integers
>> >> (of various sizes), but it is also very useful to have arrays of
>> >> complex.  While numpy has complex, it doesn't
>> >> have
>> >> complex  Has anyone thought about this?
>> >
>> >
>> > A bit. Multiplication begins to be a problem, though. Would you also
>> want
>> > fixed point multiplication with scaling, a la PPC with altivec? What
>> about
>> > division? So on and so forth. I think something like this would best be
>> > implemented in a specialized signal processing package but I am not
>> > sure of the best way to do it.
>> >
>>
>> I'd keep things as simple as possible.  No fixed point/scaling.  It's
>> simple
>> enough to explictly rescale things as you wish.
>>
>> That is (using c++ syntax):
>> complex a, b;
>> complex c = a * b;
>> complex d = d >> 4;
>>
>> Complicating life is interoperability (conversion) of types.
>>
>> I've used this concept for some years with c++/python - but not with
>> numpy.
>> It's pretty trivial to make a complex type as a C extension to
>> python.
>> Adding this to numpy would be really useful.
> 
> 
> How about fiddling with floating point to emulate integers by subclassing
> ndarray? That wouldn't buy you the speed and size advantage of true fixed
> point but would make a flexible emulator. Which raises the question, what
> are your main goals in using such a data type? Not that I don't see the
> natural utility of having complex integer numbers (Gaussian integers), but
> if you are trying to emulate hardware something more flexible might be
> appropriate.
> 
> Chuck

Yes, this is intended for modelling hardware.  I don't know what you mean
by "more flexible".  I design my hardware algorithms to use integer
arithmetic.  What did you have in mind?

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Iterate over all 1-dim views

[Neal Becker]

Suppose I have a function F(), which is defined for 1-dim arguments.  If the
user passes an n>1 dim array, I want to apply F to each 1-dim view.

For example, for a 2-d array, apply F to each row and return a 2-d result.

For a 3-d array, select each 2-d subarray and see above.  Return 3-d result.

Any suggestions on how to code something like this in numpy?

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Interested in adding new data type

[Neal Becker]

I'm interested in experimenting with adding complex data type.  I
have "Guide to Numpy".  I'm looking at section 15.3. It looks like the
first thing is a PyArray_Desc.  There doesn't seem to be much info on what
needs to go in this.  Does anyone have any examples I could look at?


___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Use of PyArray FromAny() with a PyArrayInterface object

[Neal Becker]

Matthieu Brucher wrote:

> 2007/10/12, Alan G Isaac <[EMAIL PROTECTED]>:
>>
>> On Fri, 12 Oct 2007, Matthieu Brucher apparently wrote:
>> > I'm trying to understand (but perhaps everything is in the
>> > numpy book in which case I'd rather buy the book
>> > immediately) how to use the PyArray_FromAny() function.
>>
>> This function is discussed in the NumPy Book.
>> I see perhaps a dozen references to the function
>> in the Complete API chapter.
>>
>> Even apart from that chapter, which is useful to you but
>> not to me, I have found the book to be excellent.
> 
> 
> Excellent, thank you for the tip, the book is now ordered :)
> 
> Matthieu
I don't see the text string '__array' anywhere in this book.

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] C or C++ package like NumPy?

[Neal Becker]

Charles R Harris wrote:

> On 11/1/07, Bill Baxter <[EMAIL PROTECTED]> wrote:
>>
>> Ah, ok.  Thanks.  That does look like a good example.
>> I've heard of it, but never looked too closely for some reason.  I
>> guess I always thought of it as the library that pioneered expression
>> templates but that no one actually uses.
> 
> 
> I believe it is no longer maintained. I might be wrong about that, though.
> 
> Chuck

It is being maintained, at least to some extent.  See:
[EMAIL PROTECTED]

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Proposed Roadmap Overview

[Neal Becker]

Sturla Molden wrote:

> Den 19.02.2012 01:12, skrev Nathaniel Smith:
>>
>> I don't oppose it, but I admit I'm not really clear on what the
>> supposed advantages would be. Everyone seems to agree that
>>-- Only a carefully-chosen subset of C++ features should be used
>>-- But this subset would be pretty useful
>> I wonder if anyone is actually thinking of the same subset :-).
> 
> Probably not, everybody have their own favourite subset.
> 
> 
>>
>> Chuck mentioned iterators as one advantage. I don't understand, since
>> iterators aren't even a C++ feature, they're just objects with "next"
>> and "dereference" operators. The only difference between these is
>> spelling:
>>for (my_iter i = foo.begin(); i != foo.end(); ++i) { ... }
>>for (my_iter i = my_iter_begin(foo); !my_iter_ended(&i);
>> my_iter_next(&i)) { ... }
>> So I assume he's thinking about something more, but the discussion has
>> been too high-level for me to figure out what.
> 
>

I find range interface (i.e., boost::range) is far more useful than raw 
iterator 
interface.  I always write all my algorithms using this abstraction.

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Proposed Roadmap Overview

[Neal Becker]

Nathaniel Smith wrote:

> On Sun, Feb 19, 2012 at 9:16 AM, David Cournapeau  wrote:
>> On Sun, Feb 19, 2012 at 8:08 AM, Mark Wiebe  wrote:
>>> Is there a specific
>>> target platform/compiler combination you're thinking of where we can do
>>> tests on this? I don't believe the compile times are as bad as many people
>>> suspect, can you give some simple examples of things we might do in NumPy
>>> you expect to compile slower in C++ vs C?
>>
>> Switching from gcc to g++ on the same codebase should not change much
>> compilation times. We should test, but that's not what worries me.
>> What worries me is when we start using C++ specific code, STL and co.
>> Today, scipy.sparse.sparsetools takes half of the build time  of the
>> whole scipy, and it does not even use fancy features. It also takes Gb
>> of ram when building in parallel.
> 
> I like C++ but it definitely does have issues with compilation times.
> 
> IIRC the main problem is very simple: STL and friends (e.g. Boost) are
> huge libraries, and because they use templates, the entire source code
> is in the header files. That means that as soon as you #include a few
> standard C++ headers, your innocent little source file has suddenly
> become hundreds of thousands of lines long, and it just takes the
> compiler a while to churn through megabytes of source code, no matter
> what it is. (Effectively you recompile some significant fraction of
> STL from scratch on every file, and then throw it away.)
> 
> Precompiled headers can help some, but require complex and highly
> non-portable build-system support. (E.g., gcc's precompiled header
> constraints are here:
> http://gcc.gnu.org/onlinedocs/gcc/Precompiled-Headers.html -- only one
> per source file, etc.)
> 
> To demonstrate: a trivial hello-world in C using , versus a
> trivial version in C++ using .
> 
> On my laptop (gcc 4.5.2), compiling each program 100 times in a loop requires:
>   C: 2.28 CPU seconds
>   C compiled with C++ compiler: 4.61 CPU seconds
>   C++: 17.66 CPU seconds
> Slowdown for using g++ instead of gcc: 2.0x
> Slowdown for using C++ standard library: 3.8x
> Total C++ penalty: 7.8x
> 
> Lines of code compiled in each case:
>   $ gcc -E hello.c | wc
>   8552039   16934
>   $ g++ -E hello.cc | wc
> 18569   40994  437954
> (I.e., the C++ hello world is almost half a megabyte.)
> 
> Of course we won't be using , but , 
> etc. all have the same basic character.
> 
> -- Nathaniel
> 
> (Test files attached, times were from:
>   time sh -c 'for i in $(seq 100); do gcc hello.c -o hello-c; done'
>   cp hello.c c-hello.cc
>   time sh -c 'for i in $(seq 100); do g++ c-hello.cc -o c-hello-cc; done'
>   time sh -c 'for i in $(seq 100); do g++ hello.cc -o hello-cc; done'
> and then summing the resulting user and system times.)

On Fedora linux I use ccache, which is completely transparant and makes a huge 
difference in build times.

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Proposed Roadmap Overview

[Neal Becker]

Sturla Molden wrote:

> 
> Den 18. feb. 2012 kl. 01:58 skrev Charles R Harris
> :
> 
>> 
>> 
>> On Fri, Feb 17, 2012 at 4:44 PM, David Cournapeau  wrote:
>> I don't think c++ has any significant advantage over c for high performance
>> libraries. I am not convinced by the number of people argument either: it is
>> not my experience that c++ is easier to maintain in a open source context,
>> where the level of people is far from consistent. I doubt many people did not
>> contribute to numoy because it is in c instead if c++. While this is somehow
>> subjective, there are reasons that c is much more common than c++ in that
>> context.
>> 
>> 
>> I think C++ offers much better tools than C for the sort of things in Numpy.
>> The compiler will take care of lots of things that now have to be hand
>> crafted and I wouldn't be surprised to see the code size shrink by a
>> significant factor.
> 
> The C++11 standard is fantastic. There are automatic data types, closures,
> reference counting, weak references, an improved STL with datatypes that map
> almost 1:1 against any built-in Python type, a sane threading API, regex, ect.
> Even prng is Mersenne Twister by standard. With C++11 it is finally possible
> to "write C++ (almost) like Python". On the downside, C++ takes a long term to
> learn, most C++ text books teach bad programming habits from the beginning to
> the end, and C++ becomes inherently dangerous if you write C++ like C. Many
> also abuse C++ as an bloatware generator. Templates can also be abused to
> write code that are impossible to debug. While it in theory could be better, C
> is a much smaller language. Personally I prefer C++ to C, but I am not
> convinced it will be better for NumPy.
> 

I'm all for c++11, but if you are worried about portability, dude, you have a 
bit of a problem here.

> I agree about Cython. It is nice for writing a Python interface for C, but get
> messy and unclean when used for anything else. It also has too much focus on
> adding all sorts of "new features" instead of correctness and stability. I
> don't trust it to generate bug-free code anymore.
> 
> For wrapping C, Swig might be just as good. For C++, SIP, CXX or Boost.Pyton
> work well too.
> 
> If cracy ideas are allowed, what about PyPy RPython? Or perhaps Go? Or even C#
> if a native compuler could be found?
> 
> 
c# is a non-starter if you want to run on linux.

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Proposed Roadmap Overview

[Neal Becker]

Charles R Harris wrote:

> On Fri, Feb 17, 2012 at 12:09 PM, Benjamin Root  wrote:
> 
>>
>>
>> On Fri, Feb 17, 2012 at 1:00 PM, Christopher Jordan-Squire <
>> cjord...@uw.edu> wrote:
>>
>>> On Fri, Feb 17, 2012 at 10:21 AM, Mark Wiebe  wrote:
>>> > On Fri, Feb 17, 2012 at 11:52 AM, Eric Firing 
>>> wrote:
>>> >>
>>> >> On 02/17/2012 05:39 AM, Charles R Harris wrote:
>>> >> >
>>> >> >
>>> >> > On Fri, Feb 17, 2012 at 8:01 AM, David Cournapeau <
>>> courn...@gmail.com
>>> >> > > wrote:
>>> >> >
>>> >> > Hi Travis,
>>> >> >
>>> >> > On Thu, Feb 16, 2012 at 10:39 PM, Travis Oliphant
>>> >> > mailto:tra...@continuum.io>> wrote:
>>> >> >  > Mark Wiebe and I have been discussing off and on (as well as
>>> >> > talking with Charles) a good way forward to balance two competing
>>> >> > desires:
>>> >> >  >
>>> >> >  >* addition of new features that are needed in NumPy
>>> >> >  >* improving the code-base generally and moving towards
>>> a
>>> >> > more maintainable NumPy
>>> >> >  >
>>> >> >  > I know there are load voices for just focusing on the second
>>> of
>>> >> > these and avoiding the first until we have finished that.  I
>>> >> > recognize the need to improve the code base, but I will also be
>>> >> > pushing for improvements to the feature-set and user experience
>>> in
>>> >> > the process.
>>> >> >  >
>>> >> >  > As a result, I am proposing a rough outline for releases over
>>> the
>>> >> > next year:
>>> >> >  >
>>> >> >  >* NumPy 1.7 to come out as soon as the serious bugs
>>> can be
>>> >> > eliminated.  Bryan, Francesc, Mark, and I are able to help triage
>>> >> > some of those.
>>> >> >  >
>>> >> >  >* NumPy 1.8 to come out in July which will have as many
>>> >> > ABI-compatible feature enhancements as we can add while improving
>>> >> > test coverage and code cleanup.   I will post to this list more
>>> >> > details of what we plan to address with it later.Included for
>>> >> > possible inclusion are:
>>> >> >  >* resolving the NA/missing-data issues
>>> >> >  >* finishing group-by
>>> >> >  >* incorporating the start of label arrays
>>> >> >  >* incorporating a meta-object
>>> >> >  >* a few new dtypes (variable-length string,
>>> >> > varialbe-length unicode and an enum type)
>>> >> >  >* adding ufunc support for flexible dtypes and possibly
>>> >> > structured arrays
>>> >> >  >* allowing generalized ufuncs to work on more kinds of
>>> >> > arrays besides just contiguous
>>> >> >  >* improving the ability for NumPy to receive
>>> JIT-generated
>>> >> > function pointers for ufuncs and other calculation opportunities
>>> >> >  >* adding "filters" to Input and Output
>>> >> >  >* simple computed fields for dtypes
>>> >> >  >* accepting a Data-Type specification as a class or
>>> JSON
>>> >> > file
>>> >> >  >* work towards improving the dtype-addition mechanism
>>> >> >  >* re-factoring of code so that it can compile with a
>>> C++
>>> >> > compiler and be minimally dependent on Python data-structures.
>>> >> >
>>> >> > This is a pretty exciting list of features. What is the rationale
>>> >> > for
>>> >> > code being compiled as C++ ? IMO, it will be difficult to do so
>>> >> > without preventing useful C constructs, and without removing
>>> some of
>>> >> > the existing features (like our use of C99 complex). The subset
>>> that
>>> >> > is both C and C++ compatible is quite constraining.
>>> >> >
>>> >> >
>>> >> > I'm in favor of this myself, C++ would allow a lot code cleanup and
>>> make
>>> >> > it easier to provide an extensible base, I think it would be a
>>> natural
>>> >> > fit with numpy. Of course, some C++ projects become tangled messes of
>>> >> > inheritance, but I'd be very interested in seeing what a good C++
>>> >> > designer like Mark, intimately familiar with the numpy code base,
>>> could
>>> >> > do. This opportunity might not come by again anytime soon and I
>>> think we
>>> >> > should grab onto it. The initial step would be a release whose code
>>> that
>>> >> > would compile in both C/C++, which mostly comes down to removing C++
>>> >> > keywords like 'new'.
>>> >> >
>>> >> > I did suggest running it by you for build issues, so please raise any
>>> >> > you can think of. Note that MatPlotLib is in C++, so I don't think
>>> the
>>> >> > problems are insurmountable. And choosing a set of compilers to
>>> support
>>> >> > is something that will need to be done.
>>> >>
>>> >> It's true that matplotlib relies heavily on C++, both via the Agg
>>> >> library and in its own extension code.  Personally, I don't like this;
>>> I
>>> >> think it raises the barrier to contributing.  C++ is an order of
>>> >> magnitu

[Numpy-discussion] Where is arrayobject.h?

[Neal Becker]

What is the correct way to find the installed location of arrayobject.h?

On fedora, I had been using:
(via scons):

import distutils.sysconfig
PYTHONINC = distutils.sysconfig.get_python_inc()
PYTHONLIB = distutils.sysconfig.get_python_lib(1)

NUMPYINC = PYTHONLIB + '/numpy/core/include'

But on ubuntu, this fails.  It seems numpy was installed into 
/usr/local/lib/..., while PYTHONLIB expands to /usr/lib/python2.7/dist-packages.

Is there a universal method?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Proposed Roadmap Overview

[Neal Becker]

It's great advice to say 

avoid using new

instead rely on scope and classes such as std::vector.

I just want to point out, that sometimes objects must outlive scope.

For those cases, std::shared_ptr can be helpful.

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] mkl usage

[Neal Becker]

Is mkl only used for linear algebra?  Will it speed up e.g., elementwise 
transendental functions?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] mkl usage

[Neal Becker]

Pauli Virtanen wrote:

> 23.02.2012 20:44, Francesc Alted kirjoitti:
>> On Feb 23, 2012, at 1:33 PM, Neal Becker wrote:
>> 
>>> Is mkl only used for linear algebra?  Will it speed up e.g., elementwise
>>> transendental functions?
>> 
>> Yes, MKL comes with VML that has this type of optimizations:
> 
> And also no, in the sense that Numpy and Scipy don't use VML.
> 

My question is:

"Should I purchase MKL?"

To what extent will it speed up my existing python code, without my having to 
exert (much) effort?

So that would be numpy/scipy.

I'd entertain trying other things, if it wasn't much effort.

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] mkl usage

[Neal Becker]

Francesc Alted wrote:

> On Feb 23, 2012, at 2:19 PM, Neal Becker wrote:
> 
>> Pauli Virtanen wrote:
>> 
>>> 23.02.2012 20:44, Francesc Alted kirjoitti:
>>>> On Feb 23, 2012, at 1:33 PM, Neal Becker wrote:
>>>> 
>>>>> Is mkl only used for linear algebra?  Will it speed up e.g., elementwise
>>>>> transendental functions?
>>>> 
>>>> Yes, MKL comes with VML that has this type of optimizations:
>>> 
>>> And also no, in the sense that Numpy and Scipy don't use VML.
>>> 
>> 
>> My question is:
>> 
>> "Should I purchase MKL?"
>> 
>> To what extent will it speed up my existing python code, without my having to
>> exert (much) effort?
>> 
>> So that would be numpy/scipy.
> 
> Pauli already answered you.  If you are restricted to use numpy/scipy and your
> aim is to accelerate the evaluation of transcendental functions, then there is
> no point in purchasing MKL.  If you can open your spectrum and use numexpr,
> then I think you should ponder about it.
> 
> -- Francesc Alted

Thanks.  One more thing, on theano I'm guessing MKL is required to be installed 
onto each host that would use it at runtime?  So I'd need a licensed copy for 
each host that will execute the code?  I'm guessing that because theano needs 
to 
compile code at runtime.

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Creating a bool array with Cython

[Neal Becker]

Keith Goodman wrote:

> Is this a reasonable (and fast) way to create a bool array in cython?
> 
> def makebool():
> cdef:
> int n = 2
> np.npy_intp *dims = [n]
> np.ndarray[np.uint8_t, ndim=1] a
> a = PyArray_EMPTY(1, dims, NPY_UINT8, 0)
> a[0] = 1
> a[1] = 0
> a.dtype = np.bool
> return a
> 
> Will a numpy bool array be np.uint8 on all platforms?
> 
> How can I do a.dtype=np.bool using the numpy C api? Is there any point
> (speed) in doing so?

Cython has it's own mail lists - maybe you can find more answers there.

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Proposed Roadmap Overview

[Neal Becker]

Charles R Harris wrote:

> On Tue, Feb 28, 2012 at 12:05 PM, John Hunter  wrote:
> 
>> On Sat, Feb 18, 2012 at 5:09 PM, David Cournapeau wrote:
>>
>>>
>>> There are better languages than C++ that has most of the technical
>>>
>>> benefits stated in this discussion (rust and D being the most
>>> "obvious" ones), but whose usage is unrealistic today for various
>>> reasons: knowledge, availability on "esoteric" platforms, etc… A new
>>> language is completely ridiculous.
>>>
>>
>>
>> I just saw this for the first time today: Linus Torvalds on C++ (
>> http://harmful.cat-v.org/software/c++/linus).  The post is from 2007 so
>> many of you may have seen it, but I thought it was entertainng enough and
>> on-topic enough with this thread that I'd share it in case you haven't.
>>
>>
>> The point he makes:
>>
>>   In other words, the only way to do good, efficient, and system-level and
>>   portable C++ ends up to limit yourself to all the things that
>> are basically
>>   available in C
>>
>> was interesting to me because the best C++ library I have ever worked with
>> (agg) imports *nothing* except standard C libs (no standard template
>> library).  In fact, the only includes external to external to itself
>> are math.h, stdlib.h, stdio.h, and string.h.
>>
>> To shoehorn Jamie Zawinski's famous regex quote (
>> http://regex.info/blog/2006-09-15/247).  "Some people, when confronted
>> with a problem, think “I know, I'll use boost.”   Now they have two
>> problems."
>>
>> Here is the Linus post:
>>
>> From: Linus Torvalds  linux-foundation.org>
>> Subject: Re: [RFC] Convert builin-mailinfo.c to use The Better String
>> Library.
>> Newsgroups: gmane.comp.version-control.git
>> Date: 2007-09-06 17:50:28 GMT (2 years, 14 weeks, 16 hours and 36 minutes
>> ago)
>>
>> On Wed, 5 Sep 2007, Dmitry Kakurin wrote:
>> >
>> > When I first looked at Git source code two things struck me as odd:
>> > 1. Pure C as opposed to C++. No idea why. Please don't talk about
>> portability,
>> > it's BS.
>>
>> *YOU* are full of bullshit.
>>
>> C++ is a horrible language. It's made more horrible by the fact that a lot
>> of substandard programmers use it, to the point where it's much much
>> easier to generate total and utter crap with it. Quite frankly, even if
>> the choice of C were to do *nothing* but keep the C++ programmers out,
>> that in itself would be a huge reason to use C.
>>
>> In other words: the choice of C is the only sane choice. I know Miles
>> Bader jokingly said "to piss you off", but it's actually true. I've come
>> to the conclusion that any programmer that would prefer the project to be
>> in C++ over C is likely a programmer that I really *would* prefer to piss
>> off, so that he doesn't come and screw up any project I'm involved with.
>>
>> C++ leads to really really bad design choices. You invariably start using
>> the "nice" library features of the language like STL and Boost and other
>> total and utter crap, that may "help" you program, but causes:
>>
>>  - infinite amounts of pain when they don't work (and anybody who tells me
>>that STL and especially Boost are stable and portable is just so full
>>of BS that it's not even funny)
>>
>>  - inefficient abstracted programming models where two years down the road
>>you notice that some abstraction wasn't very efficient, but now all
>>your code depends on all the nice object models around it, and you
>>cannot fix it without rewriting your app.
>>
>> In other words, the only way to do good, efficient, and system-level and
>> portable C++ ends up to limit yourself to all the things that are
>> basically available in C. And limiting your project to C means that people
>> don't screw that up, and also means that you get a lot of programmers that
>> do actually understand low-level issues and don't screw things up with any
>> idiotic "object model" crap.
>>
>> So I'm sorry, but for something like git, where efficiency was a primary
>> objective, the "advantages" of C++ is just a huge mistake. The fact that
>> we also piss off people who cannot see that is just a big additional
>> advantage.
>>
>> If you want a VCS that is written in C++, go play with Monotone. Really.
>> They use a "real database". They use "nice object-oriented libraries".
>> They use "nice C++ abstractions". And quite frankly, as a result of all
>> these design decisions that sound so appealing to some CS people, the end
>> result is a horrible and unmaintainable mess.
>>
>> But I'm sure you'd like it more than git.
>>
>>
> Yeah, Linus doesn't like C++. No doubt that is in part because of the
> attempt to rewrite Linux in C++ back in the early 90's and the resulting
> compiler and portability problems. Linus also writes C like it was his
> native tongue, he likes to work close to the metal, and he'd probably
> prefer it over Python for most problems ;) Things have improved in the
> compiler department, and I think C++ really wasn't much of an improvement
> over C until templates and

[Numpy-discussion] all elements equal

[Neal Becker]

What is a simple, efficient way to determine if all elements in an array (in my 
case, 1D) are equal?  How about close?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] all elements equal

[Neal Becker]

Keith Goodman wrote:

> On Mon, Mar 5, 2012 at 11:14 AM, Neal Becker  wrote:
>> What is a simple, efficient way to determine if all elements in an array (in
>> my case, 1D) are equal?  How about close?
> 
> For the exactly equal case, how about:
> 
> I[1] a = np.array([1,1,1,1])
> I[2] np.unique(a).size
> O[2] 1# All equal
> 
> I[3] a = np.array([1,1,1,2])
> I[4] np.unique(a).size
> O[4] 2   # All not equal

I considered this - just not sure if it's the most efficient

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] all elements equal

[Neal Becker]

Keith Goodman wrote:

> On Mon, Mar 5, 2012 at 11:52 AM, Benjamin Root  wrote:
>> Another issue to watch out for is if the array is empty.  Technically
>> speaking, that should be True, but some of the solutions offered so far
>> would fail in this case.
> 
> Good point.
> 
> For fun, here's the speed of a simple cython allclose:
> 
> I[2] a = np.ones(10)
> I[3] timeit a.min() == a.max()
> 1 loops, best of 3: 106 us per loop
> I[4] timeit allequal(a)
> 1 loops, best of 3: 68.9 us per loop
> 
> I[5] a[1] = 9
> I[6] timeit a.min() == a.max()
> 1 loops, best of 3: 102 us per loop
> I[7] timeit allequal(a)
> 100 loops, best of 3: 269 ns per loop
> 
> where
> 
> @cython.boundscheck(False)
> @cython.wraparound(False)
> def allequal(np.ndarray[np.float64_t, ndim=1] a):
> cdef:
> np.float64_t a0
> Py_ssize_t i, n=a.size
> a0 = a[0]
> for i in range(n):
> if a[i] != a0:
> return False
> return True

But doesn't this one fail on empty array?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] use for missing (ignored) data?

[Neal Becker]

I'm wondering what is the use for the ignored data feature?

I can use:

A[valid_A_indexes] = whatever

to process only the 'non-ignored' portions of A.  So at least some simple cases 
of ignored data are already supported without introducing a new type.

OTOH:

w = A[valid_A_indexes]

will copy A's data, and subsequent use of 

w[:] = something

will not update A.

Is this the reason for wanting the ignored data feature?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] use for missing (ignored) data?

[Neal Becker]

Charles R Harris wrote:

> On Wed, Mar 7, 2012 at 1:05 PM, Neal Becker  wrote:
> 
>> I'm wondering what is the use for the ignored data feature?
>>
>> I can use:
>>
>> A[valid_A_indexes] = whatever
>>
>> to process only the 'non-ignored' portions of A.  So at least some simple
>> cases
>> of ignored data are already supported without introducing a new type.
>>
>> OTOH:
>>
>> w = A[valid_A_indexes]
>>
>> will copy A's data, and subsequent use of
>>
>> w[:] = something
>>
>> will not update A.
>>
>> Is this the reason for wanting the ignored data feature?
>>
> 
> Suppose you are working with plotted data and want to turn points on/off by
> clicking on them interactively to see how that affects a fit. Why make
> multiple copies, change sizes, destroy data, and all that nonsense? Just
> have the click update the mask and redraw.
> 
> Chuck

But does

some_func (A[valid_data_mask])

actually perform a copy?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] unique along axis?

[Neal Becker]

I see unique does not take an axis arg.

Suggested way to apply unique to each column of a 2d array?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] apply 'getitem to each element of obj array?

[Neal Becker]

I have an array of object.

How can I apply attribute access to each element?

I want to do, for example,
np.all (u.some_attribute == 0) for all elements in u? 

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] apply 'getitem to each element of obj array?

[Neal Becker]

Adam Hughes wrote:

> If you are storing objects, then can't you store them in a list and just do:
> 
> for obj in objectlist:
>  obj.attribute = value
> 
> Or am I misunderstanding?
> 

It's multi-dimensional, and I wanted to avoid writing explicit loops.

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] apply 'getitem to each element of obj array?

[Neal Becker]

Ken Watford wrote:

> On Thu, Apr 5, 2012 at 11:57 AM, Olivier Delalleau  wrote:
>> Le 5 avril 2012 11:45, Neal Becker  a écrit :
>>
>> You can do:
>>
>> f = numpy.frompyfunc(lambda x: x.some_attribute == 0, 1, 1)
>>
>> Then
>> f(array_of_objects_x)
> 
> This is handy too:
> 
> agetattr = numpy.frompyfunc(getattr, 2, 1)
> 
> array_of_values = agetattr(array_of_objects, 'some_attribute')

I suppose for setitem something similar, except I don't think you can do that 
with lambda since lambda doesn't allow an assignment.

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] don't understand nditer

[Neal Becker]

Along the lines of my question about apply getitem to each element...

If I try to use nditer, I seem to run into trouble:

 for d in np.nditer (y, ['refs_ok'],['readwrite']):
   : y[...].w = 2
   : 
---
AttributeErrorTraceback (most recent call last)
/home/nbecker/hn-8psk/ in ()
  1 for d in np.nditer (y, ['refs_ok'],['readwrite']):
> 2 y[...].w = 2
  3 

AttributeError: 'numpy.ndarray' object has no attribute 'w'

y is a 2D array of 'noop' class instances:

class noop:
  pass
 
y
Out[75]: 
array([[<__main__.noop instance at 0x4241098>,
<__main__.noop instance at 0x4241098>,
<__main__.noop instance at 0x4241098>,
<__main__.noop instance at 0x4241098>,
<__main__.noop instance at 0x4241098>],
   [<__main__.noop instance at 0x4241098>,
<__main__.noop instance at 0x4241098>,
<__main__.noop instance at 0x4241098>,
<__main__.noop instance at 0x4241098>,
<__main__.noop instance at 0x4241098>]], dtype=object)

Any idea how to do this setattr per element using nditer?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] A crazy masked-array thought

[Neal Becker]

Nathaniel Smith wrote:

> On Sat, Apr 28, 2012 at 7:38 AM, Richard Hattersley
>  wrote:
>> So, assuming numpy.ndarray became a strict subclass of some new masked
>> array, it looks plausible that adding just a few checks to numpy.ndarray to
>> exclude the masked superclass would prevent much downstream code from
>> accidentally operating on masked arrays.
> 
> I think the main point I was trying to make is that it's the existence
> and content of these checks that matters. They don't necessarily have
> any relation at all to which thing Python calls a "superclass" or a
> "subclass".
> 
> -- Nathaniel

I don't agree with the argument that ma should be a superclass of ndarray.  It 
is ma that is adding features.  That makes it a subclass.  We're not talking 
mathematics here.

There is a well-known disease of OOP where everything seems to bubble up to the 
top of the class hierarchy - so that the base class becomes bloated to support 
every feature needed by subclasses.  I believe that's considered poor design.

Is there a way to support ma as a subclass of ndarray, without introducing 
overhead into ndarray?  Without having given this much real thought, I do have 
some idea.  What are the operations that we need on arrays?  The most basic are:

1. element access
2. get size (shape)

In an OO design, these would be virtual functions (or in C, pointers to 
functions).  But this would introduce unacceptable overhead.

In a generic programming design (c++ templates), we would essentially generate 
2 
copies of every function, one that operates on plain arrays, and one that 
operates on masked arrays, each using the appropriate function for element 
access, shape, etc.  This way, no uneeded overhead is introduced, (although the 
code size is increased - but this is probably of little consequence on modern 
demand-paged OS).

Following this approach, ma and ndarray don't have to have any inheritance 
relation.  OTOH, inheritance is probably useful since there are many common 
features to ma and ndarray, and a lot of code could be shared.



___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Quaternion data type

[Neal Becker]

I am quite interested in a fixed point data type.  I had produced a working 
model some time ago.

Maybe I can use some of these new efforts to provide good examples as a guide.

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] slicing and aliasing

[Neal Becker]

Will copying slices always work correctly w/r to aliasing?

That is, will:

u[a:b] = u[c:d]

always work (assuming the ranges of a:b, d:d are equal, or course)

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] varargs for logical_or, etc

[Neal Becker]

I think it's unfortunate that functions like logical_or are limited to binary.

As a workaround, I've been using this:

def apply_binary (func, *args):
if len (args) == 1:
return args[0]
elif len (args) == 2:
return func (*args)
else:
return func (
apply_binary (func, *args[:len(args)/2]),
apply_binary (func, *args[(len(args))/2:]))

Then for example:

punc2 = np.logical_and (u % 5 == 4,
   apply_binary (np.logical_or, u/5 == 3, u/5 == 8, u/5 == 
13))


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] lazy evaluation

[Neal Becker]

Would lazy eval be able to eliminate temps in doing operations such as:

np.sum (u != 23)?

That is, now ops involving selecting elements of matrixes are often performed 
by 
first constructing temp matrixes, and the operating on them.

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] possible enhancement to getitem?

[Neal Becker]

In [3]: u = np.arange(10)

In [4]: u
Out[4]: array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

In [5]: u[-2:]
Out[5]: array([8, 9])

In [6]: u[-2:2]
Out[6]: array([], dtype=int64)

I would argue for consistency it would be desirable for this to return

[8, 9, 0, 1]


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] possible enhancement to getitem?

[Neal Becker]

Robert Kern wrote:

> On Thu, Jun 7, 2012 at 7:55 PM, Neal Becker  wrote:
>> In [3]: u = np.arange(10)
>>
>> In [4]: u
>> Out[4]: array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
>>
>> In [5]: u[-2:]
>> Out[5]: array([8, 9])
>>
>> In [6]: u[-2:2]
>> Out[6]: array([], dtype=int64)
>>
>> I would argue for consistency it would be desirable for this to return
>>
>> [8, 9, 0, 1]
> 
> Unfortunately, this would be inconsistent with Python semantics:
> 
> [~]
> |1> u = range(10)
> 
> [~]
> |2> u[-2:2]
> []
> 

The fact that this proposed numpy behavior would not match python list behavior 
holds little weight for me.  I would still favor this change, unless it added 
significant overhead.  My opinion, of course.

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] trivial question?

[Neal Becker]

Maybe I'm being slow, but is there any convenient function to calculate,
for 2 vectors:

\sum_i \sum_j x_i y_j

(I had a matrix once, but it vanished without a trace)

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] trivial question?

[Neal Becker]

Robert Kern wrote:

> On Wed, Jun 20, 2012 at 3:58 PM, Neal Becker  wrote:
>> Maybe I'm being slow, but is there any convenient function to calculate,
>> for 2 vectors:
>>
>> \sum_i \sum_j x_i y_j
>>
>> (I had a matrix once, but it vanished without a trace)
> 
> np.multiply.outer(x, y).sum()
> 

I guess that's the same as

np.outer (x, y).sum()

?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] m-ary logical functions

[Neal Becker]

I've been bitten several times by this.

logical_or (a, b, c)

is silently accepted when I really meant

logical_or (logical_or (a, b), c)

because the logic functions are binary, where I expected them to be m-ary.

Dunno if anything can be done about it.

Sure would like it if they were m-ary and out was a kw arg.

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] gnu mpc 1.0 released

[Neal Becker]

Perhaps of some interest here:

http://lwn.net/Articles/507756/rss

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] blaze lib announcement

[Neal Becker]

This looks interesting:

http://code.google.com/p/blaze-lib/

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] broadcasting question

[Neal Becker]

I think this should be simple, but I'm drawing a blank

I have 2 2d matrixes

Matrix A has indexes (i, symbol)
Matrix B has indexes (state, symbol)

I combined them into a 3d matrix:

C = A[:,newaxis,:] + B[newaxis,:,:]
where C has indexes (i, state, symbol)

That works fine.

Now suppose I want to omit B (for debug), like:

C = A[:,newaxis,:]

In other words, all I want is to add a dimension into A and force it to 
broadcast along that axis.  How do I do that?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] silently ignored size mismatch (bug??)

[Neal Becker]

In [19]: u = np.arange (10)

In [20]: v = np.arange (10)

In [21]: u[v] = u

In [22]: u[v] = np.arange(11)

silence...

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] silently ignored size mismatch (bug??)

[Neal Becker]

Sounds like I'm not the only one surprised then:

http://projects.scipy.org/numpy/ticket/2220

Matthew Brett wrote:

> Hi,
> 
> On Mon, Oct 1, 2012 at 9:04 AM, Pierre Haessig 
> wrote:
>> Hi,
>>
>> Le 28/09/2012 21:02, Neal Becker a écrit :
>>> In [19]: u = np.arange (10)
>>>
>>> In [20]: v = np.arange (10)
>>>
>>> In [21]: u[v] = u
>>>
>>> In [22]: u[v] = np.arange(11)
>>>
>>> silence...
>> I've same behavior with my numpy 1.6.2.
>>
>> It indeed looks strange that the end of the data vector is dropped in
>> silence.
> 
> Same on numpy 1.4.1.  I also would have predicted an error in that
> situation.  For example in matlab:
> 
>>> a = 1:10
> 
> a =
> 
>  1 2 3 4 5 6 7 8 910
> 
>>> b = 1:10
> 
> b =
> 
>  1 2 3 4 5 6 7 8 910
> 
>>> a(b) = 11:20
> 
> a =
> 
> 11121314151617181920
> 
>>> a(b) = 11:21
> ???  In an assignment  A(I) = B, the number of elements in B and
>  I must be the same.
> 
> Is there any reason not to do that?
> 
> Best,
> 
> Matthew


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] print array in a form that can then be input

[Neal Becker]

I find it annoying that in casual use, if I print an array, that form can't be 
directly used as subsequent input (or can it?).

What do others do about this?  When I say casual, what I mean is, I write some 
long-running task and at the end, print some small array.  Now I decide I'd 
like 
to cut/paste that into a new program.

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] yet another trivial question

[Neal Becker]

I'm trying to convert some matlab code.  I see this:

b(1)=[];

AFAICT, this removes the first element of the array, shifting the others.

What is the preferred numpy equivalent?

I'm not sure if

b[:] = b[1:]

is safe or not

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] testing with amd libm/acml

[Neal Becker]

I'm trying to do a bit of benchmarking to see if amd libm/acml will help me.

I got an idea that instead of building all of numpy/scipy and all of my custom 
modules against these libraries, I could simply use:

LD_PRELOAD=/opt/amdlibm-3.0.2/lib/dynamic/libamdlibm.so:/opt/acml5.2.0/gfortran64/lib/libacml.so
 


I'm hoping that both numpy and my own dll's then will take advantage of these 
libraries.

Do you think this will work?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] testing with amd libm/acml

[Neal Becker]

David Cournapeau wrote:

> On Wed, Nov 7, 2012 at 12:35 PM, Neal Becker  wrote:
>> I'm trying to do a bit of benchmarking to see if amd libm/acml will help me.
>>
>> I got an idea that instead of building all of numpy/scipy and all of my
>> custom modules against these libraries, I could simply use:
>>
>> 
LD_PRELOAD=/opt/amdlibm-3.0.2/lib/dynamic/libamdlibm.so:/opt/acml5.2.0/gfortran64/lib/libacml.so
>> 
>>
>> I'm hoping that both numpy and my own dll's then will take advantage of these
>> libraries.
>>
>> Do you think this will work?
> 
> Quite unlikely depending on your configuration, because those
> libraries are rarely if ever ABI compatible (that's why it is such a
> pain to support).
> 
> David

When you say quite unlikely (to work), you mean 

a) unlikely that libm/acml will be used to resolve symbols in numpy/dlls at 
runtime (e.g., exp)?

or 

b) program may produce wrong results and/or crash ?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] testing with amd libm/acml

[Neal Becker]

David Cournapeau wrote:

> On Wed, Nov 7, 2012 at 1:56 PM, Neal Becker  wrote:
>> David Cournapeau wrote:
>>
>>> On Wed, Nov 7, 2012 at 12:35 PM, Neal Becker  wrote:
>>>> I'm trying to do a bit of benchmarking to see if amd libm/acml will help
>>>> me.
>>>>
>>>> I got an idea that instead of building all of numpy/scipy and all of my
>>>> custom modules against these libraries, I could simply use:
>>>>
>>>>
>> 
LD_PRELOAD=/opt/amdlibm-3.0.2/lib/dynamic/libamdlibm.so:/opt/acml5.2.0/gfortran64/lib/libacml.so
>>>> 
>>>>
>>>> I'm hoping that both numpy and my own dll's then will take advantage of
>>>> these libraries.
>>>>
>>>> Do you think this will work?
>>>
>>> Quite unlikely depending on your configuration, because those
>>> libraries are rarely if ever ABI compatible (that's why it is such a
>>> pain to support).
>>>
>>> David
>>
>> When you say quite unlikely (to work), you mean
>>
>> a) unlikely that libm/acml will be used to resolve symbols in numpy/dlls at
>> runtime (e.g., exp)?
>>
>> or
>>
>> b) program may produce wrong results and/or crash ?
> 
> Both, actually. That's not something I would use myself. Did you try
> openblas ? It is open source, simple to build, and is pretty fast,
> 
> David

Actually, for my current work, I'm more concerned with speeding up operations 
such as exp, log and basic vector arithmetic.  Any thoughts on that?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] testing with amd libm/acml

[Neal Becker]

David Cournapeau wrote:

> On Wed, Nov 7, 2012 at 1:56 PM, Neal Becker  wrote:
>> David Cournapeau wrote:
>>
>>> On Wed, Nov 7, 2012 at 12:35 PM, Neal Becker  wrote:
>>>> I'm trying to do a bit of benchmarking to see if amd libm/acml will help
>>>> me.
>>>>
>>>> I got an idea that instead of building all of numpy/scipy and all of my
>>>> custom modules against these libraries, I could simply use:
>>>>
>>>>
>> 
LD_PRELOAD=/opt/amdlibm-3.0.2/lib/dynamic/libamdlibm.so:/opt/acml5.2.0/gfortran64/lib/libacml.so
>>>> 
>>>>
>>>> I'm hoping that both numpy and my own dll's then will take advantage of
>>>> these libraries.
>>>>
>>>> Do you think this will work?
>>>
>>> Quite unlikely depending on your configuration, because those
>>> libraries are rarely if ever ABI compatible (that's why it is such a
>>> pain to support).
>>>
>>> David
>>
>> When you say quite unlikely (to work), you mean
>>
>> a) unlikely that libm/acml will be used to resolve symbols in numpy/dlls at
>> runtime (e.g., exp)?
>>
>> or
>>
>> b) program may produce wrong results and/or crash ?
> 
> Both, actually. That's not something I would use myself. Did you try
> openblas ? It is open source, simple to build, and is pretty fast,
> 
> David

In my current work, probably the largest bottlenecks are 'max*',  which are

log (\sum e^(x_i))


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] testing with amd libm/acml

[Neal Becker]

Would you expect numexpr without MKL to give a significant boost?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] numexpr question

[Neal Becker]

I'm interested in trying numexpr, but have a question (not sure where's the 
best 
forum to ask).

The examples I see use

ne.evaluate ("some string...")

When used within a loop, I would expect the compilation from the string form to 
add significant overhead.  I would have thought a pre-compiled form would be 
available, similar to a precompiled regexp.  yes/no?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] confused about univaritespline

[Neal Becker]

I don't understand why the plot of the spline continues on a negative slope at 
the end, but the plot of the integral of it flattens.

-

import numpy as np
import matplotlib.pyplot as plt

ibo = np.array ((12, 14, 16, 18, 20, 22, 24, 26, 28, 29, 29.8, 30.2))
gain_deriv = np.array ((0, 0,  0,  0,   0, 0,  .2, .4, .5, .5, 0,-2))

import scipy.interpolate
s = scipy.interpolate.UnivariateSpline(ibo, gain_deriv, s=0.1)

xs = linspace(12, 31, 100)
gain = np.vectorize (lambda x: s.integral (12, x)) (xs)

plot (xs, s(xs))
plot (xs, gain)

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] confused about univaritespline

[Neal Becker]

Pauli Virtanen wrote:

> 20.11.2012 21:11, Neal Becker kirjoitti:
>> import numpy as np
>> import matplotlib.pyplot as plt
>> 
>> ibo = np.array ((12, 14, 16, 18, 20, 22, 24, 26, 28, 29, 29.8, 30.2))
>> gain_deriv = np.array ((0, 0,  0,  0,   0, 0,  .2, .4, .5, .5, 0,-2))
>> 
>> import scipy.interpolate
>> s = scipy.interpolate.UnivariateSpline(ibo, gain_deriv, s=0.1)
>> 
>> xs = linspace(12, 31, 100)
>> gain = np.vectorize (lambda x: s.integral (12, x)) (xs)
>> 
>> plot (xs, s(xs))
>> plot (xs, gain)
> 
>>From fitpack/splint.f:
> 
> c   s(x) is considered to be identically zero outside
> c   the interval (t(k+1),t(n-k)).
> 
> Not documented in the Python wrapper, though.
> 

Ah, thanks!  Bitten one more by the fact that spline is not meant for 
extrapolation, which here I had done inadvertantly.

IMO, if spline isn't intended to be used to extrapolate, I'd prefer it to throw.

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] non-integer index misfeature?

[Neal Becker]

I think it's a misfeature that a floating point is silently accepted as an 
index.  I would prefer a warning for:

bins = np.arange (...)

for b in bins:
...
  w[b] = blah

when I meant:

for ib,b in enumerate (bins):
  w[ib] = blah

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] non-integer index misfeature?

[Neal Becker]

I'd be happy with disallowing floating point index at all.  I would think it 
was 
almost always a mistake.

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] ANN: NumPy 1.7.0rc1 release

[Neal Becker]

Are release notes available?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] phase unwrapping (1d)

[Neal Becker]

np.unwrap was too slow, so I rolled by own (in c++).

I wanted to be able to handle the case of

unwrap (arg (x1) + arg (x2))

Here, phase can change by more than 2pi.

I came up with the following algorithm, any thoughts?

In the following, y is normally set to pi.
o points to output
i points to input
nint1 finds nearest integer

  value_t prev_o = init;
  for (; i != e; ++i, ++o) {
*o = cnt * 2 * y + *i;
value_t delta = *o - prev_o;

if (delta / y > 1 or delta / y < -1) {
  int i = nint1 (delta / (2*y));
  *o -= 2*y*i;
  cnt -= i;
}

prev_o = *o;
  }


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] phase unwrapping (1d)

[Neal Becker]

Nadav Horesh wrote:

> There is an unwrap function in numpy. Doesn't it work for you?
> 

Like I had said, np.unwrap was too slow.  Profiling showed it eating up an 
absurd proportion of time.  My c++ code was much better (although still 
surprisingly slow).

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] phase unwrapping (1d)

[Neal Becker]

This code should explain all:

import numpy as np
arg = np.angle

def nint (x):
return int (x + 0.5) if x >= 0 else int (x - 0.5)

def unwrap (inp, y=np.pi, init=0, cnt=0):
o = np.empty_like (inp)
prev_o = init
for i in range (len (inp)):
o[i] = cnt * 2 * y + inp[i]
delta = o[i] - prev_o

if delta / y > 1 or delta / y < -1:
n = nint (delta / (2*y))
o[i] -= 2*y*n
cnt -= n

prev_o = o[i]

return o


u = np.linspace (0, 400, 100) * np.pi/100
v = np.cos (u) + 1j * np.sin (u)
plot (arg(v))
plot (arg(v) + arg (v))
plot (unwrap (arg (v)))
plot (unwrap (arg (v) + arg (v)))
---

Pierre Haessig wrote:

> Hi Neal,
> 
> Le 11/01/2013 16:40, Neal Becker a écrit :
>> I wanted to be able to handle the case of
>>
>> unwrap (arg (x1) + arg (x2))
>>
>> Here, phase can change by more than 2pi.
> It's not clear to me what you mean by "change more than 2pi" ? Do you
> mean that the consecutive points of in input can increase by more than
> 2pi ? If that's the case, I feel like there is no a priori information
> in the data to detect such a "giant leap".
> 
> Also, I copy-paste here for reference the numpy.wrap code from [1] :
> 
> def unwrap(p, discont=pi, axis=-1):
> p = asarray(p)
> nd = len(p.shape)
> dd = diff(p, axis=axis)
> slice1 = [slice(None, None)]*nd # full slices
> slice1[axis] = slice(1, None)
> ddmod = mod(dd+pi, 2*pi)-pi
> _nx.copyto(ddmod, pi, where=(ddmod==-pi) & (dd > 0))
> ph_correct = ddmod - dd;
> _nx.copyto(ph_correct, 0, where=abs(dd) up = array(p, copy=True, dtype='d')
> up[slice1] = p[slice1] + ph_correct.cumsum(axis)
> return up
> 
> I don't know why it's too slow though. It looks well vectorized.
> 
> Coming back to your C algorithm, I'm not C guru so that I don't have a
> clear picture of what it's doing. Do you have a Python prototype ?
> 
> Best,
> Pierre
> 
> [1]
> https://github.com/numpy/numpy/blob/master/numpy/lib/function_base.py#L1117


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] find points unique within some epsilon

[Neal Becker]

Any suggestion how to take a 2d complex array and find the set of points that 
are unique within some tolerance?  (My preferred metric here would be Euclidean 
distance)

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] another little index puzzle

[Neal Becker]

I have an array to be used for indexing.  It is 2d, where the rows are all the 
permutations of some numbers.  So:

array([[-2, -2, -2],
   [-2, -2, -1],
   [-2, -2,  0],
   [-2, -2,  1],
   [-2, -2,  2],
 ...
   [ 2,  1,  2],
   [ 2,  2, -2],
   [ 2,  2, -1],
   [ 2,  2,  0],
   [ 2,  2,  1],
   [ 2,  2,  2]])

Here the array is 125x3

I want to select all the rows of the array in which all the 3 elements are 
equal, so I can remove them.  So for example, the 1st and last row.



___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] ANN: NumPy 1.7.0 release

[Neal Becker]

Is there a way to add '-march=native' flag to gcc for the build?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] confused about tensordot

[Neal Becker]

In the following code

c = np.multiply (a, b.conj())
d = np.abs (np.sum (c, axis=0)/rows)

d2 = np.abs (np.tensordot (a, b.conj(), ((0,),(0,)))/rows)
print a.shape, b.shape, d.shape, d2.shape
 
The 1st compute steps, where I do multiply and then sum give the result I want.

I wanted to try to combine these 2 steps using tensordot, but it's not
doing what I want.

The print statement outputs this:

(1004, 13) (1004, 13) (13,) (13, 13)

The correct output should be (13,), but the tensordot output is (13,13).

It's supposed to take 2 matrixes, each (1004, 13) and do element-wise multiply, 
then sum over axis 0.

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] confused about tensordot

[Neal Becker]

Bradley M. Froehle wrote:

> Hi Neal:
> 
> The tensordot part:
>   np.tensordot (a, b.conj(), ((0,),(0,))
> 
> is returning a (13, 13) array whose [i, j]-th entry is   sum( a[k, i] *
> b.conj()[k, j] for k in xrange(1004) ).
> 
> -Brad
> 
> 
> The print statement outputs this:
>>
>> (1004, 13) (1004, 13) (13,) (13, 13)
>>
>> The correct output should be (13,), but the tensordot output is (13,13).
>>
>> It's supposed to take 2 matrixes, each (1004, 13) and do element-wise
>> multiply,
>> then sum over axis 0.
>>

Can I use tensordot to do what I want?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] FFTW bindings now implement numpy.fft interface

[Neal Becker]

Henry Gomersall wrote:

> Some of you may be interested in the latest release of my FFTW bindings.
> It can now serve as a drop in replacement* for numpy.fft and
> scipy.fftpack.
> 
> This means you can get most of the speed-up of FFTW with a one line code
> change or monkey patch existing libraries.
> 
> Lots of other goodness too of course.
> 
> Source here: https://github.com/hgomersall/pyFFTW
> pypi here: http://pypi.python.org/pypi/pyFFTW
> docs here: http://hgomersall.github.com/pyFFTW/
> 
> It's GPL3 due to license restrictions on FFTW. Get in touch if you want
> a different license and I'm sure we can reach an agreement ;)
> 
> Cheers,
> 
> Henry
> 
> *In the case where the input array is not a numpy array, it doesn't
> work. This was an oversight and will be fixed in the next release. That
> said, if you're converting from a list on every transform, you have
> better optimisations than using FFTW. One other small caveat in a corner
> case to do with repeated axes - described in the docs.

The 1st example says:
>>> import pyfftw
>>> import numpy
>>> a = pyfftw.n_byte_align_empty(128, 16, 'complex128')
>>> a[:] = numpy.random.randn(128) + 1j*numpy.random.randn(128)
>>> b = pyfftw.interfaces.numpy_fft.fft(a)

I don't see why I need to specify the alignment.  The fftw library has a 
function to allocate aligned arrays that are allocated optimally.  Why doesn't 
pyfft.n_byte_align_empty just align things correctly without me having to tell 
it the alignment?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] savetxt trouble

[Neal Becker]

I tried to save a vector as a csv, but it didn't work.

The vector is:
a[0,0]
array([-0.70710678-0.70710678j,  0.70710678+0.70710678j,
0.70710678-0.70710678j,  0.70710678+0.70710678j,
   -0.70710678-0.70710678j,  0.70710678-0.70710678j,
   -0.70710678+0.70710678j, -0.70710678+0.70710678j,
   -0.70710678-0.70710678j, -0.70710678-0.70710678j,
0.70710678-0.70710678j, -0.70710678-0.70710678j,
   -0.70710678+0.70710678j,  0.70710678-0.70710678j,
0.70710678-0.70710678j,  0.70710678+0.70710678j,
0.70710678-0.70710678j, -0.70710678-0.70710678j,
   -0.70710678-0.70710678j,  0.70710678-0.70710678j,
0.70710678+0.70710678j,  0.70710678+0.70710678j,
   -0.70710678+0.70710678j,  0.70710678+0.70710678j,
   -0.70710678-0.70710678j, -0.70710678+0.70710678j,
0.70710678-0.70710678j, -0.70710678+0.70710678j,
0.70710678+0.70710678j,  0.70710678+0.70710678j,
0.70710678-0.70710678j, -0.70710678-0.70710678j,
0.70710678-0.70710678j, -0.70710678-0.70710678j,
   -0.70710678+0.70710678j,  0.70710678+0.70710678j,
0.70710678-0.70710678j,  0.70710678-0.70710678j,
0.70710678+0.70710678j, -0.70710678+0.70710678j])

 np.savetxt ('test.out', a[0,0], delimiter=',')

The saved txt file says:
 (-7.071067811865540120e-01+-7.071067811865411334e-01j)
 (7.071067811865535679e-01+7.071067811865415775e-01j)
 (7.071067811865422437e-01+-7.071067811865529018e-01j)
 (7.071067811865520136e-01+7.071067811865431318e-01j)
 ...

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] savetxt trouble

[Neal Becker]

Robert Kern wrote:

> On Wed, Feb 20, 2013 at 1:25 PM, Neal Becker  wrote:
>> I tried to save a vector as a csv, but it didn't work.
>>
>> The vector is:
>> a[0,0]
>> array([-0.70710678-0.70710678j,  0.70710678+0.70710678j,
>> 0.70710678-0.70710678j,  0.70710678+0.70710678j,
> ...
>>  np.savetxt ('test.out', a[0,0], delimiter=',')
>>
>> The saved txt file says:
>>  (-7.071067811865540120e-01+-7.071067811865411334e-01j)
>>  (7.071067811865535679e-01+7.071067811865415775e-01j)
>>  (7.071067811865422437e-01+-7.071067811865529018e-01j)
>>  (7.071067811865520136e-01+7.071067811865431318e-01j)
>>  ...
> 
> What were you expecting? A single row? savetxt() always writes out
> len(arr) rows. Reshape your vector into a (1,N) array if you want a
> single row.
> 

Ah, thanks!

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Adopt Mersenne Twister 64bit?

[Neal Becker]

Nathaniel Smith wrote:

> On Tue, Mar 12, 2013 at 9:25 PM, Nathaniel Smith  wrote:
>> On Mon, Mar 11, 2013 at 9:46 AM, Robert Kern  wrote:
>>> On Sun, Mar 10, 2013 at 6:12 PM, Siu Kwan Lam  wrote:
 My suggestion to overcome (1) and (2) is to allow the user to select
 between the two implementations (and possibly different algorithms in the
 future). If user does not provide a choice, we use the MT19937-32 by
 default.

 numpy.random.set_state("MT19937_64", …)   # choose the 64-bit
 implementation
>>>
>>> Most likely, the different PRNGs should be different subclasses of
>>> RandomState. The module-level convenience API should probably be left
>>> alone. If you need to control the PRNG that you are using, you really
>>> need to be passing around a RandomState instance and not relying on
>>> reseeding the shared global instance.
>>
>> +1
>>
>>> Aside: I really wish we hadn't
>>> exposed `set_state()` in the module API. It's an attractive nuisance.
>>
>> And our own test suite is a serious offender in this regard, we have
>> tests that fail if you run the test suite in a non-default order...
>>   https://github.com/numpy/numpy/issues/347
>>
>> I wonder if we dare deprecate it? The whole idea of a global random
>> state is just a bad one, like every other sort of global shared state.
>> But it's one that's deeply baked into a lot of scientific programmers
>> expectations about how APIs work...
> 
> (To be clear, by 'it' here I meant np.random.set_seed(), not the whole
> np.random API. Probably. And by 'deprecate' I mean 'whine loudly in
> some fashion when people use it', not 'rip out in a few releases'. I
> think.)
> 
> -n

What do you mean that the idea of global shared state is a bad one?  How would 
you prefer the API to look?  An alternative is a stateless rng, where you have 
to pass it it's state on each invocation, which it would update and return.  I 
hope you're not advocating that. 

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Adopt Mersenne Twister 64bit?

[Neal Becker]

I guess I talked to you about 100 years ago about sharing state between numpy 
rng and code I have in c++ that wraps boost::random.  So is there a C-api for 
this RandomState object I could use to call from c++?  Maybe I could do 
something with that.

The c++ code could invoke via the python api, but that might be slower.  I'm 
just rambling here, I'd have to see the API to get some ideas.

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Adopt Mersenne Twister 64bit?

[Neal Becker]

Neal Becker wrote:

> I guess I talked to you about 100 years ago about sharing state between numpy
> rng and code I have in c++ that wraps boost::random.  So is there a C-api for
> this RandomState object I could use to call from c++?  Maybe I could do
> something with that.
> 
> The c++ code could invoke via the python api, but that might be slower.  I'm
> just rambling here, I'd have to see the API to get some ideas.

I think if I could just grab a long int from the underlying mersenne twister, 
through some c api?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] can't find doc for as_strided

[Neal Becker]

Visiting http://docs.scipy.org/doc/numpy/reference/, as search for

as_strided

or

stride_tricks

shows nothing (useful).

For that matter, I don't see a reference to numpy.lib.

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] cPickle.load says 'invalid signature'

[Neal Becker]

The file is pickle saved on i386 and loaded on x86_64.  It contains a numpy 
array (amoungst other things).

On load it says:

RuntimeError: invalid signature

Is binary format not portable?


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] New functions.

[Neal Becker]

Short-circuiting find would be nice.  Right now, to 'find' something you first 
make a bool array, then iterate over it.  If all you want is the first index 
where x[i] = e, not very efficient.

What I just described is a find with a '==' predicate.  Not sure if it's 
worthwhile to consider other predicates.

Maybe call it 'find_first'

Mark Miller wrote:

> I'd love to see something like a "count_unique" function included. The
> numpy.unique function is handy, but it can be a little awkward to
> efficiently go back and get counts of each unique value after the
> fact.
> 
> -Mark
> 
> 
> 
> On Wed, Jun 1, 2011 at 8:17 AM, Keith Goodman  wrote:
>> On Tue, May 31, 2011 at 8:41 PM, Charles R Harris
>>  wrote:
>>> On Tue, May 31, 2011 at 8:50 PM, Bruce Southey  wrote:
>>
 How about including all or some of Keith's Bottleneck package?
 He has tried to include some of the discussed functions and tried to
 make them very fast.
>>>
>>> I don't think they are sufficiently general as they are limited to 2
>>> dimensions. However, I think the moving filters should go into scipy, either
>>> in ndimage or maybe signals. Some of the others we can still speed of
>>> significantly, for instance nanmedian, by using the new functionality in
>>> numpy, i.e., numpy sort has worked with nans for a while now. It looks like
>>> call overhead dominates the nanmax times for small arrays and this might
>>> improve if the ufunc machinery is cleaned up a bit more, I don't know how
>>> far Mark got with that.
>>
>> Currently Bottleneck accelerates 1d, 2d, and 3d input. Anything else
>> falls back to a slower, non-cython version of the function. The same
>> goes for int32, int64, float32, float64.
>>
>> It should not be difficult to extend to higher nd and more dtypes
>> since everything is generated from template. The problem is that there
>> would be a LOT of cython auto-generated C code since there is a
>> separate function for each ndim, dtype, axis combination.
>>
>> Each of the ndim, dtype, axis functions currently has its own copy of
>> the algorithm (such as median). Pulling that out and reusing it should
>> save a lot of trees by reducing the auto-generated C code size.
>>
>> I recently added a partsort and argpartsort.
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] reinterpret complex <-> float

[Neal Becker]

Can I arrange to reinterpret an array of complex of length N as an array of 
float of length 2N, and vice-versa?  If so, how?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] fast numpy i/o

[Neal Becker]

I'm wondering what are good choices for fast numpy array serialization?

mmap: fast, but I guess not self-describing?
hdf5: ?
pickle: self-describing, but maybe not fast?
others?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] fast numpy i/o

[Neal Becker]

Neal Becker wrote:

> I'm wondering what are good choices for fast numpy array serialization?
> 
> mmap: fast, but I guess not self-describing?
> hdf5: ?
> pickle: self-describing, but maybe not fast?
> others?

I think, in addition, that hdf5 is the only one that easily interoperates with 
matlab?

speaking of hdf5, I see:

pyhdf5io  0.7 - Python module containing high-level hdf5 load and save 
functions.
h5py  2.0.0 - Read and write HDF5 files from Python

Any thoughts on the relative merits of these?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] write access through iterator

[Neal Becker]

Maybe I'm being dense today, but I don't see how to iterate over arrays with 
write access.  You could read through iterators like:

fl = u.flat
>>> for item in fl:
... print item

but you can't do

for item in fl:
  item = 10

(or, it won't do what you want).

Is there any way to do this?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] A bit of advice?

[Neal Becker]

I have a set of experiments that I want to plot.  There will be many plots.  
Each will show different test conditions.

Suppose I put each of the test conditions and results into a recarray.  The 
recarray could be:

arr = np.empty ((#experiments,), dtype=[('x0',int), ('x1',int), ('y0',int)]

where x0,x1 are 2 test conditions, and y0 is a result.

First I want to group the plots such according to the test conditions.  So, I 
want to first find what all the combinations of test conditions are.

Dunno if there is anything simpler than:

cases = tuple (set ((e['x0'], e['x1'])) for e in arr)

Next, need to select all those experiments which match each of these cases.  
Now 
I know of no easy way.

Any suggestions?  Perhaps I should look again at pytables?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] A bit of advice?

[Neal Becker]

Olivier Delalleau wrote:

> What about :
> dict((k, [e for e in arr if (e['x0'], e['x1']) == k]) for k in cases)
> ?

Not bad!  Thanks!

BTW, is there an easier way to get the unique keys, then this:

cases = tuple (set (tuple((e['a'],e['b'])) for e in u))

> 
> (note: it is inefficient written this way though)
> 
> -=- Olivier
> 
> 2011/6/23 Neal Becker 
> 
>> I have a set of experiments that I want to plot.  There will be many plots.
>> Each will show different test conditions.
>>
>> Suppose I put each of the test conditions and results into a recarray.  The
>> recarray could be:
>>
>> arr = np.empty ((#experiments,), dtype=[('x0',int), ('x1',int), ('y0',int)]
>>
>> where x0,x1 are 2 test conditions, and y0 is a result.
>>
>> First I want to group the plots such according to the test conditions.  So,
>> I
>> want to first find what all the combinations of test conditions are.
>>
>> Dunno if there is anything simpler than:
>>
>> cases = tuple (set ((e['x0'], e['x1'])) for e in arr)
>>
>> Next, need to select all those experiments which match each of these cases.
>>  Now
>> I know of no easy way.
>>
>> Any suggestions?  Perhaps I should look again at pytables?
>>
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] A bit of advice?

[Neal Becker]

josef.p...@gmail.com wrote:

> On Thu, Jun 23, 2011 at 8:20 AM, Neal Becker  wrote:
>> Olivier Delalleau wrote:
>>
>>> What about :
>>> dict((k, [e for e in arr if (e['x0'], e['x1']) == k]) for k in cases)
>>> ?
>>
>> Not bad!  Thanks!
>>
>> BTW, is there an easier way to get the unique keys, then this:
>>
>> cases = tuple (set (tuple((e['a'],e['b'])) for e in u))
> 
> I think you can just combine these 2
> 
> experiments = defaultdict([])  #syntax ?
> 
> for i, e in enumerate(arr):
> experiments[tuple((e['a'],e['b']))].append(i)
> #experiments[tuple((e['a'],e['b']))].append(y['c'])  #or just
> summarize results
> 
> experiments.keys()  #uniques
> 
> (just typed not checked)
> 
> Josef
> 
Yes, thanks, this works quite nicely:
experiments = defaultdict(list)
for e in arr:
 experiments[tuple((e['a'],e['b']))].append(e)


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Pleasantly surprised by recarray

[Neal Becker]

I was pleasantly surprised to find that recarrays can have a recursive 
structure, and can be defined using a nice syntax:

dtype=[('deltaf', float),('eq', bool),('filt', 
[('pulse', 'a10'), ('alpha', float)])]

However, this I discovered just by guessing.  It would be good to mention this 
in the reference doc.

BTW, one unfortunate side effect of using recarrays with elements that are not 
simple types is that identity comparison seems to fail.  That is, when 
addressing my previous problem

for e in arr:
 experiments[tuple((e['a'],e['b']))].append(e)

That would apparantly need to be:

for e in array:
  experiments[e['deltaf'], tuple(e['filt'])].append(e)

Unless e['filt'] is converted to tuple, it's type is:
 type(cases[0][4])
Out[55]: 

which does not seem to identity compare as I would want.

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] feedback request: proposal to add masks to the core ndarray

[Neal Becker]

Just 1 question before I look more closely.  What is the cost to the non-MA 
user 
of this addition?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] review request: introductory datetime documentation

[Neal Becker]

Just trying it out with 1.6:

 np.datetime64('now')
Out[6]: 2011-07-01 00:00:00

Well the time now is 07:01am.  Is this expected behaviour?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] custom atlas

[Neal Becker]

I thought I'd try to speed up numpy on my fedora system by rebuilding the atlas 
package so it would be tuned for my machine.  But when I do:

rpmbuild -ba -D 'enable_native_atlas 1' atlas.spec 

it fails with:

res/zgemvN_5000_100 : VARIATION EXCEEDS TOLERENCE, RERUN WITH HIGHER REPS.

A bit of googling has not revealed a solution.  Any hints?


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] custom atlas

[Neal Becker]

Charles R Harris wrote:

> On Tue, Jul 5, 2011 at 7:45 AM, Neal Becker  wrote:
> 
>> I thought I'd try to speed up numpy on my fedora system by rebuilding the
>> atlas
>> package so it would be tuned for my machine.  But when I do:
>>
>> rpmbuild -ba -D 'enable_native_atlas 1' atlas.spec
>>
>> it fails with:
>>
>> res/zgemvN_5000_100 : VARIATION EXCEEDS TOLERENCE, RERUN WITH HIGHER REPS.
>>
>> A bit of googling has not revealed a solution.  Any hints?
>>
>>
>>
> I've never seen that, OTOH, I haven't built ATLAS in the last few years. Do
> you have all the power saving/frequency changing options turned off? What
> version of ATLAS are you using? What CPU?
> 
> Chuck

Ah, hadn't tried turing off cpuspeed.  Try again... nope same error.

2 cpus, each:
model name  : Intel(R) Core(TM)2 Duo CPU T7500  @ 2.20GHz
stepping: 11
cpu MHz : 800.000  << that's what it says @idle
cache size  : 4096 KB


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] custom atlas

[Neal Becker]

Charles R Harris wrote:

> On Tue, Jul 5, 2011 at 8:37 AM, Charles R Harris
> wrote:
> 
>>
>>
>> On Tue, Jul 5, 2011 at 8:13 AM, Neal Becker  wrote:
>>
>>> Charles R Harris wrote:
>>>
>>> > On Tue, Jul 5, 2011 at 7:45 AM, Neal Becker 
>>> wrote:
>>> >
>>> >> I thought I'd try to speed up numpy on my fedora system by rebuilding
>>> the
>>> >> atlas
>>> >> package so it would be tuned for my machine.  But when I do:
>>> >>
>>> >> rpmbuild -ba -D 'enable_native_atlas 1' atlas.spec
>>> >>
>>> >> it fails with:
>>> >>
>>> >> res/zgemvN_5000_100 : VARIATION EXCEEDS TOLERENCE, RERUN WITH HIGHER
>>> REPS.
>>> >>
>>> >> A bit of googling has not revealed a solution.  Any hints?
>>> >>
>>> >>
>>> >>
>>> > I've never seen that, OTOH, I haven't built ATLAS in the last few years.
>>> Do
>>> > you have all the power saving/frequency changing options turned off?
>>> What
>>> > version of ATLAS are you using? What CPU?
>>> >
>>> > Chuck
>>>
>>> Ah, hadn't tried turing off cpuspeed.  Try again... nope same error.
>>>
>>> 2 cpus, each:
>>> model name  : Intel(R) Core(TM)2 Duo CPU T7500  @ 2.20GHz
>>> stepping: 11
>>> cpu MHz : 800.000  << that's what it says @idle
>>>
>>
>> You haven't got cpu frequency scaling under control. Linux? Depending on
>> the distro you can write to a file in /sys (for each cpu) or run a program
>> to make the setting, or click on a panel applet. Sometimes the scaling is
>> set in the bios also. Google is your friend here. I have
>>
>> $charris@f13 ~$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
>> ondemand
>>
>> And what you want to see is performance instead of ondemand.
>>
>>
> Here's some good info <http://tinyurl.com/o8o7b>.
> 
> Chuck

Thanks!  Good info.  But same result.

# service cpuspeed stop
# echo performance > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
# echo performance > /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor

build stopped exactly same as before.

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] NA/Missing Data Conference Call Summary

[Neal Becker]

Christopher Barker wrote:

> Dag Sverre Seljebotn wrote:
>> Here's an HPC perspective...:
> 
>> At least I feel that the transparency of NumPy is a huge part of its
>> current success. Many more than me spend half their time in C/Fortran
>> and half their time in Python.
> 
> Absolutely -- and this point has been raised a couple times in the
> discussion, so I hope it is not forgotten.
> 
>   > I tend to look at NumPy this way: Assuming you have some data in memory
>> (possibly loaded by a C or Fortran library). (Almost) no matter how it
>> is allocated, ordered, packed, aligned -- there's a way to find strides
>> and dtypes to put a nice NumPy wrapper around it and use the memory from
>> Python.
> 
> and vice-versa -- Assuming you have some data in numpy arrays, there's a
> way to process it with a C or Fortran library without copying the data.
> 
> And this is where I am skeptical of the bit-pattern idea -- while one
> can expect C and fortran and GPU, and ??? to understand NaNs for
> floating point data, is there any support in compilers or hardware for
> special bit patterns for NA values to integers? I've never seen in my
> (very limited experience).
> 
> Maybe having the mask option, too, will make that irrelevant, but I want
> to be clear about that kind of use case.
> 
> -Chris

Am I the only one that finds the idea of special values of things like int[1] 
to 
have special meanings to be really ugly?

[1] which already have defined behavior over their entire domain of bit patterns

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Warning: invalid value encountered in divide

[Neal Becker]

Warning: invalid value encountered in divide

No traceback.  How can I get more info on this?  Can this warning be converted 
to an exception so I can get a trace?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] nditer confusion

[Neal Becker]

There'a a boatload of options for nditer.  I need a simple explanation, maybe a 
few simple examples.  Is there anything that might help?

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] wierd numpy.void behavior

[Neal Becker]

I've encountered something weird about numpy.void.

arr = np.empty ((len(results),), dtype=[('deltaf', float),
('quantize', [('int', int), ('frac', 
int)])])

for i,r in enumerate (results):
arr[i] = (r[0]['deltaf'],
  tuple(r[0]['quantize_mf']))


from collections import defaultdict, namedtuple
experiments = defaultdict(list)

testcase = namedtuple ('testcase', ['quantize'])

for e in arr:
experiments[testcase(e['quantize'])].append (e)

Now it seems that when e['quantize'] is used as a dictionary key, equal values 
are not compared as equal:

In [36]: experiments
Out[36]: defaultdict(, {testcase(quantize=(0, 0)): [(1.25, (0, 
0))], testcase(quantize=(0, 0)): [(1.25, (0, 0))], testcase(quantize=(0, 0)): 
[(1.25, (0, 0))]})

See, there are 3 'testcases' inserted, all with keys quantize=(0,0).

In [37]: e['quantize']
Out[37]: (0, 0)

In [38]: type(e['quantize'])
Out[38]: 

There's something weird here.  If instead I do:

for e in arr:
experiments[testcase(tuple(e['quantize']))].append (e)

that is, convert e['quantize'] to a tuple before using it as a key, I get the 
expected behavior:

In [40]: experiments
Out[40]: defaultdict(, {testcase(quantize=(0, 0)): [(1.25, (0, 
0)), 
(1.25, (0, 0)), (1.25, (0, 0))]})


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion