Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

Antoine Pitrou wrote: > When writing C code to interact with buffer-providing objects, you > usually don't bother with memoryviews at all. You just use a Py_buffer > structure. I was taking about "typed memoryviews" which is a Cython abstraction for a Py_buffer struct. I was not taking about Py

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

On Thu, 12 May 2016 23:14:36 + (UTC) Sturla Molden wrote: > Antoine Pitrou wrote: > > > Can you define "expensive"? > > Slow enough to cause complaints on the Cython mailing list. What kind of complaints? Please be specific. > > Buffer acquisition itself only calls a single C callback and

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

Matěj Týč wrote: > Does it mean > that if you pass the numpy array to the child process using Queue, no > significant amount of data will flow through it? This is what my shared memory arrayes do. > Or I shouldn't pass it > using Queue at all and just rely on inheritance? This is what David

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

On 17.5.2016 14:13, Sturla Molden wrote: > Matěj Týč wrote: > >> - Parallel processing of HUGE data, and > This is mainly a Windows problem, as copy-on-write fork() will solve this > on any other platform. ... That sounds interesting, could you elaborate on it a bit? Does it mean that if you pas

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

Matěj Týč wrote: > - Parallel processing of HUGE data, and This is mainly a Windows problem, as copy-on-write fork() will solve this on any other platform. I am more in favor of asking Microsoft to fix their broken OS. Also observe that the usefulness of shared memory is very limited on Wind

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

On 11.5.2016 10:29, Sturla Molden wrote: > I did some work on this some years ago. ... > I am sorry, I have missed this discussion when it started. There are two cases when I had feeling that I had to use this functionality: - Parallel processing of HUGE data, and - using parallel processing i

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

Feng Yu wrote: > Also, did you checkout http://zeromq.org/blog:zero-copy ? > ZeroMQ is a dependency of Jupyter, so it is quite available. ZeroMQ is great, but it lacks some crucial features. In particular it does not support IPC on Windows. Ideally one should e.g. use Unix doman sockets on Linux

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

> > Personally I prefer a parallel programming style with queues – either to > scatter arrays to workers and collecting arrays from workers, or to chain > workers together in a pipeline (without using coroutines). But exactly how > you program is a matter of taste. I want to make it as inexpensive

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

Niki Spahiev wrote: > Apparently next Win10 will have fork as part of bash integration. It is Interix/SUA rebranded "Subsystem for Linux". It remains to be seen how long it will stay this time. Also a Python built for this subsystem will not run on the Win32 subsystem, so there is no graphics. A

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

Feng Yu wrote: > In most (half?) situations the result can be directly write back via > preallocated shared array before works are spawned. Then there is no > need to pass data back with named segments. You can work around it in various ways, this being one of them. Personally I prefer a paral

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

Antoine Pitrou pitrou.net> writes: > > On Thu, 12 May 2016 06:27:43 + (UTC) > Sturla Molden gmail.com> wrote: > > > Allan Haldane gmail.com> wrote: > > > > > You probably already know this, but I just wanted to note that the > > > mpi4py module has worked around pickle too. They discuss

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

Niki Spahiev wrote: > Apparently next Win10 will have fork as part of bash integration. That would be great. The lack of fork on Windows is very annoying. Sturla ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mail

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

Antoine Pitrou wrote: > Can you define "expensive"? Slow enough to cause complaints on the Cython mailing list. > You're assuming this is the cost of "buffer acquisition", while most > likely it's the cost of creating the memoryview object itself. Constructing a typed memoryview from a typed m

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

> Again, not everyone uses Unix. > > And on Unix it is not trival to pass data back from the child process. I > solved that problem with Sys V IPC (pickling the name of the segment). > I wonder if it is neccessary insist being able to pass large amount of data back from child to the parent process

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

On Thu, 12 May 2016 06:27:43 + (UTC) Sturla Molden wrote: > Allan Haldane wrote: > > > You probably already know this, but I just wanted to note that the > > mpi4py module has worked around pickle too. They discuss how they > > efficiently transfer numpy arrays in mpi messages here: > > htt

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

On 12.05.2016 02:02, Sturla Molden wrote: Feng Yu wrote: 1. If we are talking about shared memory and copy-on-write inheritance, then we are using 'fork'. Not available on Windows. On Unix it only allows one-way communication, from parent to child. Apparently next Win10 will have fork as p

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

Allan Haldane wrote: > You probably already know this, but I just wanted to note that the > mpi4py module has worked around pickle too. They discuss how they > efficiently transfer numpy arrays in mpi messages here: > http://pythonhosted.org/mpi4py/usrman/overview.html#communicating-python-object

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

On 05/11/2016 06:48 PM, Sturla Molden wrote: > Elliot Hallmark wrote: >> Strula, this sounds brilliant! To be clear, you're talking about >> serializing the numpy array and reconstructing it in a way that's faster >> than pickle? > > Yes. We know the binary format of NumPy arrays. We don't need

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

On 05/11/2016 06:39 PM, Joe Kington wrote: > > > In python2 it appears that multiprocessing uses pickle protocol 0 which > must cause a big slowdown (a factor of 100) relative to protocol 2, and > uses pickle instead of cPickle. > > > Even on Python 2.x, multiprocessing uses protoco

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

Feng Yu wrote: > 1. If we are talking about shared memory and copy-on-write > inheritance, then we are using 'fork'. Not available on Windows. On Unix it only allows one-way communication, from parent to child. > 2. Picking of inherited shared memory array can be done minimally by > just picki

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

Joe Kington wrote: > You're far better off just > communicating between processes as opposed to using shared memory. Yes. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

Benjamin Root wrote: > Oftentimes, if one needs to share numpy arrays for multiprocessing, I would > imagine that it is because the array is huge, right? That is a case for shared memory, but what. i was taking about is more common than this. In order for processes to cooperate, they must commu

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

Allan Haldane wrote: > That's interesting. I've also used multiprocessing with numpy and didn't > realize that. Is this true in python3 too? I am not sure. As you have noticed, pickle is faster by to orders of magnitude on Python 3. But several microseconds is also a lot, particularly if we are

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

Elliot Hallmark wrote: > Strula, this sounds brilliant! To be clear, you're talking about > serializing the numpy array and reconstructing it in a way that's faster > than pickle? Yes. We know the binary format of NumPy arrays. We don't need to invoke the machinery of pickle to serialize an arra

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

In python2 it appears that multiprocessing uses pickle protocol 0 which > must cause a big slowdown (a factor of 100) relative to protocol 2, and > uses pickle instead of cPickle. > > Even on Python 2.x, multiprocessing uses protocol 2, not protocol 0. The default for the `pickle` module changed,

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

Hi, I've been thinking and exploring this for some time. If we are to start some effort I'd like to help. Here are my comments, mostly regarding to Sturla's comments. 1. If we are talking about shared memory and copy-on-write inheritance, then we are using 'fork'. If we are free to use fork, then

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

Oftentimes, if one needs to share numpy arrays for multiprocessing, I would imagine that it is because the array is huge, right? So, the pickling approach would copy that array for each process, which defeats the purpose, right? Ben Root On Wed, May 11, 2016 at 2:01 PM, Allan Haldane wrote: > O

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

On 05/11/2016 04:29 AM, Sturla Molden wrote: > 4. The reason IPC appears expensive with NumPy is because multiprocessing > pickles the arrays. It is pickle that is slow, not the IPC. Some would say > that the pickle overhead is an integral part of the IPC ovearhead, but i > will argue that it is no

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

Strula, this sounds brilliant! To be clear, you're talking about serializing the numpy array and reconstructing it in a way that's faster than pickle? Or using shared memory and signaling array creation around that shared memory rather than using pickle? For what it's worth, I have used shared me

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

I did some work on this some years ago. I have more or less concluded that it was a waste of effort. But first let me explain what the suggested approach do not work. As it uses memory mapping to create shared memory (i.e. shared segments are not named), they must be created ahead of spawning proce

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

On Mon, Apr 11, 2016 at 5:39 AM, Matěj Týč wrote: > * ... I do see some value in providing a canonical right way to > construct shared memory arrays in NumPy, but I'm not very happy with > this solution, ... terrible code organization (with the global > variables): > * I understand that, however

[Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

Dear Numpy developers, I propose a pull request https://github.com/numpy/numpy/pull/7533 that features numpy arrays that can be shared among processes (with some effort). Why: In CPython, multiprocessing is the only way of how to exploit multi-core CPUs if your parallel code can't avoid creating P

Re: [Numpy-discussion] Numpy arrays vs typed memoryviews

I think I have said this before, but its worth a repeat: Pickle (including cPickle) is a slow hog! That might not be the overhead you see, you just haven't noticed it yet. I saw this some years ago when I worked on shared memory arrays for Numpy (cf. my account on Github). Shared memory really

Re: [Numpy-discussion] numpy arrays

On Sat, Jun 9, 2012 at 2:12 PM, bob tnur wrote: >> >> how to save  multiple files like cm1.txt,cm2.txt,cm3.txt etc and to >> produce their corresponding outputs cm1.out,cm2.out,cm3.out etc. > >    or how to modify this: >    np.savetxt(fname, (a,b), fmt='%4.8f') > You can save them to separate fi

Re: [Numpy-discussion] numpy arrays

> > Hi every body! > I have a &b numpy arrays > a=np.loadtxt('çm1.txt', dtype=np.float, skiprows=2,usecols=[1]) > b=np.loadtxt('çm1.txt', dtype=('x', np.float64), skiprows=2,usecols=[2]) > > how to save multiple files like cm1.txt,cm2.txt,cm3.txt etc and to > produce their corresponding outputs

Re: [Numpy-discussion] numpy arrays

How do you want the output files to be formatted? Binary data? Textual representation? This function can do both: http://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.tofile.html And numpy supports a variety of methods for outputting to files (and reading data back in): http://docs.s

Re: [Numpy-discussion] NumPy arrays that use memory allocated from other libraries or tools

On Wed, Sep 10, 2008 at 4:55 AM, Stéfan van der Walt <[EMAIL PROTECTED]> wrote: > 2008/9/10 Travis E. Oliphant <[EMAIL PROTECTED]>: >> The post is >> >> http://blog.enthought.com/?p=62 > > Very cool post, thank you! I agree that it would be great to have > such a mechanism in NumPy. > And then bu

Re: [Numpy-discussion] NumPy arrays that use memory allocated from other libraries or tools

2008/9/10 Travis E. Oliphant <[EMAIL PROTECTED]>: > The post is > > http://blog.enthought.com/?p=62 Very cool post, thank you! I agree that it would be great to have such a mechanism in NumPy. Cheers Stéfan ___ Numpy-discussion mailing list Numpy-discu

[Numpy-discussion] NumPy arrays that use memory allocated from other libraries or tools

I wanted to point anybody interested to a blog post that describes a useful pattern for having a NumPy array that points to the memory created by a different memory manager than the standard one used by NumPy. The pattern shows how to create a NumPy array that points to previously allocated

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

On 8/9/07, Stefan van der Walt <[EMAIL PROTECTED]> wrote: > > It doesn't really matter where the memory allocation occurs, does it? > As far as I understand, the underlying fftw function has some flag to > indicate when the data is aligned. If so, we could expose that flag > in Python, and do som

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

On 8/9/07, Charles R Harris <[EMAIL PROTECTED]> wrote: > > > On 8/9/07, David Cournapeau <[EMAIL PROTECTED]> wrote: > > Charles R Harris wrote: > > > > > > Ah, you want it in C. > > What would be the use to get SIMD aligned arrays in python ? > > If I wanted a fairly specialized routine and didn't

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

On Thu, Aug 09, 2007 at 04:52:38PM +0900, David Cournapeau wrote: > Charles R Harris wrote: > > > > Well, what you want might be very easy to do in python, we just need > > to check the default alignments for doubles and floats for some of the > > other compilers, architectures, and OS's out ther

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

On 8/9/07, David Cournapeau <[EMAIL PROTECTED]> wrote: > > Charles R Harris wrote: > > > > Ah, you want it in C. > What would be the use to get SIMD aligned arrays in python ? If I wanted a fairly specialized routine and didn't want to touch the guts of numpy, I would pass the aligned array to a

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

Charles R Harris wrote: > > Ah, you want it in C. What would be the use to get SIMD aligned arrays in python ? David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

On 8/9/07, David Cournapeau <[EMAIL PROTECTED]> wrote: > > Charles R Harris wrote: > > > > Well, what you want might be very easy to do in python, we just need > > to check the default alignments for doubles and floats for some of the > > other compilers, architectures, and OS's out there. On the o

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

On 8/9/07, Charles R Harris <[EMAIL PROTECTED]> wrote: > > > > On 8/8/07, David Cournapeau <[EMAIL PROTECTED]> wrote: > > > > Charles R Harris wrote: > > > Anne, > > > > > > On 8/8/07, *Anne Archibald* <[EMAIL PROTECTED] > > > > wrote: > > > > > > On 08/08/2007, Charles R Harris <[EMAIL PROTECT

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

Charles R Harris wrote: > > Well, what you want might be very easy to do in python, we just need > to check the default alignments for doubles and floats for some of the > other compilers, architectures, and OS's out there. On the other hand, > you might not be able to request a c malloc that is

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

On 8/8/07, David Cournapeau <[EMAIL PROTECTED]> wrote: > > Charles R Harris wrote: > > Anne, > > > > On 8/8/07, *Anne Archibald* <[EMAIL PROTECTED] > > > wrote: > > > > On 08/08/2007, Charles R Harris <[EMAIL PROTECTED] > > > wrote: > >

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

Charles R Harris wrote: > Anne, > > On 8/8/07, *Anne Archibald* <[EMAIL PROTECTED] > > wrote: > > On 08/08/2007, Charles R Harris <[EMAIL PROTECTED] > > wrote: > > > > > > On 8/8/07, Anne Archibald <[EMAIL PROTECTED] >

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

On 8/8/07, Matthieu Brucher <[EMAIL PROTECTED]> wrote: > > My 64 bit linux on Intel aligns arrays, whatever the data type, on 16 byte > > boundaries. It might be interesting to see what happens with the Intel and > > MSVC comipilers, but I expect similar results. > > > > According to the doc on the

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

> > My 64 bit linux on Intel aligns arrays, whatever the data type, on 16 byte > boundaries. It might be interesting to see what happens with the Intel and > MSVC comipilers, but I expect similar results. > According to the doc on the msdn, the data should be 16-bits aligned. Matthieu ___

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

Anne, On 8/8/07, Anne Archibald <[EMAIL PROTECTED]> wrote: > > On 08/08/2007, Charles R Harris <[EMAIL PROTECTED]> wrote: > > > > > > On 8/8/07, Anne Archibald <[EMAIL PROTECTED]> wrote: > > > Oh. Well, it's not *terrible*; it gets you an aligned array. But you > > > have to allocate the original

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

On 08/08/2007, Charles R Harris <[EMAIL PROTECTED]> wrote: > > > On 8/8/07, Anne Archibald <[EMAIL PROTECTED]> wrote: > > Oh. Well, it's not *terrible*; it gets you an aligned array. But you > > have to allocate the original array as a 1D byte array (to allow for > > arbitrary realignments) and the

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

On 8/8/07, Anne Archibald <[EMAIL PROTECTED]> wrote: > > On 08/08/2007, Stefan van der Walt <[EMAIL PROTECTED]> wrote: > > On Tue, Aug 07, 2007 at 01:33:24AM -0400, Anne Archibald wrote: > > > Well, it can be done in Python: just allocate a too-big ndarray and > > > take a slice that's the right sh

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

On 08/08/2007, Stefan van der Walt <[EMAIL PROTECTED]> wrote: > On Tue, Aug 07, 2007 at 01:33:24AM -0400, Anne Archibald wrote: > > Well, it can be done in Python: just allocate a too-big ndarray and > > take a slice that's the right shape and has the right alignment. But > > this sucks. > > Could

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

On Tue, Aug 07, 2007 at 01:33:24AM -0400, Anne Archibald wrote: > Well, it can be done in Python: just allocate a too-big ndarray and > take a slice that's the right shape and has the right alignment. But > this sucks. Could you explain to me why is this such a bad idea? Stéfan __

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

On 8/6/07, Anne Archibald <[EMAIL PROTECTED]> wrote: > > On 06/08/07, David Cournapeau <[EMAIL PROTECTED]> wrote: > > > Well, when I proposed the SIMD extension, I was willing to implement the > > proposal, and this was for a simple goal: enabling better integration > > with many numeric libraries

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

Anne Archibald wrote: > Well, it can be done in Python: just allocate a too-big ndarray and > take a slice that's the right shape and has the right alignment. But > this sucks. Stephen G. Johnson posted code earlier in this thread that > provides a portable aligned-memory allocator - it handles the

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

> > For platforms without posix_memalign, I don't see how to > implement a memory allocator with an arbitrary alignment (more > precisely, I don't see how to free it if I cannot assume a fixed > alignement: how do I know where the "real" pointer is ?). Visual Studio seems to offer a counter part

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

On 07/08/07, David Cournapeau <[EMAIL PROTECTED]> wrote: > Anne, you said previously that it was easy to allocate buffers for a > given alignment at runtime. Could you point me to a document which > explains how ? For platforms without posix_memalign, I don't see how to > implement a memory alloca

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

Anne Archibald wrote: > > I have to agree. I can hardly volunteer David for anything, and I > don't have time to implement this myself, but I think a custom > allocator is a rather special-purpose tool; if one were to implement > one, I think the way to go would be to implement a subclass of ndarra

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

On 06/08/07, David Cournapeau <[EMAIL PROTECTED]> wrote: > Well, when I proposed the SIMD extension, I was willing to implement the > proposal, and this was for a simple goal: enabling better integration > with many numeric libraries which need SIMD alignment. > > As nice as a custom allocator mig

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

Lisandro Dalcin wrote: > On 8/3/07, David Cournapeau <[EMAIL PROTECTED]> wrote: >> Here is what I can think of: >> - adding an API to know whether a given PyArrayObject has its data >> buffer 16 bytes aligned, and requesting a 16 bytes aligned >> PyArrayObject. Something like NPY_ALIGNED, ba

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

On 8/3/07, David Cournapeau <[EMAIL PROTECTED]> wrote: > Here is what I can think of: > - adding an API to know whether a given PyArrayObject has its data > buffer 16 bytes aligned, and requesting a 16 bytes aligned > PyArrayObject. Something like NPY_ALIGNED, basically. > - forcing da

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

On Aug 4, 3:24 am, "Anne Archibald" <[EMAIL PROTECTED]> wrote: > It seems to me two things are needed: > > * A mechanism for requesting numpy arrays with buffers aligned to an > arbitrary power-of-two size (basically just using posix_memalign or > some horrible hack on platforms that don't have it

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

On 04/08/07, David Cournapeau <[EMAIL PROTECTED]> wrote: > > Here's a hack that google turned up: I'd avoid hacks in favour of posix_memalign (which allows arbitrary degrees of alignment. For one thing, freeing becomes a headache (you can't free a pointer you've jiggered!). > - Check whethe

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

> > > Here's a hack that google turned up: > > (1) Use static variables instead of dynamic (stack) variables > (2) Use in-line assembly code that explicitly aligns data > (3) In C code, use "*malloc*" to explicitly allocate variables > > Here is Intel's example of (2): > >

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

On 8/3/07, Charles R Harris <[EMAIL PROTECTED]> wrote: > > > > On 8/3/07, David Cournapeau <[EMAIL PROTECTED]> wrote: > > > > Andrew Straw wrote: > > > Dear David, > > > > > > Both ideas, particularly the 2nd, would be excellent additions to > > numpy. > > > I often use the Intel IPP (Integrated Pe

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

On 8/3/07, David Cournapeau <[EMAIL PROTECTED]> wrote: > > Andrew Straw wrote: > > Dear David, > > > > Both ideas, particularly the 2nd, would be excellent additions to numpy. > > I often use the Intel IPP (Integrated Performance Primitives) Library > > together with numpy, but I have to do all my

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

Andrew Straw wrote: > Dear David, > > Both ideas, particularly the 2nd, would be excellent additions to numpy. > I often use the Intel IPP (Integrated Performance Primitives) Library > together with numpy, but I have to do all my memory allocation with the > IPP to ensure fastest operation. I th

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

Dear David, Both ideas, particularly the 2nd, would be excellent additions to numpy. I often use the Intel IPP (Integrated Performance Primitives) Library together with numpy, but I have to do all my memory allocation with the IPP to ensure fastest operation. I then create numpy views of the da

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

Hi, >Following an ongoing discussion with S. Johnson, one of the developer > of fftw3, I would be interested in what people think about adding > infrastructure in numpy related to SIMD alignement (that is 16 bytes > alignement for SSE/ALTIVEC, I don't know anything about other archs). > The pr

[Numpy-discussion] numpy arrays, data allocation and SIMD alignement

Hi, Following an ongoing discussion with S. Johnson, one of the developer of fftw3, I would be interested in what people think about adding infrastructure in numpy related to SIMD alignement (that is 16 bytes alignement for SSE/ALTIVEC, I don't know anything about other archs). The problem