Re: [Numpy-discussion] Byte aligned arrays

2012-12-21 Thread Francesc Alted
On 12/21/12 1:35 PM, Dag Sverre Seljebotn wrote: > On 12/20/2012 03:23 PM, Francesc Alted wrote: >> On 12/20/12 9:53 AM, Henry Gomersall wrote: >>> On Wed, 2012-12-19 at 19:03 +0100, Francesc Alted wrote: The only scenario that I see that this would create unaligned arrays is for mac

Re: [Numpy-discussion] Byte aligned arrays

2012-12-21 Thread Dag Sverre Seljebotn
On 12/20/2012 03:23 PM, Francesc Alted wrote: > On 12/20/12 9:53 AM, Henry Gomersall wrote: >> On Wed, 2012-12-19 at 19:03 +0100, Francesc Alted wrote: >>> The only scenario that I see that this would create unaligned arrays >>> is >>> for machines having AVX. But provided that the Intel architect

Re: [Numpy-discussion] Byte aligned arrays

2012-12-21 Thread Francesc Alted
On 12/21/12 11:58 AM, Henry Gomersall wrote: > On Fri, 2012-12-21 at 11:34 +0100, Francesc Alted wrote: >>> Also this convolution code: >>> https://github.com/hgomersall/SSE-convolution/blob/master/convolve.c >>> >>> Shows a small but repeatable speed-up (a few %) when using some >> aligned >>> loa

Re: [Numpy-discussion] Byte aligned arrays

2012-12-21 Thread Henry Gomersall
On Fri, 2012-12-21 at 11:34 +0100, Francesc Alted wrote: > > Also this convolution code: > > https://github.com/hgomersall/SSE-convolution/blob/master/convolve.c > > > > Shows a small but repeatable speed-up (a few %) when using some > aligned > > loads (as many as I can work out to use!). > > Oka

Re: [Numpy-discussion] Byte aligned arrays

2012-12-21 Thread Francesc Alted
On 12/20/12 7:35 PM, Henry Gomersall wrote: > On Thu, 2012-12-20 at 15:23 +0100, Francesc Alted wrote: >> On 12/20/12 9:53 AM, Henry Gomersall wrote: >>> On Wed, 2012-12-19 at 19:03 +0100, Francesc Alted wrote: The only scenario that I see that this would create unaligned >> arrays is >>>

Re: [Numpy-discussion] Byte aligned arrays

2012-12-20 Thread Henry Gomersall
On Thu, 2012-12-20 at 21:45 +0100, Sturla Molden wrote: > On 20.12.2012 21:24, Henry Gomersall wrote: > > > I didn't know that. It's a real pain having so many libc libs > knocking > > around. I have little experience of Windows, as you may have > guessed! > > Originally there was only one system

Re: [Numpy-discussion] Byte aligned arrays

2012-12-20 Thread Sturla Molden
On 20.12.2012 21:24, Henry Gomersall wrote: > I didn't know that. It's a real pain having so many libc libs knocking > around. I have little experience of Windows, as you may have guessed! Originally there was only one system-wide CRT on Windows (msvcrt.dll), which is why MinGW linkes with that

Re: [Numpy-discussion] Byte aligned arrays

2012-12-20 Thread Henry Gomersall
On Thu, 2012-12-20 at 21:13 +0100, Sturla Molden wrote: > On 20.12.2012 21:03, Henry Gomersall wrote: > > > Why is it important? (for my own understanding) > > Because if CRT resources are shared between different CRT versions, > bad > things will happen (the ABIs are not equivalent, errno and o

Re: [Numpy-discussion] Byte aligned arrays

2012-12-20 Thread Sturla Molden
On 20.12.2012 21:13, Sturla Molden wrote: > Because if CRT resources are shared between different CRT versions, bad > things will happen (the ABIs are not equivalent, errno and other globals > are at different addresses, etc.) For example, PyErr_SetFromErrno will return garbage if CRTs are shared

Re: [Numpy-discussion] Byte aligned arrays

2012-12-20 Thread Sturla Molden
On 20.12.2012 21:03, Henry Gomersall wrote: > Why is it important? (for my own understanding) Because if CRT resources are shared between different CRT versions, bad things will happen (the ABIs are not equivalent, errno and other globals are at different addresses, etc.) Cython code tends to s

Re: [Numpy-discussion] Byte aligned arrays

2012-12-20 Thread Henry Gomersall
On Thu, 2012-12-20 at 21:05 +0100, Sturla Molden wrote: > On 20.12.2012 20:57, Sturla Molden wrote: > > On 20.12.2012 20:52, Henry Gomersall wrote: > > > >> Perhaps the DLL should go and read MS's edicts! > > > > Do you link with same same CRT as Python? (msvcr90.dll) > > > > You should always use

Re: [Numpy-discussion] Byte aligned arrays

2012-12-20 Thread Sturla Molden
On 20.12.2012 20:57, Sturla Molden wrote: > On 20.12.2012 20:52, Henry Gomersall wrote: > >> Perhaps the DLL should go and read MS's edicts! > > Do you link with same same CRT as Python? (msvcr90.dll) > > You should always use -lmsvcr90. > > If you don't, you will link with msvcrt.dll. Here is VS2

Re: [Numpy-discussion] Byte aligned arrays

2012-12-20 Thread Henry Gomersall
On Thu, 2012-12-20 at 20:57 +0100, Sturla Molden wrote: > On 20.12.2012 20:52, Henry Gomersall wrote: > > > Perhaps the DLL should go and read MS's edicts! > > Do you link with same same CRT as Python? (msvcr90.dll) > > You should always use -lmsvcr90. > > If you don't, you will link with msvcr

Re: [Numpy-discussion] Byte aligned arrays

2012-12-20 Thread Sturla Molden
On 20.12.2012 20:52, Henry Gomersall wrote: > Perhaps the DLL should go and read MS's edicts! Do you link with same same CRT as Python? (msvcr90.dll) You should always use -lmsvcr90. If you don't, you will link with msvcrt.dll. Sturla ___ NumPy-Disc

Re: [Numpy-discussion] Byte aligned arrays

2012-12-20 Thread Henry Gomersall
On Thu, 2012-12-20 at 20:50 +0100, Sturla Molden wrote: > On 20.12.2012 18:38, Henry Gomersall wrote: > > > Except I build with MinGW. Please don't tell me I need to install > Visual > > Studio... I have about 1GB free on my windows partition! > > The same DLL is used as CRT. Perhaps the DLL sho

Re: [Numpy-discussion] Byte aligned arrays

2012-12-20 Thread Sturla Molden
On 20.12.2012 18:38, Henry Gomersall wrote: > Except I build with MinGW. Please don't tell me I need to install Visual > Studio... I have about 1GB free on my windows partition! The same DLL is used as CRT. Sturla ___ NumPy-Discussion mailing list Num

Re: [Numpy-discussion] Byte aligned arrays

2012-12-20 Thread Henry Gomersall
On Thu, 2012-12-20 at 15:23 +0100, Francesc Alted wrote: > On 12/20/12 9:53 AM, Henry Gomersall wrote: > > On Wed, 2012-12-19 at 19:03 +0100, Francesc Alted wrote: > >> The only scenario that I see that this would create unaligned > arrays > >> is > >> for machines having AVX. But provided that th

Re: [Numpy-discussion] Byte aligned arrays

2012-12-20 Thread Henry Gomersall
On Thu, 2012-12-20 at 17:48 +0100, Sturla Molden wrote: > On 19.12.2012 19:25, Henry Gomersall wrote: > > > That is not true at least under Windows 32-bit. I think also it's > not > > true for Linux 32-bit from my vague recollections of testing in a > > virtual machine. (disclaimer: both those sta

Re: [Numpy-discussion] Byte aligned arrays

2012-12-20 Thread Sturla Molden
On 20.12.2012 17:47, Henry Gomersall wrote: > On Thu, 2012-12-20 at 17:26 +0100, Sturla Molden wrote: >>return tmp[offset:offset+N]\ >> .view(dtype=d)\ >> .reshape(shape, order=order) > > Also, just for the email record, that should be > > retu

Re: [Numpy-discussion] Byte aligned arrays

2012-12-20 Thread Sturla Molden
On 19.12.2012 19:25, Henry Gomersall wrote: > That is not true at least under Windows 32-bit. I think also it's not > true for Linux 32-bit from my vague recollections of testing in a > virtual machine. (disclaimer: both those statements _may_ be out of > date). malloc is required to return memor

Re: [Numpy-discussion] Byte aligned arrays

2012-12-20 Thread Henry Gomersall
On Thu, 2012-12-20 at 17:26 +0100, Sturla Molden wrote: > return tmp[offset:offset+N]\ > .view(dtype=d)\ > .reshape(shape, order=order) Also, just for the email record, that should be return tmp[offset:offset+N*d.itemsize]\ .

Re: [Numpy-discussion] Byte aligned arrays

2012-12-20 Thread Henry Gomersall
On Thu, 2012-12-20 at 17:26 +0100, Sturla Molden wrote: > On 19.12.2012 09:40, Henry Gomersall wrote: > > I've written a few simple cython routines for assisting in creating > > byte-aligned numpy arrays. The point being for the arrays to work > with > > SSE/AVX code. > > > > https://github.com/hgo

Re: [Numpy-discussion] Byte aligned arrays

2012-12-20 Thread Sturla Molden
On 19.12.2012 09:40, Henry Gomersall wrote: > I've written a few simple cython routines for assisting in creating > byte-aligned numpy arrays. The point being for the arrays to work with > SSE/AVX code. > > https://github.com/hgomersall/pyFFTW/blob/master/pyfftw/utils.pxi Why use Cython? http://m

Re: [Numpy-discussion] Byte aligned arrays

2012-12-20 Thread Francesc Alted
On 12/20/12 9:53 AM, Henry Gomersall wrote: > On Wed, 2012-12-19 at 19:03 +0100, Francesc Alted wrote: >> The only scenario that I see that this would create unaligned arrays >> is >> for machines having AVX. But provided that the Intel architecture is >> making great strides in fetching unaligned

Re: [Numpy-discussion] Byte aligned arrays

2012-12-20 Thread Nathaniel Smith
On Thu, Dec 20, 2012 at 8:12 AM, Henry Gomersall wrote: > On Wed, 2012-12-19 at 15:10 +, Nathaniel Smith wrote: > >> >> > Is this something that can be rolled into Numpy (the feature, not >> my >> >> > particular implementation or interface - though I'd be happy for >> it to >> >> > be so)? >

Re: [Numpy-discussion] Byte aligned arrays

2012-12-20 Thread Henry Gomersall
On Wed, 2012-12-19 at 19:03 +0100, Francesc Alted wrote: > The only scenario that I see that this would create unaligned arrays > is > for machines having AVX. But provided that the Intel architecture is > making great strides in fetching unaligned data, I'd be surprised > that > the difference

Re: [Numpy-discussion] Byte aligned arrays

2012-12-20 Thread Henry Gomersall
On Thu, 2012-12-20 at 08:12 +, Henry Gomersall wrote: > On Wed, 2012-12-19 at 15:10 +, Nathaniel Smith wrote: > > > >> > Is this something that can be rolled into Numpy (the feature, > not > > my > > >> > particular implementation or interface - though I'd be happy > for > > it to > > >> >

Re: [Numpy-discussion] Byte aligned arrays

2012-12-20 Thread Henry Gomersall
On Wed, 2012-12-19 at 15:10 +, Nathaniel Smith wrote: > >> > Is this something that can be rolled into Numpy (the feature, not > my > >> > particular implementation or interface - though I'd be happy for > it to > >> > be so)? > >> > > >> > Regarding (b), I've written a test case that works fo

Re: [Numpy-discussion] Byte aligned arrays

2012-12-19 Thread David Cournapeau
On Wed, Dec 19, 2012 at 6:03 PM, Francesc Alted wrote: > On 12/19/12 5:47 PM, Henry Gomersall wrote: >> On Wed, 2012-12-19 at 15:57 +, Nathaniel Smith wrote: >>> Not sure which interface is more useful to users. On the one hand, >>> using funny dtypes makes regular non-SIMD access more cumbers

Re: [Numpy-discussion] Byte aligned arrays

2012-12-19 Thread Nathaniel Smith
On Wed, Dec 19, 2012 at 6:25 PM, Henry Gomersall wrote: > On Wed, 2012-12-19 at 19:03 +0100, Francesc Alted wrote: > >> > Finally, I think there is significant value in auto-aligning the >> array >> > based on an appropriate inspection of the cpu capabilities (or >> > alternatively, a function th

Re: [Numpy-discussion] Byte aligned arrays

2012-12-19 Thread Henry Gomersall
On Wed, 2012-12-19 at 19:03 +0100, Francesc Alted wrote: > > Finally, I think there is significant value in auto-aligning the > array > > based on an appropriate inspection of the cpu capabilities (or > > alternatively, a function that reports back the appropriate SIMD > > alignment). Again, this

Re: [Numpy-discussion] Byte aligned arrays

2012-12-19 Thread Francesc Alted
On 12/19/12 5:47 PM, Henry Gomersall wrote: > On Wed, 2012-12-19 at 15:57 +, Nathaniel Smith wrote: >> Not sure which interface is more useful to users. On the one hand, >> using funny dtypes makes regular non-SIMD access more cumbersome, and >> it forces your array size to be a multiple of the

Re: [Numpy-discussion] Byte aligned arrays

2012-12-19 Thread Henry Gomersall
On Wed, 2012-12-19 at 15:57 +, Nathaniel Smith wrote: > Not sure which interface is more useful to users. On the one hand, > using funny dtypes makes regular non-SIMD access more cumbersome, and > it forces your array size to be a multiple of the SIMD word size, > which might be inconvenient if

Re: [Numpy-discussion] Byte aligned arrays

2012-12-19 Thread Nathaniel Smith
On Wed, Dec 19, 2012 at 3:27 PM, Charles R Harris wrote: > > > On Wed, Dec 19, 2012 at 8:10 AM, Nathaniel Smith wrote: >> Right, my intuition is that it's like order="C" -- if you make a new >> array by, say, indexing, then it may or may not have order="C", no >> guarantees. So when you care, you

Re: [Numpy-discussion] Byte aligned arrays

2012-12-19 Thread Charles R Harris
On Wed, Dec 19, 2012 at 8:10 AM, Nathaniel Smith wrote: > On Wed, Dec 19, 2012 at 2:57 PM, Charles R Harris > wrote: > > > > > > On Wed, Dec 19, 2012 at 7:43 AM, Nathaniel Smith wrote: > >> > >> On Wed, Dec 19, 2012 at 8:40 AM, Henry Gomersall > wrote: > >> > I've written a few simple cython r

Re: [Numpy-discussion] Byte aligned arrays

2012-12-19 Thread Nathaniel Smith
On Wed, Dec 19, 2012 at 2:57 PM, Charles R Harris wrote: > > > On Wed, Dec 19, 2012 at 7:43 AM, Nathaniel Smith wrote: >> >> On Wed, Dec 19, 2012 at 8:40 AM, Henry Gomersall wrote: >> > I've written a few simple cython routines for assisting in creating >> > byte-aligned numpy arrays. The point

Re: [Numpy-discussion] Byte aligned arrays

2012-12-19 Thread Charles R Harris
On Wed, Dec 19, 2012 at 7:43 AM, Nathaniel Smith wrote: > On Wed, Dec 19, 2012 at 8:40 AM, Henry Gomersall wrote: > > I've written a few simple cython routines for assisting in creating > > byte-aligned numpy arrays. The point being for the arrays to work with > > SSE/AVX code. > > > > https://g

Re: [Numpy-discussion] Byte aligned arrays

2012-12-19 Thread Nathaniel Smith
On Wed, Dec 19, 2012 at 8:40 AM, Henry Gomersall wrote: > I've written a few simple cython routines for assisting in creating > byte-aligned numpy arrays. The point being for the arrays to work with > SSE/AVX code. > > https://github.com/hgomersall/pyFFTW/blob/master/pyfftw/utils.pxi > > The chang

[Numpy-discussion] Byte aligned arrays

2012-12-19 Thread Henry Gomersall
I've written a few simple cython routines for assisting in creating byte-aligned numpy arrays. The point being for the arrays to work with SSE/AVX code. https://github.com/hgomersall/pyFFTW/blob/master/pyfftw/utils.pxi The change recently has been to add a check on the CPU as to what flags are su