Re: [Numpy-discussion] np.array execution path

2012-09-22 Thread Sebastian Haase
Oh, is this actually documented - I knew that np.array would (by default) only create copies as need ... but I never knew it would - if all fits - even just return the original Python-object... Thanks, Sebastian Haase On Sat, Sep 22, 2012 at 8:12 PM, Travis Oliphant wrote: > Check to see if this

Re: [Numpy-discussion] Views of memmaps and offset

2012-09-22 Thread Charles R Harris
On Sat, Sep 22, 2012 at 12:19 PM, Charles R Harris < charlesr.har...@gmail.com> wrote: > > > On Sat, Sep 22, 2012 at 12:06 PM, Olivier Grisel > wrote: > >> 2012/9/22 Charles R Harris : >> > >> > >> > On Sat, Sep 22, 2012 at 11:52 AM, Charles R Harris >> > wrote: >> >> >> >> >> >> >> >> On Sat, S

Re: [Numpy-discussion] np.array execution path

2012-09-22 Thread Sebastian Berg
Ooops obviously thanks a lot, stupid me. Thanks was also enough to figure the rest out myself... On Sat, 2012-09-22 at 13:12 -0500, Travis Oliphant wrote: > Check to see if this expression is true > > no is o > > In the first case no and o are the same object > > > Travis > > -- > Travis Oli

Re: [Numpy-discussion] Views of memmaps and offset

2012-09-22 Thread Charles R Harris
On Sat, Sep 22, 2012 at 12:06 PM, Olivier Grisel wrote: > 2012/9/22 Charles R Harris : > > > > > > On Sat, Sep 22, 2012 at 11:52 AM, Charles R Harris > > wrote: > >> > >> > >> > >> On Sat, Sep 22, 2012 at 11:31 AM, Gael Varoquaux > >> wrote: > >>> > >>> On Sat, Sep 22, 2012 at 11:16:27AM -0600,

Re: [Numpy-discussion] np.array execution path

2012-09-22 Thread Travis Oliphant
Check to see if this expression is true no is o In the first case no and o are the same object Travis -- Travis Oliphant (on a mobile) 512-826-7480 On Sep 22, 2012, at 1:01 PM, Sebastian Berg wrote: > Hi, > > I have a bit of trouble figuring this out. I would have expected > np.asarray(a

Re: [Numpy-discussion] Views of memmaps and offset

2012-09-22 Thread Olivier Grisel
2012/9/22 Charles R Harris : > > > On Sat, Sep 22, 2012 at 11:52 AM, Charles R Harris > wrote: >> >> >> >> On Sat, Sep 22, 2012 at 11:31 AM, Gael Varoquaux >> wrote: >>> >>> On Sat, Sep 22, 2012 at 11:16:27AM -0600, Charles R Harris wrote: >>> >I think this is a bug, taking a view should prob

[Numpy-discussion] np.array execution path

2012-09-22 Thread Sebastian Berg
Hi, I have a bit of trouble figuring this out. I would have expected np.asarray(array) to go through ctors, PyArray_NewFromArray, but it seems to me it does not, so which execution path is exactly taken here? The reason I am asking is that I want to figure out this behavior/bug, and I really am no

Re: [Numpy-discussion] Views of memmaps and offset

2012-09-22 Thread Charles R Harris
On Sat, Sep 22, 2012 at 11:52 AM, Charles R Harris < charlesr.har...@gmail.com> wrote: > > > On Sat, Sep 22, 2012 at 11:31 AM, Gael Varoquaux < > gael.varoqu...@normalesup.org> wrote: > >> On Sat, Sep 22, 2012 at 11:16:27AM -0600, Charles R Harris wrote: >> >I think this is a bug, taking a vie

Re: [Numpy-discussion] Views of memmaps and offset

2012-09-22 Thread Charles R Harris
On Sat, Sep 22, 2012 at 11:31 AM, Gael Varoquaux < gael.varoqu...@normalesup.org> wrote: > On Sat, Sep 22, 2012 at 11:16:27AM -0600, Charles R Harris wrote: > >I think this is a bug, taking a view should probably update the > offset. > > OK, we can include a fix for that alongside with the pat

[Numpy-discussion] Ignore axes with dimension==1 for contiguous flags

2012-09-22 Thread Sebastian Berg
Hey, Numpy currently assumes that if "ndim > 1" then it is impossible for any array to be both C- and F-contiguous, however an axes of dimension 1 does have no effect on the memory layout. I think I have made most important changes (actually really very few), though I bet some parts of numpy still

Re: [Numpy-discussion] Views of memmaps and offset

2012-09-22 Thread Gael Varoquaux
On Sat, Sep 22, 2012 at 11:16:27AM -0600, Charles R Harris wrote: >I think this is a bug, taking a view should probably update the offset. OK, we can include a fix for that alongside with the patch to keep track of the filename. Cheers, Gaƫl ___ Nu

Re: [Numpy-discussion] Views of memmaps and offset

2012-09-22 Thread Gael Varoquaux
On Sat, Sep 22, 2012 at 06:30:27PM +0200, Olivier Grisel wrote: > The only clean solution would be to make `numpy.memmap` use a wrapper > buffer object that would keep track of the filename and offset > attributes instead of using a `mmap.mmap` instance directly. Indeed, Olivier and I have been st

Re: [Numpy-discussion] Views of memmaps and offset

2012-09-22 Thread Charles R Harris
On Sat, Sep 22, 2012 at 7:54 AM, Gael Varoquaux < gael.varoqu...@normalesup.org> wrote: > Hi list, > > I am struggling with offsets on the view of a memmaped array. Consider > the following: > > import numpy as np > > a = np.memmap('tmp.mmap', dtype=np.float64, shape=50, mode='w+') > a[:] = np.ara

Re: [Numpy-discussion] Views of memmaps and offset

2012-09-22 Thread Olivier Grisel
A posix dup (http://www.unix.com/man-page/POSIX/3posix/dup/) would not solve it as the fd is hidden inside the python `mmap.mmap` instance that is a builtin that just exposes the python buffer interface and hides the implementation details. The only clean solution would be to make `numpy.memmap` u

Re: [Numpy-discussion] Views of memmaps and offset

2012-09-22 Thread Gael Varoquaux
On Sat, Sep 22, 2012 at 10:15:52AM -0600, Charles R Harris wrote: >Would some sort of 'dup' method be useful? What do you mean by dup? G ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discus

Re: [Numpy-discussion] Views of memmaps and offset

2012-09-22 Thread Charles R Harris
On Sat, Sep 22, 2012 at 9:46 AM, Olivier Grisel wrote: > There is also a third use case that is problematic on numpy master: > > orig = np.memmap('tmp.mmap', dtype=np.float64, shape=100, mode='w+') > orig[:] = np.arange(orig.shape[0]) * -1.0 # negative markers to > detect under / overflows > > a

Re: [Numpy-discussion] Views of memmaps and offset

2012-09-22 Thread Olivier Grisel
There is also a third use case that is problematic on numpy master: orig = np.memmap('tmp.mmap', dtype=np.float64, shape=100, mode='w+') orig[:] = np.arange(orig.shape[0]) * -1.0 # negative markers to detect under / overflows a = np.memmap('tmp.mmap', dtype=np.float64, shape=50, mode='r+', offse

Re: [Numpy-discussion] Views of memmaps and offset

2012-09-22 Thread Olivier Grisel
2012/9/22 Gael Varoquaux : > Hi list, > > I am struggling with offsets on the view of a memmaped array. Consider > the following: > > import numpy as np > > a = np.memmap('tmp.mmap', dtype=np.float64, shape=50, mode='w+') > a[:] = np.arange(50) > b = a[10:] > > Here, I have a.offset == 0 and b.offs

[Numpy-discussion] Views of memmaps and offset

2012-09-22 Thread Gael Varoquaux
Hi list, I am struggling with offsets on the view of a memmaped array. Consider the following: import numpy as np a = np.memmap('tmp.mmap', dtype=np.float64, shape=50, mode='w+') a[:] = np.arange(50) b = a[10:] Here, I have a.offset == 0 and b.offset == 0. In practice, the data in b is offset c

Re: [Numpy-discussion] specifying numpy as dependency in your project, install_requires

2012-09-22 Thread Ralf Gommers
On Fri, Sep 21, 2012 at 11:39 PM, Nathaniel Smith wrote: > On Fri, Sep 21, 2012 at 9:42 PM, Ralf Gommers > wrote: > > Eh, just installing numpy with "python setup.py install" uses plain > > distutils, not setuptools. So there indeed isn't an entry in > > easy-install.pth. Which some consider a