Robert Kern gmail.com> writes:
> > Within a single machine, that sounds fine. What about processes running on
> > different nodes, with different main memories?
>
> You mean mmaping a file on a shared file system?
Yes. GPFS, I believe, presumably this:
http://en.wikipedia.org/wiki/GPFS
Horrible
Robert Kern gmail.com> writes:
> >> It's up to the virtual memory manager, but usually, it will just load
> >> those pages (chunks the size of mmap.PAGESIZE) that are touched by
> >> your request and write them back.
> >
> > What if two processes touch adjacent chunks that are smaller than a page?
Robert Kern gmail.com> writes:
> >> >> You can have each of those processes memory-map the whole file and
> >> >> just operate on their own slices. Your operating system's virtual
> >> >> memory manager should handle all of the details for you.
> >
> > Wow, I didn't know that. So as long as the ra
Robert Kern gmail.com> writes:
> On Mon, Feb 28, 2011 at 18:50, Sturla Molden molden.no> wrote:
> > Den 01.03.2011 01:15, skrev Robert Kern:
> >> You can have each of those processes memory-map the whole file and
> >> just operate on their own slices. Your operating system's virtual
> >> memory
Ralf Gommers googlemail.com> writes:
> My first question after looking at this is why we would want three
> very similar ways to load memory-mapped arrays (np.memmap, np.load,
> np.lib.format.open_memmap)? They already exist but your changes make
> those three even more similar.
If I understand
Jon Olav Vik gmail.com> writes:
> https://github.com/jonovik/numpy/compare/master...offset_memmap
> I've used this extensively on Numpy 1.4.0, but based my Git checkout on the
> current Numpy trunk. There have been some rearrangements in np.load since
then
> (it used to
load since then
(it used to be in np.lib.io and is now in np.lib.npyio), but as far as I can
see, my modifications carry over fine. I haven't had a chance to test with
Numpy trunk, though. (What is the best way to set up a test version without
affecting my working 1.4.0 setup?)
Hope this ca
Scott Sinclair gmail.com> writes:
> >>> import numpy as np
> >>> x = np.eye(3)
> >>> x
> array([[ 1., 0., 0.],
>[ 0., 1., 0.],
>[ 0., 0., 1.]])
> >>> x.resize((5,5))
> Traceback (most recent call last):
> File "", line 1, in
> ValueError: cannot resize an array that has b
