Den 11.04.2011 21:15, skrev srean:
Got you and thanks a lot for the explanation. I am not using Queues so
I think I am safe for the time being. Given that you have worked a
lot on these issues, would you recommend plain mmapped numpy arrays
over |multiprocessing.Array|
|
With multiprocessing
Den 11.04.2011 01:20, skrev Sturla Molden:
>
> Changes:
>
> - 64-bit support.
> - Memory leak on Linux/Unix should be gone (monkey patch for os._exit).
> - Added a global lock as there are callbacks to Python (the GIL is not
> sufficient serialization).
I will also add a barrier synchronization p
Got you and thanks a lot for the explanation. I am not using Queues so I
think I am safe for the time being. Given that you have worked a lot on
these issues, would you recommend plain mmapped numpy arrays over
multiprocessing.Array
Thanks again
-- srean
On Mon, Apr 11, 2011 at 1:36 PM, Sturla
Den 11.04.2011 14:58, skrev Zbigniew Jędrzejewski-Szmek:
> Hi,
> it could, but you'd have to do the parsing of data yourself. So nothing
> fancy unless you want to reimplement numpy in Java :)
Not really. Only the data buffer is stored in shared memory. If you can
pass the required fields to Java
"Shared memory" is memory mapping from the paging file (i.e. RAM), not a
file on disk. They can have a name or be anonymous. I have explained why
we need named shared memory before. If you didn't understand it, try to
pass an instance of |multiprocessing.Array over | |multiprocessing.Queue.
|S
Apologies for adding to my own post. multiprocessing.Array(...) uses an
anonymous mmapped file. I am not sure if that means it is resident on RAM or
the swap device. But my original question remains, what are the pros and
cons of using it versus numpy mmapped arrays. If multiprocessing.Array is
in
Hi everyone,
I was looking up the options that are available for shared memory arrays
and this thread came up at the right time. The doc says that multiprocessing
.Array(...) gives a shared memory array. But from the code it seems to me
that it is actually using a mmap. Is that correct a correct
On 04/11/2011 09:21 AM, Sebastian Haase wrote:
> I have a non python question:
> for Java there seems to exist a module/package/class called nio
>
> http://download.oracle.com/javase/1.4.2/docs/api/java/nio/MappedByteBuffer.html
>public abstract class MappedByteBufferextends ByteBuffer
>
On Mon, Apr 11, 2011 at 7:05 AM, Gael Varoquaux
wrote:
> Hey Sturla,
>
> It's really great that you are still working on that. I'll test the code
> under Linux.
>
> The scipy community has moved to github. If I create a repository under
> github and put the code on it, would you use it? If I find
Hey Sturla,
It's really great that you are still working on that. I'll test the code
under Linux.
The scipy community has moved to github. If I create a repository under
github and put the code on it, would you use it? If I find time, I'll add
a setup.py.
Gaël
__
Den 11.04.2011 01:20, skrev Sturla Molden:
>
> I'm an idiot with build tools, hence no setup.py. Invoke Cython and
> then cc. Compile sharedmemory_sysv.pyx for Linux/Unix or
> sharedmemory_sysv.pyx and ntqueryobject.c for Windows.
Eh, that is sharedmemory_win.pyx and ntqueryobject.c for Windows
11 matches
Mail list logo
