Re: [Numpy-discussion] Proposal: numpy.random.random_seed

2016-05-17 Thread Nathaniel Smith
On Tue, May 17, 2016 at 10:41 AM, Robert Kern wrote: > On Tue, May 17, 2016 at 6:24 PM, Nathaniel Smith wrote: >> >> On May 17, 2016 1:50 AM, "Robert Kern" wrote: >> > >> [...] >> > What you want is a function that returns many RandomState objects that >> > are hopefully spread around the MT1993

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

2016-05-17 Thread Sturla Molden
Matěj Týč wrote: > Does it mean > that if you pass the numpy array to the child process using Queue, no > significant amount of data will flow through it? This is what my shared memory arrayes do. > Or I shouldn't pass it > using Queue at all and just rely on inheritance? This is what David

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

2016-05-17 Thread Matěj Týč
On 17.5.2016 14:13, Sturla Molden wrote: > Matěj Týč wrote: > >> - Parallel processing of HUGE data, and > This is mainly a Windows problem, as copy-on-write fork() will solve this > on any other platform. ... That sounds interesting, could you elaborate on it a bit? Does it mean that if you pas

Re: [Numpy-discussion] Proposal: numpy.random.random_seed

2016-05-17 Thread Robert Kern
On Tue, May 17, 2016 at 6:24 PM, Nathaniel Smith wrote: > > On May 17, 2016 1:50 AM, "Robert Kern" wrote: > > > [...] > > What you want is a function that returns many RandomState objects that are hopefully spread around the MT19937 space enough that they are essentially independent (in the absen

Re: [Numpy-discussion] Proposal: numpy.random.random_seed

2016-05-17 Thread Nathaniel Smith
On May 17, 2016 1:50 AM, "Robert Kern" wrote: > [...] > What you want is a function that returns many RandomState objects that are hopefully spread around the MT19937 space enough that they are essentially independent (in the absence of true jumpahead). The better implementation of such a function

Re: [Numpy-discussion] Proposal: numpy.random.random_seed

2016-05-17 Thread Eric Moore
On Tue, May 17, 2016 at 9:40 AM, Sturla Molden wrote: > Stephan Hoyer wrote: > > I have recently encountered several use cases for randomly generate > random > > number seeds: > > > > 1. When writing a library of stochastic functions that take a seed as an > > input argument, and some of these f

Re: [Numpy-discussion] Proposal: numpy.random.random_seed

2016-05-17 Thread Robert Kern
On Tue, May 17, 2016 at 2:40 PM, Sturla Molden wrote: > > Stephan Hoyer wrote: > > I have recently encountered several use cases for randomly generate random > > number seeds: > > > > 1. When writing a library of stochastic functions that take a seed as an > > input argument, and some of these fu

Re: [Numpy-discussion] Proposal: numpy.random.random_seed

2016-05-17 Thread Sturla Molden
Stephan Hoyer wrote: > I have recently encountered several use cases for randomly generate random > number seeds: > > 1. When writing a library of stochastic functions that take a seed as an > input argument, and some of these functions call multiple other such > stochastic functions. Dask is one

Re: [Numpy-discussion] Proposal: numpy.random.random_seed

2016-05-17 Thread josef.pktd
On Tue, May 17, 2016 at 4:49 AM, Robert Kern wrote: > On Tue, May 17, 2016 at 9:09 AM, Stephan Hoyer wrote: > > > > On Tue, May 17, 2016 at 12:18 AM, Robert Kern > wrote: > >> > >> On Tue, May 17, 2016 at 4:54 AM, Stephan Hoyer > wrote: > >> > 1. When writing a library of stochastic functions

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

2016-05-17 Thread Sturla Molden
Matěj Týč wrote: > - Parallel processing of HUGE data, and This is mainly a Windows problem, as copy-on-write fork() will solve this on any other platform. I am more in favor of asking Microsoft to fix their broken OS. Also observe that the usefulness of shared memory is very limited on Wind

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

2016-05-17 Thread Matěj Týč
On 11.5.2016 10:29, Sturla Molden wrote: > I did some work on this some years ago. ... > I am sorry, I have missed this discussion when it started. There are two cases when I had feeling that I had to use this functionality: - Parallel processing of HUGE data, and - using parallel processing i

Re: [Numpy-discussion] Proposal: numpy.random.random_seed

2016-05-17 Thread Robert Kern
On Tue, May 17, 2016 at 9:09 AM, Stephan Hoyer wrote: > > On Tue, May 17, 2016 at 12:18 AM, Robert Kern wrote: >> >> On Tue, May 17, 2016 at 4:54 AM, Stephan Hoyer wrote: >> > 1. When writing a library of stochastic functions that take a seed as an input argument, and some of these functions cal

Re: [Numpy-discussion] Proposal: numpy.random.random_seed

2016-05-17 Thread Stephan Hoyer
On Tue, May 17, 2016 at 12:18 AM, Robert Kern wrote: > On Tue, May 17, 2016 at 4:54 AM, Stephan Hoyer wrote: > > 1. When writing a library of stochastic functions that take a seed as an > input argument, and some of these functions call multiple other such > stochastic functions. Dask is one suc

Re: [Numpy-discussion] Proposal: numpy.random.random_seed

2016-05-17 Thread Robert Kern
On Tue, May 17, 2016 at 4:54 AM, Stephan Hoyer wrote: > > I have recently encountered several use cases for randomly generate random number seeds: > > 1. When writing a library of stochastic functions that take a seed as an input argument, and some of these functions call multiple other such stoch