Re: [Numpy-discussion] array of random numbers fails to construct

2015-12-08 Thread Allan Haldane
On 12/08/2015 08:01 PM, Allan Haldane wrote: > On 12/08/2015 07:40 PM, Stephan Hoyer wrote: >> On Sun, Dec 6, 2015 at 3:55 PM, Allan Haldane > > wrote: >> >> >> I've also often wanted to generate large datasets of random uint8 >> and uint16. As a workaround, t

Re: [Numpy-discussion] array of random numbers fails to construct

2015-12-08 Thread Allan Haldane
On 12/08/2015 07:40 PM, Stephan Hoyer wrote: > On Sun, Dec 6, 2015 at 3:55 PM, Allan Haldane > wrote: > > > I've also often wanted to generate large datasets of random uint8 > and uint16. As a workaround, this is something I have used: > > np.ndarray(1

Re: [Numpy-discussion] array of random numbers fails to construct

2015-12-08 Thread Matthew Brett
Hi, On Tue, Dec 8, 2015 at 4:40 PM, Stephan Hoyer wrote: > On Sun, Dec 6, 2015 at 3:55 PM, Allan Haldane > wrote: >> >> >> I've also often wanted to generate large datasets of random uint8 and >> uint16. As a workaround, this is something I have used: >> >> np.ndarray(100, 'u1', np.random.bytes(

Re: [Numpy-discussion] array of random numbers fails to construct

2015-12-08 Thread Stephan Hoyer
On Sun, Dec 6, 2015 at 3:55 PM, Allan Haldane wrote: > > I've also often wanted to generate large datasets of random uint8 and > uint16. As a workaround, this is something I have used: > > np.ndarray(100, 'u1', np.random.bytes(100)) > > It has also crossed my mind that np.random.randint and np.r

Re: [Numpy-discussion] array of random numbers fails to construct

2015-12-08 Thread Sebastian
On 12/08/2015 02:17 AM, Warren Weckesser wrote: > On Sun, Dec 6, 2015 at 6:55 PM, Allan Haldane > wrote: > > It has also crossed my mind that np.random.randint and > np.random.rand could use an extra 'dtype' keyword. > > +1. Not a high priority, but it would

Re: [Numpy-discussion] array of random numbers fails to construct

2015-12-07 Thread Warren Weckesser
On Sun, Dec 6, 2015 at 6:55 PM, Allan Haldane wrote: > > I've also often wanted to generate large datasets of random uint8 and > uint16. As a workaround, this is something I have used: > > np.ndarray(100, 'u1', np.random.bytes(100)) > > It has also crossed my mind that np.random.randint and np.ra

Re: [Numpy-discussion] array of random numbers fails to construct

2015-12-07 Thread Elliot Hallmark
David, >I'm concluding that the .astype(np.uint8) is applied after the array is constructed, instead of during the process. That is how python works in general. astype is a method of an array, so randint needs to return the array before there is something with an astype method to call. A dtype

Re: [Numpy-discussion] array of random numbers fails to construct

2015-12-06 Thread DAVID SAROFF (RIT Student)
Allan, I see with a google search on your name that you are in the physics department at Rutgers. I got my BA in Physics there. 1975. Biological physics. A thought: Is there an entropy that can be assigned to the dna in an organism? I don't mean the usual thing, coupled to the heat bath. Evolution

Re: [Numpy-discussion] array of random numbers fails to construct

2015-12-06 Thread Allan Haldane
I've also often wanted to generate large datasets of random uint8 and uint16. As a workaround, this is something I have used: np.ndarray(100, 'u1', np.random.bytes(100)) It has also crossed my mind that np.random.randint and np.random.rand could use an extra 'dtype' keyword. It didn't look e

Re: [Numpy-discussion] array of random numbers fails to construct

2015-12-06 Thread DAVID SAROFF (RIT Student)
Matthew, That looks right. I'm concluding that the .astype(np.uint8) is applied after the array is constructed, instead of during the process. This random array is a test case. In the production analysis of radio telescope data this is how the data comes in, and there is no problem with 10GBy fil

Re: [Numpy-discussion] array of random numbers fails to construct

2015-12-06 Thread Jaime Fernández del Río
On Sun, Dec 6, 2015 at 10:07 PM, Matthew Brett wrote: > Hi, > > On Sun, Dec 6, 2015 at 12:39 PM, DAVID SAROFF (RIT Student) > wrote: > > This works. A big array of eight bit random numbers is constructed: > > > > import numpy as np > > > > spectrumArray = np.random.randint(0,255, (2**20,2**12)).

Re: [Numpy-discussion] array of random numbers fails to construct

2015-12-06 Thread Matthew Brett
Hi, On Sun, Dec 6, 2015 at 12:39 PM, DAVID SAROFF (RIT Student) wrote: > This works. A big array of eight bit random numbers is constructed: > > import numpy as np > > spectrumArray = np.random.randint(0,255, (2**20,2**12)).astype(np.uint8) > > > > This fails. It eats up all 64GBy of RAM: > > spe

[Numpy-discussion] array of random numbers fails to construct

2015-12-06 Thread DAVID SAROFF (RIT Student)
This works. A big array of eight bit random numbers is constructed: import numpy as np spectrumArray = np.random.randint(0,255, (2**20,2**12)).astype(np.uint8) This fails. It eats up all 64GBy of RAM: spectrumArray = np.random.randint(0,255, (2**21,2**12)).astype(np.uint8) The difference is