[Numpy-discussion] dtype '|S0' not understood

2009-09-23 Thread David Warde-Farley
Howdy, It seems it's possible using e.g. In [25]: dtype([('foo', str)])Out[25]: dtype([('foo', '|S0')]) to get yourself a zero-length string. However dtype('|S0') results in a TypeError: data type not understood. I understand the stupidity of creating a 0-length string field but it's concei

Re: [Numpy-discussion] something wrong with docs?

2009-09-23 Thread Fernando Perez
On Tue, Sep 22, 2009 at 11:15 PM, David Goldsmith wrote: > "It would be nice if we could move gradually > towards docs whose examples (at least those marked as such) were > always run via sphinx." > > That's a "suggestion," but given your point, it seems like you'd advocate it > being more than th

Re: [Numpy-discussion] is ndarray.base the closest base or the ultimate base?

2009-09-23 Thread Citi, Luca
> numpy.may_share_memory() should be pretty cheap. It's just arithmetic. True, but it is in python. Not something that should go in construct_arrays of ufunc_object.c, I suppose. But the same approach can be translated to C, probably. I can try if we decide http://projects.scipy.org/numpy/ticket/

Re: [Numpy-discussion] Numpy depends on OpenSSL ???

2009-09-23 Thread Mark Sienkiewicz
Robert Kern wrote: > On Wed, Sep 23, 2009 at 10:52, Mark Sienkiewicz wrote: > >> I have discovered the hard way that numpy depends on openssl. >> >> I am building a 64 bit python environment for the macintosh. I >> currently do not have a 64 bit openssl library installed, so the python >> inte

Re: [Numpy-discussion] is ndarray.base the closest base or the ultimate base?

2009-09-23 Thread David Goldsmith
So the end result is: change the docstring, correct? DG On Wed, Sep 23, 2009 at 11:49 AM, Robert Kern wrote: > On Wed, Sep 23, 2009 at 13:30, Citi, Luca wrote: > > >>> http://projects.scipy.org/numpy/ticket/1085 > > But I think in that case it was still an improvement w.r.t. the current > impl

Re: [Numpy-discussion] numpy and cython in pure python mode

2009-09-23 Thread Sebastian Haase
Thanks for all the replies ! On Wed, Sep 23, 2009 at 4:55 AM, Dag Sverre Seljebotn wrote: > Robert Kern wrote: >> On Tue, Sep 22, 2009 at 01:33, Sebastian Haase wrote: >> >>> Hi, >>> I'm not subscribed to the cython list - hoping enough people would >>> care to justify my post here: >>> > The po

[Numpy-discussion] Coercing object arrays to string (or unicode) arrays

2009-09-23 Thread Michael Droettboom
As I'm looking into fixing a number of bugs in chararray, I'm running into some surprising behavior. One of the things chararray needs to do occasionally is build up an object array of string objects, and then convert that back to a fixed-length string array. This length is sometimes predeter

Re: [Numpy-discussion] Deserialized arrays with base mutate strings

2009-09-23 Thread Robert Kern
On Wed, Sep 23, 2009 at 13:59, Pauli Virtanen wrote: > ke, 2009-09-23 kello 10:01 +0200, Hrvoje Niksic kirjoitti: > [clip] >> I guess this one could be prevented by verifying that the buffer is >> writable when setting the "writable" flag.  When deserializing arrays, I >> don't see a reason for th

Re: [Numpy-discussion] Deserialized arrays with base mutate strings

2009-09-23 Thread Pauli Virtanen
ke, 2009-09-23 kello 10:01 +0200, Hrvoje Niksic kirjoitti: [clip] > I guess this one could be prevented by verifying that the buffer is > writable when setting the "writable" flag. When deserializing arrays, I > don't see a reason for the "base" property to even exist - sharing of > the buffer

Re: [Numpy-discussion] is ndarray.base the closest base or the ultimate base?

2009-09-23 Thread Robert Kern
On Wed, Sep 23, 2009 at 13:30, Citi, Luca wrote: >>> http://projects.scipy.org/numpy/ticket/1085 > But I think in that case it was still an improvement w.r.t. the current > implementation > which is buggy. At least it shields 95% of users from unexpected results. > Using memory addresses and ext

Re: [Numpy-discussion] is ndarray.base the closest base or the ultimate base?

2009-09-23 Thread Citi, Luca
> Lacking a robust use case, I would prefer to keep the current > behavior. It is likely that nothing would break if we changed it, but > without a use case, I would prefer to be conservative. Fair enough. >> When working on >> http://projects.scipy.org/numpy/ticket/1085 >> I had to walk the chai

Re: [Numpy-discussion] Create numpy array from a list error

2009-09-23 Thread David Warde-Farley
On 23-Sep-09, at 10:06 AM, Dave Wood wrote: > Hi all, > > I've got a fairly large (but not huge, 58mb) tab seperated text > file, with > approximately 200 columns and 56k rows of numbers and strings. > > Here's a snippet of my code to create a numpy matrix from the data > file... > > > >

Re: [Numpy-discussion] simple indexing question

2009-09-23 Thread Neal Becker
josef.p...@gmail.com wrote: > On Wed, Sep 23, 2009 at 11:12 AM, Neal Becker wrote: >> Robert Cimrman wrote: >> >>> Neal Becker wrote: I have an array: In [12]: a Out[12]: array([[0, 1, 2, 3, 4], [5, 6, 7, 8, 9]]) And a selection array: In [13]: b Out[1

Re: [Numpy-discussion] is ndarray.base the closest base or the ultimate base?

2009-09-23 Thread Robert Kern
On Tue, Sep 22, 2009 at 17:14, Citi, Luca wrote: > My vote (if I am entitled to) goes to "change the code". > Whether or not the addressee of .base is an array, it should be "the object > that has to be kept alive such that the data does not get deallocated" rather > "one object which will keep

Re: [Numpy-discussion] Numpy 2D array from a list error

2009-09-23 Thread Dave Wood
Ignore that last mail, I hit send instead of save by mistake. Between you you seem to be right, it's a problem with loading the array of strings. There must be some large strings in the first 'rowname' column. If this column is left out, it works fine (even as strings). Many thanks, sorry for all

Re: [Numpy-discussion] Numpy 2D array from a list error

2009-09-23 Thread Dave Wood
Appologies for the multiple posts, people. My posting to the forum was pending for a long time, so I deleted it and tried emailing directly. I didn't think they'd all be sent out. Gokan, thanks for the reply, I hope you get this one. "Here I use loadtxt to read ~89 MB txt file. Can you use loadtxt

Re: [Numpy-discussion] Create numpy array from a list error

2009-09-23 Thread Gökhan Sever
On Wed, Sep 23, 2009 at 9:06 AM, Dave Wood wrote: > Hi all, > > I've got a fairly large (but not huge, 58mb) tab seperated text file, with > approximately 200 columns and 56k rows of numbers and strings. > > Here's a snippet of my code to create a numpy matrix from the data file... > > > > d

Re: [Numpy-discussion] Numpy 2D array from a list error

2009-09-23 Thread Christopher Barker
Dave Wood wrote: > Well, I suppose they are all considered to be strings here. I haven't > tried to convert the numbers to floats yet. This could be an issue. For strings, numpy creates an array of strings, all of the same length, so each element is as big as the largest one: In [13]: l Out[13]

Re: [Numpy-discussion] Numpy depends on OpenSSL ???

2009-09-23 Thread David Cournapeau
On Thu, Sep 24, 2009 at 1:20 AM, Charles R Harris wrote: > In any case, we should find a fix. I don't think we do - we requires a standard python install, and a python without hashlib is crippled. If you can't build python without openssl, I would consider this a python bug. cheers, David _

Re: [Numpy-discussion] Numpy depends on OpenSSL ???

2009-09-23 Thread David Warde-Farley
On 23-Sep-09, at 11:52 AM, Mark Sienkiewicz wrote: > I am building a 64 bit python environment for the macintosh. I > currently do not have a 64 bit openssl library installed, so the > python > interpreter does not have hashlib. (hashlib gets its md5 function > from > the openssl library.)

Re: [Numpy-discussion] Create numpy array from a list error

2009-09-23 Thread Gökhan Sever
On Wed, Sep 23, 2009 at 9:06 AM, Dave Wood wrote: > Hi all, > > I've got a fairly large (but not huge, 58mb) tab seperated text file, with > approximately 200 columns and 56k rows of numbers and strings. > > Here's a snippet of my code to create a numpy matrix from the data file... > > > > d

Re: [Numpy-discussion] Numpy depends on OpenSSL ???

2009-09-23 Thread Charles R Harris
On Wed, Sep 23, 2009 at 9:52 AM, Mark Sienkiewicz wrote: > I have discovered the hard way that numpy depends on openssl. > > I am building a 64 bit python environment for the macintosh. I > currently do not have a 64 bit openssl library installed, so the python > interpreter does not have hashlib

Re: [Numpy-discussion] Numpy depends on OpenSSL ???

2009-09-23 Thread Robert Kern
On Wed, Sep 23, 2009 at 10:52, Mark Sienkiewicz wrote: > I have discovered the hard way that numpy depends on openssl. > > I am building a 64 bit python environment for the macintosh.  I > currently do not have a 64 bit openssl library installed, so the python > interpreter does not have hashlib.

Re: [Numpy-discussion] simple indexing question

2009-09-23 Thread Robert Cimrman
josef.p...@gmail.com wrote: > On Wed, Sep 23, 2009 at 11:12 AM, Neal Becker wrote: >> Robert Cimrman wrote: >> >>> Neal Becker wrote: I have an array: In [12]: a Out[12]: array([[0, 1, 2, 3, 4], [5, 6, 7, 8, 9]]) And a selection array: In [13]: b

[Numpy-discussion] Numpy depends on OpenSSL ???

2009-09-23 Thread Mark Sienkiewicz
I have discovered the hard way that numpy depends on openssl. I am building a 64 bit python environment for the macintosh. I currently do not have a 64 bit openssl library installed, so the python interpreter does not have hashlib. (hashlib gets its md5 function from the openssl library.) Th

Re: [Numpy-discussion] Numpy 2D array from a list error

2009-09-23 Thread Skipper Seabold
On Wed, Sep 23, 2009 at 9:42 AM, davew wrote: > > Hi, > > I've got a fairly large (but not huge, 58mb) tab seperated text file, with > approximately 200 columns and 56k rows of numbers and strings. > > Here's a snippet of my code to create a numpy matrix from the data file... > > > > data

Re: [Numpy-discussion] Numpy 2D array from a list error

2009-09-23 Thread Bruce Southey
On 09/23/2009 10:00 AM, Dave Wood wrote: "If the text file has 'numbers and strings' how is numpy meant to know what dtype to use? Please try genfromtxt especially if columns contain both numbers and strings." Well, I suppose they are all considered to be strings here. I haven't tried to convert

Re: [Numpy-discussion] simple indexing question

2009-09-23 Thread josef . pktd
On Wed, Sep 23, 2009 at 11:12 AM, Neal Becker wrote: > Robert Cimrman wrote: > >> Neal Becker wrote: >>> I have an array: >>> In [12]: a >>> Out[12]: >>> array([[0, 1, 2, 3, 4], >>>        [5, 6, 7, 8, 9]]) >>> >>> And a selection array: >>> In [13]: b >>> Out[13]: array([1, 1, 1, 1, 1]) >>> >>> I

Re: [Numpy-discussion] simple indexing question

2009-09-23 Thread Neal Becker
Robert Cimrman wrote: > Neal Becker wrote: >> I have an array: >> In [12]: a >> Out[12]: >> array([[0, 1, 2, 3, 4], >>[5, 6, 7, 8, 9]]) >> >> And a selection array: >> In [13]: b >> Out[13]: array([1, 1, 1, 1, 1]) >> >> I want a 1-dimensional output, where the array b selects an element

Re: [Numpy-discussion] cummax

2009-09-23 Thread Charles R Harris
On Wed, Sep 23, 2009 at 8:34 AM, Nissim Karpenstein wrote: > Hi, > > I want a cummax function where given an array inp it returns this: > > numpy.array([inp[:i].max() for i in xrange(1,len(inp)+1)]). > > Various python versions equivalent to the above are quite slow (though a > single python loop

Re: [Numpy-discussion] Numpy 2D array from a list error

2009-09-23 Thread Dave Wood
"If the text file has 'numbers and strings' how is numpy meant to know what dtype to use? Please try genfromtxt especially if columns contain both numbers and strings." Well, I suppose they are all considered to be strings here. I haven't tried to convert the numbers to floats yet. "What happens

[Numpy-discussion] cummax

2009-09-23 Thread Nissim Karpenstein
Hi, I want a cummax function where given an array inp it returns this: numpy.array([inp[:i].max() for i in xrange(1,len(inp)+1)]). Various python versions equivalent to the above are quite slow (though a single python loop is much faster than a python loop with a nested numpy C loop as shown abo

Re: [Numpy-discussion] Numpy 2D array from a list error

2009-09-23 Thread Bruce Southey
On 09/23/2009 08:42 AM, davew wrote: > Hi, > > I've got a fairly large (but not huge, 58mb) tab seperated text file, with > approximately 200 columns and 56k rows of numbers and strings. > > Here's a snippet of my code to create a numpy matrix from the data file... > > > > data = map(lambd

[Numpy-discussion] Create numpy array from a list error

2009-09-23 Thread Dave Wood
Hi all, I've got a fairly large (but not huge, 58mb) tab seperated text file, with approximately 200 columns and 56k rows of numbers and strings. Here's a snippet of my code to create a numpy matrix from the data file... data = map(lambda x : x.strip().split('\t'), sys.stdin.readlines()) d

Re: [Numpy-discussion] simple indexing question

2009-09-23 Thread Robert Cimrman
Neal Becker wrote: > I have an array: > In [12]: a > Out[12]: > array([[0, 1, 2, 3, 4], >[5, 6, 7, 8, 9]]) > > And a selection array: > In [13]: b > Out[13]: array([1, 1, 1, 1, 1]) > > I want a 1-dimensional output, where the array b selects an element from > each column of a, where if

[Numpy-discussion] simple indexing question

2009-09-23 Thread Neal Becker
I have an array: In [12]: a Out[12]: array([[0, 1, 2, 3, 4], [5, 6, 7, 8, 9]]) And a selection array: In [13]: b Out[13]: array([1, 1, 1, 1, 1]) I want a 1-dimensional output, where the array b selects an element from each column of a, where if b[i]=0 select element from 0th row of a and

[Numpy-discussion] Numpy 2D array from a list error

2009-09-23 Thread davew0000
Hi, I've got a fairly large (but not huge, 58mb) tab seperated text file, with approximately 200 columns and 56k rows of numbers and strings. Here's a snippet of my code to create a numpy matrix from the data file... data = map(lambda x : x.strip().split('\t'), sys.stdin.readlines())

Re: [Numpy-discussion] numpy and cython in pure python mode

2009-09-23 Thread Dag Sverre Seljebotn
Robert Kern wrote: > On Tue, Sep 22, 2009 at 01:33, Sebastian Haase wrote: > >> Hi, >> I'm not subscribed to the cython list - hoping enough people would >> care to justify my post here: >> The post might be justified, but it is a question of available knowledge as well. I nearly missed t

[Numpy-discussion] current stautus of numpy -> Excel

2009-09-23 Thread Tim Michelsen
FYI: Here is a summary of how one can 1) write numpy arrays to Excel 2) interact with numpy/scipy/... from Excel http://groups.google.com/group/python-excel/msg/3881b7e7ae210cc7 Best regards, Timmie ___ NumPy-Discussion mailing list NumPy-Discussion@

Re: [Numpy-discussion] Deserialized arrays with base mutate strings

2009-09-23 Thread Hrvoje Niksic
Pauli Virtanen wrote: > Wed, 23 Sep 2009 09:15:44 +0200, Hrvoje Niksic wrote: > [clip] >> Numpy arrays with the "base" property are deserialized as arrays >> pointing to a storage contained within a Python string. This is a >> problem since such arrays are mutable and can mutate existing strings.

Re: [Numpy-discussion] Deserialized arrays with base mutate strings

2009-09-23 Thread Pauli Virtanen
Wed, 23 Sep 2009 09:15:44 +0200, Hrvoje Niksic wrote: [clip] > Numpy arrays with the "base" property are deserialized as arrays > pointing to a storage contained within a Python string. This is a > problem since such arrays are mutable and can mutate existing strings. > Here is how to create one:

[Numpy-discussion] Deserialized arrays with base mutate strings

2009-09-23 Thread Hrvoje Niksic
Numpy arrays with the "base" property are deserialized as arrays pointing to a storage contained within a Python string. This is a problem since such arrays are mutable and can mutate existing strings. Here is how to create one: >>> import numpy, cPickle as p >>> a = numpy.array([1, 2, 3])

Re: [Numpy-discussion] something wrong with docs?

2009-09-23 Thread Pauli Virtanen
Tue, 22 Sep 2009 23:15:56 -0700, David Goldsmith wrote: [clip] > "It would be nice if we could move gradually towards docs whose examples > (at least those marked as such) were always run via sphinx." Also the >>> examples are doctestable, via numpy.test(doctests=True), or enabling Sphinx's docte