Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Thu, Jan 23, 2014 at 11:49 AM, Chris Barker wrote: > Thanks for poking into this all. I've lost track a bit, but I think: > > The 'S' type is clearly broken on py3 (at least). I think that gives us > room to change it, and backward compatibly is less of an issue because it's > broken already --

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On 24 January 2014 01:09, Chris Barker wrote: > On Thu, Jan 23, 2014 at 4:02 PM, Oscar Benjamin > wrote: >> >> On 23 January 2014 21:51, Chris Barker wrote: >> > >> > However, I would prefer latin-1 -- that way you might get garbage for >> > the >> > non-ascii parts, but it wouldn't raise an e

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Thu, Jan 23, 2014 at 3:56 PM, wrote: > > I'm not sure anymore, after all these threads I think bytes should be > bytes and strings should be strings > exactly -- that's the py3 model, and I think we really soudl try to conform to it, it's really the only way to have a robust solution. > I l

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Thu, Jan 23, 2014 at 4:02 PM, Oscar Benjamin wrote: > On 23 January 2014 21:51, Chris Barker wrote: > > > > However, I would prefer latin-1 -- that way you might get garbage for > the > > non-ascii parts, but it wouldn't raise an exception and it round-trips > > through encoding/decoding. An

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On 23 January 2014 21:51, Chris Barker wrote: > > However, I would prefer latin-1 -- that way you might get garbage for the > non-ascii parts, but it wouldn't raise an exception and it round-trips > through encoding/decoding. And you would have a somewhat more useful subset > -- including the la

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Thu, Jan 23, 2014 at 4:51 PM, Chris Barker wrote: > On Thu, Jan 23, 2014 at 12:10 PM, wrote: >> >> > Exactly -- but what should those conversion/casting rules be? We can't >> > decide that unless we decide if 'S' is for text or for arbitrary bytes >> > -- it >> > can't be both. I say text, tha

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Thu, Jan 23, 2014 at 12:10 PM, wrote: > > Exactly -- but what should those conversion/casting rules be? We can't > > decide that unless we decide if 'S' is for text or for arbitrary bytes > -- it > > can't be both. I say text, that's what it's mostly trying to do already. > But > > if it's byt

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Thu, Jan 23, 2014 at 1:36 PM, Oscar Benjamin wrote: > On 23 January 2014 17:42, wrote: >> On Thu, Jan 23, 2014 at 12:13 PM, wrote: >>> On Thu, Jan 23, 2014 at 11:58 AM, wrote: No, a view doesn't change the memory, it just changes the interpretation and there shouldn't be an

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Thu, Jan 23, 2014 at 2:45 PM, Chris Barker wrote: > On Thu, Jan 23, 2014 at 11:18 AM, wrote: > >> >> I think this is just inconsistent casting rules in numpy, >> >> numpy should either refuse to assign the wrong type, instead of using >> the repr as in some of the earlier examples of Oscar >>

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

>> > numpy arrays need a decode and encode method > > >> I'm not sure that they do. Rather there needs to be a text dtype that >> knows what encoding to use in order to have a binary interface as >> exposed by .tostring() and friends and but produce unicode strings >> when indexed from Python code.

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Thu, Jan 23, 2014 at 11:18 AM, wrote: > I think this is just inconsistent casting rules in numpy, > > numpy should either refuse to assign the wrong type, instead of using > the repr as in some of the earlier examples of Oscar > > >>> s = np.inf > >>> np.array((s,), dtype=int)[0] == s > Trace

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Thu, Jan 23, 2014 at 1:49 PM, Chris Barker wrote: > > s = 'a string' > np.array((s,), dtype='S')[0] == s > > Gives you False, rather than True on py2. This is because a py3 string is > translated to the 'S' type (presumable with the default encoding, another > maybe not a good idea, but return

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

Thanks for poking into this all. I've lost track a bit, but I think: The 'S' type is clearly broken on py3 (at least). I think that gives us room to change it, and backward compatibly is less of an issue because it's broken already -- do we need to preserve bug-for-bug compatibility? Maybe, but I

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On 23 January 2014 17:42, wrote: > On Thu, Jan 23, 2014 at 12:13 PM, wrote: >> On Thu, Jan 23, 2014 at 11:58 AM, wrote: >>> >>> No, a view doesn't change the memory, it just changes the >>> interpretation and there shouldn't be any conversion involved. >>> astype does type conversion, but it

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Thu, Jan 23, 2014 at 12:13 PM, wrote: > On Thu, Jan 23, 2014 at 11:58 AM, wrote: >> On Thu, Jan 23, 2014 at 11:43 AM, Oscar Benjamin >> wrote: >>> On Thu, Jan 23, 2014 at 11:23:09AM -0500, josef.p...@gmail.com wrote: another curious example, encode utf-8 to latin-1 bytes

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Thu, Jan 23, 2014 at 11:58 AM, wrote: > On Thu, Jan 23, 2014 at 11:43 AM, Oscar Benjamin > wrote: >> On Thu, Jan 23, 2014 at 11:23:09AM -0500, josef.p...@gmail.com wrote: >>> >>> another curious example, encode utf-8 to latin-1 bytes >>> >>> >>> b >>> array(['Õsc', 'zxc'], >>> dtype='>>

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Thu, Jan 23, 2014 at 11:43 AM, Oscar Benjamin wrote: > On Thu, Jan 23, 2014 at 11:23:09AM -0500, josef.p...@gmail.com wrote: >> >> another curious example, encode utf-8 to latin-1 bytes >> >> >>> b >> array(['Õsc', 'zxc'], >> dtype='> >>> b[0].encode('utf8') >> b'\xc3\x95sc' >> >>> b[0].e

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Thu, Jan 23, 2014 at 11:23:09AM -0500, josef.p...@gmail.com wrote: > > another curious example, encode utf-8 to latin-1 bytes > > >>> b > array(['Õsc', 'zxc'], > dtype=' >>> b[0].encode('utf8') > b'\xc3\x95sc' > >>> b[0].encode('latin1') > b'\xd5sc' > >>> b.astype('S') > Traceback (most

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Thu, Jan 23, 2014 at 10:41 AM, wrote: > On Thu, Jan 23, 2014 at 5:45 AM, Oscar Benjamin > wrote: >> On Wed, Jan 22, 2014 at 05:53:26PM -0800, Chris Barker - NOAA Federal wrote: >>> On Jan 22, 2014, at 1:13 PM, Oscar Benjamin >>> wrote: >>> >>> > >>> > It's not safe to stop removing the null

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Thu, Jan 23, 2014 at 5:45 AM, Oscar Benjamin wrote: > On Wed, Jan 22, 2014 at 05:53:26PM -0800, Chris Barker - NOAA Federal wrote: >> On Jan 22, 2014, at 1:13 PM, Oscar Benjamin >> wrote: >> >> > >> > It's not safe to stop removing the null bytes. This is how numpy determines >> > the length

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Thu, Jan 23, 2014 at 5:45 AM, Oscar Benjamin wrote: > On Wed, Jan 22, 2014 at 05:53:26PM -0800, Chris Barker - NOAA Federal wrote: >> On Jan 22, 2014, at 1:13 PM, Oscar Benjamin >> wrote: >> >> > >> > It's not safe to stop removing the null bytes. This is how numpy determines >> > the length

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Wed, Jan 22, 2014 at 05:53:26PM -0800, Chris Barker - NOAA Federal wrote: > On Jan 22, 2014, at 1:13 PM, Oscar Benjamin > wrote: > > > > > It's not safe to stop removing the null bytes. This is how numpy determines > > the length of the strings in a dtype='S' array. The strings are not > > "f

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Jan 22, 2014, at 1:13 PM, Oscar Benjamin wrote: > > It's not safe to stop removing the null bytes. This is how numpy determines > the length of the strings in a dtype='S' array. The strings are not > "fixed-width" but rather have a maximum width. Exactly--but folks have told us on this list t

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Wed, Jan 22, 2014 at 12:07:28PM -0800, Chris Barker wrote: > On Wed, Jan 22, 2014 at 2:46 AM, Oscar Benjamin > wrote: > > > BTW, as much as the fixed-width 'S' dtype doesn't really work for str in > > Python 3 it's also a poor fit for bytes since it strips trailing nulls: > > > > >>> a = np.ar

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Wed, Jan 22, 2014 at 2:46 AM, Oscar Benjamin wrote: > BTW, as much as the fixed-width 'S' dtype doesn't really work for str in > Python 3 it's also a poor fit for bytes since it strips trailing nulls: > > >>> a = np.array(['a\0s\0', 'qwert'], dtype='S') > >>> a > array([b'a\x00s', b'qwert'], >

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

Hi Oscar, > Is it fair to say that people should really be using vlen utf-8 strings for > text? Is it problematic because of the need to interface with non-Python > libraries using the same hdf5 file? The general recommendation has been to use fixed-width strings for exactly that reason; FORTRAN

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Tue, Jan 21, 2014 at 06:54:33PM -0700, Andrew Collette wrote: > Hi Chris, > > > it looks from here: > > http://www.hdfgroup.org/HDF5/doc/ADGuide/WhatsNew180.html > > > > that HDF uses utf-8 for unicode strings -- so you _could_ roundtrip with a > > lot of calls to encode/decode -- which could b

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

Hi Chris, > it looks from here: > http://www.hdfgroup.org/HDF5/doc/ADGuide/WhatsNew180.html > > that HDF uses utf-8 for unicode strings -- so you _could_ roundtrip with a > lot of calls to encode/decode -- which could be pretty slow, compared to > other ways to dump numpy arrays into HDF-5 -- that

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Tue, Jan 21, 2014 at 3:22 PM, Andrew Collette wrote: > Just stumbled on this discussion (I'm the lead author of h5py). > > We would be overjoyed if there were a 1-byte text type available in > NumPy. cool -- it looks like someone is going to get a draft PEP going -- so stay tuned, and add you

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

Hi Chris, Just stumbled on this discussion (I'm the lead author of h5py). We would be overjoyed if there were a 1-byte text type available in NumPy. String handling is the source of major pain right now in the HDF5 world. All HDF5 strings are text (opaque types are used for binary data), but we

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

Gestuur vanaf my Samsung S3 Mini Original message From: josef.p...@gmail.com Date: 18/01/2014 04:16 (GMT+02:00) To: Discussion of Numerical Python Subject: Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array It looks like both recfromtxt and

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

It looks like both recfromtxt and loadtxt are flexible enough to handle string/bytes en/decoding, - with a bit of work and using enough information >>> dtype=[('f0', '>> data = numpy.recfromtxt(open('Õscar_3.txt',"rb"), dtype=dtype, >>> delimiter=',',converters={3:lambda x: x.decode('utf8')}) >>>

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Fri, Jan 17, 2014 at 4:43 PM, wrote: > On Fri, Jan 17, 2014 at 4:20 PM, Chris Barker > wrote: > > On Fri, Jan 17, 2014 at 12:36 PM, wrote: > >> > >> > ('S' ?) -- which is probably not what you want particularly if you > >> > specify > >> > an encoding. Though I can't figure out at the moment

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Fri, Jan 17, 2014 at 1:43 PM, wrote: > > 2) Either: > > a) open as a binary file and use bytes for anything that doesn't > parse > > as text -- this means that the user will need to do the conversion to > text > > themselves > > > > b) decode as latin-1: this would work well for ascii an

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Fri, Jan 17, 2014 at 4:20 PM, Chris Barker wrote: > On Fri, Jan 17, 2014 at 12:36 PM, wrote: >> >> > ('S' ?) -- which is probably not what you want particularly if you >> > specify >> > an encoding. Though I can't figure out at the moment why the previous >> > one >> > failed -- where did the

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Fri, Jan 17, 2014 at 12:36 PM, wrote: > > ('S' ?) -- which is probably not what you want particularly if you > specify > > an encoding. Though I can't figure out at the moment why the previous one > > failed -- where did the bytes object come from when the encoding was > > specified? > > Yes,

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

Small note: Being an English speaker I don't normally use non-ascii characters in > filenames but my system (Ubuntu Linux) still uses utf-8 rather than > latin-1 or > (and rightly so!). just to be really clear -- encoding for filenames and encoding for file content have nothing to do with each-o

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Fri, Jan 17, 2014 at 3:17 PM, Chris Barker wrote: > >>> numpy.recfromtxt(open('Õscar_3.txt',"r", encoding='utf8'), > delimiter=',') >> >> Traceback (most recent call last): >> File "", line 1, in >> numpy.recfromtxt(open('Õscar_3.txt',"r", encoding='utf8'), >> delimiter=',') >> File "

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Fri, Jan 17, 2014 at 5:18 AM, Freddie Witherden wrote: > In terms of HDF5 it is interesting to look at how h5py -- which has to > go between NumPy types and HDF5 conventions -- handles the problem as > described here: > > http://www.h5py.org/docs/topics/strings.html from that: """All strin

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

>>> numpy.recfromtxt(open('Õscar_3.txt',"r", encoding='utf8'), delimiter=',') > Traceback (most recent call last): > File "", line 1, in > numpy.recfromtxt(open('Õscar_3.txt',"r", encoding='utf8'), > delimiter=',') > File "C:\Programs\Python33\lib\site-packages\numpy\lib\npyio.py", > lin

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Fri, Jan 17, 2014 at 1:38 AM, Julian Taylor < jtaylor.deb...@googlemail.com> wrote: > > This thread is getting a little out of hand which is my fault for > initially mixing different topics in one mail, > still a bit mixed ;-) -- but I think the loadtxt issue requires a lot less discussion,

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Fri, Jan 17, 2014 at 2:18 PM, Julian Taylor wrote: > On 17.01.2014 15:12, Julian Taylor wrote: >> On Fri, Jan 17, 2014 at 2:40 PM, Oscar Benjamin >> mailto:oscar.j.benja...@gmail.com>> wrote: >> >> On Fri, Jan 17, 2014 at 02:10:19PM +0100, Julian Taylor wrote: >> > On Fri, Jan 17, 2014

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On 17.01.2014 15:12, Julian Taylor wrote: > On Fri, Jan 17, 2014 at 2:40 PM, Oscar Benjamin > mailto:oscar.j.benja...@gmail.com>> wrote: > > On Fri, Jan 17, 2014 at 02:10:19PM +0100, Julian Taylor wrote: > > On Fri, Jan 17, 2014 at 1:44 PM, Oscar Benjamin > > mailto:oscar.j.benja...@gm

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

17.01.2014 15:09, Aldcroft, Thomas kirjoitti: [clip] > I've been playing around with porting a stack of analysis libraries > to Python 3 and this is a very timely thread and comment. What I > discovered right away is that all the string data coming from > binary HDF5 files show up (as expected) as

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Fri, Jan 17, 2014 at 10:58:25AM -0500, josef.p...@gmail.com wrote: > On Fri, Jan 17, 2014 at 10:26 AM, Oscar Benjamin > wrote: > > On Fri, Jan 17, 2014 at 03:12:32PM +0100, Julian Taylor wrote: > > > > You don't show how you created the file. I think that in your case the > > content of 'filena

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Fri, Jan 17, 2014 at 10:26 AM, Oscar Benjamin wrote: > On Fri, Jan 17, 2014 at 03:12:32PM +0100, Julian Taylor wrote: >> On Fri, Jan 17, 2014 at 2:40 PM, Oscar Benjamin >> wrote: >> >> > On Fri, Jan 17, 2014 at 02:10:19PM +0100, Julian Taylor wrote: >> > > >> > > no, the right solution is to ad

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Fri, Jan 17, 2014 at 03:12:32PM +0100, Julian Taylor wrote: > On Fri, Jan 17, 2014 at 2:40 PM, Oscar Benjamin > wrote: > > > On Fri, Jan 17, 2014 at 02:10:19PM +0100, Julian Taylor wrote: > > > > > > no, the right solution is to add an encoding argument. > > > Its a 4 line patch for python2 and

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Fri, Jan 17, 2014 at 2:40 PM, Oscar Benjamin wrote: > On Fri, Jan 17, 2014 at 02:10:19PM +0100, Julian Taylor wrote: > > On Fri, Jan 17, 2014 at 1:44 PM, Oscar Benjamin > > wrote: > > > > > On Fri, Jan 17, 2014 at 10:59:27AM +, Pauli Virtanen wrote: > > > > Julian Taylor googlemail.com> wr

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Fri, Jan 17, 2014 at 8:40 AM, Oscar Benjamin wrote: > On Fri, Jan 17, 2014 at 02:10:19PM +0100, Julian Taylor wrote: >> On Fri, Jan 17, 2014 at 1:44 PM, Oscar Benjamin >> wrote: >> >> > On Fri, Jan 17, 2014 at 10:59:27AM +, Pauli Virtanen wrote: >> > > Julian Taylor googlemail.com> writes:

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Fri, Jan 17, 2014 at 02:10:19PM +0100, Julian Taylor wrote: > On Fri, Jan 17, 2014 at 1:44 PM, Oscar Benjamin > wrote: > > > On Fri, Jan 17, 2014 at 10:59:27AM +, Pauli Virtanen wrote: > > > Julian Taylor googlemail.com> writes: > > > [clip] > > > > > > > For backward compatibility we *ca

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Fri, Jan 17, 2014 at 2:10 PM, Julian Taylor < jtaylor.deb...@googlemail.com> wrote: > On Fri, Jan 17, 2014 at 1:44 PM, Oscar Benjamin < > oscar.j.benja...@gmail.com> wrote:... > ... > No latin1 de/encoding is required for anything, I don't know why you would > want do to that in this context. >

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On 17/01/14 13:09, Aldcroft, Thomas wrote: > I've been playing around with porting a stack of analysis libraries to > Python 3 and this is a very timely thread and comment. What I > discovered right away is that all the string data coming from binary > HDF5 files show up (as expected) as 'S' type,

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Fri, Jan 17, 2014 at 1:44 PM, Oscar Benjamin wrote: > On Fri, Jan 17, 2014 at 10:59:27AM +, Pauli Virtanen wrote: > > Julian Taylor googlemail.com> writes: > > [clip] > > > > For backward compatibility we *cannot* change S. > > Do you mean to say that loadtxt cannot be changed from decodi

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Fri, Jan 17, 2014 at 5:59 AM, Pauli Virtanen wrote: > Julian Taylor googlemail.com> writes: > [clip] > > - inconvenience in dealing with strings in python 3. > > > > bytes are not strings in python3 which means ascii data is either a byte > > array which can be inconvenient to deal with or 4

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Fri, Jan 17, 2014 at 10:59:27AM +, Pauli Virtanen wrote: > Julian Taylor googlemail.com> writes: > [clip] > > - inconvenience in dealing with strings in python 3. > > > > bytes are not strings in python3 which means ascii data is either a byte > > array which can be inconvenient to deal wi

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Fri, Jan 17, 2014 at 5:59 AM, Pauli Virtanen wrote: > Julian Taylor googlemail.com> writes: > [clip] >> - inconvenience in dealing with strings in python 3. >> >> bytes are not strings in python3 which means ascii data is either a byte >> array which can be inconvenient to deal with or 4 byte

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

Julian Taylor googlemail.com> writes: [clip] > For backward compatibility we *cannot* change S. > Maybe we could change the meaning of 'a' but it would be safer > to add a new dtype, possibly 'S' can be deprecated in favor > of 'B' when we have a specific encoding dtype. Note that the rename 'S'

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

Julian Taylor googlemail.com> writes: [clip] > - inconvenience in dealing with strings in python 3. > > bytes are not strings in python3 which means ascii data is either a byte > array which can be inconvenient to deal with or 4 byte unicode which > wastes space. > > A proposal to fix this would

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

This thread is getting a little out of hand which is my fault for initially mixing different topics in one mail, so let me try to summarize: We have three issues here: - a loadtxt bug when loading strings in python3 this has nothing to do with encodings or dtypes it is a bug that should be fixed.

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Thu, Jan 16, 2014 at 2:43 AM, Oscar Benjamin wrote: > > My proposal: > > > > loadtxt accepts an encoding argument. > > > > default is ascii -- that's what it's doing now, anyway, yes? > > No it's loading the file reading a line, encoding the line with latin-1, > and > then putting the repr of t

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Wed, Jan 15, 2014 at 11:40:58AM -0800, Chris Barker wrote: > On Wed, Jan 15, 2014 at 9:57 AM, Charles R Harris > wrote: > > > > There was a discussion of this long ago and UCS-4 was chosen as the numpy > > standard. There are just too many complications that arise in supporting > > both. > >

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Wed, Jan 15, 2014 at 3:58 PM, Julian Taylor < jtaylor.deb...@googlemail.com> wrote: > > fileContent=loadtxt(filePath,dtype='S') > > this gives you bytes not a string, this can only be fixed by adding new > dtypes, or changing the behavior or dtype 'S', but yes, the other thread. But the OP's

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On 15 January 2014 12:38, Julian Taylor wrote: > On 01/15/2014 11:25 AM, Daπid wrote: >> On 15 January 2014 11:12, Hedieh Ebrahimi > > wrote: >> >> I try to print my fileContent array after I read it and it looks >> like this : >> >> ["b'C:Users

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On 16.01.2014 00:42, Chris Barker wrote: > bump back to the OP: > On Wed, Jan 15, 2014 at 2:12 AM, Hedieh Ebrahimi > mailto:hedieh.ebrah...@amphos21.com>> wrote: > > fileContent=loadtxt(filePath,dtype=str) > > > do either of these work for you? > > fileContent=loadtxt(filePath,dtype='S') t

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

bump back to the OP: On Wed, Jan 15, 2014 at 2:12 AM, Hedieh Ebrahimi < hedieh.ebrah...@amphos21.com> wrote: > fileContent=loadtxt(filePath,dtype=str) > do either of these work for you? fileContent=loadtxt(filePath,dtype='S') or fileContent=loadtxt(filePath,dtype=np.unicode) -Chris -- Chr

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array (Charles R Harris)

On Wed, Jan 15, 2014 at 9:52 AM, wrote: > Date: Wed, 15 Jan 2014 10:57:51 -0700 > From: Charles R Harris > Subject: Re: [Numpy-discussion] using loadtxt to load a text file in > to a numpy array > To: Discussion of Numerical Python > Message-ID: > < &g

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Wed, Jan 15, 2014 at 9:57 AM, Charles R Harris wrote: > There was a discussion of this long ago and UCS-4 was chosen as the numpy > standard. There are just too many complications that arise in supporting > both. > fair enough -- but loadtxt appears to be broken just the same. Any proposals

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Wed, Jan 15, 2014 at 10:27 AM, Chris Barker wrote: > On Wed, Jan 15, 2014 at 4:38 AM, Julian Taylor < > jtaylor.deb...@googlemail.com> wrote: > >> > I try to print my fileContent array after I read it and it looks >> > like this : >> > >> > ["b'C:UsersDocumentsProject\\

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Wed, Jan 15, 2014 at 4:38 AM, Julian Taylor < jtaylor.deb...@googlemail.com> wrote: > > I try to print my fileContent array after I read it and it looks > > like this : > > > > ["b'C:UsersDocumentsProjectmytextfile1.txt'" > > "b'C:UsersDocumentsPro

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On 01/15/2014 01:38 PM, Julian Taylor wrote: > On 01/15/2014 11:25 AM, Daπid wrote: >> On 15 January 2014 11:12, Hedieh Ebrahimi for utf 8 data: > > d = np.loadtxt(file, dtype='utf8') > ups this is a very bad example as we can't have utf8 as its variable length, but we can have ascii and uc

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On 01/15/2014 11:25 AM, Daπid wrote: > On 15 January 2014 11:12, Hedieh Ebrahimi > wrote: > > I try to print my fileContent array after I read it and it looks > like this : > > ["b'C:UsersDocumentsProjectmytextfile1.txt'" > "b'

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On 15 January 2014 11:12, Hedieh Ebrahimi wrote: > I try to print my fileContent array after I read it and it looks like this > : > > ["b'C:UsersDocumentsProjectmytextfile1.txt'" > "b'C:UsersDocumentsProjectmytextfile2.txt'" > "b'C:UsersDocumentsProjec

[Numpy-discussion] using loadtxt to load a text file in to a numpy array

Hello, I am trying to use the following line of code : fileContent=loadtxt(filePath,dtype=str) in order to load a text file located at path= filePath in to a numpy array called fileContent. I´ve simplifed my file for the purpose of this question but the file looks something like this: file Co