Re: [Numpy-discussion] Bug in genfromtxt with usecols and converters

2014-08-27 Thread Derek Homeier
On 26 Aug 2014, at 09:05 pm, Adrian Altenhoff wrote: >> But you are right that the problem with using the first_values, which should >> of course be valid, >> somehow stems from the use of usecols, it seems that in that loop >> >>for (i, conv) in user_converters.items(): >> >> i in user_c

Re: [Numpy-discussion] Bug in genfromtxt with usecols and converters

2014-08-26 Thread Adrian Altenhoff
Hi Derek, > But you are right that the problem with using the first_values, which should > of course be valid, > somehow stems from the use of usecols, it seems that in that loop > > for (i, conv) in user_converters.items(): > > i in user_converters and in usecols get out of sync. This cert

Re: [Numpy-discussion] Bug in genfromtxt with usecols and converters

2014-08-26 Thread Derek Homeier
Hi Adrian, >> not sure whether to call it a bug; the error seems to arise before reading >> any actual data >> (even on reading from an empty string); when genfromtxt is checking the >> filling_values used >> to substitute missing or invalid data it is apparently testing on default >> testing v

Re: [Numpy-discussion] Bug in genfromtxt with usecols and converters

2014-08-26 Thread Adrian Altenhoff
Hi Derek, thanks for your answer. > not sure whether to call it a bug; the error seems to arise before reading > any actual data > (even on reading from an empty string); when genfromtxt is checking the > filling_values used > to substitute missing or invalid data it is apparently testing on def

Re: [Numpy-discussion] Bug in genfromtxt with usecols and converters

2014-08-26 Thread Derek Homeier
Hi Adrian, > I tried to load data from a csv file into numpy using genfromtxt. I need > only a subset of the columns and want to apply some conversions to the > data. attached is a minimal script showing the error. > In brief, I want to load columns 1,2 and 4. But in the converter > function for t

[Numpy-discussion] Bug in genfromtxt with usecols and converters

2014-08-25 Thread Adrian Altenhoff
Hi, I tried to load data from a csv file into numpy using genfromtxt. I need only a subset of the columns and want to apply some conversions to the data. attached is a minimal script showing the error. In brief, I want to load columns 1,2 and 4. But in the converter function for the 4th column, I

Re: [Numpy-discussion] bug in genfromtxt for python 3.2

2011-03-30 Thread Matthew Brett
Hi, On Wed, Mar 30, 2011 at 11:32 AM, Pauli Virtanen wrote: > On Wed, 30 Mar 2011 10:37:45 -0700, Matthew Brett wrote: > [clip] >> imagine I'm working with a non-latin default encoding, and I've opened a >> file: >> >> fobj = open('my_nonlatin.txt', 'rt') >> >> in python 3.2.  That might contain

Re: [Numpy-discussion] bug in genfromtxt for python 3.2

2011-03-30 Thread Pauli Virtanen
On Wed, 30 Mar 2011 10:37:45 -0700, Matthew Brett wrote: [clip] > imagine I'm working with a non-latin default encoding, and I've opened a > file: > > fobj = open('my_nonlatin.txt', 'rt') > > in python 3.2. That might contain numbers and non-latin text. I can't > pass that into 'genfromtxt' be

Re: [Numpy-discussion] bug in genfromtxt for python 3.2

2011-03-30 Thread Ralf Gommers
On Wed, Mar 30, 2011 at 7:37 PM, Matthew Brett wrote: > Hi, > > On Wed, Mar 30, 2011 at 10:02 AM, Ralf Gommers > wrote: >> On Wed, Mar 30, 2011 at 3:39 AM, Matthew Brett >> wrote: >>> Hi, >>> >>> On Mon, Mar 28, 2011 at 11:29 PM,   wrote: numpy/lib/test_io.py    only uses StringIO in the t

Re: [Numpy-discussion] bug in genfromtxt for python 3.2

2011-03-30 Thread Matthew Brett
Hi, On Wed, Mar 30, 2011 at 10:02 AM, Ralf Gommers wrote: > On Wed, Mar 30, 2011 at 3:39 AM, Matthew Brett > wrote: >> Hi, >> >> On Mon, Mar 28, 2011 at 11:29 PM,   wrote: >>> numpy/lib/test_io.py    only uses StringIO in the test, no actual csv file >>> >>> If I give the filename than I get a

Re: [Numpy-discussion] bug in genfromtxt for python 3.2

2011-03-30 Thread Ralf Gommers
On Wed, Mar 30, 2011 at 3:39 AM, Matthew Brett wrote: > Hi, > > On Mon, Mar 28, 2011 at 11:29 PM,   wrote: >> numpy/lib/test_io.py    only uses StringIO in the test, no actual csv file >> >> If I give the filename than I get a  TypeError: Can't convert 'bytes' >> object to str implicitly >> >> >>

Re: [Numpy-discussion] bug in genfromtxt for python 3.2

2011-03-29 Thread Matthew Brett
Hi, On Mon, Mar 28, 2011 at 11:29 PM, wrote: > numpy/lib/test_io.py    only uses StringIO in the test, no actual csv file > > If I give the filename than I get a  TypeError: Can't convert 'bytes' > object to str implicitly > > > from the statsmodels mailing list example > > data = recfromtxt

[Numpy-discussion] bug in genfromtxt for python 3.2

2011-03-28 Thread josef . pktd
numpy/lib/test_io.pyonly uses StringIO in the test, no actual csv file If I give the filename than I get a TypeError: Can't convert 'bytes' object to str implicitly from the statsmodels mailing list example data = recfromtxt(open('./star98.csv', "U"), delimiter=",", skip_header=1, >>

Re: [Numpy-discussion] bug in genfromtxt with missing values?

2011-01-25 Thread Pierre GM
On Jan 25, 2011, at 9:06 PM, Bruce Southey wrote: > Your filling_values is zero so there is this line (1295?) in the code: > user_filling_values = filling_values or [] > > Which of cause presumes your filling_values is not something like 0 or [0]. That's the bug. I forgot that filling_values cou

Re: [Numpy-discussion] bug in genfromtxt with missing values?

2011-01-25 Thread Bruce Southey
On 01/25/2011 10:56 AM, Skipper Seabold wrote: > On Tue, Jan 25, 2011 at 11:17 AM, Pierre GM wrote: >> On Jan 24, 2011, at 11:47 PM, Skipper Seabold wrote: >> >>> Am I misreading the docs or missing something? Consider the following >>> adapted from here: >>> http://docs.scipy.org/doc/numpy/user/

Re: [Numpy-discussion] bug in genfromtxt with missing values?

2011-01-25 Thread Skipper Seabold
On Tue, Jan 25, 2011 at 11:17 AM, Pierre GM wrote: > > On Jan 24, 2011, at 11:47 PM, Skipper Seabold wrote: > >> Am I misreading the docs or missing something?  Consider the following >> adapted from here: >> http://docs.scipy.org/doc/numpy/user/basics.io.genfromtxt.html >> >> from StringIO import

Re: [Numpy-discussion] bug in genfromtxt with missing values?

2011-01-25 Thread Pierre GM
On Jan 24, 2011, at 11:47 PM, Skipper Seabold wrote: > Am I misreading the docs or missing something? Consider the following > adapted from here: > http://docs.scipy.org/doc/numpy/user/basics.io.genfromtxt.html > > from StringIO import StringIO > import numpy as np > > data = "1, 2, 3\n4, ,5"

[Numpy-discussion] bug in genfromtxt with missing values?

2011-01-24 Thread Skipper Seabold
Am I misreading the docs or missing something? Consider the following adapted from here: http://docs.scipy.org/doc/numpy/user/basics.io.genfromtxt.html from StringIO import StringIO import numpy as np data = "1, 2, 3\n4, ,5" np.genfromtxt(StringIO(data), delimiter=",", names="a,b,c", missing_va

[Numpy-discussion] bug in genfromtxt

2010-10-12 Thread Skipper Seabold
With names=True, if you specify the dtype manually and specify usecols, it doesn't grab the correct names along with the columns. Should be a simple fix, but haven't had time to write a patch yet. from stringIO import stringIO import numpy as np np.__version__ #1.5.0 s = StringIO("col0,col1,col2,c