Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-14 Thread Ralf Gommers
On Wed, Dec 14, 2011 at 9:54 PM, Chris Barker wrote: > On Wed, Dec 14, 2011 at 11:36 AM, Benjamin Root wrote: > >>> well, yes, though it does do a lot -- do you have a smpler one in mind? > >>> > >> Just looking at what I normally wouldn't need for simple data files > and/or > >> what a beginnin

Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-14 Thread Chris Barker
On Wed, Dec 14, 2011 at 11:36 AM, Benjamin Root wrote: >>> well, yes, though it does do a lot -- do you have a smpler one in mind? >>> >> Just looking at what I normally wouldn't need for simple data files and/or >> what a beginning user won't understand at once, the `unpack` and `ndmin` >> keywor

Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-14 Thread Benjamin Root
On Wed, Dec 14, 2011 at 1:22 PM, Ralf Gommers wrote: > > > On Wed, Dec 14, 2011 at 4:11 PM, Bruce Southey wrote: > >> ** >> On 12/14/2011 01:03 AM, Chris Barker wrote: >> >> >> >> On Tue, Dec 13, 2011 at 1:21 PM, Ralf Gommers < >> ralf.gomm...@googlemail.com> wrote: >> >>> >>> genfromtxt sure l

Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-14 Thread Ralf Gommers
On Wed, Dec 14, 2011 at 4:11 PM, Bruce Southey wrote: > ** > On 12/14/2011 01:03 AM, Chris Barker wrote: > > > > On Tue, Dec 13, 2011 at 1:21 PM, Ralf Gommers > wrote: > >> >> genfromtxt sure looks close for an API >>> >> >> This I don't agree with. It has a huge amount of keywords that just >

Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-14 Thread Bruce Southey
On 12/14/2011 01:03 AM, Chris Barker wrote: On Tue, Dec 13, 2011 at 1:21 PM, Ralf Gommers mailto:ralf.gomm...@googlemail.com>> wrote: genfromtxt sure looks close for an API This I don't agree with. It has a huge amount of keywords that just confuse or intimidate a beginnin

Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-13 Thread Chris Barker
On Tue, Dec 13, 2011 at 1:21 PM, Ralf Gommers wrote: > > genfromtxt sure looks close for an API >> > > This I don't agree with. It has a huge amount of keywords that just > confuse or intimidate a beginning user. There should be a dead simple > interface, even the loadtxt API is on the heavy side.

Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-13 Thread Wes McKinney
On Mon, Dec 12, 2011 at 12:34 PM, Warren Weckesser wrote: > > > On Mon, Dec 12, 2011 at 10:22 AM, Chris.Barker > wrote: >> >> On 12/11/11 8:40 AM, Ralf Gommers wrote: >> > On Wed, Dec 7, 2011 at 7:50 PM, Chris.Barker > >     * If we have a good, fast ascii (or unicode?) to array reader, >> > hope

Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-13 Thread Ralf Gommers
On Tue, Dec 13, 2011 at 10:07 PM, Chris Barker wrote: > On Tue, Dec 13, 2011 at 11:29 AM, Bruce Southey wrote: > >> ** >> Reading data is hard and writing code that suits the diversity in the >> Numerical Python community is even harder! >> >> > yup > > Both loadtxt and genfromtxt functions (other

Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-13 Thread Chris Barker
On Tue, Dec 13, 2011 at 11:29 AM, Bruce Southey wrote: > ** > Reading data is hard and writing code that suits the diversity in the > Numerical Python community is even harder! > > yup Both loadtxt and genfromtxt functions (other functions are perhaps less > important) perhaps need an upgrade to

Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-13 Thread Bruce Southey
On 12/13/2011 12:08 PM, Chris Barker wrote: NOTE: Let's keep this on the list. On Tue, Dec 13, 2011 at 9:19 AM, denis > wrote: Chris, unified, consistent save / load is a nice goal 1) header lines with date, pwd etc.: "where'd this come from ?"

Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-13 Thread Chris Barker
NOTE: Let's keep this on the list. On Tue, Dec 13, 2011 at 9:19 AM, denis wrote: > Chris, > unified, consistent save / load is a nice goal > > 1) header lines with date, pwd etc.: "where'd this come from ?" > ># (5, 5) svm.py bz/py/ml/svm 2011-12-13 Dec 11:56 -- automatic ># 80.6 %

Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-12 Thread Warren Weckesser
On Mon, Dec 12, 2011 at 10:22 AM, Chris.Barker wrote: > On 12/11/11 8:40 AM, Ralf Gommers wrote: > > On Wed, Dec 7, 2011 at 7:50 PM, Chris.Barker > * If we have a good, fast ascii (or unicode?) to array reader, > hopefully > > it could be leveraged for use in the more complex cases. So th

Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-12 Thread Chris.Barker
On 12/11/11 8:40 AM, Ralf Gommers wrote: > On Wed, Dec 7, 2011 at 7:50 PM, Chris.Barker * If we have a good, fast ascii (or unicode?) to array reader, hopefully > it could be leveraged for use in the more complex cases. So that rather > than genfromtxt() being written from scratch, it

Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-11 Thread Ralf Gommers
On Wed, Dec 7, 2011 at 7:50 PM, Chris.Barker wrote: > Hi folks, > > This is a continuation of a conversation already started, but i gave it > a new, more appropriate, thread and subject. > > On 12/6/11 2:13 PM, Wes McKinney wrote: > > we should start talking > > about building a *high performance

[Numpy-discussion] Fast Reading of ASCII files

2011-12-07 Thread Chris.Barker
Hi folks, This is a continuation of a conversation already started, but i gave it a new, more appropriate, thread and subject. On 12/6/11 2:13 PM, Wes McKinney wrote: > we should start talking > about building a *high performance* flat file loading solution with > good column type inference and