On 3/11/2012 6:56 PM Robert Sjoblom said...
I'm sorry if the subject is vague, but I can't really explain it very
well. I've been away from programming for a while now (I got a
daughter and a year after that a son, so I've been busy with family
matters). As such, my skills are definitely rusty.
On 12/03/12 03:28, Steven D'Aprano wrote:
Another approach may be to read the whole file into memory in one big
chunk. 1.1 million lines, by (say) 50 characters per line comes to about
53 MB per file, which should be small enough to read into memory and
process it in one chunk. Something like th
On Mon, Mar 12, 2012 at 05:46:39AM +0100, Robert Sjoblom wrote:
> > You haven't shown us the critical part: how are you getting the lines in
> > the first place?
>
> Ah, yes --
> with open(address, "r", encoding="cp1252") as instream:
> for line in instream:
Seems reasonable.
> > (Also, you
Erik Rise gave a good talk today at PyCon about a parsing library he's
working on called Parsimonious. You could maybe look into what he's doing
there, and see if that helps you any... Follow him on Twitter at @erikrose
to see when his session's video is up. His session was named "Parsing
Horrible
> You haven't shown us the critical part: how are you getting the lines in
> the first place?
Ah, yes --
with open(address, "r", encoding="cp1252") as instream:
for line in instream:
> (Also, you shouldn't shadow built-ins like list as you do above, unless
> you know what you are doing. If yo
On Mon, Mar 12, 2012 at 02:56:36AM +0100, Robert Sjoblom wrote:
> In the file I'm parsing, I'm looking for specific lines. I don't know
> the content of these lines but I do know the content that appears two
> lines before. As such I thought that maybe I'd flag for a found line
> and then flag the
I'm sorry if the subject is vague, but I can't really explain it very
well. I've been away from programming for a while now (I got a
daughter and a year after that a son, so I've been busy with family
matters). As such, my skills are definitely rusty.
In the file I'm parsing, I'm looking for speci
