Re: [Tutor] How to parse large files

2015-11-01 Thread Danny Yoo
On Sun, Nov 1, 2015 at 5:17 PM, Peter Otten <__pete...@web.de> wrote: > jarod_v6--- via Tutor wrote: > >> Thanks!! >> I use python2.7 Can Also use in that version? > > Yes. Hi Jarod, Also for reference, here are the equivalent 2.7 doc links: https://docs.python.org/2/reference/compound_stm

Re: [Tutor] How to parse large files

2015-11-01 Thread Peter Otten
Alan Gauld wrote: > On 01/11/15 20:33, jarod_v6--- via Tutor wrote: >> Thanks!! >> I use python2.7 Can Also use in that version? >> I don't understand why use partition and not split(). what is the reason >> for that? > > I'm confused? Where did partition v split enter into things? > Am I missin

Re: [Tutor] How to parse large files

2015-11-01 Thread Alan Gauld
On 01/11/15 20:33, jarod_v6--- via Tutor wrote: Thanks!! I use python2.7 Can Also use in that version? I don't understand why use partition and not split(). what is the reason for that? I'm confused? Where did partition v split enter into things? Am I missing a message some place? -- Alan G

Re: [Tutor] How to parse large files

2015-11-01 Thread Peter Otten
jarod_v6--- via Tutor wrote: > Thanks!! > I use python2.7 Can Also use in that version? Yes. > I don't understand why use partition and not split(). what is the reason > for that? If there is exactly one "\t" in the line key, value = line.split("\t") and key, _tab, value = line.partition("\

Re: [Tutor] How to parse large files

2015-11-01 Thread jarod_v6--- via Tutor
Thanks!! I use python2.7 Can Also use in that version? I don't understand why use partition and not split(). what is the reason for that? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/ma

Re: [Tutor] How to parse large files

2015-11-01 Thread Peter Otten
Danny Yoo wrote: >> AttributeErrorTraceback (most recent call >> last) in () >> > 1 with shelve.open("diz5") as db: >> 2 with open("tmp1.txt") as instream: >> 3 for line in instream: >> 4 assert line.count("\t") == 1 >>

Re: [Tutor] How to parse large files

2015-11-01 Thread Danny Yoo
> AttributeErrorTraceback (most recent call last) > in () > > 1 with shelve.open("diz5") as db: > 2 with open("tmp1.txt") as instream: > 3 for line in instream: > 4 assert line.count("\t") == 1 > 5 key, _ta

Re: [Tutor] How to parse large files

2015-11-01 Thread jarod_v6--- via Tutor
My file have 1960607 rows but I don't understand why I'm not able to create a dictionary in fast way I try to use also gc.disable but Not work. I need to have dictionary but I have this erro: with shelve.open("diz5") as db: with open("tmp1.txt") as instream: for line in instream:

Re: [Tutor] How to parse large files

2015-10-28 Thread Peter Otten
Danny Yoo wrote: > There are several out there; one that comes standard in Python 3 is > the "dbm" module: > > https://docs.python.org/3.5/library/dbm.html > > Instead of doing: > > diz5 = {} > ... > > we'd do something like this: > > with diz5 = dbm.open('diz5, 'c'): > ... > > And otherwise

Re: [Tutor] How to parse large files

2015-10-27 Thread Danny Yoo
> Instead of doing: > > diz5 = {} > ... > > we'd do something like this: > > with diz5 = dbm.open('diz5, 'c'): Sorry, I'm getting my syntax completely wrong here. My apologies. This should be: with dbm.open('diz5', 'c') as diz5: ... Apologies: I just got back from work

Re: [Tutor] How to parse large files

2015-10-27 Thread Danny Yoo
On Tue, Oct 27, 2015 at 2:32 PM, jarod_v6--- via Tutor wrote: > Hi! > I want to reads two files and create simple dictionary. Input file contain > more than 1 rows > > diz5 = {} > with open("tmp1.txt") as p: > for i in p: > lines = i.rstrip("\n").split("\t") > diz5.setde

Re: [Tutor] How to parse large files

2015-10-27 Thread Steven D'Aprano
On Tue, Oct 27, 2015 at 10:32:51PM +0100, jarod_v6--- via Tutor wrote: > Hi! > I want to reads two files and create simple dictionary. Input file contain > more than 1 rows Since you are talking about a tab-delimited CSV file, you should use the csv module: https://pymotw.com/2/csv/ http

Re: [Tutor] How to parse large files

2015-10-27 Thread Mark Lawrence
On 27/10/2015 21:32, jarod_v6--- via Tutor wrote: Hi! I want to reads two files and create simple dictionary. Input file contain more than 1 rows diz5 = {} with open("tmp1.txt") as p: for i in p: lines = i.rstrip("\n").split("\t") diz5.setdefault(lines[0],set()).ad

[Tutor] How to parse large files

2015-10-27 Thread jarod_v6--- via Tutor
Hi! I want to reads two files and create simple dictionary. Input file contain more than 1 rows diz5 = {} with open("tmp1.txt") as p: for i in p: lines = i.rstrip("\n").split("\t") diz5.setdefault(lines[0],set()).add(lines[1]) diz3 = {} with open("tmp2.txt") as p: