Hey Frederic:
thanks for the response. I really want it to do it your way but I am
a bad programmer. Do you have any sample code? your method seems
correct
2009/7/10 Frédéric Bastien :
> Can you do it by chunk instead of by row? If the chunk is not too big the
> sort could be faster then the ac
Can you do it by chunk instead of by row? If the chunk is not too big the
sort could be faster then the access to the multiple dictionnary access. But
don't forget, you change an algo of O(n), by O(nlogn) with a lower constant.
So the n should not be too big. Just try different value.
Frédéric Bas
The problem is the array is very large. We are talking about 200+ million rows.
On Thu, Jul 9, 2009 at 4:41 AM, David Warde-Farley wrote:
> On 9-Jul-09, at 1:12 AM, Mag Gam wrote:
>
>> Here is what I have, which does it 1x1:
>>
>> z={} #dictionary
>> r=csv.reader(file)
>> for i,row in enumerate(
On 9-Jul-09, at 1:12 AM, Mag Gam wrote:
> Here is what I have, which does it 1x1:
>
> z={} #dictionary
> r=csv.reader(file)
> for i,row in enumerate(r):
> p="/MIT/"+row[1]
>
> if p not in z:
>z[p]=0:
> else:
>z[p]+=1
>
> arr[p]['chem'][z[p]]=tuple(row) #this loads the array 1 x 1
>
>
Hey All
I am reading thru a file and trying to store the values into another
array, but instead of storing the values 1 by 1, I would like to store
them in bulk sets for optimization purposes.
Here is what I have, which does it 1x1:
z={} #dictionary
r=csv.reader(file)
for i,row in enumerate(r):
