Hello, I am trying to merge 20 segments, total size 13GB, using Nutch 1.0 segment merger on a single computer. I have 100GB free in temp partition. Still, Nutch runs out of free space on the device. This does not seem right.
Is there anything I can do to reduce the use of temp space? Perhaps some option in Hadoop configuration limiting the amount of "parallel" jobs generated? Or, can I get around this problem by merging the segments one by one? Thanks, Arkadi
