ML, Upto what mapred.child.ulimit values have you tried submitting with? How large of a dict do you build in your program?
On Sun, Jul 29, 2012 at 1:54 PM, Mapred Learn <[email protected]> wrote: > + CDH users > > Sent from my iPhone > > On Jul 29, 2012, at 1:17 AM, Mapred Learn <[email protected]> wrote: > >> hi, >> One of my programs create a huge python dictionary and reducers fails with >> Memory Error everytime. >> >> Is there a way to specify reducer memory to be a bigger value for reducers >> to succeed ? >> >> I know we shuold not have this requirement in first place and not cerate >> this kind of dictionary, but still can I finish this job with giving more >> memory in jar command ? >> >> >> Thanks, >> JJ >> -- Harsh J
