It seems indeed. We had good results with 20.203 and a large value for reuse configured. What changed?
Thanks > I wouldn't use jvm reuse at this point. It's had a number of issues over > time and I've consistently switched it off for a long while now. > > Arun > > On Dec 26, 2011, at 2:50 PM, Markus Jelsma wrote: > > Hi, > > > >> Markus, > >> > >> Good to know you fixed it now :) > >> > >> Also consider raising reduce slowstart to a more suitable value, like > >> 80-85% -- this is pretty beneficial in both heap consumption and more > >> performant Reduce slots. The property name is > >> "mapred.reduce.slowstart.completed.maps" and carries a float value > >> (percentage). > > > > Is that going to help if one has a non-zero value for reusing JVM > > instances? We have little concurrent jobs running because the process > > flow is linear and we prefer the benefit of copying map output while > > it's running. > > For this particular job, however, it is not the case since all mappers > > finish almost simultaneously so changing that value would not make a > > difference right? 80% finishes within a matter of seconds. > > > > For the record, it's an Apache Nutch fetcher job configured to terminate > > after 1 hour. This downloads URL's, hence the disabling of speculative > > execution.
