Thanks, I'll check it out. On Tue, 27 Jun 2017 at 10:22 PM, Jasson Chenwei <[email protected]> wrote:
> hi, Hassan > > Actually, I didn't find any tryout for implementing > checkpoint&&replication based fault tolerance in the community yet. > I think the reason is the overhead is much larger than the gain, given the > fact that each map task only runs for 30s~40s. However, I have ever read > some academic papers that propose fault tolerance approach. > > Check out this: > http://ieeexplore.ieee.org/document/7161515/\ > > > > Wei > > > On Mon, Jun 26, 2017 at 9:16 PM, Hassan Asghar <[email protected]> > wrote: > >> Thank you for the clarification. i am talking about fault toldrance in >> map reduce, is there any algorithm implement in it?? For Checkpointing and >> Replication??? >> >> On Mon, 26 Jun 2017 at 3:32 AM, Jasson Chenwei <[email protected]> >> wrote: >> >>> hi, Hassan. >>> >>> First, YARN( the scheduler) doesn't provide any fault tolerance >>> techniques, But applications(e.g., MapReduce or Spark) do. >>> >>> For MapReduce, its fault tolerance is based on speculative execution, >>> which simply re-launched failed tasks. >>> For Spark, it does provide checkpoint API which users can leverage in >>> their code by designating which RDDs should be checkpointed at what time. >>> For more details about Spark checkpoint, you can refer to Spark docs. If >>> checkpoint is not enabled, it fails back to speculative execution, the same >>> as MapReduce. >>> >>> >>> >>> Wei >>> >>> On Sat, Jun 24, 2017 at 8:44 PM, Hassan Asghar <[email protected]> >>> wrote: >>> >>>> Dear users, >>>> >>>> I am performing a comparative study on different fault tolerance >>>> techniques, so, my question is that, how can we implement checkpointing >>>> based and replication based fault tolerance in hadoop, is there any patch >>>> already implement so that i can use that in my hadoop cluster?? >>>> >>>> >>>> Regards, >>>> Hassan Asghar >>>> >>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: [email protected] >>>> For additional commands, e-mail: [email protected] >>>> >>>> >>> -- >> Dear, >> >> Best Regards >> ------------------ >> Hassan Asghar >> *M*:*+923400400374 <+92%20340%200400374>* >> *E*: *[email protected] <[email protected]>* >> > > -- Dear, Best Regards ------------------ Hassan Asghar *M*:*+923400400374* *E*: *[email protected] <[email protected]>*
