Thank you for the clarification. i am talking about fault toldrance in map reduce, is there any algorithm implement in it?? For Checkpointing and Replication???
On Mon, 26 Jun 2017 at 3:32 AM, Jasson Chenwei <[email protected]> wrote: > hi, Hassan. > > First, YARN( the scheduler) doesn't provide any fault tolerance > techniques, But applications(e.g., MapReduce or Spark) do. > > For MapReduce, its fault tolerance is based on speculative execution, > which simply re-launched failed tasks. > For Spark, it does provide checkpoint API which users can leverage in > their code by designating which RDDs should be checkpointed at what time. > For more details about Spark checkpoint, you can refer to Spark docs. If > checkpoint is not enabled, it fails back to speculative execution, the same > as MapReduce. > > > > Wei > > On Sat, Jun 24, 2017 at 8:44 PM, Hassan Asghar <[email protected]> > wrote: > >> Dear users, >> >> I am performing a comparative study on different fault tolerance >> techniques, so, my question is that, how can we implement checkpointing >> based and replication based fault tolerance in hadoop, is there any patch >> already implement so that i can use that in my hadoop cluster?? >> >> >> Regards, >> Hassan Asghar >> > --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >> > -- Dear, Best Regards ------------------ Hassan Asghar *M*:*+923400400374* *E*: *[email protected] <[email protected]>*
