hi, Hassan. First, YARN( the scheduler) doesn't provide any fault tolerance techniques, But applications(e.g., MapReduce or Spark) do.
For MapReduce, its fault tolerance is based on speculative execution, which simply re-launched failed tasks. For Spark, it does provide checkpoint API which users can leverage in their code by designating which RDDs should be checkpointed at what time. For more details about Spark checkpoint, you can refer to Spark docs. If checkpoint is not enabled, it fails back to speculative execution, the same as MapReduce. Wei On Sat, Jun 24, 2017 at 8:44 PM, Hassan Asghar <[email protected]> wrote: > Dear users, > > I am performing a comparative study on different fault tolerance > techniques, so, my question is that, how can we implement checkpointing > based and replication based fault tolerance in hadoop, is there any patch > already implement so that i can use that in my hadoop cluster?? > > > Regards, > Hassan Asghar > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
