How to voluntarily fail the RSCDriver in Livy?

[sanchay javeria]

Hey Livy devs,

I am working on a use-case where I want to fail an interactive Livy
session's Yarn application if any statement in the session fails. My
question is how do I voluntarily fail the RSCDriver after stopping the
sparkContext?

I tried throwing an exception in RSCDrvier.java inside Netty's handler like
so:

public void handle(ChannelHandlerContext ctx, EndSession msg) throws Exception {
  if (livyConf.getBoolean(TEST_STUCK_END_SESSION)) {
    LOG.warn("Ignoring EndSession request because
TEST_STUCK_END_SESSION is set.");
  } else {
    LOG.debug("Shutting down due to EndSession request.");
    // shutdown();
    throw new Exception("This is a testing exception.");
  }
}

but the Yarn application just stays in the RUNNING state although I see
this exception in the Livy logs. Is there a way to fail the Yarn app with
this exception?


Thanks,
Sanchay