Hi All, I am trying to understand Livy internals for Interactive Job submission. For batch jobs, I can see that BatchSession.Scala instantiates a new instance of SparkProcessBuilder and runs spark-submit script with appropriate arguments.
For InteractiveSession, I see that no spark-submit or spark-shell is launched. Majorly, I have following questions with respect to this: In the code, SparkProcessBuilder seems to be only used for BatchSession and doesn't seem to be used for InteractiveSession. How does IntearctiveSession Spark Application get submitted to Yarn? Is it via Spark (like running spark-submit, spark-shell remotely) or Livy submits the request to Yarn RM directly? Could you kindly point to the code which does the Spark application submission for Interactive jobs? Appreciate your help. Thanks, Akshat
