wyb opened a new pull request #3716: URL: https://github.com/apache/incubator-doris/pull/3716
After user creates a spark load job which status is PENDING, Fe will schedule and submit the spark etl job. 1. Begin transaction 2. Create a SparkLoadPendingTask for submitting etl job 2.1 Create etl job configuration according to https://github.com/apache/incubator-doris/issues/3010#issuecomment-635174675 2.2 Upload the configuration file and job jar to HDFS with broker 2.3 Submit etl job to spark cluster 2.4 Wait for etl job submission result 3. Update job state to ETL and log job update info if etl job is submitted successfully #3433 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
