Hi everyone, I'd like to start a discussion on streampark support submit spark 
task to yarn

Reference:
GitHub PR: https://github.com/apache/incubator-streampark/pull/3537

Currently completed functions:
1.streampark-console module adds spark related interfaces to support spark task 
submit, spark environment creation, and Release application building 
pipeline.These interfaces are new and will not affect previous Flink-related 
interfaces

2.Added streampark-spark-client module to support spark jar task submit

3.Modified the streampark-flink-packer module to support the Building pipeline 
for spark yarn application mode.The subsequent plan is to extract this module 
from flink

4.`t_spark_env` and `t_spark_app` are added to the database


Features under development:
1.StreamPark support spark sql task submit
2.spark task status tracking

Sample task submission











Reply via email to