Hi everyone, I'd like to start a discussion on streampark support for pyflink
Reference: GitHub issue: https://github.com/apache/incubator-streampark/issues/2437 GitHub PR: https://github.com/apache/incubator-streampark/pull/2956 streampark pyflink currently supports yarn, k8s will be supported in the future 1. Before using pyflink, users need to compile venv.zip and upload it to hdfs and $WORKSPACE/python(whether this can be optimized, directly compile venv.zip and upload it to streampark directory or other solutions), please refer to the document: https://github.com/apache/incubator-streampark/pull/2956/files#diff-abc9ac8d84878215b2c86971e3320fbe0a7333a9a6dde9eb82b6 583393792b40 2.yarn application mode can be extended by adding PythonOptions parameters, such as' python.archives' and 'python.client.executable'. 3.yarn perjob mode can be extended by modifying getJobGraph to generate different Jobgraphs based on whether python or jar is uploaded
