GitHub user liuyq2step created a discussion: Modified Flink Configuration Not Effective in YARN Application Mode Unless Cached Directory on HDFS is Cleared
When submitting a Flink job in yarn-application mode through StreamPark, if I modify the Flink configuration (e.g., flink-conf.yaml), the changes do not take effect in the newly submitted job unless I manually delete the corresponding Flink installation directory within the StreamPark workspace on HDFS. For example, after modifying the configuration, I need to delete: hdfs://uat-cluster/bg/streampark/workspace/flink/flink-1.20.2/ This indicates that StreamPark (or the underlying Flink on YARN mechanism) might be relying on a cached version of the Flink distribution on HDFS. It does not automatically upload the modified configuration files when resubmitting a job for the same version. This behavior forces manual cleanup and can be error-prone. Question: Is this the expected behavior? If so, could the documentation be updated to mention this requirement? If not, could an automatic mechanism be considered to check for configuration changes or provide an option to force re-upload? StreamPark Version: 2.1.6 Flink Version: 1.20.2 Execution Mode: yarn-application GitHub link: https://github.com/apache/streampark/discussions/4308 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
