The GitHub Actions job "Flink CDC CI" on 
flink-cdc.git/FLINK-40005-cli-executor-read-local-file has failed.
Run started by GitHub user Zhile (triggered by leonardBang).

Head commit for run:
0e10c043582b0944591490f37fc376dba9a1f79d / Jiyong Wang <[email protected]>
[FLINK-40005][cdc-cli] CliExecutor application mode resolves pipeline 
definition from local path, remote path or inline content

In application mode, CliExecutor.main received args[0] and passed it to
YamlPipelineDefinitionParser.parse(String, Configuration), which treats the
String as YAML content. The path string itself was parsed as YAML and failed
validation with: Missing required field "source" in top-level configuration.

The two native application deployment executors share CliExecutor.main but pass
args[0] with different semantics:
- K8SApplicationDeploymentExecutor sets APPLICATION_ARGS = 
commandLine.getArgList(),
  so args[0] is the pipeline definition FILE PATH (shipped into the JobManager
  container, e.g. mounted via a ConfigMap by the Flink Kubernetes Operator).
- YarnApplicationDeploymentExecutor reads the file on the client side and sets
  APPLICATION_ARGS to the pipeline definition CONTENT.

Resolve args[0] in three cases, in order:
1. An explicit-scheme path (s3://, hdfs://, oss://, file://) is read through
   Flink's FileSystem so the matching plugin resolves it. The scheme is 
explicit,
   so it is not at risk of being hijacked by the cluster default FileSystem.
2. A bare local file path is read with the local JVM file API (java.nio
   Files.readAllBytes) instead of Flink's FileSystem, whose cluster default may
   be S3/HDFS and would not resolve a local path shipped next to the JobManager.
3. Otherwise the value is already the pipeline definition content (YARN) and is
   parsed verbatim.

This fixes K8S application mode without regressing YARN application mode, and
also supports placing the pipeline definition file on remote storage.

Also update the Kubernetes deployment docs (EN + ZH): the FlinkDeployment 
example
now uses entryClass org.apache.flink.cdc.cli.CliExecutor without 
--use-mini-cluster,
and the "native application mode is not supported" note is removed.

Regression introduced by #3643 (FLINK-35360, Support Yarn application mode for
yaml job).

Report URL: https://github.com/apache/flink-cdc/actions/runs/28425046701

With regards,
GitHub Actions via GitBox

Reply via email to