88fantasy opened a new pull request, #4486:
URL: https://github.com/apache/streampark/pull/4486
## What changes were proposed in this pull request
Issue Number: close #4481
Fixes `BuildPipeline#execStep()` throwing `NullPointerException` whenever a
pipeline step's `Callable` legitimately returns `null` — which is the normal
case for steps that only perform a side effect (e.g. "create/clean the build
workspace") and have nothing meaningful to hand to the next step. This breaks
the build/release pipeline for a Flink SQL application (and, per the ~20
affected call sites, potentially every deploy mode) at its very first step.
## Brief change log
- `BuildPipeline#execStep()`: return `R` directly instead of `Optional<R>`,
and throw `pipelineException()` directly from the `catch` block on failure,
instead of returning `Optional.empty()` for the caller to detect via
`.orElseThrow(...)`. This removes the null ambiguity entirely — Java `Optional`
cannot distinguish "empty because the step failed" from "empty because it
succeeded with a `null` value," which is exactly what made a one-line
`Optional.of` → `Optional.ofNullable` swap insufficient (it would just make
`.orElseThrow()` mis-fire on every successful null-returning step instead of
crashing at `Optional.of`).
- Drop the now-redundant `.orElseThrow(() -> pipelineException())` /
`.orElseThrow(this::pipelineException)` at all ~20 call sites:
`BuildPipeline#runYarnSqlBuildSteps`, `FlinkRemoteBuildPipeline`,
`FlinkK8sSessionBuildPipeline`, `AbstractK8sApplicationBuildPipeline`,
`FlinkK8sApplicationBuildPipeline`, `SparkK8sApplicationBuildPipeline`.
No caller had extra logic in its `orElseThrow` lambda beyond `throw
pipelineException();` — every occurrence checked and removed identically.
## Verifying this change
Manually verified against a real Flink 2.2.1 standalone cluster:
- Before this change: `POST /flink/pipe/build` for a Flink SQL application
always fails at step 1/2 ("Create building workspace") with
`NullPointerException` at `Optional.of(null)`, even though the workspace
directory is genuinely created on disk. The failure is invisible via the REST
API (`hasError: false`, `errorSummary: null`) and only shows up in raw stdout
(`streampark.out`), not the console's own error log — because the exception is
thrown from a background thread pool task before `GlobalExceptionHandler` is
ever involved.
- After this change: `POST /flink/pipe/build` for the same application
completes with `pipeStatus: SUCCESS` for both steps, and produces a real 6.2 MB
shaded jar at `workspace/workspace/<appId>/streampark-flinkjob_<jobName>.jar`.
`./mvnw -pl
streampark-flink/streampark-flink-packer,streampark-console/streampark-console-service
-am clean compile checkstyle:check spotless:check` passes with 0 violations
across all touched modules.
## Does this pull request potentially affect one of the following parts
- Dependencies (does it add or upgrade a dependency): no
- Anything that affects deployment: no
- The persistence of application state: no
- The direction of network connections: no
- Anything that affects any api: no (internal `BuildPipeline` contract used
only within `streampark-flink-packer` and its console caller;
`execStep`/`runYarnSqlBuildSteps` are `protected`, not part of any
public/external API)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
