andygrove opened a new issue, #6143:
URL: https://github.com/apache/datafusion-comet/issues/6143
### Describe the bug
When an Iceberg write fails under the split-operator plan,
`IcebergCommitExec` aborts and rethrows the raw cause
(`IcebergCommitExec.scala:103` for a job failure, `:119` for a commit failure).
On Spark 3.x, Spark's `V2TableWriteExec.writeWithV2` wraps a non-fatal
failure, including a commit-time `ValidationException` or
`CommitFailedException`, in a `SparkException` ("Writing job aborted"). With
the split plan on, the same failure reaches user code with a different
exception type, so code that catches the Spark exception (or inspects its
cause) behaves differently depending on a Comet setting.
### Steps to reproduce
On Spark 3.5, make an Iceberg append fail at commit (for example a
concurrent conflicting write under serializable isolation) with
`spark.comet.write.iceberg.splitOperator.enabled` off and then on, and compare
the exception type thrown by the action.
### Expected behavior
The split plan throws the same exception type as Spark's own write path on
each Spark version: wrap on the versions where Spark wraps, rethrow where Spark
rethrows. A version shim is probably the simplest place for this.
### Additional context
Minor, but user visible once the split plan is on by default (#5644). Found
in an audit of the native Iceberg write path. Part of #5649.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
