rishi-rana commented on code in PR #18159:
URL: https://github.com/apache/iceberg/pull/18159#discussion_r4077378312
##########
spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/RewriteOperationForRowLineage.scala:
##########
@@ -42,6 +42,7 @@ trait RewriteOperationForRowLineage extends
RewriteRowLevelCommand {
r.table match {
case sparkTable: SparkTable =>
TableUtil.supportsRowLineage(sparkTable.table())
+ case _ => false
Review Comment:
Added in b1981589a: `TestRowLineageWithForeignCatalog`, with a minimal v2
catalog whose tables are
not Iceberg tables, and a `DELETE` against one of them.
The test is coupled to this line — dropping the branch makes it fail with:
```
scala.MatchError: ...TestRowLineageWithForeignCatalog$ForeignTable
(of class ...TestRowLineageWithForeignCatalog$ForeignTable)
```
One thing worth flagging while writing it: a plain non-Iceberg table under
`spark_catalog`
(`CREATE TABLE ... USING parquet`) does *not* reach this branch. It resolves
to a v1 relation, so
the outer `case _ => false` already handles it and you get a normal
`UNSUPPORTED_FEATURE.TABLE_OPERATION`
error. It takes a genuine third-party v2 catalog registered alongside
Iceberg to get here, which is
why the test needs its own catalog rather than reusing an existing fixture.
Also note this branch fixes a latent issue rather than only one I
introduced: `shouldUpdatePlan` is
shared with `RewriteUpdateTableForRowLineage` and
`RewriteMergeIntoTableForRowLineage`, so `UPDATE`
and `MERGE` against a foreign v2 catalog could already hit the same
`MatchError` before this PR. The
new `DELETE` rule just made it easier to reach. Happy to split it into its
own PR if you would rather
keep this one to the plan validation fix.
Full `spark-extensions` 3.5 suite passes with the new test (0 failures), and
`spotlessCheck` /
`scalastyleMain` are clean.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
