aokolnychyi commented on code in PR #9050:
URL: https://github.com/apache/iceberg/pull/9050#discussion_r1393529465
##########
spark/v3.4/spark/src/main/java/org/apache/iceberg/spark/source/SparkWrite.java:
##########
@@ -390,7 +390,7 @@ private Expression conflictDetectionFilter() {
filter = Expressions.and(filter, expr);
}
- return filter;
+ return SERIALIZABLE == isolationLevel ? Expressions.alwaysTrue() :
filter;
Review Comment:
The validation on the Iceberg side works correctly. The open question is
whether the optimizer rule is safe to apply under these conditions and whether
our Spark planning rule behaves correctly. I'll need to check with fresh eyes
tomorrow.
From what I see, the scope of this (potential) issue is very limited. We
must trigger constant folding in the ON condition and push that predicate to
the target table.
@nastra, do you think you will have some time to test whether other engines
perform this kind of replacement in ON conditions? I tested Postgres and it did
not do that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
