amogh-jahagirdar commented on code in PR #11561:
URL: https://github.com/apache/iceberg/pull/11561#discussion_r1873686457
##########
spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkBatchQueryScan.java:
##########
@@ -162,13 +162,14 @@ public void filter(Predicate[] predicates) {
}
}
- protected Map<String, DeleteFileSet> rewritableDeletes() {
+ protected Map<String, DeleteFileSet> rewritableDeletes(boolean forDVs) {
Map<String, DeleteFileSet> rewritableDeletes = Maps.newHashMap();
for (ScanTask task : tasks()) {
FileScanTask fileScanTask = task.asFileScanTask();
for (DeleteFile deleteFile : fileScanTask.deletes()) {
- if (ContentFileUtil.isFileScoped(deleteFile)) {
+ // Both file-scoped and partition-scoped position deletes must be
rewritten for DVs
+ if (forDVs || ContentFileUtil.isFileScoped(deleteFile)) {
Review Comment:
Ah I fixed to avoid broadcasting eq. deletes unnecessarily but need to
figure out how to add a test which would inspect the broadcast contents, will a
publish a separate PR for that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
