Revanth14 opened a new issue, #2141:
URL: https://github.com/apache/iceberg-go/issues/2141
### Feature Request / Improvement
### Problem
Position-delete files that iceberg-go writes for v2 tables (merge-on-read
`Delete`) leave `referenced_data_file` unset and record `file_path` bounds
under the table's default metrics mode, `truncate(16)`. For a delete against a
single data file, the bounds come out as:
lower = "/var/folders/rl/"
upper = "/var/folders/rl0"
`referencedDataFilePath` cannot resolve a single target from these, so
`validateNoNewDeletesForRewrittenFiles` falls back to a `(specID, partition)`
match. As a result, a retried copy-on-write `Delete`/`Overwrite` (or a
`RewriteFiles`) is rejected with `ErrConflictingDeleteFiles` when a concurrent
merge-on-read delete targets a **different** data file in the same partition,
or any file at all if the table is unpartitioned.
The result is safe (over-rejection, never lost deletes), but it causes
unnecessary commit failures under concurrent writers.
### Reference behavior
Java writes position deletes with `MetricsConfig.forPositionDelete()`, which
keeps full metrics for `file_path`, so a delete file that targets one data file
has exact, equal bounds and the path-scoped match applies. iceberg-go already
does this when `referenced_data_file` is set (`table/data_file_meta.go`, which
forces `full` metrics on the `file_path` column), but the transaction's v2
writer (`positionDeleteRecordsToDataFiles` in `table/arrow_utils.go`) passes
table properties through unchanged.
### Proposed fix
- Force `full` metrics on `file_path` (and `pos`) in the v2 position-delete
writer, matching `MetricsConfig.forPositionDelete()`.
- Optionally, set `referenced_data_file` when a written delete file targets
exactly one data file.
Note: the unpartitioned v2 path bin-packs deletes across data files, so a
single delete file can still span several targets. In that case the bounds
legitimately differ and the partition fallback remains correct.
v3 is unaffected: deletion vectors always carry `referenced_data_file`.
### Context
Raised in review of #2099 by @zeroshade and @laskoviymishka.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
