JeonDaehong opened a new pull request, #3454: URL: https://github.com/apache/iceberg-rust/pull/3454
## Which issue does this PR close? - Related to #3400. ## What changes are included in this PR? `build_deletes_row_selection` walks the delete vector one position at a time, so a run of deleted rows costs one iteration per row. On files with many contiguous deletes this is a large part of the read. This builds the selection from deleted ranges instead (`DeleteVector::ranges`): - Short runs are still walked with `next`. A run switches to roaring's `next_range` after 4 consecutive positions, because `next_range` binary-searches the rest of an array container on each call and is slower than `next` when most runs are single rows. - Row groups that are not selected are skipped with `skip_to` instead of walking their deletes. - `DeleteVectorIterator` and its `advance_to` are no longer used and are removed. That is the function #3400 is about; the new skip logic is tested for both cases there (I left a note on the issue). Results are identical to `main` (checked in the benchmark below and in the tests). <details> <summary>Benchmark (c7i.2xlarge, release, 7 runs alternating old/new, median)</summary> Reading a 20M-row Parquet file (4 numeric columns, 20 row groups, local disk), selection build + read: | deletes | main | this PR | |---|---|---| | 1000 large runs, 50% | 190 ms | 125 ms | | runs of 100, 50% | 223 ms | 159 ms | | runs of 20, 50% | 279 ms | 224 ms | | 99% deleted | 260 ms | 135 ms | | 5% scattered single rows | 270 ms | 271 ms | | pairs, 10% | 270 ms | 269 ms | | 0.01% | 163 ms | 163 ms | The last three are within run-to-run noise. Selection build alone on 100M rows: 329 ms → 0.05 ms (large runs), 689 ms → 59 ms (99%), 172 ms → 174 ms (5% scattered, ranges overlap). The share of the build in a read is smaller with more or wider columns or remote storage. I also tried switching after 8, 16 and 32 positions; 4 was best for runs and showed no slowdown on scattered deletes. </details> I did the same kind of change for Java's DV iteration in apache/iceberg#18027. ## Are these changes tested? Yes. New tests in `delete_vector.rs` compare `ranges()` and `skip_to` against a reference model on mixed sets (single rows, pairs, runs around the switch point, runs across 2^16 and 2^32), plus the two cases from #3400. A new row selection test crosses 2^32 with and without row group selection. Existing positional delete tests pass unchanged. Breaking `skip_to` or dropping a buffered run in four different ways makes the new tests fail. `cargo test -p iceberg --lib`, clippy and fmt are clean. ## AI Disclosure I used Claude Code to write the change, the tests and the benchmark, and to run them. I reviewed it before opening the PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
