wirybeaver commented on issue #2186:
URL: https://github.com/apache/iceberg-rust/issues/2186#issuecomment-3994808698
Hi @glitchy , thanks for the context!
To keep the momentum going for both merge_into and the upcoming DataFusion
integration, I’d suggest we decouple the basic RowDeltaAction (CoW) from the
more complex MoR implementation.
Since I have the CoW implementation ready and tested for my `merge_into` PoC
(upsert PoC will be exposed soon), merging the "bare" RowDeltaAction now would:
- Unblock downstream PRs for merge_into and update immediately.
- Reduce the scope of your MoR PR, making it easier and faster for
maintainers to review.
I’m happy to rebase on your `OverwriteAction` as soon as it lands to ensure
zero conflicts. This way, we aren't serializing the work—I can push the CoW
foundation through now, and you can layer the complex MoR logic on top of it
without having to worry about the boilerplate.
In terms of datafusion's support on `merge_into`, I already noticed this
unsupported problem, mentioned this in my epic note and plan to contribute a
merge_into on the datafusion side. We can first implement a merge_into
TableProvider hook like this PR do on the datafusion side:
https://github.com/apache/datafusion/pull/19142
Does that sound like a fair way to get both features landed faster?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
