zeroshade commented on issue #33: URL: https://github.com/apache/iceberg-cpp/issues/33#issuecomment-2598844931
I would also be in favor of `nanoarrow` over rolling your own data protocol. I'll also note that for AI/ML use cases, NVIDIA Rapids (libcudf) uses `nanoarrow` for importing and exporting data in Arrow format (leveraging the C Device ABIs). > However, I want to give some users the option to allow them to customize their own data protocol. As long as a user can produce a plan / tasks for a given query, they would be able to use their own customized data protocol for consuming and executing that plan, right? It seems like it would be kinda difficult to really allow full customization of data protocol while still enabling full handling of filtering out deletions, transforms, and so on within this library while still maintaining a sane API for the interfaces internally. At least IMHO -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
