javrasya commented on issue #9410: URL: https://github.com/apache/iceberg/issues/9410#issuecomment-1889832176
It still feels weird to allow that big of a split to be created. Wouldn't it possible to make the deleted files lazy and rather be loaded in the respective task node, instead of the coordinator node. It is network cost in the cluster and wouldn't it slow down the planning especially when the table is an upserted kind which is going to have so many EQUALITY_DELETES, or am I interpreting it wrongly 🤔 ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
