laskoviymishka commented on issue #437: URL: https://github.com/apache/iceberg-go/issues/437#issuecomment-2941581439
This is a significant issue, and unfortunately, there isn't a simple fix. The current behavior is problematic: we rely on context in two separate places. Cancelling the context used by the catalog effectively renders the table unusable. On the other hand, cancelling the context used by PlanFiles only gives the illusion of cleaning up resources - it doesn't actually stop any ongoing I/O operations. This is the crux of the problem. From a user perspective, if the context passed to `PlanFiles` is cancelled, all I/O operations should stop immediately. To achieve that, we need to ensure that I/O is bound to the closest context—in this case, the one passed to `PlanFiles`. We’re faced with two options for addressing the current behavior: 1. Decouple the catalog context from later I/O entirely, and shift the responsibility to the PlanFiles context. 2. Combine both contexts - one from the catalog and one for `PlanFiles` - and propagate them together. Whichever path we choose, one thing is clear: all I/O operations must respect the context passed to PlanFiles(ctx). To support that, we should stop passing pre-constructed I/O objects and instead construct them ad hoc within PlanFiles, using the provided context -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
