lirui-apache commented on issue #13155: URL: https://github.com/apache/iceberg/issues/13155#issuecomment-2909799838
> > I think when a snapshot is being created, previous manifests without live entries will not be tracked. So maybe we have to apply the incremental snapshots one by one? And probably limit the max number of such snapshots, i.e. if there're too many, a full computation could be a better choice. > > We can apply one by one and throw exception if the snapshots removed (expired) in between? So, user does full compute? Thats what make sense to me. Yeah, given a target snapshot, we can walk through the ancestors and look for an existing stats file. If we can find one, it means all snapshots in between are still there, then we apply them one by one. A possible corner case is an ExpireSnapshot action comes in after we found the ancestors, but before we apply them. I suppose users can retry and fall back to full compute in that case? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
