JonasJ-ap commented on PR #6642: URL: https://github.com/apache/iceberg/pull/6642#issuecomment-1426338304
Souds Good! Thank you for your suggestions. After the last community sync, I re-investigate the whole process of this demo and realize that there is no proper way to guarantee the correct order of timeline replay. In the new proposal, the expected migration guarantee is that we always migrate the table at the state of lastest COMPACTION. In other words, we only include the most updated base file in each file group of the hudi table. In this way, users can choose to do compaction before the migration if they want the most updated table or do nothing if they do not want to include newly arrived data. I will start to investigate the proper way to implement this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
