pvary commented on issue #14699: URL: https://github.com/apache/iceberg/issues/14699#issuecomment-3587257462
@Marwen94: You can't set the equality ids and the overwrite mode at the same sink - that is the issue you have seen. If the overwrite mode is set to true, then it means that every partition which is changed is overwritten with the new data - every old data file is removed, and the new data files are inserted. Still, the unmodified partitions are not dropped. If the equality ids are set, then the old data is retained, and only the rows are updated. So the records are kept and if the same row (based on the equality columns) is arrived then the old row is removed, and the new row is inserted. Reading this page could help: https://iceberg.apache.org/docs/nightly/flink-writes/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
