atifiu commented on issue #8045: URL: https://github.com/apache/iceberg/issues/8045#issuecomment-1760787847
@szehon-ho Thanks for the fix. I am facing the same issue on iceberg 1.3.0 while trying to remove delete files using proc `rewrite_position_delete_files` . Reason why I have remove delete files is the fact that aggregate pushdown is failing with this error message `SparkScanBuilder: Skipping aggregate pushdown: detected row level deletes`. https://github.com/apache/iceberg/pull/6252#issuecomment-1757848584 And I am still not sure how delete files were created when I have defined Merge on Read for dml operations. https://github.com/apache/iceberg/pull/6252#issuecomment-1758873680 So my questions to you is how can we remove delete files if we are still using 1.3.0 ? Is it somehow possible to manually remove reference of delete files without corrupting the metadata ? Thanks for your help. ``` 23/10/13 00:16:56 ERROR RewritePositionDeleteFilesSparkAction: Failure during rewrite group FileGroupInfo{globalIndex=1, partitionIndex=1, partition=org.apache.iceberg.util.StructProjection@3162902b} org.apache.spark.sql.AnalysisException: cannot resolve '(partition.`page_view_dtm_day` = 18384)' due to data type mismatch: differing types in '(partition.`page_view_dtm_day` = 18384)' (date and int).; 'Filter (partition#4925.page_view_dtm_day = 18384) +- RelationV2[content#4921, file_path#4922, file_format#4923, spec_id#4924, partition#4925, record_count#4926L, file_size_in_bytes#4927L, column_sizes#4928, value_counts#4929, null_value_counts#4930, nan_value_counts#4931, lower_bounds#4932, upper_bounds#4933, key_metadata#4934, split_offsets#4935, equality_ids#4936, sort_order_id#4937, readable_metrics#4938] ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
