kevinjqliu commented on PR #929: URL: https://github.com/apache/iceberg-python/pull/929#issuecomment-2239713152
@syun64 already pointed to the cost/benefits of upgrading. I lean more towards correctness than performance. What is the correctness issue if we do not upgrade? As I understand from the above, if the parquet file is of type `string`, we read it as `large_string` but write it as `string` again. As for updating the minimum dependency to pyarrow 17.0.0, I would prefer to wait for the new arrow version to be baked for a time before we require all new versions of Pyiceberg to use it. I also think the 0.7.0 release's feature set is getting massive. We can add this upgrade as a fast-follow release. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
