sungwy commented on PR #1569: URL: https://github.com/apache/iceberg-python/pull/1569#issuecomment-2621569571
Thank you for following up on the suggestion @lloyd-EA . I'm glad we included this integration test to find out the issue with Spark's timestamp conversion. I think we will have to investigate the corresponding schema evolution handling logic in Spark-Iceberg to understand if this is a bug, or an expected behavior with Spark. If it is not possible to fix this issue in Spark-Iceberg, I think it'll mean that we will not be able to support adding `ns` timestamp files until [V3 spec](https://iceberg.apache.org/spec/#version-3-extended-types-and-capabilities) is supported. Please do keep in mind that Iceberg spec is a language agnostic spec, and hence ruling out the possibility of using the same Iceberg table with other tools (like Trino or Spark) may impact you down the line when your application system grows. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
