Re: [I] Block writing to sorted tables [iceberg-python]

2024-11-05 Thread via GitHub
kevinjqliu commented on issue #1247: URL: https://github.com/apache/iceberg-python/issues/1247#issuecomment-2458708958 Thanks for the explanation! It looks like Sung was right and the table's sort order is not a constraint that must be enforced across the entire table and in all data files

Re: [I] Block writing to sorted tables [iceberg-python]

2024-11-05 Thread via GitHub
kevinjqliu closed issue #1247: Block writing to sorted tables URL: https://github.com/apache/iceberg-python/issues/1247 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

Re: [I] Block writing to sorted tables [iceberg-python]

2024-10-31 Thread via GitHub
Fokko commented on issue #1247: URL: https://github.com/apache/iceberg-python/issues/1247#issuecomment-2450831332 I don't think we should block the writes as that's pretty aggressive. What we do today when a table has a sort order, we write the data, but set the sort-order to none:

Re: [I] Block writing to sorted tables [iceberg-python]

2024-10-30 Thread via GitHub
kevinjqliu commented on issue #1247: URL: https://github.com/apache/iceberg-python/issues/1247#issuecomment-2448132613 According to #271, "We fail when we see a sort order". -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [I] Block writing to sorted tables [iceberg-python]

2024-10-30 Thread via GitHub
kevinjqliu commented on issue #1247: URL: https://github.com/apache/iceberg-python/issues/1247#issuecomment-2448078653 This might be a correctness issue. Maybe we can disable writing to a table with sort order for now, similar to what we do with equality deletes https://github.com/a

Re: [I] Block writing to sorted tables [iceberg-python]

2024-10-30 Thread via GitHub
kevinjqliu commented on issue #1247: URL: https://github.com/apache/iceberg-python/issues/1247#issuecomment-2448076726 > From the [docs](https://iceberg.apache.org/docs/latest/spark-ddl/#alter-table-write-ordered-by) I'm understanding that sort order is purely suggestive, and it is up to t

Re: [I] Block writing to sorted tables [iceberg-python]

2024-10-30 Thread via GitHub
sungwy commented on issue #1247: URL: https://github.com/apache/iceberg-python/issues/1247#issuecomment-2447112519 Hi @kevinjqliu thanks for raising this. From the [docs](https://iceberg.apache.org/docs/latest/spark-ddl/#alter-table-write-ordered-by) I'm understanding that sort order is pu

[I] Block writing to sorted tables [iceberg-python]

2024-10-25 Thread via GitHub
kevinjqliu opened a new issue, #1247: URL: https://github.com/apache/iceberg-python/issues/1247 ### Apache Iceberg version None ### Please describe the bug 🐞 Verify that we disallow writing an unsorted data frame to a table with sort order. -- This is an automated mes