Re: [PR] Flink 1.17: Support specifying equality columns with write options [iceberg]

2024-01-15 Thread via GitHub
manuzhang closed pull request #8195: Flink 1.17: Support specifying equality columns with write options URL: https://github.com/apache/iceberg/pull/8195 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Flink 1.17: Support specifying equality columns with write options [iceberg]

2024-01-15 Thread via GitHub
manuzhang commented on PR #8195: URL: https://github.com/apache/iceberg/pull/8195#issuecomment-1892165875 Closing this since Spark table can support Flink SQL upsert operation with identifier fields. -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [PR] Flink 1.17: Support specifying equality columns with write options [iceberg]

2023-12-15 Thread via GitHub
pvary commented on PR #8195: URL: https://github.com/apache/iceberg/pull/8195#issuecomment-1858158123 > However, there's no primary key in Spark or Trino. I think it should be possible to provide the identity fields there too. The java API provides the possibility through the `S

Re: [PR] Flink 1.17: Support specifying equality columns with write options [iceberg]

2023-12-15 Thread via GitHub
manuzhang commented on PR #8195: URL: https://github.com/apache/iceberg/pull/8195#issuecomment-1858100586 However, there's no primary key in Spark or Trino. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] Flink 1.17: Support specifying equality columns with write options [iceberg]

2023-12-14 Thread via GitHub
pvary commented on PR #8195: URL: https://github.com/apache/iceberg/pull/8195#issuecomment-1855873133 @manuzhang: The main point of Iceberg is that we have an engine independent table format. The columns are either primary keys, or not. If they are primary keys, they should be primary keys

Re: [PR] Flink 1.17: Support specifying equality columns with write options [iceberg]

2023-12-14 Thread via GitHub
manuzhang commented on PR #8195: URL: https://github.com/apache/iceberg/pull/8195#issuecomment-1855777652 @pvary I've updated my description. Our use case is that the table is created in Spark while data are sinked from Flink -- This is an automated message from the Apache Git Service. To

Re: [PR] Flink 1.17: Support specifying equality columns with write options [iceberg]

2023-12-14 Thread via GitHub
pvary commented on PR #8195: URL: https://github.com/apache/iceberg/pull/8195#issuecomment-1855685224 > Currently, equality columns are parsed from primary key specified in Flink create table statement. However, an Iceberg table might have been created without primary key, but we still want

Re: [PR] Flink 1.17: Support specifying equality columns with write options [iceberg]

2023-12-14 Thread via GitHub
manuzhang commented on PR #8195: URL: https://github.com/apache/iceberg/pull/8195#issuecomment-1855640410 @stevenzwu @pvary please help review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the