Re: [I] Flink: Maintenance - RewriteManifestFiles [iceberg]

2025-07-09 Thread via GitHub
Guosmilesmile commented on issue #10305: URL: https://github.com/apache/iceberg/issues/10305#issuecomment-3055910794 I will start working on the development of this feature. Please feel free to ping me if there is any related information. -- This is an automated message from the Apache Gi

Re: [I] Flink: Maintenance - DeleteOrphanFiles [iceberg]

2025-07-09 Thread via GitHub
Guosmilesmile commented on issue #10306: URL: https://github.com/apache/iceberg/issues/10306#issuecomment-3055901714 There is a related PR in progress. https://github.com/apache/iceberg/pull/13302 -- This is an automated message from the Apache Git Service. To respond to the message, plea

Re: [PR] Scan Delete Support Part 6: Equality Delete Parsing [iceberg-rust]

2025-07-09 Thread via GitHub
sdd commented on code in PR #1017: URL: https://github.com/apache/iceberg-rust/pull/1017#discussion_r2196737201 ## crates/iceberg/src/arrow/caching_delete_file_loader.rs: ## @@ -308,28 +319,231 @@ impl CachingDeleteFileLoader { Ok(result) } -/// Parses record

Re: [I] Delegate `delete` to JUnit [iceberg]

2025-07-09 Thread via GitHub
ebyhr commented on issue #13506: URL: https://github.com/apache/iceberg/issues/13506#issuecomment-3055653279 @saumyapandey1998 We can change the logic to create a file under `temp` directory something like: ```java File testFile = File.createTempFile("junit", null, temp.toFile());

Re: [PR] Update the range in format-version (for v3/v4) [iceberg]

2025-07-09 Thread via GitHub
ajantha-bhat commented on code in PR #13505: URL: https://github.com/apache/iceberg/pull/13505#discussion_r2196625118 ## open-api/rest-catalog-open-api.py: ## @@ -1147,7 +1147,7 @@ class NotExpression(BaseModel): class TableMetadata(BaseModel): -format_version: int = Fi

Re: [PR] Update the range in format-version (for v3/v4) [iceberg]

2025-07-09 Thread via GitHub
ajantha-bhat commented on code in PR #13505: URL: https://github.com/apache/iceberg/pull/13505#discussion_r2196624700 ## open-api/rest-catalog-open-api.yaml: ## @@ -2511,7 +2511,7 @@ components: format-version: type: integer minimum: 1 - m

Re: [PR] Spark: Support Parquet dictionary encoded UUIDs [iceberg]

2025-07-09 Thread via GitHub
nastra commented on PR #13324: URL: https://github.com/apache/iceberg/pull/13324#issuecomment-3055598765 @Fokko are you planning to port this over to Spark 4? I think it would be good to get this out with 1.10 -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Core: Fix a cast that is too narrow [iceberg]

2025-07-09 Thread via GitHub
nastra commented on PR #12743: URL: https://github.com/apache/iceberg/pull/12743#issuecomment-3055593039 @angelo-DNAStack can you please fix formatting by running `./gradlew spotlessApply`? -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [I] Delegate `delete` to JUnit [iceberg]

2025-07-09 Thread via GitHub
saumyapandey1998 commented on issue #13506: URL: https://github.com/apache/iceberg/issues/13506#issuecomment-3055323499 hi @Fokko! I would like to work on this issue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Spark,Core: Refactor Delete OrphanFiles by moving common code from Spark to core [iceberg]

2025-07-09 Thread via GitHub
Guosmilesmile commented on PR #13429: URL: https://github.com/apache/iceberg/pull/13429#issuecomment-3055069748 Hey @RussellSpitzer @szehon-ho , if you have time, please take a moment to help review this. Thank you very much! GuoYu -- This is an automated message from the Apach

[PR] Add RoaringPositionBitmap [iceberg-python]

2025-07-09 Thread via GitHub
stevie9868 opened a new pull request, #2193: URL: https://github.com/apache/iceberg-python/pull/2193 # Rationale for this change This is the first change to support deletion vector write in pyiceberg, where we build an abstraction on top of the Bitmap for various opera

Re: [PR] [#13278] : Upgrade junit 5 to 5.13.x [iceberg]

2025-07-09 Thread via GitHub
manuzhang commented on PR #13280: URL: https://github.com/apache/iceberg/pull/13280#issuecomment-3054791352 @raveendra11 #13490 has been merged. Could you please rebase and try upgrading junit again? -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [PR] Build: Bump nessie from 0.104.1 to 0.104.2 [iceberg]

2025-07-09 Thread via GitHub
dependabot[bot] commented on PR #13314: URL: https://github.com/apache/iceberg/pull/13314#issuecomment-3054786593 OK, I won't notify you again about this release, but will get in touch when a new version is available. You can also ignore all major, minor, or patch releases for a dependency

Re: [PR] Build: Bump nessie from 0.104.1 to 0.104.2 [iceberg]

2025-07-09 Thread via GitHub
manuzhang closed pull request #13314: Build: Bump nessie from 0.104.1 to 0.104.2 URL: https://github.com/apache/iceberg/pull/13314 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] Build: Bump nessie from 0.104.1 to 0.104.2 [iceberg]

2025-07-09 Thread via GitHub
manuzhang commented on PR #13314: URL: https://github.com/apache/iceberg/pull/13314#issuecomment-3054786011 Superseded by #13490 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [I] [suggestion] Write path optimization [iceberg-python]

2025-07-09 Thread via GitHub
github-actions[bot] commented on issue #926: URL: https://github.com/apache/iceberg-python/issues/926#issuecomment-3054582883 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity oc

Re: [I] Add REST catalog integration tests [iceberg-python]

2025-07-09 Thread via GitHub
github-actions[bot] commented on issue #1439: URL: https://github.com/apache/iceberg-python/issues/1439#issuecomment-3054582828 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity

Re: [PR] fix: coerce UUID to String in readable_metrics to avoid ClassCastException in Spark [iceberg]

2025-07-09 Thread via GitHub
github-actions[bot] commented on PR #13087: URL: https://github.com/apache/iceberg/pull/13087#issuecomment-3054570780 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Core: Enhance logging messages for snapshot expiration [iceberg]

2025-07-09 Thread via GitHub
github-actions[bot] commented on PR #13279: URL: https://github.com/apache/iceberg/pull/13279#issuecomment-3054570969 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Spark 4.0: Display write metrics on SQL UI [iceberg]

2025-07-09 Thread via GitHub
github-actions[bot] commented on PR #13253: URL: https://github.com/apache/iceberg/pull/13253#issuecomment-3054570897 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] Handling of Source Id => Source Ids [iceberg]

2025-07-09 Thread via GitHub
github-actions[bot] commented on issue #10762: URL: https://github.com/apache/iceberg/issues/10762#issuecomment-3054570410 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Spark: Support Parquet dictionary encoded UUIDs [iceberg]

2025-07-09 Thread via GitHub
liamzwbao commented on code in PR #13324: URL: https://github.com/apache/iceberg/pull/13324#discussion_r2196215334 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/data/vectorized/parquet/TestParquetVectorizedReads.java: ## @@ -400,4 +400,19 @@ public void testUnsuppor

Re: [PR] Spark: Support Parquet dictionary encoded UUIDs [iceberg]

2025-07-09 Thread via GitHub
liamzwbao commented on code in PR #13324: URL: https://github.com/apache/iceberg/pull/13324#discussion_r2196215334 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/data/vectorized/parquet/TestParquetVectorizedReads.java: ## @@ -400,4 +400,19 @@ public void testUnsuppor

Re: [PR] Make metrics reporting asynchronous [iceberg]

2025-07-09 Thread via GitHub
danielcweeks commented on code in PR #13507: URL: https://github.com/apache/iceberg/pull/13507#discussion_r2196213424 ## core/src/main/java/org/apache/iceberg/rest/RESTMetricsReporter.java: ## @@ -51,15 +56,30 @@ public void report(MetricsReport report) { return; }

Re: [PR] Make metrics reporting asynchronous [iceberg]

2025-07-09 Thread via GitHub
danielcweeks commented on code in PR #13507: URL: https://github.com/apache/iceberg/pull/13507#discussion_r2196210942 ## core/src/main/java/org/apache/iceberg/rest/RESTMetricsReporter.java: ## @@ -36,12 +39,14 @@ class RESTMetricsReporter implements MetricsReporter { private

[PR] Make metrics reporting asynchronous [iceberg]

2025-07-09 Thread via GitHub
anoopj opened a new pull request, #13507: URL: https://github.com/apache/iceberg/pull/13507 Right now the implementation of RESTMetricsReporter is synchronous and the REST call can take a significant amount of time. Change the reporting to be async instead. -- This is an automated messag

[PR] Add RemovePartitionStatisticsUpdate and SetPartitionStatisticsUpdate [iceberg-python]

2025-07-09 Thread via GitHub
rambleraptor opened a new pull request, #2192: URL: https://github.com/apache/iceberg-python/pull/2192 Closes #2191 # Rationale for this change apache/iceberg supports these two event types. We should do the same to match the Java implementation and allow users to alte

Re: [PR] Spark: Support Parquet dictionary encoded UUIDs [iceberg]

2025-07-09 Thread via GitHub
Fokko commented on PR #13324: URL: https://github.com/apache/iceberg/pull/13324#issuecomment-3054290736 Thanks @RussellSpitzer, @kevinjqliu and @dingo4dev for the review 🚀 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [PR] Spark: Support Parquet dictionary encoded UUIDs [iceberg]

2025-07-09 Thread via GitHub
Fokko commented on code in PR #13324: URL: https://github.com/apache/iceberg/pull/13324#discussion_r2196110621 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/data/vectorized/parquet/TestParquetVectorizedReads.java: ## @@ -400,4 +400,19 @@ public void testUnsupportedR

Re: [I] Spark: Cannot read or write UUID columns [iceberg]

2025-07-09 Thread via GitHub
Fokko closed issue #4581: Spark: Cannot read or write UUID columns URL: https://github.com/apache/iceberg/issues/4581 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

Re: [PR] Spark: Support Parquet dictionary encoded UUIDs [iceberg]

2025-07-09 Thread via GitHub
Fokko merged PR #13324: URL: https://github.com/apache/iceberg/pull/13324 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

[I] Delegate `delete` to JUnit [iceberg]

2025-07-09 Thread via GitHub
Fokko opened a new issue, #13506: URL: https://github.com/apache/iceberg/issues/13506 ### Feature Request / Improvement For the `TestParquetVectorizedReads` we delete the temp directory by hand: https://github.com/apache/iceberg/blob/c71e3eb4d0b4674758d1ed3c196fac9aed3c9ac2/spa

Re: [PR] Spark: Support Parquet dictionary encoded UUIDs [iceberg]

2025-07-09 Thread via GitHub
Fokko commented on code in PR #13324: URL: https://github.com/apache/iceberg/pull/13324#discussion_r2196106687 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/data/vectorized/parquet/TestParquetVectorizedReads.java: ## @@ -400,4 +400,19 @@ public void testUnsupportedR

[I] Support SetPartitionStatistics and RemoveParitionStatistics [iceberg-python]

2025-07-09 Thread via GitHub
rambleraptor opened a new issue, #2191: URL: https://github.com/apache/iceberg-python/issues/2191 ### Feature Request / Improvement apache/iceberg supports the table update events `SetPartitionStatistics` and `RemovePartitionStatistics`. We should support those as well to match the Ja

Re: [PR] Build: Bump junit from 5.12.2 to 5.13.3 [iceberg]

2025-07-09 Thread via GitHub
Fokko commented on PR #13469: URL: https://github.com/apache/iceberg/pull/13469#issuecomment-3054159155 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] Spark 4.0: Row Lineage support [iceberg]

2025-07-09 Thread via GitHub
stevenzwu commented on code in PR #13310: URL: https://github.com/apache/iceberg/pull/13310#discussion_r2196013023 ## spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/SparkPositionDeltaWrite.java: ## @@ -736,7 +769,14 @@ public void update(InternalRow meta, Interna

Re: [PR] Use short string in Variant when possible [iceberg]

2025-07-09 Thread via GitHub
RussellSpitzer commented on PR #13284: URL: https://github.com/apache/iceberg/pull/13284#issuecomment-3054049439 @aihuaxu are you happy with the new test? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] API, Flink: fix typos in javadoc [iceberg]

2025-07-09 Thread via GitHub
stevenzwu commented on PR #13503: URL: https://github.com/apache/iceberg/pull/13503#issuecomment-3053902183 thanks @laurentgo for the fix -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] feat(transaction): Add initial support for update spec in transaction API [iceberg-go]

2025-07-09 Thread via GitHub
lliangyu-lin commented on code in PR #467: URL: https://github.com/apache/iceberg-go/pull/467#discussion_r2195907682 ## table/update_spec.go: ## @@ -0,0 +1,367 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the

Re: [PR] API, Flink: fix typos in javadoc [iceberg]

2025-07-09 Thread via GitHub
stevenzwu merged PR #13503: URL: https://github.com/apache/iceberg/pull/13503 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [I] Can't partition by nested field [iceberg-python]

2025-07-09 Thread via GitHub
geruh commented on issue #2095: URL: https://github.com/apache/iceberg-python/issues/2095#issuecomment-3053775392 Hey @nvartolomei, Thanks for raising this issue. Currently, we don’t support partitioning on non-primitive type columns, so the referenced struct type should trigger a v

Re: [PR] feat(transaction): Add initial support for update spec in transaction API [iceberg-go]

2025-07-09 Thread via GitHub
zeroshade commented on code in PR #467: URL: https://github.com/apache/iceberg-go/pull/467#discussion_r2195814663 ## table/update_spec.go: ## @@ -0,0 +1,367 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOT

Re: [PR] feat(transaction): Add initial support for update spec in transaction API [iceberg-go]

2025-07-09 Thread via GitHub
lliangyu-lin commented on code in PR #467: URL: https://github.com/apache/iceberg-go/pull/467#discussion_r2195803110 ## table/update_spec.go: ## @@ -0,0 +1,367 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the

Re: [PR] Spark: Use native table FileIO instead of Hadoop to save file list in RewriteTablePath [iceberg]

2025-07-09 Thread via GitHub
szehon-ho commented on PR #13459: URL: https://github.com/apache/iceberg/pull/13459#issuecomment-3053697095 thanks @NikitaMatskevich , also @kevinjqliu , @singhpk234 for thoughtful reviews! -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] Spark: Use native table FileIO instead of Hadoop to save file list in RewriteTablePath [iceberg]

2025-07-09 Thread via GitHub
szehon-ho commented on PR #13459: URL: https://github.com/apache/iceberg/pull/13459#issuecomment-3053694475 @kevinjqliu not as far as I know, i guess it will get bundled correctly in the appropriate spark-iceberg jar. @NikitaMatskevich thanks it looks good to me -- This is an autom

Re: [PR] Spark: Use native table FileIO instead of Hadoop to save file list in RewriteTablePath [iceberg]

2025-07-09 Thread via GitHub
szehon-ho merged PR #13459: URL: https://github.com/apache/iceberg/pull/13459 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

[PR] Update the range in format-version (for v3/v4) [iceberg]

2025-07-09 Thread via GitHub
jbonofre opened a new pull request, #13505: URL: https://github.com/apache/iceberg/pull/13505 Should not we update `format-version` in REST spec to deal with V3 and anticipate V4 ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[PR] Docs: Add Ryft to list of vendors and blog posts [iceberg]

2025-07-09 Thread via GitHub
yogevyuval opened a new pull request, #13504: URL: https://github.com/apache/iceberg/pull/13504 Added Ryft to the list of vendors, as well as links to the Iceberg blog posts. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [PR] Spark-3.5: Add procedure to compute partition stats [iceberg]

2025-07-09 Thread via GitHub
szehon-ho commented on code in PR #13480: URL: https://github.com/apache/iceberg/pull/13480#discussion_r2195750881 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/procedures/ComputePartitionStatsProcedure.java: ## @@ -0,0 +1,130 @@ +/* + * Licensed to the Apache Softw

Re: [PR] update daft links [iceberg-python]

2025-07-09 Thread via GitHub
ccmao1130 commented on PR #2169: URL: https://github.com/apache/iceberg-python/pull/2169#issuecomment-3053649259 Thanks Kevin! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] Core: Fix a cast that is too narrow [iceberg]

2025-07-09 Thread via GitHub
angelo-DNAStack commented on PR #12743: URL: https://github.com/apache/iceberg/pull/12743#issuecomment-3053644747 > I've left a few comments that would be good to address around testing, but the fix LGTM. @angelo-DNAStack could you please address those so that we could get this into Iceberg

Re: [PR] Core: Fix a cast that is too narrow [iceberg]

2025-07-09 Thread via GitHub
angelo-DNAStack commented on code in PR #12743: URL: https://github.com/apache/iceberg/pull/12743#discussion_r2195742075 ## core/src/test/java/org/apache/iceberg/TestClientPoolImpl.java: ## @@ -84,6 +141,18 @@ static class RetryableException extends RuntimeException {} stat

Re: [PR] Core: Fix a cast that is too narrow [iceberg]

2025-07-09 Thread via GitHub
angelo-DNAStack commented on code in PR #12743: URL: https://github.com/apache/iceberg/pull/12743#discussion_r2195741397 ## core/src/test/java/org/apache/iceberg/TestClientPoolImpl.java: ## @@ -44,6 +46,29 @@ public void testRetrySucceedsWithinMaxAttempts() throws Exception {

Re: [PR] feat(transaction): Add initial support for update spec in transaction API [iceberg-go]

2025-07-09 Thread via GitHub
lliangyu-lin commented on code in PR #467: URL: https://github.com/apache/iceberg-go/pull/467#discussion_r2195720556 ## partitions.go: ## @@ -28,7 +28,7 @@ import ( ) const ( - partitionDataIDStart = 1000 + PartitionDataIDStart = 1000 InitialPartition

Re: [PR] feat(transaction): Add initial support for update spec in transaction API [iceberg-go]

2025-07-09 Thread via GitHub
lliangyu-lin commented on code in PR #467: URL: https://github.com/apache/iceberg-go/pull/467#discussion_r2195720556 ## partitions.go: ## @@ -28,7 +28,7 @@ import ( ) const ( - partitionDataIDStart = 1000 + PartitionDataIDStart = 1000 InitialPartition

Re: [PR] feat(transaction): Add initial support for update spec in transaction API [iceberg-go]

2025-07-09 Thread via GitHub
lliangyu-lin commented on code in PR #467: URL: https://github.com/apache/iceberg-go/pull/467#discussion_r2195720556 ## partitions.go: ## @@ -28,7 +28,7 @@ import ( ) const ( - partitionDataIDStart = 1000 + PartitionDataIDStart = 1000 InitialPartition

Re: [PR] docs: remove mentiones of dynamodb [iceberg-go]

2025-07-09 Thread via GitHub
zeroshade merged PR #481: URL: https://github.com/apache/iceberg-go/pull/481 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

[PR] API, Flink: fix typos in javadoc [iceberg]

2025-07-09 Thread via GitHub
laurentgo opened a new pull request, #13503: URL: https://github.com/apache/iceberg/pull/13503 Remove superfluous `/**` string inside javadoc section -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] feat(table): Support Dynamic Partition Overwrite [iceberg-go]

2025-07-09 Thread via GitHub
dttung2905 commented on code in PR #482: URL: https://github.com/apache/iceberg-go/pull/482#discussion_r2195653721 ## table/transaction.go: ## @@ -343,6 +345,250 @@ func (t *Transaction) AddFiles(ctx context.Context, files []string, snapshotProp return t.apply(updates,

Re: [PR] refactor: Move memory catalog into core [iceberg-rust]

2025-07-09 Thread via GitHub
Xuanwo commented on PR #1480: URL: https://github.com/apache/iceberg-rust/pull/1480#issuecomment-3053494515 cc @liurenjie1024 please review again. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] [Parquet][Variant] The Parquet writer should set the `VARIANT` `logicalType` on the produced parquet files. [iceberg]

2025-07-09 Thread via GitHub
manirajv06 commented on issue #13498: URL: https://github.com/apache/iceberg/issues/13498#issuecomment-3053448582 I will work on this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] [docs] Tidy up left-hand navigation [iceberg]

2025-07-09 Thread via GitHub
stevenzwu commented on code in PR #13491: URL: https://github.com/apache/iceberg/pull/13491#discussion_r2195446926 ## docs/mkdocs.yml: ## @@ -22,69 +22,79 @@ plugins: nav: - index.md - - Tables: -- branching.md -- configuration.md -- evolution.md -- mainte

Re: [PR] [docs] Tidy up left-hand navigation [iceberg]

2025-07-09 Thread via GitHub
stevenzwu commented on code in PR #13491: URL: https://github.com/apache/iceberg/pull/13491#discussion_r2195446926 ## docs/mkdocs.yml: ## @@ -22,69 +22,79 @@ plugins: nav: - index.md - - Tables: -- branching.md -- configuration.md -- evolution.md -- mainte

Re: [PR] Proposal: IRC Events endpoint [iceberg]

2025-07-09 Thread via GitHub
c-thiel commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2195501429 ## open-api/rest-catalog-open-api.yaml: ## @@ -4245,8 +4247,6 @@ components: table-uuid: type: string format: uuid -metadata:

Re: [PR] Proposal: IRC Events endpoint [iceberg]

2025-07-09 Thread via GitHub
c-thiel commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2195491352 ## open-api/rest-catalog-open-api.py: ## @@ -1389,6 +1558,101 @@ class CommitTableResponse(BaseModel): metadata: TableMetadata +class EventsResponse(BaseMode

Re: [PR] Proposal: IRC Events endpoint [iceberg]

2025-07-09 Thread via GitHub
c-thiel commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2195489249 ## open-api/rest-catalog-open-api.yaml: ## @@ -3405,6 +3488,131 @@ components: allOf: - $ref: '#/components/schemas/ScanTasks' +Actor: + ty

Re: [PR] Proposal: IRC Events endpoint [iceberg]

2025-07-09 Thread via GitHub
c-thiel commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2195488964 ## open-api/rest-catalog-open-api.yaml: ## @@ -3041,6 +3096,34 @@ components: - $ref: '#/components/schemas/AddViewVersionUpdate' - $ref: '#/compone

Re: [PR] [docs] Tidy up left-hand navigation [iceberg]

2025-07-09 Thread via GitHub
stevenzwu commented on code in PR #13491: URL: https://github.com/apache/iceberg/pull/13491#discussion_r2195421247 ## docs/mkdocs.yml: ## @@ -22,69 +22,79 @@ plugins: nav: - index.md - - Tables: -- branching.md -- configuration.md -- evolution.md -- mainte

Re: [PR] [docs] Tidy up left-hand navigation [iceberg]

2025-07-09 Thread via GitHub
stevenzwu commented on code in PR #13491: URL: https://github.com/apache/iceberg/pull/13491#discussion_r2195446926 ## docs/mkdocs.yml: ## @@ -22,69 +22,79 @@ plugins: nav: - index.md - - Tables: -- branching.md -- configuration.md -- evolution.md -- mainte

Re: [PR] Use short string in Variant when possible [iceberg]

2025-07-09 Thread via GitHub
manirajv06 commented on PR #13284: URL: https://github.com/apache/iceberg/pull/13284#issuecomment-3053260545 > Still having issues with spotless, be sure to run gradlew spotlessApply Didn't fixed properly last time. Taken care now. -- This is an automated message from the Apache Git

Re: [PR] [docs] Tidy up left-hand navigation [iceberg]

2025-07-09 Thread via GitHub
stevenzwu commented on code in PR #13491: URL: https://github.com/apache/iceberg/pull/13491#discussion_r2195421247 ## docs/mkdocs.yml: ## @@ -22,69 +22,79 @@ plugins: nav: - index.md - - Tables: -- branching.md -- configuration.md -- evolution.md -- mainte

Re: [PR] Use short string in Variant when possible [iceberg]

2025-07-09 Thread via GitHub
RussellSpitzer commented on PR #13284: URL: https://github.com/apache/iceberg/pull/13284#issuecomment-3053212403 Still having issues with spotless, be sure to run gradlew spotlessApply -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] Build: Bump nessie to 0.104.2 skipping tests in JDK 11 [iceberg]

2025-07-09 Thread via GitHub
nastra merged PR #13490: URL: https://github.com/apache/iceberg/pull/13490 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [I] Docs: Merge latest doc with latest version doc [iceberg]

2025-07-09 Thread via GitHub
rmoff commented on issue #12917: URL: https://github.com/apache/iceberg/issues/12917#issuecomment-3053146897 What about ``` 1.9.1 > Other versions ``` ``` 1.9.1 V Other versions nightly 1.9.0 1.8.1 etc ``` -- This is an automated

Re: [PR] Spark 4.0: Port Avro lineage reader test changes from #13070 [iceberg]

2025-07-09 Thread via GitHub
stevenzwu merged PR #13496: URL: https://github.com/apache/iceberg/pull/13496 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Enforce that test classes start with "Test" instead of suffix [iceberg]

2025-07-09 Thread via GitHub
CuteChuanChuan commented on PR #13466: URL: https://github.com/apache/iceberg/pull/13466#issuecomment-3053087117 Hi @nastra , Thanks for your review! I've addressed all the feedback points: ✅ Added suppression rule to limit TestClassNamingConvention to test directories only

Re: [I] Docs: Merge latest doc with latest version doc [iceberg]

2025-07-09 Thread via GitHub
manuzhang commented on issue #12917: URL: https://github.com/apache/iceberg/issues/12917#issuecomment-3053075925 I think we'd better keep `nightly` for people to check documentation changes immediately. -- This is an automated message from the Apache Git Service. To respond to the message

Re: [I] Docs: Merge latest doc with latest version doc [iceberg]

2025-07-09 Thread via GitHub
rmoff commented on issue #12917: URL: https://github.com/apache/iceberg/issues/12917#issuecomment-3053050185 I agree. So how about this kind of thing. 1. Default to latest version (drop the `latest` terminology) https://github.com/user-attachments/assets/7e192be1-62d4-4350-a52b

Re: [PR] AWS: Prevent excessive creation of auth sessions in S3V4RestSignerClient [iceberg]

2025-07-09 Thread via GitHub
danielcweeks merged PR #13215: URL: https://github.com/apache/iceberg/pull/13215 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceb

Re: [PR] AWS: Prevent excessive creation of auth sessions in S3V4RestSignerClient [iceberg]

2025-07-09 Thread via GitHub
danielcweeks commented on code in PR #13215: URL: https://github.com/apache/iceberg/pull/13215#discussion_r2195272150 ## aws/src/main/java/org/apache/iceberg/aws/s3/signer/S3V4RestSignerClient.java: ## @@ -76,15 +80,25 @@ public abstract class S3V4RestSignerClient private s

Re: [PR] Enforce that test classes start with "Test" instead of suffix [iceberg]

2025-07-09 Thread via GitHub
CuteChuanChuan commented on code in PR #13466: URL: https://github.com/apache/iceberg/pull/13466#discussion_r2195225210 ## .baseline/checkstyle/checkstyle.xml: ## @@ -431,6 +431,11 @@ + Review Comment: Thanks for the feedback!

Re: [PR] Enforce that test classes start with "Test" instead of suffix [iceberg]

2025-07-09 Thread via GitHub
CuteChuanChuan commented on code in PR #13466: URL: https://github.com/apache/iceberg/pull/13466#discussion_r219585 ## core/src/test/java/org/apache/iceberg/data/TestData.java: ## @@ -53,7 +53,7 @@ import org.junit.jupiter.params.provider.FieldSource; import org.junit.jupi

Re: [I] Docs: Merge latest doc with latest version doc [iceberg]

2025-07-09 Thread via GitHub
manuzhang commented on issue #12917: URL: https://github.com/apache/iceberg/issues/12917#issuecomment-3052933253 Flink website does have such terminology as stable, LTS, snapshot in the `Documentation` pull-down. https://github.com/user-attachments/assets/d8cd7e9a-58a6-4aa9-807b-33a70

Re: [I] [docs] Default to the latest stable release [iceberg]

2025-07-09 Thread via GitHub
rmoff commented on issue #13492: URL: https://github.com/apache/iceberg/issues/13492#issuecomment-3052916403 see also #12917 As well as not defaulting to `nightly`; do we even _need_ `nightly` ? -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [I] Kafka Connect sink fails to write snapshot when using dynamic routing with SMTs [iceberg]

2025-07-09 Thread via GitHub
raphaelauv commented on issue #13457: URL: https://github.com/apache/iceberg/issues/13457#issuecomment-3052906943 I can reproduce my setup : ```json { "name": "X", "config": { "connector.class": "org.apache.iceberg.connect.IcebergSinkConnector", "

[PR] AWS: Add dot as supported character for DB in Glue [iceberg]

2025-07-09 Thread via GitHub
catarinaaa opened a new pull request, #13502: URL: https://github.com/apache/iceberg/pull/13502 ## Description On AWS Glue it's possible to create namespaces with a dot character however in iceberg this is not considered a valid namespace. Therefore this PR adds the . "dot" as a valid ch

Re: [I] Docs: Merge latest doc with latest version doc [iceberg]

2025-07-09 Thread via GitHub
rmoff commented on issue #12917: URL: https://github.com/apache/iceberg/issues/12917#issuecomment-3052840457 I agree that _both_ `latest` and `1.9.1` becomes confusing (is 1.9.1 not the latest?) and takes up more nav space. Before we fix it thought, a couple of thoughts: ## Nav

Re: [PR] Build: Bump nessie to 0.104.2 skipping tests in JDK 11 [iceberg]

2025-07-09 Thread via GitHub
nastra commented on code in PR #13490: URL: https://github.com/apache/iceberg/pull/13490#discussion_r2194950381 ## build.gradle: ## @@ -906,6 +906,7 @@ project(':iceberg-arrow') { } project(':iceberg-nessie') { + test.onlyIf { JavaVersion.current() != JavaVersion.VERSION_11

Re: [PR] Implement error handling mechanism for DataException [iceberg]

2025-07-09 Thread via GitHub
jaakla commented on PR #12973: URL: https://github.com/apache/iceberg/pull/12973#issuecomment-3052242586 Please keep it active, this feature is a must to use Iceberg Kafka Connector in any meaningful data scale -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Core/REST: generify AuthSessionCache [iceberg]

2025-07-09 Thread via GitHub
adutra commented on PR #12562: URL: https://github.com/apache/iceberg/pull/12562#issuecomment-3052239562 > If you have the object, you can produce and identifier string for the key. Not always. There is no "session ID" in my use case. You could obviously use the object's hash, but ho

Re: [PR] Build: Bump nessie to 0.104.2 skipping tests in JDK 11 [iceberg]

2025-07-09 Thread via GitHub
ajantha-bhat commented on code in PR #13490: URL: https://github.com/apache/iceberg/pull/13490#discussion_r2194722503 ## build.gradle: ## @@ -906,6 +906,7 @@ project(':iceberg-arrow') { } project(':iceberg-nessie') { + test.onlyIf { JavaVersion.current() != JavaVersion.VERS

[PR] Hardcode BSD sed if running on MacOS [iceberg]

2025-07-09 Thread via GitHub
rmoff opened a new pull request, #13501: URL: https://github.com/apache/iceberg/pull/13501 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-ma

[I] Delete/Update/Merge (Spark Scala)-API instead of SQL? [iceberg]

2025-07-09 Thread via GitHub
christianknoepfle opened a new issue, #13500: URL: https://github.com/apache/iceberg/issues/13500 ### Query engine Spark ### Question Hi, in Delta I have an API for Delete/Update/Merge in Spark. So I can write ``mytable.delete(col("foo") === lit(2))`` (usi

[I] [docs] Docs build doesn't work on Mac with GNU sed [iceberg]

2025-07-09 Thread via GitHub
rmoff opened a new issue, #13499: URL: https://github.com/apache/iceberg/issues/13499 ### Feature Request / Improvement The common.sh caters for the BSD version of `sed` that MacOS ships with here: https://github.com/apache/iceberg/blob/main/site/dev/common.sh#L178 Howeve

Re: [PR] refactor: Move memory catalog into core [iceberg-rust]

2025-07-09 Thread via GitHub
Xuanwo commented on code in PR #1480: URL: https://github.com/apache/iceberg-rust/pull/1480#discussion_r2194701328 ## crates/iceberg/src/catalog/mod.rs: ## @@ -17,6 +17,8 @@ //! Catalog API for Apache Iceberg +pub mod memory; Review Comment: Makes sense. -- This is

[I] [Parquet][Variant] The Parquet writer should set the `VARIANT` `logicalType` on the produced parquet files. [iceberg]

2025-07-09 Thread via GitHub
Tishj opened a new issue, #13498: URL: https://github.com/apache/iceberg/issues/13498 ### Apache Iceberg version 1.9.1 (latest release) ### Query engine None ### Please describe the bug 🐞 As defined by the [VariantEncoding](https://github.com/apache/parquet

Re: [PR] AWS: Prevent excessive creation of auth sessions in S3V4RestSignerClient [iceberg]

2025-07-09 Thread via GitHub
adutra commented on code in PR #13215: URL: https://github.com/apache/iceberg/pull/13215#discussion_r2194568622 ## aws/src/main/java/org/apache/iceberg/aws/s3/signer/S3V4RestSignerClient.java: ## @@ -76,15 +80,25 @@ public abstract class S3V4RestSignerClient private static

[PR] Make 'Drop table' behavior same in hive4 and spark3.5 when use hiveca… [iceberg]

2025-07-09 Thread via GitHub
hidataplus opened a new pull request, #13497: URL: https://github.com/apache/iceberg/pull/13497 1. add catalog property "delete-table-rootdir",default value: true. if want to make behavior as before, set this to fasle. 2. if delete-table-rootdir = true and external.table.purge = true, it

Re: [PR] feat(table): Support Dynamic Partition Overwrite [iceberg-go]

2025-07-09 Thread via GitHub
laskoviymishka commented on code in PR #482: URL: https://github.com/apache/iceberg-go/pull/482#discussion_r2194537734 ## table/transaction.go: ## @@ -343,6 +345,250 @@ func (t *Transaction) AddFiles(ctx context.Context, files []string, snapshotProp return t.apply(updat

Re: [PR] Flink: Dynamic Iceberg Sink: Add table update code for schema comparison and evolution [iceberg]

2025-07-09 Thread via GitHub
mxm commented on code in PR #13032: URL: https://github.com/apache/iceberg/pull/13032#discussion_r2194337608 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/TableUpdater.java: ## @@ -0,0 +1,206 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

  1   2   >