Re: [I] Why is the default value of write.metadata.delete-after-commit.enabled set to false? [iceberg]

2025-07-10 Thread via GitHub
lintingbin commented on issue #13330: URL: https://github.com/apache/iceberg/issues/13330#issuecomment-3060845343 > Setting `write.metadata.previous-versions-max` at the catalog level is unsupported for now. I can send a PR in a few days. We hope to provide a feature for configuring d

Re: [I] Why is the default value of write.metadata.delete-after-commit.enabled set to false? [iceberg]

2025-07-10 Thread via GitHub
ebyhr commented on issue #13330: URL: https://github.com/apache/iceberg/issues/13330#issuecomment-3060826127 Setting `write.metadata.previous-versions-max` at the catalog level is unsupported for now. I can send a PR in a few days. -- This is an automated message from the Apache Git Serv

Re: [PR] Build: Bump junit from 5.12.2 to 5.13.3 [iceberg]

2025-07-10 Thread via GitHub
manuzhang commented on PR #13469: URL: https://github.com/apache/iceberg/pull/13469#issuecomment-3060798962 @Fokko sorry, I didn't notice the PR has been rebased, and the base version is already 5.13.2. Please help to reopen the PR. -- This is an automated message from the Apache Git Serv

Re: [PR] Build: Bump junit from 5.12.2 to 5.13.3 [iceberg]

2025-07-10 Thread via GitHub
manuzhang commented on PR #13469: URL: https://github.com/apache/iceberg/pull/13469#issuecomment-3060791974 Superseded by https://github.com/apache/iceberg/pull/13280 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Build: Bump junit from 5.12.2 to 5.13.3 [iceberg]

2025-07-10 Thread via GitHub
manuzhang closed pull request #13469: Build: Bump junit from 5.12.2 to 5.13.3 URL: https://github.com/apache/iceberg/pull/13469 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Build: Bump junit from 5.12.2 to 5.13.3 [iceberg]

2025-07-10 Thread via GitHub
dependabot[bot] commented on PR #13469: URL: https://github.com/apache/iceberg/pull/13469#issuecomment-3060789426 OK, I won't notify you again about this release, but will get in touch when a new version is available. You can also ignore all major, minor, or patch releases for a dependency

Re: [PR] Build: Bump junit from 5.12.2 to 5.13.3 [iceberg]

2025-07-10 Thread via GitHub
manuzhang commented on PR #13469: URL: https://github.com/apache/iceberg/pull/13469#issuecomment-3060789333 Superseded by https://github.com/apache/iceberg/pull/13280 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [I] Partition stats task tracker [iceberg]

2025-07-10 Thread via GitHub
ajantha-bhat commented on issue #8450: URL: https://github.com/apache/iceberg/issues/8450#issuecomment-3060786325 Closing this task as completed today. V4 and future work can still be tracked in this even after closing it. -- This is an automated message from the Apache Git Service. To r

Re: [I] Partition stats task tracker [iceberg]

2025-07-10 Thread via GitHub
ajantha-bhat closed issue #8450: Partition stats task tracker URL: https://github.com/apache/iceberg/issues/8450 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [I] Why is the default value of write.metadata.delete-after-commit.enabled set to false? [iceberg]

2025-07-10 Thread via GitHub
manuzhang commented on issue #13330: URL: https://github.com/apache/iceberg/issues/13330#issuecomment-3060785866 @ebyhr do you know how to set catalog default config via Trino? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [PR] Build: Bump JUnit5 from 5.12.2 to 5.13.2 [iceberg]

2025-07-10 Thread via GitHub
nastra merged PR #13280: URL: https://github.com/apache/iceberg/pull/13280 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Core: Fix a cast that is too narrow [iceberg]

2025-07-10 Thread via GitHub
nastra merged PR #12743: URL: https://github.com/apache/iceberg/pull/12743 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [I] Retry logic in JDBC catalog fails with class cast exception if driver exception class does not extend SQLTransientException [iceberg]

2025-07-10 Thread via GitHub
nastra closed issue #11176: Retry logic in JDBC catalog fails with class cast exception if driver exception class does not extend SQLTransientException URL: https://github.com/apache/iceberg/issues/11176 -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [PR] Spark-3.5: Add procedure to compute partition stats [iceberg]

2025-07-10 Thread via GitHub
nastra merged PR #13480: URL: https://github.com/apache/iceberg/pull/13480 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [I] Spark procedure to compute partition stats. [iceberg]

2025-07-10 Thread via GitHub
nastra closed issue #10106: Spark procedure to compute partition stats. URL: https://github.com/apache/iceberg/issues/10106 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

Re: [PR] Make metrics reporting asynchronous [iceberg]

2025-07-10 Thread via GitHub
nastra commented on code in PR #13507: URL: https://github.com/apache/iceberg/pull/13507#discussion_r2199700864 ## core/src/main/java/org/apache/iceberg/rest/RESTMetricsReporter.java: ## @@ -51,15 +57,20 @@ public void report(MetricsReport report) { return; } -

Re: [PR] Spark-3.5: Add procedure to compute partition stats [iceberg]

2025-07-10 Thread via GitHub
ajantha-bhat commented on PR #13480: URL: https://github.com/apache/iceberg/pull/13480#issuecomment-3060658989 PR for spark-4.0: https://github.com/apache/iceberg/pull/13523 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[PR] Spark-4.0: Add procedure to compute partition stats [iceberg]

2025-07-10 Thread via GitHub
ajantha-bhat opened a new pull request, #13523: URL: https://github.com/apache/iceberg/pull/13523 Just a clean port from spark 3.5: https://github.com/apache/iceberg/pull/13480 And adopted the changes for spark-4.0 as per https://github.com/apache/iceberg/pull/13106. -- This is an au

Re: [PR] Docs: Meetup Guidelines [iceberg]

2025-07-10 Thread via GitHub
ebyhr commented on code in PR #13520: URL: https://github.com/apache/iceberg/pull/13520#discussion_r2199471668 ## site/docs/community.md: ## @@ -38,37 +62,31 @@ Issues are tracked in GitHub: [open-issues]: https://github.com/apache/iceberg/issues [new-issue]: https://github.co

Re: [I] optimize `_combine_positional_deletes` [iceberg-python]

2025-07-10 Thread via GitHub
geruh commented on issue #1271: URL: https://github.com/apache/iceberg-python/issues/1271#issuecomment-3060167245 They added support for the arange function in pyarrow in https://github.com/apache/arrow/pull/46778/files. I pulled the latest Arrow branch, built it locally, and integrated it

Re: [I] Remove `numpy` as a dependency [iceberg-python]

2025-07-10 Thread via GitHub
geruh commented on issue #1259: URL: https://github.com/apache/iceberg-python/issues/1259#issuecomment-3060166331 They added support for the arange function in pyarrow in https://github.com/apache/arrow/pull/46778/files. I pulled the latest Arrow branch, built it locally, and integrated it

[I] Merge snapshots into 1 under transaction of multiple operations [iceberg-python]

2025-07-10 Thread via GitHub
jim-ngoo opened a new issue, #2201: URL: https://github.com/apache/iceberg-python/issues/2201 ### Question For my use case, I have a daily cron job that batch process and append data but I only want a single snapshot record after the whole process. I tried to do them under a single t

Re: [PR] Docs: Add BladePipe to list of vendors and blog posts [iceberg]

2025-07-10 Thread via GitHub
ChocZoe commented on code in PR #13510: URL: https://github.com/apache/iceberg/pull/13510#discussion_r2199319004 ## site/docs/vendors.md: ## @@ -26,13 +26,19 @@ This page contains some of the vendors who are shipping and supporting Apache Ic AWS provides a [comprehensive sui

[I] Question on field id reassignment on table metadata creation [iceberg-rust]

2025-07-10 Thread via GitHub
dentiny opened a new issue, #1499: URL: https://github.com/apache/iceberg-rust/issues/1499 ### Is your feature request related to a problem or challenge? Hi community, I'm confused why do we need to reassign field id on table (metadata) creation? Imagine such a case, iceberg ta

Re: [PR] Spark-3.5: Add procedure to compute partition stats [iceberg]

2025-07-10 Thread via GitHub
ajantha-bhat commented on PR #13480: URL: https://github.com/apache/iceberg/pull/13480#issuecomment-3059996628 > btw, we usually start with spark 4.0. do you have pr for it? I know. We started this work on march: https://github.com/apache/iceberg/pull/12451 So, spark 4 was not ther

Re: [PR] Spark-3.5: Add procedure to compute partition stats [iceberg]

2025-07-10 Thread via GitHub
szehon-ho commented on PR #13480: URL: https://github.com/apache/iceberg/pull/13480#issuecomment-3059959926 btw, we usually start with spark 4.0. do you have pr for it? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

Re: [PR] Spark: Support Parquet dictionary encoded UUIDs [iceberg]

2025-07-10 Thread via GitHub
kevinjqliu commented on PR #13324: URL: https://github.com/apache/iceberg/pull/13324#issuecomment-3059891043 and perhaps spark 3.4 as well https://grep.app/search?f.repo=apache%2Ficeberg&q=UTF8String+ofRow%28FixedSizeBinaryVector -- This is an automated message from the Apache Git Ser

Re: [PR] Spark-3.5: Add procedure to compute partition stats [iceberg]

2025-07-10 Thread via GitHub
szehon-ho commented on code in PR #13480: URL: https://github.com/apache/iceberg/pull/13480#discussion_r2199089234 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/procedures/ComputePartitionStatsProcedure.java: ## @@ -0,0 +1,118 @@ +/* + * Licensed to the Apache Softw

Re: [I] [discuss] `Transaction` API's `autocommit` [iceberg-python]

2025-07-10 Thread via GitHub
github-actions[bot] commented on issue #1253: URL: https://github.com/apache/iceberg-python/issues/1253#issuecomment-3059680622 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity

Re: [I] [BUG] pyiceberg hanging on multiprocessing [iceberg-python]

2025-07-10 Thread via GitHub
github-actions[bot] commented on issue #1488: URL: https://github.com/apache/iceberg-python/issues/1488#issuecomment-3059680567 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity

Re: [PR] Spark 4.0: Provide method for returning Dataset of Orphan Files from Remove Orph… [iceberg]

2025-07-10 Thread via GitHub
github-actions[bot] commented on PR #13289: URL: https://github.com/apache/iceberg/pull/13289#issuecomment-3059675899 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] [PoC] Test Flink and Spark 4.0 using Hive 4 metastore [iceberg]

2025-07-10 Thread via GitHub
github-actions[bot] commented on PR #13262: URL: https://github.com/apache/iceberg/pull/13262#issuecomment-3059675832 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Encryption for REST catalog [iceberg]

2025-07-10 Thread via GitHub
github-actions[bot] commented on PR #13224: URL: https://github.com/apache/iceberg/pull/13224#issuecomment-3059675775 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] Encryption for REST catalog [iceberg]

2025-07-10 Thread via GitHub
github-actions[bot] closed pull request #13224: Encryption for REST catalog URL: https://github.com/apache/iceberg/pull/13224 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Build: Bump com.aliyun.oss:aliyun-sdk-oss from 3.10.2 to 3.18.2 [iceberg]

2025-07-10 Thread via GitHub
dependabot[bot] commented on PR #12968: URL: https://github.com/apache/iceberg/pull/12968#issuecomment-3059675692 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, l

Re: [PR] Enhanced License and Notice Report Generation [iceberg]

2025-07-10 Thread via GitHub
github-actions[bot] commented on PR #13220: URL: https://github.com/apache/iceberg/pull/13220#issuecomment-3059675694 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] Enhanced License and Notice Report Generation [iceberg]

2025-07-10 Thread via GitHub
github-actions[bot] closed pull request #13220: Enhanced License and Notice Report Generation URL: https://github.com/apache/iceberg/pull/13220 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [PR] Build: Bump com.aliyun.oss:aliyun-sdk-oss from 3.10.2 to 3.18.2 [iceberg]

2025-07-10 Thread via GitHub
github-actions[bot] closed pull request #12968: Build: Bump com.aliyun.oss:aliyun-sdk-oss from 3.10.2 to 3.18.2 URL: https://github.com/apache/iceberg/pull/12968 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] Build: Bump com.aliyun.oss:aliyun-sdk-oss from 3.10.2 to 3.18.2 [iceberg]

2025-07-10 Thread via GitHub
github-actions[bot] commented on PR #12968: URL: https://github.com/apache/iceberg/pull/12968#issuecomment-3059675552 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

[PR] chore: expose field id mapping from schema [iceberg-rust]

2025-07-10 Thread via GitHub
dentiny opened a new pull request, #1498: URL: https://github.com/apache/iceberg-rust/pull/1498 ## What changes are included in this PR? Motivation: - I would like to leverage naming mapping, to compensate the issue that table metadata reassigns field id on creation. - Name mapp

Re: [PR] Make metrics reporting asynchronous [iceberg]

2025-07-10 Thread via GitHub
anoopj commented on code in PR #13507: URL: https://github.com/apache/iceberg/pull/13507#discussion_r2199017447 ## core/src/main/java/org/apache/iceberg/SystemConfigs.java: ## @@ -53,6 +53,17 @@ private SystemConfigs() {} Math.max(2, 4 * Runtime.getRuntime().available

Re: [PR] Make metrics reporting asynchronous [iceberg]

2025-07-10 Thread via GitHub
amogh-jahagirdar commented on code in PR #13507: URL: https://github.com/apache/iceberg/pull/13507#discussion_r2198953282 ## core/src/main/java/org/apache/iceberg/SystemConfigs.java: ## @@ -53,6 +53,17 @@ private SystemConfigs() {} Math.max(2, 4 * Runtime.getRuntime()

Re: [PR] Azure: Add extra error logging in ADLSFileIO [iceberg]

2025-07-10 Thread via GitHub
kevinjqliu commented on PR #13517: URL: https://github.com/apache/iceberg/pull/13517#issuecomment-3059349927 cc @bryanck @Fokko who authored the initial adls pr in #8303 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Make metrics reporting asynchronous [iceberg]

2025-07-10 Thread via GitHub
anoopj commented on code in PR #13507: URL: https://github.com/apache/iceberg/pull/13507#discussion_r2198883672 ## core/src/main/java/org/apache/iceberg/rest/RESTMetricsReporter.java: ## @@ -51,15 +53,17 @@ public void report(MetricsReport report) { return; } -

Re: [PR] feat: Reassign field ids for schema [iceberg-rust]

2025-07-10 Thread via GitHub
dentiny commented on PR #615: URL: https://github.com/apache/iceberg-rust/pull/615#issuecomment-3059287463 > When new TableMetadata is created, field ids should be reassign to ensure that field numbering is normalized and started from 0. I'm not sure if I mis-read, but it seems field

Re: [PR] TableMetadataBuilder [iceberg-rust]

2025-07-10 Thread via GitHub
dentiny commented on code in PR #587: URL: https://github.com/apache/iceberg-rust/pull/587#discussion_r2198859868 ## crates/iceberg/src/spec/table_metadata_builder.rs: ## @@ -0,0 +1,2160 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor li

Re: [PR] Make metrics reporting asynchronous [iceberg]

2025-07-10 Thread via GitHub
amogh-jahagirdar commented on code in PR #13507: URL: https://github.com/apache/iceberg/pull/13507#discussion_r2198843241 ## core/src/main/java/org/apache/iceberg/rest/RESTMetricsReporter.java: ## @@ -51,15 +53,17 @@ public void report(MetricsReport report) { return;

Re: [PR] Spark 3.5: Add configuration to disable executor cache for delete files [iceberg]

2025-07-10 Thread via GitHub
anuragmantri commented on code in PR #12893: URL: https://github.com/apache/iceberg/pull/12893#discussion_r2198815727 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/BatchDataReader.java: ## @@ -50,15 +50,17 @@ class BatchDataReader extends BaseBatchReader B

Re: [PR] Proposal: IRC Events endpoint [iceberg]

2025-07-10 Thread via GitHub
rdblue commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2198696465 ## open-api/rest-catalog-open-api.yaml: ## @@ -3964,6 +4144,292 @@ components: metadata: $ref: '#/components/schemas/TableMetadata' +EventsRe

Re: [PR] Proposal: IRC Events endpoint [iceberg]

2025-07-10 Thread via GitHub
rdblue commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2198691557 ## open-api/rest-catalog-open-api.yaml: ## @@ -3964,6 +4144,292 @@ components: metadata: $ref: '#/components/schemas/TableMetadata' +EventsRe

Re: [PR] Proposal: IRC Events endpoint [iceberg]

2025-07-10 Thread via GitHub
rdblue commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2198687577 ## open-api/rest-catalog-open-api.yaml: ## @@ -3964,6 +4144,292 @@ components: metadata: $ref: '#/components/schemas/TableMetadata' +EventsRe

Re: [PR] Proposal: IRC Events endpoint [iceberg]

2025-07-10 Thread via GitHub
rdblue commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2198685486 ## open-api/rest-catalog-open-api.yaml: ## @@ -3964,6 +4144,292 @@ components: metadata: $ref: '#/components/schemas/TableMetadata' +EventsRe

Re: [PR] Proposal: IRC Events endpoint [iceberg]

2025-07-10 Thread via GitHub
rdblue commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2198683592 ## open-api/rest-catalog-open-api.yaml: ## @@ -3964,6 +4144,292 @@ components: metadata: $ref: '#/components/schemas/TableMetadata' +EventsRe

Re: [PR] Docs: metadata deletion doc fix [iceberg]

2025-07-10 Thread via GitHub
yguy-ryft commented on PR #13432: URL: https://github.com/apache/iceberg/pull/13432#issuecomment-3058919294 @gaborkaszab @kevinjqliu are you guys able to review this one? I think it could clarify some confusion around metadata "expiration" @gaborkaszab I saw you were involved in a Slack

Re: [PR] Proposal: IRC Events endpoint [iceberg]

2025-07-10 Thread via GitHub
rdblue commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2198679627 ## open-api/rest-catalog-open-api.yaml: ## @@ -3964,6 +4144,292 @@ components: metadata: $ref: '#/components/schemas/TableMetadata' +EventsRe

Re: [PR] Proposal: IRC Events endpoint [iceberg]

2025-07-10 Thread via GitHub
rdblue commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2198674690 ## open-api/rest-catalog-open-api.yaml: ## @@ -3964,6 +4144,292 @@ components: metadata: $ref: '#/components/schemas/TableMetadata' +EventsRe

Re: [PR] Proposal: IRC Events endpoint [iceberg]

2025-07-10 Thread via GitHub
rdblue commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2198667019 ## open-api/rest-catalog-open-api.yaml: ## @@ -3964,6 +4144,292 @@ components: metadata: $ref: '#/components/schemas/TableMetadata' +EventsRe

Re: [PR] Proposal: IRC Events endpoint [iceberg]

2025-07-10 Thread via GitHub
rdblue commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2198649389 ## open-api/rest-catalog-open-api.yaml: ## @@ -3405,6 +3490,101 @@ components: allOf: - $ref: '#/components/schemas/ScanTasks' +GetEventsRequest:

Re: [PR] Proposal: IRC Events endpoint [iceberg]

2025-07-10 Thread via GitHub
rdblue commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2198664432 ## open-api/rest-catalog-open-api.yaml: ## @@ -3964,6 +4144,292 @@ components: metadata: $ref: '#/components/schemas/TableMetadata' +EventsRe

Re: [PR] Proposal: IRC Events endpoint [iceberg]

2025-07-10 Thread via GitHub
rdblue commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2198657540 ## open-api/rest-catalog-open-api.yaml: ## @@ -3964,6 +4144,292 @@ components: metadata: $ref: '#/components/schemas/TableMetadata' +EventsRe

Re: [PR] Proposal: IRC Events endpoint [iceberg]

2025-07-10 Thread via GitHub
rdblue commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2198654847 ## open-api/rest-catalog-open-api.yaml: ## @@ -3964,6 +4144,292 @@ components: metadata: $ref: '#/components/schemas/TableMetadata' +EventsRe

Re: [PR] Proposal: IRC Events endpoint [iceberg]

2025-07-10 Thread via GitHub
rdblue commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2198649389 ## open-api/rest-catalog-open-api.yaml: ## @@ -3405,6 +3490,101 @@ components: allOf: - $ref: '#/components/schemas/ScanTasks' +GetEventsRequest:

Re: [PR] Proposal: IRC Events endpoint [iceberg]

2025-07-10 Thread via GitHub
rdblue commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2198645763 ## open-api/rest-catalog-open-api.yaml: ## @@ -3405,6 +3490,101 @@ components: allOf: - $ref: '#/components/schemas/ScanTasks' +GetEventsRequest:

Re: [PR] Proposal: IRC Events endpoint [iceberg]

2025-07-10 Thread via GitHub
rdblue commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2198637148 ## open-api/rest-catalog-open-api.yaml: ## @@ -3405,6 +3490,101 @@ components: allOf: - $ref: '#/components/schemas/ScanTasks' +GetEventsRequest:

Re: [PR] Proposal: IRC Events endpoint [iceberg]

2025-07-10 Thread via GitHub
rdblue commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2198632797 ## open-api/rest-catalog-open-api.yaml: ## @@ -3405,6 +3490,101 @@ components: allOf: - $ref: '#/components/schemas/ScanTasks' +GetEventsRequest:

Re: [PR] Proposal: IRC Events endpoint [iceberg]

2025-07-10 Thread via GitHub
rdblue commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2198626283 ## open-api/rest-catalog-open-api.yaml: ## @@ -3405,6 +3490,101 @@ components: allOf: - $ref: '#/components/schemas/ScanTasks' +GetEventsRequest:

Re: [PR] Proposal: IRC Events endpoint [iceberg]

2025-07-10 Thread via GitHub
rdblue commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2198610884 ## open-api/rest-catalog-open-api.yaml: ## @@ -3405,6 +3490,101 @@ components: allOf: - $ref: '#/components/schemas/ScanTasks' +GetEventsRequest:

Re: [I] [Spec] filed path update for Variant array metadata [iceberg]

2025-07-10 Thread via GitHub
rdblue commented on issue #13462: URL: https://github.com/apache/iceberg/issues/13462#issuecomment-3058845654 Thanks for the clarification! I'd prefer not going to option 3 because of the extra space and I'm not convinced that it is going to be useful. Plus you have a few edge cases like wh

Re: [PR] Docs: Add Ryft to list of vendors and blog posts [iceberg]

2025-07-10 Thread via GitHub
amogh-jahagirdar merged PR #13504: URL: https://github.com/apache/iceberg/pull/13504 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] Spark 3.5: Add configuration to disable executor cache for delete files [iceberg]

2025-07-10 Thread via GitHub
RussellSpitzer commented on code in PR #12893: URL: https://github.com/apache/iceberg/pull/12893#discussion_r2198588667 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/BatchDataReader.java: ## @@ -50,15 +50,17 @@ class BatchDataReader extends BaseBatchReader

Re: [PR] Docs: Add Ryft to list of vendors and blog posts [iceberg]

2025-07-10 Thread via GitHub
yogevyuval commented on PR #13504: URL: https://github.com/apache/iceberg/pull/13504#issuecomment-3058822454 @danielcweeks Thanks! Removed that one - appreciate if you can review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] Docs: Add Ryft to list of vendors and blog posts [iceberg]

2025-07-10 Thread via GitHub
yogevyuval commented on code in PR #13504: URL: https://github.com/apache/iceberg/pull/13504#discussion_r2198583538 ## site/docs/blogs.md: ## @@ -21,6 +21,15 @@ title: "Blogs" ## Iceberg Blogs Here is a list of company blogs that talk about Iceberg. The blogs are ordered fr

Re: [PR] Docs: Add Ryft to list of vendors and blog posts [iceberg]

2025-07-10 Thread via GitHub
danielcweeks commented on code in PR #13504: URL: https://github.com/apache/iceberg/pull/13504#discussion_r2198579278 ## site/docs/blogs.md: ## @@ -21,6 +21,15 @@ title: "Blogs" ## Iceberg Blogs Here is a list of company blogs that talk about Iceberg. The blogs are ordered

Re: [PR] Add RemovePartitionStatisticsUpdate and SetPartitionStatisticsUpdate [iceberg-python]

2025-07-10 Thread via GitHub
smaheshwar-pltr commented on code in PR #2192: URL: https://github.com/apache/iceberg-python/pull/2192#discussion_r2198547493 ## pyiceberg/table/statistics.py: ## @@ -52,3 +52,10 @@ def filter_statistics_by_snapshot_id( reject_snapshot_id: int, ) -> List[StatisticsFile]:

Re: [PR] Add RemovePartitionStatisticsUpdate and SetPartitionStatisticsUpdate [iceberg-python]

2025-07-10 Thread via GitHub
smaheshwar-pltr commented on code in PR #2192: URL: https://github.com/apache/iceberg-python/pull/2192#discussion_r2198547493 ## pyiceberg/table/statistics.py: ## @@ -52,3 +52,10 @@ def filter_statistics_by_snapshot_id( reject_snapshot_id: int, ) -> List[StatisticsFile]:

Re: [PR] Add RemovePartitionStatisticsUpdate and SetPartitionStatisticsUpdate [iceberg-python]

2025-07-10 Thread via GitHub
smaheshwar-pltr commented on code in PR #2192: URL: https://github.com/apache/iceberg-python/pull/2192#discussion_r2198543820 ## pyiceberg/table/statistics.py: ## @@ -52,3 +52,10 @@ def filter_statistics_by_snapshot_id( reject_snapshot_id: int, ) -> List[StatisticsFile]:

Re: [PR] refactor: consolidate snapshot expiration into MaintenanceTable [iceberg-python]

2025-07-10 Thread via GitHub
ForeverAngry commented on PR #2143: URL: https://github.com/apache/iceberg-python/pull/2143#issuecomment-3058700259 > This gonna be awesome, left a few comments. > > One general question I have is I dont us making use of the following table properties anywhere where I think we should

Re: [PR] Allow updating table scans with cached properties and non-argument members [iceberg-python]

2025-07-10 Thread via GitHub
smaheshwar-pltr commented on code in PR #2178: URL: https://github.com/apache/iceberg-python/pull/2178#discussion_r2198512874 ## pyiceberg/table/__init__.py: ## @@ -1691,7 +1691,12 @@ def to_polars(self) -> pl.DataFrame: ... def update(self: S, **overrides: Any) -> S:

Re: [PR] Add `create_view` to REST Catalog [iceberg-python]

2025-07-10 Thread via GitHub
rambleraptor commented on PR #2154: URL: https://github.com/apache/iceberg-python/pull/2154#issuecomment-3058669570 @sungwy @jayceslesar @Fokko can I get a review on this when you can? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] Allow updating table scans with cached properties and non-argument members [iceberg-python]

2025-07-10 Thread via GitHub
smaheshwar-pltr commented on PR #2178: URL: https://github.com/apache/iceberg-python/pull/2178#issuecomment-3058666929 After some thought, we can maybe narrow down to a few approaches: 1. Inspect constructor of subclass and match attribute names to parameter names; I've implemented this h

[PR] REST: Freshness aware table loading [iceberg]

2025-07-10 Thread via GitHub
hsingh574 opened a new pull request, #13522: URL: https://github.com/apache/iceberg/pull/13522 Fixes #13309 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[PR] Add RemoveSchemasUpdate event [iceberg-python]

2025-07-10 Thread via GitHub
rambleraptor opened a new pull request, #2200: URL: https://github.com/apache/iceberg-python/pull/2200 # Rationale for this change This is the last non-view related TableMetadata update event missing from Java. It allows users to remove schemas, as long as they exist a

Re: [I] Pyiceberg allows dropping the sort order column and causes table corruption on AWS Glue Catalog [iceberg-python]

2025-07-10 Thread via GitHub
geruh commented on issue #2166: URL: https://github.com/apache/iceberg-python/issues/2166#issuecomment-3058645521 Hi @mwa28, have you started working on this? It would be great to get it in before the next release. If not, I can take it up. -- This is an automated message from the Apache

Re: [PR] Remove cached properties before updating table scans [iceberg-python]

2025-07-10 Thread via GitHub
smaheshwar-pltr commented on PR #2178: URL: https://github.com/apache/iceberg-python/pull/2178#issuecomment-3058636169 After some thought, we can maybe narrow down to a few approaches: - Inspect constructor of subclass; I've implemented this here now ([9edb166](https://github.com/apache/i

Re: [PR] [docs] Add Confluent to vendors page [iceberg]

2025-07-10 Thread via GitHub
amogh-jahagirdar commented on code in PR #13512: URL: https://github.com/apache/iceberg/pull/13512#discussion_r2198487135 ## site/docs/vendors.md: ## @@ -61,6 +61,10 @@ the same copy of data using Spark and run analytics or AI with our [Machine Learning](https://www.cloudera.c

Re: [PR] Remove cached properties before updating table scans [iceberg-python]

2025-07-10 Thread via GitHub
smaheshwar-pltr commented on code in PR #2178: URL: https://github.com/apache/iceberg-python/pull/2178#discussion_r2198078628 ## pyiceberg/table/__init__.py: ## @@ -1689,9 +1689,13 @@ def to_pandas(self, **kwargs: Any) -> pd.DataFrame: ... @abstractmethod def to_polars

[PR] [Draft] Fix `TableScan` updating with `arguments` method [iceberg-python]

2025-07-10 Thread via GitHub
smaheshwar-pltr opened a new pull request, #2199: URL: https://github.com/apache/iceberg-python/pull/2199 # Rationale for this change # Are these changes tested? # Are there any user-facing changes? -- This is an automated message from the Apac

[PR] Add RemovePartitionSpecsUpdate event [iceberg-python]

2025-07-10 Thread via GitHub
rambleraptor opened a new pull request, #2198: URL: https://github.com/apache/iceberg-python/pull/2198 # Rationale for this change This adds RemovePartitionSpecsUpdate, another missing TableMetadata update event. It also ends up testing AddPartitionSpecUpdate, w

Re: [PR] Spark 4.0: Row Lineage support [iceberg]

2025-07-10 Thread via GitHub
stevenzwu commented on code in PR #13310: URL: https://github.com/apache/iceberg/pull/13310#discussion_r2198413757 ## spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/SparkPositionDeltaWrite.java: ## @@ -426,17 +428,35 @@ public DeltaWriter createWriter(int partit

Re: [PR] Spark 4.0: Row Lineage support [iceberg]

2025-07-10 Thread via GitHub
stevenzwu commented on code in PR #13310: URL: https://github.com/apache/iceberg/pull/13310#discussion_r2198413757 ## spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/SparkPositionDeltaWrite.java: ## @@ -426,17 +428,35 @@ public DeltaWriter createWriter(int partit

Re: [PR] refactor: consolidate snapshot expiration into MaintenanceTable [iceberg-python]

2025-07-10 Thread via GitHub
ForeverAngry commented on code in PR #2143: URL: https://github.com/apache/iceberg-python/pull/2143#discussion_r2198460499 ## pyiceberg/table/maintenance.py: ## @@ -0,0 +1,399 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreeme

Re: [PR] Build: Bump software.amazon.s3.analyticsaccelerator:analyticsaccelerator-s3 from 1.0.0 to 1.2.0 [iceberg]

2025-07-10 Thread via GitHub
fuatbasik commented on PR #13417: URL: https://github.com/apache/iceberg/pull/13417#issuecomment-3058555493 There is an Analytics Accelerator version 1.2.1 that fixes this build failure. We can close this pull requests and update to 1.2.1 directly. -- This is an automated message from th

Re: [PR] Add RemovePartitionStatisticsUpdate and SetPartitionStatisticsUpdate [iceberg-python]

2025-07-10 Thread via GitHub
rambleraptor commented on PR #2192: URL: https://github.com/apache/iceberg-python/pull/2192#issuecomment-3058503363 @kevinjqliu please merge whenever you're ready! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] Add RemovePartitionStatisticsUpdate and SetPartitionStatisticsUpdate [iceberg-python]

2025-07-10 Thread via GitHub
rambleraptor commented on code in PR #2192: URL: https://github.com/apache/iceberg-python/pull/2192#discussion_r2198416689 ## pyiceberg/table/statistics.py: ## @@ -52,3 +52,10 @@ def filter_statistics_by_snapshot_id( reject_snapshot_id: int, ) -> List[StatisticsFile]:

Re: [PR] Spark 4.0: Row Lineage support [iceberg]

2025-07-10 Thread via GitHub
stevenzwu commented on code in PR #13310: URL: https://github.com/apache/iceberg/pull/13310#discussion_r2198413757 ## spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/SparkPositionDeltaWrite.java: ## @@ -426,17 +428,35 @@ public DeltaWriter createWriter(int partit

Re: [PR] Spark 4.0: Row Lineage support [iceberg]

2025-07-10 Thread via GitHub
stevenzwu commented on code in PR #13310: URL: https://github.com/apache/iceberg/pull/13310#discussion_r2198398116 ## spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java: ## @@ -260,18 +260,52 @@ public MetadataColumn[] metadataColumns() { DataTyp

Re: [PR] Spark 4.0: Row Lineage support [iceberg]

2025-07-10 Thread via GitHub
amogh-jahagirdar commented on code in PR #13310: URL: https://github.com/apache/iceberg/pull/13310#discussion_r2198395943 ## spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/SparkPositionDeltaWrite.java: ## @@ -426,17 +428,35 @@ public DeltaWriter createWriter(int

Re: [PR] AWS: Add support to run all integration tests when S3 Analytics Accelerator is enabled [iceberg]

2025-07-10 Thread via GitHub
geruh commented on code in PR #13347: URL: https://github.com/apache/iceberg/pull/13347#discussion_r2198391319 ## aws/src/integration/java/org/apache/iceberg/aws/s3/TestS3InputStream.java: ## @@ -160,28 +187,36 @@ private void readAndCheckRanges( .isEqualTo(Arrays.copyO

Re: [PR] Spark 4.0: Row Lineage support [iceberg]

2025-07-10 Thread via GitHub
amogh-jahagirdar commented on code in PR #13310: URL: https://github.com/apache/iceberg/pull/13310#discussion_r2198388161 ## spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/SparkPositionDeltaWrite.java: ## @@ -426,17 +428,35 @@ public DeltaWriter createWriter(int

Re: [PR] Spark 4.0: Row Lineage support [iceberg]

2025-07-10 Thread via GitHub
amogh-jahagirdar commented on code in PR #13310: URL: https://github.com/apache/iceberg/pull/13310#discussion_r2198388161 ## spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/SparkPositionDeltaWrite.java: ## @@ -426,17 +428,35 @@ public DeltaWriter createWriter(int

  1   2   >