Re: [PR] Flink: Dynamic Iceberg Sink: Add table update code for schema comparison and evolution [iceberg]

2025-05-20 Thread via GitHub
b-rick commented on code in PR #13032: URL: https://github.com/apache/iceberg/pull/13032#discussion_r2099407477 ## flink/v2.0/flink/src/test/java/org/apache/iceberg/flink/sink/dynamic/TestCompareSchemasVisitor.java: ## @@ -0,0 +1,209 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] Spark-3.5: Add spark action to compute partition stats [iceberg]

2025-05-20 Thread via GitHub
ajantha-bhat commented on code in PR #12450: URL: https://github.com/apache/iceberg/pull/12450#discussion_r2099442303 ## api/src/main/java/org/apache/iceberg/actions/ComputePartitionStats.java: ## @@ -0,0 +1,43 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under on

Re: [PR] Spark-3.5: Add spark action to compute partition stats [iceberg]

2025-05-20 Thread via GitHub
ajantha-bhat commented on code in PR #12450: URL: https://github.com/apache/iceberg/pull/12450#discussion_r2099439655 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/ComputePartitionStatsSparkAction.java: ## @@ -80,6 +88,16 @@ public Result execute() { }

Re: [PR] Flink: Dynamic Iceberg Sink: Add table update code for schema comparison and evolution [iceberg]

2025-05-20 Thread via GitHub
b-rick commented on code in PR #13032: URL: https://github.com/apache/iceberg/pull/13032#discussion_r2099407477 ## flink/v2.0/flink/src/test/java/org/apache/iceberg/flink/sink/dynamic/TestCompareSchemasVisitor.java: ## @@ -0,0 +1,209 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] feat: Add basic operations for `UpdateSchema` [iceberg-rust]

2025-05-20 Thread via GitHub
jonathanc-n commented on code in PR #1172: URL: https://github.com/apache/iceberg-rust/pull/1172#discussion_r2099265733 ## crates/iceberg/src/spec/schema/mod.rs: ## @@ -21,6 +21,7 @@ use std::collections::{HashMap, HashSet}; use std::fmt::{Display, Formatter}; use std::sync::A

Re: [PR] feat: Add basic operations for `UpdateSchema` [iceberg-rust]

2025-05-20 Thread via GitHub
CTTY commented on code in PR #1172: URL: https://github.com/apache/iceberg-rust/pull/1172#discussion_r2099236486 ## crates/iceberg/src/spec/schema/mod.rs: ## @@ -21,6 +21,7 @@ use std::collections::{HashMap, HashSet}; use std::fmt::{Display, Formatter}; use std::sync::Arc; +

Re: [I] [REST Catalog] OAuth 2 grant type "refresh_token" not implemented [iceberg]

2025-05-20 Thread via GitHub
nika-qubit commented on issue #12196: URL: https://github.com/apache/iceberg/issues/12196#issuecomment-2896409199 Thanks for the response @adutra When I tried using dremio/iceberg-auth-manager with spark and these conf: ``` ...rest.auth.oauth2.token-endpoint=https://oauth2.googl

Re: [I] Reorganize Spark Time Travel doc [iceberg]

2025-05-20 Thread via GitHub
Bhargavkonidena commented on issue #13064: URL: https://github.com/apache/iceberg/issues/13064#issuecomment-2896410203 Hi @manuzhang I am just done with PR for this issue. Please review it. -- This is an automated message from the Apache Git Service. To respond to the message, please l

[PR] Fix Issue #13064 [iceberg]

2025-05-20 Thread via GitHub
Bhargavkonidena opened a new pull request, #13113: URL: https://github.com/apache/iceberg/pull/13113 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

Re: [PR] fix: bump avro version to enable uuid type test case [iceberg-cpp]

2025-05-20 Thread via GitHub
wgtmac commented on PR #103: URL: https://github.com/apache/iceberg-cpp/pull/103#issuecomment-2896397857 @Xuanwo Thanks for the comment! Ideally it should be downloaded from a released tar ball with checksum verified. However, I have fixed many issues on the avro-cpp side so we have to use

Re: [PR] chore: bump vortex-jni to 0.34.0 [iceberg]

2025-05-20 Thread via GitHub
a10y closed pull request #13112: chore: bump vortex-jni to 0.34.0 URL: https://github.com/apache/iceberg/pull/13112 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

Re: [PR] feat: add avro input&output stream based on arrow stream impl [iceberg-cpp]

2025-05-20 Thread via GitHub
wgtmac commented on code in PR #105: URL: https://github.com/apache/iceberg-cpp/pull/105#discussion_r2099154550 ## src/iceberg/avro/avro_stream.cc: ## @@ -0,0 +1,148 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements.

Re: [PR] feat: Support metadata table "Entries" [iceberg-rust]

2025-05-20 Thread via GitHub
geruh commented on code in PR #863: URL: https://github.com/apache/iceberg-rust/pull/863#discussion_r2099144298 ## crates/iceberg/src/inspect/entries.rs: ## @@ -0,0 +1,603 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreement

Re: [I] Best Practices for Storing and Querying Full History and Latest Versions [iceberg]

2025-05-20 Thread via GitHub
github-actions[bot] commented on issue #11618: URL: https://github.com/apache/iceberg/issues/11618#issuecomment-2896113671 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] PyIceberg Cookbook [iceberg-python]

2025-05-20 Thread via GitHub
github-actions[bot] commented on issue #1201: URL: https://github.com/apache/iceberg-python/issues/1201#issuecomment-2896117270 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the A

Re: [I] PyIceberg Cookbook [iceberg-python]

2025-05-20 Thread via GitHub
github-actions[bot] closed issue #1201: PyIceberg Cookbook URL: https://github.com/apache/iceberg-python/issues/1201 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

Re: [I] Explicitly declare supported public APIs with '__all__' [iceberg-python]

2025-05-20 Thread via GitHub
github-actions[bot] commented on issue #1099: URL: https://github.com/apache/iceberg-python/issues/1099#issuecomment-2896117288 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity

[PR] feat(transaction): Add update statistics [iceberg-rust]

2025-05-20 Thread via GitHub
CTTY opened a new pull request, #1359: URL: https://github.com/apache/iceberg-rust/pull/1359 ## Which issue does this PR close? - Closes #1358 ## What changes are included in this PR? - Add new transaction api to update statistics ## Are these changes test

Re: [I] What's the use of old metadata file, why not delete by default? [iceberg]

2025-05-20 Thread via GitHub
github-actions[bot] closed issue #11206: What's the use of old metadata file, why not delete by default? URL: https://github.com/apache/iceberg/issues/11206 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] [Spark] Identity partition on required column generates nullable partition tuple in manifest file [iceberg]

2025-05-20 Thread via GitHub
github-actions[bot] commented on issue #11300: URL: https://github.com/apache/iceberg/issues/11300#issuecomment-2896113432 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] add_files with RestCatalog, S3FileIO [iceberg]

2025-05-20 Thread via GitHub
github-actions[bot] commented on issue #11558: URL: https://github.com/apache/iceberg/issues/11558#issuecomment-2896113596 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] kafka connect iceberg connect: option to fail connector on N number of failed commit cycles [iceberg]

2025-05-20 Thread via GitHub
github-actions[bot] closed issue #11468: kafka connect iceberg connect: option to fail connector on N number of failed commit cycles URL: https://github.com/apache/iceberg/issues/11468 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [I] How does client use hadoopcatlog to read the iceberg table writen by hivecatalog? [iceberg]

2025-05-20 Thread via GitHub
github-actions[bot] commented on issue #11375: URL: https://github.com/apache/iceberg/issues/11375#issuecomment-2896113455 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] When write.object-storage.enabled=true, it is difficult to gather information for individual partition of partitioned tables [iceberg]

2025-05-20 Thread via GitHub
github-actions[bot] commented on issue #11488: URL: https://github.com/apache/iceberg/issues/11488#issuecomment-2896113513 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] REST Catalog S3 Signer Endpoint should be Catalog specific [iceberg]

2025-05-20 Thread via GitHub
github-actions[bot] commented on issue #11608: URL: https://github.com/apache/iceberg/issues/11608#issuecomment-2896113633 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] Spark-hive catalog drop table XX purge not use purge code in HiveCatalog.dropTable() [iceberg]

2025-05-20 Thread via GitHub
github-actions[bot] commented on issue #11484: URL: https://github.com/apache/iceberg/issues/11484#issuecomment-2896113494 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] kafka connect iceberg connect: option to fail connector on N number of failed commit cycles [iceberg]

2025-05-20 Thread via GitHub
github-actions[bot] commented on issue #11468: URL: https://github.com/apache/iceberg/issues/11468#issuecomment-2896113469 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] Spark-hive catalog drop table XX purge not use purge code in HiveCatalog.dropTable() [iceberg]

2025-05-20 Thread via GitHub
github-actions[bot] closed issue #11484: Spark-hive catalog drop table XX purge not use purge code in HiveCatalog.dropTable() URL: https://github.com/apache/iceberg/issues/11484 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [I] How does client use hadoopcatlog to read the iceberg table writen by hivecatalog? [iceberg]

2025-05-20 Thread via GitHub
github-actions[bot] closed issue #11375: How does client use hadoopcatlog to read the iceberg table writen by hivecatalog? URL: https://github.com/apache/iceberg/issues/11375 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

Re: [I] What's the use of old metadata file, why not delete by default? [iceberg]

2025-05-20 Thread via GitHub
github-actions[bot] commented on issue #11206: URL: https://github.com/apache/iceberg/issues/11206#issuecomment-2896113414 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] [Spark] Identity partition on required column generates nullable partition tuple in manifest file [iceberg]

2025-05-20 Thread via GitHub
github-actions[bot] closed issue #11300: [Spark] Identity partition on required column generates nullable partition tuple in manifest file URL: https://github.com/apache/iceberg/issues/11300 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[I] Support update statistics in transaction [iceberg-rust]

2025-05-20 Thread via GitHub
CTTY opened a new issue, #1358: URL: https://github.com/apache/iceberg-rust/issues/1358 ### Is your feature request related to a problem or challenge? Support update statistics as a transaction API ### Describe the solution you'd like Add an API like below and implement

Re: [PR] Range distribution iceberg sink [iceberg]

2025-05-20 Thread via GitHub
rodmeneses commented on code in PR #12071: URL: https://github.com/apache/iceberg/pull/12071#discussion_r2099051067 ## flink/v1.20/flink/src/test/java/org/apache/iceberg/flink/sink/TestIcebergSink.java: ## @@ -145,57 +145,6 @@ void testWriteRowWithTableSchema() throws Exception

Re: [PR] Range distribution iceberg sink [iceberg]

2025-05-20 Thread via GitHub
rodmeneses commented on code in PR #12071: URL: https://github.com/apache/iceberg/pull/12071#discussion_r2098984273 ## flink/v1.20/flink/src/test/java/org/apache/iceberg/flink/sink/TestIcebergSink.java: ## @@ -140,6 +140,13 @@ void testWriteRowWithTableSchema() throws Exception

Re: [PR] feat: Introduce snapshot summary properties [iceberg-rust]

2025-05-20 Thread via GitHub
jonathanc-n commented on code in PR #1336: URL: https://github.com/apache/iceberg-rust/pull/1336#discussion_r2099043059 ## crates/iceberg/src/spec/table_metadata_builder.rs: ## @@ -1241,6 +1241,33 @@ impl TableMetadataBuilder { Ok(self) } + +/// Add summary p

[PR] Build: increase gradle jvm heap size from 1 GB to 1.5 GB. [iceberg]

2025-05-20 Thread via GitHub
stevenzwu opened a new pull request, #13110: URL: https://github.com/apache/iceberg/pull/13110 Twice, I have ran into OOMError with gradle build https://github.com/apache/iceberg/actions/runs/15146709248/job/42584083395 https://github.com/apache/iceberg/actions/runs/15146709248/job/

Re: [PR] feat: Introduce snapshot summary properties [iceberg-rust]

2025-05-20 Thread via GitHub
dentiny commented on code in PR #1336: URL: https://github.com/apache/iceberg-rust/pull/1336#discussion_r2099036931 ## crates/iceberg/src/spec/table_metadata_builder.rs: ## @@ -1241,6 +1241,33 @@ impl TableMetadataBuilder { Ok(self) } + +/// Add summary prope

Re: [PR] Flink: Migrate Flink TableSchema for IcebergSource [iceberg]

2025-05-20 Thread via GitHub
liamzwbao commented on code in PR #13072: URL: https://github.com/apache/iceberg/pull/13072#discussion_r2099013967 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/FlinkSchemaUtil.java: ## @@ -137,7 +136,9 @@ private static Schema freshIdentifierFieldIds(Schema iceber

Re: [PR] feat: Add basic operations for `UpdateSchema` [iceberg-rust]

2025-05-20 Thread via GitHub
CTTY commented on code in PR #1172: URL: https://github.com/apache/iceberg-rust/pull/1172#discussion_r2099007104 ## crates/iceberg/src/spec/schema/update.rs: ## @@ -0,0 +1,461 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agree

[I] How to create an apache iceberg table with geometry column? [iceberg]

2025-05-20 Thread via GitHub
likawa3b opened a new issue, #13109: URL: https://github.com/apache/iceberg/issues/13109 ### Query engine Spark and Sedona ### Question ``` from pyspark.sql import SparkSession from sedona.spark import SedonaContext from pyspark.sql.functions import expr s

Re: [PR] Flink: Migrate Flink TableSchema for IcebergSource [iceberg]

2025-05-20 Thread via GitHub
liamzwbao commented on PR #13072: URL: https://github.com/apache/iceberg/pull/13072#issuecomment-2896030358 > I remember previous attempts to remove TableSchema from Flink are failed. Could you please check if we were able to backport these changes to Flink 1.19/1.20? Hi @pvary, than

Re: [PR] Range distribution iceberg sink [iceberg]

2025-05-20 Thread via GitHub
rodmeneses commented on code in PR #12071: URL: https://github.com/apache/iceberg/pull/12071#discussion_r2098984273 ## flink/v1.20/flink/src/test/java/org/apache/iceberg/flink/sink/TestIcebergSink.java: ## @@ -140,6 +140,13 @@ void testWriteRowWithTableSchema() throws Exception

Re: [PR] feat: Support `PartitionStats` [iceberg-rust]

2025-05-20 Thread via GitHub
jonathanc-n commented on code in PR #: URL: https://github.com/apache/iceberg-rust/pull/#discussion_r2098993484 ## crates/iceberg/src/spec/statistic_file.rs: ## @@ -69,6 +72,137 @@ pub struct PartitionStatisticsFile { pub file_size_in_bytes: i64, } +#[derive(Clon

Re: [PR] Range distribution iceberg sink [iceberg]

2025-05-20 Thread via GitHub
rodmeneses commented on code in PR #12071: URL: https://github.com/apache/iceberg/pull/12071#discussion_r2098985225 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -692,20 +729,51 @@ private DataStream distributeDataStream(DataStream inp

Re: [I] [DISCUSS] A catalog loader api. [iceberg-rust]

2025-05-20 Thread via GitHub
gsoundar commented on issue #1228: URL: https://github.com/apache/iceberg-rust/issues/1228#issuecomment-2895995222 @liurenjie1024 For the `load_catalog` function, how do you plan to avoid cyclic dependency ? Catalog does not include packages of specific implementations. I can help wi

Re: [PR] Fix projected fields predicate evaluation [iceberg-python]

2025-05-20 Thread via GitHub
Erigara commented on code in PR #2029: URL: https://github.com/apache/iceberg-python/pull/2029#discussion_r2098978290 ## pyiceberg/expressions/visitors.py: ## @@ -894,12 +895,17 @@ def visit_unbound_predicate(self, predicate: UnboundPredicate[L]) -> BooleanExpr def visit

Re: [PR] CORE: [PARQUET] Log corrupted parquet filenames to trace bad nodes that may have written them. [iceberg]

2025-05-20 Thread via GitHub
ebyhr commented on code in PR #13108: URL: https://github.com/apache/iceberg/pull/13108#discussion_r2098952765 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetReader.java: ## @@ -120,18 +125,27 @@ public boolean hasNext() { @Override public T next() { -

Re: [I] feat: Support default values in `UpdateSchema` [iceberg-rust]

2025-05-20 Thread via GitHub
jonathanc-n commented on issue #1357: URL: https://github.com/apache/iceberg-rust/issues/1357#issuecomment-2895954073 This can be marked as a good first issue @Xuanwo @liurenjie1024 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

Re: [PR] feat: Introduce snapshot summary properties [iceberg-rust]

2025-05-20 Thread via GitHub
jonathanc-n commented on code in PR #1336: URL: https://github.com/apache/iceberg-rust/pull/1336#discussion_r2098955404 ## crates/iceberg/src/spec/table_metadata_builder.rs: ## @@ -1241,6 +1241,33 @@ impl TableMetadataBuilder { Ok(self) } + +/// Add summary p

Re: [PR] feat: Introduce snapshot summary properties [iceberg-rust]

2025-05-20 Thread via GitHub
jonathanc-n commented on code in PR #1336: URL: https://github.com/apache/iceberg-rust/pull/1336#discussion_r2098948747 ## crates/iceberg/src/transaction/mod.rs: ## @@ -128,6 +128,18 @@ impl<'a> Transaction<'a> { Ok(self) } +/// Add snapshot summary propertie

Re: [PR] feat: Introduce snapshot summary properties [iceberg-rust]

2025-05-20 Thread via GitHub
jonathanc-n commented on code in PR #1336: URL: https://github.com/apache/iceberg-rust/pull/1336#discussion_r2098948747 ## crates/iceberg/src/transaction/mod.rs: ## @@ -128,6 +128,18 @@ impl<'a> Transaction<'a> { Ok(self) } +/// Add snapshot summary propertie

Re: [PR] feat(transaction): Support snapshot validation [iceberg-rust]

2025-05-20 Thread via GitHub
jonathanc-n commented on code in PR #1353: URL: https://github.com/apache/iceberg-rust/pull/1353#discussion_r2098930807 ## crates/iceberg/src/transaction/snapshot.rs: ## @@ -307,6 +308,11 @@ impl<'a> SnapshotProduceAction<'a> { .await?; let next_seq_num =

Re: [I] Modify REST Tests to Bind Loopback instead of Localhost [iceberg]

2025-05-20 Thread via GitHub
stevenzwu commented on issue #13097: URL: https://github.com/apache/iceberg/issues/13097#issuecomment-2895920631 There are also some Spark tests have the same problem. it resolves to the public IP from `eth0` and then failed when resolving hostname from the public IP in the restricted firew

Re: [PR] feat: Add basic operations for `UpdateSchema` [iceberg-rust]

2025-05-20 Thread via GitHub
jonathanc-n commented on code in PR #1172: URL: https://github.com/apache/iceberg-rust/pull/1172#discussion_r2098906726 ## crates/iceberg/src/spec/schema/update.rs: ## @@ -0,0 +1,461 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor licens

Re: [PR] Spark-3.5: Add spark action to compute partition stats [iceberg]

2025-05-20 Thread via GitHub
karuppayya commented on code in PR #12450: URL: https://github.com/apache/iceberg/pull/12450#discussion_r2098882746 ## api/src/main/java/org/apache/iceberg/actions/ComputePartitionStats.java: ## @@ -34,6 +34,16 @@ public interface ComputePartitionStats */ ComputePartition

Re: [PR] feat: Add basic operations for `UpdateSchema` [iceberg-rust]

2025-05-20 Thread via GitHub
jonathanc-n commented on code in PR #1172: URL: https://github.com/apache/iceberg-rust/pull/1172#discussion_r2098920505 ## crates/iceberg/src/spec/schema/update.rs: ## @@ -0,0 +1,461 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor licens

Re: [PR] feat: Add basic operations for `UpdateSchema` [iceberg-rust]

2025-05-20 Thread via GitHub
jonathanc-n commented on code in PR #1172: URL: https://github.com/apache/iceberg-rust/pull/1172#discussion_r2098904751 ## crates/iceberg/src/spec/schema/update.rs: ## @@ -0,0 +1,461 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor licens

[I] feat: Support default values in `UpdateSchema` [iceberg-rust]

2025-05-20 Thread via GitHub
jonathanc-n opened a new issue, #1357: URL: https://github.com/apache/iceberg-rust/issues/1357 ### Is your feature request related to a problem or challenge? ``` There is a recent PR to support default values in UpdateSchema, it would be good to port that to iceberg-rs as well: ht

Re: [PR] CORE: [PARQUET] Log corrupted parquet filenames to trace bad nodes that may have written them. [iceberg]

2025-05-20 Thread via GitHub
bryanck commented on PR #13108: URL: https://github.com/apache/iceberg/pull/13108#issuecomment-2895871030 LGTM, though looks like you need to run `spotlessApply`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] feat: Add basic operations for `UpdateSchema` [iceberg-rust]

2025-05-20 Thread via GitHub
jonathanc-n commented on code in PR #1172: URL: https://github.com/apache/iceberg-rust/pull/1172#discussion_r2098900355 ## crates/iceberg/src/spec/schema/mod.rs: ## @@ -21,6 +21,7 @@ use std::collections::{HashMap, HashSet}; use std::fmt::{Display, Formatter}; use std::sync::A

Re: [PR] Range distribution iceberg sink [iceberg]

2025-05-20 Thread via GitHub
rodmeneses commented on code in PR #12071: URL: https://github.com/apache/iceberg/pull/12071#discussion_r2098898675 ## flink/v1.20/flink/src/test/java/org/apache/iceberg/flink/sink/TestFlinkIcebergSinkV2DistributionMode.java: ## @@ -0,0 +1,495 @@ +/* + * Licensed to the Apache S

Re: [PR] feat: Add basic operations for `UpdateSchema` [iceberg-rust]

2025-05-20 Thread via GitHub
jonathanc-n commented on code in PR #1172: URL: https://github.com/apache/iceberg-rust/pull/1172#discussion_r2098894436 ## crates/iceberg/src/spec/schema/update.rs: ## @@ -0,0 +1,461 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor licens

Re: [PR] fix: Writing Avro for Spark [iceberg-go]

2025-05-20 Thread via GitHub
zeroshade commented on PR #435: URL: https://github.com/apache/iceberg-go/pull/435#issuecomment-2895814025 @hendoxc Looks like the issue is https://github.com/trinodb/trino/issues/25702 and not something on our end here as far as I can tell. If I create the catalog in trino with `ice

Re: [PR] Spark 3.5: Structured Streaming read limit support follow-up [iceberg]

2025-05-20 Thread via GitHub
huaxingao commented on PR #12260: URL: https://github.com/apache/iceberg/pull/12260#issuecomment-2895788757 Merged. Thanks @wypoon for the PR! Thanks @singhpk234 @sririshindra for reviewing! -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] Spark 3.5: Structured Streaming read limit support follow-up [iceberg]

2025-05-20 Thread via GitHub
huaxingao merged PR #12260: URL: https://github.com/apache/iceberg/pull/12260 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Core, Data: File Format API interfaces [iceberg]

2025-05-20 Thread via GitHub
RussellSpitzer commented on PR #12774: URL: https://github.com/apache/iceberg/pull/12774#issuecomment-2895768606 @huaxingao, @pvary Could you take a look from a comet prospective? I know you have some custom code that would be using this as well -- This is an automated message from the Ap

Re: [PR] Core, Data: File Format API interfaces [iceberg]

2025-05-20 Thread via GitHub
RussellSpitzer commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2098824679 ## data/src/main/java/org/apache/iceberg/data/FileAccessFactoryRegistry.java: ## @@ -0,0 +1,269 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] fix: Writing Avro for Spark [iceberg-go]

2025-05-20 Thread via GitHub
zeroshade commented on PR #435: URL: https://github.com/apache/iceberg-go/pull/435#issuecomment-2895762271 @Fokko Added a test in the latest commit! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] feat(catalog/sql): Add views related ops [iceberg-go]

2025-05-20 Thread via GitHub
lliangyu-lin commented on code in PR #414: URL: https://github.com/apache/iceberg-go/pull/414#discussion_r2098816486 ## catalog/sql/sql_test.go: ## @@ -280,9 +280,13 @@ func (s *SqliteCatalogTestSuite) TestCreationOneTableExists() { _, err := sqldb.Exec(`CREATE TABLE "i

Re: [PR] feat(catalog/sql): Add views related ops [iceberg-go]

2025-05-20 Thread via GitHub
lliangyu-lin commented on code in PR #414: URL: https://github.com/apache/iceberg-go/pull/414#discussion_r2098807582 ## catalog/sql/sql.go: ## @@ -809,3 +821,369 @@ func (c *Catalog) UpdateNamespaceProperties(ctx context.Context, namespace table func (c *Catalog) CheckNamespac

Re: [PR] Range distribution iceberg sink [iceberg]

2025-05-20 Thread via GitHub
stevenzwu commented on code in PR #12071: URL: https://github.com/apache/iceberg/pull/12071#discussion_r2098767251 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -375,22 +381,43 @@ public Builder flinkConf(ReadableConfig config) {

Re: [PR] Build: Bump cython from 3.0.12 to 3.1.1 [iceberg-python]

2025-05-20 Thread via GitHub
Fokko merged PR #2022: URL: https://github.com/apache/iceberg-python/pull/2022 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] API: Compute truncate decimal result precision based on lowest value bound [iceberg]

2025-05-20 Thread via GitHub
lliangyu-lin commented on code in PR #12969: URL: https://github.com/apache/iceberg/pull/12969#discussion_r2098773053 ## api/src/test/java/org/apache/iceberg/transforms/TestTruncate.java: ## @@ -85,6 +87,43 @@ public void testTruncateDecimal() { assertThat(trunc.apply(new B

Re: [PR] API: Compute truncate decimal result precision based on lowest value bound [iceberg]

2025-05-20 Thread via GitHub
lliangyu-lin commented on code in PR #12969: URL: https://github.com/apache/iceberg/pull/12969#discussion_r2098773053 ## api/src/test/java/org/apache/iceberg/transforms/TestTruncate.java: ## @@ -85,6 +87,43 @@ public void testTruncateDecimal() { assertThat(trunc.apply(new B

Re: [PR] Core, AWS: use loopback address explicitly for jetty server in test. [iceberg]

2025-05-20 Thread via GitHub
stevenzwu commented on PR #13101: URL: https://github.com/apache/iceberg/pull/13101#issuecomment-2895687019 sth was messed up in last force push. reopen the PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [PR] Core, AWS: use loopback address explicitly for jetty server in test. [iceberg]

2025-05-20 Thread via GitHub
stevenzwu commented on code in PR #13101: URL: https://github.com/apache/iceberg/pull/13101#discussion_r2098758424 ## core/src/test/java/org/apache/iceberg/rest/TestRESTCatalog.java: ## @@ -166,7 +167,7 @@ public T execute( servletContext.addServlet(new ServletHolder(new

Re: [PR] Range distribution iceberg sink [iceberg]

2025-05-20 Thread via GitHub
rodmeneses commented on code in PR #12071: URL: https://github.com/apache/iceberg/pull/12071#discussion_r2098754472 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -645,70 +676,121 @@ private DataStream distributeDataStream(DataStream in

Re: [PR] Build: Bump mkdocs-material from 9.6.13 to 9.6.14 [iceberg-python]

2025-05-20 Thread via GitHub
Fokko merged PR #2024: URL: https://github.com/apache/iceberg-python/pull/2024 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Core, AWS: use loopback address explicitly for jetty server in test. [iceberg]

2025-05-20 Thread via GitHub
stevenzwu closed pull request #13101: Core, AWS: use loopback address explicitly for jetty server in test. URL: https://github.com/apache/iceberg/pull/13101 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] feat: `validate_deleted_data_files` [iceberg-python]

2025-05-20 Thread via GitHub
Fokko merged PR #1938: URL: https://github.com/apache/iceberg-python/pull/1938 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Core, Data: File Format API interfaces [iceberg]

2025-05-20 Thread via GitHub
RussellSpitzer commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2098728163 ## core/src/main/java/org/apache/iceberg/io/WriteBuilder.java: ## @@ -0,0 +1,120 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

Re: [I] Support Concurrency Safety Validation: Implement `validateDeletedDataFiles` [iceberg-python]

2025-05-20 Thread via GitHub
Fokko closed issue #1928: Support Concurrency Safety Validation: Implement `validateDeletedDataFiles` URL: https://github.com/apache/iceberg-python/issues/1928 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] feat: `validate_deleted_data_files` [iceberg-python]

2025-05-20 Thread via GitHub
Fokko commented on code in PR #1938: URL: https://github.com/apache/iceberg-python/pull/1938#discussion_r2098728278 ## pyiceberg/table/update/validate.py: ## @@ -69,3 +75,78 @@ def validation_history( raise ValidationException("No matching snapshot found.") retur

Re: [PR] core: Send single snapshot to remove rather than in bulk [iceberg]

2025-05-20 Thread via GitHub
aihuaxu commented on code in PR #13100: URL: https://github.com/apache/iceberg/pull/13100#discussion_r2098728228 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1838,11 +1832,7 @@ private static List updateSnapshotLog( Set intermediateSnapshotIds = int

Re: [PR] Core, Data: File Format API interfaces [iceberg]

2025-05-20 Thread via GitHub
RussellSpitzer commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2098726421 ## core/src/main/java/org/apache/iceberg/io/WriteBuilder.java: ## @@ -0,0 +1,120 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

Re: [PR] core: Send single snapshot to remove rather than in bulk [iceberg]

2025-05-20 Thread via GitHub
amogh-jahagirdar commented on code in PR #13100: URL: https://github.com/apache/iceberg/pull/13100#discussion_r2098703160 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1838,11 +1832,7 @@ private static List updateSnapshotLog( Set intermediateSnapshot

Re: [PR] Core, Data: File Format API interfaces [iceberg]

2025-05-20 Thread via GitHub
RussellSpitzer commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2098711363 ## core/src/main/java/org/apache/iceberg/io/WriteBuilder.java: ## @@ -0,0 +1,120 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

Re: [PR] core: Send single snapshot to remove rather than in bulk [iceberg]

2025-05-20 Thread via GitHub
amogh-jahagirdar commented on code in PR #13100: URL: https://github.com/apache/iceberg/pull/13100#discussion_r2098703160 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1838,11 +1832,7 @@ private static List updateSnapshotLog( Set intermediateSnapshot

Re: [PR] Build: Bump huggingface-hub from 0.31.2 to 0.31.4 [iceberg-python]

2025-05-20 Thread via GitHub
Fokko merged PR #2020: URL: https://github.com/apache/iceberg-python/pull/2020 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Build: Bump mypy-boto3-glue from 1.38.12 to 1.38.18 [iceberg-python]

2025-05-20 Thread via GitHub
Fokko merged PR #2021: URL: https://github.com/apache/iceberg-python/pull/2021 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Build: Bump getdaft from 0.4.14 to 0.4.15 [iceberg-python]

2025-05-20 Thread via GitHub
Fokko merged PR #2023: URL: https://github.com/apache/iceberg-python/pull/2023 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Build: Bump sqlalchemy from 2.0.40 to 2.0.41 [iceberg-python]

2025-05-20 Thread via GitHub
Fokko merged PR #2025: URL: https://github.com/apache/iceberg-python/pull/2025 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Build: Bump fastavro from 1.10.0 to 1.11.1 [iceberg-python]

2025-05-20 Thread via GitHub
Fokko merged PR #2026: URL: https://github.com/apache/iceberg-python/pull/2026 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Build: Bump setuptools from 78.1.0 to 78.1.1 [iceberg-python]

2025-05-20 Thread via GitHub
Fokko merged PR #2027: URL: https://github.com/apache/iceberg-python/pull/2027 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] feat(transaction): Support snapshot validation [iceberg-rust]

2025-05-20 Thread via GitHub
jonathanc-n commented on PR #1353: URL: https://github.com/apache/iceberg-rust/pull/1353#issuecomment-2895515739 I'll tkae a look later today! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] feat: add manifest related structures [iceberg-cpp]

2025-05-20 Thread via GitHub
Fokko merged PR #91: URL: https://github.com/apache/iceberg-cpp/pull/91 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [PR] feat: add manifest related structures [iceberg-cpp]

2025-05-20 Thread via GitHub
Fokko commented on PR #91: URL: https://github.com/apache/iceberg-cpp/pull/91#issuecomment-2895504012 Thanks @zhjwpku for adding this, and thanks @wgtmac and @gty404 for the review 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] fix: Writing Avro for Spark [iceberg-go]

2025-05-20 Thread via GitHub
Fokko commented on PR #435: URL: https://github.com/apache/iceberg-go/pull/435#issuecomment-2895497194 Should we also include a test as part of this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] fix: Writing Avro for Spark [iceberg-go]

2025-05-20 Thread via GitHub
hendoxc commented on PR #435: URL: https://github.com/apache/iceberg-go/pull/435#issuecomment-2895484610 > @hendoxc Can you take a look and verify this fixes your issue? I gave this a go with ``` replace github.com/apache/iceberg-go v0.2.1-0.20250510221226-f32578d5a843 => github

Re: [PR] Core, Data: File Format API interfaces [iceberg]

2025-05-20 Thread via GitHub
stevenzwu commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2098442003 ## core/src/main/java/org/apache/iceberg/io/WriteBuilder.java: ## @@ -0,0 +1,120 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more c

  1   2   >