Re: [PR] feat: avro support applying field-ids based on name mapping [iceberg-cpp]

mapleFU commented on code in PR #127: URL: https://github.com/apache/iceberg-cpp/pull/127#discussion_r2163038104 ## src/iceberg/avro/avro_reader.cc: ## @@ -96,11 +99,22 @@ class AvroBatchReader::Impl { // Validate field ids in the file schema. HasIdVisitor has_id_visit

Re: [PR] feat: avro support applying field-ids based on name mapping [iceberg-cpp]

MisterRaindrop commented on code in PR #127: URL: https://github.com/apache/iceberg-cpp/pull/127#discussion_r2163004896 ## src/iceberg/CMakeLists.txt: ## @@ -98,7 +98,8 @@ if(ICEBERG_BUILD_BUNDLE) avro/avro_data_util.cc avro/avro_reader.cc avro/avro_schema_u

Re: [PR] feat: avro support applying field-ids based on name mapping [iceberg-cpp]

MisterRaindrop commented on code in PR #127: URL: https://github.com/apache/iceberg-cpp/pull/127#discussion_r2163003327 ## src/iceberg/CMakeLists.txt: ## @@ -98,7 +98,8 @@ if(ICEBERG_BUILD_BUNDLE) avro/avro_data_util.cc avro/avro_reader.cc avro/avro_schema_u

[PR] Flink:Backport Revise the display of the task name in TableMaintenance to show the specific task name [iceberg]

Guosmilesmile opened a new pull request, #13372: URL: https://github.com/apache/iceberg/pull/13372 Back port https://github.com/apache/iceberg/pull/13024 to Flink 1.19 and 1.20 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [PR] feat: add support for avro to arrow data conversion [iceberg-cpp]

wgtmac commented on PR #124: URL: https://github.com/apache/iceberg-cpp/pull/124#issuecomment-2998714792 This PR is complete. Please take a look @Fokko -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] Flink:Backport Revise the display of the task name in TableMaintenance to show the specific task name [iceberg]

Guosmilesmile commented on PR #13372: URL: https://github.com/apache/iceberg/pull/13372#issuecomment-2998676482 This is a clearn backport -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [I] `iceberg-rest-fixture` errors with SQLiteException under high concurrent load [iceberg]

kevinjqliu commented on issue #13366: URL: https://github.com/apache/iceberg/issues/13366#issuecomment-2998579092 > I think it would be good to move back to the file-based repository. 👍 Made the change in #13367 > I think it also makes more sense with the minio setup we use to

Re: [PR] feat: add support for avro to arrow data conversion [iceberg-cpp]

wgtmac commented on code in PR #124: URL: https://github.com/apache/iceberg-cpp/pull/124#discussion_r2162826126 ## src/iceberg/avro/avro_data_util_internal.h: ## @@ -26,10 +26,21 @@ namespace iceberg::avro { +/// \brief Append an Avro datum to an Arrow array builder. +/// +

[PR] ci: enable sanitizer [iceberg-cpp]

wgtmac opened a new pull request, #129: URL: https://github.com/apache/iceberg-cpp/pull/129 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

Re: [PR] Build: Bump cachetools from 5.5.2 to 6.1.0 [iceberg-python]

kevinjqliu commented on PR #2145: URL: https://github.com/apache/iceberg-python/pull/2145#issuecomment-2998554781 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] feat: avro support applying field-ids based on name mapping [iceberg-cpp]

wgtmac commented on code in PR #127: URL: https://github.com/apache/iceberg-cpp/pull/127#discussion_r2162828781 ## src/iceberg/CMakeLists.txt: ## @@ -98,7 +98,8 @@ if(ICEBERG_BUILD_BUNDLE) avro/avro_data_util.cc avro/avro_reader.cc avro/avro_schema_util.cc -

Re: [PR] feat: add support for avro to arrow data conversion [iceberg-cpp]

wgtmac commented on code in PR #124: URL: https://github.com/apache/iceberg-cpp/pull/124#discussion_r2162825065 ## src/iceberg/avro/avro_data_util.cc: ## @@ -17,16 +17,440 @@ * under the License. */ +#include +#include +#include +#include +#include +#include +#includ

Re: [PR] Flink: Supports delete orphan files in TableMaintenance [iceberg]

Guosmilesmile commented on code in PR #13302: URL: https://github.com/apache/iceberg/pull/13302#discussion_r2162824352 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/maintenance/api/DeleteOrphanFiles.java: ## @@ -0,0 +1,358 @@ +/* + * Licensed to the Apache Software

Re: [PR] feat(transaction): Implement TransactionAction for FastAppendAction [iceberg-rust]

liurenjie1024 commented on code in PR #1448: URL: https://github.com/apache/iceberg-rust/pull/1448#discussion_r2162823355 ## crates/iceberg/src/spec/manifest/writer.rs: ## @@ -127,7 +127,7 @@ impl ManifestWriter { pub(crate) fn new( output: OutputFile, sna

Re: [PR] Build: Use Java 21 to publish snapshot to Maven [iceberg]

pan3793 commented on PR #13369: URL: https://github.com/apache/iceberg/pull/13369#issuecomment-2998507805 cc @huaxingao @pvary @RussellSpitzer -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] Iceberg + MinIO S3 - Invalid signature after 3 hours [iceberg]

omurad commented on issue #13045: URL: https://github.com/apache/iceberg/issues/13045#issuecomment-2998476255 @c-thiel Fixed by disabling request signing on my Nessie REST catalog (`requestSigningEnabled` Helm chart value). So it was not an iceberg connector issue after all. I hope this hel

Re: [I] Iceberg + MinIO S3 - Invalid signature after 3 hours [iceberg]

omurad closed issue #13045: Iceberg + MinIO S3 - Invalid signature after 3 hours URL: https://github.com/apache/iceberg/issues/13045 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] Feature: Write to branches [iceberg-python]

vinjai commented on code in PR #941: URL: https://github.com/apache/iceberg-python/pull/941#discussion_r2162799415 ## pyiceberg/table/__init__.py: ## @@ -774,12 +797,24 @@ def upsert( matched_predicate = upsert_util.create_match_filter(df, join_cols) # We mus

[PR] Build: Bump cachetools from 5.5.2 to 6.1.0 [iceberg-python]

dependabot[bot] opened a new pull request, #2145: URL: https://github.com/apache/iceberg-python/pull/2145 Bumps [cachetools](https://github.com/tkem/cachetools) from 5.5.2 to 6.1.0. Changelog Sourced from https://github.com/tkem/cachetools/blob/master/CHANGELOG.rst";>cachetools's c

Re: [PR] Core, Data: File Format API interfaces [iceberg]

rdblue commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2162731545 ## core/src/main/java/org/apache/iceberg/io/ObjectModel.java: ## @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contr

[PR] Build: Bump mypy-boto3-glue from 1.38.22 to 1.38.42 [iceberg-python]

dependabot[bot] opened a new pull request, #2144: URL: https://github.com/apache/iceberg-python/pull/2144 Bumps [mypy-boto3-glue](https://github.com/youtype/mypy_boto3_builder) from 1.38.22 to 1.38.42. Release notes Sourced from https://github.com/youtype/mypy_boto3_builder/release

Re: [PR] Spark 4.0: Migrate Iceberg Stored Procedures to Spark built-in implementations [iceberg]

github-actions[bot] commented on PR #13106: URL: https://github.com/apache/iceberg/pull/13106#issuecomment-2998354465 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] AWS: add view support for Glue Catalog [iceberg]

github-actions[bot] closed pull request #12544: AWS: add view support for Glue Catalog URL: https://github.com/apache/iceberg/pull/12544 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] AWS: add view support for Glue Catalog [iceberg]

github-actions[bot] commented on PR #12544: URL: https://github.com/apache/iceberg/pull/12544#issuecomment-2998354383 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] Spark: support rewrite on specified target branch [iceberg]

github-actions[bot] commented on PR #12257: URL: https://github.com/apache/iceberg/pull/12257#issuecomment-2998354343 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] Use apache/spark image in the quickstart [iceberg]

github-actions[bot] closed issue #11746: Use apache/spark image in the quickstart URL: https://github.com/apache/iceberg/issues/11746 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [I] Use apache/spark image in the quickstart [iceberg]

github-actions[bot] commented on issue #11746: URL: https://github.com/apache/iceberg/issues/11746#issuecomment-2998354230 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] Rest Catalog: spark catalog api fails to work with rest based catalog [iceberg]

github-actions[bot] commented on issue #11741: URL: https://github.com/apache/iceberg/issues/11741#issuecomment-2998354198 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] Rest Catalog: spark catalog api fails to work with rest based catalog [iceberg]

github-actions[bot] closed issue #11741: Rest Catalog: spark catalog api fails to work with rest based catalog URL: https://github.com/apache/iceberg/issues/11741 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] [wip]: Decouple Hadoop Configuration from FlinkCategoryFactory [iceberg]

dyrnq commented on PR #4768: URL: https://github.com/apache/iceberg/pull/4768#issuecomment-2998330345 Mark -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

Re: [PR] Core, Data: File Format API interfaces [iceberg]

rdblue commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2162712202 ## core/src/main/java/org/apache/iceberg/io/FileAccessFactory.java: ## @@ -0,0 +1,139 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] Core, Data: File Format API interfaces [iceberg]

rdblue commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2162710191 ## core/src/main/java/org/apache/iceberg/io/ReadBuilder.java: ## @@ -0,0 +1,136 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contr

Re: [PR] Core, Data: File Format API interfaces [iceberg]

rdblue commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2162711922 ## core/src/main/java/org/apache/iceberg/io/FileAccessFactory.java: ## @@ -0,0 +1,139 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] Core, Data: File Format API interfaces [iceberg]

rdblue commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2162709047 ## core/src/main/java/org/apache/iceberg/io/ReadBuilder.java: ## @@ -0,0 +1,131 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contr

Re: [PR] Core, Data: File Format API interfaces [iceberg]

rdblue commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2162709629 ## core/src/main/java/org/apache/iceberg/io/ReadBuilder.java: ## @@ -0,0 +1,136 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contr

Re: [PR] Core, Data: File Format API interfaces [iceberg]

rdblue commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2162702140 ## core/src/main/java/org/apache/iceberg/io/ReadBuilder.java: ## @@ -0,0 +1,136 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contr

Re: [PR] Core, Data: File Format API interfaces [iceberg]

rdblue commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2162701774 ## core/src/main/java/org/apache/iceberg/io/ReadBuilder.java: ## @@ -0,0 +1,136 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contr

Re: [PR] Core, Data: File Format API interfaces [iceberg]

rdblue commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2162700119 ## core/src/main/java/org/apache/iceberg/io/ReadBuilder.java: ## @@ -0,0 +1,136 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contr

Re: [PR] Core, Data: File Format API interfaces [iceberg]

rdblue commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2162696513 ## core/src/main/java/org/apache/iceberg/io/FileAccessFactory.java: ## @@ -0,0 +1,139 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] Core, Data: File Format API interfaces [iceberg]

rdblue commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2162696513 ## core/src/main/java/org/apache/iceberg/io/FileAccessFactory.java: ## @@ -0,0 +1,139 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] Core, Data: File Format API interfaces [iceberg]

rdblue commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2162695074 ## core/src/main/java/org/apache/iceberg/io/FileAccessFactory.java: ## @@ -0,0 +1,139 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] Core, Data: File Format API interfaces [iceberg]

rdblue commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2162683580 ## core/src/main/java/org/apache/iceberg/io/FileAccessFactory.java: ## @@ -0,0 +1,139 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] Core: Interface based DataFile reader and writer API [iceberg]

rdblue commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r2162672594 ## data/src/main/java/org/apache/iceberg/data/RegistryBasedFileWriterFactory.java: ## @@ -0,0 +1,191 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] refactor: consolidate snapshot expiration into MaintenanceTable [iceberg-python]

ForeverAngry commented on PR #2143: URL: https://github.com/apache/iceberg-python/pull/2143#issuecomment-2998235100 @Fokko @jayceslesar let me know if you guys prefer i stack this pr into the #1200 or if you both would rather i wait until the #1200 is merged into `main`, and then rebase on

[PR] feat(catalog): Add TableCommit::apply to help update metadata [iceberg-rust]

CTTY opened a new pull request, #1464: URL: https://github.com/apache/iceberg-rust/pull/1464 ## Which issue does this PR close? - Closes https://github.com/apache/iceberg-rust/issues/1386 ## What changes are included in this PR? - Added `TableCommit::apply` to app

[PR] refactor: consolidate snapshot expiration into MaintenanceTable [iceberg-python]

ForeverAngry opened a new pull request, #2143: URL: https://github.com/apache/iceberg-python/pull/2143 # Rationale for this change Consolidates snapshot expiration functionality from the standalone `ExpireSnapshots` class into the `MaintenanceTable` class for a uni

Re: [PR] Spark: Make maxRecordPerMicrobatch a soft limit [iceberg]

stevenzwu commented on code in PR #12988: URL: https://github.com/apache/iceberg/pull/12988#discussion_r2162658679 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestStructuredStreamingRead3.java: ## @@ -200,7 +198,7 @@ public void testReadStreamWithMaxRows2()

[I] refactor: consolidate snapshot expiration into MaintenanceTable [iceberg-python]

ForeverAngry opened a new issue, #2142: URL: https://github.com/apache/iceberg-python/issues/2142 ### Feature Request / Improvement This issue tracks to the work planned and left over from #1880. It depends on, and stacks onto the work done in #1200. -- This is an automated messag

Re: [I] Proposal: Implement table maintenance operations [iceberg-rust]

cmcarthur closed issue #1453: Proposal: Implement table maintenance operations URL: https://github.com/apache/iceberg-rust/issues/1453 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] test: use `create or replace` [iceberg-go]

kevinjqliu commented on PR #466: URL: https://github.com/apache/iceberg-go/pull/466#issuecomment-2998157820 Lets wait for https://github.com/apache/iceberg-python/pull/2140 to be merged first -- This is an automated message from the Apache Git Service. To respond to the message, please l

Re: [PR] test: use `create or replace` [iceberg-python]

kevinjqliu commented on PR #2140: URL: https://github.com/apache/iceberg-python/pull/2140#issuecomment-2998157247 Follow up to https://github.com/apache/iceberg-python/pull/2137#discussion_r2162478760 I guess we can remove it since the test passed -- This is an automated message f

Re: [I] Avoid Vec allocation in Transaction::do_commit [iceberg-rust]

CTTY commented on issue #1437: URL: https://github.com/apache/iceberg-rust/issues/1437#issuecomment-2998147938 This is closed by https://github.com/apache/iceberg-rust/pull/1451 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [I] Avoid Vec allocation in Transaction::do_commit [iceberg-rust]

CTTY closed issue #1437: Avoid Vec allocation in Transaction::do_commit URL: https://github.com/apache/iceberg-rust/issues/1437 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] Add retry in `Transaction::commit` method [iceberg-rust]

CTTY commented on issue #1387: URL: https://github.com/apache/iceberg-rust/issues/1387#issuecomment-2998145703 There is a discussion going on about the design of retry/retrable errors, please see this thread: https://github.com/apache/iceberg-rust/pull/1383#discussion_r2158961780 Thi

Re: [PR] docs: add `Transaction` example [iceberg-rust]

CTTY commented on PR #1436: URL: https://github.com/apache/iceberg-rust/pull/1436#issuecomment-2998140719 Hi @jdockerty , with the closing of https://github.com/apache/iceberg-rust/pull/1451, Transaction API semantic should be finalized as of now. Please feel free to resume this effort!

Re: [PR] Core, Data: File Format API interfaces [iceberg]

rdblue commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2162622583 ## core/src/main/java/org/apache/iceberg/io/FileAccessFactory.java: ## @@ -0,0 +1,139 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] feat(transaction): Implement TransactionAction for FastAppendAction [iceberg-rust]

dentiny commented on code in PR #1448: URL: https://github.com/apache/iceberg-rust/pull/1448#discussion_r2162620617 ## crates/iceberg/src/transaction/append.rs: ## @@ -62,101 +59,51 @@ impl FastAppendAction { } /// Add data files to the snapshot. -pub fn add_data

Re: [PR] Core: Interface based DataFile reader and writer API [iceberg]

rdblue commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r2162619276 ## data/src/main/java/org/apache/iceberg/data/RegistryBasedFileWriterFactory.java: ## @@ -0,0 +1,196 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] feat(transaction): Implement TransactionAction for FastAppendAction [iceberg-rust]

CTTY commented on code in PR #1448: URL: https://github.com/apache/iceberg-rust/pull/1448#discussion_r2162617409 ## crates/iceberg/src/transaction/append.rs: ## @@ -62,101 +59,51 @@ impl FastAppendAction { } /// Add data files to the snapshot. -pub fn add_data_fi

Re: [PR] feat(transaction): Implement TransactionAction for FastAppendAction [iceberg-rust]

CTTY commented on code in PR #1448: URL: https://github.com/apache/iceberg-rust/pull/1448#discussion_r2162616750 ## crates/iceberg/src/spec/manifest/writer.rs: ## @@ -127,7 +127,7 @@ impl ManifestWriter { pub(crate) fn new( output: OutputFile, snapshot_id:

[PR] test: use `create or replace` [iceberg-go]

kevinjqliu opened a new pull request, #466: URL: https://github.com/apache/iceberg-go/pull/466 Remove all mentions of `CREATE OR REPLACE` issue, https://github.com/apache/iceberg/issues/8756 Similar to https://github.com/apache/iceberg-python/pull/2137 and https://github.com/apache/icebe

[PR] Added new configuration parameter hive.kerberos-service-name (#2032) [iceberg-python]

gardenia opened a new pull request, #2141: URL: https://github.com/apache/iceberg-python/pull/2141 # Rationale for this change Added new configuration parameter hive.kerberos-service-name (#2032) hive.kerberos-service-name Defaults to "hive"

Re: [PR] Add warehouse parameter to the REST Catalog doc [iceberg-python]

kevinjqliu merged PR #2066: URL: https://github.com/apache/iceberg-python/pull/2066 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

[PR] test: use `create or replace` [iceberg-python]

kevinjqliu opened a new pull request, #2140: URL: https://github.com/apache/iceberg-python/pull/2140 # Rationale for this change Follow up to #2137 Remove all mentions of `CREATE OR REPLACE` issue, https://github.com/apache/iceberg/issues/8756 # Are these cha

Re: [PR] Core: Interface based DataFile reader and writer API [iceberg]

stevenzwu commented on PR #12298: URL: https://github.com/apache/iceberg/pull/12298#issuecomment-2998113103 Right now, there are two questions that Peter and I are discussing offline. 1) is it necessary to have a globally shared `FileAccessFactoryRegistry` in the iceberg-data module a

Re: [PR] Core: Interface based DataFile reader and writer API [iceberg]

rdblue commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r2162604687 ## data/src/main/java/org/apache/iceberg/data/RegistryBasedFileWriterFactory.java: ## @@ -0,0 +1,196 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] Spark-3.5, 4.0: Add unit tests for ColumnarBatchUtil [iceberg]

anuragmantri commented on PR #12275: URL: https://github.com/apache/iceberg/pull/12275#issuecomment-2998073354 > shall we also have a test for the mixed case (both equality + position deletes)? Yes, added a mixed case test. -- This is an automated message from the Apache Git Servi

Re: [PR] AWS: Support metrics tracking when using Analytics Accelerator stream [iceberg]

stubz151 commented on code in PR #13348: URL: https://github.com/apache/iceberg/pull/13348#discussion_r2162031142 ## aws/src/main/java/org/apache/iceberg/aws/s3/AnalyticsAcceleratorInputStreamWrapper.java: ## @@ -19,31 +19,47 @@ package org.apache.iceberg.aws.s3; import java

Re: [PR] Spark-3.5, 4.0: Add unit tests for ColumnarBatchUtil [iceberg]

anuragmantri commented on code in PR #12275: URL: https://github.com/apache/iceberg/pull/12275#discussion_r2162592802 ## spark/v4.0/spark/src/test/java/org/apache/iceberg/spark/data/vectorized/TestColumnarBatchUtil.java: ## @@ -0,0 +1,254 @@ +/* + * Licensed to the Apache Softwa

Re: [PR] Core: Interface based DataFile reader and writer API [iceberg]

rdblue commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r2162586644 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkFileWriterFactory.java: ## @@ -68,119 +54,34 @@ class SparkFileWriterFactory extends BaseFileWr

Re: [PR] [infra] publish rc to pypi as part of release process [iceberg-rust]

kevinjqliu commented on code in PR #1449: URL: https://github.com/apache/iceberg-rust/pull/1449#discussion_r2162430124 ## .github/workflows/publish.yml: ## @@ -20,7 +20,10 @@ name: Publish on: push: tags: - - "*" + # Trigger this workflow when tag follows the

Re: [PR] Core, Data: File Format API interfaces [iceberg]

rdblue commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2162569873 ## build.gradle: ## @@ -373,6 +373,7 @@ project(':iceberg-data') { dependencies { implementation project(path: ':iceberg-bundled-guava', configuration: 'shado

Re: [PR] Core, Data: File Format API interfaces [iceberg]

rdblue commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2162569005 ## api/src/main/java/org/apache/iceberg/io/FileReader.java: ## @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contribu

Re: [PR] [infra] publish rc to pypi as part of release process [iceberg-rust]

Fokko commented on code in PR #1449: URL: https://github.com/apache/iceberg-rust/pull/1449#discussion_r2162411590 ## .github/workflows/publish.yml: ## @@ -20,7 +20,10 @@ name: Publish on: push: tags: - - "*" + # Trigger this workflow when tag follows the versi

[PR] Docs: Condens Python API docs [iceberg-python]

Fokko opened a new pull request, #2139: URL: https://github.com/apache/iceberg-python/pull/2139 I noticed that the docs needed some TLC. - Collapsed some lines to make it more compact. - Avoid imports where possible (eg transforms). - Update docs. - Add an example of the `to_a

Re: [PR] Run `test_create_table_transaction` for Rest V1 tables [iceberg-python]

Fokko commented on PR #2137: URL: https://github.com/apache/iceberg-python/pull/2137#issuecomment-2997900787 Thanks @kevinjqliu for the review 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [I] Duplicate File Remediation [iceberg-python]

MrDerecho commented on issue #2130: URL: https://github.com/apache/iceberg-python/issues/2130#issuecomment-2997900165 @kevinjqliu, for context I am referring to Trino (Athena) tables can deal with duplicate files referenced in the metadata- other upstream consumers i.e. snowflake external

Re: [PR] Run `test_create_table_transaction` for Rest V1 tables [iceberg-python]

Fokko merged PR #2137: URL: https://github.com/apache/iceberg-python/pull/2137 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Run `test_create_table_transaction` for Rest V1 tables [iceberg-python]

Fokko commented on code in PR #2137: URL: https://github.com/apache/iceberg-python/pull/2137#discussion_r2162478760 ## dev/provision.py: ## @@ -279,7 +279,7 @@ Review Comment: Unfortunally not, the `OR REPLACE` syntax requires a SQL extension I believe, so I'll leave that

Re: [PR] Run `test_create_table_transaction` for Rest V1 tables [iceberg-python]

kevinjqliu commented on code in PR #2137: URL: https://github.com/apache/iceberg-python/pull/2137#discussion_r2162473528 ## dev/provision.py: ## @@ -279,7 +279,7 @@ Review Comment: good catch, i think we can also remove L276-L278 -- This is an automated message from th

Re: [PR] Optimise RowData evolution [iceberg]

pvary commented on code in PR #13340: URL: https://github.com/apache/iceberg/pull/13340#discussion_r2162466950 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/TableMetadataCache.java: ## @@ -220,37 +238,59 @@ SchemaInfo getSchemaInfo() { */ static

Re: [PR] [infra] publish rc to pypi as part of release process [iceberg-rust]

kevinjqliu commented on code in PR #1449: URL: https://github.com/apache/iceberg-rust/pull/1449#discussion_r2162430124 ## .github/workflows/publish.yml: ## @@ -20,7 +20,10 @@ name: Publish on: push: tags: - - "*" + # Trigger this workflow when tag follows the

Re: [PR] ICEBERG-13343: fix race condition in JdbcCatalog [iceberg]

Fokko commented on code in PR #13345: URL: https://github.com/apache/iceberg/pull/13345#discussion_r2162391125 ## core/src/test/java/org/apache/iceberg/jdbc/TestJdbcTableConcurrency.java: ## @@ -149,4 +190,1015 @@ public synchronized void testConcurrentConnections() throws Inte

Re: [PR] Flink: Fix flaky test in testTwoSinksInDisjointedDAG [iceberg]

rodmeneses commented on PR #13349: URL: https://github.com/apache/iceberg/pull/13349#issuecomment-2997701792 Hi @Guosmilesmile Thanks for reporting this issue. I have a couple question/comments: 1. How "flaky" is this unit test? is it flaky in a particular Flink Version? Have you been ab

Re: [I] `iceberg-rest-fixture` errors with SQLiteException under high concurrent load [iceberg]

Fokko commented on issue #13366: URL: https://github.com/apache/iceberg/issues/13366#issuecomment-2997679063 Nice catch, glad we still have the old repository to check. I think it would be good to move back to the file-based repository. I think it also makes more sense with the minio setup

Re: [PR] Optimise RowData evolution [iceberg]

mxm commented on code in PR #13340: URL: https://github.com/apache/iceberg/pull/13340#discussion_r2162371074 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/TableMetadataCache.java: ## @@ -220,37 +238,59 @@ SchemaInfo getSchemaInfo() { */ static cl

Re: [PR] feature: expire snapshots action [iceberg-rust]

cmcarthur closed pull request #1455: feature: expire snapshots action URL: https://github.com/apache/iceberg-rust/pull/1455 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

Re: [PR] feature: expire snapshots action [iceberg-rust]

cmcarthur commented on PR #1455: URL: https://github.com/apache/iceberg-rust/pull/1455#issuecomment-2997360798 @liurenjie1024 yes, I understand. I'll break this PR down as follows: 1. change(s) to ensure that the needed public APIs are exposed for building table maintenance directly i

Re: [I] Maintenance: Expire Snapshots Action [iceberg-rust]

cmcarthur closed issue #1454: Maintenance: Expire Snapshots Action URL: https://github.com/apache/iceberg-rust/issues/1454 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [PR] add toString implementations to SparkPositionDeltaWrite [iceberg]

singhpk234 commented on code in PR #13371: URL: https://github.com/apache/iceberg/pull/13371#discussion_r2162037598 ## spark/v3.4/spark/src/main/java/org/apache/iceberg/spark/source/SparkPositionDeltaWrite.java: ## @@ -266,6 +266,11 @@ public void commit(WriterCommitMessage[] me

Re: [PR] Optimise RowData evolution [iceberg]

mxm commented on code in PR #13340: URL: https://github.com/apache/iceberg/pull/13340#discussion_r2162319680 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/TableMetadataCache.java: ## @@ -220,37 +238,59 @@ SchemaInfo getSchemaInfo() { */ static cl

Re: [PR] Flink: Migrate Flink `TableSchema` to `Schema`/`ResolvedSchema` [iceberg]

mxm commented on code in PR #13072: URL: https://github.com/apache/iceberg/pull/13072#discussion_r2162279624 ## flink/v2.0/flink/src/test/java/org/apache/iceberg/flink/sink/TestFlinkIcebergSinkV2DistributionMode.java: ## @@ -122,7 +122,7 @@ public void before() throws IOExceptio

Re: [I] discussion: token refresh mechanism for rest client [iceberg-rust]

cmcarthur commented on issue #437: URL: https://github.com/apache/iceberg-rust/issues/437#issuecomment-2997408926 as an alternative, can we simply expose a public `refresh_token` method that allows the caller to implement their own token invalidation (for my use case, via option 1, using to

Re: [I] Issue with kerberized hive metastore [iceberg-python]

gardenia commented on issue #2032: URL: https://github.com/apache/iceberg-python/issues/2032#issuecomment-2997385005 > I'm interested in working on this issue if there's no takers! @kris-gaudel I wrote the basics of a patch for this last week and just haven't had a chance to upload i

Re: [PR] Core: Fix numeric overflow of timestamp nano literal [iceberg]

stevenzwu commented on code in PR #11775: URL: https://github.com/apache/iceberg/pull/11775#discussion_r2162151134 ## api/src/main/java/org/apache/iceberg/expressions/Literals.java: ## @@ -300,8 +300,7 @@ public Literal to(Type type) { case TIMESTAMP: return

Re: [I] Implement an `AddFields` operation [iceberg-rust]

cmcarthur closed issue #1174: Implement an `AddFields` operation URL: https://github.com/apache/iceberg-rust/issues/1174 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

Re: [PR] Core: Fix numeric overflow of timestamp nano literal [iceberg]

stevenzwu commented on code in PR #11775: URL: https://github.com/apache/iceberg/pull/11775#discussion_r2162151134 ## api/src/main/java/org/apache/iceberg/expressions/Literals.java: ## @@ -300,8 +300,7 @@ public Literal to(Type type) { case TIMESTAMP: return

Re: [PR] Build and test hive-metastore with Hive 2, 3 and 4 with a single source set [iceberg]

wypoon commented on PR #12721: URL: https://github.com/apache/iceberg/pull/12721#issuecomment-2997309476 I believe that there are many organizations using Iceberg currently that are using the HiveCatalog. That is why I am keen for this work to go in. Hive 2 and 3 are EOL, but still being us

Re: [I] Issue with kerberized hive metastore [iceberg-python]

kris-gaudel commented on issue #2032: URL: https://github.com/apache/iceberg-python/issues/2032#issuecomment-2997276931 I'm interested in working on this issue if there's no takers! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

Re: [PR] Further refactor Parquet readers for v2 support [iceberg]

eric-maynard commented on PR #13290: URL: https://github.com/apache/iceberg/pull/13290#issuecomment-2997253263 Thanks @huaxingao! I've added Javadocs About the scary diff @RussellSpitzer, it should be fixed but unfortunately I can't remove the tags which got auto-added when the diff w

  1   2   >