Re: [PR] Add iceberg_arrow library [iceberg-cpp]

2024-12-25 Thread via GitHub
wgtmac commented on code in PR #6: URL: https://github.com/apache/iceberg-cpp/pull/6#discussion_r1897581776 ## cmake_modules/ThirdpartyToolchain.cmake: ## @@ -0,0 +1,139 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

Re: [PR] Add iceberg_arrow library [iceberg-cpp]

2024-12-25 Thread via GitHub
wgtmac commented on code in PR #6: URL: https://github.com/apache/iceberg-cpp/pull/6#discussion_r1897581776 ## cmake_modules/ThirdpartyToolchain.cmake: ## @@ -0,0 +1,139 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

Re: [PR] Add iceberg_arrow library [iceberg-cpp]

2024-12-25 Thread via GitHub
lidavidm commented on code in PR #6: URL: https://github.com/apache/iceberg-cpp/pull/6#discussion_r1897591862 ## src/CMakeLists.txt: ## @@ -15,5 +15,20 @@ # specific language governing permissions and limitations # under the License. +add_subdirectory(arrow) add_subdirector

Re: [PR] Add iceberg_arrow library [iceberg-cpp]

2024-12-25 Thread via GitHub
wgtmac commented on code in PR #6: URL: https://github.com/apache/iceberg-cpp/pull/6#discussion_r1897586676 ## src/CMakeLists.txt: ## @@ -15,5 +15,20 @@ # specific language governing permissions and limitations # under the License. +add_subdirectory(arrow) add_subdirectory(

[I] CatalogUtil:dropTableData method doesn't remove old Puffin files [iceberg]

2024-12-25 Thread via GitHub
ebyhr opened a new issue, #11876: URL: https://github.com/apache/iceberg/issues/11876 ### Apache Iceberg version 1.7.1 (latest release) ### Query engine Trino ### Please describe the bug 🐞 https://github.com/apache/iceberg/pull/9305 and https://github.com/a

Re: [I] Support Adding File Metadata Directly [iceberg-python]

2024-12-25 Thread via GitHub
Fokko commented on issue #1470: URL: https://github.com/apache/iceberg-python/issues/1470#issuecomment-2562201472 Hey @subthedubdub Thanks for reaching out here. We allow appending data files directly: ```python with tbl.transaction() as txn: with txn.update_snapshot() as s

Re: [PR] Add iceberg_arrow library [iceberg-cpp]

2024-12-25 Thread via GitHub
wgtmac commented on code in PR #6: URL: https://github.com/apache/iceberg-cpp/pull/6#discussion_r1897676702 ## src/CMakeLists.txt: ## @@ -15,5 +15,20 @@ # specific language governing permissions and limitations # under the License. +add_subdirectory(arrow) add_subdirectory(

[PR] feat: Expose disable_config_load opendal GCS option [iceberg-rust]

2024-12-25 Thread via GitHub
chenzl25 opened a new pull request, #847: URL: https://github.com/apache/iceberg-rust/pull/847 - Expose `gcs.disable-config-load`, `gcs.disable-vm-metadata` and `gcs.allow-anonymous` for GCS. -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] API: Replace deprecated asList with asInstanceOf in TestExceptionUtil [iceberg]

2024-12-25 Thread via GitHub
Fokko merged PR #11875: URL: https://github.com/apache/iceberg/pull/11875 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] feat: Expose disable_config_load opendal GCS option [iceberg-rust]

2024-12-25 Thread via GitHub
Xuanwo commented on code in PR #847: URL: https://github.com/apache/iceberg-rust/pull/847#discussion_r1897698605 ## crates/iceberg/src/io/storage_gcs.rs: ## @@ -41,6 +41,17 @@ pub const GCS_CREDENTIALS_JSON: &str = "gcs.credentials-json"; /// Google Cloud Storage token pub co

[PR] Build: Fix ignoring `license-check.yml` in PR [iceberg]

2024-12-25 Thread via GitHub
manuzhang opened a new pull request, #11873: URL: https://github.com/apache/iceberg/pull/11873 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[PR] API: Replace deprecated asList with asInstanceOf in TestExceptionUtil [iceberg]

2024-12-25 Thread via GitHub
ebyhr opened a new pull request, #11875: URL: https://github.com/apache/iceberg/pull/11875 `asList` method is deprecated: https://www.javadoc.io/doc/org.assertj/assertj-core/latest/org/assertj/core/api/AbstractAssert.html#asList() -- This is an automated message from the Apache Git Servic

Re: [PR] Spark 3.5: Avoid deprecated method [iceberg]

2024-12-25 Thread via GitHub
ebyhr commented on code in PR #11874: URL: https://github.com/apache/iceberg/pull/11874#discussion_r1897321567 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestSparkCatalogOperations.java: ## @@ -107,8 +107,8 @@ public void testAlterTable() throws NoSuchTableExcept

Re: [PR] Spark 3.5: Avoid deprecated method [iceberg]

2024-12-25 Thread via GitHub
ebyhr commented on code in PR #11874: URL: https://github.com/apache/iceberg/pull/11874#discussion_r1897322776 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestCompressionSettings.java: ## @@ -251,7 +251,8 @@ private String getCompressionType(InputFile input

Re: [PR] Spark 3.5: Avoid deprecated method [iceberg]

2024-12-25 Thread via GitHub
ebyhr commented on code in PR #11874: URL: https://github.com/apache/iceberg/pull/11874#discussion_r1897323379 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/data/ParquetWithSparkSchemaVisitor.java: ## @@ -59,106 +59,101 @@ public static T visit(DataType sType, Type

Re: [PR] Spark 3.5: Avoid deprecated method [iceberg]

2024-12-25 Thread via GitHub
ebyhr commented on code in PR #11874: URL: https://github.com/apache/iceberg/pull/11874#discussion_r1897322776 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestCompressionSettings.java: ## @@ -251,7 +251,8 @@ private String getCompressionType(InputFile input

Re: [PR] Spark 3.5: Avoid deprecated method [iceberg]

2024-12-25 Thread via GitHub
ebyhr commented on code in PR #11874: URL: https://github.com/apache/iceberg/pull/11874#discussion_r1897322861 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestSparkCatalogOperations.java: ## @@ -107,8 +107,8 @@ public void testAlterTable() throws NoSuchTableExcept

[PR] Spark 3.5: Avoid deprecated method [iceberg]

2024-12-25 Thread via GitHub
ebyhr opened a new pull request, #11874: URL: https://github.com/apache/iceberg/pull/11874 I would recommend reviewing `ParquetWithSparkSchemaVisitor.java` with the hiding whitespace option. I changed `switch` to `if` because `LogicalTypeAnnotation` isn't enum. -- This is an automated m

Re: [I] Flaky test `TestCopyOnWriteDelete > testDeleteWithSnapshotIsolation()` [iceberg]

2024-12-25 Thread via GitHub
ebyhr commented on issue #11651: URL: https://github.com/apache/iceberg/issues/11651#issuecomment-2561844859 https://github.com/apache/iceberg/actions/runs/12491852295/job/34858377591?pr=11873 -- This is an automated message from the Apache Git Service. To respond to the message, please l

Re: [PR] Build: Fix ignoring `license-check.yml` in PR [iceberg]

2024-12-25 Thread via GitHub
ebyhr commented on PR #11873: URL: https://github.com/apache/iceberg/pull/11873#issuecomment-2561844827 CI hit #11651 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] chore: Use nightly toolchain for check [iceberg-rust]

2024-12-25 Thread via GitHub
xxchan commented on code in PR #445: URL: https://github.com/apache/iceberg-rust/pull/445#discussion_r1897190259 ## .github/workflows/ci.yml: ## @@ -63,8 +72,14 @@ jobs: - windows-latest steps: - uses: actions/checkout@v4 + + - name: Setup Rust toolch

Re: [PR] chore: Use nightly toolchain for check [iceberg-rust]

2024-12-25 Thread via GitHub
xxchan commented on code in PR #445: URL: https://github.com/apache/iceberg-rust/pull/445#discussion_r1897190743 ## .github/workflows/ci.yml: ## @@ -63,8 +72,14 @@ jobs: - windows-latest steps: - uses: actions/checkout@v4 + + - name: Setup Rust toolch

Re: [PR] Core: Use FileIO for hadoop table metadata file operations [iceberg]

2024-12-25 Thread via GitHub
IzzelAliz commented on code in PR #11690: URL: https://github.com/apache/iceberg/pull/11690#discussion_r1897338510 ## core/src/main/java/org/apache/iceberg/hadoop/HadoopTableOperations.java: ## @@ -237,15 +237,15 @@ Path getMetadataFile(int metadataVersion) throws IOException {

Re: [PR] fix: support both gs and gcs schemes for google cloud storage [iceberg-rust]

2024-12-25 Thread via GitHub
Xuanwo merged PR #845: URL: https://github.com/apache/iceberg-rust/pull/845 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] Rest catalog integration testing [iceberg-python]

2024-12-25 Thread via GitHub
AhmedNader42 commented on PR #1469: URL: https://github.com/apache/iceberg-python/pull/1469#issuecomment-2561825232 I had a look at the failing tests, and it appears the issue is in the sigv4 authentication option. But I can't reproduce it on local. Any ideas where to proceed from he

[PR] feat: Support metadata table "Metadata Log Entries" [iceberg-rust]

2024-12-25 Thread via GitHub
rshkv opened a new pull request, #846: URL: https://github.com/apache/iceberg-rust/pull/846 Re #823. This adds support for the [Metadata Log Entries][Java] metadata table. This is building on @xxchan's unmerged #822. I'll update and rebase this PR when #822 merges. Metadata Lo

Re: [I] Flink: IcebergTableSink to write data into multiple iceberg tables [iceberg]

2024-12-25 Thread via GitHub
thuongle2210 commented on issue #2208: URL: https://github.com/apache/iceberg/issues/2208#issuecomment-2561938516 > @caseylucas keyBy can help address some of the problems. not sure about your datasets. just watch out the traffic imbalance for the keyBy shuffle. How did you ensure Fli

Re: [I] Flink: IcebergTableSink to write data into multiple iceberg tables [iceberg]

2024-12-25 Thread via GitHub
thuongle2210 commented on issue #2208: URL: https://github.com/apache/iceberg/issues/2208#issuecomment-2561938944 > Currently, Flink Iceberg sink can only write to one table with exactly-once semantic. > > If the table partition doesn't fit your requirements, maybe you can use filter

Re: [I] Flink: IcebergTableSink to write data into multiple iceberg tables [iceberg]

2024-12-25 Thread via GitHub
thuongle2210 commented on issue #2208: URL: https://github.com/apache/iceberg/issues/2208#issuecomment-2561938744 > Currently, Flink Iceberg sink can only write to one table with exactly-once semantic. > > If the table partition doesn't fit your requirements, maybe you can use filter

Re: [PR] feat: Support metadata table "Metadata Log Entries" [iceberg-rust]

2024-12-25 Thread via GitHub
rshkv commented on code in PR #846: URL: https://github.com/apache/iceberg-rust/pull/846#discussion_r1897369513 ## crates/iceberg/src/metadata_scan.rs: ## @@ -0,0 +1,404 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

Re: [PR] feat: Support metadata table "Metadata Log Entries" [iceberg-rust]

2024-12-25 Thread via GitHub
rshkv commented on code in PR #846: URL: https://github.com/apache/iceberg-rust/pull/846#discussion_r1897371026 ## crates/iceberg/src/metadata_scan.rs: ## @@ -0,0 +1,404 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

Re: [PR] Build: Fix ignoring `license-check.yml` in PR [iceberg]

2024-12-25 Thread via GitHub
Fokko merged PR #11873: URL: https://github.com/apache/iceberg/pull/11873 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] feat: Support metadata table "Metadata Log Entries" [iceberg-rust]

2024-12-25 Thread via GitHub
rshkv commented on code in PR #846: URL: https://github.com/apache/iceberg-rust/pull/846#discussion_r1897375353 ## crates/iceberg/src/metadata_scan.rs: ## @@ -0,0 +1,407 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

Re: [PR] feat: Support metadata table "Metadata Log Entries" [iceberg-rust]

2024-12-25 Thread via GitHub
rshkv commented on code in PR #846: URL: https://github.com/apache/iceberg-rust/pull/846#discussion_r1897375800 ## crates/iceberg/src/scan.rs: ## @@ -1019,7 +1022,7 @@ mod tests { .metadata(table_metadata) .identifier(TableIdent::from_strs(["db"

Re: [PR] Core: Replace deprecated Schema.toString with SchemaFormatter [iceberg]

2024-12-25 Thread via GitHub
Fokko merged PR #11867: URL: https://github.com/apache/iceberg/pull/11867 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] feat: support metadata table "snapshots" [iceberg-rust]

2024-12-25 Thread via GitHub
rshkv commented on code in PR #822: URL: https://github.com/apache/iceberg-rust/pull/822#discussion_r1897394640 ## crates/iceberg/src/metadata_scan.rs: ## @@ -0,0 +1,265 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

Re: [PR] feat: support metadata table "snapshots" [iceberg-rust]

2024-12-25 Thread via GitHub
rshkv commented on code in PR #822: URL: https://github.com/apache/iceberg-rust/pull/822#discussion_r1897394640 ## crates/iceberg/src/metadata_scan.rs: ## @@ -0,0 +1,265 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

Re: [I] Can't add column with the same name as a deleted and previously partitioned key column [iceberg]

2024-12-25 Thread via GitHub
github-actions[bot] commented on issue #10487: URL: https://github.com/apache/iceberg/issues/10487#issuecomment-2562033204 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] Java visibility issue for `DataStatisticsSerializer` [iceberg]

2024-12-25 Thread via GitHub
github-actions[bot] commented on issue #10588: URL: https://github.com/apache/iceberg/issues/10588#issuecomment-2562033258 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] Java visibility issue for `TableScanContext` [iceberg]

2024-12-25 Thread via GitHub
github-actions[bot] commented on issue #10589: URL: https://github.com/apache/iceberg/issues/10589#issuecomment-2562033278 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] Java visibility issue for `org.apache.iceberg.ManifestEntry` [iceberg]

2024-12-25 Thread via GitHub
github-actions[bot] commented on issue #10585: URL: https://github.com/apache/iceberg/issues/10585#issuecomment-2562033228 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] Potential NPE in `o.a.i.orc.OrcValueReaders.StructReader#readInternal` [iceberg]

2024-12-25 Thread via GitHub
github-actions[bot] commented on issue #10594: URL: https://github.com/apache/iceberg/issues/10594#issuecomment-2562033299 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] Java visibility issue for `org.apache.iceberg.dell.ecs.EcsURI` [iceberg]

2024-12-25 Thread via GitHub
github-actions[bot] commented on issue #10587: URL: https://github.com/apache/iceberg/issues/10587#issuecomment-2562033243 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] Java visibility issue for `org.apache.iceberg.expressions.BoundAggregate.Aggregator` [iceberg]

2024-12-25 Thread via GitHub
github-actions[bot] commented on issue #10584: URL: https://github.com/apache/iceberg/issues/10584#issuecomment-2562033223 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] Can't add column with the same name as a deleted and previously partitioned key column [iceberg]

2024-12-25 Thread via GitHub
github-actions[bot] closed issue #10487: Can't add column with the same name as a deleted and previously partitioned key column URL: https://github.com/apache/iceberg/issues/10487 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [I] `o.a.i.util.Tasks.Builder.runSingleThreaded` possbily broken [iceberg]

2024-12-25 Thread via GitHub
github-actions[bot] commented on issue #10597: URL: https://github.com/apache/iceberg/issues/10597#issuecomment-2562033309 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] Java visibility issue for `SnowflakeTableMetadata` and `SnowflakeIdentifier` [iceberg]

2024-12-25 Thread via GitHub
github-actions[bot] commented on issue #10586: URL: https://github.com/apache/iceberg/issues/10586#issuecomment-2562033237 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Core: Add support for `view-default` property in catalog [iceberg]

2024-12-25 Thread via GitHub
github-actions[bot] commented on PR #11064: URL: https://github.com/apache/iceberg/pull/11064#issuecomment-2562033317 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] Potential bug in `o.a.i.mapping.MappingUtil.UpdateMapping.addNewFields()` [iceberg]

2024-12-25 Thread via GitHub
github-actions[bot] commented on issue #10596: URL: https://github.com/apache/iceberg/issues/10596#issuecomment-2562033304 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] Java visibility issues in `o.a.i.flink.sink.shuffle.AggregatedStatisticsSerializer` [iceberg]

2024-12-25 Thread via GitHub
github-actions[bot] commented on issue #10590: URL: https://github.com/apache/iceberg/issues/10590#issuecomment-2562033291 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Core: Add support for `view-default` property in catalog [iceberg]

2024-12-25 Thread via GitHub
ebyhr commented on PR #11064: URL: https://github.com/apache/iceberg/pull/11064#issuecomment-2562034661 Rebased on main branch to resolve conflicts. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t