[PR] [1.7.x] AWS, Core, GCP: Support relative credential endpoint / pass OAuth2 token to credential provider [iceberg]

2025-01-14 Thread via GitHub
nastra opened a new pull request, #11971: URL: https://github.com/apache/iceberg/pull/11971 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

Re: [PR] Build: Bump openapi-generator plugin from 6.6.0 to 7.10.0 [iceberg]

2025-01-14 Thread via GitHub
Fokko commented on PR #11970: URL: https://github.com/apache/iceberg/pull/11970#issuecomment-2591832013 Thanks @nastra We should probably run the Java tests also when making changes to `open-api/*` -- This is an automated message from the Apache Git Service. To respond to the message, ple

Re: [PR] Revert "Open-API: Bump to OpenAPI 3.1" [iceberg]

2025-01-14 Thread via GitHub
nastra closed pull request #11969: Revert "Open-API: Bump to OpenAPI 3.1" URL: https://github.com/apache/iceberg/pull/11969 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

Re: [PR] Revert "Open-API: Bump to OpenAPI 3.1" [iceberg]

2025-01-14 Thread via GitHub
nastra commented on PR #11969: URL: https://github.com/apache/iceberg/pull/11969#issuecomment-2591828590 Revert isn't needed anymore as this is fixed by https://github.com/apache/iceberg/pull/11970 -- This is an automated message from the Apache Git Service. To respond to the message, ple

Re: [I] Bucket name getting appended to minIO service name [iceberg-python]

2025-01-14 Thread via GitHub
ArijitSinghEDA closed issue #908: Bucket name getting appended to minIO service name URL: https://github.com/apache/iceberg-python/issues/908 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] ci: check MSRV correctly [iceberg-rust]

2025-01-14 Thread via GitHub
xxchan commented on PR #849: URL: https://github.com/apache/iceberg-rust/pull/849#issuecomment-2591790418 @Xuanwo CI is green now! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] AWS, Core, GCP: Support relative credential endpoint / pass OAuth2 token to credential provider [iceberg]

2025-01-14 Thread via GitHub
nastra closed pull request #11954: AWS, Core, GCP: Support relative credential endpoint / pass OAuth2 token to credential provider URL: https://github.com/apache/iceberg/pull/11954 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [I] Creating Delete Vectors using Java API or Spark [iceberg]

2025-01-14 Thread via GitHub
nastra commented on issue #11968: URL: https://github.com/apache/iceberg/issues/11968#issuecomment-2591782721 Not sure where you got the `write.delete.vector.enabled` property from but that property doesn't exist. The DV code hasn't been officially released yet. You could either wait for

Re: [I] Partition writes not creating expected directory hierarchy on S3 (MinIO) [iceberg-rust]

2025-01-14 Thread via GitHub
hackintoshrao commented on issue #891: URL: https://github.com/apache/iceberg-rust/issues/891#issuecomment-2591778008 Thank you, @ZENOTME, for the response. I work with a startup building a distributed query engine for large-scale Iceberg tables (>1PB). The `partition-specific` hiera

Re: [PR] feat(catalog): Standardize Catalog create table function [iceberg-go]

2025-01-14 Thread via GitHub
Fokko commented on PR #245: URL: https://github.com/apache/iceberg-go/pull/245#issuecomment-2591769403 Thanks @zeroshade 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] feat(catalog): Standardize Catalog create table function [iceberg-go]

2025-01-14 Thread via GitHub
Fokko merged PR #245: URL: https://github.com/apache/iceberg-go/pull/245 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] feat(catalog): Standardize Catalog create table function [iceberg-go]

2025-01-14 Thread via GitHub
Fokko commented on code in PR #245: URL: https://github.com/apache/iceberg-go/pull/245#discussion_r1916018700 ## table/metadata.go: ## @@ -1029,3 +1030,76 @@ func (m *metadataV2) UnmarshalJSON(b []byte) error { m.preValidate() return m.validate() } + +const Defa

Re: [PR] AWS, Core, GCP: Support relative credential endpoint / pass OAuth2 token to credential provider [iceberg]

2025-01-14 Thread via GitHub
nastra closed pull request #11954: AWS, Core, GCP: Support relative credential endpoint / pass OAuth2 token to credential provider URL: https://github.com/apache/iceberg/pull/11954 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
HonahX commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1916016403 ## core/src/test/java/org/apache/iceberg/TestTableMetadata.java: ## @@ -314,31 +328,43 @@ public void testBackwardCompat() throws Exception { assertThat(metadata.

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
HonahX commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1916012887 ## core/src/test/java/org/apache/iceberg/TestTableMetadata.java: ## @@ -160,31 +177,26 @@ public void testJsonConversion() throws Exception { .build()

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
HonahX commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1916012081 ## core/src/test/java/org/apache/iceberg/TestTableMetadata.java: ## @@ -96,31 +97,47 @@ public class TestTableMetadata { public TableOperations ops = new LocalTab

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
HonahX commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1916011232 ## core/src/test/java/org/apache/iceberg/TestTableMetadata.java: ## @@ -96,31 +97,47 @@ public class TestTableMetadata { public TableOperations ops = new LocalTab

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
HonahX commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1916010795 ## core/src/test/java/org/apache/iceberg/TestMetadataUpdateParser.java: ## @@ -364,15 +364,15 @@ public void testAddSnapshotToJson() throws IOException { String m

Re: [PR] Doc: Add DELETE ORPHAN-FILES example to Hive page [iceberg]

2025-01-14 Thread via GitHub
ebyhr commented on PR #11896: URL: https://github.com/apache/iceberg/pull/11896#issuecomment-2591729549 @pvary Can you take a look at this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Docs: add apache amoro(incubating) with iceberg (#11965) [iceberg]

2025-01-14 Thread via GitHub
ebyhr commented on code in PR #11966: URL: https://github.com/apache/iceberg/pull/11966#discussion_r1915973689 ## docs/docs/amoro.md: ## @@ -0,0 +1,90 @@ +--- +title: "Apache Amoro" +--- + + +# Apache Amoro With Iceberg + +**[Apache Amoro(incubating)](https://amoro.apache.org/do

Re: [PR] Spec: add variant type [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #10831: URL: https://github.com/apache/iceberg/pull/10831#discussion_r1915973279 ## format/spec.md: ## @@ -182,6 +182,21 @@ A **`list`** is a collection of values with some element type. The element field A **`map`** is a collection of

Re: [PR] ci: check MSRV correctly [iceberg-rust]

2025-01-14 Thread via GitHub
Xuanwo commented on PR #849: URL: https://github.com/apache/iceberg-rust/pull/849#issuecomment-2591683936 > If the current way looks good, I can try to fix compilation later. Current way looks really great to me. -- This is an automated message from the Apache Git Service. To respon

[I] Creating Delete Vectors using Java API or Spark [iceberg]

2025-01-14 Thread via GitHub
piyushdubey opened a new issue, #11968: URL: https://github.com/apache/iceberg/issues/11968 ### Query engine Spark ### Question Q1 - Is it possible to create Deletion Vectors in Apache Iceberg? Is a Deletion Vector file generated for a specific Deletion type Positional v

Re: [PR] Spark 3.5: Fix empty scan issue when start timestamp retrieves root snapshot and end timestamp is missing [iceberg]

2025-01-14 Thread via GitHub
lliangyu-lin commented on PR #11967: URL: https://github.com/apache/iceberg/pull/11967#issuecomment-2591666267 Thanks @manuzhang for the review. cc: @flyrain @Fokko May I get some help on additional reviews? -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] ci: check MSRV correctly [iceberg-rust]

2025-01-14 Thread via GitHub
xxchan commented on PR #849: URL: https://github.com/apache/iceberg-rust/pull/849#issuecomment-2591666123 https://github.com/user-attachments/assets/15d7dcad-8de5-472b-ab3a-1db9e1d35690"; /> Oops, it still doesn't compile. I guess it's because some dependencies' failed to specify cor

Re: [PR] feat(catalog): Add Catalog Registry [iceberg-go]

2025-01-14 Thread via GitHub
kevinjqliu commented on code in PR #244: URL: https://github.com/apache/iceberg-go/pull/244#discussion_r1915921905 ## catalog/registry.go: ## @@ -0,0 +1,135 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOT

Re: [I] Does Iceberg have a command similar to Hive MSCK to repair tables? [iceberg]

2025-01-14 Thread via GitHub
SGITLOGIN commented on issue #11961: URL: https://github.com/apache/iceberg/issues/11961#issuecomment-2591579937 Hello,I found the solution, which can be achieved through register_table -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [I] Does Iceberg have a command similar to Hive MSCK to repair tables? [iceberg]

2025-01-14 Thread via GitHub
SGITLOGIN closed issue #11961: Does Iceberg have a command similar to Hive MSCK to repair tables? URL: https://github.com/apache/iceberg/issues/11961 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] ci: check MSRV correctly [iceberg-rust]

2025-01-14 Thread via GitHub
Xuanwo commented on PR #849: URL: https://github.com/apache/iceberg-rust/pull/849#issuecomment-2591560448 Really appreciate your work here! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [I] [DISCUSSION] Project Goal [iceberg-cpp]

2025-01-14 Thread via GitHub
wgtmac commented on issue #2: URL: https://github.com/apache/iceberg-cpp/issues/2#issuecomment-2591560268 > Is it possible to allow users to customize and choose their own parser (like options) or use Arrow for parsing? Because I want a cleaner dependency. You mean the JSON parser?

Re: [PR] Docs: add apache amoro(incubating) with iceberg (#11965) [iceberg]

2025-01-14 Thread via GitHub
XBaith commented on code in PR #11966: URL: https://github.com/apache/iceberg/pull/11966#discussion_r1915887251 ## docs/docs/amoro.md: ## @@ -0,0 +1,90 @@ +--- +title: "Apache Amoro" +--- + + +# Apache Amoro With Iceberg + +**[Apache Amoro(incubating)](https://amoro.apache.org/d

Re: [PR] Docs: add apache amoro(incubating) with iceberg (#11965) [iceberg]

2025-01-14 Thread via GitHub
XBaith commented on code in PR #11966: URL: https://github.com/apache/iceberg/pull/11966#discussion_r1915882328 ## docs/docs/amoro.md: ## @@ -0,0 +1,90 @@ +--- +title: "Apache Amoro" +--- + + +# Apache Amoro With Iceberg + +**[Apache Amoro(incubating)](https://amoro.apache.org/d

Re: [I] [DISCUSSION] Project Goal [iceberg-cpp]

2025-01-14 Thread via GitHub
MisterRaindrop commented on issue #2: URL: https://github.com/apache/iceberg-cpp/issues/2#issuecomment-2591542225 @wgtmac I’m quite curious. I only saw the reading part. Will there be writing designs? -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [I] Does Iceberg have a command similar to Hive MSCK to repair tables? [iceberg]

2025-01-14 Thread via GitHub
SGITLOGIN commented on issue #11961: URL: https://github.com/apache/iceberg/issues/11961#issuecomment-2591465958 > > I create the same table again. How can I restore this part of the data? > > What do you mean by `this part of the data`? Which catalog are you using? @manuzhang H

Re: [PR] Core: Fix numeric overflow of timestamp nano literal [iceberg]

2025-01-14 Thread via GitHub
ebyhr commented on code in PR #11775: URL: https://github.com/apache/iceberg/pull/11775#discussion_r1915834371 ## api/src/main/java/org/apache/iceberg/expressions/Literals.java: ## @@ -300,8 +300,7 @@ public Literal to(Type type) { case TIMESTAMP: return (Li

Re: [PR] Spec: add variant type [iceberg]

2025-01-14 Thread via GitHub
aihuaxu commented on code in PR #10831: URL: https://github.com/apache/iceberg/pull/10831#discussion_r1915818592 ## format/spec.md: ## @@ -182,6 +182,21 @@ A **`list`** is a collection of values with some element type. The element field A **`map`** is a collection of key-val

Re: [PR] Spec: Document Snapshot Summary Optional Fields for Standardization [iceberg]

2025-01-14 Thread via GitHub
manuzhang commented on code in PR #11660: URL: https://github.com/apache/iceberg/pull/11660#discussion_r1915810877 ## format/spec.md: ## @@ -1633,3 +1633,50 @@ might indicate different snapshot IDs for a specific timestamp. The discrepancie When processing point in time quer

Re: [PR] Spec: Document Snapshot Summary Optional Fields for Standardization [iceberg]

2025-01-14 Thread via GitHub
rdblue commented on code in PR #11660: URL: https://github.com/apache/iceberg/pull/11660#discussion_r1915780926 ## format/spec.md: ## @@ -1633,3 +1633,50 @@ might indicate different snapshot IDs for a specific timestamp. The discrepancie When processing point in time queries

Re: [PR] feat(catalog): Add Catalog Registry [iceberg-go]

2025-01-14 Thread via GitHub
zeroshade commented on code in PR #244: URL: https://github.com/apache/iceberg-go/pull/244#discussion_r1915663329 ## catalog/registry.go: ## @@ -0,0 +1,135 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTI

Re: [PR] Support WASB scheme in ADLSFileIO [iceberg]

2025-01-14 Thread via GitHub
amogh-jahagirdar commented on code in PR #11830: URL: https://github.com/apache/iceberg/pull/11830#discussion_r1915560522 ## azure/src/test/java/org/apache/iceberg/azure/adlsv2/ADLSLocationTest.java: ## @@ -89,4 +101,17 @@ public void testQuestionMarkInFileName(String path) {

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915587999 ## core/src/test/java/org/apache/iceberg/TestTableMetadata.java: ## @@ -357,43 +383,48 @@ public void testInvalidMainBranch() throws IOException { asser

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915586726 ## core/src/test/java/org/apache/iceberg/TestTableMetadata.java: ## @@ -314,31 +328,43 @@ public void testBackwardCompat() throws Exception { assertThat(m

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915570223 ## core/src/test/java/org/apache/iceberg/TestTableMetadata.java: ## @@ -160,31 +177,26 @@ public void testJsonConversion() throws Exception {

Re: [PR] Spec: Support geo type [iceberg]

2025-01-14 Thread via GitHub
flyrain commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1915566336 ## format/spec.md: ## @@ -1633,3 +1683,8 @@ might indicate different snapshot IDs for a specific timestamp. The discrepancie When processing point in time queries

Re: [PR] [draft] AWS: support hadoop bulk delete API [iceberg]

2025-01-14 Thread via GitHub
steveloughran commented on PR #10233: URL: https://github.com/apache/iceberg/pull/10233#issuecomment-2590997379 For anyone watching this, there's a full integration test suite in the hadoop test code: https://github.com/apache/hadoop/pull/7285 All is good, though as it's the first jav

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915510199 ## core/src/test/java/org/apache/iceberg/TestSnapshotJson.java: ## @@ -42,8 +42,15 @@ public void testJsonConversion() throws IOException { String manifes

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915513619 ## core/src/test/java/org/apache/iceberg/TestTableMetadata.java: ## @@ -96,31 +97,47 @@ public class TestTableMetadata { public TableOperations ops = new

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915512319 ## core/src/test/java/org/apache/iceberg/TestTableMetadata.java: ## @@ -96,31 +97,47 @@ public class TestTableMetadata { public TableOperations ops = new

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915509758 ## core/src/test/java/org/apache/iceberg/TestSnapshotJson.java: ## @@ -61,8 +68,15 @@ public void testJsonConversionWithoutSchemaId() throws IOException {

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915508273 ## core/src/main/java/org/apache/iceberg/TableMetadataParser.java: ## @@ -352,6 +352,7 @@ public static TableMetadata fromJson(String metadataLocation, JsonNo

Re: [PR] Spec: Document Snapshot Summary Optional Fields for Standardization [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11660: URL: https://github.com/apache/iceberg/pull/11660#discussion_r1915505397 ## format/spec.md: ## @@ -1633,3 +1633,57 @@ might indicate different snapshot IDs for a specific timestamp. The discrepancie When processing point in time

Re: [PR] Spec: Document Snapshot Summary Optional Fields for Standardization [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on PR #11660: URL: https://github.com/apache/iceberg/pull/11660#issuecomment-2590959103 @HonahX I think we are good on this one, I don't think we had any negative feedback from the dev list. So I think we are good to merge if you are ready. -- This is an automated

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
HonahX commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915460216 ## core/src/main/java/org/apache/iceberg/TableMetadataParser.java: ## @@ -352,6 +352,7 @@ public static TableMetadata fromJson(String metadataLocation, JsonNode node)

Re: [PR] Spec: add variant type [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #10831: URL: https://github.com/apache/iceberg/pull/10831#discussion_r1915466398 ## format/spec.md: ## @@ -182,6 +182,21 @@ A **`list`** is a collection of values with some element type. The element field A **`map`** is a collection of

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
HonahX commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915460216 ## core/src/main/java/org/apache/iceberg/TableMetadataParser.java: ## @@ -352,6 +352,7 @@ public static TableMetadata fromJson(String metadataLocation, JsonNode node)

Re: [PR] Open-API: Bump to OpenAPI 3.1 [iceberg]

2025-01-14 Thread via GitHub
Fokko commented on PR #11955: URL: https://github.com/apache/iceberg/pull/11955#issuecomment-2590904626 Thanks @jbonofre, @nastra and @hussein-awala for the review 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Open-API: Bump to OpenAPI 3.1 [iceberg]

2025-01-14 Thread via GitHub
Fokko merged PR #11955: URL: https://github.com/apache/iceberg/pull/11955 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
HonahX commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915429803 ## core/src/test/java/org/apache/iceberg/TestMetadataUpdateParser.java: ## @@ -364,15 +364,15 @@ public void testAddSnapshotToJson() throws IOException { String m

Re: [PR] Support WASB scheme in ADLSFileIO [iceberg]

2025-01-14 Thread via GitHub
amogh-jahagirdar commented on code in PR #11830: URL: https://github.com/apache/iceberg/pull/11830#discussion_r1915391549 ## azure/src/test/java/org/apache/iceberg/azure/adlsv2/ADLSLocationTest.java: ## @@ -89,4 +101,17 @@ public void testQuestionMarkInFileName(String path) {

Re: [PR] Support WASB scheme in ADLSFileIO [iceberg]

2025-01-14 Thread via GitHub
amogh-jahagirdar commented on code in PR #11830: URL: https://github.com/apache/iceberg/pull/11830#discussion_r1915391549 ## azure/src/test/java/org/apache/iceberg/azure/adlsv2/ADLSLocationTest.java: ## @@ -89,4 +101,17 @@ public void testQuestionMarkInFileName(String path) {

Re: [PR] Support WASB scheme in ADLSFileIO [iceberg]

2025-01-14 Thread via GitHub
amogh-jahagirdar commented on code in PR #11830: URL: https://github.com/apache/iceberg/pull/11830#discussion_r1915391549 ## azure/src/test/java/org/apache/iceberg/azure/adlsv2/ADLSLocationTest.java: ## @@ -89,4 +101,17 @@ public void testQuestionMarkInFileName(String path) {

Re: [PR] Support WASB scheme in ADLSFileIO [iceberg]

2025-01-14 Thread via GitHub
amogh-jahagirdar commented on code in PR #11830: URL: https://github.com/apache/iceberg/pull/11830#discussion_r1915382736 ## azure/src/test/java/org/apache/iceberg/azure/adlsv2/ADLSLocationTest.java: ## @@ -89,4 +101,17 @@ public void testQuestionMarkInFileName(String path) {

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915381785 ## core/src/test/java/org/apache/iceberg/TestMetadataUpdateParser.java: ## @@ -364,15 +364,15 @@ public void testAddSnapshotToJson() throws IOException {

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915381785 ## core/src/test/java/org/apache/iceberg/TestMetadataUpdateParser.java: ## @@ -364,15 +364,15 @@ public void testAddSnapshotToJson() throws IOException {

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915347026 ## core/src/test/java/org/apache/iceberg/MetadataTestUtils.java: ## @@ -0,0 +1,336 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915373560 ## core/src/test/java/org/apache/iceberg/MetadataTestUtils.java: ## @@ -0,0 +1,336 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915353274 ## core/src/main/java/org/apache/iceberg/TableMetadataParser.java: ## @@ -352,6 +352,7 @@ public static TableMetadata fromJson(String metadataLocation, JsonNo

Re: [I] Getting all snapshots through table.inspect.snapshots() [iceberg-python]

2025-01-14 Thread via GitHub
kevinjqliu closed issue #1512: Getting all snapshots through table.inspect.snapshots() URL: https://github.com/apache/iceberg-python/issues/1512 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] Getting all snapshots through table.inspect.snapshots() [iceberg-python]

2025-01-14 Thread via GitHub
kevinjqliu commented on issue #1512: URL: https://github.com/apache/iceberg-python/issues/1512#issuecomment-2590718401 x-posting from the other issue (https://github.com/apache/iceberg-python/issues/1105#issuecomment-2588075721) > To retain Nessie’s consistency and cross-branch/tag i

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915347026 ## core/src/test/java/org/apache/iceberg/MetadataTestUtils.java: ## @@ -0,0 +1,336 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
HonahX commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915306849 ## core/src/test/java/org/apache/iceberg/MetadataTestUtils.java: ## @@ -0,0 +1,336 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more co

Re: [PR] Core, API, Spec: Metadata Row Lineage [iceberg]

2025-01-14 Thread via GitHub
HonahX commented on code in PR #11948: URL: https://github.com/apache/iceberg/pull/11948#discussion_r1915308572 ## core/src/test/java/org/apache/iceberg/TestSnapshotJson.java: ## @@ -43,7 +43,16 @@ public void testJsonConversion() throws IOException { Snapshot expected =

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
HonahX commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915306849 ## core/src/test/java/org/apache/iceberg/MetadataTestUtils.java: ## @@ -0,0 +1,336 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more co

Re: [PR] Core, API, Spec: Metadata Row Lineage [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11948: URL: https://github.com/apache/iceberg/pull/11948#discussion_r1915289834 ## core/src/test/java/org/apache/iceberg/TestSnapshotJson.java: ## @@ -43,7 +43,16 @@ public void testJsonConversion() throws IOException { Snapshot exp

Re: [PR] Metadata Row Lineage [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on PR #11948: URL: https://github.com/apache/iceberg/pull/11948#issuecomment-2590607674 @nastra Very interested on your thoughts on my test suite here. I'm thinking about how to get away from relying on the Junit4 Test wrapper annotation in TestBase. I know I did so

Re: [I] create_changelog_view returns no record when end-timestamp is missing [iceberg]

2025-01-14 Thread via GitHub
flyrain commented on issue #11922: URL: https://github.com/apache/iceberg/issues/11922#issuecomment-2590603660 Thanks @lliangyu-lin to take it. Let's aim for the release 1.7.2. cc @jbonofre -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [I] Flaky test `TestCopyOnWriteDelete > testDeleteWithSnapshotIsolation()` [iceberg]

2025-01-14 Thread via GitHub
hantangwangd commented on issue #11651: URL: https://github.com/apache/iceberg/issues/11651#issuecomment-2590588696 +1, see: https://github.com/apache/iceberg/actions/runs/12769765775/job/35593064023?pr=11901 -- This is an automated message from the Apache Git Service. To respond to the m

Re: [PR] Azure: Support vended credentials refresh in ADLSFileIO. [iceberg]

2025-01-14 Thread via GitHub
danielcweeks commented on code in PR #11577: URL: https://github.com/apache/iceberg/pull/11577#discussion_r1915252528 ## azure/src/test/java/org/apache/iceberg/azure/adlsv2/VendedAdlsCredentialProviderTest.java: ## @@ -0,0 +1,315 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] OpenAPI: Changes for freshness-aware table loading [iceberg]

2025-01-14 Thread via GitHub
danielcweeks commented on code in PR #11946: URL: https://github.com/apache/iceberg/pull/11946#discussion_r1915221784 ## open-api/rest-catalog-open-api.yaml: ## @@ -930,6 +930,14 @@ paths: key. For example, "urn:ietf:params:oauth:token-type:jwt=". parameters:

Re: [PR] Spark 3.4: Add view support to SparkSessionCatalog [iceberg]

2025-01-14 Thread via GitHub
danielcweeks commented on PR #11797: URL: https://github.com/apache/iceberg/pull/11797#issuecomment-2590492899 @nastra can we get this rebased? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Core: Fix loading a table in CachingCatalog with metadata table name [iceberg]

2025-01-14 Thread via GitHub
nastra merged PR #11738: URL: https://github.com/apache/iceberg/pull/11738 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Core: Fix loading a table in CachingCatalog with metadata table name [iceberg]

2025-01-14 Thread via GitHub
nastra commented on PR #11738: URL: https://github.com/apache/iceberg/pull/11738#issuecomment-2590458971 merging as the CI failure isn't related to the changes being proposed here -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [I] How to move Iceberg table from one location to another [iceberg]

2025-01-14 Thread via GitHub
manuzhang closed issue #11645: How to move Iceberg table from one location to another URL: https://github.com/apache/iceberg/issues/11645 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] Why the metric values are all N/A on the spark webui? [iceberg]

2025-01-14 Thread via GitHub
manuzhang closed issue #11794: Why the metric values are all N/A on the spark webui? URL: https://github.com/apache/iceberg/issues/11794 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Docs: add apache amoro(incubating) with iceberg (#11965) [iceberg]

2025-01-14 Thread via GitHub
nastra commented on code in PR #11966: URL: https://github.com/apache/iceberg/pull/11966#discussion_r1915111401 ## docs/docs/amoro.md: ## @@ -0,0 +1,90 @@ +--- +title: "Apache Amoro" +--- + + +# Apache Amoro With Iceberg + +**[Apache Amoro(incubating)](https://amoro.apache.org/d

Re: [PR] Docs: add apache amoro(incubating) with iceberg (#11965) [iceberg]

2025-01-14 Thread via GitHub
nastra commented on code in PR #11966: URL: https://github.com/apache/iceberg/pull/11966#discussion_r1915106856 ## docs/docs/amoro.md: ## @@ -0,0 +1,90 @@ +--- +title: "Apache Amoro" +--- + + +# Apache Amoro With Iceberg + +**[Apache Amoro(incubating)](https://amoro.apache.org/d

Re: [PR] Add ResidualVisitor to compute residuals [iceberg-python]

2025-01-14 Thread via GitHub
Fokko commented on code in PR #1388: URL: https://github.com/apache/iceberg-python/pull/1388#discussion_r1915102464 ## pyiceberg/table/__init__.py: ## @@ -1522,8 +1555,9 @@ def plan_files(self) -> Iterable[FileScanTask]: data_entry, posi

Re: [PR] AWS, Core, GCP: Support relative credential endpoint / pass OAuth2 token to credential provider [iceberg]

2025-01-14 Thread via GitHub
nastra commented on code in PR #11954: URL: https://github.com/apache/iceberg/pull/11954#discussion_r1915070130 ## core/src/main/java/org/apache/iceberg/rest/RESTUtil.java: ## @@ -215,4 +215,19 @@ public static Namespace decodeNamespace(String encodedNs) { return Namespac

Re: [PR] AWS, Core, GCP: Support relative credential endpoint / pass OAuth2 token to credential provider [iceberg]

2025-01-14 Thread via GitHub
nastra commented on code in PR #11954: URL: https://github.com/apache/iceberg/pull/11954#discussion_r1915069030 ## core/src/main/java/org/apache/iceberg/rest/RESTUtil.java: ## @@ -215,4 +215,19 @@ public static Namespace decodeNamespace(String encodedNs) { return Namespac

Re: [PR] Add ResidualVisitor to compute residuals [iceberg-python]

2025-01-14 Thread via GitHub
tusharchou commented on code in PR #1388: URL: https://github.com/apache/iceberg-python/pull/1388#discussion_r1915063325 ## pyiceberg/table/__init__.py: ## @@ -1522,8 +1555,9 @@ def plan_files(self) -> Iterable[FileScanTask]: data_entry,

Re: [PR] Fix CMake to support being a subproject [iceberg-cpp]

2025-01-14 Thread via GitHub
Xuanwo merged PR #29: URL: https://github.com/apache/iceberg-cpp/pull/29 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [I] [CMake] usage problem [iceberg-cpp]

2025-01-14 Thread via GitHub
Xuanwo closed issue #28: [CMake] usage problem URL: https://github.com/apache/iceberg-cpp/issues/28 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issu

Re: [PR] Metadata Row Lineage [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11948: URL: https://github.com/apache/iceberg/pull/11948#discussion_r1914954813 ## core/src/test/java/org/apache/iceberg/TestRowLineageMetadata.java: ## @@ -0,0 +1,213 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

Re: [PR] Fix CMake to support being a subproject [iceberg-cpp]

2025-01-14 Thread via GitHub
wgtmac commented on PR #29: URL: https://github.com/apache/iceberg-cpp/pull/29#issuecomment-2590100956 Thanks for the confirmation! @zhjwpku @Fokko @Xuanwo Could you please merge this? It is a minor fix. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Core: Fix loading a table in CachingCatalog with metadata table name [iceberg]

2025-01-14 Thread via GitHub
nastra commented on PR #11738: URL: https://github.com/apache/iceberg/pull/11738#issuecomment-2590097929 @gaborkaszab can you rebase please to fix CI? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Fix CMake to support being a subproject [iceberg-cpp]

2025-01-14 Thread via GitHub
wgtmac commented on PR #29: URL: https://github.com/apache/iceberg-cpp/pull/29#issuecomment-2590048249 @zhjwpku Could you please try this? ``` cmake_minimum_required(VERSION 3.25) project(dummy VERSION 0.1.0 LANGUAGES CXX) include(FetchContent)

Re: [I] Partition writes not creating expected directory hierarchy on S3 (MinIO) [iceberg-rust]

2025-01-14 Thread via GitHub
ZENOTME commented on issue #891: URL: https://github.com/apache/iceberg-rust/issues/891#issuecomment-2590004754 Thanks @hackintoshrao for pointing out this. I think the partition-specific directory is not necessary to partition write?🤔 But it indeed reminds me to think about how to support

Re: [I] [CMake] usage problem [iceberg-cpp]

2025-01-14 Thread via GitHub
wgtmac commented on issue #28: URL: https://github.com/apache/iceberg-cpp/issues/28#issuecomment-258768 Let me try :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] docs: update `README.md` fix brand name `macOS` [iceberg]

2025-01-14 Thread via GitHub
Fokko merged PR #11964: URL: https://github.com/apache/iceberg/pull/11964 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Add pyspark DataFusion integration test [iceberg-rust]

2025-01-14 Thread via GitHub
gruuya commented on code in PR #850: URL: https://github.com/apache/iceberg-rust/pull/850#discussion_r1914826504 ## crates/integration_tests/Cargo.toml: ## @@ -27,9 +27,11 @@ rust-version = { workspace = true } [dependencies] arrow-array = { workspace = true } arrow-schema =

  1   2   >