Re: [PR] Core: Fix test failure caused by check empty namespace on REST session catalog [iceberg]

2025-01-14 Thread via GitHub
hantangwangd commented on PR #11960: URL: https://github.com/apache/iceberg/pull/11960#issuecomment-2589346183 @nastra My pleasure! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] Flink: Maintenance - RewriteDataFiles [iceberg]

2025-01-14 Thread via GitHub
mxm commented on PR #11497: URL: https://github.com/apache/iceberg/pull/11497#issuecomment-2589350651 @pvary @stevenzwu Could we wrap this up? I don't see any fundamental issues here. The only open point from my perspective is table refresh during planning time, but even that could be handl

Re: [I] Javadoc issues [iceberg]

2025-01-14 Thread via GitHub
Fokko commented on issue #10378: URL: https://github.com/apache/iceberg/issues/10378#issuecomment-2589292476 Thanks @jbonofre I'm bumping this one to 1.8.0 because we can publish the docs without an actual release đź‘Ť This way the 1.7.2 release looks like it is ready to go. -- This is an a

Re: [I] Javadoc issues [iceberg]

2025-01-14 Thread via GitHub
jbonofre commented on issue #10378: URL: https://github.com/apache/iceberg/issues/10378#issuecomment-2589309002 @Fokko it sounds good. Agree. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Build: Bump mkdocs-autorefs from 1.2.0 to 1.3.0 [iceberg-python]

2025-01-14 Thread via GitHub
Fokko merged PR #1513: URL: https://github.com/apache/iceberg-python/pull/1513 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Metadata Row Lineage [iceberg]

2025-01-14 Thread via GitHub
pvary commented on code in PR #11948: URL: https://github.com/apache/iceberg/pull/11948#discussion_r1914481119 ## core/src/test/java/org/apache/iceberg/TestRowLineageMetadata.java: ## @@ -0,0 +1,213 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mor

Re: [PR] Flink: Maintenance - RewriteDataFiles [iceberg]

2025-01-14 Thread via GitHub
pvary commented on PR #11497: URL: https://github.com/apache/iceberg/pull/11497#issuecomment-2589410701 > @pvary @stevenzwu Could we wrap this up? I don't see any fundamental issues here. The only open point from my perspective is table refresh during planning time, but even that could be h

Re: [PR] Add Doxygen for generating API documentation [iceberg-cpp]

2025-01-14 Thread via GitHub
raulcd commented on code in PR #27: URL: https://github.com/apache/iceberg-cpp/pull/27#discussion_r1914657246 ## docs/README.md: ## @@ -0,0 +1,27 @@ + + +# Documentation + +To build the documentation: + +#. Install [Doxygen][doxygen]. Review Comment: Was this supposed to be

Re: [PR] ci: check MSRV correctly [iceberg-rust]

2025-01-14 Thread via GitHub
xxchan commented on PR #849: URL: https://github.com/apache/iceberg-rust/pull/849#issuecomment-2589822788 @Xuanwo I explored several ways to pin dependencies: 1. Modify `Cargo.lock` to get it work. - Drawback: Need quite some manual work. And perhaps we need to disable dependabot

Re: [PR] [docs] Update docs link about Apache Doris and update vendors list [iceberg]

2025-01-14 Thread via GitHub
nastra merged PR #11956: URL: https://github.com/apache/iceberg/pull/11956 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
nastra commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1914536361 ## core/src/test/java/org/apache/iceberg/MetadataTestUtils.java: ## @@ -0,0 +1,336 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more co

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
nastra commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1914536816 ## core/src/test/java/org/apache/iceberg/MetadataTestUtils.java: ## @@ -0,0 +1,336 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more co

Re: [PR] Spark 3.5: Add query runner in test module [iceberg]

2025-01-14 Thread via GitHub
ebyhr commented on PR #11758: URL: https://github.com/apache/iceberg/pull/11758#issuecomment-2589422855 Added a few small comments to the class. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] Core: Fix test failure caused by check empty namespace on REST session catalog [iceberg]

2025-01-14 Thread via GitHub
nastra merged PR #11960: URL: https://github.com/apache/iceberg/pull/11960 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Use SupportsPrefixOperations for Remove OrphanFile Procedure [iceberg]

2025-01-14 Thread via GitHub
ismailsimsek commented on code in PR #11906: URL: https://github.com/apache/iceberg/pull/11906#discussion_r1914531213 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/DeleteOrphanFilesSparkAction.java: ## @@ -292,19 +294,49 @@ private Dataset validFileIdentDS()

[I] [CMake] usage problem [iceberg-cpp]

2025-01-14 Thread via GitHub
zhjwpku opened a new issue, #28: URL: https://github.com/apache/iceberg-cpp/issues/28 I am testing using iceberg-cpp as a subproject using FetchContent(not install it first), here is the CMakeLists.txt ``` cmake_minimum_required(VERSION 3.25) project(dummy VERSION

Re: [PR] Kafka Connect: Add SMTs for Debezium and AWS DMS [iceberg]

2025-01-14 Thread via GitHub
ismailsimsek commented on code in PR #11936: URL: https://github.com/apache/iceberg/pull/11936#discussion_r1909022386 ## kafka-connect/build.gradle: ## @@ -242,3 +243,19 @@ project(':iceberg-kafka-connect:iceberg-kafka-connect-runtime') { assemble.dependsOn distZip, hiveDi

Re: [PR] docs: update `README.md` fix brand name `macOS` [iceberg]

2025-01-14 Thread via GitHub
Fokko merged PR #11964: URL: https://github.com/apache/iceberg/pull/11964 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [I] [CMake] usage problem [iceberg-cpp]

2025-01-14 Thread via GitHub
wgtmac commented on issue #28: URL: https://github.com/apache/iceberg-cpp/issues/28#issuecomment-258768 Let me try :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Core: Fix loading a table in CachingCatalog with metadata table name [iceberg]

2025-01-14 Thread via GitHub
nastra commented on PR #11738: URL: https://github.com/apache/iceberg/pull/11738#issuecomment-2590097929 @gaborkaszab can you rebase please to fix CI? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Fix CMake to support being a subproject [iceberg-cpp]

2025-01-14 Thread via GitHub
Xuanwo merged PR #29: URL: https://github.com/apache/iceberg-cpp/pull/29 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [I] [CMake] usage problem [iceberg-cpp]

2025-01-14 Thread via GitHub
Xuanwo closed issue #28: [CMake] usage problem URL: https://github.com/apache/iceberg-cpp/issues/28 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issu

Re: [PR] Spark 3.4: Add view support to SparkSessionCatalog [iceberg]

2025-01-14 Thread via GitHub
danielcweeks commented on PR #11797: URL: https://github.com/apache/iceberg/pull/11797#issuecomment-2590492899 @nastra can we get this rebased? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Core: Fix loading a table in CachingCatalog with metadata table name [iceberg]

2025-01-14 Thread via GitHub
nastra commented on PR #11738: URL: https://github.com/apache/iceberg/pull/11738#issuecomment-2590458971 merging as the CI failure isn't related to the changes being proposed here -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] Core: Fix loading a table in CachingCatalog with metadata table name [iceberg]

2025-01-14 Thread via GitHub
nastra merged PR #11738: URL: https://github.com/apache/iceberg/pull/11738 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] OpenAPI: Changes for freshness-aware table loading [iceberg]

2025-01-14 Thread via GitHub
danielcweeks commented on code in PR #11946: URL: https://github.com/apache/iceberg/pull/11946#discussion_r1915221784 ## open-api/rest-catalog-open-api.yaml: ## @@ -930,6 +930,14 @@ paths: key. For example, "urn:ietf:params:oauth:token-type:jwt=". parameters:

Re: [PR] Azure: Support vended credentials refresh in ADLSFileIO. [iceberg]

2025-01-14 Thread via GitHub
danielcweeks commented on code in PR #11577: URL: https://github.com/apache/iceberg/pull/11577#discussion_r1915252528 ## azure/src/test/java/org/apache/iceberg/azure/adlsv2/VendedAdlsCredentialProviderTest.java: ## @@ -0,0 +1,315 @@ +/* + * Licensed to the Apache Software Founda

Re: [I] Flaky test `TestCopyOnWriteDelete > testDeleteWithSnapshotIsolation()` [iceberg]

2025-01-14 Thread via GitHub
hantangwangd commented on issue #11651: URL: https://github.com/apache/iceberg/issues/11651#issuecomment-2590588696 +1, see: https://github.com/apache/iceberg/actions/runs/12769765775/job/35593064023?pr=11901 -- This is an automated message from the Apache Git Service. To respond to the m

Re: [I] create_changelog_view returns no record when end-timestamp is missing [iceberg]

2025-01-14 Thread via GitHub
flyrain commented on issue #11922: URL: https://github.com/apache/iceberg/issues/11922#issuecomment-2590603660 Thanks @lliangyu-lin to take it. Let's aim for the release 1.7.2. cc @jbonofre -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] Metadata Row Lineage [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on PR #11948: URL: https://github.com/apache/iceberg/pull/11948#issuecomment-2590607674 @nastra Very interested on your thoughts on my test suite here. I'm thinking about how to get away from relying on the Junit4 Test wrapper annotation in TestBase. I know I did so

Re: [PR] Add pyspark DataFusion integration test [iceberg-rust]

2025-01-14 Thread via GitHub
gruuya commented on code in PR #850: URL: https://github.com/apache/iceberg-rust/pull/850#discussion_r1914826504 ## crates/integration_tests/Cargo.toml: ## @@ -27,9 +27,11 @@ rust-version = { workspace = true } [dependencies] arrow-array = { workspace = true } arrow-schema =

Re: [PR] Fix CMake to support being a subproject [iceberg-cpp]

2025-01-14 Thread via GitHub
wgtmac commented on PR #29: URL: https://github.com/apache/iceberg-cpp/pull/29#issuecomment-2590048249 @zhjwpku Could you please try this? ``` cmake_minimum_required(VERSION 3.25) project(dummy VERSION 0.1.0 LANGUAGES CXX) include(FetchContent)

Re: [PR] Metadata Row Lineage [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11948: URL: https://github.com/apache/iceberg/pull/11948#discussion_r1914954813 ## core/src/test/java/org/apache/iceberg/TestRowLineageMetadata.java: ## @@ -0,0 +1,213 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

Re: [I] Partition writes not creating expected directory hierarchy on S3 (MinIO) [iceberg-rust]

2025-01-14 Thread via GitHub
ZENOTME commented on issue #891: URL: https://github.com/apache/iceberg-rust/issues/891#issuecomment-2590004754 Thanks @hackintoshrao for pointing out this. I think the partition-specific directory is not necessary to partition write?🤔 But it indeed reminds me to think about how to support

Re: [PR] Fix CMake to support being a subproject [iceberg-cpp]

2025-01-14 Thread via GitHub
wgtmac commented on PR #29: URL: https://github.com/apache/iceberg-cpp/pull/29#issuecomment-2590100956 Thanks for the confirmation! @zhjwpku @Fokko @Xuanwo Could you please merge this? It is a minor fix. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] AWS, Core, GCP: Support relative credential endpoint / pass OAuth2 token to credential provider [iceberg]

2025-01-14 Thread via GitHub
nastra commented on code in PR #11954: URL: https://github.com/apache/iceberg/pull/11954#discussion_r1915069030 ## core/src/main/java/org/apache/iceberg/rest/RESTUtil.java: ## @@ -215,4 +215,19 @@ public static Namespace decodeNamespace(String encodedNs) { return Namespac

Re: [PR] Add ResidualVisitor to compute residuals [iceberg-python]

2025-01-14 Thread via GitHub
tusharchou commented on code in PR #1388: URL: https://github.com/apache/iceberg-python/pull/1388#discussion_r1915063325 ## pyiceberg/table/__init__.py: ## @@ -1522,8 +1555,9 @@ def plan_files(self) -> Iterable[FileScanTask]: data_entry,

Re: [PR] AWS, Core, GCP: Support relative credential endpoint / pass OAuth2 token to credential provider [iceberg]

2025-01-14 Thread via GitHub
nastra commented on code in PR #11954: URL: https://github.com/apache/iceberg/pull/11954#discussion_r1915070130 ## core/src/main/java/org/apache/iceberg/rest/RESTUtil.java: ## @@ -215,4 +215,19 @@ public static Namespace decodeNamespace(String encodedNs) { return Namespac

Re: [PR] Add ResidualVisitor to compute residuals [iceberg-python]

2025-01-14 Thread via GitHub
Fokko commented on code in PR #1388: URL: https://github.com/apache/iceberg-python/pull/1388#discussion_r1915102464 ## pyiceberg/table/__init__.py: ## @@ -1522,8 +1555,9 @@ def plan_files(self) -> Iterable[FileScanTask]: data_entry, posi

Re: [PR] Docs: add apache amoro(incubating) with iceberg (#11965) [iceberg]

2025-01-14 Thread via GitHub
nastra commented on code in PR #11966: URL: https://github.com/apache/iceberg/pull/11966#discussion_r1915106856 ## docs/docs/amoro.md: ## @@ -0,0 +1,90 @@ +--- +title: "Apache Amoro" +--- + + +# Apache Amoro With Iceberg + +**[Apache Amoro(incubating)](https://amoro.apache.org/d

Re: [PR] Docs: add apache amoro(incubating) with iceberg (#11965) [iceberg]

2025-01-14 Thread via GitHub
nastra commented on code in PR #11966: URL: https://github.com/apache/iceberg/pull/11966#discussion_r1915111401 ## docs/docs/amoro.md: ## @@ -0,0 +1,90 @@ +--- +title: "Apache Amoro" +--- + + +# Apache Amoro With Iceberg + +**[Apache Amoro(incubating)](https://amoro.apache.org/d

Re: [I] How to move Iceberg table from one location to another [iceberg]

2025-01-14 Thread via GitHub
manuzhang closed issue #11645: How to move Iceberg table from one location to another URL: https://github.com/apache/iceberg/issues/11645 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] Why the metric values are all N/A on the spark webui? [iceberg]

2025-01-14 Thread via GitHub
manuzhang closed issue #11794: Why the metric values are all N/A on the spark webui? URL: https://github.com/apache/iceberg/issues/11794 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
HonahX commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915306849 ## core/src/test/java/org/apache/iceberg/MetadataTestUtils.java: ## @@ -0,0 +1,336 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more co

Re: [PR] Core, API, Spec: Metadata Row Lineage [iceberg]

2025-01-14 Thread via GitHub
HonahX commented on code in PR #11948: URL: https://github.com/apache/iceberg/pull/11948#discussion_r1915308572 ## core/src/test/java/org/apache/iceberg/TestSnapshotJson.java: ## @@ -43,7 +43,16 @@ public void testJsonConversion() throws IOException { Snapshot expected =

Re: [PR] Core, API, Spec: Metadata Row Lineage [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11948: URL: https://github.com/apache/iceberg/pull/11948#discussion_r1915289834 ## core/src/test/java/org/apache/iceberg/TestSnapshotJson.java: ## @@ -43,7 +43,16 @@ public void testJsonConversion() throws IOException { Snapshot exp

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
HonahX commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915306849 ## core/src/test/java/org/apache/iceberg/MetadataTestUtils.java: ## @@ -0,0 +1,336 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more co

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915347026 ## core/src/test/java/org/apache/iceberg/MetadataTestUtils.java: ## @@ -0,0 +1,336 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [I] Getting all snapshots through table.inspect.snapshots() [iceberg-python]

2025-01-14 Thread via GitHub
kevinjqliu commented on issue #1512: URL: https://github.com/apache/iceberg-python/issues/1512#issuecomment-2590718401 x-posting from the other issue (https://github.com/apache/iceberg-python/issues/1105#issuecomment-2588075721) > To retain Nessie’s consistency and cross-branch/tag i

Re: [I] Getting all snapshots through table.inspect.snapshots() [iceberg-python]

2025-01-14 Thread via GitHub
kevinjqliu closed issue #1512: Getting all snapshots through table.inspect.snapshots() URL: https://github.com/apache/iceberg-python/issues/1512 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915353274 ## core/src/main/java/org/apache/iceberg/TableMetadataParser.java: ## @@ -352,6 +352,7 @@ public static TableMetadata fromJson(String metadataLocation, JsonNo

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915373560 ## core/src/test/java/org/apache/iceberg/MetadataTestUtils.java: ## @@ -0,0 +1,336 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915347026 ## core/src/test/java/org/apache/iceberg/MetadataTestUtils.java: ## @@ -0,0 +1,336 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915381785 ## core/src/test/java/org/apache/iceberg/TestMetadataUpdateParser.java: ## @@ -364,15 +364,15 @@ public void testAddSnapshotToJson() throws IOException {

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915381785 ## core/src/test/java/org/apache/iceberg/TestMetadataUpdateParser.java: ## @@ -364,15 +364,15 @@ public void testAddSnapshotToJson() throws IOException {

Re: [PR] Support WASB scheme in ADLSFileIO [iceberg]

2025-01-14 Thread via GitHub
amogh-jahagirdar commented on code in PR #11830: URL: https://github.com/apache/iceberg/pull/11830#discussion_r1915382736 ## azure/src/test/java/org/apache/iceberg/azure/adlsv2/ADLSLocationTest.java: ## @@ -89,4 +101,17 @@ public void testQuestionMarkInFileName(String path) {

Re: [PR] Support WASB scheme in ADLSFileIO [iceberg]

2025-01-14 Thread via GitHub
amogh-jahagirdar commented on code in PR #11830: URL: https://github.com/apache/iceberg/pull/11830#discussion_r1915391549 ## azure/src/test/java/org/apache/iceberg/azure/adlsv2/ADLSLocationTest.java: ## @@ -89,4 +101,17 @@ public void testQuestionMarkInFileName(String path) {

Re: [PR] Spec: Document Snapshot Summary Optional Fields for Standardization [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on PR #11660: URL: https://github.com/apache/iceberg/pull/11660#issuecomment-2590959103 @HonahX I think we are good on this one, I don't think we had any negative feedback from the dev list. So I think we are good to merge if you are ready. -- This is an automated

Re: [PR] Spec: Document Snapshot Summary Optional Fields for Standardization [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11660: URL: https://github.com/apache/iceberg/pull/11660#discussion_r1915505397 ## format/spec.md: ## @@ -1633,3 +1633,57 @@ might indicate different snapshot IDs for a specific timestamp. The discrepancie When processing point in time

Re: [PR] Support WASB scheme in ADLSFileIO [iceberg]

2025-01-14 Thread via GitHub
amogh-jahagirdar commented on code in PR #11830: URL: https://github.com/apache/iceberg/pull/11830#discussion_r1915391549 ## azure/src/test/java/org/apache/iceberg/azure/adlsv2/ADLSLocationTest.java: ## @@ -89,4 +101,17 @@ public void testQuestionMarkInFileName(String path) {

Re: [PR] Open-API: Bump to OpenAPI 3.1 [iceberg]

2025-01-14 Thread via GitHub
Fokko commented on PR #11955: URL: https://github.com/apache/iceberg/pull/11955#issuecomment-2590904626 Thanks @jbonofre, @nastra and @hussein-awala for the review 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
HonahX commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915460216 ## core/src/main/java/org/apache/iceberg/TableMetadataParser.java: ## @@ -352,6 +352,7 @@ public static TableMetadata fromJson(String metadataLocation, JsonNode node)

Re: [PR] [draft] AWS: support hadoop bulk delete API [iceberg]

2025-01-14 Thread via GitHub
steveloughran commented on PR #10233: URL: https://github.com/apache/iceberg/pull/10233#issuecomment-2590997379 For anyone watching this, there's a full integration test suite in the hadoop test code: https://github.com/apache/hadoop/pull/7285 All is good, though as it's the first jav

Re: [PR] Support WASB scheme in ADLSFileIO [iceberg]

2025-01-14 Thread via GitHub
amogh-jahagirdar commented on code in PR #11830: URL: https://github.com/apache/iceberg/pull/11830#discussion_r1915391549 ## azure/src/test/java/org/apache/iceberg/azure/adlsv2/ADLSLocationTest.java: ## @@ -89,4 +101,17 @@ public void testQuestionMarkInFileName(String path) {

Re: [PR] Core: Fix numeric overflow of timestamp nano literal [iceberg]

2025-01-14 Thread via GitHub
ebyhr commented on code in PR #11775: URL: https://github.com/apache/iceberg/pull/11775#discussion_r1915834371 ## api/src/main/java/org/apache/iceberg/expressions/Literals.java: ## @@ -300,8 +300,7 @@ public Literal to(Type type) { case TIMESTAMP: return (Li

Re: [I] Does Iceberg have a command similar to Hive MSCK to repair tables? [iceberg]

2025-01-14 Thread via GitHub
SGITLOGIN commented on issue #11961: URL: https://github.com/apache/iceberg/issues/11961#issuecomment-2591465958 > > I create the same table again. How can I restore this part of the data? > > What do you mean by `this part of the data`? Which catalog are you using? @manuzhang H

Re: [PR] Build: Bump openapi-generator plugin from 6.6.0 to 7.10.0 [iceberg]

2025-01-14 Thread via GitHub
Fokko commented on PR #11970: URL: https://github.com/apache/iceberg/pull/11970#issuecomment-2591832013 Thanks @nastra We should probably run the Java tests also when making changes to `open-api/*` -- This is an automated message from the Apache Git Service. To respond to the message, ple

[PR] [1.7.x] AWS, Core, GCP: Support relative credential endpoint / pass OAuth2 token to credential provider [iceberg]

2025-01-14 Thread via GitHub
nastra opened a new pull request, #11971: URL: https://github.com/apache/iceberg/pull/11971 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

Re: [PR] feat(catalog): Add Catalog Registry [iceberg-go]

2025-01-14 Thread via GitHub
kevinjqliu commented on code in PR #244: URL: https://github.com/apache/iceberg-go/pull/244#discussion_r1915921905 ## catalog/registry.go: ## @@ -0,0 +1,135 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOT

Re: [PR] ci: check MSRV correctly [iceberg-rust]

2025-01-14 Thread via GitHub
Xuanwo commented on PR #849: URL: https://github.com/apache/iceberg-rust/pull/849#issuecomment-2591683936 > If the current way looks good, I can try to fix compilation later. Current way looks really great to me. -- This is an automated message from the Apache Git Service. To respon

Re: [PR] Doc: Add DELETE ORPHAN-FILES example to Hive page [iceberg]

2025-01-14 Thread via GitHub
ebyhr commented on PR #11896: URL: https://github.com/apache/iceberg/pull/11896#issuecomment-2591729549 @pvary Can you take a look at this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
HonahX commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1916010795 ## core/src/test/java/org/apache/iceberg/TestMetadataUpdateParser.java: ## @@ -364,15 +364,15 @@ public void testAddSnapshotToJson() throws IOException { String m

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
HonahX commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1916011232 ## core/src/test/java/org/apache/iceberg/TestTableMetadata.java: ## @@ -96,31 +97,47 @@ public class TestTableMetadata { public TableOperations ops = new LocalTab

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
HonahX commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1916012887 ## core/src/test/java/org/apache/iceberg/TestTableMetadata.java: ## @@ -160,31 +177,26 @@ public void testJsonConversion() throws Exception { .build()

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
HonahX commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1916012081 ## core/src/test/java/org/apache/iceberg/TestTableMetadata.java: ## @@ -96,31 +97,47 @@ public class TestTableMetadata { public TableOperations ops = new LocalTab

[I] Creating Delete Vectors using Java API or Spark [iceberg]

2025-01-14 Thread via GitHub
piyushdubey opened a new issue, #11968: URL: https://github.com/apache/iceberg/issues/11968 ### Query engine Spark ### Question Q1 - Is it possible to create Deletion Vectors in Apache Iceberg? Is a Deletion Vector file generated for a specific Deletion type Positional v

Re: [PR] ci: check MSRV correctly [iceberg-rust]

2025-01-14 Thread via GitHub
xxchan commented on PR #849: URL: https://github.com/apache/iceberg-rust/pull/849#issuecomment-2591666123 https://github.com/user-attachments/assets/15d7dcad-8de5-472b-ab3a-1db9e1d35690"; /> Oops, it still doesn't compile. I guess it's because some dependencies' failed to specify cor

Re: [PR] Spark 3.5: Fix empty scan issue when start timestamp retrieves root snapshot and end timestamp is missing [iceberg]

2025-01-14 Thread via GitHub
lliangyu-lin commented on PR #11967: URL: https://github.com/apache/iceberg/pull/11967#issuecomment-2591666267 Thanks @manuzhang for the review. cc: @flyrain @Fokko May I get some help on additional reviews? -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Docs: add apache amoro(incubating) with iceberg (#11965) [iceberg]

2025-01-14 Thread via GitHub
ebyhr commented on code in PR #11966: URL: https://github.com/apache/iceberg/pull/11966#discussion_r1915973689 ## docs/docs/amoro.md: ## @@ -0,0 +1,90 @@ +--- +title: "Apache Amoro" +--- + + +# Apache Amoro With Iceberg + +**[Apache Amoro(incubating)](https://amoro.apache.org/do

Re: [PR] Spec: add variant type [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #10831: URL: https://github.com/apache/iceberg/pull/10831#discussion_r1915973279 ## format/spec.md: ## @@ -182,6 +182,21 @@ A **`list`** is a collection of values with some element type. The element field A **`map`** is a collection of

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
HonahX commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1916016403 ## core/src/test/java/org/apache/iceberg/TestTableMetadata.java: ## @@ -314,31 +328,43 @@ public void testBackwardCompat() throws Exception { assertThat(metadata.

Re: [PR] feat(catalog): Standardize Catalog create table function [iceberg-go]

2025-01-14 Thread via GitHub
Fokko merged PR #245: URL: https://github.com/apache/iceberg-go/pull/245 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] feat(catalog): Standardize Catalog create table function [iceberg-go]

2025-01-14 Thread via GitHub
Fokko commented on code in PR #245: URL: https://github.com/apache/iceberg-go/pull/245#discussion_r1916018700 ## table/metadata.go: ## @@ -1029,3 +1030,76 @@ func (m *metadataV2) UnmarshalJSON(b []byte) error { m.preValidate() return m.validate() } + +const Defa

Re: [PR] AWS, Core, GCP: Support relative credential endpoint / pass OAuth2 token to credential provider [iceberg]

2025-01-14 Thread via GitHub
nastra closed pull request #11954: AWS, Core, GCP: Support relative credential endpoint / pass OAuth2 token to credential provider URL: https://github.com/apache/iceberg/pull/11954 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] feat(catalog): Standardize Catalog create table function [iceberg-go]

2025-01-14 Thread via GitHub
Fokko commented on PR #245: URL: https://github.com/apache/iceberg-go/pull/245#issuecomment-2591769403 Thanks @zeroshade 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] Partition writes not creating expected directory hierarchy on S3 (MinIO) [iceberg-rust]

2025-01-14 Thread via GitHub
hackintoshrao commented on issue #891: URL: https://github.com/apache/iceberg-rust/issues/891#issuecomment-2591778008 Thank you, @ZENOTME, for the response. I work with a startup building a distributed query engine for large-scale Iceberg tables (>1PB). The `partition-specific` hiera

Re: [I] Creating Delete Vectors using Java API or Spark [iceberg]

2025-01-14 Thread via GitHub
nastra commented on issue #11968: URL: https://github.com/apache/iceberg/issues/11968#issuecomment-2591782721 Not sure where you got the `write.delete.vector.enabled` property from but that property doesn't exist. The DV code hasn't been officially released yet. You could either wait for

Re: [PR] AWS, Core, GCP: Support relative credential endpoint / pass OAuth2 token to credential provider [iceberg]

2025-01-14 Thread via GitHub
nastra closed pull request #11954: AWS, Core, GCP: Support relative credential endpoint / pass OAuth2 token to credential provider URL: https://github.com/apache/iceberg/pull/11954 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] ci: check MSRV correctly [iceberg-rust]

2025-01-14 Thread via GitHub
xxchan commented on PR #849: URL: https://github.com/apache/iceberg-rust/pull/849#issuecomment-2591790418 @Xuanwo CI is green now! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
HonahX commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915429803 ## core/src/test/java/org/apache/iceberg/TestMetadataUpdateParser.java: ## @@ -364,15 +364,15 @@ public void testAddSnapshotToJson() throws IOException { String m

Re: [PR] Support WASB scheme in ADLSFileIO [iceberg]

2025-01-14 Thread via GitHub
amogh-jahagirdar commented on code in PR #11830: URL: https://github.com/apache/iceberg/pull/11830#discussion_r1915560522 ## azure/src/test/java/org/apache/iceberg/azure/adlsv2/ADLSLocationTest.java: ## @@ -89,4 +101,17 @@ public void testQuestionMarkInFileName(String path) {

Re: [PR] Spec: add variant type [iceberg]

2025-01-14 Thread via GitHub
aihuaxu commented on code in PR #10831: URL: https://github.com/apache/iceberg/pull/10831#discussion_r1915818592 ## format/spec.md: ## @@ -182,6 +182,21 @@ A **`list`** is a collection of values with some element type. The element field A **`map`** is a collection of key-val

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915508273 ## core/src/main/java/org/apache/iceberg/TableMetadataParser.java: ## @@ -352,6 +352,7 @@ public static TableMetadata fromJson(String metadataLocation, JsonNo

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-14 Thread via GitHub
RussellSpitzer commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1915509758 ## core/src/test/java/org/apache/iceberg/TestSnapshotJson.java: ## @@ -61,8 +68,15 @@ public void testJsonConversionWithoutSchemaId() throws IOException {

Re: [PR] feat(catalog): Add Catalog Registry [iceberg-go]

2025-01-14 Thread via GitHub
zeroshade commented on code in PR #244: URL: https://github.com/apache/iceberg-go/pull/244#discussion_r1915663329 ## catalog/registry.go: ## @@ -0,0 +1,135 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTI

Re: [I] Does Iceberg have a command similar to Hive MSCK to repair tables? [iceberg]

2025-01-14 Thread via GitHub
SGITLOGIN closed issue #11961: Does Iceberg have a command similar to Hive MSCK to repair tables? URL: https://github.com/apache/iceberg/issues/11961 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] Does Iceberg have a command similar to Hive MSCK to repair tables? [iceberg]

2025-01-14 Thread via GitHub
SGITLOGIN commented on issue #11961: URL: https://github.com/apache/iceberg/issues/11961#issuecomment-2591579937 Hello,I found the solution, which can be achieved through register_table -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] Docs: add apache amoro(incubating) with iceberg (#11965) [iceberg]

2025-01-14 Thread via GitHub
XBaith commented on code in PR #11966: URL: https://github.com/apache/iceberg/pull/11966#discussion_r1915882328 ## docs/docs/amoro.md: ## @@ -0,0 +1,90 @@ +--- +title: "Apache Amoro" +--- + + +# Apache Amoro With Iceberg + +**[Apache Amoro(incubating)](https://amoro.apache.org/d

Re: [PR] Docs: add apache amoro(incubating) with iceberg (#11965) [iceberg]

2025-01-14 Thread via GitHub
XBaith commented on code in PR #11966: URL: https://github.com/apache/iceberg/pull/11966#discussion_r1915887251 ## docs/docs/amoro.md: ## @@ -0,0 +1,90 @@ +--- +title: "Apache Amoro" +--- + + +# Apache Amoro With Iceberg + +**[Apache Amoro(incubating)](https://amoro.apache.org/d

Re: [PR] ci: check MSRV correctly [iceberg-rust]

2025-01-14 Thread via GitHub
Xuanwo commented on PR #849: URL: https://github.com/apache/iceberg-rust/pull/849#issuecomment-2591560448 Really appreciate your work here! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

  1   2   >