Re: [PR] Core: Fix test failure caused by check empty namespace on REST session catalog [iceberg]

2025-01-13 Thread via GitHub
hantangwangd commented on code in PR #11960: URL: https://github.com/apache/iceberg/pull/11960#discussion_r1914391256 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -659,9 +659,9 @@ public List listNamespaces(SessionContext context, Namespace names

Re: [I] create_changelog_view returns no record when end-timestamp is missing [iceberg]

2025-01-13 Thread via GitHub
lliangyu-lin commented on issue #11922: URL: https://github.com/apache/iceberg/issues/11922#issuecomment-2589245565 I'm able to reproduce the issue. I'm happy to take a try on fixing the issue and add additional test coverages. -- This is an automated message from the Apache Git Service.

Re: [PR] Core: Fix tests around empty namespaces [iceberg]

2025-01-13 Thread via GitHub
nastra closed pull request #11962: Core: Fix tests around empty namespaces URL: https://github.com/apache/iceberg/pull/11962 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Core: Fix test failure caused by check empty namespace on REST session catalog [iceberg]

2025-01-13 Thread via GitHub
nastra commented on code in PR #11960: URL: https://github.com/apache/iceberg/pull/11960#discussion_r1914380512 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -659,9 +659,9 @@ public List listNamespaces(SessionContext context, Namespace namespac

Re: [PR] Core: Fix tests around empty namespaces [iceberg]

2025-01-13 Thread via GitHub
nastra commented on PR #11962: URL: https://github.com/apache/iceberg/pull/11962#issuecomment-2589236132 Actually I think https://github.com/apache/iceberg/pull/11960 is the better fix for this -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Core: Fix test failure caused by check empty namespace on REST session catalog [iceberg]

2025-01-13 Thread via GitHub
nastra commented on code in PR #11960: URL: https://github.com/apache/iceberg/pull/11960#discussion_r1914380512 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -659,9 +659,9 @@ public List listNamespaces(SessionContext context, Namespace namespac

Re: [PR] Spark 3.5: Add query runner in test module [iceberg]

2025-01-13 Thread via GitHub
nastra commented on PR #11758: URL: https://github.com/apache/iceberg/pull/11758#issuecomment-2589228337 I think it could be quite helpful but a few small comments or a small readme would be good to have -- This is an automated message from the Apache Git Service. To respond to the messag

Re: [PR] Core: Fix test failure caused by check empty namespace on REST session catalog [iceberg]

2025-01-13 Thread via GitHub
hantangwangd commented on PR #11960: URL: https://github.com/apache/iceberg/pull/11960#issuecomment-2589227245 Hi @nastra , can you please take a look at this fix when available? Thanks a lot! -- This is an automated message from the Apache Git Service. To respond to the message, please l

[I] Does Iceberg have a command similar to Hive MSCK to repair tables? [iceberg]

2025-01-13 Thread via GitHub
SGITLOGIN opened a new issue, #11961: URL: https://github.com/apache/iceberg/issues/11961 ### Query engine Spark 3.4.2 ### Question When Iceberg executes drop table without adding PURGE, the metadata and data files are not deleted. I create the same table again. How

Re: [I] insert branch failed:spark_catalog requires a single-part namespace, but got `iceberg_db`.`sample_oss_time_travel`. [iceberg]

2025-01-13 Thread via GitHub
SGITLOGIN closed issue #11952: insert branch failed:spark_catalog requires a single-part namespace, but got `iceberg_db`.`sample_oss_time_travel`. URL: https://github.com/apache/iceberg/issues/11952 -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [I] insert branch failed:spark_catalog requires a single-part namespace, but got `iceberg_db`.`sample_oss_time_travel`. [iceberg]

2025-01-13 Thread via GitHub
SGITLOGIN commented on issue #11952: URL: https://github.com/apache/iceberg/issues/11952#issuecomment-2589132191 > When you use the branch from SparkSQL, you need to set `branch_` before your branch name. Specifically, for your INSERT query, you need to specify the branch name like: >

Re: [I] insert branch failed:spark_catalog requires a single-part namespace, but got `iceberg_db`.`sample_oss_time_travel`. [iceberg]

2025-01-13 Thread via GitHub
tomtongue commented on issue #11952: URL: https://github.com/apache/iceberg/issues/11952#issuecomment-2588929264 When you use the branch from SparkSQL, you need to set `branch_` as its branch name. Specifically, for your INSERT query, you need to specify the branch name like: ```

Re: [PR] Spark 3.5: Procedure to rewrite table path [iceberg]

2025-01-13 Thread via GitHub
dramaticlly commented on PR #11931: URL: https://github.com/apache/iceberg/pull/11931#issuecomment-257095 > @dramaticlly would you mind updating doc as well? Yes @manuzhang , I plan to have a followup PR with spark procedure documentation once this is merged. -- This is an aut

Re: [PR] Spark 3.5: Procedure to rewrite table path [iceberg]

2025-01-13 Thread via GitHub
manuzhang commented on PR #11931: URL: https://github.com/apache/iceberg/pull/11931#issuecomment-252806 @dramaticlly would you mind updating doc as well? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[PR] Spark 3.5: Support RewriteManifestsProcedure with a target size parameter [iceberg]

2025-01-13 Thread via GitHub
majian1998 opened a new pull request, #11959: URL: https://github.com/apache/iceberg/pull/11959 Current Limitations: The rewrite manifest currently has limited parameters. To set or change the manifest size, you must go through the table config, which restricts flexibility. We wish for rewr

Re: [PR] Add Doxygen for generating API documentation [iceberg-cpp]

2025-01-13 Thread via GitHub
lidavidm commented on PR #27: URL: https://github.com/apache/iceberg-cpp/pull/27#issuecomment-2588799249 How it looks: ![image](https://github.com/user-attachments/assets/11dc7489-58a6-48c2-a9ca-c8358748c1f8) -- This is an automated message from the Apache Git Service. To respo

Re: [PR] feat(puffin): Parse Puffin FileMetadata [iceberg-rust]

2025-01-13 Thread via GitHub
liurenjie1024 merged PR #765: URL: https://github.com/apache/iceberg-rust/pull/765 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [I] insert branch failed:spark_catalog requires a single-part namespace, but got `iceberg_db`.`sample_oss_time_travel`. [iceberg]

2025-01-13 Thread via GitHub
SGITLOGIN commented on issue #11952: URL: https://github.com/apache/iceberg/issues/11952#issuecomment-2588579221 yes,I tried SET spark. wap. branch=audit; There is no problem, It is possible that there is an issue with Spark parsing catalog.namespace.table.branch-audit -- This is an auto

Re: [PR] [docs] Update docs link about Apache Doris and update vendors list [iceberg]

2025-01-13 Thread via GitHub
ebyhr commented on code in PR #11956: URL: https://github.com/apache/iceberg/pull/11956#discussion_r1913957973 ## site/docs/vendors.md: ## @@ -89,3 +89,8 @@ Starburst is a commercial offering for the [Trino query engine](https://trino.io ### [Upsolver](https://upsolver.com)

Re: [I] [Spec][Upstream] Mapping from DecimalType to Parquet physical type not aligned with spec [iceberg-python]

2025-01-13 Thread via GitHub
github-actions[bot] commented on issue #936: URL: https://github.com/apache/iceberg-python/issues/936#issuecomment-2588486293 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity oc

Re: [I] PyIceberg with Azure Storage Account (500 Internal Server Error) [iceberg-python]

2025-01-13 Thread via GitHub
github-actions[bot] commented on issue #939: URL: https://github.com/apache/iceberg-python/issues/939#issuecomment-2588486282 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity oc

Re: [PR] [Views] Update view spec with table identifier requirements [iceberg]

2025-01-13 Thread via GitHub
github-actions[bot] commented on PR #11365: URL: https://github.com/apache/iceberg/pull/11365#issuecomment-2588484684 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] empty partition folders after delete the data or drop the table [iceberg]

2025-01-13 Thread via GitHub
github-actions[bot] commented on issue #9956: URL: https://github.com/apache/iceberg/issues/9956#issuecomment-2588484637 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Unencoded Variable Length Column Size Statistics [iceberg]

2025-01-13 Thread via GitHub
github-actions[bot] commented on issue #10703: URL: https://github.com/apache/iceberg/issues/10703#issuecomment-2588484662 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] [WIP][Core] Restrict adding column of StructType with Empty Fields [iceberg]

2025-01-13 Thread via GitHub
github-actions[bot] commented on PR #11755: URL: https://github.com/apache/iceberg/pull/11755#issuecomment-2588484724 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] Requesting help with Deployment to Google Kubernetes Engine [iceberg]

2025-01-13 Thread via GitHub
github-actions[bot] commented on issue #10713: URL: https://github.com/apache/iceberg/issues/10713#issuecomment-2588484669 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] ParallelIterable: Queue Size w/ O(1) [iceberg]

2025-01-13 Thread via GitHub
tbaeg commented on PR #11895: URL: https://github.com/apache/iceberg/pull/11895#issuecomment-2588475018 I think incremental improvement for the existing implementation (even if slated for rewrite) should be included. Of note, we cherry-picked commits from https://github.com/apache/ic

Re: [PR] Spark 3.5: Add query runner in test module [iceberg]

2025-01-13 Thread via GitHub
ebyhr commented on PR #11758: URL: https://github.com/apache/iceberg/pull/11758#issuecomment-2588429690 Note that this query runner allows inspecting at breakpoints. This class is helpful to debug codes without writing tests when community members reported issues. -- This is an automate

[PR] Build: Bump mkdocs-autorefs from 1.2.0 to 1.3.0 [iceberg-python]

2025-01-13 Thread via GitHub
dependabot[bot] opened a new pull request, #1513: URL: https://github.com/apache/iceberg-python/pull/1513 Bumps [mkdocs-autorefs](https://github.com/mkdocstrings/autorefs) from 1.2.0 to 1.3.0. Release notes Sourced from https://github.com/mkdocstrings/autorefs/releases";>mkdocs-aut

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/glue from 1.105.0 to 1.105.1 [iceberg-go]

2025-01-13 Thread via GitHub
zeroshade merged PR #247: URL: https://github.com/apache/iceberg-go/pull/247 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.72.0 to 1.72.2 [iceberg-go]

2025-01-13 Thread via GitHub
zeroshade merged PR #249: URL: https://github.com/apache/iceberg-go/pull/249 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] build(deps): bump google.golang.org/api from 0.214.0 to 0.216.0 [iceberg-go]

2025-01-13 Thread via GitHub
zeroshade merged PR #248: URL: https://github.com/apache/iceberg-go/pull/248 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] refactor: improve manifest scanning organization and concurrency [iceberg-go]

2025-01-13 Thread via GitHub
zeroshade merged PR #252: URL: https://github.com/apache/iceberg-go/pull/252 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] refactor: improve manifest scanning organization and concurrency [iceberg-go]

2025-01-13 Thread via GitHub
zeroshade commented on code in PR #252: URL: https://github.com/apache/iceberg-go/pull/252#discussion_r1913882185 ## table/scanner.go: ## @@ -259,129 +286,119 @@ func matchDeletesToData(entry iceberg.ManifestEntry, positionalDeletes []iceberg return out, nil } -func

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.17.48 to 1.17.51 [iceberg-go]

2025-01-13 Thread via GitHub
dependabot[bot] commented on PR #250: URL: https://github.com/apache/iceberg-go/pull/250#issuecomment-2588346627 Looks like github.com/aws/aws-sdk-go-v2/credentials is up-to-date now, so this is no longer needed. -- This is an automated message from the Apache Git Service. To respond to t

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.17.48 to 1.17.51 [iceberg-go]

2025-01-13 Thread via GitHub
dependabot[bot] closed pull request #250: build(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.17.48 to 1.17.51 URL: https://github.com/apache/iceberg-go/pull/250 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.28.7 to 1.28.10 [iceberg-go]

2025-01-13 Thread via GitHub
zeroshade merged PR #251: URL: https://github.com/apache/iceberg-go/pull/251 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] refactor: improve manifest scanning organization and concurrency [iceberg-go]

2025-01-13 Thread via GitHub
zeroshade commented on code in PR #252: URL: https://github.com/apache/iceberg-go/pull/252#discussion_r1913854651 ## table/scanner.go: ## @@ -259,129 +286,119 @@ func matchDeletesToData(entry iceberg.ManifestEntry, positionalDeletes []iceberg return out, nil } -func

Re: [PR] refactor: improve manifest scanning organization and concurrency [iceberg-go]

2025-01-13 Thread via GitHub
iskakaushik commented on code in PR #252: URL: https://github.com/apache/iceberg-go/pull/252#discussion_r1913861589 ## table/scanner.go: ## @@ -259,129 +286,119 @@ func matchDeletesToData(entry iceberg.ManifestEntry, positionalDeletes []iceberg return out, nil } -fun

Re: [PR] chore(dependencies): update Arrow and Substrait dependencies [iceberg-go]

2025-01-13 Thread via GitHub
zeroshade merged PR #253: URL: https://github.com/apache/iceberg-go/pull/253 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [I] [feature] UpdateSchema.add_column supports both parent and child in the same transaction [iceberg-python]

2025-01-13 Thread via GitHub
jiakai-li commented on issue #1493: URL: https://github.com/apache/iceberg-python/issues/1493#issuecomment-2588295176 Sure @kevinjqliu , I think this is a very interesting API to support. I checked the java side as well to get some idea about how they tackle the issue. Seems it's not suppo

Re: [PR] feat(catalog): Standardize Catalog create table function [iceberg-go]

2025-01-13 Thread via GitHub
zeroshade commented on PR #245: URL: https://github.com/apache/iceberg-go/pull/245#issuecomment-2588263730 @Fokko any further comments? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Core, Spark: Rewrite data files with high delete ratio [iceberg]

2025-01-13 Thread via GitHub
singhpk234 commented on code in PR #11825: URL: https://github.com/apache/iceberg/pull/11825#discussion_r1913830176 ## core/src/main/java/org/apache/iceberg/actions/SizeBasedDataRewriter.java: ## @@ -84,13 +86,30 @@ private boolean shouldRewrite(List group) { return enoughI

Re: [PR] feat(catalog): Standardize Catalog create table function [iceberg-go]

2025-01-13 Thread via GitHub
zeroshade commented on code in PR #245: URL: https://github.com/apache/iceberg-go/pull/245#discussion_r1913825350 ## catalog/rest.go: ## @@ -663,18 +631,40 @@ func splitIdentForPath(ident table.Identifier) (string, string, error) { return strings.Join(NamespaceFromIdent

Re: [PR] feat(catalog): Standardize Catalog create table function [iceberg-go]

2025-01-13 Thread via GitHub
kevinjqliu commented on code in PR #245: URL: https://github.com/apache/iceberg-go/pull/245#discussion_r1913823798 ## catalog/catalog.go: ## @@ -195,3 +200,36 @@ func TableNameFromIdent(ident table.Identifier) string { func NamespaceFromIdent(ident table.Identifier) table.Ident

Re: [PR] feat(catalog): Standardize Catalog create table function [iceberg-go]

2025-01-13 Thread via GitHub
zeroshade commented on code in PR #245: URL: https://github.com/apache/iceberg-go/pull/245#discussion_r1913822331 ## catalog/catalog.go: ## @@ -195,3 +200,36 @@ func TableNameFromIdent(ident table.Identifier) string { func NamespaceFromIdent(ident table.Identifier) table.Identi

Re: [PR] feat(catalog): Standardize Catalog create table function [iceberg-go]

2025-01-13 Thread via GitHub
kevinjqliu commented on code in PR #245: URL: https://github.com/apache/iceberg-go/pull/245#discussion_r1908136750 ## catalog/catalog.go: ## @@ -195,3 +200,36 @@ func TableNameFromIdent(ident table.Identifier) string { func NamespaceFromIdent(ident table.Identifier) table.Ident

Re: [I] [feature] UpdateSchema.add_column supports both parent and child in the same transaction [iceberg-python]

2025-01-13 Thread via GitHub
kevinjqliu commented on issue #1493: URL: https://github.com/apache/iceberg-python/issues/1493#issuecomment-2588188799 Thanks! Thats a good workaround. I think a more generic use case is to be able to modify pending updates. `details` here is a pending update, it has not yet been

Re: [I] [feature] UpdateSchema.add_column supports both parent and child in the same transaction [iceberg-python]

2025-01-13 Thread via GitHub
jiakai-li commented on issue #1493: URL: https://github.com/apache/iceberg-python/issues/1493#issuecomment-2588111953 Hey @kevinjqliu , after some investigation, I realized this feature might require a bigger change than I originally expected (which could be a good thing though, so we can

Re: [PR] Metadata Row Lineage [iceberg]

2025-01-13 Thread via GitHub
RussellSpitzer commented on code in PR #11948: URL: https://github.com/apache/iceberg/pull/11948#discussion_r1913751486 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1468,6 +1510,28 @@ public Builder setPreviousFileLocation(String previousFileLocation) {

Re: [PR] Metadata Row Lineage [iceberg]

2025-01-13 Thread via GitHub
RussellSpitzer commented on code in PR #11948: URL: https://github.com/apache/iceberg/pull/11948#discussion_r1913749663 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1230,6 +1260,18 @@ public Builder addSnapshot(Snapshot snapshot) { snapshotsById.put

Re: [PR] Metadata Row Lineage [iceberg]

2025-01-13 Thread via GitHub
RussellSpitzer commented on code in PR #11948: URL: https://github.com/apache/iceberg/pull/11948#discussion_r1913745405 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1230,6 +1260,18 @@ public Builder addSnapshot(Snapshot snapshot) { snapshotsById.put

Re: [I] [Nessie REST Catalog] Iceberg table not listing all snapshots [iceberg-python]

2025-01-13 Thread via GitHub
kevinjqliu commented on issue #1105: URL: https://github.com/apache/iceberg-python/issues/1105#issuecomment-2588075721 > To retain Nessie’s consistency and cross-branch/tag isolation guarantees, we have deliberately chosen to only return the state of a table or view as a single snapshot in

[PR] Core, Spark: Minimize executor memory pressure in broadcast of data to delete files [iceberg]

2025-01-13 Thread via GitHub
amogh-jahagirdar opened a new pull request, #11957: URL: https://github.com/apache/iceberg/pull/11957 Leaving in draft now, but in an effort to save executor memory as part of the broadcast of data to. file scoped deletes, we can remove the referenced manifest location because that's only n

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-13 Thread via GitHub
HonahX commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1913661284 ## core/src/test/resources/TableMetadataV2StatisticsFiles.json: ## Review Comment: It is referenced in the following test: https://github.com/apache/iceberg/b

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-13 Thread via GitHub
HonahX commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1913661284 ## core/src/test/resources/TableMetadataV2StatisticsFiles.json: ## Review Comment: It is still referenced in the following test: https://github.com/apache/ice

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-01-13 Thread via GitHub
HonahX commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1913661284 ## core/src/test/resources/TableMetadataV2StatisticsFiles.json: ## Review Comment: It is referenced in the following test: https://github.com/apache/iceberg/b

Re: [PR] [docs] Replace examples of Hadoop catalog with JDBC catalog [iceberg]

2025-01-13 Thread via GitHub
kevinjqliu commented on code in PR #11845: URL: https://github.com/apache/iceberg/pull/11845#discussion_r1913641156 ## site/docs/spark-quickstart.md: ## @@ -267,44 +271,109 @@ To read a table, simply use the Iceberg table's name. df = spark.table("demo.nyc.taxis").show()

Re: [PR] Improve `LocationProvider` unit tests [iceberg-python]

2025-01-13 Thread via GitHub
kevinjqliu commented on PR #1511: URL: https://github.com/apache/iceberg-python/pull/1511#issuecomment-2587762579 Thanks for the follow up @smaheshwar-pltr -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] AWS, Core, GCP: Support relative credential endpoint / pass OAuth2 token to credential provider [iceberg]

2025-01-13 Thread via GitHub
danielcweeks commented on PR #11954: URL: https://github.com/apache/iceberg/pull/11954#issuecomment-2587762161 Minor comments and we should add a description, but LGTM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [PR] Improve `LocationProvider` unit tests [iceberg-python]

2025-01-13 Thread via GitHub
kevinjqliu merged PR #1511: URL: https://github.com/apache/iceberg-python/pull/1511 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

Re: [PR] AWS, Core, GCP: Support relative credential endpoint / pass OAuth2 token to credential provider [iceberg]

2025-01-13 Thread via GitHub
danielcweeks commented on code in PR #11954: URL: https://github.com/apache/iceberg/pull/11954#discussion_r1913564118 ## aws/src/test/java/org/apache/iceberg/aws/AwsClientPropertiesTest.java: ## @@ -140,4 +141,54 @@ public void refreshCredentialsEndpointSetButRefreshDisabled()

Re: [PR] AWS, Core, GCP: Support relative credential endpoint / pass OAuth2 token to credential provider [iceberg]

2025-01-13 Thread via GitHub
danielcweeks commented on code in PR #11954: URL: https://github.com/apache/iceberg/pull/11954#discussion_r1913468816 ## core/src/main/java/org/apache/iceberg/rest/RESTUtil.java: ## @@ -215,4 +215,19 @@ public static Namespace decodeNamespace(String encodedNs) { return Na

Re: [PR] [Docs] Update spark-getting-started docs page to make the example valid [iceberg]

2025-01-13 Thread via GitHub
kevinjqliu commented on PR #11923: URL: https://github.com/apache/iceberg/pull/11923#issuecomment-2587748916 looks like we'd have to rerun spotless check again ``` Run './gradlew :iceberg-spark:iceberg-spark-runtime-3.3_2.12:spotlessApply' to fix these violations. ``` -- This

Re: [I] Getting all snapshots through table.inspect.snapshots() [iceberg-python]

2025-01-13 Thread via GitHub
kevinjqliu commented on issue #1512: URL: https://github.com/apache/iceberg-python/issues/1512#issuecomment-2587742210 I saw on Slack that this is using the Nessie catalog which has a known issue with returning only 1 snapshot https://github.com/apache/iceberg-python/issues/1105 -- This

Re: [I] Iceberg REST Spec Extensions for Access Decision Exchange [iceberg]

2025-01-13 Thread via GitHub
kulte commented on issue #10395: URL: https://github.com/apache/iceberg/issues/10395#issuecomment-2587723940 Is this really not planned now officially, or is this a `github-actions` stale automation? -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [PR] AWS, Core, GCP: Support relative credential endpoint / pass OAuth2 token to credential provider [iceberg]

2025-01-13 Thread via GitHub
creechy commented on code in PR #11954: URL: https://github.com/apache/iceberg/pull/11954#discussion_r1913536698 ## core/src/main/java/org/apache/iceberg/rest/RESTUtil.java: ## @@ -215,4 +215,19 @@ public static Namespace decodeNamespace(String encodedNs) { return Namespa

Re: [PR] ParallelIterable: Queue Size w/ O(1) [iceberg]

2025-01-13 Thread via GitHub
shanielh closed pull request #11895: ParallelIterable: Queue Size w/ O(1) URL: https://github.com/apache/iceberg/pull/11895 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

Re: [I] Forbidden Exception creating Polaris Rest catalog with Flink 1.20 [iceberg]

2025-01-13 Thread via GitHub
shantanu-dahiya commented on issue #11836: URL: https://github.com/apache/iceberg/issues/11836#issuecomment-2587651479 Envoy has [an issue](https://github.com/envoyproxy/envoy/issues/36305) and an [open PR](https://github.com/envoyproxy/envoy/pull/37642) to add configuration that allows upg

Re: [I] insert branch failed:spark_catalog requires a single-part namespace, but got `iceberg_db`.`sample_oss_time_travel`. [iceberg]

2025-01-13 Thread via GitHub
anish749 commented on issue #11952: URL: https://github.com/apache/iceberg/issues/11952#issuecomment-2587642557 I am facing the same issue with AWS S3 tables, when writing to a branch. so ```sql MERGE INTO `catalog`.namespace.table.branch_audit ... ``` fails because S3 t

Re: [PR] Metadata Row Lineage [iceberg]

2025-01-13 Thread via GitHub
RussellSpitzer commented on code in PR #11948: URL: https://github.com/apache/iceberg/pull/11948#discussion_r1913489410 ## core/src/test/java/org/apache/iceberg/TestRowLineageMetadata.java: ## @@ -0,0 +1,213 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

Re: [PR] Metadata Row Lineage [iceberg]

2025-01-13 Thread via GitHub
RussellSpitzer commented on code in PR #11948: URL: https://github.com/apache/iceberg/pull/11948#discussion_r1913488919 ## core/src/test/java/org/apache/iceberg/TestRowLineageMetadata.java: ## @@ -0,0 +1,213 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

Re: [PR] Metadata Row Lineage [iceberg]

2025-01-13 Thread via GitHub
RussellSpitzer commented on code in PR #11948: URL: https://github.com/apache/iceberg/pull/11948#discussion_r1913485862 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1468,6 +1510,28 @@ public Builder setPreviousFileLocation(String previousFileLocation) {

Re: [PR] Spec: Add cross-region bucket access property to config [iceberg]

2025-01-13 Thread via GitHub
RussellSpitzer commented on PR #11260: URL: https://github.com/apache/iceberg/pull/11260#issuecomment-2587600228 Approved and Merged, thanks @munendrasn for the PR and @Fokko and @nastra for reviewing -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] Spec: Add cross-region bucket access property to config [iceberg]

2025-01-13 Thread via GitHub
RussellSpitzer merged PR #11260: URL: https://github.com/apache/iceberg/pull/11260 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] AWS, Core, GCP: Support relative credential endpoint / pass OAuth2 token to credential provider [iceberg]

2025-01-13 Thread via GitHub
danielcweeks commented on code in PR #11954: URL: https://github.com/apache/iceberg/pull/11954#discussion_r1913468816 ## core/src/main/java/org/apache/iceberg/rest/RESTUtil.java: ## @@ -215,4 +215,19 @@ public static Namespace decodeNamespace(String encodedNs) { return Na

Re: [PR] ParallelIterable: Queue Size w/ O(1) [iceberg]

2025-01-13 Thread via GitHub
RussellSpitzer commented on PR #11895: URL: https://github.com/apache/iceberg/pull/11895#issuecomment-2587582525 > @RussellSpitzer, I see that #11768 is closed now, we use the PR in a forked version for over a week now and we've observed no issues, any chance to merge this? BTW, the fix for

Re: [PR] Core: Add tests for catalogs supporting empty namespaces [iceberg]

2025-01-13 Thread via GitHub
Fokko merged PR #9890: URL: https://github.com/apache/iceberg/pull/9890 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [PR] Add ResidualVisitor to compute residuals [iceberg-python]

2025-01-13 Thread via GitHub
Fokko commented on code in PR #1388: URL: https://github.com/apache/iceberg-python/pull/1388#discussion_r1913339866 ## pyiceberg/expressions/visitors.py: ## @@ -1731,3 +1731,214 @@ def _can_contain_nulls(self, field_id: int) -> bool: def _can_contain_nans(self, field_id:

Re: [PR] Flink 1.20: Support default values in Parquet reader [iceberg]

2025-01-13 Thread via GitHub
jbonofre commented on PR #11839: URL: https://github.com/apache/iceberg/pull/11839#issuecomment-2587420383 @Fokko FYI, I'm working on the tests. I will update this PR soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [PR] Open-API: Bump to OpenAPI 3.1 [iceberg]

2025-01-13 Thread via GitHub
jbonofre commented on code in PR #11955: URL: https://github.com/apache/iceberg/pull/11955#discussion_r1913370539 ## open-api/rest-catalog-open-api.py: ## @@ -132,11 +132,15 @@ class ExpressionType(BaseModel): class TrueExpression(BaseModel): -type: ExpressionType +

Re: [PR] Core: Add Catalog Transactions API [iceberg]

2025-01-13 Thread via GitHub
ajantha-bhat commented on PR #6948: URL: https://github.com/apache/iceberg/pull/6948#issuecomment-2587386729 Some of the users are asking for multi-table transactions. I would love to see progress on this. Happy to take up the pending work if the author is busy. -- This is an automated m

[PR] Core: Fix loading a table in CachingCatalog with metadata table name [iceberg]

2025-01-13 Thread via GitHub
gaborkaszab opened a new pull request, #11738: URL: https://github.com/apache/iceberg/pull/11738 If a regular table had a metadata table name then CachingCatalog throws a NoSuchTableException when loading that table. -- This is an automated message from the Apache Git Service. To respond

Re: [PR] Core: Fix loading a table in CachingCatalog with metadata table name [iceberg]

2025-01-13 Thread via GitHub
nastra closed pull request #11738: Core: Fix loading a table in CachingCatalog with metadata table name URL: https://github.com/apache/iceberg/pull/11738 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Open-API: Fix compilation errors in generated Java classes due to mismatched return types [iceberg]

2025-01-13 Thread via GitHub
VladimirYushkevich commented on code in PR #11806: URL: https://github.com/apache/iceberg/pull/11806#discussion_r1913331085 ## open-api/rest-catalog-open-api.py: ## @@ -267,26 +267,57 @@ class ViewVersion(BaseModel): default_namespace: Namespace = Field(..., alias='default-

Re: [PR] Use `ObjectStoreLocationProvider` by default [iceberg-python]

2025-01-13 Thread via GitHub
Fokko commented on PR #1509: URL: https://github.com/apache/iceberg-python/pull/1509#issuecomment-2587320031 Thanks for the quick follow up @smaheshwar-pltr, and thanks for the review @kevinjqliu 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, pl

Re: [PR] Use `ObjectStoreLocationProvider` by default [iceberg-python]

2025-01-13 Thread via GitHub
Fokko merged PR #1509: URL: https://github.com/apache/iceberg-python/pull/1509 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [I] [DISCUSSION] Set up documentation/website generator [iceberg-cpp]

2025-01-13 Thread via GitHub
lidavidm commented on issue #26: URL: https://github.com/apache/iceberg-cpp/issues/26#issuecomment-2587189407 How does it work for C++ though? I see a few random plugins (like cxxdox) but I'm not confident in them vs Doxygen. (I also don't see from a bit of searching how mkdocs handles the

Re: [PR] Modified exception objects being thrown when converting Pyarrow tables [iceberg-python]

2025-01-13 Thread via GitHub
DevChrisCross commented on code in PR #1498: URL: https://github.com/apache/iceberg-python/pull/1498#discussion_r1913239486 ## pyiceberg/exceptions.py: ## @@ -14,6 +14,9 @@ # KIND, either express or implied. See the License for the # specific language governing permissions

Re: [PR] Modified exception objects being thrown when converting Pyarrow tables [iceberg-python]

2025-01-13 Thread via GitHub
Fokko commented on code in PR #1498: URL: https://github.com/apache/iceberg-python/pull/1498#discussion_r1913230599 ## pyiceberg/io/pyarrow.py: ## @@ -1003,6 +1000,20 @@ def _(obj: pa.DictionaryType, visitor: PyArrowSchemaVisitor[T]) -> T: return visit_pyarrow(obj.value_ty

Re: [PR] Modified exception objects being thrown when converting Pyarrow tables [iceberg-python]

2025-01-13 Thread via GitHub
Fokko commented on code in PR #1498: URL: https://github.com/apache/iceberg-python/pull/1498#discussion_r1913227992 ## pyiceberg/exceptions.py: ## @@ -14,6 +14,9 @@ # KIND, either express or implied. See the License for the # specific language governing permissions and limi

Re: [PR] Implement column projection [iceberg-python]

2025-01-13 Thread via GitHub
Fokko commented on code in PR #1443: URL: https://github.com/apache/iceberg-python/pull/1443#discussion_r1913225607 ## pyiceberg/io/pyarrow.py: ## @@ -1216,6 +1216,25 @@ def _field_id(self, field: pa.Field) -> int: return -1 +def _get_column_projection_values( +

Re: [PR] Spec: Add cross-region bucket access property to config [iceberg]

2025-01-13 Thread via GitHub
munendrasn commented on PR #11260: URL: https://github.com/apache/iceberg/pull/11260#issuecomment-2587128406 Reached over to iceberg-slack to get the additional review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [PR] Core: Fix loading a table in CachingCatalog with metadata table name [iceberg]

2025-01-13 Thread via GitHub
gaborkaszab commented on code in PR #11738: URL: https://github.com/apache/iceberg/pull/11738#discussion_r1913194574 ## core/src/test/java/org/apache/iceberg/hadoop/TestCachingCatalog.java: ## @@ -166,6 +168,42 @@ public void testTableName() throws Exception { .isEqualT

Re: [PR] Implement column projection [iceberg-python]

2025-01-13 Thread via GitHub
Fokko commented on code in PR #1443: URL: https://github.com/apache/iceberg-python/pull/1443#discussion_r1913166114 ## pyiceberg/io/pyarrow.py: ## @@ -1216,6 +1216,25 @@ def _field_id(self, field: pa.Field) -> int: return -1 +def _get_column_projection_values( +

Re: [I] Javadoc issues [iceberg]

2025-01-13 Thread via GitHub
jbonofre commented on issue #10378: URL: https://github.com/apache/iceberg/issues/10378#issuecomment-2587027874 I'm resuming work on this one to include 1.7.2 and 1.8.0. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [I] [DISCUSSION] Set up documentation/website generator [iceberg-cpp]

2025-01-13 Thread via GitHub
Fokko commented on issue #26: URL: https://github.com/apache/iceberg-cpp/issues/26#issuecomment-2587026463 @pitrou It is pretty similar, mkdocs feels a bit more modern to me. Referencing the Python docs works pretty well: https://py.iceberg.apache.org/reference/pyiceberg/io/ -- This is a

Re: [I] [DISCUSSION] Set up documentation/website generator [iceberg-cpp]

2025-01-13 Thread via GitHub
pitrou commented on issue #26: URL: https://github.com/apache/iceberg-cpp/issues/26#issuecomment-2587019412 How does mkdocs compare to Sphinx? Sphinx really has excellent cross-referencing capabilities (to API docs, to glossary, to other pages, to index references, even to other Spinx docs)

Re: [I] [DISCUSSION] Set up documentation/website generator [iceberg-cpp]

2025-01-13 Thread via GitHub
Fokko commented on issue #26: URL: https://github.com/apache/iceberg-cpp/issues/26#issuecomment-2587009767 I don't have any opinion on the C++ documentation generation, but we use mkdocs at PyIceberg, and it works really well: https://github.com/apache/iceberg-python/tree/main/mkdocs --

Re: [PR] Build: Bump com.google.cloud:libraries-bom from 26.51.0 to 26.52.0 [iceberg]

2025-01-13 Thread via GitHub
Fokko merged PR #11846: URL: https://github.com/apache/iceberg/pull/11846 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

  1   2   >