Re: [PR] Set version in path instead of OS [iceberg-python]

2024-02-13 Thread via GitHub
nastra merged PR #422: URL: https://github.com/apache/iceberg-python/pull/422 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Core: Add property to prevent loss of view representation when replacing a view [iceberg]

2024-02-13 Thread via GitHub
nastra commented on code in PR #9620: URL: https://github.com/apache/iceberg/pull/9620#discussion_r1489047441 ## core/src/main/java/org/apache/iceberg/view/ViewMetadata.java: ## @@ -510,5 +529,29 @@ static List updateHistory(List history, Set< private Stream changes(Class

Re: [PR] Build: Bump sqlalchemy from 2.0.26 to 2.0.27 [iceberg-python]

2024-02-13 Thread via GitHub
Fokko merged PR #424: URL: https://github.com/apache/iceberg-python/pull/424 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Build: Bump duckdb from 0.9.2 to 0.10.0 [iceberg-python]

2024-02-13 Thread via GitHub
Fokko merged PR #425: URL: https://github.com/apache/iceberg-python/pull/425 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Build: Bump getdaft from 0.2.13 to 0.2.14 [iceberg-python]

2024-02-13 Thread via GitHub
Fokko merged PR #426: URL: https://github.com/apache/iceberg-python/pull/426 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Build: Bump fastavro from 1.9.3 to 1.9.4 [iceberg-python]

2024-02-13 Thread via GitHub
Fokko merged PR #427: URL: https://github.com/apache/iceberg-python/pull/427 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Core, Spark: Remove dangling deletes as part of RewriteDataFilesAction [iceberg]

2024-02-13 Thread via GitHub
ajantha-bhat commented on code in PR #9724: URL: https://github.com/apache/iceberg/pull/9724#discussion_r1489002574 ## api/src/main/java/org/apache/iceberg/actions/RewriteDataFiles.java: ## @@ -99,6 +99,20 @@ public interface RewriteDataFiles boolean USE_STARTING_SEQUENCE_N

Re: [PR] Core, Spark: Remove dangling deletes as part of RewriteDataFilesAction [iceberg]

2024-02-13 Thread via GitHub
ajantha-bhat commented on code in PR #9724: URL: https://github.com/apache/iceberg/pull/9724#discussion_r1489002574 ## api/src/main/java/org/apache/iceberg/actions/RewriteDataFiles.java: ## @@ -99,6 +99,20 @@ public interface RewriteDataFiles boolean USE_STARTING_SEQUENCE_N

Re: [I] Operations on partition columns in `WHERE` clause not used in pruning [iceberg]

2024-02-13 Thread via GitHub
amogh-jahagirdar commented on issue #9678: URL: https://github.com/apache/iceberg/issues/9678#issuecomment-1943105412 Thanks for reporting @mgmarino . I'd reach out to AWS support with the query details to see if Athena can do anything on their side. At least here, we can take a loo

Re: [I] Null values in metadata_log_entries [iceberg]

2024-02-13 Thread via GitHub
amogh-jahagirdar commented on issue #9723: URL: https://github.com/apache/iceberg/issues/9723#issuecomment-1943095150 The short answer to your question is yes, I think this is expected behavior for the scenario you described because a replace will reset the history entries (the snapshot-log

Re: [PR] Core: Add property to prevent loss of view representation when replacing a view [iceberg]

2024-02-13 Thread via GitHub
amogh-jahagirdar commented on code in PR #9620: URL: https://github.com/apache/iceberg/pull/9620#discussion_r1488225269 ## core/src/main/java/org/apache/iceberg/view/ViewMetadata.java: ## @@ -510,5 +529,29 @@ static List updateHistory(List history, Set< private Stream cha

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-02-13 Thread via GitHub
jackye1995 commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1488794633 ## open-api/rest-catalog-open-api.yaml: ## @@ -532,6 +532,103 @@ paths: 5XX: $ref: '#/components/responses/ServerErrorResponse' + /v1/{prefix}

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-02-13 Thread via GitHub
jackye1995 commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1488794339 ## open-api/rest-catalog-open-api.yaml: ## @@ -2068,6 +2165,146 @@ components: items: $ref: '#/components/schemas/PartitionStatisticsFile'

Re: [I] java.lang.IllegalArgumentException: requirement failed: length (-6235972) cannot be smaller than -1 [iceberg]

2024-02-13 Thread via GitHub
amogh-jahagirdar commented on issue #9689: URL: https://github.com/apache/iceberg/issues/9689#issuecomment-1942945448 Thanks for the repro steps @rjayapalan ! I just wanted to ask a clarifying question which would help narrow down possible culprits before I go ahead and attempt a repro.

Re: [PR] Add pagination to open api spec for listing of namespaces, tables, views [iceberg]

2024-02-13 Thread via GitHub
jackye1995 commented on code in PR #9660: URL: https://github.com/apache/iceberg/pull/9660#discussion_r1488770326 ## open-api/rest-catalog-open-api.yaml: ## @@ -1482,6 +1490,33 @@ components: explode: false example: "vended-credentials,remote-signing" +page-t

Re: [PR] Add pagination to open api spec for listing of namespaces, tables, views [iceberg]

2024-02-13 Thread via GitHub
jackye1995 commented on code in PR #9660: URL: https://github.com/apache/iceberg/pull/9660#discussion_r1488769635 ## open-api/rest-catalog-open-api.yaml: ## @@ -1482,6 +1490,33 @@ components: explode: false example: "vended-credentials,remote-signing" +page-t

Re: [PR] Add pagination to open api spec for listing of namespaces, tables, views [iceberg]

2024-02-13 Thread via GitHub
jackye1995 commented on code in PR #9660: URL: https://github.com/apache/iceberg/pull/9660#discussion_r1488768254 ## open-api/rest-catalog-open-api.yaml: ## @@ -1482,6 +1490,33 @@ components: explode: false example: "vended-credentials,remote-signing" +page-t

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-02-13 Thread via GitHub
dramaticlly commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1488764915 ## open-api/rest-catalog-open-api.yaml: ## @@ -532,6 +532,103 @@ paths: 5XX: $ref: '#/components/responses/ServerErrorResponse' + /v1/{prefix

Re: [PR] Dynamically support Spark native engine in Iceberg [iceberg]

2024-02-13 Thread via GitHub
huaxingao commented on PR #9721: URL: https://github.com/apache/iceberg/pull/9721#issuecomment-1942910134 cc @aokolnychyi @RussellSpitzer @flyrain -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Dynamically support Spark native engine in Iceberg [iceberg]

2024-02-13 Thread via GitHub
huaxingao commented on PR #9721: URL: https://github.com/apache/iceberg/pull/9721#issuecomment-1942910442 also cc @rdblue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [I] Support iceberg hadoop catalog in python library [iceberg-python]

2024-02-13 Thread via GitHub
brianfromoregon commented on issue #17: URL: https://github.com/apache/iceberg-python/issues/17#issuecomment-1942909223 @Fokko forgot the ping. Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-02-13 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1488746581 ## open-api/rest-catalog-open-api.yaml: ## @@ -2800,6 +3057,53 @@ components: additionalProperties: type: string +PreplanTableRequest: +

Re: [I] Run iceberg-mr and iceberg-hive tests on Hive 3.1.2 as well [iceberg]

2024-02-13 Thread via GitHub
github-actions[bot] commented on issue #1371: URL: https://github.com/apache/iceberg/issues/1371#issuecomment-1942885322 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Missing docs for Entries metadata table [iceberg]

2024-02-13 Thread via GitHub
github-actions[bot] commented on issue #1365: URL: https://github.com/apache/iceberg/issues/1365#issuecomment-1942885301 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Core, Spark: Remove dangling deletes as part of RewriteDataFilesAction [iceberg]

2024-02-13 Thread via GitHub
dramaticlly commented on PR #9724: URL: https://github.com/apache/iceberg/pull/9724#issuecomment-1942867308 @szehon-ho can I ask for your eyes first? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[PR] Core, Spark: Remove dangling deletes as part of RewriteDataFilesAction [iceberg]

2024-02-13 Thread via GitHub
dramaticlly opened a new pull request, #9724: URL: https://github.com/apache/iceberg/pull/9724 Goal: Attempt to clean up the dangling deletes as part of Spark RewriteDataFilesAction, it can be controlled by feature flag `remove-dangling-deletes` and it's by default turned on. Most of the

[I] Null values in metadata_log_entries [iceberg]

2024-02-13 Thread via GitHub
oneonestar opened a new issue, #9723: URL: https://github.com/apache/iceberg/issues/9723 ### Query engine Spark ### Question In `metadata_log_entries` table, `latest_snapshot_id, latest_schema_id, latest_sequence_number` return null in some cases. Also, those values beca

Re: [I] Core: checkpoint validation in BaseOverwriteFiles [iceberg]

2024-02-13 Thread via GitHub
amogh-jahagirdar commented on issue #9718: URL: https://github.com/apache/iceberg/issues/9718#issuecomment-1942824540 @hrishisd Thanks for reporting this, all the details provided make sense. The solution for keeping track of the last validated snapshot so retries are more efficient makes l

Re: [I] Drop the SQL issue when attempting to drop an Iceberg table whose location does not exist [iceberg]

2024-02-13 Thread via GitHub
HanzhiWang1105 commented on issue #7227: URL: https://github.com/apache/iceberg/issues/7227#issuecomment-1942809558 Hi all, I hit same issue. Any idea how to get the workaround and drop the table before the fixed PR is released? Thanks! -- This is an automated message from the Apache G

[PR] Make InMemoryFileIO map shared access across instances [iceberg]

2024-02-13 Thread via GitHub
geruh opened a new pull request, #9722: URL: https://github.com/apache/iceberg/pull/9722 This PR addresses the issue with InMemoryFileIO not being resolved by the RESTCatalog Tests identified in #9604. In the RESTCatalog tests are backed by the `InMemoryCatalog` which uses `InMemoryF

Re: [PR] Chore: build wheels for all macos versions including arm macos-14 [iceberg-python]

2024-02-13 Thread via GitHub
Fokko commented on PR #416: URL: https://github.com/apache/iceberg-python/pull/416#issuecomment-1942782294 Great idea: https://github.com/apache/iceberg-python/actions/runs/7893859717 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

Re: [PR] Fix environment variable parsing [iceberg-python]

2024-02-13 Thread via GitHub
hussein-awala commented on code in PR #423: URL: https://github.com/apache/iceberg-python/pull/423#discussion_r1488694863 ## tests/utils/test_config.py: ## @@ -41,6 +41,11 @@ def test_from_environment_variables_uppercase() -> None: assert Config().get_catalog_config("PRODUC

Re: [PR] Fix environment variable parsing [iceberg-python]

2024-02-13 Thread via GitHub
hussein-awala commented on code in PR #423: URL: https://github.com/apache/iceberg-python/pull/423#discussion_r1488694863 ## tests/utils/test_config.py: ## @@ -41,6 +41,11 @@ def test_from_environment_variables_uppercase() -> None: assert Config().get_catalog_config("PRODUC

[PR] Build: Bump fastavro from 1.9.3 to 1.9.4 [iceberg-python]

2024-02-13 Thread via GitHub
dependabot[bot] opened a new pull request, #427: URL: https://github.com/apache/iceberg-python/pull/427 Bumps [fastavro](https://github.com/fastavro/fastavro) from 1.9.3 to 1.9.4. Changelog Sourced from https://github.com/fastavro/fastavro/blob/master/ChangeLog";>fastavro's changel

[PR] Build: Bump getdaft from 0.2.13 to 0.2.14 [iceberg-python]

2024-02-13 Thread via GitHub
dependabot[bot] opened a new pull request, #426: URL: https://github.com/apache/iceberg-python/pull/426 Bumps [getdaft](https://github.com/Eventual-Inc/Daft) from 0.2.13 to 0.2.14. Release notes Sourced from https://github.com/Eventual-Inc/Daft/releases";>getdaft's releases.

[PR] Build: Bump duckdb from 0.9.2 to 0.10.0 [iceberg-python]

2024-02-13 Thread via GitHub
dependabot[bot] opened a new pull request, #425: URL: https://github.com/apache/iceberg-python/pull/425 Bumps [duckdb](https://github.com/duckdb/duckdb) from 0.9.2 to 0.10.0. Release notes Sourced from https://github.com/duckdb/duckdb/releases";>duckdb's releases. DuckDB 0.10

[PR] Build: Bump sqlalchemy from 2.0.26 to 2.0.27 [iceberg-python]

2024-02-13 Thread via GitHub
dependabot[bot] opened a new pull request, #424: URL: https://github.com/apache/iceberg-python/pull/424 Bumps [sqlalchemy](https://github.com/sqlalchemy/sqlalchemy) from 2.0.26 to 2.0.27. Release notes Sourced from https://github.com/sqlalchemy/sqlalchemy/releases";>sqlalchemy's r

Re: [PR] Chore: build wheels for all macos versions including arm macos-14 [iceberg-python]

2024-02-13 Thread via GitHub
hussein-awala commented on PR #416: URL: https://github.com/apache/iceberg-python/pull/416#issuecomment-1942769204 Great 🎉 Could we create a new RC (RC6) to test the creation of the wheels before voting? -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Add pagination to open api spec for listing of namespaces, tables, views [iceberg]

2024-02-13 Thread via GitHub
rahil-c commented on code in PR #9660: URL: https://github.com/apache/iceberg/pull/9660#discussion_r1488679806 ## open-api/rest-catalog-open-api.yaml: ## @@ -1482,6 +1490,33 @@ components: explode: false example: "vended-credentials,remote-signing" +page-toke

[PR] Fix environment variable parsing [iceberg-python]

2024-02-13 Thread via GitHub
Fokko opened a new pull request, #423: URL: https://github.com/apache/iceberg-python/pull/423 ```bash export PYICEBERG_CATALOG__SOMETHING__S3__REGION=eu-north-1 ``` Before: ```python >>> from pyiceberg.catalog import load_catalog >>> load_catalog('something').propert

Re: [PR] Add pagination to open api spec for listing of namespaces, tables, views [iceberg]

2024-02-13 Thread via GitHub
rahil-c commented on code in PR #9660: URL: https://github.com/apache/iceberg/pull/9660#discussion_r1488679806 ## open-api/rest-catalog-open-api.yaml: ## @@ -1482,6 +1490,33 @@ components: explode: false example: "vended-credentials,remote-signing" +page-toke

[PR] Dynamically support Spark native engine in Iceberg [iceberg]

2024-02-13 Thread via GitHub
huaxingao opened a new pull request, #9721: URL: https://github.com/apache/iceberg/pull/9721 This PR is to introduce a dynamic plugin mechanism to support Spark native execution engines, e.g. [Comet](https://github.com/apache/arrow-datafusion-comet) Currently in Iceberg, when vectori

[PR] Set version in path instead of OS [iceberg-python]

2024-02-13 Thread via GitHub
Fokko opened a new pull request, #422: URL: https://github.com/apache/iceberg-python/pull/422 Bundling this by version makes it easier for the release -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Chore: build wheels for all macos versions including arm macos-14 [iceberg-python]

2024-02-13 Thread via GitHub
Fokko merged PR #416: URL: https://github.com/apache/iceberg-python/pull/416 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] docs: Add section on fetching the remote [iceberg-python]

2024-02-13 Thread via GitHub
rdblue merged PR #421: URL: https://github.com/apache/iceberg-python/pull/421 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Update `LICENSE` and `NOTICE` [iceberg-python]

2024-02-13 Thread via GitHub
rdblue merged PR #413: URL: https://github.com/apache/iceberg-python/pull/413 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] OpenAPI: Add AppendDataFileUpdate to TableUpdate for rest appends [iceberg]

2024-02-13 Thread via GitHub
amogh-jahagirdar commented on code in PR #9717: URL: https://github.com/apache/iceberg/pull/9717#discussion_r1488618480 ## open-api/rest-catalog-open-api.yaml: ## @@ -3324,6 +3348,97 @@ components: type: integer format: int64 +TypeValue: + oneOf:

[I] How to insert overwrite with a single commit [iceberg]

2024-02-13 Thread via GitHub
difin opened a new issue, #9720: URL: https://github.com/apache/iceberg/issues/9720 ### Query engine Apache Hive ### Question Hello Iceberg Community, Background: implementation of Iceberg compaction in Apache Hive. Presently, Apache Hive has Major Query-Based

Re: [I] rewrite_data_files procedure fails with Premature end of Content-Length when using S3 client [iceberg]

2024-02-13 Thread via GitHub
paulpaul1076 commented on issue #9679: URL: https://github.com/apache/iceberg/issues/9679#issuecomment-1942611489 @nastra thank you very much, I will try tomorrow and let you know! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

Re: [I] Core: complete FileScanTaskParser for other FileScanTask implementation classes (like StaticDataTask) [iceberg]

2024-02-13 Thread via GitHub
aokolnychyi commented on issue #9597: URL: https://github.com/apache/iceberg/issues/9597#issuecomment-1942454151 Given that the task serialization is part of the spec, shall we raise this discussion during the sync? -- This is an automated message from the Apache Git Service. To respond t

Re: [I] Core: complete FileScanTaskParser for other FileScanTask implementation classes (like StaticDataTask) [iceberg]

2024-02-13 Thread via GitHub
aokolnychyi commented on issue #9597: URL: https://github.com/apache/iceberg/issues/9597#issuecomment-1942452335 I think it is reasonably to have a field in JSON that would indicate the task type. I'd also avoid any changes in task APIs, we can leverage that enum only in the parser. I doubt

Re: [PR] Chore: build wheels for all macos versions including arm macos-14 [iceberg-python]

2024-02-13 Thread via GitHub
hussein-awala commented on PR #416: URL: https://github.com/apache/iceberg-python/pull/416#issuecomment-1942375890 Actually, it's much simpler than I thought, the installed Python is only used to install poetry, then we build the wheels for the range of Python version, so bumping the Python

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-02-13 Thread via GitHub
jackye1995 commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1488517904 ## open-api/rest-catalog-open-api.yaml: ## @@ -2800,6 +3057,53 @@ components: additionalProperties: type: string +PreplanTableRequest:

Re: [PR] Core: Common metadata for TableMetadata and ViewMetadata [iceberg]

2024-02-13 Thread via GitHub
szehon-ho commented on code in PR #9682: URL: https://github.com/apache/iceberg/pull/9682#discussion_r1488515592 ## core/src/main/java/org/apache/iceberg/BaseMetadata.java: ## @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contrib

Re: [PR] Add pagination to open api spec for listing of namespaces, tables, views [iceberg]

2024-02-13 Thread via GitHub
jackye1995 commented on code in PR #9660: URL: https://github.com/apache/iceberg/pull/9660#discussion_r1488509954 ## open-api/rest-catalog-open-api.yaml: ## @@ -1482,6 +1490,33 @@ components: explode: false example: "vended-credentials,remote-signing" +page-t

Re: [PR] Add pagination to open api spec for listing of namespaces, tables, views [iceberg]

2024-02-13 Thread via GitHub
jackye1995 commented on code in PR #9660: URL: https://github.com/apache/iceberg/pull/9660#discussion_r1488490615 ## open-api/rest-catalog-open-api.yaml: ## @@ -1482,6 +1490,34 @@ components: explode: false example: "vended-credentials,remote-signing" +page-t

Re: [PR] Add pagination to open api spec for listing of namespaces, tables, views [iceberg]

2024-02-13 Thread via GitHub
jackye1995 commented on code in PR #9660: URL: https://github.com/apache/iceberg/pull/9660#discussion_r1488490615 ## open-api/rest-catalog-open-api.yaml: ## @@ -1482,6 +1490,34 @@ components: explode: false example: "vended-credentials,remote-signing" +page-t

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-02-13 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1488506328 ## open-api/rest-catalog-open-api.yaml: ## @@ -2800,6 +3062,58 @@ components: additionalProperties: type: string +PrePlanTableRequest: +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-02-13 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1488505978 ## open-api/rest-catalog-open-api.yaml: ## @@ -2800,6 +3057,53 @@ components: additionalProperties: type: string +PreplanTableRequest: +

Re: [PR] Add pagination to open api spec for listing of namespaces, tables, views [iceberg]

2024-02-13 Thread via GitHub
jackye1995 commented on code in PR #9660: URL: https://github.com/apache/iceberg/pull/9660#discussion_r1488505531 ## open-api/rest-catalog-open-api.yaml: ## @@ -1482,6 +1490,34 @@ components: explode: false example: "vended-credentials,remote-signing" +page-t

Re: [PR] Add pagination to open api spec for listing of namespaces, tables, views [iceberg]

2024-02-13 Thread via GitHub
rahil-c commented on code in PR #9660: URL: https://github.com/apache/iceberg/pull/9660#discussion_r1488502107 ## open-api/rest-catalog-open-api.yaml: ## @@ -1482,6 +1490,34 @@ components: explode: false example: "vended-credentials,remote-signing" +page-toke

Re: [PR] Add pagination to open api spec for listing of namespaces, tables, views [iceberg]

2024-02-13 Thread via GitHub
jackye1995 commented on code in PR #9660: URL: https://github.com/apache/iceberg/pull/9660#discussion_r1488490615 ## open-api/rest-catalog-open-api.yaml: ## @@ -1482,6 +1490,34 @@ components: explode: false example: "vended-credentials,remote-signing" +page-t

Re: [PR] Update `LICENSE` and `NOTICE` [iceberg-python]

2024-02-13 Thread via GitHub
Fokko commented on code in PR #413: URL: https://github.com/apache/iceberg-python/pull/413#discussion_r1488486128 ## NOTICE: ## @@ -1,37 +1,8 @@ Apache Iceberg -Copyright 2017-2024 The Apache Software Foundation +Copyright 2017-2022 The Apache Software Foundation Review Comm

[PR] docs: Add section on fetching the remote [iceberg-python]

2024-02-13 Thread via GitHub
Fokko opened a new pull request, #421: URL: https://github.com/apache/iceberg-python/pull/421 Before creating the tag -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] Add pagination to open api spec for listing of namespaces, tables, views [iceberg]

2024-02-13 Thread via GitHub
rahil-c commented on code in PR #9660: URL: https://github.com/apache/iceberg/pull/9660#discussion_r1488430408 ## open-api/rest-catalog-open-api.py: ## @@ -629,7 +638,7 @@ class TransformTerm(BaseModel): term: Reference -class ReportMetricsRequest2(CommitReport): +class

Re: [PR] Add pagination to open api spec for listing of namespaces, tables, views [iceberg]

2024-02-13 Thread via GitHub
rahil-c commented on code in PR #9660: URL: https://github.com/apache/iceberg/pull/9660#discussion_r1488452604 ## open-api/rest-catalog-open-api.yaml: ## @@ -1482,6 +1490,34 @@ components: explode: false example: "vended-credentials,remote-signing" +page-toke

Re: [PR] Chore: build wheels for all macos versions including arm macos-14 [iceberg-python]

2024-02-13 Thread via GitHub
Fokko commented on PR #416: URL: https://github.com/apache/iceberg-python/pull/416#issuecomment-1942243868 @hussein-awala Thanks! It would be great to get this in 🚀 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Chore: build wheels for all macos versions including arm macos-14 [iceberg-python]

2024-02-13 Thread via GitHub
hussein-awala commented on PR #416: URL: https://github.com/apache/iceberg-python/pull/416#issuecomment-1942241914 Yeah, setup-python doesn't provide an arm-compatible build (https://github.com/actions/setup-python/issues/705), I will try to implement a workaround for this, which I use for

Re: [PR] Feat: Add fail_if_exists param to create_table [iceberg-python]

2024-02-13 Thread via GitHub
hussein-awala commented on PR #415: URL: https://github.com/apache/iceberg-python/pull/415#issuecomment-1942231488 Exactly, this implements `CREATE TABLE IF NOT EXISTS` and not `CREATE OR REPLACE`. If you prefer to have a new method `create_table_if_not_exists`, I can move the table

Re: [PR] Add pagination to open api spec for listing of namespaces, tables, views [iceberg]

2024-02-13 Thread via GitHub
rahil-c commented on code in PR #9660: URL: https://github.com/apache/iceberg/pull/9660#discussion_r1488452604 ## open-api/rest-catalog-open-api.yaml: ## @@ -1482,6 +1490,34 @@ components: explode: false example: "vended-credentials,remote-signing" +page-toke

[PR] Flink: Move ParquetReader to LogicalTypeAnnotationVisitor [iceberg]

2024-02-13 Thread via GitHub
Fokko opened a new pull request, #9719: URL: https://github.com/apache/iceberg/pull/9719 This will enable nanosecond timestamp support, since the OriginalType does not represent the nanosecond timestamp. -- This is an automated message from the Apache Git Service. To respond to the messag

Re: [PR] Add pagination to open api spec for listing of namespaces, tables, views [iceberg]

2024-02-13 Thread via GitHub
rahil-c commented on code in PR #9660: URL: https://github.com/apache/iceberg/pull/9660#discussion_r1488452604 ## open-api/rest-catalog-open-api.yaml: ## @@ -1482,6 +1490,34 @@ components: explode: false example: "vended-credentials,remote-signing" +page-toke

Re: [PR] Add pagination to open api spec for listing of namespaces, tables, views [iceberg]

2024-02-13 Thread via GitHub
rahil-c commented on code in PR #9660: URL: https://github.com/apache/iceberg/pull/9660#discussion_r1488430408 ## open-api/rest-catalog-open-api.py: ## @@ -629,7 +638,7 @@ class TransformTerm(BaseModel): term: Reference -class ReportMetricsRequest2(CommitReport): +class

Re: [PR] Add pagination to open api spec for listing of namespaces, tables, views [iceberg]

2024-02-13 Thread via GitHub
rahil-c commented on code in PR #9660: URL: https://github.com/apache/iceberg/pull/9660#discussion_r1488429491 ## open-api/rest-catalog-open-api.yaml: ## @@ -1581,6 +1617,13 @@ components: type: string example: [ "accounting", "tax" ] +NextPageToken: +

Re: [PR] feat(catalog): add initial rest catalog impl [iceberg-go]

2024-02-13 Thread via GitHub
zeroshade commented on code in PR #58: URL: https://github.com/apache/iceberg-go/pull/58#discussion_r1488412489 ## catalog/catalog.go: ## @@ -47,19 +52,136 @@ func WithAwsConfig(cfg aws.Config) Option { } } +func WithCredential(cred string) Option { Review Comment:

Re: [PR] Add pagination to open api spec for listing of namespaces, tables, views [iceberg]

2024-02-13 Thread via GitHub
rahil-c commented on code in PR #9660: URL: https://github.com/apache/iceberg/pull/9660#discussion_r1488399115 ## open-api/rest-catalog-open-api.yaml: ## @@ -1482,6 +1490,34 @@ components: explode: false example: "vended-credentials,remote-signing" +page-toke

Re: [PR] Add pagination to open api spec for listing of namespaces, tables, views [iceberg]

2024-02-13 Thread via GitHub
rahil-c commented on code in PR #9660: URL: https://github.com/apache/iceberg/pull/9660#discussion_r1488399115 ## open-api/rest-catalog-open-api.yaml: ## @@ -1482,6 +1490,34 @@ components: explode: false example: "vended-credentials,remote-signing" +page-toke

Re: [I] Writer Design [iceberg-rust]

2024-02-13 Thread via GitHub
ZENOTME commented on issue #34: URL: https://github.com/apache/iceberg-rust/issues/34#issuecomment-1942155782 > @ZENOTME are you intending on implementing writer support until completion yourself? I have implemented the writer support in [icelake](https://github.com/icelake-io/icelak

Re: [PR] feat(catalog): add initial rest catalog impl [iceberg-go]

2024-02-13 Thread via GitHub
zeroshade commented on code in PR #58: URL: https://github.com/apache/iceberg-go/pull/58#discussion_r1488374463 ## catalog/rest_test.go: ## @@ -0,0 +1,818 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTIC

Re: [PR] feat(catalog): add initial rest catalog impl [iceberg-go]

2024-02-13 Thread via GitHub
zeroshade commented on PR #58: URL: https://github.com/apache/iceberg-go/pull/58#issuecomment-1942110612 @nastra So I've figured out the issue: The properties are correctly being propagated to the FileIO object, however it looks like the tabular api doesn't like the Go Iceberg user-ag

Re: [I] Writer Design [iceberg-rust]

2024-02-13 Thread via GitHub
lukekim commented on issue #34: URL: https://github.com/apache/iceberg-rust/issues/34#issuecomment-1942107647 @ZENOTME are you intending on implementing writer support until completion yourself? -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] detect breaking changes [iceberg-python]

2024-02-13 Thread via GitHub
syun64 commented on PR #394: URL: https://github.com/apache/iceberg-python/pull/394#issuecomment-1942088750 | Breakage Kind | Description | Ignore | Rationale

[PR] Update `LICENSE` and `NOTICE` [iceberg-python]

2024-02-13 Thread via GitHub
Fokko opened a new pull request, #413: URL: https://github.com/apache/iceberg-python/pull/413 This reverts commit e9e265a1878a55a4385580bf800382fd75c912c9. If I'm reading [the page right](https://infra.apache.org/licensing-howto.html), the following justifies the NOTICE updates:

Re: [PR] Update `LICENSE` and `NOTICE` [iceberg-python]

2024-02-13 Thread via GitHub
Fokko closed pull request #413: Update `LICENSE` and `NOTICE` URL: https://github.com/apache/iceberg-python/pull/413 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-02-13 Thread via GitHub
jackye1995 commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1488229757 ## open-api/rest-catalog-open-api.yaml: ## @@ -2800,6 +3057,53 @@ components: additionalProperties: type: string +PreplanTableRequest:

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-02-13 Thread via GitHub
jackye1995 commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1488228243 ## open-api/rest-catalog-open-api.yaml: ## @@ -2800,6 +3057,53 @@ components: additionalProperties: type: string +PreplanTableRequest:

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-02-13 Thread via GitHub
jackye1995 commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1488225765 ## open-api/rest-catalog-open-api.yaml: ## @@ -2800,6 +3062,58 @@ components: additionalProperties: type: string +PrePlanTableRequest:

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-02-13 Thread via GitHub
jackye1995 commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1488225063 ## open-api/rest-catalog-open-api.yaml: ## @@ -2800,6 +3062,58 @@ components: additionalProperties: type: string +PrePlanTableRequest:

Re: [PR] Allow mmhash3 v4 [iceberg-python]

2024-02-13 Thread via GitHub
Fokko commented on PR #420: URL: https://github.com/apache/iceberg-python/pull/420#issuecomment-1941975922 I'm actually the maintainer of `mmhash3`, I forked it to support Python 3.11 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

Re: [PR] feat(catalog): add initial rest catalog impl [iceberg-go]

2024-02-13 Thread via GitHub
nastra commented on code in PR #58: URL: https://github.com/apache/iceberg-go/pull/58#discussion_r1488126836 ## catalog/catalog.go: ## @@ -47,19 +52,136 @@ func WithAwsConfig(cfg aws.Config) Option { } } +func WithCredential(cred string) Option { + return func(o

Re: [PR] feat(catalog): add initial rest catalog impl [iceberg-go]

2024-02-13 Thread via GitHub
zeroshade commented on PR #58: URL: https://github.com/apache/iceberg-go/pull/58#issuecomment-1941810329 @nastra Hmm. So, setting the env vars `AWS_REGION`, `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` should all work and get picked up by the FileIO. But I haven't tried testing

Re: [PR] feat(catalog): add initial rest catalog impl [iceberg-go]

2024-02-13 Thread via GitHub
zeroshade commented on code in PR #58: URL: https://github.com/apache/iceberg-go/pull/58#discussion_r1488083197 ## catalog/catalog.go: ## @@ -47,19 +52,136 @@ func WithAwsConfig(cfg aws.Config) Option { } } +func WithCredential(cred string) Option { + return fun

Re: [PR] feat(catalog): add initial rest catalog impl [iceberg-go]

2024-02-13 Thread via GitHub
zeroshade commented on code in PR #58: URL: https://github.com/apache/iceberg-go/pull/58#discussion_r1488074526 ## dev/Dockerfile: ## @@ -0,0 +1,66 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more Review Comment: honestly i was just following the conf

Re: [I] check if table exist [iceberg-python]

2024-02-13 Thread via GitHub
syun64 commented on issue #406: URL: https://github.com/apache/iceberg-python/issues/406#issuecomment-1941783224 I think `table_exists` function that @djouallah proposed and the PR that @hussein-awala is working on to support `CREATE TABLE IF NOT EXISTS` both serve different purposes. And I

[PR] Allow mmhash3 v4 [iceberg-python]

2024-02-13 Thread via GitHub
jonashaag opened a new pull request, #420: URL: https://github.com/apache/iceberg-python/pull/420 Let's give this a try. Unblocks Python 3.12 support on conda-forge -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

Re: [PR] Feat: Add fail_if_exists param to create_table [iceberg-python]

2024-02-13 Thread via GitHub
syun64 commented on PR #415: URL: https://github.com/apache/iceberg-python/pull/415#issuecomment-1941762880 > I think we should have some more discussion around this. I would prefer to have, analog to SQL, a `CREATE OR REPLACE` API. This feels analogous to the SQL `CREATE TABLE IF NOT

Re: [PR] [WIP] Change Append/Overwrite API to accept snapshot properties [iceberg-python]

2024-02-13 Thread via GitHub
Gowthami03B commented on code in PR #419: URL: https://github.com/apache/iceberg-python/pull/419#discussion_r1488029645 ## pyiceberg/table/__init__.py: ## @@ -974,7 +974,7 @@ def name_mapping(self) -> NameMapping: else: return create_mapping_from_schema(sel

Re: [I] InMemoryCatalog's FiloIO in memory map isn't persistent in RestCatalog [iceberg]

2024-02-13 Thread via GitHub
jbonofre commented on issue #9604: URL: https://github.com/apache/iceberg/issues/9604#issuecomment-1941643985 It makes sense to improve the `TestRESTCatalog` with `InMemoryCatalog` backend. -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] [WIP] Migrate Write sub-classes in spark-extensions to JUnit5 and AssertJ style [iceberg]

2024-02-13 Thread via GitHub
tomtongue commented on PR #9670: URL: https://github.com/apache/iceberg/pull/9670#issuecomment-1941476354 Working on the fix of errors. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Add test to capture Duckdb behavior [iceberg-python]

2024-02-13 Thread via GitHub
Fokko merged PR #389: URL: https://github.com/apache/iceberg-python/pull/389 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

  1   2   >