Re: [PR] Flink: Bump minor versions [iceberg]

2024-03-11 Thread via GitHub
ajantha-bhat commented on PR #9875: URL: https://github.com/apache/iceberg/pull/9875#issuecomment-1990903882 cc: @Fokko, @nastra -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] Add issue template and docs for iceberg proposals [iceberg]

2024-03-11 Thread via GitHub
jbonofre commented on PR #9932: URL: https://github.com/apache/iceberg/pull/9932#issuecomment-1990901054 @danielcweeks That's because my user is not in the `collaborators` list in `.asf.yml`. Let me check with asfinfra if we can have more than 10 entries in `collaborators`. -- This is an

Re: [PR] Docs: Enhance create_changelog_view usage [iceberg]

2024-03-11 Thread via GitHub
manuzhang commented on code in PR #9889: URL: https://github.com/apache/iceberg/pull/9889#discussion_r1520932990 ## docs/docs/spark-procedures.md: ## @@ -823,6 +823,8 @@ second snapshot deleted 1 record. |2 | Bob |INSERT |0 |5390529835796506035| |1 | Ali

Re: [PR] Docs: Enhance create_changelog_view usage [iceberg]

2024-03-11 Thread via GitHub
manuzhang commented on code in PR #9889: URL: https://github.com/apache/iceberg/pull/9889#discussion_r1520931489 ## docs/docs/spark-procedures.md: ## @@ -756,8 +756,8 @@ Creates a view that contains the changes from a given table. | `table` | ✔️ | string

Re: [PR] Hive: Add View support for HIVE catalog [iceberg]

2024-03-11 Thread via GitHub
nk1506 commented on code in PR #9852: URL: https://github.com/apache/iceberg/pull/9852#discussion_r1520927889 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -250,29 +368,53 @@ public void renameTable(TableIdentifier from, TableIdentifier origina

Re: [PR] [WIP][Discussion]CreateTableTransaction Implementation [iceberg-python]

2024-03-11 Thread via GitHub
HonahX commented on code in PR #498: URL: https://github.com/apache/iceberg-python/pull/498#discussion_r1520912688 ## pyiceberg/table/metadata.py: ## @@ -287,6 +287,31 @@ def _generate_snapshot_id() -> int: return snapshot_id +class IncompleteTableMetadata(TableMetadata

Re: [PR] Hive: Add View support for HIVE catalog [iceberg]

2024-03-11 Thread via GitHub
nk1506 commented on code in PR #9852: URL: https://github.com/apache/iceberg/pull/9852#discussion_r1520888231 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveOperationsBase.java: ## @@ -181,4 +296,214 @@ default Table newHmsTable(String hmsTableOwner) { return

Re: [PR] Hive: Add View support for HIVE catalog [iceberg]

2024-03-11 Thread via GitHub
nk1506 commented on code in PR #9852: URL: https://github.com/apache/iceberg/pull/9852#discussion_r1520875974 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveOperationsBase.java: ## @@ -181,4 +264,220 @@ default Table newHmsTable(String hmsTableOwner) { return

Re: [PR] feat: implement OAuth for catalog rest client [iceberg-rust]

2024-03-11 Thread via GitHub
TennyZhuang commented on PR #254: URL: https://github.com/apache/iceberg-rust/pull/254#issuecomment-1990529796 > Is there any way to extend the Dockerfile with the Env for credentials? I can't find any codes about OAuth in the image. Only an open issue seems related. https://gi

Re: [PR] feat: implement OAuth for catalog rest client [iceberg-rust]

2024-03-11 Thread via GitHub
TennyZhuang commented on code in PR #254: URL: https://github.com/apache/iceberg-rust/pull/254#discussion_r1520864738 ## crates/catalog/rest/src/catalog.rs: ## @@ -497,13 +516,56 @@ impl RestCatalog { client: config.try_create_rest_client()?, config,

Re: [PR] feat: implement OAuth for catalog rest client [iceberg-rust]

2024-03-11 Thread via GitHub
marvinlanhenke commented on PR #254: URL: https://github.com/apache/iceberg-rust/pull/254#issuecomment-1990440706 > Resolve #238 > > The docker image of tabular doesn't support credential settings, so it's impossible to write a test. > > I just test it manually using their hos

Re: [PR] Build: Bump pypa/cibuildwheel from 2.16.5 to 2.17.0 [iceberg-python]

2024-03-11 Thread via GitHub
HonahX merged PR #517: URL: https://github.com/apache/iceberg-python/pull/517 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Docs: Fix release notes indentation [iceberg]

2024-03-11 Thread via GitHub
Fokko merged PR #9933: URL: https://github.com/apache/iceberg/pull/9933 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [PR] Hive: Add View support for HIVE catalog [iceberg]

2024-03-11 Thread via GitHub
nk1506 commented on code in PR #9852: URL: https://github.com/apache/iceberg/pull/9852#discussion_r1520828123 ## core/src/main/java/org/apache/iceberg/BaseMetadata.java: ## @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributo

[I] OSSFileIO not compatible with aliyun-sdk-oss higher than 3.11.3 [iceberg]

2024-03-11 Thread via GitHub
769484623 opened a new issue, #9934: URL: https://github.com/apache/iceberg/issues/9934 ### Apache Iceberg version 1.4.3 ### Query engine Flink ### Please describe the bug 🐞 ```java import org.apache.hadoop.conf.Configuration; import org.apache.iceberg

Re: [I] iceberg-core: Switch tests to JUnit5 + AssertJ-style assertions [iceberg]

2024-03-11 Thread via GitHub
tomtongue commented on issue #9085: URL: https://github.com/apache/iceberg/issues/9085#issuecomment-1990115699 @nastra If possible, could you assign this issue with me? I'll have the owenership for this core migration. -- This is an automated message from the Apache Git Service. To respon

Re: [I] feat: Binding unbound expression to bound expression. [iceberg-rust]

2024-03-11 Thread via GitHub
liurenjie1024 closed issue #149: feat: Binding unbound expression to bound expression. URL: https://github.com/apache/iceberg-rust/issues/149 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] feat: Implement binding expression [iceberg-rust]

2024-03-11 Thread via GitHub
liurenjie1024 merged PR #231: URL: https://github.com/apache/iceberg-rust/pull/231 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [I] Add hive metastore catalog support [iceberg-rust]

2024-03-11 Thread via GitHub
liurenjie1024 commented on issue #113: URL: https://github.com/apache/iceberg-rust/issues/113#issuecomment-1990044567 @marvinlanhenke Thanks for the work. I'll also try to work on setting up `TableMetadataBuilder` this week. -- This is an automated message from the Apache Git Service. To

Re: [PR] feat: Implement binding expression [iceberg-rust]

2024-03-11 Thread via GitHub
liurenjie1024 commented on PR #231: URL: https://github.com/apache/iceberg-rust/pull/231#issuecomment-1990023145 > Let's say that you have a table with number: i32 column. At some point, you'll evolve this column to number: i64. New data will be written as a long, but the old data will stil

Re: [PR] Parquet, Arrow: Refactor vectorized reader [iceberg]

2024-03-11 Thread via GitHub
wgtmac commented on PR #9772: URL: https://github.com/apache/iceberg/pull/9772#issuecomment-1989767289 @anthonysgro Yes, that is on my radar. I will work on it once v2 encodings of all primitive types have been supported. -- This is an automated message from the Apache Git Service. To res

Re: [PR] Add issue template and docs for iceberg proposals [iceberg]

2024-03-11 Thread via GitHub
liurenjie1024 commented on code in PR #9932: URL: https://github.com/apache/iceberg/pull/9932#discussion_r1520676297 ## .github/ISSUE_TEMPLATE/iceberg_proposal.yml: ## @@ -0,0 +1,50 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor licens

Re: [PR] Add issue template and docs for iceberg proposals [iceberg]

2024-03-11 Thread via GitHub
ajantha-bhat commented on PR #9932: URL: https://github.com/apache/iceberg/pull/9932#issuecomment-1989736901 > @jbonofre I'm not sure why I can't add you as a reviewer. What I observed is only collaborators, committers can be added as reviewer. https://github.com/apache/iceberg/blob

[PR] Docs: Fix release notes indentation [iceberg]

2024-03-11 Thread via GitHub
ajantha-bhat opened a new pull request, #9933: URL: https://github.com/apache/iceberg/pull/9933 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] Add issue template and docs for iceberg proposals [iceberg]

2024-03-11 Thread via GitHub
danielcweeks commented on PR #9932: URL: https://github.com/apache/iceberg/pull/9932#issuecomment-1989711615 @jbonofre I'm not sure why I can't add you as a reviewer. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [I] Catalogs Do Not Easily Support Full State Rollback [iceberg]

2024-03-11 Thread via GitHub
github-actions[bot] commented on issue #1944: URL: https://github.com/apache/iceberg/issues/1944#issuecomment-1989676374 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Add in support for vectorized parquet reads on data files with row position (_pos) in the metadata column [iceberg]

2024-03-11 Thread via GitHub
github-actions[bot] commented on issue #1540: URL: https://github.com/apache/iceberg/issues/1540#issuecomment-1989676204 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Add in support for vectorized parquet reads on data files with row position (_pos) in the metadata column [iceberg]

2024-03-11 Thread via GitHub
github-actions[bot] closed issue #1540: Add in support for vectorized parquet reads on data files with row position (_pos) in the metadata column URL: https://github.com/apache/iceberg/issues/1540 -- This is an automated message from the Apache Git Service. To respond to the message, please l

Re: [I] Error when creating a table `InvalidObjectException` [iceberg]

2024-03-11 Thread via GitHub
github-actions[bot] commented on issue #1533: URL: https://github.com/apache/iceberg/issues/1533#issuecomment-1989676186 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Error when creating a table `InvalidObjectException` [iceberg]

2024-03-11 Thread via GitHub
github-actions[bot] closed issue #1533: Error when creating a table `InvalidObjectException` URL: https://github.com/apache/iceberg/issues/1533 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [I] Reading snapshot of table uses current schema [iceberg]

2024-03-11 Thread via GitHub
github-actions[bot] commented on issue #1501: URL: https://github.com/apache/iceberg/issues/1501#issuecomment-1989676167 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Reading snapshot of table uses current schema [iceberg]

2024-03-11 Thread via GitHub
github-actions[bot] closed issue #1501: Reading snapshot of table uses current schema URL: https://github.com/apache/iceberg/issues/1501 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] Read custom metadata on iceberg snapshots via spark options [iceberg]

2024-03-11 Thread via GitHub
github-actions[bot] closed issue #1500: Read custom metadata on iceberg snapshots via spark options URL: https://github.com/apache/iceberg/issues/1500 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] Update version-hint.txt atomically [iceberg]

2024-03-11 Thread via GitHub
github-actions[bot] commented on issue #1496: URL: https://github.com/apache/iceberg/issues/1496#issuecomment-1989676127 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Update version-hint.txt atomically [iceberg]

2024-03-11 Thread via GitHub
github-actions[bot] closed issue #1496: Update version-hint.txt atomically URL: https://github.com/apache/iceberg/issues/1496 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [I] Read custom metadata on iceberg snapshots via spark options [iceberg]

2024-03-11 Thread via GitHub
github-actions[bot] commented on issue #1500: URL: https://github.com/apache/iceberg/issues/1500#issuecomment-1989676150 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Custom partition functions [iceberg]

2024-03-11 Thread via GitHub
github-actions[bot] commented on issue #1482: URL: https://github.com/apache/iceberg/issues/1482#issuecomment-1989676097 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Spark 3.5: Fix system function pushdown in CoW row-level commands [iceberg]

2024-03-11 Thread via GitHub
aokolnychyi commented on PR #9873: URL: https://github.com/apache/iceberg/pull/9873#issuecomment-1989658865 Thanks, @rdblue! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Spark 3.5: Fix system function pushdown in CoW row-level commands [iceberg]

2024-03-11 Thread via GitHub
aokolnychyi merged PR #9873: URL: https://github.com/apache/iceberg/pull/9873 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

[PR] Build: Bump pypa/cibuildwheel from 2.16.5 to 2.17.0 [iceberg-python]

2024-03-11 Thread via GitHub
dependabot[bot] opened a new pull request, #517: URL: https://github.com/apache/iceberg-python/pull/517 Bumps [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel) from 2.16.5 to 2.17.0. Release notes Sourced from https://github.com/pypa/cibuildwheel/releases";>pypa/cibuildwhee

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1520474875 ## open-api/rest-catalog-open-api.yaml: ## @@ -537,6 +537,104 @@ paths: 5XX: $ref: '#/components/responses/ServerErrorResponse' + /v1/{prefix}/nam

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1520474556 ## open-api/rest-catalog-open-api.yaml: ## @@ -2106,6 +2204,28 @@ components: items: $ref: '#/components/schemas/PartitionStatisticsFile' +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1520473745 ## open-api/rest-catalog-open-api.yaml: ## @@ -2106,6 +2204,28 @@ components: items: $ref: '#/components/schemas/PartitionStatisticsFile' +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1520472141 ## open-api/rest-catalog-open-api.yaml: ## @@ -2838,6 +2978,63 @@ components: additionalProperties: type: string +PreplanTableRequest: +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1520471915 ## open-api/rest-catalog-open-api.yaml: ## @@ -2838,6 +2978,63 @@ components: additionalProperties: type: string +PreplanTableRequest: +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1520470991 ## open-api/rest-catalog-open-api.yaml: ## @@ -2838,6 +2978,63 @@ components: additionalProperties: type: string +PreplanTableRequest: +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1520470429 ## open-api/rest-catalog-open-api.yaml: ## @@ -2838,6 +2978,63 @@ components: additionalProperties: type: string +PreplanTableRequest: +

Re: [PR] Hive: Add View support for HIVE catalog [iceberg]

2024-03-11 Thread via GitHub
pvary commented on code in PR #9852: URL: https://github.com/apache/iceberg/pull/9852#discussion_r1520469971 ## core/src/main/java/org/apache/iceberg/BaseMetastoreTableOperations.java: ## @@ -309,65 +304,20 @@ protected enum CommitStatus { * @return Commit Status of Success,

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1520467411 ## open-api/rest-catalog-open-api.yaml: ## @@ -2838,6 +2978,63 @@ components: additionalProperties: type: string +PreplanTableRequest: +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1520466297 ## open-api/rest-catalog-open-api.yaml: ## @@ -2838,6 +2978,63 @@ components: additionalProperties: type: string +PreplanTableRequest: +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1520465666 ## open-api/rest-catalog-open-api.yaml: ## @@ -2838,6 +2978,63 @@ components: additionalProperties: type: string +PreplanTableRequest: +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1520464769 ## open-api/rest-catalog-open-api.yaml: ## @@ -532,6 +532,100 @@ paths: 5XX: $ref: '#/components/responses/ServerErrorResponse' + /v1/{prefix}/nam

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1520463982 ## open-api/rest-catalog-open-api.yaml: ## @@ -3804,6 +4016,21 @@ components: } } +UnprocessableEntityResponse: + description: +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1520462916 ## open-api/rest-catalog-open-api.yaml: ## @@ -3789,6 +3986,21 @@ components: EmptyResponse: $ref: '#/components/examples/ListNamespacesEmpty

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1520462117 ## open-api/rest-catalog-open-api.yaml: ## @@ -3789,6 +3986,21 @@ components: EmptyResponse: $ref: '#/components/examples/ListNamespacesEmpty

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1520455028 ## open-api/rest-catalog-open-api.yaml: ## @@ -2068,6 +2162,145 @@ components: items: $ref: '#/components/schemas/PartitionStatisticsFile' +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1520454032 ## open-api/rest-catalog-open-api.yaml: ## @@ -2068,6 +2162,145 @@ components: items: $ref: '#/components/schemas/PartitionStatisticsFile' +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1520453171 ## open-api/rest-catalog-open-api.yaml: ## @@ -537,6 +537,104 @@ paths: 5XX: $ref: '#/components/responses/ServerErrorResponse' + /v1/{prefix}/nam

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1520451408 ## open-api/rest-catalog-open-api.yaml: ## @@ -537,6 +537,104 @@ paths: 5XX: $ref: '#/components/responses/ServerErrorResponse' + /v1/{prefix}/nam

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1520451266 ## open-api/rest-catalog-open-api.yaml: ## @@ -537,6 +537,108 @@ paths: 5XX: $ref: '#/components/responses/ServerErrorResponse' + /v1/{prefix}/nam

Re: [PR] Feature make oauth `audience` configurable [iceberg]

2024-03-11 Thread via GitHub
flyrain merged PR #9839: URL: https://github.com/apache/iceberg/pull/9839 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Update site to 1.5.0 docs [iceberg]

2024-03-11 Thread via GitHub
Fokko merged PR #9931: URL: https://github.com/apache/iceberg/pull/9931 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [PR] Add Javadoc for Iceberg release version 1.5.0 [iceberg]

2024-03-11 Thread via GitHub
Fokko merged PR #9930: URL: https://github.com/apache/iceberg/pull/9930 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1520444991 ## open-api/rest-catalog-open-api.yaml: ## @@ -2068,6 +2162,145 @@ components: items: $ref: '#/components/schemas/PartitionStatisticsFile' +

Re: [PR] Hive: Add View support for HIVE catalog [iceberg]

2024-03-11 Thread via GitHub
szehon-ho commented on code in PR #9852: URL: https://github.com/apache/iceberg/pull/9852#discussion_r1520424415 ## core/src/main/java/org/apache/iceberg/BaseMetastoreTableOperations.java: ## @@ -309,65 +304,20 @@ protected enum CommitStatus { * @return Commit Status of Succ

Re: [PR] Hive: Add View support for HIVE catalog [iceberg]

2024-03-11 Thread via GitHub
szehon-ho commented on code in PR #9852: URL: https://github.com/apache/iceberg/pull/9852#discussion_r1520424415 ## core/src/main/java/org/apache/iceberg/BaseMetastoreTableOperations.java: ## @@ -309,65 +304,20 @@ protected enum CommitStatus { * @return Commit Status of Succ

Re: [PR] Hive: Add View support for HIVE catalog [iceberg]

2024-03-11 Thread via GitHub
szehon-ho commented on code in PR #9852: URL: https://github.com/apache/iceberg/pull/9852#discussion_r1520440313 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveOperationsBase.java: ## @@ -181,4 +264,220 @@ default Table newHmsTable(String hmsTableOwner) { ret

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1520435413 ## api/src/test/java/org/apache/iceberg/expressions/TestStringLiteralConversions.java: ## @@ -132,6 +132,38 @@ public void testStringToTimestampLiteral() { .isE

Re: [PR] Hive: Add View support for HIVE catalog [iceberg]

2024-03-11 Thread via GitHub
szehon-ho commented on code in PR #9852: URL: https://github.com/apache/iceberg/pull/9852#discussion_r1520430836 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -222,24 +220,144 @@ public boolean dropTable(TableIdentifier identifier, boolean purg

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1520434845 ## api/src/test/java/org/apache/iceberg/expressions/TestStringLiteralConversions.java: ## @@ -132,6 +132,38 @@ public void testStringToTimestampLiteral() { .isE

Re: [PR] Feature make oauth `audience` configurable [iceberg]

2024-03-11 Thread via GitHub
flyrain commented on PR #9839: URL: https://github.com/apache/iceberg/pull/9839#issuecomment-1989448970 +1 Thanks @himadripal for the PR. Thanks @nastra for the review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1520433728 ## api/src/test/java/org/apache/iceberg/transforms/TestBucketing.java: ## @@ -107,17 +107,65 @@ public void testSpecValues() { .isEqualTo(-662762989); Li

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1520429456 ## api/src/main/java/org/apache/iceberg/expressions/Literals.java: ## @@ -298,7 +299,7 @@ public Literal to(Type type) { case TIME: return (Literal)

Re: [PR] Adding 1.5.0 to the docs branch [iceberg]

2024-03-11 Thread via GitHub
Fokko merged PR #9929: URL: https://github.com/apache/iceberg/pull/9929 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [I] Calling `rewrite_position_delete_files` rewrites into same amount of files [iceberg]

2024-03-11 Thread via GitHub
bk-mz commented on issue #9833: URL: https://github.com/apache/iceberg/issues/9833#issuecomment-1989436414 I.e. ![image](https://github.com/apache/iceberg/assets/892781/b6678068-243c-41ea-8aa2-a9f82e836407) Odd folder 475031 -> 1710111600 -> Sun Mar 10 2024 23:00:00 GMT+

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1520419670 ## api/src/main/java/org/apache/iceberg/expressions/Literals.java: ## @@ -501,19 +538,17 @@ public Literal to(Type type) { return (Literal) new TimeLiteral(t

Re: [I] Calling `rewrite_position_delete_files` rewrites into same amount of files [iceberg]

2024-03-11 Thread via GitHub
bk-mz commented on issue #9833: URL: https://github.com/apache/iceberg/issues/9833#issuecomment-1989424746 >do the files have incorrect partition tuples (the values actually used for filtering)? how do I check this? Something is odd, because I see bloated small-file partitions, i.

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1520417598 ## api/src/main/java/org/apache/iceberg/util/DateTimeUtil.java: ## @@ -120,19 +176,32 @@ public static long isoTimeToMicros(String timeString) { } public static

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1520417082 ## api/src/main/java/org/apache/iceberg/util/DateTimeUtil.java: ## @@ -120,19 +176,32 @@ public static long isoTimeToMicros(String timeString) { } public static

Re: [PR] feat: Implement binding expression [iceberg-rust]

2024-03-11 Thread via GitHub
Fokko commented on code in PR #231: URL: https://github.com/apache/iceberg-rust/pull/231#discussion_r1520407376 ## crates/iceberg/src/expr/predicate.rs: ## @@ -172,6 +223,146 @@ pub enum Predicate { Set(SetExpression), } +impl Bind for Predicate { +type Bound = Bound

[PR] Add Javadoc for Iceberg release version 1.5.0 [iceberg]

2024-03-11 Thread via GitHub
bitsondatadev opened a new pull request, #9930: URL: https://github.com/apache/iceberg/pull/9930 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1520412384 ## api/src/main/java/org/apache/iceberg/util/DateTimeUtil.java: ## @@ -106,11 +146,27 @@ public static String microsToIsoTimestamptz(long micros) { return localDate

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1520403209 ## api/src/main/java/org/apache/iceberg/expressions/Literals.java: ## @@ -426,23 +427,55 @@ protected Type.TypeID typeId() { } static class TimestampLiteral exte

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1520398143 ## api/src/main/java/org/apache/iceberg/expressions/Literals.java: ## @@ -501,19 +538,17 @@ public Literal to(Type type) { return (Literal) new TimeLiteral(t

Re: [I] Accessing Iceberg tables without catalog [iceberg]

2024-03-11 Thread via GitHub
jarias1 commented on issue #5512: URL: https://github.com/apache/iceberg/issues/5512#issuecomment-1989380058 it's been almost a year, can't we read Iceberg tables from S3 path (without catalog) yet? -- This is an automated message from the Apache Git Service. To respond to the message, pl

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1520384546 ## api/src/main/java/org/apache/iceberg/expressions/ExpressionUtil.java: ## @@ -595,11 +613,17 @@ private static String sanitizeString(CharSequence value, long now, int

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

2024-03-11 Thread via GitHub
Fokko commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1520360025 ## api/src/main/java/org/apache/iceberg/transforms/Days.java: ## @@ -55,14 +56,15 @@ public boolean satisfiesOrderOf(Transform other) { } if (other instanceof

Re: [PR] Spark 3.5: Fix system function pushdown in CoW row-level commands [iceberg]

2024-03-11 Thread via GitHub
aokolnychyi commented on code in PR #9873: URL: https://github.com/apache/iceberg/pull/9873#discussion_r1520370692 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/SparkPlanUtil.java: ## @@ -53,6 +58,49 @@ private static SparkPlan actualPlan(Spark

Re: [PR] Adding 1.5.0 to the docs branch [iceberg]

2024-03-11 Thread via GitHub
bitsondatadev commented on PR #9929: URL: https://github.com/apache/iceberg/pull/9929#issuecomment-1989331209 Hold up, I didn't update the search indexes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[PR] Adding 1.5.0 to the docs branch [iceberg]

2024-03-11 Thread via GitHub
bitsondatadev opened a new pull request, #9929: URL: https://github.com/apache/iceberg/pull/9929 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

Re: [PR] Spark 3.5: Fix system function pushdown in CoW row-level commands [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9873: URL: https://github.com/apache/iceberg/pull/9873#discussion_r1520350448 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/SparkPlanUtil.java: ## @@ -53,6 +58,49 @@ private static SparkPlan actualPlan(SparkPlan

Re: [PR] feat: implement OAuth for catalog rest client [iceberg-rust]

2024-03-11 Thread via GitHub
himadripal commented on code in PR #254: URL: https://github.com/apache/iceberg-rust/pull/254#discussion_r1520307628 ## crates/catalog/rest/src/catalog.rs: ## @@ -497,13 +516,56 @@ impl RestCatalog { client: config.try_create_rest_client()?, config,

Re: [PR] Parquet, Arrow: Refactor vectorized reader [iceberg]

2024-03-11 Thread via GitHub
anthonysgro commented on PR #9772: URL: https://github.com/apache/iceberg/pull/9772#issuecomment-1989273962 @wgtmac thanks so much for taking this issue on! I am just a user of iceberg (I have never contributed), but I was curious--is it also possible to include a resolution for https://git

Re: [PR] Fix pagination description based on new feedback [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9917: URL: https://github.com/apache/iceberg/pull/9917#discussion_r1520320944 ## open-api/rest-catalog-open-api.yaml: ## @@ -1610,13 +1610,27 @@ components: PageToken: description: -An opaque token which allows clients to mak

Re: [PR] Fix pagination description based on new feedback [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9917: URL: https://github.com/apache/iceberg/pull/9917#discussion_r1520319849 ## open-api/rest-catalog-open-api.yaml: ## @@ -1610,13 +1610,27 @@ components: PageToken: description: -An opaque token which allows clients to mak

Re: [PR] [WIP] Add `PartitionEvaluator` to allow filtering of files in a table scan (Issue #152) [iceberg-rust]

2024-03-11 Thread via GitHub
sdd commented on code in PR #241: URL: https://github.com/apache/iceberg-rust/pull/241#discussion_r1520319219 ## crates/iceberg/src/scan.rs: ## @@ -52,6 +62,18 @@ impl<'a> TableScanBuilder<'a> { self } +/// Sets the scan as being case-insensitive +pub fn

Re: [PR] Fix pagination description based on new feedback [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9917: URL: https://github.com/apache/iceberg/pull/9917#discussion_r1520319098 ## open-api/rest-catalog-open-api.yaml: ## @@ -1610,13 +1610,27 @@ components: PageToken: description: -An opaque token which allows clients to mak

Re: [PR] Fix pagination description based on new feedback [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9917: URL: https://github.com/apache/iceberg/pull/9917#discussion_r1520316991 ## open-api/rest-catalog-open-api.yaml: ## @@ -1610,13 +1610,27 @@ components: PageToken: description: -An opaque token which allows clients to mak

Re: [I] Iceberg containing Parquet v2 files cannot be read unless read.parquet.vectorization.enabled is set to false [iceberg]

2024-03-11 Thread via GitHub
anthonysgro commented on issue #8430: URL: https://github.com/apache/iceberg/issues/8430#issuecomment-1989262118 I believe this is already being tracked here: https://github.com/apache/iceberg/issues/7162 -- This is an automated message from the Apache Git Service. To respond to the messa

Re: [PR] Views, Spark: Add support for Materialized Views; Integrate with Spark SQL [iceberg]

2024-03-11 Thread via GitHub
rdblue commented on code in PR #9830: URL: https://github.com/apache/iceberg/pull/9830#discussion_r1520310556 ## core/src/main/java/org/apache/iceberg/view/ViewVersionReplace.java: ## @@ -56,6 +57,14 @@ public ViewVersion apply() { } ViewMetadata internalApply() { +/

  1   2   3   >