Re: [PR] fix: chrono dep [iceberg-rust]

2024-03-29 Thread via GitHub
liurenjie1024 commented on PR #274: URL: https://github.com/apache/iceberg-rust/pull/274#issuecomment-2026807571 cc @odysa Should we close this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] fix: chrono dep [iceberg-rust]

2024-03-29 Thread via GitHub
Xuanwo commented on PR #274: URL: https://github.com/apache/iceberg-rust/pull/274#issuecomment-2026814405 > I think this could be solved by `cargo update`? This is required to make sure our users depends on `>= 0.4.34` since old versions don't have this new type. -- This is an auto

Re: [PR] fix: chrono dep [iceberg-rust]

2024-03-29 Thread via GitHub
liurenjie1024 commented on PR #274: URL: https://github.com/apache/iceberg-rust/pull/274#issuecomment-2026824492 > > I think this could be solved by `cargo update`? > > > > This is required to make sure our users depends on `>= 0.4.34` since old versions don't have this new ty

Re: [PR] fix: chrono dep [iceberg-rust]

2024-03-29 Thread via GitHub
liurenjie1024 merged PR #274: URL: https://github.com/apache/iceberg-rust/pull/274 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

[PR] Aliyun: Add loading oss file into memory option to OSS client properties [iceberg]

2024-03-29 Thread via GitHub
Zelin1203 opened a new pull request, #10062: URL: https://github.com/apache/iceberg/pull/10062 OSS will close connections that have not sent or received data for more than 1 minute. When the backpressure time of Flink is too long, it will throw an exception and cause the Flink task to crash

Re: [I] [core] Table commit successful but metadata json was never created leaving table in a bad state [iceberg]

2024-03-29 Thread via GitHub
stevenzwu closed issue #9997: [core] Table commit successful but metadata json was never created leaving table in a bad state URL: https://github.com/apache/iceberg/issues/9997 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] [core] fix #9997 - Handle s3a file upload interrupt which results in table metadata pointing to files that doesn't exist [iceberg]

2024-03-29 Thread via GitHub
stevenzwu merged PR #9998: URL: https://github.com/apache/iceberg/pull/9998 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] Flink: backport PR #9321 for range partitioner on map statistics [iceberg]

2024-03-29 Thread via GitHub
stevenzwu commented on PR #10061: URL: https://github.com/apache/iceberg/pull/10061#issuecomment-2027565289 @pvary yes, it is direct copy of new files from 1.17 to 1.16 and 1.18 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-03-29 Thread via GitHub
jqin61 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1544752616 ## pyiceberg/table/__init__.py: ## @@ -2489,16 +2488,28 @@ def _add_and_move_fields( class WriteTask: write_uuid: uuid.UUID task_id: int +schema: Sch

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-03-29 Thread via GitHub
jqin61 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1544754492 ## pyiceberg/table/__init__.py: ## @@ -2526,25 +2537,44 @@ def _dataframe_to_data_files( """ from pyiceberg.io.pyarrow import bin_pack_arrow_table, write_

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-03-29 Thread via GitHub
jqin61 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1544754857 ## pyiceberg/table/__init__.py: ## @@ -2526,25 +2537,44 @@ def _dataframe_to_data_files( """ from pyiceberg.io.pyarrow import bin_pack_arrow_table, write_

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-03-29 Thread via GitHub
jqin61 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1544756546 ## pyiceberg/table/__init__.py: ## @@ -3108,3 +3138,127 @@ def snapshots(self) -> "pa.Table": snapshots, schema=snapshots_schema,

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-03-29 Thread via GitHub
jqin61 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1544770143 ## pyiceberg/table/__init__.py: ## @@ -3108,3 +3138,127 @@ def snapshots(self) -> "pa.Table": snapshots, schema=snapshots_schema,

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-03-29 Thread via GitHub
jqin61 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1544770723 ## pyiceberg/table/__init__.py: ## @@ -3108,3 +3138,127 @@ def snapshots(self) -> "pa.Table": snapshots, schema=snapshots_schema,

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-03-29 Thread via GitHub
jqin61 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1544783412 ## pyiceberg/table/__init__.py: ## @@ -3108,3 +3138,127 @@ def snapshots(self) -> "pa.Table": snapshots, schema=snapshots_schema,

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-03-29 Thread via GitHub
jqin61 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1544785650 ## pyiceberg/table/__init__.py: ## @@ -3108,3 +3138,127 @@ def snapshots(self) -> "pa.Table": snapshots, schema=snapshots_schema,

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-03-29 Thread via GitHub
jqin61 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1544785650 ## pyiceberg/table/__init__.py: ## @@ -3108,3 +3138,127 @@ def snapshots(self) -> "pa.Table": snapshots, schema=snapshots_schema,

Re: [PR] [0.6.x] Backport PR #523 to cast data to iceberg table's pyarrow schema [iceberg-python]

2024-03-29 Thread via GitHub
HonahX commented on PR #559: URL: https://github.com/apache/iceberg-python/pull/559#issuecomment-2027657047 @kevinjqliu I did not include that PR since that adds a new API and I think for a patch release we should not add any new functionality. I added that one to 0.7.0 milestone. WDYT? -

Re: [PR] [0.6.x] Backport PR #523 to cast data to iceberg table's pyarrow schema [iceberg-python]

2024-03-29 Thread via GitHub
kevinjqliu commented on PR #559: URL: https://github.com/apache/iceberg-python/pull/559#issuecomment-2027661302 that makes sense, thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] GCP: Add Iceberg Catalog for GCP BigLake Metastore [iceberg]

2024-03-29 Thread via GitHub
istreeter commented on PR #7412: URL: https://github.com/apache/iceberg/pull/7412#issuecomment-2027773396 I just came across this PR -- it looks like this would be a hugely valuable addition to Iceberg! My company would certainly benefit from this. It seems to have got stuck though.

Re: [PR] Add Pagination To List Apis [iceberg]

2024-03-29 Thread via GitHub
rahil-c commented on code in PR #9782: URL: https://github.com/apache/iceberg/pull/9782#discussion_r1544946200 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -114,6 +114,9 @@ public class RESTSessionCatalog extends BaseViewSessionCatalog private

Re: [PR] Spark: Don't allow branch_ usage with TIMESTAMP AS OF [iceberg]

2024-03-29 Thread via GitHub
singhpk234 commented on code in PR #10059: URL: https://github.com/apache/iceberg/pull/10059#discussion_r1544949819 ## spark/v3.4/spark/src/test/java/org/apache/iceberg/spark/sql/TestSelect.java: ## @@ -448,6 +448,12 @@ public void testInvalidTimeTravelAgainstBranchIdentifierWi

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-29 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1544956648 ## open-api/rest-catalog-open-api.yaml: ## @@ -537,6 +537,113 @@ paths: 5XX: $ref: '#/components/responses/ServerErrorResponse' + /v1/{prefix}/na

Re: [PR] Add Pagination To List Apis [iceberg]

2024-03-29 Thread via GitHub
danielcweeks commented on PR #9782: URL: https://github.com/apache/iceberg/pull/9782#issuecomment-2027829487 I just want to add a few high-level comments here: 1. It looks like `pageSize` is currently required, but I believe the default should be that if it is not set, no pagination shoul

Re: [I] Reading from iceberg table through spark thrift server using jdbc taking larger time [iceberg]

2024-03-29 Thread via GitHub
github-actions[bot] commented on issue #2302: URL: https://github.com/apache/iceberg/issues/2302#issuecomment-2027836401 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Flink: Support nested projection in iceberg table source [iceberg]

2024-03-29 Thread via GitHub
github-actions[bot] commented on issue #2312: URL: https://github.com/apache/iceberg/issues/2312#issuecomment-2027836421 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Flink: Provide the fully covered SQL integration test cases to verify the data correctness. [iceberg]

2024-03-29 Thread via GitHub
github-actions[bot] commented on issue #2313: URL: https://github.com/apache/iceberg/issues/2313#issuecomment-2027836433 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Error while evolving partition column of a table [iceberg]

2024-03-29 Thread via GitHub
github-actions[bot] commented on issue #2327: URL: https://github.com/apache/iceberg/issues/2327#issuecomment-2027836446 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Merge into is Error。Error:java.lang.NoSuchMethodError [iceberg]

2024-03-29 Thread via GitHub
github-actions[bot] commented on issue #2344: URL: https://github.com/apache/iceberg/issues/2344#issuecomment-2027836466 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] how to set Catalogs in structured streaming local model [iceberg]

2024-03-29 Thread via GitHub
github-actions[bot] commented on issue #2322: URL: https://github.com/apache/iceberg/issues/2322#issuecomment-2027836441 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Test: Add row-level API for TestTable to make the unit test more easy. [iceberg]

2024-03-29 Thread via GitHub
github-actions[bot] commented on issue #2334: URL: https://github.com/apache/iceberg/issues/2334#issuecomment-2027836459 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Add Pagination To List Apis [iceberg]

2024-03-29 Thread via GitHub
danielcweeks commented on code in PR #9782: URL: https://github.com/apache/iceberg/pull/9782#discussion_r1545017898 ## core/src/main/java/org/apache/iceberg/rest/PaginatedList.java: ## @@ -0,0 +1,269 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

[PR] Hive: Check e.getMessage() is not null [iceberg]

2024-03-29 Thread via GitHub
lurnagao-dahua opened a new pull request, #10069: URL: https://github.com/apache/iceberg/pull/10069 Hi, I get an Metadata not found in metadata location for table error when trying to query the data generated by flink flink: 1.16.2 iceberg: 1.4.3 The reason is that in some cases

Re: [PR] Spark: Don't allow branch_ usage with TIMESTAMP AS OF [iceberg]

2024-03-29 Thread via GitHub
amogh-jahagirdar commented on code in PR #10059: URL: https://github.com/apache/iceberg/pull/10059#discussion_r1545052196 ## spark/v3.4/spark/src/test/java/org/apache/iceberg/spark/sql/TestSelect.java: ## @@ -448,6 +448,12 @@ public void testInvalidTimeTravelAgainstBranchIdenti

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-29 Thread via GitHub
amogh-jahagirdar commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1545049249 ## open-api/rest-catalog-open-api.yaml: ## @@ -2838,6 +2991,76 @@ components: additionalProperties: type: string +PreplanTableReq

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-29 Thread via GitHub
amogh-jahagirdar commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1545072167 ## open-api/rest-catalog-open-api.yaml: ## @@ -537,6 +537,113 @@ paths: 5XX: $ref: '#/components/responses/ServerErrorResponse' + /v1/{p

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-29 Thread via GitHub
amogh-jahagirdar commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1545072167 ## open-api/rest-catalog-open-api.yaml: ## @@ -537,6 +537,113 @@ paths: 5XX: $ref: '#/components/responses/ServerErrorResponse' + /v1/{p