Re: [PR] Use the correct spec when rewiting existing manifests [iceberg-python]

2024-09-11 Thread via GitHub
Fokko commented on code in PR #1157: URL: https://github.com/apache/iceberg-python/pull/1157#discussion_r1755433036 ## tests/integration/test_writes/test_writes.py: ## @@ -1409,3 +1399,52 @@ def test_delete_threshold() -> None: assert len(iceberg_table.scan(row_filter=Not(

Re: [PR] Bump boto3 from 1.35.7 to 1.35.16 [iceberg-python]

2024-09-11 Thread via GitHub
Fokko merged PR #1159: URL: https://github.com/apache/iceberg-python/pull/1159 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Bump pydantic from 2.9.0 to 2.9.1 [iceberg-python]

2024-09-11 Thread via GitHub
Fokko merged PR #1154: URL: https://github.com/apache/iceberg-python/pull/1154 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] OpenAPI: Standardize credentials in loadTable/loadView responses [iceberg]

2024-09-11 Thread via GitHub
flyrain commented on code in PR #10722: URL: https://github.com/apache/iceberg/pull/10722#discussion_r1755461632 ## open-api/rest-catalog-open-api.yaml: ## @@ -3129,6 +3204,11 @@ components: - `s3.secret-access-key`: secret for credentials that provide access to data

Re: [PR] Remove unnecessary _ensure_tables_exist method [iceberg-python]

2024-09-11 Thread via GitHub
isc-patrick commented on PR #1155: URL: https://github.com/apache/iceberg-python/pull/1155#issuecomment-2344540314 It does not look like CREATE TABLE IF NOT EXISTS is what is used. It is specific to each dialect and how they implement the has_table method, but the ones I looked at use metad

Re: [PR] OpenAPI: Standardize credentials in loadTable/loadView responses [iceberg]

2024-09-11 Thread via GitHub
snazy commented on code in PR #10722: URL: https://github.com/apache/iceberg/pull/10722#discussion_r1755464604 ## open-api/rest-catalog-open-api.yaml: ## @@ -3103,6 +3103,81 @@ components: uuid: type: string +ADLSCredentials: + type: object +

[I] Streaming read from Iceberg table in S3 cause checkpoint related error [iceberg]

2024-09-11 Thread via GitHub
Giangblackk opened a new issue, #3: URL: https://github.com/apache/iceberg/issues/3 ### Apache Iceberg version 1.6.1 (latest release) ### Query engine Spark ### Please describe the bug 🐞 I faced the error when streaming read from an Iceberg table in

[I] PyIceberg - MetaException(message='java.lang.IllegalArgumentException: bucket is null/empty') [iceberg-python]

2024-09-11 Thread via GitHub
malopezh opened a new issue, #1165: URL: https://github.com/apache/iceberg-python/issues/1165 ### Apache Iceberg version 0.7.1 (latest release) ### Please describe the bug 🐞 Problem: Trying to create table in OCI Object Storage. Metadata is successfully created but

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-09-11 Thread via GitHub
brunsgaard commented on PR #11039: URL: https://github.com/apache/iceberg/pull/11039#issuecomment-2344644652 Hi @hesham-medhat , I assume this PR will also enable read/write support for Flink pipelines that use BigLake external tables with BigQuery? -- This is an automated message from

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-09-11 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r178263 ## build.gradle: ## @@ -647,20 +647,43 @@ project(':iceberg-gcp') { implementation project(':iceberg-common') implementation project(':iceberg-core') +

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-09-11 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1755564516 ## gcp/src/main/java/org/apache/iceberg/gcp/bigquery/BigQueryClientImpl.java: ## @@ -0,0 +1,481 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-09-11 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1755570976 ## gcp/src/main/java/org/apache/iceberg/gcp/bigquery/BigQueryClientImpl.java: ## @@ -0,0 +1,481 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-09-11 Thread via GitHub
rdblue commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1755571530 ## gcp/src/main/java/org/apache/iceberg/gcp/bigquery/BigQueryClientImpl.java: ## @@ -0,0 +1,481 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

Re: [PR] Spark 3.4: Action to compute table stats [iceberg]

2024-09-11 Thread via GitHub
dramaticlly commented on code in PR #11106: URL: https://github.com/apache/iceberg/pull/11106#discussion_r1755572668 ## spark/v3.4/spark/src/main/java/org/apache/iceberg/spark/actions/NDVSketchUtil.java: ## @@ -0,0 +1,93 @@ +/* + * Licensed to the Apache Software Foundation (ASF

Re: [PR] Use the correct spec when rewiting existing manifests [iceberg-python]

2024-09-11 Thread via GitHub
sungwy merged PR #1157: URL: https://github.com/apache/iceberg-python/pull/1157 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@icebe

Re: [I] Issue when overwriting data with row filter [iceberg-python]

2024-09-11 Thread via GitHub
sungwy closed issue #1108: Issue when overwriting data with row filter URL: https://github.com/apache/iceberg-python/issues/1108 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] Issue when overwriting data with row filter [iceberg-python]

2024-09-11 Thread via GitHub
sungwy closed issue #1108: Issue when overwriting data with row filter URL: https://github.com/apache/iceberg-python/issues/1108 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] Streaming read from Iceberg table in S3 cause checkpoint related error [iceberg]

2024-09-11 Thread via GitHub
singhpk234 commented on issue #3: URL: https://github.com/apache/iceberg/issues/3#issuecomment-2344753500 we should be using ResolvingFileIO instead of S3FileIO, this is very reason why we did introduce that in the first place. details : https://github.com/apache/iceberg/pull/359

Re: [PR] Spec: Support geo type [iceberg]

2024-09-11 Thread via GitHub
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1755699062 ## format/spec.md: ## @@ -444,28 +456,28 @@ The schema of a manifest file is a struct called `manifest_entry` with the follo `data_file` is a struct with the fo

Re: [PR] Spec: Support geo type [iceberg]

2024-09-11 Thread via GitHub
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1755699062 ## format/spec.md: ## @@ -444,28 +456,28 @@ The schema of a manifest file is a struct called `manifest_entry` with the follo `data_file` is a struct with the fo

Re: [PR] Spec: Support geo type [iceberg]

2024-09-11 Thread via GitHub
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1755702448 ## format/spec.md: ## @@ -444,28 +456,28 @@ The schema of a manifest file is a struct called `manifest_entry` with the follo `data_file` is a struct with the fo

Re: [PR] Spec: Support geo type [iceberg]

2024-09-11 Thread via GitHub
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1755699062 ## format/spec.md: ## @@ -444,28 +456,28 @@ The schema of a manifest file is a struct called `manifest_entry` with the follo `data_file` is a struct with the fo

Re: [PR] Spec: Support geo type [iceberg]

2024-09-11 Thread via GitHub
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1755704472 ## format/spec.md: ## @@ -198,6 +199,9 @@ Notes: - Timestamp values _with time zone_ represent a point in time: values are stored as UTC and do not retain a s

Re: [PR] Spec: Support geo type [iceberg]

2024-09-11 Thread via GitHub
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1755704472 ## format/spec.md: ## @@ -198,6 +199,9 @@ Notes: - Timestamp values _with time zone_ represent a point in time: values are stored as UTC and do not retain a s

Re: [PR] Spec: Support geo type [iceberg]

2024-09-11 Thread via GitHub
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1755705907 ## format/spec.md: ## @@ -198,6 +199,9 @@ Notes: - Timestamp values _with time zone_ represent a point in time: values are stored as UTC and do not retain a s

Re: [PR] Spec: Support geo type [iceberg]

2024-09-11 Thread via GitHub
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1755706590 ## format/spec.md: ## @@ -1407,3 +1432,9 @@ Iceberg supports two types of histories for tables. A history of previous "curre might indicate different snapshot IDs

Re: [PR] Spec: Support geo type [iceberg]

2024-09-11 Thread via GitHub
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1755705776 ## format/spec.md: ## @@ -190,6 +190,7 @@ Supported primitive types are defined in the table below. Primitive types added | | **`uuid`**

Re: [PR] Spec: Support geo type [iceberg]

2024-09-11 Thread via GitHub
szehon-ho commented on PR #10981: URL: https://github.com/apache/iceberg/pull/10981#issuecomment-2344767216 @jiayuasu @Kontinuation @wgtmac @flyrain @rdblue sorry for the delay, as I only got access now. Updated the pr. -- This is an automated message from the Apache Git Service. To resp

Re: [PR] Spec: Support geo type [iceberg]

2024-09-11 Thread via GitHub
flyrain commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1755713095 ## format/spec.md: ## @@ -323,16 +327,17 @@ Partition field IDs must be reused if an existing partition spec contains an equ Partition Transforms -| Transf

[I] Iceberg defaulting to URLConnectionHttpClient instead of Apache HTTP Client [iceberg]

2024-09-11 Thread via GitHub
anthonysgro opened a new issue, #6: URL: https://github.com/apache/iceberg/issues/6 ### Apache Iceberg version 1.5.0 ### Query engine Spark ### Please describe the bug 🐞 Receiving this stack trace when reading from cross-account iceberg glue table:

Re: [PR] API, Core: Add default value APIs and Avro implementation [iceberg]

2024-09-11 Thread via GitHub
rdblue commented on PR #9502: URL: https://github.com/apache/iceberg/pull/9502#issuecomment-2344814631 > One more thing, we're also leaking V3 spec features into the codebase. Should we guard that with a flag? For #9008, we added a check in `TableMetadata.Builder` to prevent v1 and v2

Re: [PR] Bug Fix: Use historical partition field name [iceberg-python]

2024-09-11 Thread via GitHub
sungwy commented on code in PR #1161: URL: https://github.com/apache/iceberg-python/pull/1161#discussion_r1755686149 ## pyiceberg/table/update/spec.py: ## @@ -280,7 +280,7 @@ def _partition_field(self, transform_key: Tuple[int, Transform[Any, Any]], name: for field

Re: [PR] OpenAPI: Standardize credentials in loadTable/loadView responses [iceberg]

2024-09-11 Thread via GitHub
dimas-b commented on code in PR #10722: URL: https://github.com/apache/iceberg/pull/10722#discussion_r1755855988 ## open-api/rest-catalog-open-api.yaml: ## @@ -3129,6 +3204,11 @@ components: - `s3.secret-access-key`: secret for credentials that provide access to data

Re: [PR] OpenAPI: Standardize credentials in loadTable/loadView responses [iceberg]

2024-09-11 Thread via GitHub
dimas-b commented on code in PR #10722: URL: https://github.com/apache/iceberg/pull/10722#discussion_r1755859795 ## open-api/rest-catalog-open-api.yaml: ## @@ -3129,6 +3204,11 @@ components: - `s3.secret-access-key`: secret for credentials that provide access to data

Re: [PR] Increase Partition Start Id to 10000 [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] commented on PR #6369: URL: https://github.com/apache/iceberg/pull/6369#issuecomment-2345009303 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Increase Partition Start Id to 10000 [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed pull request #6369: Increase Partition Start Id to 1 URL: https://github.com/apache/iceberg/pull/6369 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] Docs: Update Iceberg Hive documentation (#6337) [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed pull request #6380: Docs: Update Iceberg Hive documentation (#6337) URL: https://github.com/apache/iceberg/pull/6380 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [I] iceberg-1.1.0 - flink sql create hive catalog error [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed issue #6522: iceberg-1.1.0 - flink sql create hive catalog error URL: https://github.com/apache/iceberg/issues/6522 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] Docs: Update Iceberg Hive documentation (#6337) [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] commented on PR #6380: URL: https://github.com/apache/iceberg/pull/6380#issuecomment-2345009333 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] fix delete when in upsert mode using two discontinuous primary keys [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed pull request #6560: fix delete when in upsert mode using two discontinuous primary keys URL: https://github.com/apache/iceberg/pull/6560 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] fix delete when in upsert mode using two discontinuous primary keys [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] commented on PR #6560: URL: https://github.com/apache/iceberg/pull/6560#issuecomment-2345009382 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Data: java api add GenericTaskWriter and add write demo to Doc. [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed pull request #6571: Data: java api add GenericTaskWriter and add write demo to Doc. URL: https://github.com/apache/iceberg/pull/6571 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [I] iceberg-1.1.0 - flink sql create hive catalog error [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] commented on issue #6522: URL: https://github.com/apache/iceberg/issues/6522#issuecomment-2345009360 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [PR] Data: java api add GenericTaskWriter and add write demo to Doc. [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] commented on PR #6571: URL: https://github.com/apache/iceberg/pull/6571#issuecomment-2345009410 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Fix the problem of shadow jar without modifying meta-inf/services at the same time [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed pull request #6577: Fix the problem of shadow jar without modifying meta-inf/services at the same time URL: https://github.com/apache/iceberg/pull/6577 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [PR] Spark 3.3: Add RemoveDanglingDeletes action [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] commented on PR #6581: URL: https://github.com/apache/iceberg/pull/6581#issuecomment-2345009464 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Spark 3.3: Add RemoveDanglingDeletes action [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed pull request #6581: Spark 3.3: Add RemoveDanglingDeletes action URL: https://github.com/apache/iceberg/pull/6581 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] Add a Spark procedure to collect NDV [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] commented on PR #6582: URL: https://github.com/apache/iceberg/pull/6582#issuecomment-2345009488 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Add a Spark procedure to collect NDV [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed pull request #6582: Add a Spark procedure to collect NDV URL: https://github.com/apache/iceberg/pull/6582 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] Core: Add information_schema.namespaces table [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] commented on PR #6599: URL: https://github.com/apache/iceberg/pull/6599#issuecomment-2345009526 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Core: Add information_schema.namespaces table [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed pull request #6599: Core: Add information_schema.namespaces table URL: https://github.com/apache/iceberg/pull/6599 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] Docs: Update the description of binary and varbinary type in the type conversion section [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] commented on PR #6608: URL: https://github.com/apache/iceberg/pull/6608#issuecomment-2345009546 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Docs: Update the description of binary and varbinary type in the type conversion section [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed pull request #6608: Docs: Update the description of binary and varbinary type in the type conversion section URL: https://github.com/apache/iceberg/pull/6608 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [I] Merge into does not work with spark temp table [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed issue #6615: Merge into does not work with spark temp table URL: https://github.com/apache/iceberg/issues/6615 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [HiveCatalog] Support Altering and Dropping Table Ownership [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed pull request #6621: [HiveCatalog] Support Altering and Dropping Table Ownership URL: https://github.com/apache/iceberg/pull/6621 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] 🎨 Add "parallelism" parameter to "add_files" syscall and MigrateTable, SnapshotTable. [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] commented on PR #6624: URL: https://github.com/apache/iceberg/pull/6624#issuecomment-2345009638 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] 🎨 Add "parallelism" parameter to "add_files" syscall and MigrateTable, SnapshotTable. [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed pull request #6624: 🎨 Add "parallelism" parameter to "add_files" syscall and MigrateTable, SnapshotTable. URL: https://github.com/apache/iceberg/pull/6624 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] [HiveCatalog] Support Altering and Dropping Table Ownership [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] commented on PR #6621: URL: https://github.com/apache/iceberg/pull/6621#issuecomment-2345009600 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Core: Update error msg [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] commented on PR #6626: URL: https://github.com/apache/iceberg/pull/6626#issuecomment-2345009664 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [I] Purpose of MAX_CONTINUOUS_EMPTY_COMMITS in IcebergFilesCommitter [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] commented on issue #6630: URL: https://github.com/apache/iceberg/issues/6630#issuecomment-2345009691 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [PR] WIP: Support Snapshot Copy-On-Write Hudi Table to Iceberg Table [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] commented on PR #6642: URL: https://github.com/apache/iceberg/pull/6642#issuecomment-2345009733 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] WIP: Support Snapshot Copy-On-Write Hudi Table to Iceberg Table [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed pull request #6642: WIP: Support Snapshot Copy-On-Write Hudi Table to Iceberg Table URL: https://github.com/apache/iceberg/pull/6642 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] AWS: support force register table in GlueCatalog [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] commented on PR #6742: URL: https://github.com/apache/iceberg/pull/6742#issuecomment-2345009853 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] AWS: support force register table in GlueCatalog [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed pull request #6742: AWS: support force register table in GlueCatalog URL: https://github.com/apache/iceberg/pull/6742 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [I] Iceberg add_files procedure with partition_filter scan non needed folders [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed issue #7027: Iceberg add_files procedure with partition_filter scan non needed folders URL: https://github.com/apache/iceberg/issues/7027 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] Support bulk remove orphan files [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed issue #7111: Support bulk remove orphan files URL: https://github.com/apache/iceberg/issues/7111 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

Re: [I] Replace Thread.sleep() usage in test code with Awaitility [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] commented on issue #7154: URL: https://github.com/apache/iceberg/issues/7154#issuecomment-2345009924 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Support commit operations in pyiceberg [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed issue #7259: Support commit operations in pyiceberg URL: https://github.com/apache/iceberg/issues/7259 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] EMR 6.10.0 Cannot migrate a table from a non-Iceberg Spark Session Catalog. Found spark_catalog [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed issue #7317: EMR 6.10.0 Cannot migrate a table from a non-Iceberg Spark Session Catalog. Found spark_catalog URL: https://github.com/apache/iceberg/issues/7317 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

Re: [I] Using Iceberg from EKS to access resource in another aws account loads instance role by default [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed issue #7344: Using Iceberg from EKS to access resource in another aws account loads instance role by default URL: https://github.com/apache/iceberg/issues/7344 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

Re: [I] Using Iceberg from EKS to access resource in another aws account loads instance role by default [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] commented on issue #7344: URL: https://github.com/apache/iceberg/issues/7344#issuecomment-2345010035 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [PR] Core: Update error msg [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed pull request #6626: Core: Update error msg URL: https://github.com/apache/iceberg/pull/6626 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

Re: [I] Purpose of MAX_CONTINUOUS_EMPTY_COMMITS in IcebergFilesCommitter [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed issue #6630: Purpose of MAX_CONTINUOUS_EMPTY_COMMITS in IcebergFilesCommitter URL: https://github.com/apache/iceberg/issues/6630 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] Unclear messaging about Glue catalog locking [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed issue #6636: Unclear messaging about Glue catalog locking URL: https://github.com/apache/iceberg/issues/6636 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [I] Unclear messaging about Glue catalog locking [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] commented on issue #6636: URL: https://github.com/apache/iceberg/issues/6636#issuecomment-2345009713 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] ManifestReader does not return metrics with null filters [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed issue #6658: ManifestReader does not return metrics with null filters URL: https://github.com/apache/iceberg/issues/6658 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [I] ManifestReader does not return metrics with null filters [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] commented on issue #6658: URL: https://github.com/apache/iceberg/issues/6658#issuecomment-2345009752 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [PR] Add REST catalog spec to iceberg-core.jar [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] commented on PR #6672: URL: https://github.com/apache/iceberg/pull/6672#issuecomment-2345009779 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Add REST catalog spec to iceberg-core.jar [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed pull request #6672: Add REST catalog spec to iceberg-core.jar URL: https://github.com/apache/iceberg/pull/6672 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] Integrate CRT with Iceberg S3 client [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed issue #6739: Integrate CRT with Iceberg S3 client URL: https://github.com/apache/iceberg/issues/6739 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] Integrate CRT with Iceberg S3 client [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] commented on issue #6739: URL: https://github.com/apache/iceberg/issues/6739#issuecomment-2345009835 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Iceberg add_files procedure with partition_filter scan non needed folders [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] commented on issue #7027: URL: https://github.com/apache/iceberg/issues/7027#issuecomment-2345009886 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Replace Thread.sleep() usage in test code with Awaitility [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed issue #7154: Replace Thread.sleep() usage in test code with Awaitility URL: https://github.com/apache/iceberg/issues/7154 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [I] Drop the SQL issue when attempting to drop an Iceberg table whose location does not exist [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed issue #7227: Drop the SQL issue when attempting to drop an Iceberg table whose location does not exist URL: https://github.com/apache/iceberg/issues/7227 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [I] Drop the SQL issue when attempting to drop an Iceberg table whose location does not exist [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] commented on issue #7227: URL: https://github.com/apache/iceberg/issues/7227#issuecomment-2345009947 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Support commit operations in pyiceberg [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] commented on issue #7259: URL: https://github.com/apache/iceberg/issues/7259#issuecomment-2345009970 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] EMR 6.10.0 Cannot migrate a table from a non-Iceberg Spark Session Catalog. Found spark_catalog [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] commented on issue #7317: URL: https://github.com/apache/iceberg/issues/7317#issuecomment-2345009993 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] location parameter of remove_orphan_files procedure relative to table location [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed issue #7334: location parameter of remove_orphan_files procedure relative to table location URL: https://github.com/apache/iceberg/issues/7334 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [I] Support bulk remove orphan files [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] commented on issue #7111: URL: https://github.com/apache/iceberg/issues/7111#issuecomment-2345009906 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] location parameter of remove_orphan_files procedure relative to table location [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] commented on issue #7334: URL: https://github.com/apache/iceberg/issues/7334#issuecomment-2345010011 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] iceberg mor table execute merge very very slow [iceberg]

2024-09-11 Thread via GitHub
github-actions[bot] closed issue #7431: iceberg mor table execute merge very very slow URL: https://github.com/apache/iceberg/issues/7431 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-09-11 Thread via GitHub
hesham-medhat commented on PR #11039: URL: https://github.com/apache/iceberg/pull/11039#issuecomment-2345020069 > Hi @hesham-medhat , I assume this PR will also enable read/write support for Flink pipelines that use BigLake external tables with BigQuery? Hi @brunsgaard! No. This one i

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-09-11 Thread via GitHub
hesham-medhat commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1755903459 ## gcp/src/main/java/org/apache/iceberg/gcp/bigquery/BigQueryClientImpl.java: ## @@ -0,0 +1,481 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2024-09-11 Thread via GitHub
hesham-medhat commented on code in PR #11039: URL: https://github.com/apache/iceberg/pull/11039#discussion_r1755908398 ## build.gradle: ## @@ -647,20 +647,43 @@ project(':iceberg-gcp') { implementation project(':iceberg-common') implementation project(':iceberg-core')

Re: [I] Enable initializing DataFusion IcebergCatalogProvider using specific schemas (rather than all) [iceberg-rust]

2024-09-11 Thread via GitHub
liurenjie1024 commented on issue #601: URL: https://github.com/apache/iceberg-rust/issues/601#issuecomment-2345167409 Thanks @a-agmon for raising this. The reason currently we need to fetch all schemas at once is to simplify implementation, since datafusion's catalog api currently is not as

Re: [I] Iceberg defaulting to URLConnectionHttpClient instead of Apache HTTP Client [iceberg]

2024-09-11 Thread via GitHub
anthonysgro commented on issue #6: URL: https://github.com/apache/iceberg/issues/6#issuecomment-2345208962 ended up fixing this by just resetting the config right before reading from the table: ``` sparkSession.sparkContext.setConf(s"spark.sql.catalog.${CATALOG}.http-clien

Re: [I] Issue when overwriting data with row filter [iceberg-python]

2024-09-11 Thread via GitHub
JasperHG90 commented on issue #1108: URL: https://github.com/apache/iceberg-python/issues/1108#issuecomment-2345258759 Thanks @Fokko ! Big fan of your work. I'll check it out! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [PR] Flink: Maintenance - Lock remover [iceberg]

2024-09-11 Thread via GitHub
pvary merged PR #11010: URL: https://github.com/apache/iceberg/pull/11010 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Flink: Maintenance - Lock remover [iceberg]

2024-09-11 Thread via GitHub
pvary commented on PR #11010: URL: https://github.com/apache/iceberg/pull/11010#issuecomment-2345295003 Merged to main. Thanks for the review @stevenzwu and @advancedxy! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Use Snapshot's statistics file in SparkScan [iceberg]

2024-09-11 Thread via GitHub
jeesou commented on PR #11040: URL: https://github.com/apache/iceberg/pull/11040#issuecomment-2345310369 LGTM I have picked up the PR changes and tested it out, Its working fine. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

<    1   2   3   >