Re: [PR] Spark: Fix flaky tests due to socket bind failure [iceberg]

2025-05-12 Thread via GitHub
nastra commented on code in PR #13017: URL: https://github.com/apache/iceberg/pull/13017#discussion_r2086062975 ## open-api/src/testFixtures/java/org/apache/iceberg/rest/RESTServerExtension.java: ## @@ -58,8 +60,23 @@ public RESTCatalog client() { public void beforeAll(Extens

Re: [PR] nessie_list_namespaces_throw_exception [iceberg]

2025-05-12 Thread via GitHub
nastra commented on code in PR #12901: URL: https://github.com/apache/iceberg/pull/12901#discussion_r2086053168 ## nessie/src/test/java/org/apache/iceberg/nessie/TestBranchVisibility.java: ## @@ -455,7 +456,9 @@ public void testWithRefAndHash() throws NessieConflictException, N

Re: [PR] nessie_list_namespaces_throw_exception [iceberg]

2025-05-12 Thread via GitHub
nastra commented on code in PR #12901: URL: https://github.com/apache/iceberg/pull/12901#discussion_r2086058824 ## nessie/src/test/java/org/apache/iceberg/nessie/TestNessieCatalog.java: ## @@ -193,8 +205,16 @@ public void testWarehouseLocationWithTrailingSlash() { @Test

Re: [PR] nessie_list_namespaces_throw_exception [iceberg]

2025-05-12 Thread via GitHub
nastra commented on code in PR #12901: URL: https://github.com/apache/iceberg/pull/12901#discussion_r2086057524 ## nessie/src/test/java/org/apache/iceberg/nessie/TestNessieCatalog.java: ## @@ -96,6 +100,14 @@ public void afterEach() throws IOException { } } + @Test +

Re: [PR] nessie_list_namespaces_throw_exception [iceberg]

2025-05-12 Thread via GitHub
nastra commented on code in PR #12901: URL: https://github.com/apache/iceberg/pull/12901#discussion_r2086058184 ## nessie/src/test/java/org/apache/iceberg/nessie/TestNessieCatalog.java: ## @@ -193,8 +205,16 @@ public void testWarehouseLocationWithTrailingSlash() { @Test

Re: [PR] nessie_list_namespaces_throw_exception [iceberg]

2025-05-12 Thread via GitHub
nastra commented on code in PR #12901: URL: https://github.com/apache/iceberg/pull/12901#discussion_r2086055889 ## nessie/src/test/java/org/apache/iceberg/nessie/TestMultipleClients.java: ## @@ -76,9 +76,11 @@ public void testListNamespaces() throws NessieConflictException, Nes

Re: [PR] nessie_list_namespaces_throw_exception [iceberg]

2025-05-12 Thread via GitHub
nastra commented on code in PR #12901: URL: https://github.com/apache/iceberg/pull/12901#discussion_r2086051251 ## nessie/src/test/java/org/apache/iceberg/nessie/TestBranchVisibility.java: ## @@ -455,7 +456,9 @@ public void testWithRefAndHash() throws NessieConflictException, N

Re: [PR] nessie_list_namespaces_throw_exception [iceberg]

2025-05-12 Thread via GitHub
nastra commented on code in PR #12901: URL: https://github.com/apache/iceberg/pull/12901#discussion_r2086051251 ## nessie/src/test/java/org/apache/iceberg/nessie/TestBranchVisibility.java: ## @@ -455,7 +456,9 @@ public void testWithRefAndHash() throws NessieConflictException, N

Re: [PR] nessie_list_namespaces_throw_exception [iceberg]

2025-05-12 Thread via GitHub
nastra commented on code in PR #12901: URL: https://github.com/apache/iceberg/pull/12901#discussion_r2086049477 ## nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java: ## @@ -270,6 +270,15 @@ public List listNamespaces(Namespace namespace) throws NoSuchNames

Re: [PR] nessie_list_namespaces_throw_exception [iceberg]

2025-05-12 Thread via GitHub
nastra commented on code in PR #12901: URL: https://github.com/apache/iceberg/pull/12901#discussion_r2086049916 ## nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java: ## @@ -270,6 +270,15 @@ public List listNamespaces(Namespace namespace) throws NoSuchNames

Re: [PR] feat(catalog/sql): Add views related ops [iceberg-go]

2025-05-12 Thread via GitHub
dttung2905 commented on PR #414: URL: https://github.com/apache/iceberg-go/pull/414#issuecomment-2875158585 > Hey @dttung2905, just wanted to check if you're still actively working on this PR. If not, I'd be happy to pick it up and continue. Let me know your thoughts! > > > >

Re: [PR] feat(catalog/sql): Add views related ops [iceberg-go]

2025-05-12 Thread via GitHub
Shreyas220 commented on PR #414: URL: https://github.com/apache/iceberg-go/pull/414#issuecomment-2875108202 Hey @dttung2905, just wanted to check if you're still actively working on this PR. If not, I'd be happy to pick it up and continue. Let me know your thoughts! -- This is an

Re: [PR] feat:Add hadoop catalog mode(include s3,s3a(minio) and hdfs ) [iceberg-rust]

2025-05-12 Thread via GitHub
Xuanwo commented on PR #1313: URL: https://github.com/apache/iceberg-rust/pull/1313#issuecomment-2875100255 > iceberg-python is just a wrapper for pyiceberg-core, and pyiceberg-core is located within the iceberg-rust repository Hi, this understanding is incorrect. While pyiceb

Re: [PR] feat:Add hadoop catalog mode(include s3,s3a(minio) and hdfs ) [iceberg-rust]

2025-05-12 Thread via GitHub
awol2005ex commented on PR #1313: URL: https://github.com/apache/iceberg-rust/pull/1313#issuecomment-2875084933 > > > > > Hi, @awol2005ex thanks for the contribution. The community direction is to deprecate pure file system catalog(like hadoop catalog). If you need to query iceberg table on

Re: [PR] feat: add visit type support [iceberg-cpp]

2025-05-12 Thread via GitHub
Xuanwo merged PR #94: URL: https://github.com/apache/iceberg-cpp/pull/94 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [I] Transactions do not support Upsert [iceberg-python]

2025-05-12 Thread via GitHub
koenvo commented on issue #1776: URL: https://github.com/apache/iceberg-python/issues/1776#issuecomment-2875049585 Thanks for the reminder. Let me finish this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [I] Transactions do not support Upsert [iceberg-python]

2025-05-12 Thread via GitHub
potatochipcoconut commented on issue #1776: URL: https://github.com/apache/iceberg-python/issues/1776#issuecomment-2875036376 @koenvo how's it looking? I'm also interested in this feature. -- This is an automated message from the Apache Git Service. To respond to the message, please log o

Re: [PR] [REST SPEC] Add first-row-id in the data files for Row Lineage [iceberg]

2025-05-12 Thread via GitHub
stevenzwu commented on code in PR #13010: URL: https://github.com/apache/iceberg/pull/13010#discussion_r2085907445 ## open-api/rest-catalog-open-api.py: ## @@ -991,6 +991,9 @@ class ValueMap(BaseModel): class DataFile(ContentFile): content: str = Field(..., const=True) +

Re: [PR] Add table property to disable/enable parquet column statistics #12770 [iceberg]

2025-05-12 Thread via GitHub
huaxiangsun commented on code in PR #12771: URL: https://github.com/apache/iceberg/pull/12771#discussion_r2085907320 ## parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java: ## @@ -306,33 +309,35 @@ private WriteBuilder createContextFunc( return this; } +

Re: [PR] [REST SPEC] Add first-row-id in the data files for Row Lineage [iceberg]

2025-05-12 Thread via GitHub
stevenzwu commented on code in PR #13010: URL: https://github.com/apache/iceberg/pull/13010#discussion_r2085907445 ## open-api/rest-catalog-open-api.py: ## @@ -991,6 +991,9 @@ class ValueMap(BaseModel): class DataFile(ContentFile): content: str = Field(..., const=True) +

Re: [PR] Support Scan Planning in Rest Client [iceberg]

2025-05-12 Thread via GitHub
singhpk234 closed pull request #13004: Support Scan Planning in Rest Client URL: https://github.com/apache/iceberg/pull/13004 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [I] Proposal on the return type of org.apache.iceberg.catalog.Catalog#listTables [iceberg]

2025-05-12 Thread via GitHub
manuzhang commented on issue #13037: URL: https://github.com/apache/iceberg/issues/13037#issuecomment-2875014599 There is already a parameter `pageSize` that clients can ask servers to return a limited number of results. -- This is an automated message from the Apache Git Service. To resp

Re: [PR] feat: Add API to set location in the transacation [iceberg-rust]

2025-05-12 Thread via GitHub
CTTY commented on code in PR #1317: URL: https://github.com/apache/iceberg-rust/pull/1317#discussion_r2085877334 ## crates/iceberg/src/transaction/mod.rs: ## @@ -335,6 +341,22 @@ mod tests { ); } +#[test] Review Comment: I think this makes sense, many tra

[I] Add integration tests for transaction APIs [iceberg-rust]

2025-05-12 Thread via GitHub
CTTY opened a new issue, #1322: URL: https://github.com/apache/iceberg-rust/issues/1322 ### Is your feature request related to a problem or challenge? Now many APIs in `transaction/mod.rs` don't have an integration tests to test if they actually work after committing the transaction.

Re: [PR] feat:Add hadoop catalog mode(include s3,s3a(minio) and hdfs ) [iceberg-rust]

2025-05-12 Thread via GitHub
liurenjie1024 commented on PR #1313: URL: https://github.com/apache/iceberg-rust/pull/1313#issuecomment-2874982132 > > > > Hi, @awol2005ex thanks for the contribution. The community direction is to deprecate pure file system catalog(like hadoop catalog). If you need to query iceberg table o

Re: [PR] chore: bump up arrow/parquet/datafusion [iceberg-rust]

2025-05-12 Thread via GitHub
Xuanwo merged PR #1294: URL: https://github.com/apache/iceberg-rust/pull/1294 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Build: Run SPARK CI on include path patterns [iceberg]

2025-05-12 Thread via GitHub
manuzhang closed pull request #13033: Build: Run SPARK CI on include path patterns URL: https://github.com/apache/iceberg/pull/13033 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] Spark: Fix flaky tests due to socket bind failure [iceberg]

2025-05-12 Thread via GitHub
manuzhang commented on PR #13017: URL: https://github.com/apache/iceberg/pull/13017#issuecomment-2874947011 Spark CI is not triggered, but `RESTServerExtension` is used in `org.apache.iceberg.spark.TestBaseWithCatalog`. -- This is an automated message from the Apache Git Service. To respo

Re: [PR] feat:Add hadoop catalog mode(include s3,s3a(minio) and hdfs ) [iceberg-rust]

2025-05-12 Thread via GitHub
awol2005ex commented on PR #1313: URL: https://github.com/apache/iceberg-rust/pull/1313#issuecomment-2874920417 > > > Hi, @awol2005ex thanks for the contribution. The community direction is to deprecate pure file system catalog(like hadoop catalog). If you need to query iceberg table on fil

Re: [PR] feat: add manifest related structures [iceberg-cpp]

2025-05-12 Thread via GitHub
wgtmac commented on code in PR #91: URL: https://github.com/apache/iceberg-cpp/pull/91#discussion_r2085835400 ## src/iceberg/schema_field.h: ## @@ -86,4 +86,16 @@ class ICEBERG_EXPORT SchemaField : public iceberg::util::Formattable { bool optional_; }; +template +inline

Re: [PR] [REST SPEC] Add first-row-id in the data files for Row Lineage [iceberg]

2025-05-12 Thread via GitHub
singhpk234 commented on PR #13010: URL: https://github.com/apache/iceberg/pull/13010#issuecomment-2874913231 > is there a way we can autogenerate clients/objects from this and check One thought on this that comes to my mind is (open to other ideas), Open API does supports generating

Re: [PR] Flink: Support zookeeper lock in TableMaintenance [iceberg]

2025-05-12 Thread via GitHub
Guosmilesmile commented on PR #12810: URL: https://github.com/apache/iceberg/pull/12810#issuecomment-2874886395 @mxm As we discussed offline, Flink always includes the Curator library, so it is fine to bundle Curator for iceberg. But in checkstyleMain, import repackaged classes will make th

Re: [PR] feat:Add hadoop catalog mode(include s3,s3a(minio) and hdfs ) [iceberg-rust]

2025-05-12 Thread via GitHub
liurenjie1024 commented on PR #1313: URL: https://github.com/apache/iceberg-rust/pull/1313#issuecomment-2874867295 > > Hi, @awol2005ex thanks for the contribution. The community direction is to deprecate pure file system catalog(like hadoop catalog). If you need to query iceberg table on fi

Re: [PR] Spark4.0 rc5 integration with Iceberg [iceberg]

2025-05-12 Thread via GitHub
amogh-jahagirdar commented on code in PR #13022: URL: https://github.com/apache/iceberg/pull/13022#discussion_r2085808982 ## core/src/main/java/org/apache/iceberg/MetadataColumns.java: ## @@ -51,7 +51,7 @@ private MetadataColumns() {} public static final int SPEC_ID_COLUMN_ID

Re: [PR] feat:Add hadoop catalog mode(include s3,s3a(minio) and hdfs ) [iceberg-rust]

2025-05-12 Thread via GitHub
awol2005ex commented on PR #1313: URL: https://github.com/apache/iceberg-rust/pull/1313#issuecomment-2874799554 > Hi, @awol2005ex thanks for the contribution. The community direction is to deprecate pure file system catalog(like hadoop catalog). If you need to query iceberg table on file sy

Re: [PR] feat: implement initial MemoryCatalog functionality with namespace and table support [iceberg-cpp]

2025-05-12 Thread via GitHub
wgtmac commented on code in PR #80: URL: https://github.com/apache/iceberg-cpp/pull/80#discussion_r2085776055 ## src/iceberg/catalog/memory_catalog.h: ## @@ -0,0 +1,204 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreemen

Re: [PR] feat: Add API to set location in the transacation [iceberg-rust]

2025-05-12 Thread via GitHub
jonathanc-n commented on code in PR #1317: URL: https://github.com/apache/iceberg-rust/pull/1317#discussion_r2085773454 ## crates/iceberg/src/transaction/mod.rs: ## @@ -335,6 +341,22 @@ mod tests { ); } +#[test] Review Comment: I believe we do not have on

Re: [PR] feat: Add API to set location in the transacation [iceberg-rust]

2025-05-12 Thread via GitHub
jonathanc-n commented on code in PR #1317: URL: https://github.com/apache/iceberg-rust/pull/1317#discussion_r2085772767 ## crates/iceberg/src/transaction/mod.rs: ## @@ -335,6 +341,22 @@ mod tests { ); } +#[test] Review Comment: Should we make a integratio

[PR] fix: doc typo for `Schema.name_by_field_id()` [iceberg-rust]

2025-05-12 Thread via GitHub
burmecia opened a new pull request, #1321: URL: https://github.com/apache/iceberg-rust/pull/1321 ## Which issue does this PR close? - N/A ## What changes are included in this PR? This PR is to fix a typo in function `Schema.name_by_field_id()` doc string. ## Are th

Re: [PR] Build: Bump getdaft from 0.4.13 to 0.4.14 [iceberg-python]

2025-05-12 Thread via GitHub
Fokko merged PR #1991: URL: https://github.com/apache/iceberg-python/pull/1991 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Catalog: Add BigQuery Metastore Catalog Support [iceberg]

2025-05-12 Thread via GitHub
ebyhr commented on code in PR #12808: URL: https://github.com/apache/iceberg/pull/12808#discussion_r2085738367 ## bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/BigQueryMetastoreClientImpl.java: ## @@ -0,0 +1,579 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[I] Add dedicated credential property to BigQuery metastore catalog [iceberg]

2025-05-12 Thread via GitHub
ebyhr opened a new issue, #13036: URL: https://github.com/apache/iceberg/issues/13036 ### Feature Request / Improvement This is a follow-up of https://github.com/apache/iceberg/pull/12808. The PR uses the `GoogleCredentials.getApplicationDefault()` method to create the BigQuer

Re: [PR] fix(transforms): DayTransform [iceberg-go]

2025-05-12 Thread via GitHub
jhump commented on code in PR #423: URL: https://github.com/apache/iceberg-go/pull/423#discussion_r2085734360 ## transforms.go: ## @@ -720,7 +720,7 @@ func (t DayTransform) MarshalText() ([]byte, error) { func (DayTransform) String() string { return "day" } -func (DayTransf

Re: [PR] fix(transforms): DayTransform [iceberg-go]

2025-05-12 Thread via GitHub
jhump commented on code in PR #423: URL: https://github.com/apache/iceberg-go/pull/423#discussion_r2085734360 ## transforms.go: ## @@ -720,7 +720,7 @@ func (t DayTransform) MarshalText() ([]byte, error) { func (DayTransform) String() string { return "day" } -func (DayTransf

Re: [PR] Flink 2.0: Remove the JUnit4 dependency [iceberg]

2025-05-12 Thread via GitHub
JeonDaehong commented on code in PR #13021: URL: https://github.com/apache/iceberg/pull/13021#discussion_r2085725629 ## flink/v2.0/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailover.java: ## @@ -314,20 +313,6 @@ private void createBoundedStreams(Strea

Re: [PR] Build: Bump pyroaring from 1.0.0 to 1.0.1 [iceberg-python]

2025-05-12 Thread via GitHub
Fokko merged PR #1992: URL: https://github.com/apache/iceberg-python/pull/1992 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Build: Bump mkdocs-material from 9.6.12 to 9.6.13 [iceberg-python]

2025-05-12 Thread via GitHub
Fokko merged PR #1993: URL: https://github.com/apache/iceberg-python/pull/1993 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Add table property to disable/enable parquet column statistics #12770 [iceberg]

2025-05-12 Thread via GitHub
huaxiangsun commented on PR #12771: URL: https://github.com/apache/iceberg/pull/12771#issuecomment-2874662297 > Thanks @huaxiangsun for the PR! Thanks @pvary @dramaticlly @szehon-ho for the review! Thanks @huaxingao! -- This is an automated message from the Apache Git Service. To r

Re: [PR] Build: Bump mypy-boto3-glue from 1.38.0 to 1.38.12 [iceberg-python]

2025-05-12 Thread via GitHub
Fokko merged PR #1990: URL: https://github.com/apache/iceberg-python/pull/1990 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Materialized View Spec [iceberg]

2025-05-12 Thread via GitHub
RussellSpitzer commented on code in PR #11041: URL: https://github.com/apache/iceberg/pull/11041#discussion_r2085454632 ## format/view-spec.md: ## @@ -42,12 +42,28 @@ An atomic swap of one view metadata file for another provides the basis for maki Writers create view metadat

Re: [PR] [PoC] Test Flink with Hive 4 metastore and Spark with Hive 3 metastore [iceberg]

2025-05-12 Thread via GitHub
github-actions[bot] commented on PR #12693: URL: https://github.com/apache/iceberg/pull/12693#issuecomment-2874628178 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [I] Iceberg metadata table is not created if using MYSQL JDBC catalog and there are existing iceberg meta tables in another database. [iceberg]

2025-05-12 Thread via GitHub
github-actions[bot] closed issue #11423: Iceberg metadata table is not created if using MYSQL JDBC catalog and there are existing iceberg meta tables in another database. URL: https://github.com/apache/iceberg/issues/11423 -- This is an automated message from the Apache Git Service. To respo

Re: [I] Iceberg metadata table is not created if using MYSQL JDBC catalog and there are existing iceberg meta tables in another database. [iceberg]

2025-05-12 Thread via GitHub
github-actions[bot] commented on issue #11423: URL: https://github.com/apache/iceberg/issues/11423#issuecomment-2874627974 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [PR] [PoC] Test Flink with Hive 4 metastore and Spark with Hive 3 metastore [iceberg]

2025-05-12 Thread via GitHub
github-actions[bot] closed pull request #12693: [PoC] Test Flink with Hive 4 metastore and Spark with Hive 3 metastore URL: https://github.com/apache/iceberg/pull/12693 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [I] Flink Dynamic Sink [iceberg]

2025-05-12 Thread via GitHub
github-actions[bot] commented on issue #11536: URL: https://github.com/apache/iceberg/issues/11536#issuecomment-2874628074 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] Iceberg does not work with Spark's default hive metastore (embedded Derby database) [iceberg]

2025-05-12 Thread via GitHub
github-actions[bot] commented on issue #7847: URL: https://github.com/apache/iceberg/issues/7847#issuecomment-2874627811 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Equality delete lost after compact data files [iceberg]

2025-05-12 Thread via GitHub
github-actions[bot] commented on issue #10312: URL: https://github.com/apache/iceberg/issues/10312#issuecomment-2874627862 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] Iceberg does not work with Spark's default hive metastore (embedded Derby database) [iceberg]

2025-05-12 Thread via GitHub
github-actions[bot] closed issue #7847: Iceberg does not work with Spark's default hive metastore (embedded Derby database) URL: https://github.com/apache/iceberg/issues/7847 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [I] Clarify Error Logs for GCS Directory/Object Conflicts [iceberg-python]

2025-05-12 Thread via GitHub
HyunWooZZ commented on issue #1952: URL: https://github.com/apache/iceberg-python/issues/1952#issuecomment-2874603831 @kevinjqliu There is nothing. I ran the gsutil ls command beforehand, and the seoul_data_v1/warehouse/test/ directory did not exist. As you know, 0-6f9631fe-18

Re: [I] Dependency on pyiceberg_core while still marked as Optional [iceberg-python]

2025-05-12 Thread via GitHub
kevinjqliu commented on issue #1987: URL: https://github.com/apache/iceberg-python/issues/1987#issuecomment-2874433972 Thanks for reporting! `pyiceberg_core` / `pyiceberg-core` is marked optional since its only used when writing with pyarrow fs for tables with partition transforms.

Re: [PR] Add table property to disable/enable parquet column statistics #12770 [iceberg]

2025-05-12 Thread via GitHub
huaxingao merged PR #12771: URL: https://github.com/apache/iceberg/pull/12771 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Add table property to disable/enable parquet column statistics #12770 [iceberg]

2025-05-12 Thread via GitHub
huaxingao commented on PR #12771: URL: https://github.com/apache/iceberg/pull/12771#issuecomment-2874414193 Thanks @huaxiangsun for the PR! Thanks @pvary @dramaticlly @szehon-ho for the review! -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [I] Feature request: make `blob` struct public [iceberg-rust]

2025-05-12 Thread via GitHub
dentiny commented on issue #1311: URL: https://github.com/apache/iceberg-rust/issues/1311#issuecomment-2874342257 > I think the fact that it is `puffin::Blob`, is quite clear. In ambiguous contexts, someone could always just `use puffin::Blob as PuffinBlob` - we already have similar situati

[I] follow up issue for #12770, to add default parquet column statistic enable config flag [iceberg]

2025-05-12 Thread via GitHub
huaxiangsun opened a new issue, #13035: URL: https://github.com/apache/iceberg/issues/13035 ### Feature Request / Improvement We need to add support to configure Iceberg table's default parquet column statistics enable flag after parquet-java module is upgraded to include fix for htt

[PR] feat: Implement remove properties CLI using catalog commit table API and improve references [iceberg-go]

2025-05-12 Thread via GitHub
lliangyu-lin opened a new pull request, #429: URL: https://github.com/apache/iceberg-go/pull/429 ### Description * Close the gap in properties CLI using the commit table implementation * Update references and loggings as mentioned in #424 ### Testing ``` go run . properties

Re: [PR] feat: Implement remove properties CLI using catalog commit table API and improve references [iceberg-go]

2025-05-12 Thread via GitHub
lliangyu-lin commented on code in PR #429: URL: https://github.com/apache/iceberg-go/pull/429#discussion_r2085621667 ## README.md: ## @@ -70,9 +70,9 @@ $ cd iceberg-go/cmd/iceberg && go build . | Create New Snapshot | X | | | | X | | Rename Tabl

Re: [PR] Add table property to disable/enable parquet column statistics #12770 [iceberg]

2025-05-12 Thread via GitHub
huaxiangsun commented on PR #12771: URL: https://github.com/apache/iceberg/pull/12771#issuecomment-2874340542 @szehon-ho Can you help to merge the patch? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[PR] Expose puffin blob constructor [iceberg-rust]

2025-05-12 Thread via GitHub
dentiny opened a new pull request, #1320: URL: https://github.com/apache/iceberg-rust/pull/1320 ## Which issue does this PR close? This PR expose public construction function for puffin blob, as discussed in the linked issue. The main motivation is: puffin blob is supposed to hold

Re: [PR] Add table property to disable/enable parquet column statistics #12770 [iceberg]

2025-05-12 Thread via GitHub
huaxiangsun commented on PR #12771: URL: https://github.com/apache/iceberg/pull/12771#issuecomment-2874339230 > LGTM! Do you plan to follow up with default column stats config once parquet change is adopted in iceberg? Yeah, created https://github.com/apache/iceberg/issues/13035 to tr

Re: [PR] chore: minor update to manifest sanity check error message [iceberg-rust]

2025-05-12 Thread via GitHub
sdd merged PR #1278: URL: https://github.com/apache/iceberg-rust/pull/1278 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Adding VSCode Devcontainer Configs [iceberg]

2025-05-12 Thread via GitHub
ig-raiderler commented on PR #13034: URL: https://github.com/apache/iceberg/pull/13034#issuecomment-2874243964 Great implementation of this feature. I particularly like the approach to error handling. -- This is an automated message from the Apache Git Service. To respond to the message,

[PR] Build: Bump mkdocs-material from 9.6.12 to 9.6.13 [iceberg-python]

2025-05-12 Thread via GitHub
dependabot[bot] opened a new pull request, #1993: URL: https://github.com/apache/iceberg-python/pull/1993 Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.6.12 to 9.6.13. Release notes Sourced from https://github.com/squidfunk/mkdocs-material/releases";>

[PR] Build: Bump pyroaring from 1.0.0 to 1.0.1 [iceberg-python]

2025-05-12 Thread via GitHub
dependabot[bot] opened a new pull request, #1992: URL: https://github.com/apache/iceberg-python/pull/1992 Bumps [pyroaring](https://github.com/Ezibenroc/PyRoaringBitMap) from 1.0.0 to 1.0.1. Release notes Sourced from https://github.com/Ezibenroc/PyRoaringBitMap/releases";>pyroarin

[PR] Build: Bump getdaft from 0.4.13 to 0.4.14 [iceberg-python]

2025-05-12 Thread via GitHub
dependabot[bot] opened a new pull request, #1991: URL: https://github.com/apache/iceberg-python/pull/1991 Bumps [getdaft](https://github.com/Eventual-Inc/Daft) from 0.4.13 to 0.4.14. Release notes Sourced from https://github.com/Eventual-Inc/Daft/releases";>getdaft's releases.

[PR] Build: Bump mypy-boto3-glue from 1.38.0 to 1.38.12 [iceberg-python]

2025-05-12 Thread via GitHub
dependabot[bot] opened a new pull request, #1990: URL: https://github.com/apache/iceberg-python/pull/1990 Bumps [mypy-boto3-glue](https://github.com/youtype/mypy_boto3_builder) from 1.38.0 to 1.38.12. Release notes Sourced from https://github.com/youtype/mypy_boto3_builder/releases

Re: [PR] Catalog: Add BigQuery Metastore Catalog Support [iceberg]

2025-05-12 Thread via GitHub
danielcweeks commented on code in PR #12808: URL: https://github.com/apache/iceberg/pull/12808#discussion_r2085606448 ## bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/BigQueryMetastoreClientImpl.java: ## @@ -0,0 +1,660 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] Spark4.0 rc5 integration with Iceberg [iceberg]

2025-05-12 Thread via GitHub
huaxingao commented on code in PR #13022: URL: https://github.com/apache/iceberg/pull/13022#discussion_r2085605022 ## core/src/main/java/org/apache/iceberg/MetadataColumns.java: ## @@ -51,7 +51,7 @@ private MetadataColumns() {} public static final int SPEC_ID_COLUMN_ID = Inte

[I] Question / Feature request: make `FileRead` trait `Sync` [iceberg-rust]

2025-05-12 Thread via GitHub
dentiny opened a new issue, #1318: URL: https://github.com/apache/iceberg-rust/issues/1318 ### Is your feature request related to a problem or challenge? Hi team, this is a half feature request, half question issue. TLDR: I don't see problem to make `FileRead` trait `Sync`, with

[PR] feat: declare `FileRead` trait to be `Sync`-safe [iceberg-rust]

2025-05-12 Thread via GitHub
dentiny opened a new pull request, #1319: URL: https://github.com/apache/iceberg-rust/pull/1319 ## Which issue does this PR close? - Closes https://github.com/apache/iceberg-rust/issues/1318 ## What changes are included in this PR? Declare `FileRead` trait `Sync`-safe.

Re: [PR] Materialized View Spec [iceberg]

2025-05-12 Thread via GitHub
RussellSpitzer commented on code in PR #11041: URL: https://github.com/apache/iceberg/pull/11041#discussion_r2085552249 ## format/view-spec.md: ## @@ -160,6 +179,57 @@ Each entry in `version-log` is a struct with the following fields: | _required_ | `timestamp-ms` | Timestamp

Re: [PR] Add API to set location in the transacation [iceberg-rust]

2025-05-12 Thread via GitHub
CTTY commented on PR #1315: URL: https://github.com/apache/iceberg-rust/pull/1315#issuecomment-287410 Closed this as duplicate, please find the new PR: #1317 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Add API to set location in the transacation [iceberg-rust]

2025-05-12 Thread via GitHub
CTTY closed pull request #1315: Add API to set location in the transacation URL: https://github.com/apache/iceberg-rust/pull/1315 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Materialized View Spec [iceberg]

2025-05-12 Thread via GitHub
RussellSpitzer commented on code in PR #11041: URL: https://github.com/apache/iceberg/pull/11041#discussion_r2085537930 ## format/view-spec.md: ## @@ -42,12 +42,28 @@ An atomic swap of one view metadata file for another provides the basis for maki Writers create view metadat

[PR] feat: Add API to set location in the transacation [iceberg-rust]

2025-05-12 Thread via GitHub
CTTY opened a new pull request, #1317: URL: https://github.com/apache/iceberg-rust/pull/1317 ## Which issue does this PR close? - Closes #1316 ## What changes are included in this PR? - Added new API `set_location` in `transaction` ## Are these changes

[I] Support set_location in transaction [iceberg-rust]

2025-05-12 Thread via GitHub
CTTY opened a new issue, #1316: URL: https://github.com/apache/iceberg-rust/issues/1316 ### Is your feature request related to a problem or challenge? Add API in `transaction` to support `TableUpdate::SetLocation` ### Describe the solution you'd like Something as simple

[PR] Add API to set location in the transacation [iceberg-rust]

2025-05-12 Thread via GitHub
CTTY opened a new pull request, #1315: URL: https://github.com/apache/iceberg-rust/pull/1315 ## Which issue does this PR close? - Closes #. ## What changes are included in this PR? - Added new API `set_location` in `transaction` ## Are these changes tes

Re: [PR] fix(table) getPartitionRecord to use field ID rather than field name [iceberg-go]

2025-05-12 Thread via GitHub
loicalleyne commented on code in PR #427: URL: https://github.com/apache/iceberg-go/pull/427#discussion_r2085472448 ## manifest.go: ## @@ -415,31 +415,33 @@ func getFieldIDMap(sc avro.Schema) (map[string]int, map[int]avro.LogicalType) { } result := make(map[st

Re: [PR] Materialized View Spec [iceberg]

2025-05-12 Thread via GitHub
RussellSpitzer commented on code in PR #11041: URL: https://github.com/apache/iceberg/pull/11041#discussion_r2085454632 ## format/view-spec.md: ## @@ -42,12 +42,28 @@ An atomic swap of one view metadata file for another provides the basis for maki Writers create view metadat

Re: [PR] Materialized View Spec [iceberg]

2025-05-12 Thread via GitHub
RussellSpitzer commented on code in PR #11041: URL: https://github.com/apache/iceberg/pull/11041#discussion_r2085450857 ## format/view-spec.md: ## @@ -42,12 +42,28 @@ An atomic swap of one view metadata file for another provides the basis for maki Writers create view metadat

Re: [PR] Materialized View Spec [iceberg]

2025-05-12 Thread via GitHub
RussellSpitzer commented on code in PR #11041: URL: https://github.com/apache/iceberg/pull/11041#discussion_r2085437752 ## format/view-spec.md: ## @@ -42,12 +42,28 @@ An atomic swap of one view metadata file for another provides the basis for maki Writers create view metadat

Re: [PR] Spark: Custom snapshot property from session configuration [iceberg]

2025-05-12 Thread via GitHub
RussellSpitzer commented on PR #12999: URL: https://github.com/apache/iceberg/pull/12999#issuecomment-2873928244 Looks like there is a build issue @cccs-jory, please update and let me know when to run tests again :) -- This is an automated message from the Apache Git Service. To respond t

Re: [PR] REST spec: Add encryption keys [iceberg]

2025-05-12 Thread via GitHub
rdblue commented on PR #12987: URL: https://github.com/apache/iceberg/pull/12987#issuecomment-2873885748 Merging this now that the vote passed. Thanks for the reviews! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] REST spec: Add encryption keys [iceberg]

2025-05-12 Thread via GitHub
rdblue merged PR #12987: URL: https://github.com/apache/iceberg/pull/12987 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] build(deps): bump the gomod_updates group with 4 updates [iceberg-go]

2025-05-12 Thread via GitHub
zeroshade merged PR #428: URL: https://github.com/apache/iceberg-go/pull/428 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Spark: Custom snapshot property from session configuration [iceberg]

2025-05-12 Thread via GitHub
RussellSpitzer commented on code in PR #12999: URL: https://github.com/apache/iceberg/pull/12999#discussion_r2085373090 ## spark/v3.4/spark/src/test/java/org/apache/iceberg/spark/TestSparkWriteConf.java: ## @@ -486,6 +486,36 @@ public void testDeleteFileWriteConf() { } }

Re: [PR] chore: minor update to manifest sanity check error message [iceberg-rust]

2025-05-12 Thread via GitHub
dentiny commented on PR #1278: URL: https://github.com/apache/iceberg-rust/pull/1278#issuecomment-2873857930 > LGTM, thanks for this @dentiny Thank you for the code review @sdd ! Could you please help merge when you have some time? Thank you! -- This is an automated message from th

Re: [PR] Build: Bump mkdocs-material from 9.6.12 to 9.6.13 [iceberg]

2025-05-12 Thread via GitHub
Fokko merged PR #13029: URL: https://github.com/apache/iceberg/pull/13029 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Adding VSCode Devcontainer Configs [iceberg]

2025-05-12 Thread via GitHub
bpkroth commented on code in PR #13034: URL: https://github.com/apache/iceberg/pull/13034#discussion_r2085326688 ## .devcontainer/devcontainer.json: ## @@ -0,0 +1,54 @@ +// For format details, see https://aka.ms/devcontainer.json. For config options, see the +// README at: http

Re: [PR] Spark: Custom snapshot property from session configuration [iceberg]

2025-05-12 Thread via GitHub
RussellSpitzer commented on PR #12999: URL: https://github.com/apache/iceberg/pull/12999#issuecomment-2873804902 Please only target one Spark Version in this PR for easier reviewing, we can do a fast followup that changes earlier versions after we merge this. -- This is an automated messa

Re: [PR] Parquet: Fix to ensure that last updated sequence numbers for V2 and earlier tables are null [iceberg]

2025-05-12 Thread via GitHub
RussellSpitzer commented on PR #13001: URL: https://github.com/apache/iceberg/pull/13001#issuecomment-2873802043 Merged! Thanks @amogh-jahagirdar for the fix and @nastra for reviewing. I think this one should get into 1.9.1 as well -- This is an automated message from the Apache Git Servi

  1   2   3   >