Re: [PR] 16x New dremio iceberg blogs sept 2025 [iceberg]

2025-09-20 Thread via GitHub
RussellSpitzer closed pull request #14105: 16x New dremio iceberg blogs sept 2025 URL: https://github.com/apache/iceberg/pull/14105 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] 16x New dremio iceberg blogs sept 2025 [iceberg]

2025-09-20 Thread via GitHub
RussellSpitzer commented on PR #14105: URL: https://github.com/apache/iceberg/pull/14105#issuecomment-3315138855 Discussion of the future of the blogs page continues here https://lists.apache.org/thread/8n5jygtol1bgyw6ow7wjt7c8m3v3t0l7 -- This is an automated message from the Apache Git S

Re: [PR] Build: Bump hadoop from 3.4.1 to 3.4.2. [iceberg]

2025-09-20 Thread via GitHub
huaxingao commented on PR #14125: URL: https://github.com/apache/iceberg/pull/14125#issuecomment-3315131566 Thanks @slfan1989 for the PR! Thanks @Fokko for the review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] Build: Bump hadoop from 3.4.1 to 3.4.2. [iceberg]

2025-09-20 Thread via GitHub
huaxingao merged PR #14125: URL: https://github.com/apache/iceberg/pull/14125 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] BigQuery: Add table validity check for BigQueryMetastoreCatalog [iceberg]

2025-09-20 Thread via GitHub
nastra commented on code in PR #14113: URL: https://github.com/apache/iceberg/pull/14113#discussion_r2364037650 ## bigquery/src/test/java/org/apache/iceberg/gcp/bigquery/TestBigQueryCatalog.java: ## @@ -21,11 +21,19 @@ import static org.apache.iceberg.CatalogUtil.ICEBERG_CATALO

Re: [PR] Docs: improve local editing experience [iceberg]

2025-09-20 Thread via GitHub
kevinjqliu commented on PR #13664: URL: https://github.com/apache/iceberg/pull/13664#issuecomment-3315101304 hey @yeswanth-s1th thanks for working on this! I tried to run this locally too and got the same error I took a look at the code path for `make serve` and it looks like we're

Re: [PR] Manifest list encryption [iceberg]

2025-09-20 Thread via GitHub
RussellSpitzer commented on code in PR #7770: URL: https://github.com/apache/iceberg/pull/7770#discussion_r2342348861 ## core/src/main/java/org/apache/iceberg/encryption/StandardEncryptionManager.java: ## @@ -18,21 +18,49 @@ */ package org.apache.iceberg.encryption; +import

Re: [PR] Flink: Dynamic Sink: Add error stream for graceful error handling [iceberg]

2025-09-20 Thread via GitHub
pvary commented on code in PR #13963: URL: https://github.com/apache/iceberg/pull/13963#discussion_r2343879909 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicRecordProcessor.java: ## @@ -37,17 +39,21 @@ class DynamicRecordProcessor extends Process

Re: [PR] feat: bump datafusion to 48 [iceberg-rust]

2025-09-20 Thread via GitHub
kevinjqliu commented on PR #1501: URL: https://github.com/apache/iceberg-rust/pull/1501#issuecomment-3275292345 Thank you for the PR @colinmarc and thanks for the review @liurenjie1024 @fo -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] Parquet, Core: Allows Internal Parquet Readers to use Custom Types [iceberg]

2025-09-20 Thread via GitHub
RussellSpitzer commented on code in PR #14040: URL: https://github.com/apache/iceberg/pull/14040#discussion_r2337453015 ## core/src/test/java/org/apache/iceberg/TestInternalData.java: ## @@ -0,0 +1,245 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [I] Schema evolution issue with structs nested inside map fields (Parquet-backed Iceberg tables) [iceberg]

2025-09-20 Thread via GitHub
amogh-jahagirdar commented on issue #14043: URL: https://github.com/apache/iceberg/issues/14043#issuecomment-3299330390 Yeah adding fields to structs which are map _values_ should work. The project has a spark integration test which actually validates this https://github.com/apache/iceberg/

[PR] Spark 3.4: Backport: Add procedure and action to compute partition stats. [iceberg]

2025-09-20 Thread via GitHub
slfan1989 opened a new pull request, #14034: URL: https://github.com/apache/iceberg/pull/14034 backport #12450, #13480 to Spark3.4 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] Flink: Dynamic Sink: Add error stream for graceful error handling [iceberg]

2025-09-20 Thread via GitHub
pvary commented on code in PR #13963: URL: https://github.com/apache/iceberg/pull/13963#discussion_r2343883388 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicTableUpdateOperator.java: ## @@ -64,10 +72,37 @@ public void open(OpenContext openContext

Re: [I] Support deletion in Apache Flink [iceberg]

2025-09-20 Thread via GitHub
rlokugamage commented on issue #8718: URL: https://github.com/apache/iceberg/issues/8718#issuecomment-3300493405 awesome, do you know if it's writing equality or positional deletes? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [I] [DISCUSS] Selecting a C++ HTTP Client Library for REST [iceberg-cpp]

2025-09-20 Thread via GitHub
dongxiao1198 commented on issue #232: URL: https://github.com/apache/iceberg-cpp/issues/232#issuecomment-3301180973 Libcurl is a low-level common lib,has a lot of settings, but it is stable & small & no-extra dependency. CPR is much more modern and easy to use, but itself is a 3rd lib, we

Re: [PR] feat: support deleted rows in Comet [iceberg]

2025-09-20 Thread via GitHub
parthchandra commented on PR #14062: URL: https://github.com/apache/iceberg/pull/14062#issuecomment-3285963787 @huaxingao @anuragmantri @hsiang-c -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Flink: Dynamic Sink: Add error stream for graceful error handling [iceberg]

2025-09-20 Thread via GitHub
pvary commented on code in PR #13963: URL: https://github.com/apache/iceberg/pull/13963#discussion_r2343900993 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicIcebergSink.java: ## @@ -374,32 +416,86 @@ public DataStreamSink append() {

Re: [I] No pushdown filters applied if temporal type was optimized into timestamp type by datafusion [iceberg-rust]

2025-09-20 Thread via GitHub
github-actions[bot] commented on issue #942: URL: https://github.com/apache/iceberg-rust/issues/942#issuecomment-3287224386 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occu

Re: [I] Connection lost during persistTable with Hive Locks can cause overwritten metadata location [iceberg]

2025-09-20 Thread via GitHub
pvary commented on issue #14096: URL: https://github.com/apache/iceberg/issues/14096#issuecomment-3311947928 > Is there a benefit to not passing the expected metadata location when using hive locks? Is it just performance? The original goal was to have a way to revert back to the orig

Re: [I] Dectect schema evolution or partition evolution for append DataFile [iceberg-rust]

2025-09-20 Thread via GitHub
github-actions[bot] commented on issue #777: URL: https://github.com/apache/iceberg-rust/issues/777#issuecomment-3294404653 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occu

Re: [PR] Core: Refactor: Rename resp to response in RESTCatalogAdapter [iceberg]

2025-09-20 Thread via GitHub
Fokko commented on PR #14051: URL: https://github.com/apache/iceberg/pull/14051#issuecomment-3281705581 Thanks @gaborkaszab for bringing this in line, and thanks @huaxingao and @amogh-jahagirdar for the quick review 🚀 -- This is an automated message from the Apache Git Service. To respon

Re: [PR] Spark 4.0: Refactor Spark procedures to consistently use ProcedureInput for parameter handling. [iceberg]

2025-09-20 Thread via GitHub
nastra commented on code in PR #13913: URL: https://github.com/apache/iceberg/pull/13913#discussion_r2362940532 ## spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/procedures/ExpireSnapshotsProcedure.java: ## @@ -104,13 +118,14 @@ public ProcedureParameter[] parameters()

Re: [PR] BigQuery: Add table validity check for BigQueryMetastoreCatalog [iceberg]

2025-09-20 Thread via GitHub
kevinjqliu commented on code in PR #14113: URL: https://github.com/apache/iceberg/pull/14113#discussion_r2364015282 ## bigquery/src/test/java/org/apache/iceberg/gcp/bigquery/TestBigQueryCatalog.java: ## @@ -171,4 +181,85 @@ public void renameTableNamespaceMissing() { public v

Re: [PR] Core: Fix Scan Plan API resource paths [iceberg]

2025-09-20 Thread via GitHub
amogh-jahagirdar merged PR #14120: URL: https://github.com/apache/iceberg/pull/14120 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] Core: Fix Scan Plan API resource paths [iceberg]

2025-09-20 Thread via GitHub
amogh-jahagirdar commented on PR #14120: URL: https://github.com/apache/iceberg/pull/14120#issuecomment-3315071443 I'll go ahead and merge, thanks @singhpk234 and thanks everyone for reviewing! -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Core: Fix Scan Plan API resource paths [iceberg]

2025-09-20 Thread via GitHub
amogh-jahagirdar commented on PR #14120: URL: https://github.com/apache/iceberg/pull/14120#issuecomment-3315071303 >@amogh-jahagirdar Isn't duck db using these routes? No, not to my knowledge since duckDB is implemented in C++. On the server side, I also don't know of any production serve

Re: [I] Improve Site Build Time [iceberg]

2025-09-20 Thread via GitHub
kevinjqliu commented on issue #14124: URL: https://github.com/apache/iceberg/issues/14124#issuecomment-3315065129 +1! i raised the same issue here https://github.com/apache/iceberg/issues/13661 and theres some work already in progress :) -- This is an automated message from the Apache

Re: [I] cmake hang [iceberg-cpp]

2025-09-20 Thread via GitHub
Tangruilin commented on issue #240: URL: https://github.com/apache/iceberg-cpp/issues/240#issuecomment-3315058505 > > You might need to set up http(s) proxy. It may hang when trying to download vendored dependencies. > > https://private-user-images.githubusercontent.com/23651891/49194

Re: [I] cmake hang [iceberg-cpp]

2025-09-20 Thread via GitHub
Tangruilin commented on issue #240: URL: https://github.com/apache/iceberg-cpp/issues/240#issuecomment-3315058120 > You might need to set up http(s) proxy. It may hang when trying to download vendored dependencies. https://github.com/user-attachments/assets/690dc3b4-301b-4522-ab66-851

Re: [I] build deps mirror for China [iceberg-cpp]

2025-09-20 Thread via GitHub
wgtmac commented on issue #239: URL: https://github.com/apache/iceberg-cpp/issues/239#issuecomment-3314996422 Perhaps we can add alternative URLs for 3rd party dependencies except avro-cpp which is based on an unreleased version. -- This is an automated message from the Apache Git Service

Re: [I] cmake hang [iceberg-cpp]

2025-09-20 Thread via GitHub
wgtmac commented on issue #240: URL: https://github.com/apache/iceberg-cpp/issues/240#issuecomment-3314995705 You might need to set up http(s) proxy. It may hang when trying to download vendored dependencies. -- This is an automated message from the Apache Git Service. To respond to the m

[I] cmake hang [iceberg-cpp]

2025-09-20 Thread via GitHub
Tangruilin opened a new issue, #240: URL: https://github.com/apache/iceberg-cpp/issues/240 https://github.com/user-attachments/assets/394cfc33-900d-4d95-bcbc-9f4a5fd2a965"; /> command `cmake -S . -B build -G Ninja -DCMAKE_INSTALL_PREFIX=./install -DICEBERG_BUILD_STATIC=ON -DICEBERG_BUILD

Re: [PR] Spark: Fix Z-order UDF to correctly handle DateType [iceberg]

2025-09-20 Thread via GitHub
nastra commented on code in PR #14108: URL: https://github.com/apache/iceberg/pull/14108#discussion_r2363020061 ## spark/v4.0/spark/src/test/java/org/apache/iceberg/spark/actions/TestRewriteDataFilesAction.java: ## @@ -2110,6 +2113,29 @@ public void testExecutorCacheForDeleteFil

[I] Conversion of pyarrow schema to pyiceberg schema [iceberg-python]

2025-09-20 Thread via GitHub
din14970 opened a new issue, #2488: URL: https://github.com/apache/iceberg-python/issues/2488 ### Feature Request / Improvement Converting a pyiceberg schema to a pyarrow schema is natively supported in pyiceberg with `as_arrow()`. However, it would be convenient to be able to conver

Re: [PR] Support Snowflake-Managed Iceberg Tables via SnowflakeCatalog [iceberg-python]

2025-09-20 Thread via GitHub
corleyma commented on PR #1834: URL: https://github.com/apache/iceberg-python/pull/1834#issuecomment-3314140342 @monti-python I think the best way to unblock yourself for Snowflake is to do like this article suggests, and sync the Snowflake-managed Iceberg table with an "Open Catalog" (aka

Re: [PR] Spark 4.0: Refactor Spark procedures to consistently use ProcedureInput for parameter handling. [iceberg]

2025-09-20 Thread via GitHub
slfan1989 commented on code in PR #13913: URL: https://github.com/apache/iceberg/pull/13913#discussion_r2364993020 ## spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/procedures/ProcedureInput.java: ## @@ -80,6 +82,22 @@ public Integer asInt(ProcedureParameter param, Inte

Re: [PR] BigQuery: Add table validity check for BigQueryMetastoreCatalog [iceberg]

2025-09-20 Thread via GitHub
nastra commented on code in PR #14113: URL: https://github.com/apache/iceberg/pull/14113#discussion_r2364027684 ## bigquery/src/test/java/org/apache/iceberg/gcp/bigquery/TestBigQueryCatalog.java: ## @@ -144,9 +154,7 @@ public void createTableTransaction(int formatVersion) { @

[I] Remove Spark 3.4 in Iceberg 1.12 release [iceberg]

2025-09-20 Thread via GitHub
kevinjqliu opened a new issue, #14121: URL: https://github.com/apache/iceberg/issues/14121 ### Feature Request / Improvement Current Spark 3.4 deprecation: * devlist thread, https://lists.apache.org/thread/n8qy5nsdw35vjog5lb4kp15989b800cv * Marked for deprecation, https://github

Re: [I] [Spec] Linking Schema ID to Data & Delete Files [iceberg]

2025-09-20 Thread via GitHub
manirajv06 commented on issue #13855: URL: https://github.com/apache/iceberg/issues/13855#issuecomment-3314798350 > Just so I understand the new field on the java class will be entirely ephemeral, and populated after opening the parquet footer? Yes. I've created a [draft PR](h

[PR] Collect list of columns written in file [iceberg]

2025-09-20 Thread via GitHub
manirajv06 opened a new pull request, #14126: URL: https://github.com/apache/iceberg/pull/14126 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] feat(metadata): validated SortOrders constructor, validations & tests [iceberg-go]

2025-09-20 Thread via GitHub
twuebi commented on code in PR #571: URL: https://github.com/apache/iceberg-go/pull/571#discussion_r2365426450 ## table/sorting.go: ## @@ -125,24 +126,102 @@ const ( ) // A default Sort Order indicating no sort order at all -var UnsortedSortOrder = SortOrder{OrderID: Unsorte

[I] build deps mirror for China [iceberg-cpp]

2025-09-20 Thread via GitHub
Tangruilin opened a new issue, #239: URL: https://github.com/apache/iceberg-cpp/issues/239 now the cmake deps are all github.com. developers in China are easy to build failed occurs to network. -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [I] Connection lost during persistTable with Hive Locks can cause overwritten metadata location [iceberg]

2025-09-19 Thread via GitHub
Noremac201 commented on issue #14096: URL: https://github.com/apache/iceberg/issues/14096#issuecomment-3313572113 My expectation is that the API call fails, precisely because HMS's insert fails for this out of order write due to `expected_parameter_key`. However, since the client lost conne

Re: [I] [docs] Indicate navigation links that take the user to an external site [iceberg]

2025-09-19 Thread via GitHub
kevinjqliu commented on issue #13514: URL: https://github.com/apache/iceberg/issues/13514#issuecomment-3314158982 found link-marker from https://github.com/mkdocs/catalog/blob/main/README.md#-links--references https://github.com/timmeinerzhagen/mkdocs-link-marker Looks like this is

Re: [PR] BigQuery: Add table validity check for BigQueryMetastoreCatalog [iceberg]

2025-09-19 Thread via GitHub
nastra commented on code in PR #14113: URL: https://github.com/apache/iceberg/pull/14113#discussion_r2364039562 ## bigquery/src/test/java/org/apache/iceberg/gcp/bigquery/TestBigQueryCatalog.java: ## @@ -39,6 +47,8 @@ import org.junit.jupiter.api.io.TempDir; import org.junit.ju

[PR] Build: Bump hadoop from 3.4.1 to 3.4.2. [iceberg]

2025-09-19 Thread via GitHub
slfan1989 opened a new pull request, #14125: URL: https://github.com/apache/iceberg/pull/14125 Hadoop 3.4.2 has been released, and we can consider upgrading to this version. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] follow up issue for #12770, to add default parquet column statistic enable config flag [iceberg]

2025-09-19 Thread via GitHub
pvary commented on issue #13035: URL: https://github.com/apache/iceberg/issues/13035#issuecomment-3311988177 The parquet version is 1.16.0 now, so we can move forward with this config. @huaxiangsun: Are you planning to work on this? -- This is an automated message from the Apache Gi

[PR] perf(DataScan): parallelize FileScanTask with executor [iceberg-python]

2025-09-19 Thread via GitHub
jayceslesar opened a new pull request, #2487: URL: https://github.com/apache/iceberg-python/pull/2487 # Rationale for this change Performance! `plan_files` should generally be faster. ## Are these changes tested? Existing tests pass -- This is an automated message from the

Re: [PR] Fixed the limit bug and added test for count() method and documentation for count() [iceberg-python]

2025-09-19 Thread via GitHub
tushar-choudhary-tc commented on PR #2423: URL: https://github.com/apache/iceberg-python/pull/2423#issuecomment-3314608206 Hey @kevinjqliu this looks good to me -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] API: Follow up on adding Variant data type to implement sanitizing for Variant [iceberg]

2025-09-19 Thread via GitHub
aihuaxu closed issue #11479: API: Follow up on adding Variant data type to implement sanitizing for Variant URL: https://github.com/apache/iceberg/issues/11479 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Remove user flag from poetry installation [iceberg-python]

2025-09-19 Thread via GitHub
jayceslesar commented on PR #2486: URL: https://github.com/apache/iceberg-python/pull/2486#issuecomment-3312884835 ah I wonder if this has been the source of my issues, I was just manually created a venv with uv, installing poetry and then running `make install-dependencies` but I imagine t

Re: [I] Exploring Enhanced Compaction Support in Rust [iceberg-rust]

2025-09-19 Thread via GitHub
github-actions[bot] commented on issue #657: URL: https://github.com/apache/iceberg-rust/issues/657#issuecomment-3314270219 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occu

Re: [PR] fix: validate schema and partition field name conflicts during updates [iceberg]

2025-09-19 Thread via GitHub
github-actions[bot] commented on PR #13835: URL: https://github.com/apache/iceberg/pull/13835#issuecomment-3314270621 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] feat: support longer width for long and decimal truncation [iceberg]

2025-09-19 Thread via GitHub
github-actions[bot] commented on PR #13860: URL: https://github.com/apache/iceberg/pull/13860#issuecomment-3314270671 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] discuss: re-export arrow types [iceberg-rust]

2025-09-19 Thread via GitHub
github-actions[bot] commented on issue #636: URL: https://github.com/apache/iceberg-rust/issues/636#issuecomment-3314270249 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occu

Re: [PR] Make view metadata path configurable by `write.metadata.path` [iceberg]

2025-09-19 Thread via GitHub
flyrain commented on PR #12017: URL: https://github.com/apache/iceberg/pull/12017#issuecomment-3314215098 @tomtongue thanks for the reply. Sorry for the delay reply. I can see that additional `metadata/` isn't necessary, if we could redesign it, we would remove it. Given it is there

Re: [PR] fix(Catalog): Handle NotFound exception for missing metadata file [iceberg]

2025-09-19 Thread via GitHub
flyrain commented on code in PR #13143: URL: https://github.com/apache/iceberg/pull/13143#discussion_r2364803142 ## core/src/test/java/org/apache/iceberg/catalog/CatalogTests.java: ## @@ -959,6 +966,35 @@ public void testLoadMissingTable() { .hasMessageStartingWith("Tab

Re: [PR] Docs: lint markdown files in site build [iceberg]

2025-09-19 Thread via GitHub
kevinjqliu commented on code in PR #13977: URL: https://github.com/apache/iceberg/pull/13977#discussion_r2364811503 ## site/dev/common.sh: ## @@ -233,6 +233,16 @@ pull_versioned_docs () { create_nightly } +scan_markdown_files () { + echo " --> check markdown file styles

Re: [PR] Site: Remove Blogs and Talks From Site [iceberg]

2025-09-19 Thread via GitHub
dramaticlly commented on code in PR #14110: URL: https://github.com/apache/iceberg/pull/14110#discussion_r2364802996 ## docs/docs/aws.md: ## @@ -816,4 +815,4 @@ to run fully managed Apache Flink applications. You can include Iceberg in your [AWS Redshift Spectrum or Redshift

Re: [PR] Core, OpenAPI: Implement refs snapshot mode in reference IRC [iceberg]

2025-09-19 Thread via GitHub
nastra commented on code in PR #14060: URL: https://github.com/apache/iceberg/pull/14060#discussion_r2362966624 ## core/src/main/java/org/apache/iceberg/rest/HTTPRequest.java: ## @@ -33,6 +33,7 @@ @Value.Immutable @SuppressWarnings({"ImmutablesStyle", "SafeLoggingPropagation"}

Re: [PR] Site: Remove Blogs and Talks From Site [iceberg]

2025-09-19 Thread via GitHub
szehon-ho commented on code in PR #14110: URL: https://github.com/apache/iceberg/pull/14110#discussion_r2364784520 ## docs/docs/aws.md: ## @@ -816,4 +815,4 @@ to run fully managed Apache Flink applications. You can include Iceberg in your [AWS Redshift Spectrum or Redshift Se

Re: [I] [docs] Indicate navigation links that take the user to an external site [iceberg]

2025-09-19 Thread via GitHub
kevinjqliu commented on issue #13514: URL: https://github.com/apache/iceberg/issues/13514#issuecomment-3314150188 +1! this would be great. Do you know how we can go about implementing this? Maybe there's a plugin to do this -- This is an automated message from the Apache Git Service. To r

[I] Improve Site Build Time [iceberg]

2025-09-19 Thread via GitHub
RussellSpitzer opened a new issue, #14124: URL: https://github.com/apache/iceberg/issues/14124 ### Feature Request / Improvement The site, with versioned api docs, now takes an extremely long time to build. We should figure out how to keep the old versioned apis static or something t

Re: [PR] Support Snowflake-Managed Iceberg Tables via SnowflakeCatalog [iceberg-python]

2025-09-19 Thread via GitHub
monti-python commented on PR #1834: URL: https://github.com/apache/iceberg-python/pull/1834#issuecomment-3313941885 @iamontheinet @jdanielmyers What's your take on this? At my org, we'd love to use Snowflake-managed Iceberg tables but the lack of support for them in pyiceberg has been

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-09-19 Thread via GitHub
RussellSpitzer commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r2363274764 ## core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java: ## @@ -299,6 +309,14 @@ protected MetricsReporter metricsReporter() { return metricsRe

Re: [PR] Detect the case to identify missing column from the file using file's max field id in StrictMetricsEvaluator #13397 [iceberg]

2025-09-19 Thread via GitHub
manirajv06 commented on PR #13398: URL: https://github.com/apache/iceberg/pull/13398#issuecomment-3313138367 To remove "stale" label -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] API: Add geospatial bounding box types and implement intersects checking [iceberg]

2025-09-19 Thread via GitHub
jiayuasu commented on PR #12667: URL: https://github.com/apache/iceberg/pull/12667#issuecomment-3313995873 Yes, I think we can add it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] BigQuery: Add table validity check for BigQueryMetastoreCatalog [iceberg]

2025-09-19 Thread via GitHub
nastra commented on code in PR #14113: URL: https://github.com/apache/iceberg/pull/14113#discussion_r2364025070 ## bigquery/src/test/java/org/apache/iceberg/gcp/bigquery/TestBigQueryCatalog.java: ## @@ -144,9 +154,61 @@ public void createTableTransaction(int formatVersion) {

Re: [PR] Core, OpenAPI: Implement refs snapshot mode in reference IRC [iceberg]

2025-09-19 Thread via GitHub
huaxingao commented on code in PR #14060: URL: https://github.com/apache/iceberg/pull/14060#discussion_r2364639777 ## core/src/test/java/org/apache/iceberg/rest/RESTCatalogAdapter.java: ## @@ -426,7 +428,10 @@ public T handleRequest( case LOAD_TABLE: {

Re: [PR] Spark: Fix Z-order UDF to correctly handle DateType [iceberg]

2025-09-19 Thread via GitHub
RussellSpitzer commented on PR #14108: URL: https://github.com/apache/iceberg/pull/14108#issuecomment-3313962844 ``` Execution failed for task ':iceberg-spark:iceberg-spark-4.0_2.13:spotlessJavaCheck'. > The following files had format violations: src/test/java/org/apache/iceb

Re: [PR] Add SQL UDF spec [iceberg]

2025-09-19 Thread via GitHub
stevenzwu commented on code in PR #14117: URL: https://github.com/apache/iceberg/pull/14117#discussion_r2363929131 ## format/udf-spec.md: ## @@ -0,0 +1,221 @@ +--- +title: "SQL UDF Spec" +--- + + +# Iceberg SQL UDF Spec + +## Background and Motivation + +A SQL user-defined funct

Re: [PR] BigQuery: Add table validity check for BigQueryMetastoreCatalog [iceberg]

2025-09-19 Thread via GitHub
kevinjqliu commented on PR #14113: URL: https://github.com/apache/iceberg/pull/14113#issuecomment-3313759432 Thanks @thomaschow and thanks everyone for the review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] [Core] Add mergeAppendTest to ensure consist distribution of data files in manifests [iceberg]

2025-09-19 Thread via GitHub
huaxingao merged PR #14111: URL: https://github.com/apache/iceberg/pull/14111 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] [Core] Add mergeAppendTest to ensure consist distribution of data files in manifests [iceberg]

2025-09-19 Thread via GitHub
huaxingao commented on PR #14111: URL: https://github.com/apache/iceberg/pull/14111#issuecomment-3313816075 Thanks @dramaticlly for the PR! Thanks @stevenzwu @singhpk234 for the review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] Core: Fix Scan Plan API resource paths [iceberg]

2025-09-19 Thread via GitHub
stevenzwu commented on code in PR #14120: URL: https://github.com/apache/iceberg/pull/14120#discussion_r2364520510 ## .palantir/revapi.yml: ## @@ -1365,6 +1365,18 @@ acceptedBreaks: justification: "Removing deprecations for 1.10.0" "1.10.0": org.apache.iceberg:ice

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-09-19 Thread via GitHub
dramaticlly commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r2364430805 ## core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java: ## @@ -299,6 +309,14 @@ protected MetricsReporter metricsReporter() { return metricsRepor

Re: [PR] Add "stop signs" for sensitive information/issues to all issue templates [iceberg]

2025-09-19 Thread via GitHub
kevinjqliu merged PR #14103: URL: https://github.com/apache/iceberg/pull/14103 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] [SPEC] Add finer grained read restrictions as part of loadTable [iceberg]

2025-09-19 Thread via GitHub
stevenzwu commented on code in PR #13879: URL: https://github.com/apache/iceberg/pull/13879#discussion_r2364497749 ## open-api/rest-catalog-open-api.yaml: ## @@ -3260,6 +3260,74 @@ components: additionalProperties: type: string +ReadRestrictions: +

Re: [PR] [SPEC] Add finer grained read restrictions as part of loadTable [iceberg]

2025-09-19 Thread via GitHub
stevenzwu commented on code in PR #13879: URL: https://github.com/apache/iceberg/pull/13879#discussion_r2364495758 ## open-api/rest-catalog-open-api.yaml: ## @@ -3260,6 +3260,74 @@ components: additionalProperties: type: string +ReadRestrictions: +

Re: [PR] BigQuery: Add table validity check for BigQueryMetastoreCatalog [iceberg]

2025-09-19 Thread via GitHub
kevinjqliu merged PR #14113: URL: https://github.com/apache/iceberg/pull/14113 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Core: Fix Scan Plan API resource paths [iceberg]

2025-09-19 Thread via GitHub
RussellSpitzer commented on PR #14120: URL: https://github.com/apache/iceberg/pull/14120#issuecomment-3313753837 @amogh-jahagirdar Isn't duck db using these routes? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] Core: Fix Scan Plan API resource paths [iceberg]

2025-09-19 Thread via GitHub
dramaticlly commented on code in PR #14120: URL: https://github.com/apache/iceberg/pull/14120#discussion_r2364470319 ## .palantir/revapi.yml: ## @@ -1365,6 +1365,18 @@ acceptedBreaks: justification: "Removing deprecations for 1.10.0" "1.10.0": org.apache.iceberg:i

[I] Migrating existing Spark tables with Variant type fails with Iceberg [iceberg]

2025-09-19 Thread via GitHub
hariuserx opened a new issue, #14123: URL: https://github.com/apache/iceberg/issues/14123 ### Apache Iceberg version 1.10.0 (latest release) ### Query engine Spark ### Please describe the bug 🐞 Spark 4.0 introduced "Variant type" (https://www.databricks.com

Re: [PR] Add SQL UDF spec [iceberg]

2025-09-19 Thread via GitHub
stevenzwu commented on code in PR #14117: URL: https://github.com/apache/iceberg/pull/14117#discussion_r2364008862 ## format/udf-spec.md: ## @@ -0,0 +1,221 @@ +--- +title: "SQL UDF Spec" +--- + + +# Iceberg SQL UDF Spec + +## Background and Motivation + +A SQL user-defined funct

Re: [PR] Core: Fix Scan Plan API resource paths [iceberg]

2025-09-19 Thread via GitHub
amogh-jahagirdar commented on code in PR #14120: URL: https://github.com/apache/iceberg/pull/14120#discussion_r2364214139 ## .palantir/revapi.yml: ## @@ -1365,6 +1365,18 @@ acceptedBreaks: justification: "Removing deprecations for 1.10.0" "1.10.0": org.apache.iceb

[I] StaticTable::from_metadata_file() fails to read metadata for a table that was upgraded from v1 to v2 [iceberg-rust]

2025-09-19 Thread via GitHub
aditya-subrahmanyan opened a new issue, #1697: URL: https://github.com/apache/iceberg-rust/issues/1697 ### Apache Iceberg Rust version 0.5.1 ### Describe the bug According to the [spec](https://iceberg.apache.org/spec/#version-2), the `sequence-number` field for a snapsh

[PR] Spark: Remove Spark 3.4 support [iceberg]

2025-09-19 Thread via GitHub
kevinjqliu opened a new pull request, #14122: URL: https://github.com/apache/iceberg/pull/14122 Closes #14121 Spark 3.4 support is deprecated in 1.11.0, so this PR removes it from main branch after 1.11.0 release. Screenshots for the doc changes -- This is an automated

Re: [PR] BigQuery: Add table validity check for BigQueryMetastoreCatalog [iceberg]

2025-09-19 Thread via GitHub
thomaschow commented on code in PR #14113: URL: https://github.com/apache/iceberg/pull/14113#discussion_r2364159875 ## bigquery/src/test/java/org/apache/iceberg/gcp/bigquery/TestBigQueryCatalog.java: ## @@ -171,4 +181,85 @@ public void renameTableNamespaceMissing() { public v

Re: [I] feedback from release verification [iceberg-python]

2025-09-19 Thread via GitHub
kevinjqliu closed issue #2425: feedback from release verification URL: https://github.com/apache/iceberg-python/issues/2425 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

Re: [PR] Fix variant type filtering in ParquetMetricsRowGroupFilter [iceberg]

2025-09-19 Thread via GitHub
nastra commented on code in PR #14081: URL: https://github.com/apache/iceberg/pull/14081#discussion_r2364016484 ## data/src/test/java/org/apache/iceberg/data/TestMetricsRowGroupFilter.java: ## @@ -988,6 +1004,67 @@ public void testTransformFilter() { .isTrue(); } +

Re: [I] feedback from release verification [iceberg-python]

2025-09-19 Thread via GitHub
kevinjqliu commented on issue #2425: URL: https://github.com/apache/iceberg-python/issues/2425#issuecomment-3313160671 Thanks @geruh i believe we address most of the issues above. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [PR] API: Add geospatial bounding box types and implement intersects checking [iceberg]

2025-09-19 Thread via GitHub
szehon-ho commented on PR #12667: URL: https://github.com/apache/iceberg/pull/12667#issuecomment-3313395441 Sure I can make a thread, but may be end of day. So, assuming this, we add validation against wraparound bounds (like xmax > xmin) here then? -- This is an automated message from

Re: [PR] Core: Fix Scan Plan API resource paths [iceberg]

2025-09-19 Thread via GitHub
nastra commented on code in PR #14120: URL: https://github.com/apache/iceberg/pull/14120#discussion_r2364080561 ## .palantir/revapi.yml: ## @@ -1365,6 +1365,18 @@ acceptedBreaks: justification: "Removing deprecations for 1.10.0" "1.10.0": org.apache.iceberg:iceber

Re: [PR] Fix variant type filtering in ParquetMetricsRowGroupFilter [iceberg]

2025-09-19 Thread via GitHub
nastra commented on code in PR #14081: URL: https://github.com/apache/iceberg/pull/14081#discussion_r2364015645 ## data/src/test/java/org/apache/iceberg/data/TestMetricsRowGroupFilter.java: ## @@ -988,6 +1004,67 @@ public void testTransformFilter() { .isTrue(); } +

Re: [PR] Core: Fix Scan Plan API resource paths [iceberg]

2025-09-19 Thread via GitHub
nastra commented on code in PR #14120: URL: https://github.com/apache/iceberg/pull/14120#discussion_r2364076023 ## .palantir/revapi.yml: ## @@ -1365,6 +1365,18 @@ acceptedBreaks: justification: "Removing deprecations for 1.10.0" "1.10.0": org.apache.iceberg:iceber

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-09-19 Thread via GitHub
RussellSpitzer commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r2363249976 ## core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java: ## @@ -71,23 +70,34 @@ public Table loadTable(TableIdentifier identifier) { } @Over

Re: [PR] BigQuery: Add table validity check for BigQueryMetastoreCatalog [iceberg]

2025-09-19 Thread via GitHub
nastra commented on code in PR #14113: URL: https://github.com/apache/iceberg/pull/14113#discussion_r2364038865 ## bigquery/src/test/java/org/apache/iceberg/gcp/bigquery/TestBigQueryCatalog.java: ## @@ -21,11 +21,19 @@ import static org.apache.iceberg.CatalogUtil.ICEBERG_CATALO

Re: [PR] BigQuery: Add table validity check for BigQueryMetastoreCatalog [iceberg]

2025-09-19 Thread via GitHub
nastra commented on code in PR #14113: URL: https://github.com/apache/iceberg/pull/14113#discussion_r2364034397 ## bigquery/src/test/java/org/apache/iceberg/gcp/bigquery/TestBigQueryCatalog.java: ## @@ -171,4 +177,30 @@ public void renameTableNamespaceMissing() { public void

Re: [PR] BigQuery: Add table validity check for BigQueryMetastoreCatalog [iceberg]

2025-09-19 Thread via GitHub
nastra commented on code in PR #14113: URL: https://github.com/apache/iceberg/pull/14113#discussion_r2364030812 ## bigquery/src/test/java/org/apache/iceberg/gcp/bigquery/TestBigQueryCatalog.java: ## @@ -144,9 +154,6 @@ public void createTableTransaction(int formatVersion) { @

Re: [I] follow up issue for #12770, to add default parquet column statistic enable config flag [iceberg]

2025-09-19 Thread via GitHub
huaxiangsun commented on issue #13035: URL: https://github.com/apache/iceberg/issues/13035#issuecomment-3312474796 Thanks for reminding, let me upload a patch today. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

  1   2   3   4   5   6   7   8   9   10   >