Re: [PR] Update schema projection to support `initial-defaults` [iceberg-python]

Fokko merged PR #1644: URL: https://github.com/apache/iceberg-python/pull/1644 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [I] Implement default-value projection [iceberg-python]

Fokko closed issue #1836: Implement default-value projection URL: https://github.com/apache/iceberg-python/issues/1836 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

Re: [PR] dev integration: add pyiceberg[pyarrow] to Dockerfile to resolve missing pyarrow import error [iceberg-python]

Fokko commented on PR #2174: URL: https://github.com/apache/iceberg-python/pull/2174#issuecomment-3047601215 Thanks @kevinjqliu -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] dev integration: add pyiceberg[pyarrow] to Dockerfile to resolve missing pyarrow import error [iceberg-python]

Fokko merged PR #2174: URL: https://github.com/apache/iceberg-python/pull/2174 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] dev integration: add pyiceberg[pyarrow] to Dockerfile to resolve missing pyarrow import error [iceberg-python]

Fokko commented on code in PR #2174: URL: https://github.com/apache/iceberg-python/pull/2174#discussion_r2191636802 ## dev/Dockerfile: ## @@ -13,7 +13,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -FROM python:3.9

Re: [PR] feat: avro support applying field-ids based on name mapping [iceberg-cpp]

wgtmac commented on code in PR #127: URL: https://github.com/apache/iceberg-cpp/pull/127#discussion_r2189060171 ## src/iceberg/avro/avro_schema_util_internal.h: ## @@ -144,4 +145,39 @@ std::string ToString(const ::avro::LogicalType::Type& logical_type); /// \return True if the

Re: [I] org.apache.iceberg.gcp.gcs.OAuth2RefreshCredentialsHandler is not serializable [iceberg]

silentsokolov commented on issue #13482: URL: https://github.com/apache/iceberg/issues/13482#issuecomment-3047532935 Yep, it works. Thanks you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] org.apache.iceberg.gcp.gcs.OAuth2RefreshCredentialsHandler is not serializable [iceberg]

silentsokolov closed issue #13482: org.apache.iceberg.gcp.gcs.OAuth2RefreshCredentialsHandler is not serializable URL: https://github.com/apache/iceberg/issues/13482 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] Cannot create table named after metadata table in Spark using REST catalog [iceberg]

elphastori commented on issue #13388: URL: https://github.com/apache/iceberg/issues/13388#issuecomment-3047488688 > its for both the REST client and the underlying catalog. Okay, I wasn't sure whether the REST catalog supports empty namespaces. I'll try updating the server to not load

Re: [PR] feat: add spdlog dependency to implement logging [iceberg-cpp]

yingcai-cy commented on PR #140: URL: https://github.com/apache/iceberg-cpp/pull/140#issuecomment-3047436848 +1 to pluggable logging interface，I'll take a look into this later. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [PR] Add support for Bodo DataFrame [iceberg-python]

kevinjqliu commented on PR #2167: URL: https://github.com/apache/iceberg-python/pull/2167#issuecomment-3047387247 maybe try rebase `main` again, idk what CI is doing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Improve Docs for IRC Catalog Connection [iceberg-python]

kevinjqliu commented on code in PR #2175: URL: https://github.com/apache/iceberg-python/pull/2175#discussion_r2191469681 ## mkdocs/docs/configuration.md: ## @@ -339,25 +339,19 @@ catalog: | Key | Example | Description

Re: [PR] Encryption integration and test [iceberg]

ggershinsky commented on PR #13066: URL: https://github.com/apache/iceberg/pull/13066#issuecomment-3047343797 wip -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

Re: [PR] Spark 4.0: Row Lineage support [iceberg]

amogh-jahagirdar commented on code in PR #13310: URL: https://github.com/apache/iceberg/pull/13310#discussion_r2191443467 ## spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/ExtractRowLineageFromMetadata.java: ## @@ -0,0 +1,69 @@ +/* + * Licensed to the Apache Soft

Re: [PR] Spark 4.0: Row Lineage support [iceberg]

amogh-jahagirdar commented on code in PR #13310: URL: https://github.com/apache/iceberg/pull/13310#discussion_r2191443467 ## spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/ExtractRowLineageFromMetadata.java: ## @@ -0,0 +1,69 @@ +/* + * Licensed to the Apache Soft

Re: [PR] Spark 4.0: Row Lineage support [iceberg]

amogh-jahagirdar commented on code in PR #13310: URL: https://github.com/apache/iceberg/pull/13310#discussion_r2191440182 ## spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/SparkMetadataColumn.java: ## @@ -20,17 +20,40 @@ import org.apache.spark.sql.connector.c

Re: [PR] Spark 4.0: Row Lineage support [iceberg]

stevenzwu commented on code in PR #13310: URL: https://github.com/apache/iceberg/pull/13310#discussion_r2191288421 ## spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/ExtractRowLineageFromMetadata.java: ## @@ -0,0 +1,69 @@ +/* + * Licensed to the Apache Software Fo

[PR] feat: add manifest list reader [iceberg-cpp]

dongxiao1198 opened a new pull request, #143: URL: https://github.com/apache/iceberg-cpp/pull/143 - Add manifest list reader - Integrate with avro reader - Add simple ut -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [PR] Build: Bump nessie from 0.104.1 to 0.104.2 [iceberg]

manuzhang commented on PR #13314: URL: https://github.com/apache/iceberg/pull/13314#issuecomment-3047205080 @snazy Could you please take a look from nessie side? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] [Spec] filed path update for Variant array metadata [iceberg]

RussellSpitzer commented on issue #13462: URL: https://github.com/apache/iceberg/issues/13462#issuecomment-3047188898 For 2,3 I was suggesting keeping the key but having the value be an array type itself. In 2, a one element array with the aggregate max In 3, a multi element ar

Re: [PR] Add support for Bodo DataFrame [iceberg-python]

ehsantn commented on PR #2167: URL: https://github.com/apache/iceberg-python/pull/2167#issuecomment-3047167766 > @ehsantn i merged a few library upgrades. could you rebase this PR? Done. I assume the CI failure is not related to this PR? The test doesn't seem relevant. -- This is a

Re: [PR] Build: Bump mkdocs-material from 9.6.14 to 9.6.15 [iceberg-python]

kevinjqliu merged PR #2187: URL: https://github.com/apache/iceberg-python/pull/2187 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

Re: [PR] Add support for Bodo DataFrame [iceberg-python]

kevinjqliu commented on PR #2167: URL: https://github.com/apache/iceberg-python/pull/2167#issuecomment-3047159027 @ehsantn i merged a few library upgrades. could you rebase this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

Re: [PR] Build: Bump huggingface-hub from 0.33.1 to 0.33.2 [iceberg-python]

kevinjqliu merged PR #2184: URL: https://github.com/apache/iceberg-python/pull/2184 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

Re: [PR] Use short string in Variant when possible [iceberg]

manirajv06 commented on code in PR #13284: URL: https://github.com/apache/iceberg/pull/13284#discussion_r2191344004 ## api/src/test/java/org/apache/iceberg/variants/TestSerializedObject.java: ## @@ -182,70 +182,59 @@ public void testMixedValueTypes() { assertThat(actualInne

Re: [PR] update daft links [iceberg-python]

kevinjqliu commented on PR #2169: URL: https://github.com/apache/iceberg-python/pull/2169#issuecomment-3047138488 @ccmao1130 could you run `make install` and then `poetry lock` locally? That should resolve the CI issue -- This is an automated message from the Apache Git Service. To respon

Re: [PR] Build: Bump mkdocs-material from 9.6.14 to 9.6.15 [iceberg-python]

kevinjqliu commented on PR #2187: URL: https://github.com/apache/iceberg-python/pull/2187#issuecomment-3047136644 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] Build: Bump pandas from 2.3.0 to 2.3.1 [iceberg-python]

kevinjqliu merged PR #2181: URL: https://github.com/apache/iceberg-python/pull/2181 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

Re: [PR] Build: Bump pypa/cibuildwheel from 3.0.0 to 3.0.1 [iceberg-python]

kevinjqliu merged PR #2186: URL: https://github.com/apache/iceberg-python/pull/2186 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

Re: [PR] Build: Bump moto from 5.1.6 to 5.1.8 [iceberg-python]

kevinjqliu merged PR #2182: URL: https://github.com/apache/iceberg-python/pull/2182 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

Re: [PR] Build: Bump coverage from 7.9.1 to 7.9.2 [iceberg-python]

kevinjqliu merged PR #2185: URL: https://github.com/apache/iceberg-python/pull/2185 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

Re: [PR] Build: Bump typing-extensions from 4.14.0 to 4.14.1 [iceberg-python]

kevinjqliu merged PR #2183: URL: https://github.com/apache/iceberg-python/pull/2183 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

[PR] Build: Bump mkdocs-material from 9.6.14 to 9.6.15 [iceberg-python]

dependabot[bot] opened a new pull request, #2187: URL: https://github.com/apache/iceberg-python/pull/2187 Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.6.14 to 9.6.15. Release notes Sourced from https://github.com/squidfunk/mkdocs-material/releases";>

[PR] Build: Bump pypa/cibuildwheel from 3.0.0 to 3.0.1 [iceberg-python]

dependabot[bot] opened a new pull request, #2186: URL: https://github.com/apache/iceberg-python/pull/2186 Bumps [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel) from 3.0.0 to 3.0.1. Release notes Sourced from https://github.com/pypa/cibuildwheel/releases";>pypa/cibuildwhee

[PR] Build: Bump coverage from 7.9.1 to 7.9.2 [iceberg-python]

dependabot[bot] opened a new pull request, #2185: URL: https://github.com/apache/iceberg-python/pull/2185 Bumps [coverage](https://github.com/nedbat/coveragepy) from 7.9.1 to 7.9.2. Changelog Sourced from https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst";>coverage's cha

[PR] Build: Bump huggingface-hub from 0.33.1 to 0.33.2 [iceberg-python]

dependabot[bot] opened a new pull request, #2184: URL: https://github.com/apache/iceberg-python/pull/2184 Bumps [huggingface-hub](https://github.com/huggingface/huggingface_hub) from 0.33.1 to 0.33.2. Release notes Sourced from https://github.com/huggingface/huggingface_hub/release

[PR] Build: Bump typing-extensions from 4.14.0 to 4.14.1 [iceberg-python]

dependabot[bot] opened a new pull request, #2183: URL: https://github.com/apache/iceberg-python/pull/2183 Bumps [typing-extensions](https://github.com/python/typing_extensions) from 4.14.0 to 4.14.1. Release notes Sourced from https://github.com/python/typing_extensions/releases";>

Re: [I] [bug] Schema validation should reject field names that are invalid Avro identifiers [iceberg-python]

kris-gaudel commented on issue #2123: URL: https://github.com/apache/iceberg-python/issues/2123#issuecomment-3047002807 @kevinjqliu Would be great to hear your thoughts on this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[PR] Core: Support timestamp nanos as default values [iceberg]

ebyhr opened a new pull request, #13487: URL: https://github.com/apache/iceberg/pull/13487 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-ma

[PR] Build: Bump moto from 5.1.6 to 5.1.8 [iceberg-python]

dependabot[bot] opened a new pull request, #2182: URL: https://github.com/apache/iceberg-python/pull/2182 Bumps [moto](https://github.com/getmoto/moto) from 5.1.6 to 5.1.8. Changelog Sourced from https://github.com/getmoto/moto/blob/master/CHANGELOG.md";>moto's changelog. Mo

[PR] Build: Bump pandas from 2.3.0 to 2.3.1 [iceberg-python]

dependabot[bot] opened a new pull request, #2181: URL: https://github.com/apache/iceberg-python/pull/2181 Bumps [pandas](https://github.com/pandas-dev/pandas) from 2.3.0 to 2.3.1. Release notes Sourced from https://github.com/pandas-dev/pandas/releases";>pandas's releases. P

Re: [I] Kafka Connect: How to connect to Hive Metastore with SSL [iceberg]

github-actions[bot] commented on issue #11925: URL: https://github.com/apache/iceberg/issues/11925#issuecomment-3046935717 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] How to force iceberg table to create an empty snapshot [iceberg-python]

geruh commented on issue #2177: URL: https://github.com/apache/iceberg-python/issues/2177#issuecomment-3046896340 Hey @fusion, usually when we insert/modify table data in iceberg snapshots get produced, but it looks like this can be forced in both libraries (java/python) by

Re: [I] [Spec] filed path update for Variant array metadata [iceberg]

rdblue commented on issue #13462: URL: https://github.com/apache/iceberg/issues/13462#issuecomment-3046805127 I think I understand the problem: because the bounds for a scalar and an array are represented using the same key (`$["tags"]`) there is no way to distinguish in this representation

Re: [PR] refactor: consolidate snapshot expiration into MaintenanceTable [iceberg-python]

ForeverAngry commented on code in PR #2143: URL: https://github.com/apache/iceberg-python/pull/2143#discussion_r2191172858 ## pyiceberg/table/inspect.py: ## @@ -668,10 +666,20 @@ def data_files(self, snapshot_id: Optional[int] = None) -> "pa.Table": def delete_files(self,

Re: [PR] refactor: consolidate snapshot expiration into MaintenanceTable [iceberg-python]

ForeverAngry commented on code in PR #2143: URL: https://github.com/apache/iceberg-python/pull/2143#discussion_r2191158861 ## pyiceberg/table/maintenance.py: ## @@ -0,0 +1,372 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreeme

[PR] writing v3 table metadata [iceberg-python]

rambleraptor opened a new pull request, #2180: URL: https://github.com/apache/iceberg-python/pull/2180 Part of #1818 This might be early. This enables writing v3 metadata # Rationale for this change We'll need this at some point for v3 support. # Are

Re: [PR] refactor: consolidate snapshot expiration into MaintenanceTable [iceberg-python]

ForeverAngry commented on code in PR #2143: URL: https://github.com/apache/iceberg-python/pull/2143#discussion_r2191155682 ## pyiceberg/table/maintenance.py: ## @@ -0,0 +1,372 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreeme

Re: [PR] refactor: consolidate snapshot expiration into MaintenanceTable [iceberg-python]

ForeverAngry commented on code in PR #2143: URL: https://github.com/apache/iceberg-python/pull/2143#discussion_r2191154305 ## pyiceberg/table/inspect.py: ## @@ -650,14 +651,11 @@ def _files(self, snapshot_id: Optional[int] = None, data_file_filter: Optional[S snapsho

Re: [PR] Metrics reporting [iceberg-rust]

DerGut commented on code in PR #1496: URL: https://github.com/apache/iceberg-rust/pull/1496#discussion_r2191153369 ## crates/iceberg/src/metrics.rs: ## @@ -0,0 +1,154 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. S

Re: [PR] Metrics reporting [iceberg-rust]

DerGut commented on code in PR #1496: URL: https://github.com/apache/iceberg-rust/pull/1496#discussion_r2191124944 ## crates/iceberg/src/scan/metrics.rs: ## Review Comment: Because the planning stage is parallelized and multiple threads are responsible for fetching manifes

Re: [PR] Spark 4.0: Row Lineage support [iceberg]

amogh-jahagirdar commented on code in PR #13310: URL: https://github.com/apache/iceberg/pull/13310#discussion_r2191110242 ## spark/v4.0/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRowLevelOperationsWithLineage.java: ## @@ -0,0 +1,491 @@ +/* + * License

Re: [PR] Metrics reporting [iceberg-rust]

DerGut commented on code in PR #1496: URL: https://github.com/apache/iceberg-rust/pull/1496#discussion_r2191109073 ## crates/iceberg/src/scan/mod.rs: ## @@ -336,94 +374,211 @@ impl TableScan { return Ok(Box::pin(futures::stream::empty())); }; -let

Re: [PR] Metrics reporting [iceberg-rust]

DerGut commented on code in PR #1496: URL: https://github.com/apache/iceberg-rust/pull/1496#discussion_r2191102714 ## crates/iceberg/src/scan/mod.rs: ## Review Comment: Because I had to thread the metrics building throughout the planning stage, I heavily refactored this fi

Re: [PR] Spark 4.0: Row Lineage support [iceberg]

amogh-jahagirdar commented on code in PR #13310: URL: https://github.com/apache/iceberg/pull/13310#discussion_r2191101009 ## spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/SparkWriteBuilder.java: ## @@ -116,16 +119,36 @@ public WriteBuilder overwrite(Filter[] fil

Re: [PR] Metrics reporting [iceberg-rust]

DerGut commented on code in PR #1496: URL: https://github.com/apache/iceberg-rust/pull/1496#discussion_r2191093209 ## crates/iceberg/src/scan/context.rs: ## @@ -186,16 +187,25 @@ impl PlanContext { tx_data: Sender, delete_file_idx: DeleteFileIndex, del

Re: [PR] spec: add clarification about the Geometry type calculation [iceberg]

rdblue commented on PR #13227: URL: https://github.com/apache/iceberg/pull/13227#issuecomment-3046661554 > Just a note that I don't think this clarification is needed...a Cartesian bounding box like the one described here is perfectly legal in for the geometry type in GeoArrow's box type, G

Re: [PR] Spec: Add DV information in overview [iceberg]

stevenzwu commented on code in PR #13189: URL: https://github.com/apache/iceberg/pull/13189#discussion_r2191089719 ## format/spec.md: ## @@ -101,10 +101,10 @@ Inheriting the sequence number from manifest metadata allows writing a new manif Row-level deletes are stored in del

Re: [PR] Spec: Add DV information in overview [iceberg]

stevenzwu commented on code in PR #13189: URL: https://github.com/apache/iceberg/pull/13189#discussion_r2191089719 ## format/spec.md: ## @@ -101,10 +101,10 @@ Inheriting the sequence number from manifest metadata allows writing a new manif Row-level deletes are stored in del

Re: [PR] Metrics reporting [iceberg-rust]

DerGut commented on code in PR #1496: URL: https://github.com/apache/iceberg-rust/pull/1496#discussion_r2191088574 ## crates/iceberg/src/metrics.rs: ## @@ -0,0 +1,154 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. S

Re: [PR] Spec: Add DV information in overview [iceberg]

stevenzwu commented on code in PR #13189: URL: https://github.com/apache/iceberg/pull/13189#discussion_r2191086264 ## format/spec.md: ## @@ -101,10 +101,10 @@ Inheriting the sequence number from manifest metadata allows writing a new manif Row-level deletes are stored in del

Re: [PR] Spark 4.0: Migrate Iceberg Stored Procedures to Spark built-in implementations [iceberg]

szehon-ho commented on PR #13106: URL: https://github.com/apache/iceberg/pull/13106#issuecomment-3046643863 Merged, thanks @pan3793 for the great work, and everyone for jumping on the reviews! -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Spark 4.0: Migrate Iceberg Stored Procedures to Spark built-in implementations [iceberg]

szehon-ho merged PR #13106: URL: https://github.com/apache/iceberg/pull/13106 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Spark: Throw unsupported for ADD COLUMN with default value [iceberg]

amogh-jahagirdar merged PR #13464: URL: https://github.com/apache/iceberg/pull/13464 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] Metrics reporting [iceberg-rust]

DerGut commented on code in PR #1496: URL: https://github.com/apache/iceberg-rust/pull/1496#discussion_r2191074184 ## crates/iceberg/Cargo.toml: ## @@ -90,6 +90,7 @@ typed-builder = { workspace = true } url = { workspace = true } uuid = { workspace = true } zstd = { workspace

Re: [PR] Spark: Throw unsupported for ADD COLUMN with default value [iceberg]

amogh-jahagirdar commented on PR #13464: URL: https://github.com/apache/iceberg/pull/13464#issuecomment-3046632045 Thanks for the reviews @singhpk234 @nastra @RussellSpitzer ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[PR] Metrics reporting [iceberg-rust]

DerGut opened a new pull request, #1496: URL: https://github.com/apache/iceberg-rust/pull/1496 ## Which issue does this PR close? - Closes #1466. ## What changes are included in this PR? As mentioned in the issue description, this PR adds an implementation for th

Re: [PR] Metrics reporting [iceberg-rust]

DerGut commented on code in PR #1496: URL: https://github.com/apache/iceberg-rust/pull/1496#discussion_r2191068576 ## crates/iceberg/src/metrics.rs: ## @@ -0,0 +1,154 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. S

Re: [PR] Metrics reporting [iceberg-rust]

DerGut commented on code in PR #1496: URL: https://github.com/apache/iceberg-rust/pull/1496#discussion_r2191064385 ## crates/iceberg/src/metrics.rs: ## @@ -0,0 +1,154 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. S

Re: [PR] refactor: consolidate snapshot expiration into MaintenanceTable [iceberg-python]

jayceslesar commented on code in PR #2143: URL: https://github.com/apache/iceberg-python/pull/2143#discussion_r2191041938 ## pyiceberg/table/inspect.py: ## @@ -668,10 +666,20 @@ def data_files(self, snapshot_id: Optional[int] = None) -> "pa.Table": def delete_files(self, s

Re: [PR] Metrics reporting [iceberg-rust]

DerGut commented on code in PR #1496: URL: https://github.com/apache/iceberg-rust/pull/1496#discussion_r2191062453 ## crates/iceberg/src/delete_file_index.rs: ## Review Comment: At this point, the `DeleteFileIndex` is only used by the `scan` module. I don't think it will b

Re: [PR] Core: Support DV for partition stats [iceberg]

stevenzwu commented on PR #13425: URL: https://github.com/apache/iceberg/pull/13425#issuecomment-3046596628 thanks @ajantha-bhat for the contribution and @pvary @nastra for the reviews -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] Core: Support DV for partition stats [iceberg]

stevenzwu commented on PR #13425: URL: https://github.com/apache/iceberg/pull/13425#issuecomment-3046594626 merge this now. if there are more review comments, we can follow up separately -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [I] Support DV for partition stats [iceberg]

stevenzwu closed issue #13180: Support DV for partition stats URL: https://github.com/apache/iceberg/issues/13180 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

Re: [PR] Core: Support DV for partition stats [iceberg]

stevenzwu merged PR #13425: URL: https://github.com/apache/iceberg/pull/13425 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Core: Support DV for partition stats [iceberg]

stevenzwu commented on code in PR #13425: URL: https://github.com/apache/iceberg/pull/13425#discussion_r2191047299 ## core/src/main/java/org/apache/iceberg/PartitionStatsHandler.java: ## @@ -89,17 +91,57 @@ private PartitionStatsHandler() {} NestedField.optional(11, "last

Re: [PR] docs: Introduce website with docs [iceberg-go]

zeroshade merged PR #478: URL: https://github.com/apache/iceberg-go/pull/478 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Remove cached properties before updating table scans [iceberg-python]

jayceslesar commented on PR #2178: URL: https://github.com/apache/iceberg-python/pull/2178#issuecomment-3046566095 you could add `**kwargs` in the `TableScan` constructor too right as a way to "fix"? I dont really think there is a good way to deal with this hahahaha -- This is an automate

Re: [PR] docs: Introduce website with docs [iceberg-go]

laskoviymishka commented on code in PR #478: URL: https://github.com/apache/iceberg-go/pull/478#discussion_r2191022375 ## .github/workflows/go-release-docs.yml: ## @@ -0,0 +1,62 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license ag

Re: [PR] docs: Introduce website with docs [iceberg-go]

zeroshade commented on code in PR #478: URL: https://github.com/apache/iceberg-go/pull/478#discussion_r2191021499 ## .github/workflows/go-release-docs.yml: ## @@ -0,0 +1,62 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreeme

Re: [PR] Use short string in Variant when possible [iceberg]

aihuaxu commented on code in PR #13284: URL: https://github.com/apache/iceberg/pull/13284#discussion_r2190999519 ## api/src/test/java/org/apache/iceberg/variants/TestSerializedObject.java: ## @@ -182,70 +182,59 @@ public void testMixedValueTypes() { assertThat(actualInner.g

Re: [PR] feat (cli): Add Support CreateTableOpt with table properties, partition spec & sort order [iceberg-go]

zeroshade commented on code in PR #475: URL: https://github.com/apache/iceberg-go/pull/475#discussion_r2191006370 ## cmd/iceberg/main.go: ## @@ -83,7 +83,13 @@ Options: --description TEXT specify a description for the namespace --location-uri TEXT specify a loca

Re: [PR] feat (cli): Add Support CreateTableOpt with table properties, partition spec & sort order [iceberg-go]

dttung2905 commented on code in PR #475: URL: https://github.com/apache/iceberg-go/pull/475#discussion_r2190997317 ## cmd/iceberg/main.go: ## @@ -83,7 +83,13 @@ Options: --description TEXT specify a description for the namespace --location-uri TEXT specify a loc

Re: [I] [Spec] filed path update for Variant array metadata [iceberg]

RussellSpitzer commented on issue #13462: URL: https://github.com/apache/iceberg/issues/13462#issuecomment-3046477449 I think I understand the problem now (and I see a typo in the Spec ([missing *for](https://github.com/apache/iceberg/pull/12658/files#diff-36347a47c3bf67ea2ef6309ea9620181403

[PR] Fix: Packed Parquet Int96 timestamps read incorrectly #13485 [iceberg]

xajohnson opened a new pull request, #13486: URL: https://github.com/apache/iceberg/pull/13486 fix: use byte offset in setLong for TimestampInt96Reader.nextDictEncodedVal to prevent buffer corruption https://github.com/apache/iceberg/issues/13485 -- This is an automated message fro

Re: [PR] Spark: Throw unsupported for ADD COLUMN with default value [iceberg]

amogh-jahagirdar commented on code in PR #13464: URL: https://github.com/apache/iceberg/pull/13464#discussion_r2190942553 ## spark/v3.4/spark/src/main/java/org/apache/iceberg/spark/Spark3Util.java: ## @@ -234,6 +234,13 @@ private static void apply(UpdateSchema pendingUpdate, Ta

Re: [PR] Add support for Bodo DataFrame [iceberg-python]

ehsantn commented on PR #2167: URL: https://github.com/apache/iceberg-python/pull/2167#issuecomment-3046375134 Ok, updated Bodo to support Python 3.9 so this should work now. Tried `poetry install --all-extras` in an Ubuntu environment and it works. -- This is an automated message from th

Re: [PR] refine: refine the interface of SnapshotProducer [iceberg-rust]

CTTY commented on PR #1490: URL: https://github.com/apache/iceberg-rust/pull/1490#issuecomment-3046327641 > so I think there is no case that SnapshotProducer has different table with TransactionAction::commit? This is correct. In this case, can we refactor `SnapshotProducer::validate

Re: [PR] Spark 4.0: Migrate Iceberg Stored Procedures to Spark built-in implementations [iceberg]

szehon-ho commented on PR #13106: URL: https://github.com/apache/iceberg/pull/13106#issuecomment-3046284271 I guess Spark just needs all Expression to mark whether they are deterministic , for things like knowing whether you can use them in various places like filter aggregate, merge condit

Re: [PR] Read ManifestList V1 with V2 projection. [iceberg-rust]

rambleraptor commented on code in PR #1482: URL: https://github.com/apache/iceberg-rust/pull/1482#discussion_r2190851276 ## crates/iceberg/src/avro/schema.rs: ## @@ -43,6 +43,41 @@ const MAP_LOGICAL_TYPE: &str = "map"; // This const may better to maintain in avro-rs. const LOG

Re: [PR] Read ManifestList V1 with V2 projection. [iceberg-rust]

rambleraptor commented on code in PR #1482: URL: https://github.com/apache/iceberg-rust/pull/1482#discussion_r2190851276 ## crates/iceberg/src/avro/schema.rs: ## @@ -43,6 +43,41 @@ const MAP_LOGICAL_TYPE: &str = "map"; // This const may better to maintain in avro-rs. const LOG

Re: [PR] update daft links [iceberg-python]

ccmao1130 commented on code in PR #2169: URL: https://github.com/apache/iceberg-python/pull/2169#discussion_r2190844881 ## pyproject.toml: ## @@ -78,7 +78,7 @@ gcsfs = { version = ">=2023.1.0", optional = true } huggingface-hub = { version = ">=0.24.0", optional = true } psyco

Re: [PR] build(deps): bump the gomod_updates group with 3 updates [iceberg-go]

zeroshade merged PR #476: URL: https://github.com/apache/iceberg-go/pull/476 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] feat: adding schema update methods [iceberg-go]

zeroshade commented on code in PR #431: URL: https://github.com/apache/iceberg-go/pull/431#discussion_r2190824187 ## table/update_schema.go: ## @@ -0,0 +1,606 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the N

Re: [PR] Core: Registering tables to nonexistent target namespace leads to metadata deletion in HiveCatalog [iceberg]

hsiang-c commented on code in PR #13434: URL: https://github.com/apache/iceberg/pull/13434#discussion_r2190804849 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcTableOperations.java: ## @@ -173,7 +173,7 @@ private void createTable(String newMetadataLocation) throws SQLExcept

Re: [PR] feat (cli): Add Support CreateTableOpt with table properties, partition spec & sort order [iceberg-go]

zeroshade commented on code in PR #475: URL: https://github.com/apache/iceberg-go/pull/475#discussion_r2190804963 ## cmd/iceberg/utils.go: ## @@ -0,0 +1,125 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOT

Re: [I] Change iceberg folder structure to facilitate easier migrations from snapshot parquet tables [iceberg]

abhinigam commented on issue #13477: URL: https://github.com/apache/iceberg/issues/13477#issuecomment-3046195367 > There's a [proposal](https://github.com/apache/iceberg/issues/13141) to support relative paths in iceberg. Maybe that helps? Thanks @aheev since both the issues are relat

Re: [PR] Core: Keep track of data files to be removed for orphaned DV detection [iceberg]

stevenzwu commented on code in PR #13222: URL: https://github.com/apache/iceberg/pull/13222#discussion_r2190783604 ## core/src/main/java/org/apache/iceberg/ManifestFilterManager.java: ## @@ -224,7 +235,9 @@ List filterManifests(Schema tableSchema, List manife private boolean

Re: [PR] Read ManifestList V1 with V2 projection. [iceberg-rust]

rambleraptor commented on code in PR #1482: URL: https://github.com/apache/iceberg-rust/pull/1482#discussion_r2190804837 ## crates/iceberg/src/spec/manifest/_serde.rs: ## @@ -398,4 +398,73 @@ mod tests { assert_eq!(data_files, actual_data_file); } + +#[tokio:

Re: [PR] feat (cli): Add Support CreateTableOpt with table properties, partition spec & sort order [iceberg-go]

zeroshade commented on code in PR #475: URL: https://github.com/apache/iceberg-go/pull/475#discussion_r2190800727 ## cmd/iceberg/main.go: ## @@ -279,6 +288,31 @@ func main() { opts = append(opts, catalog.WithLocation(cfg.LocationURI))

  1   2   3   >