Re: [PR] Core, API, Spec: Metadata Row Lineage [iceberg]

RussellSpitzer commented on code in PR #11948: URL: https://github.com/apache/iceberg/pull/11948#discussion_r1920826812 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1230,6 +1265,18 @@ public Builder addSnapshot(Snapshot snapshot) { snapshotsById.put

Re: [PR] Core, API, Spec: Metadata Row Lineage [iceberg]

RussellSpitzer commented on code in PR #11948: URL: https://github.com/apache/iceberg/pull/11948#discussion_r1920830866 ## core/src/main/java/org/apache/iceberg/TableMetadataParser.java: ## @@ -110,6 +110,8 @@ private TableMetadataParser() {} static final String METADATA_LOG

Re: [I] Is it possible to create a table without instantiating an Iceberg catalog? [iceberg-python]

aaron-siegel commented on issue #1535: URL: https://github.com/apache/iceberg-python/issues/1535#issuecomment-2599230589 Yes this worked for me, thank you! Aaron -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [I] Is it possible to create a table without instantiating an Iceberg catalog? [iceberg-python]

Fokko closed issue #1535: Is it possible to create a table without instantiating an Iceberg catalog? URL: https://github.com/apache/iceberg-python/issues/1535 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [I] Is it possible to create a table without instantiating an Iceberg catalog? [iceberg-python]

Fokko commented on issue #1535: URL: https://github.com/apache/iceberg-python/issues/1535#issuecomment-2599235562 @aaron-siegel Any time, let us know if we can improve the documentation. I'll close this issue for now, thanks for asking. -- This is an automated message from the Apache Git

Re: [PR] WIP: Deletion vectors [iceberg-python]

Fokko commented on PR #1516: URL: https://github.com/apache/iceberg-python/pull/1516#issuecomment-2599098751 I want to try using the [Python `roaringbitmap`](https://pypi.org/project/roaringbitmap/) API first, otherwise, we can go the Iceberg-Rust route. -- This is an automated message f

Re: [I] Is it possible to create a table without instantiating an Iceberg catalog? [iceberg-python]

aaron-siegel commented on issue #1535: URL: https://github.com/apache/iceberg-python/issues/1535#issuecomment-2599098815 @Fokko Thanks! How would we specify configuration to pyiceberg? The docs state that pyiceberg expects to find a `.yaml` file in `$HOME`; is there anyway to override this

Re: [PR] Core, API, Spec: Metadata Row Lineage [iceberg]

RussellSpitzer commented on code in PR #11948: URL: https://github.com/apache/iceberg/pull/11948#discussion_r1920812731 ## core/src/main/java/org/apache/iceberg/BaseSnapshot.java: ## @@ -61,7 +63,9 @@ class BaseSnapshot implements Snapshot { String operation, Map s

[PR] Auth Manager API part 5: SigV4 Auth Manager [iceberg]

adutra opened a new pull request, #11995: URL: https://github.com/apache/iceberg/pull/11995 5th PR for the Auth Manager API. Previous ones: * #11992 * #11844 * #11809 * #11769 This PR introduces the SigV4 auth manager. It can be reviewed independently of [part4](https

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2599392946 Thanks @bitsondatadev for all this great feedback. I'll get working on your suggestions and push an update next week and will address all your concerns. -- This is an automa

Re: [I] com/fasterxml/jackson/core/exc/StreamConstraintsException [iceberg]

github-actions[bot] commented on issue #9355: URL: https://github.com/apache/iceberg/issues/9355#issuecomment-2599401807 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Hive: partitioning is not working [iceberg]

github-actions[bot] closed issue #9329: Hive: partitioning is not working URL: https://github.com/apache/iceberg/issues/9329 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [I] Is iceberg writing metadata atomic [iceberg]

github-actions[bot] commented on issue #9328: URL: https://github.com/apache/iceberg/issues/9328#issuecomment-2599401681 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] How to improve write speed for data in the same partition? [iceberg]

github-actions[bot] closed issue #9330: How to improve write speed for data in the same partition? URL: https://github.com/apache/iceberg/issues/9330 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] com/fasterxml/jackson/core/exc/StreamConstraintsException [iceberg]

github-actions[bot] closed issue #9355: com/fasterxml/jackson/core/exc/StreamConstraintsException URL: https://github.com/apache/iceberg/issues/9355 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] Regarding the issue of decimal scale [iceberg]

github-actions[bot] commented on issue #9350: URL: https://github.com/apache/iceberg/issues/9350#issuecomment-2599401767 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Iceberg Disaster Recovery [iceberg]

github-actions[bot] commented on issue #10723: URL: https://github.com/apache/iceberg/issues/10723#issuecomment-2599401987 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Add scan planning api request and response models, parsers [iceberg]

github-actions[bot] commented on PR #11369: URL: https://github.com/apache/iceberg/pull/11369#issuecomment-2599402066 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [I] Hive: partitioning is not working [iceberg]

github-actions[bot] commented on issue #9329: URL: https://github.com/apache/iceberg/issues/9329#issuecomment-2599401703 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] How to improve write speed for data in the same partition? [iceberg]

github-actions[bot] commented on issue #9330: URL: https://github.com/apache/iceberg/issues/9330#issuecomment-2599401729 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [PR] Core: Add reference snapshot ID/timestamps to AllEntriesTable and AllManifestsTable [iceberg]

github-actions[bot] commented on PR #9335: URL: https://github.com/apache/iceberg/pull/9335#issuecomment-2599401738 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] GCP: Implement SupportsRecoveryOperations for GCSFileIO [iceberg]

github-actions[bot] commented on PR #11565: URL: https://github.com/apache/iceberg/pull/11565#issuecomment-2599402092 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] Spark3.5 deprecate a few SparkCatalog APIs [iceberg]

github-actions[bot] commented on PR #11807: URL: https://github.com/apache/iceberg/pull/11807#issuecomment-2599402177 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] Regarding the issue of decimal scale [iceberg]

github-actions[bot] closed issue #9350: Regarding the issue of decimal scale URL: https://github.com/apache/iceberg/issues/9350 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Add scan planning api request and response models, parsers [iceberg]

github-actions[bot] closed pull request #11369: Add scan planning api request and response models, parsers URL: https://github.com/apache/iceberg/pull/11369 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] dropDeleteFilesOlderthan should be partition level instead of table level [iceberg]

github-actions[bot] commented on issue #9383: URL: https://github.com/apache/iceberg/issues/9383#issuecomment-2599401841 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] dropDeleteFilesOlderthan should be partition level instead of table level [iceberg]

github-actions[bot] closed issue #9383: dropDeleteFilesOlderthan should be partition level instead of table level URL: https://github.com/apache/iceberg/issues/9383 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] Iceberg the condition function of org.apache.iceberg.expressions, not able to use time-related values as value [iceberg]

github-actions[bot] closed issue #9431: Iceberg the condition function of org.apache.iceberg.expressions, not able to use time-related values as value URL: https://github.com/apache/iceberg/issues/9431 -- This is an automated message from the Apache Git Service. To respond to the message, ple

Re: [I] Snowflake Iceberg Partitioned data read issue [iceberg]

github-actions[bot] commented on issue #9404: URL: https://github.com/apache/iceberg/issues/9404#issuecomment-2599401856 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Is iceberg writing metadata atomic [iceberg]

github-actions[bot] closed issue #9328: Is iceberg writing metadata atomic URL: https://github.com/apache/iceberg/issues/9328 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [I] Data: Allow classes of different packages to implement DeleteFilter [iceberg]

github-actions[bot] commented on issue #9351: URL: https://github.com/apache/iceberg/issues/9351#issuecomment-2599401784 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] When using the Flink upsert mode, the speed of reading data from the iceberg table is very slow. [iceberg]

github-actions[bot] closed issue #9363: When using the Flink upsert mode, the speed of reading data from the iceberg table is very slow. URL: https://github.com/apache/iceberg/issues/9363 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [I] When using the Flink upsert mode, the speed of reading data from the iceberg table is very slow. [iceberg]

github-actions[bot] commented on issue #9363: URL: https://github.com/apache/iceberg/issues/9363#issuecomment-2599401815 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [PR] GCP: Implement SupportsRecoveryOperations for GCSFileIO [iceberg]

github-actions[bot] closed pull request #11565: GCP: Implement SupportsRecoveryOperations for GCSFileIO URL: https://github.com/apache/iceberg/pull/11565 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [I] manifest list missing error after commit failed exception [iceberg]

github-actions[bot] closed issue #9406: manifest list missing error after commit failed exception URL: https://github.com/apache/iceberg/issues/9406 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [I] Data: Allow classes of different packages to implement DeleteFilter [iceberg]

github-actions[bot] closed issue #9351: Data: Allow classes of different packages to implement DeleteFilter URL: https://github.com/apache/iceberg/issues/9351 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [I] manifest list missing error after commit failed exception [iceberg]

github-actions[bot] commented on issue #9406: URL: https://github.com/apache/iceberg/issues/9406#issuecomment-2599401894 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Is it possible to create a table without instantiating an Iceberg catalog? [iceberg-python]

Fokko commented on issue #1535: URL: https://github.com/apache/iceberg-python/issues/1535#issuecomment-2599094360 @aaron-siegel The `SqlCatalog` with `sqlite` should do the trick then, you can find examples here: https://py.iceberg.apache.org/#connecting-to-a-catalog -- This is an automa

Re: [PR] Spec: Support geo type [iceberg]

szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1920848298 ## format/spec.md: ## @@ -205,15 +205,40 @@ Supported primitive types are defined in the table below. Primitive types added | | **`uuid`**

Re: [PR] Core, API, Spec: Metadata Row Lineage [iceberg]

RussellSpitzer commented on code in PR #11948: URL: https://github.com/apache/iceberg/pull/11948#discussion_r1920848418 ## core/src/test/java/org/apache/iceberg/TestRowLineageMetadata.java: ## @@ -0,0 +1,339 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

Re: [PR] Core, API, Spec: Metadata Row Lineage [iceberg]

RussellSpitzer commented on code in PR #11948: URL: https://github.com/apache/iceberg/pull/11948#discussion_r1920838303 ## core/src/test/java/org/apache/iceberg/TestMetadataUpdateParser.java: ## @@ -360,6 +360,8 @@ public void testAddSnapshotToJson() throws IOException { lo

Re: [PR] Spec: Support geo type [iceberg]

szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1920848298 ## format/spec.md: ## @@ -205,15 +205,40 @@ Supported primitive types are defined in the table below. Primitive types added | | **`uuid`**

Re: [PR] Spec: Support geo type [iceberg]

szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1920848298 ## format/spec.md: ## @@ -205,15 +205,40 @@ Supported primitive types are defined in the table below. Primitive types added | | **`uuid`**

[PR] Build: Bump boto3 from 1.35.93 to 1.36.1 [iceberg-python]

dependabot[bot] opened a new pull request, #1536: URL: https://github.com/apache/iceberg-python/pull/1536 Bumps [boto3](https://github.com/boto/boto3) from 1.35.93 to 1.36.1. Commits https://github.com/boto/boto3/commit/9613882980328c8316ee418ec680f8d08ecae692";>9613882 Merge b

Re: [PR] refine: refine interface of ManifestWriter [iceberg-rust]

liurenjie1024 commented on PR #738: URL: https://github.com/apache/iceberg-rust/pull/738#issuecomment-2599575267 Let's wait for a moment to see if @Xuanwo has other comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] feat: support scan nested type(struct, map, list) [iceberg-rust]

liurenjie1024 commented on code in PR #882: URL: https://github.com/apache/iceberg-rust/pull/882#discussion_r1920988957 ## crates/iceberg/src/arrow/schema.rs: ## @@ -43,7 +43,9 @@ use crate::spec::{ use crate::{Error, ErrorKind}; /// When iceberg map type convert to Arrow ma

Re: [PR] Spec: Support geo type [iceberg]

dmitrykoval commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1921000694 ## format/spec.md: ## @@ -205,15 +205,40 @@ Supported primitive types are defined in the table below. Primitive types added | | **`uuid`**

Re: [PR] feat: support scan nested type(struct, map, list) [iceberg-rust]

liurenjie1024 commented on code in PR #882: URL: https://github.com/apache/iceberg-rust/pull/882#discussion_r1921001935 ## crates/integration_tests/tests/scan_all_type.rs: ## Review Comment: A good example could be found here: https://github.com/apache/iceberg/blob/1f6ff6c

Re: [PR] Use SupportsPrefixOperations for Remove OrphanFile Procedure [iceberg]

danielcweeks commented on code in PR #11906: URL: https://github.com/apache/iceberg/pull/11906#discussion_r1920903258 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/DeleteOrphanFilesSparkAction.java: ## @@ -292,19 +294,49 @@ private Dataset validFileIdentDS()

Re: [PR] Use SupportsPrefixOperations for Remove OrphanFile Procedure [iceberg]

danielcweeks commented on code in PR #11906: URL: https://github.com/apache/iceberg/pull/11906#discussion_r1920907162 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/DeleteOrphanFilesSparkAction.java: ## @@ -292,19 +294,49 @@ private Dataset validFileIdentDS()

Re: [I] Is it possible to create a table without instantiating an Iceberg catalog? [iceberg-python]

Fokko commented on issue #1535: URL: https://github.com/apache/iceberg-python/issues/1535#issuecomment-2599109331 The YAML is mostly to avoid leaking secrets/credentials into the Python code. The example is unauthenticated, so you can directly pass as the properties which is where the `dic

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

bitsondatadev commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2599251493 > @kevinjqliu - i'm doing this work on behalf of my company and when i ran my tests, i used a standard python virtual environment venv; i haven't figured out quite yet how to

Re: [I] [feature] UpdateSchema.add_column supports both parent and child in the same transaction [iceberg-python]

jiakai-li commented on issue #1493: URL: https://github.com/apache/iceberg-python/issues/1493#issuecomment-2599376964 Thank you @kevinjqliu , I think I have a better understanding now :-) I believe I have some idea, and will push an update once it gets closer. I'm currently on leave so it'

Re: [I] Is it possible to create a table without instantiating an Iceberg catalog? [iceberg-python]

aaron-siegel commented on issue #1535: URL: https://github.com/apache/iceberg-python/issues/1535#issuecomment-2599083869 I'm referring more to using no catalog at all, rather than a file-based catalog; something akin to the InMemoryCatalog provided by Java iceberg that we can throw away on

Re: [PR] Core, API, Spec: Metadata Row Lineage [iceberg]

RussellSpitzer commented on code in PR #11948: URL: https://github.com/apache/iceberg/pull/11948#discussion_r1920826065 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -262,6 +265,8 @@ public String toString() { private volatile Map snapshotsById; privat

[I] Iceberg Maven artifacts do not declare proper dependencies [iceberg]

wallacms opened a new issue, #11994: URL: https://github.com/apache/iceberg/issues/11994 ### Apache Iceberg version 1.7.1 (latest release) ### Query engine None ### Please describe the bug 🐞 If I have a Maven project that declares a single dependency like th

Re: [I] Is it possible to create a table without instantiating an Iceberg catalog? [iceberg-python]

aaron-siegel commented on issue #1535: URL: https://github.com/apache/iceberg-python/issues/1535#issuecomment-2599257180 @Fokko Re improving the documentation, yes - I now see that it was staring me in the face in "Getting Started", but I was looking through the "API" section, which doesn'

Re: [PR] Build: Add plugin to generate license and notice files [iceberg]

RussellSpitzer commented on PR #11977: URL: https://github.com/apache/iceberg/pull/11977#issuecomment-2599272113 I would like to be clear that it is extremely important that we assume all other engineers are contributing in good faith and the interaction above is failing to meet that standa

Re: [PR] Spec: Add added-rows field to Snapshot [iceberg]

RussellSpitzer merged PR #11976: URL: https://github.com/apache/iceberg/pull/11976 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] API: add hashcode cache in StructType [iceberg]

amogh-jahagirdar merged PR #11764: URL: https://github.com/apache/iceberg/pull/11764 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] API: add hashcode cache in StructType [iceberg]

amogh-jahagirdar commented on PR #11764: URL: https://github.com/apache/iceberg/pull/11764#issuecomment-2599281346 I'm going to go ahead and merge this, it's been ready for a while and didn't want to miss this change going in for the next release! Thank you @wzx140 and thank you @singhpk234

Re: [PR] Spec: Add added-rows field to Snapshot [iceberg]

RussellSpitzer commented on PR #11976: URL: https://github.com/apache/iceberg/pull/11976#issuecomment-2599281195 Merged, Thanks @sfc-gh-aixu, @amogh-jahagirdar , @dramaticlly , @danielcweeks , @stevenzwu , @Fokko for reviewing! -- This is an automated message from the Apache Git Service.

Re: [PR] WIP: Add headers for type/field/schema [iceberg-cpp]

gaborkaszab commented on code in PR #31: URL: https://github.com/apache/iceberg-cpp/pull/31#discussion_r1919778107 ## src/iceberg/schema.h: ## @@ -0,0 +1,60 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the

Re: [PR] Azure: Support vended credentials refresh in ADLSFileIO. [iceberg]

adutra commented on code in PR #11577: URL: https://github.com/apache/iceberg/pull/11577#discussion_r1919880274 ## azure/src/main/java/org/apache/iceberg/azure/adlsv2/VendedAdlsCredentialProvider.java: ## @@ -0,0 +1,188 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] WIP: Add headers for type/field/schema [iceberg-cpp]

lidavidm commented on code in PR #31: URL: https://github.com/apache/iceberg-cpp/pull/31#discussion_r1919981922 ## src/iceberg/schema.h: ## @@ -0,0 +1,60 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NO

Re: [PR] WIP: Add headers for type/field/schema [iceberg-cpp]

lidavidm commented on code in PR #31: URL: https://github.com/apache/iceberg-cpp/pull/31#discussion_r1919981227 ## src/iceberg/schema.h: ## @@ -0,0 +1,60 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NO

Re: [PR] WIP: Add headers for type/field/schema [iceberg-cpp]

lidavidm commented on code in PR #31: URL: https://github.com/apache/iceberg-cpp/pull/31#discussion_r1919988333 ## src/iceberg/type.h: ## @@ -0,0 +1,149 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOT

Re: [PR] WIP: Add headers for type/field/schema [iceberg-cpp]

lidavidm commented on code in PR #31: URL: https://github.com/apache/iceberg-cpp/pull/31#discussion_r1919984826 ## src/iceberg/type.h: ## @@ -0,0 +1,149 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOT

Re: [PR] WIP: Add headers for type/field/schema [iceberg-cpp]

gaborkaszab commented on code in PR #31: URL: https://github.com/apache/iceberg-cpp/pull/31#discussion_r1919778107 ## src/iceberg/schema.h: ## @@ -0,0 +1,60 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the

Re: [PR] WIP: Add headers for type/field/schema [iceberg-cpp]

lidavidm commented on code in PR #31: URL: https://github.com/apache/iceberg-cpp/pull/31#discussion_r1919982972 ## src/iceberg/schema.h: ## @@ -0,0 +1,60 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NO

Re: [PR] feat(datafusion): Support cast operations [iceberg-rust]

Fokko commented on PR #821: URL: https://github.com/apache/iceberg-rust/pull/821#issuecomment-2598109685 @ryzhyk @Xuanwo Sorry about that, I think this is actually good to go 👍 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] [WIP]: Data, Flink, Spark: Set PUFFIN as delete file format for V3 [iceberg]

nastra closed pull request #11588: [WIP]: Data, Flink, Spark: Set PUFFIN as delete file format for V3 URL: https://github.com/apache/iceberg/pull/11588 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [I] How to provide Iceberg partitions metadata to upper-layer data platform services? [iceberg]

KnightChess closed issue #11926: How to provide Iceberg partitions metadata to upper-layer data platform services? URL: https://github.com/apache/iceberg/issues/11926 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[PR] Add ALDS support for PyArrow [iceberg-python]

Fokko opened a new pull request, #1532: URL: https://github.com/apache/iceberg-python/pull/1532 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] Spark 3.3: Backport support for default values [iceberg]

manuzhang commented on PR #11988: URL: https://github.com/apache/iceberg/pull/11988#issuecomment-2598310322 Here's [what we say on "Deprecated"](https://iceberg.apache.org/multi-engine-support/#engine-version-lifecycle). > Deprecated: an engine version is no longer actively maintained

Re: [I] Namespace names with dot(.) not supported in Iceberg REST API [iceberg]

nastra commented on issue #11990: URL: https://github.com/apache/iceberg/issues/11990#issuecomment-2598314036 @mingnuj can you please share which issue you ran into? Any stack traces and error messages will help -- This is an automated message from the Apache Git Service. To respond to th

Re: [PR] refine: refine interface of ManifestWriter [iceberg-rust]

liurenjie1024 commented on code in PR #738: URL: https://github.com/apache/iceberg-rust/pull/738#discussion_r1920149166 ## crates/iceberg/src/spec/manifest.rs: ## @@ -210,38 +284,207 @@ impl ManifestWriter { deleted_rows: 0, min_seq_num: None,

Re: [PR] Parquet: Add readers and writers for the internal object model [iceberg]

ajantha-bhat commented on code in PR #11904: URL: https://github.com/apache/iceberg/pull/11904#discussion_r1920166221 ## parquet/src/main/java/org/apache/iceberg/data/parquet/BaseParquetReaders.java: ## @@ -76,6 +67,31 @@ protected ParquetValueReader createReader( protected a

Re: [PR] Add ResidualVisitor to compute residuals [iceberg-python]

tusharchou commented on code in PR #1388: URL: https://github.com/apache/iceberg-python/pull/1388#discussion_r1920208710 ## pyiceberg/table/__init__.py: ## @@ -1596,6 +1630,43 @@ def to_ray(self) -> ray.data.dataset.Dataset: return ray.data.from_arrow(self.to_arrow())

Re: [I] Namespace names with dot(.) not supported in Iceberg REST API [iceberg]

mingnuj commented on issue #11990: URL: https://github.com/apache/iceberg/issues/11990#issuecomment-2598347992 > [@mingnuj](https://github.com/mingnuj) can you please share which issue you ran into? Any stack traces and error messages will help Thank you for your response. I'm sorry f

Re: [PR] Core: List namespaces/tables when testing identifier with a dot [iceberg]

nastra commented on code in PR #11991: URL: https://github.com/apache/iceberg/pull/11991#discussion_r1920229779 ## core/src/test/java/org/apache/iceberg/catalog/CatalogTests.java: ## @@ -244,7 +244,7 @@ public void testLoadNamespaceMetadata() { assertThatThrownBy(() -> ca

Re: [PR] Core: List namespaces/tables when testing identifier with a dot [iceberg]

nastra commented on code in PR #11991: URL: https://github.com/apache/iceberg/pull/11991#discussion_r1920230431 ## core/src/test/java/org/apache/iceberg/catalog/CatalogTests.java: ## @@ -499,6 +499,8 @@ public void testNamespaceWithDot() { catalog.createNamespace(withDot);

Re: [PR] Spark: Fix Puffin suffix for DV files [iceberg]

amogh-jahagirdar merged PR #11986: URL: https://github.com/apache/iceberg/pull/11986 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] Config for deciding whether to use Iceberg Time type [iceberg]

bryanck commented on PR #11174: URL: https://github.com/apache/iceberg/pull/11174#issuecomment-2598431590 We have https://github.com/apache/iceberg/pull/11936 for a new SMT project. We can start adding some common transforms to that, rather than mixing transform logic into the sink itself.

Re: [I] Creating Delete Vectors using Java API or Spark [iceberg]

amogh-jahagirdar closed issue #11968: Creating Delete Vectors using Java API or Spark URL: https://github.com/apache/iceberg/issues/11968 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Add ResidualVisitor to compute residuals [iceberg-python]

tusharchou commented on code in PR #1388: URL: https://github.com/apache/iceberg-python/pull/1388#discussion_r1920238317 ## pyiceberg/expressions/visitors.py: ## @@ -1731,3 +1731,214 @@ def _can_contain_nulls(self, field_id: int) -> bool: def _can_contain_nans(self, field

Re: [PR] Kafka Connect: Add SMTs for Debezium and AWS DMS [iceberg]

bryanck commented on code in PR #11936: URL: https://github.com/apache/iceberg/pull/11936#discussion_r1920242587 ## kafka-connect/build.gradle: ## @@ -242,3 +243,19 @@ project(':iceberg-kafka-connect:iceberg-kafka-connect-runtime') { assemble.dependsOn distZip, hiveDistZip

Re: [PR] Kafka Connect: Add SMTs for Debezium and AWS DMS [iceberg]

bryanck commented on code in PR #11936: URL: https://github.com/apache/iceberg/pull/11936#discussion_r1920238659 ## gradle/libs.versions.toml: ## @@ -159,6 +161,7 @@ jaxb-runtime = { module = "org.glassfish.jaxb:jaxb-runtime", version.ref = "jaxb kafka-clients = { module = "or

Re: [PR] Parquet: Add readers and writers for the internal object model [iceberg]

ajantha-bhat commented on code in PR #11904: URL: https://github.com/apache/iceberg/pull/11904#discussion_r1920247487 ## parquet/src/main/java/org/apache/iceberg/data/parquet/BaseParquetWriter.java: ## @@ -50,6 +42,27 @@ protected ParquetValueWriter createWriter(MessageType typ

Re: [PR] Spark: Fix Puffin suffix for DV files [iceberg]

amogh-jahagirdar commented on PR #11986: URL: https://github.com/apache/iceberg/pull/11986#issuecomment-2598427717 Thanks for reviewing @nastra @Fokko! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Fix comment on `WRITE_OBJECT_STORE_PARTITIONED_PATHS` table property [iceberg]

smaheshwar-pltr commented on PR #11798: URL: https://github.com/apache/iceberg/pull/11798#issuecomment-2598499580 Any update on this? Happy to change if https://github.com/apache/iceberg/pull/11798#discussion_r1888502532 is baseless. I think fixing a misleading comment would be an i

[PR] Core: List namespaces/tables when testing identifier with a dot [iceberg]

nastra opened a new pull request, #11991: URL: https://github.com/apache/iceberg/pull/11991 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

Re: [PR] docs: Add docstrings for Identifier, Properties, RecursiveDict [iceberg-python]

pawamoy commented on PR #1530: URL: https://github.com/apache/iceberg-python/pull/1530#issuecomment-2598415684 @rodrigc do note that I'm just a passer-by! I have no authority here :smile: I'm the maintainer of mkdocstrings and found this issue while stalking a bit, just wanted to help :slig

Re: [PR] Azure: Support vended credentials refresh in ADLSFileIO. [iceberg]

nastra commented on code in PR #11577: URL: https://github.com/apache/iceberg/pull/11577#discussion_r1920173884 ## azure/src/main/java/org/apache/iceberg/azure/adlsv2/AzureSasCredentialRefresher.java: ## @@ -0,0 +1,69 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

Re: [I] Validation Error in ConfigResponse Model When connecting Nessie with PyIceberg using RestCatalog [iceberg-python]

Fokko closed issue #1524: Validation Error in ConfigResponse Model When connecting Nessie with PyIceberg using RestCatalog URL: https://github.com/apache/iceberg-python/issues/1524 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [I] Validation Error in ConfigResponse Model When connecting Nessie with PyIceberg using RestCatalog [iceberg-python]

Fokko commented on issue #1524: URL: https://github.com/apache/iceberg-python/issues/1524#issuecomment-2598349707 Thanks for confirming @heman026, closing this one -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] WIP: Add headers for type/field/schema [iceberg-cpp]

wgtmac commented on code in PR #31: URL: https://github.com/apache/iceberg-cpp/pull/31#discussion_r1920212826 ## src/iceberg/schema.h: ## @@ -0,0 +1,60 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTI

Re: [PR] fix: remove check-docstring-first pre-commit check [iceberg-python]

rodrigc commented on PR #1531: URL: https://github.com/apache/iceberg-python/pull/1531#issuecomment-2598397023 @Fokko I dont see any way around this other than removing this check. #1530 is blocked on this and @pawamoy suggested doing this: https://github.com/apache/iceberg-python/pull/153

Re: [PR] Add ResidualVisitor to compute residuals [iceberg-python]

tusharchou commented on code in PR #1388: URL: https://github.com/apache/iceberg-python/pull/1388#discussion_r1920210749 ## pyiceberg/table/__init__.py: ## @@ -1596,6 +1630,43 @@ def to_ray(self) -> ray.data.dataset.Dataset: return ray.data.from_arrow(self.to_arrow())

Re: [I] [Bug] Error in overwrite(): pyarrow.lib.ArrowInvalid: offset overflow with large dataset (~3M rows) [iceberg-python]

Fokko commented on issue #1491: URL: https://github.com/apache/iceberg-python/issues/1491#issuecomment-2598611790 I can take a stab at this if nobody objects. @bigluck Why are you still at 0.7.1? 😆 -- This is an automated message from the Apache Git Service. To respond to the mess

  1   2   >