Re: [PR] fix 0.5.x release `cargo publish` [iceberg-rust]

kevinjqliu commented on PR #1379: URL: https://github.com/apache/iceberg-rust/pull/1379#issuecomment-2910948039 im going to test the github action on my fork first before merging -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[PR] Core: Don't copy stats of delete files based on ManifestReader.dropStats [iceberg]

ebyhr opened a new pull request, #13161: URL: https://github.com/apache/iceberg/pull/13161 I've been investigating the OOM issue in the Trino coordinator. It turns out that a significant portion of memory usage comes from the statistics stored in `DeleteFileIndex`. To address this,

Re: [PR] feat(memory): Implement update_table for MemoryCatalog [iceberg-rust]

liurenjie1024 commented on PR #1381: URL: https://github.com/apache/iceberg-rust/pull/1381#issuecomment-2911729753 Hi, @hsingh574 Thanks for contributing. I've created #1382 to add initial support for write, which also include a framework for transaction to execute. I think it should be par

Re: [I] [EPIC] Iceberg-rust Write support [iceberg-rust]

liurenjie1024 closed issue #700: [EPIC] Iceberg-rust Write support URL: https://github.com/apache/iceberg-rust/issues/700 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [I] Iceberg Kafka Connector experiences a constant hanging lag for low-volume topics [iceberg]

PhillHenry commented on issue #11818: URL: https://github.com/apache/iceberg/issues/11818#issuecomment-2911858587 @fenil25 I think this might be due to the comparison of the number of messages received not being greater or equal to the number of partitions (see [this line](https://github.c

Re: [PR] Experiment implementation for catalog builder [iceberg-rust]

sdd commented on PR #1231: URL: https://github.com/apache/iceberg-rust/pull/1231#issuecomment-2911881967 Makes sense to me -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] Iceberg Kafka Connector experiences a constant hanging lag for low-volume topics [iceberg]

PhillHenry commented on issue #11818: URL: https://github.com/apache/iceberg/issues/11818#issuecomment-2911983708 I've written [a test](https://gist.github.com/PhillHenry/02a4a14ca280d665040c5aadb7a2bccf) that demonstrates. If the number of messages is >= `TEST_TOPIC_PARTITIONS` then this

[PR] Bump iceberg-rust version to 0.5.1 [iceberg-rust]

kevinjqliu opened a new pull request, #1380: URL: https://github.com/apache/iceberg-rust/pull/1380 ## Which issue does this PR close? - Closes #. ## What changes are included in this PR? Bump version to `0.5.1` Update `CHANGELOG.md` Generate dependencies

[PR] Core: Fix incremental compute of partition stats [iceberg]

ajantha-bhat opened a new pull request, #13163: URL: https://github.com/apache/iceberg/pull/13163 Fixes an edge case where deleted entires information is not carried to next snapshot. So, incremental stats compute was wrong for this copy on write case. Fixes: #13155 Note: Aft

Re: [I] INT96 timestamp is read as OffsetDateTime, not LocalDateTime [iceberg]

elphastori commented on issue #12266: URL: https://github.com/apache/iceberg/issues/12266#issuecomment-2912175829 I've done some digging and found that `GenericParquetReaders` initially read `int96` timestamps without timezone information. This behavior was changed from using `LocalDa

Re: [I] Incrementally computing partition stats can miss deleted files [iceberg]

ajantha-bhat commented on issue #13155: URL: https://github.com/apache/iceberg/issues/13155#issuecomment-2912175962 > Yeah, given a target snapshot, we can walk through the ancestors and look for an existing stats file. If we can find one, it means all snapshots in between are still there,

Re: [I] Spark 4.0 [iceberg]

nastra closed issue #13162: Spark 4.0 URL: https://github.com/apache/iceberg/issues/13162 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubsc

Re: [I] Spark 4.0 [iceberg]

nastra commented on issue #13162: URL: https://github.com/apache/iceberg/issues/13162#issuecomment-2912307091 Spark 4 just made it into Iceberg and support will be shipped with the next Iceberg release -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] Build: Upgrade to Gradle 8.14.1 [iceberg]

nastra merged PR #13149: URL: https://github.com/apache/iceberg/pull/13149 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

[I] Nested field IDs in user-defined schema are reassigned during table creation, causing query failures in external engines [iceberg]

sclee01 opened a new issue, #13164: URL: https://github.com/apache/iceberg/issues/13164 ### Apache Iceberg version 1.9.0 (latest release) ### Query engine Snowflake ### Please describe the bug 🐞 ### Description When creating an Iceberg table using a m

Re: [PR] Build: Bump software.amazon.awssdk:bom from 2.31.45 to 2.31.50 [iceberg]

nastra merged PR #13148: URL: https://github.com/apache/iceberg/pull/13148 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Build: Bump org.apache.httpcomponents.client5:httpclient5 from 5.4.4 to 5.5 [iceberg]

nastra merged PR #13147: URL: https://github.com/apache/iceberg/pull/13147 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Build: Bump com.google.cloud:libraries-bom from 26.60.0 to 26.61.0 [iceberg]

nastra merged PR #13146: URL: https://github.com/apache/iceberg/pull/13146 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Spark 3.5: Update to Spark 3.5.6 [iceberg]

ajantha-bhat commented on PR #13142: URL: https://github.com/apache/iceberg/pull/13142#issuecomment-2912344021 Thanks @drexler-sky for the PR. I would suggest keeping the PR in draft till the official release is available. But it was good to test it out in advance 👍 -- This is an aut

Re: [PR] Build: Bump com.palantir.gradle.gitversion:gradle-git-version from 3.2.0 to 3.3.0 [iceberg]

nastra merged PR #13144: URL: https://github.com/apache/iceberg/pull/13144 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] SPARK: Remove dependency on hadoop's filesystem class from remove orphan files [iceberg]

pvary commented on PR #12254: URL: https://github.com/apache/iceberg/pull/12254#issuecomment-2912397591 @RussellSpitzer: Any comments on the PR? If you are ok with this, I can merge -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [I] Add docs of Spark SQL functions for Iceberg transforms [iceberg]

manuzhang commented on issue #13156: URL: https://github.com/apache/iceberg/issues/13156#issuecomment-2912026202 @VerneZhong Welcome. You may refer to examples, `TestSpark***Function.java`, under https://github.com/apache/iceberg/tree/main/spark/v3.5/spark/src/test/java/org/apache/iceberg/s

Re: [I] Iceberg Kafka Connector experiences a constant hanging lag for low-volume topics [iceberg]

PhillHenry commented on issue #11818: URL: https://github.com/apache/iceberg/issues/11818#issuecomment-2912223823 Actually, @fenil25 if I set `tasks.max` to `1` in the config for `IcebergSinkConnector`, my test in the gist passes. That is, there will only be one SinkTask. I'm guessing th

[I] Spark 4.0 [iceberg]

kesavkolla opened a new issue, #13162: URL: https://github.com/apache/iceberg/issues/13162 ### Feature Request / Improvement Current release of iceberg 1.9.0 doesn't work with spark 4.0. Now spark 4.0 released it would be great to add that support into iceberg. ### Query engine

Re: [PR] Docs: add column descriptions for entries metadata table [iceberg]

manuzhang commented on code in PR #13104: URL: https://github.com/apache/iceberg/pull/13104#discussion_r2109157835 ## docs/docs/spark-queries.md: ## @@ -301,6 +301,16 @@ SELECT * FROM prod.db.table.entries; | -- | -- | -- | -- | -- | -- | | 2 | 57897183625154 | 0 | 0 | {"cont

[PR] Flink: Backport the removal of MiniClusterWithClientResource dependency to 1.19 and 1.20 [iceberg]

JeonDaehong opened a new pull request, #13165: URL: https://github.com/apache/iceberg/pull/13165 I have backported the changes from [Remove the MiniClusterWithClientResource dependency](https://github.com/apache/iceberg/pull/13021#issuecomment-2903837698) to the 1.19 and 1.20 branches.

Re: [PR] Proposal: IRC Events endpoint [iceberg]

snazy commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2109196548 ## open-api/rest-catalog-open-api.yaml: ## @@ -3405,6 +3488,131 @@ components: allOf: - $ref: '#/components/schemas/ScanTasks' +Actor: + type

Re: [PR] Docs: add column descriptions for entries metadata table [iceberg]

elphastori commented on code in PR #13104: URL: https://github.com/apache/iceberg/pull/13104#discussion_r2109211954 ## docs/docs/spark-queries.md: ## @@ -301,6 +301,16 @@ SELECT * FROM prod.db.table.entries; | -- | -- | -- | -- | -- | -- | | 2 | 57897183625154 | 0 | 0 | {"con

Re: [PR] Core: Don't copy stats of delete files based on ManifestReader.dropStats [iceberg]

ebyhr commented on PR #13161: URL: https://github.com/apache/iceberg/pull/13161#issuecomment-2911565200 Looking into CI failures. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] feat: basic table scan planning [iceberg-cpp]

lidavidm commented on code in PR #112: URL: https://github.com/apache/iceberg-cpp/pull/112#discussion_r2108439814 ## src/iceberg/result.h: ## @@ -31,6 +31,7 @@ namespace iceberg { enum class ErrorKind { kAlreadyExists, kCommitStateUnknown, + kDataInvalid, Review Comment

[PR] feat(memory): Implement update_table for MemoryCatalog [iceberg-rust]

hsingh574 opened a new pull request, #1381: URL: https://github.com/apache/iceberg-rust/pull/1381 My initial motivation was to work on #1322, but for that the MemoryCatalog needs to support update_table, so this is that implementation, verified with a few basic tests. There's more t

Re: [PR] feat: basic table scan planning [iceberg-cpp]

gty404 commented on code in PR #112: URL: https://github.com/apache/iceberg-cpp/pull/112#discussion_r2108542013 ## src/iceberg/result.h: ## @@ -31,6 +31,7 @@ namespace iceberg { enum class ErrorKind { kAlreadyExists, kCommitStateUnknown, + kDataInvalid, Review Comment:

[I] [EPIC] Support for appending data to iceberg table. [iceberg-rust]

liurenjie1024 opened a new issue, #1382: URL: https://github.com/apache/iceberg-rust/issues/1382 ### What's the feature are you trying to implement? This issue is used to track implementing appending data to iceberg table. In summary, after we complete all subtasks in this issue, we s

Re: [I] [EPIC] Iceberg-rust Write support [iceberg-rust]

liurenjie1024 commented on issue #700: URL: https://github.com/apache/iceberg-rust/issues/700#issuecomment-2911636262 New tracking issue here: https://github.com/apache/iceberg-rust/issues/1382 -- This is an automated message from the Apache Git Service. To respond to the message, please l

Re: [PR] Fix projected fields predicate evaluation [iceberg-python]

Erigara commented on code in PR #2029: URL: https://github.com/apache/iceberg-python/pull/2029#discussion_r2108588368 ## pyiceberg/expressions/visitors.py: ## @@ -894,12 +895,17 @@ def visit_unbound_predicate(self, predicate: UnboundPredicate[L]) -> BooleanExpr def visit

Re: [PR] Proposal: IRC Events endpoint [iceberg]

nastra commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2108580061 ## open-api/rest-catalog-open-api.yaml: ## @@ -3041,6 +3096,34 @@ components: - $ref: '#/components/schemas/AddViewVersionUpdate' - $ref: '#/componen

Re: [I] [EPIC] Support for appending data to iceberg table. [iceberg-rust]

jonathanc-n commented on issue #1382: URL: https://github.com/apache/iceberg-rust/issues/1382#issuecomment-2911730515 Thanks for creating this @liurenjie1024! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] static table metadata access support [iceberg-cpp]

gty404 commented on code in PR #111: URL: https://github.com/apache/iceberg-cpp/pull/111#discussion_r2108619009 ## src/iceberg/table.cc: ## @@ -0,0 +1,173 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the N

Re: [PR] Flink: Backport the removal of MiniClusterWithClientResource dependency to 1.19 and 1.20 [iceberg]

JeonDaehong commented on PR #13165: URL: https://github.com/apache/iceberg/pull/13165#issuecomment-2912548643 @nastra Hello ! I noticed that spotlessApply is not supported for iceberg-flink:iceberg-flink-1.19 or iceberg-flink:iceberg-flink-1.20, so I proceeded using iceberg-f

Re: [PR] Flink: Migrate Flink TableSchema for IcebergSource [iceberg]

pvary commented on code in PR #13072: URL: https://github.com/apache/iceberg/pull/13072#discussion_r2109094485 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/FlinkSchemaUtil.java: ## @@ -192,7 +222,9 @@ public static Type convert(LogicalType flinkType) { * *

Re: [PR] Docs: add column descriptions for entries metadata table [iceberg]

elphastori commented on code in PR #13104: URL: https://github.com/apache/iceberg/pull/13104#discussion_r2109211954 ## docs/docs/spark-queries.md: ## @@ -301,6 +301,16 @@ SELECT * FROM prod.db.table.entries; | -- | -- | -- | -- | -- | -- | | 2 | 57897183625154 | 0 | 0 | {"con

Re: [PR] Feature: Write to branches [iceberg-python]

vinjai commented on PR #941: URL: https://github.com/apache/iceberg-python/pull/941#issuecomment-2912737391 Hey @Fokko This PR is ready for review again -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Hive: Throw exception for when listing a non-existing namespace [iceberg]

pvary commented on code in PR #13130: URL: https://github.com/apache/iceberg/pull/13130#discussion_r2110052098 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCatalog.java: ## @@ -1209,11 +1208,4 @@ public void testDatabaseLocationWithSlashInWarehouseDir() {

[PR] Fix the CI [iceberg-python]

Fokko opened a new pull request, #2049: URL: https://github.com/apache/iceberg-python/pull/2049 # Rationale for this change # Are these changes tested? # Are there any user-facing changes? -- This is an automated message from the Apache Git Ser

Re: [PR] SPARK: Remove dependency on hadoop's filesystem class from remove orphan files [iceberg]

RussellSpitzer commented on code in PR #12254: URL: https://github.com/apache/iceberg/pull/12254#discussion_r2110065489 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/DeleteOrphanFilesSparkAction.java: ## @@ -303,39 +310,89 @@ private Dataset listedFileDS() {

Re: [PR] feat: implement initial MemoryCatalog functionality with namespace and table support [iceberg-cpp]

Fokko commented on PR #80: URL: https://github.com/apache/iceberg-cpp/pull/80#issuecomment-2913814348 Yes, looks good @gty404 thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] feat: implement initial MemoryCatalog functionality with namespace and table support [iceberg-cpp]

Fokko merged PR #80: URL: https://github.com/apache/iceberg-cpp/pull/80 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [PR] SPARK: Remove dependency on hadoop's filesystem class from remove orphan files [iceberg]

RussellSpitzer commented on code in PR #12254: URL: https://github.com/apache/iceberg/pull/12254#discussion_r2110075002 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/DeleteOrphanFilesSparkAction.java: ## @@ -303,39 +310,89 @@ private Dataset listedFileDS() {

Re: [PR] SPARK: Remove dependency on hadoop's filesystem class from remove orphan files [iceberg]

RussellSpitzer commented on code in PR #12254: URL: https://github.com/apache/iceberg/pull/12254#discussion_r2110088911 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/actions/TestRemoveOrphanFilesAction.java: ## @@ -344,6 +374,8 @@ public void testWapFilesAreKept() {

Re: [PR] AWS: Support multiple storage credential prefixes [iceberg]

nastra commented on PR #12799: URL: https://github.com/apache/iceberg/pull/12799#issuecomment-2913084781 @and124578963 can you please open a new issue with some additional details about your catalog configuration? -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] feat(memory): Implement update_table for MemoryCatalog [iceberg-rust]

hsingh574 commented on PR #1381: URL: https://github.com/apache/iceberg-rust/pull/1381#issuecomment-2912907798 Hi, @liurenjie1024 thanks for taking a look. Regarding the commit path functionality mentioned in #1382 , do you think there's anything blocking this PR? I'm also unsure about the

Re: [PR] AWS: Support multiple storage credential prefixes [iceberg]

and124578963 commented on PR #12799: URL: https://github.com/apache/iceberg/pull/12799#issuecomment-2912973308 Hey. I had a problem with this commit. As I understand my path start with s3a://, not s3://. How to configure it? After removing the commit, all is working well. ``` 25

Re: [I] [EPIC] Support for appending data to iceberg table. [iceberg-rust]

ZENOTME commented on issue #1382: URL: https://github.com/apache/iceberg-rust/issues/1382#issuecomment-2912574552 Thanks @liurenjie1024 for this! Let's move! > Add a method in TableCommit to apply changes to TableMetadata to create a new TableMetadata I don't get why we need th

Re: [PR] Spark 3.5: Update to Spark 3.5.6 [iceberg]

huaxingao commented on PR #13142: URL: https://github.com/apache/iceberg/pull/13142#issuecomment-2913181284 @drexler-sky I would also suggest keeping this in draft. I accidentally changed it to draft—sorry about that! Please feel free to mark it as ready for review when it's ready. Thanks!

Re: [PR] fix: upgrade spark version [iceberg-rust]

kevinjqliu merged PR #1384: URL: https://github.com/apache/iceberg-rust/pull/1384 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ice

Re: [PR] SPARK: Remove dependency on hadoop's filesystem class from remove orphan files [iceberg]

RussellSpitzer commented on code in PR #12254: URL: https://github.com/apache/iceberg/pull/12254#discussion_r2110093075 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/actions/TestRemoveOrphanFilesAction.java: ## @@ -1067,6 +1148,38 @@ public void testRemoveOrphanFile

Re: [PR] Build: Bump moto from 5.1.4 to 5.1.5 [iceberg-python]

Fokko merged PR #2039: URL: https://github.com/apache/iceberg-python/pull/2039 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Build: Bump mypy-boto3-glue from 1.38.18 to 1.38.22 [iceberg-python]

Fokko merged PR #2038: URL: https://github.com/apache/iceberg-python/pull/2038 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Build: Bump pydantic from 2.11.4 to 2.11.5 [iceberg-python]

Fokko merged PR #2045: URL: https://github.com/apache/iceberg-python/pull/2045 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Build: Bump getdaft from 0.4.15 to 0.4.16 [iceberg-python]

Fokko merged PR #2042: URL: https://github.com/apache/iceberg-python/pull/2042 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Build: Bump huggingface-hub from 0.31.4 to 0.32.1 [iceberg-python]

Fokko merged PR #2046: URL: https://github.com/apache/iceberg-python/pull/2046 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Build: Bump coverage from 7.8.0 to 7.8.2 [iceberg-python]

Fokko merged PR #2047: URL: https://github.com/apache/iceberg-python/pull/2047 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Build: Bump thrift from 0.21.0 to 0.22.0 [iceberg-python]

Fokko merged PR #2043: URL: https://github.com/apache/iceberg-python/pull/2043 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] SPARK: Remove dependency on hadoop's filesystem class from remove orphan files [iceberg]

RussellSpitzer commented on PR #12254: URL: https://github.com/apache/iceberg/pull/12254#issuecomment-2914044262 The only real fix I need is the error message, I have some other suggestions there to look into but they are more nits -- This is an automated message from the Apache Git Servi

Re: [PR] Build: Bump cachetools from 5.5.2 to 6.0.0 [iceberg-python]

kevinjqliu merged PR #2041: URL: https://github.com/apache/iceberg-python/pull/2041 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

Re: [PR] Core: Fix incremental compute of partition stats [iceberg]

lirui-apache commented on PR #13163: URL: https://github.com/apache/iceberg/pull/13163#issuecomment-2914862185 @ajantha-bhat Thanks for working on this! Left some comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [PR] Core: Fix numeric overflow of timestamp nano literal [iceberg]

ajantha-bhat commented on PR #11775: URL: https://github.com/apache/iceberg/pull/11775#issuecomment-2914902672 We faced this at Dremio too : https://github.com/apache/iceberg/issues/13160 Looks forward to this fix. -- This is an automated message from the Apache Git Service. To res

Re: [PR] REST: Avoid table corruption on self conflicts due to internal HTTP retries [iceberg]

singhpk234 commented on code in PR #12818: URL: https://github.com/apache/iceberg/pull/12818#discussion_r2110958150 ## core/src/main/java/org/apache/iceberg/rest/responses/ErrorResponse.java: ## @@ -126,9 +134,14 @@ public Builder responseCode(Integer responseCode) { retu

Re: [I] Literals.LongLiteral conversion issue to TimestampNanoLiteral [iceberg]

ebyhr commented on issue #13160: URL: https://github.com/apache/iceberg/issues/13160#issuecomment-2914854172 We have an existing PR #11775 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] Docs: add column descriptions for entries metadata table [iceberg]

nastra commented on code in PR #13104: URL: https://github.com/apache/iceberg/pull/13104#discussion_r2110970830 ## docs/docs/spark-queries.md: ## @@ -301,6 +301,16 @@ SELECT * FROM prod.db.table.entries; | -- | -- | -- | -- | -- | -- | | 2 | 57897183625154 | 0 | 0 | {"content

[I] Add a method in `TableCommit` to apply changes to `TableMetadata` to create a new `TableMetadata`. [iceberg-rust]

liurenjie1024 opened a new issue, #1385: URL: https://github.com/apache/iceberg-rust/issues/1385 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

[I] Implement saving `TableMetadata` to new location. [iceberg-rust]

liurenjie1024 opened a new issue, #1388: URL: https://github.com/apache/iceberg-rust/issues/1388 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

Re: [I] [EPIC] Support for appending data to iceberg table. [iceberg-rust]

CTTY commented on issue #1382: URL: https://github.com/apache/iceberg-rust/issues/1382#issuecomment-2915116041 Hi @liurenjie1024 , thanks for having this! Would it be ok if I take up the commit path work? I can start working on a POC tomorrow -- This is an automated message from t

Re: [PR] feat: support decompress gzip metadata [iceberg-cpp]

wgtmac commented on PR #108: URL: https://github.com/apache/iceberg-cpp/pull/108#issuecomment-2914954957 Iceberg itself only requires gzip for metadata json file. Specific file format libraries we use for Parquet and Avro have already supported different codecs so we don't need to deal with

Re: [PR] Hive: Throw exception for when listing a non-existing namespace [iceberg]

pvary commented on code in PR #13130: URL: https://github.com/apache/iceberg/pull/13130#discussion_r2110933234 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCatalog.java: ## @@ -1209,11 +1208,4 @@ public void testDatabaseLocationWithSlashInWarehouseDir() {

Re: [PR] Core: Make pageToken query parameter optional [iceberg]

geruh commented on code in PR #13129: URL: https://github.com/apache/iceberg/pull/13129#discussion_r2110998730 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -565,13 +568,16 @@ public List listNamespaces(SessionContext context, Namespace namespac

Re: [I] Spark: add IcebergHiveConnectorDelegationTokenProvider for HiveCatalog [iceberg]

pan3793 commented on issue #13116: URL: https://github.com/apache/iceberg/issues/13116#issuecomment-2915058090 Glad to see that the community is considering multiple HMS cases, actually, a similar work has been done in Kyuubi Spark Hive Connector - a Hive connector based on Spark DSv2 API t

Re: [PR] spark 4.0 : SPJ : add hour to day reducer [iceberg]

himadripal commented on PR #13166: URL: https://github.com/apache/iceberg/pull/13166#issuecomment-2915116839 To trigger build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[I] Add a method in TableCommit to apply changes to TableMetadata to create a new TableMetadata [iceberg-rust]

liurenjie1024 opened a new issue, #1386: URL: https://github.com/apache/iceberg-rust/issues/1386 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

Re: [PR] spark 4.0 : SPJ : add hour to day reducer [iceberg]

himadripal closed pull request #13166: spark 4.0 : SPJ : add hour to day reducer URL: https://github.com/apache/iceberg/pull/13166 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[I] Add retry in `Transaction::commit` method [iceberg-rust]

liurenjie1024 opened a new issue, #1387: URL: https://github.com/apache/iceberg-rust/issues/1387 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

Re: [PR] Core: Fix incremental compute of partition stats [iceberg]

lirui-apache commented on code in PR #13163: URL: https://github.com/apache/iceberg/pull/13163#discussion_r2110839769 ## core/src/main/java/org/apache/iceberg/PartitionStatsHandler.java: ## @@ -336,16 +336,22 @@ private static PartitionMap computeStatsDiff( Sets.newHas

Re: [PR] feat: add asan and ubsan support to cmake [iceberg-cpp]

wgtmac commented on code in PR #107: URL: https://github.com/apache/iceberg-cpp/pull/107#discussion_r2110912596 ## .github/workflows/sanitizer_test.yml: ## @@ -0,0 +1,63 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

Re: [PR] SPARK: Remove dependency on hadoop's filesystem class from remove orphan files [iceberg]

pvary commented on code in PR #12254: URL: https://github.com/apache/iceberg/pull/12254#discussion_r2110925431 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/actions/TestRemoveOrphanFilesAction.java: ## @@ -107,9 +115,12 @@ public abstract class TestRemoveOrphanFiles

Re: [PR] Core: Make pageToken query parameter optional [iceberg]

kevinjqliu commented on code in PR #13129: URL: https://github.com/apache/iceberg/pull/13129#discussion_r2110923274 ## core/src/main/java/org/apache/iceberg/rest/CatalogHandlers.java: ## @@ -111,6 +111,10 @@ public CommitFailedException wrapped() { } private static Pair

[I] Add integration test for SQL catalog [iceberg-go]

dttung2905 opened a new issue, #442: URL: https://github.com/apache/iceberg-go/issues/442 ### Feature Request / Improvement A place holder to track the feedback in this [comment](https://github.com/apache/iceberg-go/pull/414#issuecomment-2910706995). To be implemented once the PR htt

Re: [I] Incrementally computing partition stats can miss deleted files [iceberg]

ajantha-bhat commented on issue #13155: URL: https://github.com/apache/iceberg/issues/13155#issuecomment-2913380025 I think other option is to just consider DELETED entires from previous snapshots as it won't be carried out for next snapshot. Now I considered all the entries from all the pr

Re: [PR] fix: upgrade spark version [iceberg-rust]

ZENOTME commented on PR #1384: URL: https://github.com/apache/iceberg-rust/pull/1384#issuecomment-2913336403 cc @liurenjie1024 @Xuanwo @Fokko @kevinjqliu -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [PR] Build: Bump getdaft from 0.4.15 to 0.4.16 [iceberg-python]

kevinjqliu commented on PR #2042: URL: https://github.com/apache/iceberg-python/pull/2042#issuecomment-2913912801 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Build: Bump cachetools from 5.5.2 to 6.0.0 [iceberg-python]

kevinjqliu commented on PR #2041: URL: https://github.com/apache/iceberg-python/pull/2041#issuecomment-2913912859 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Build: Bump mypy-boto3-glue from 1.38.18 to 1.38.22 [iceberg-python]

kevinjqliu commented on PR #2038: URL: https://github.com/apache/iceberg-python/pull/2038#issuecomment-2913913073 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Build: Bump mkdocstrings-python from 1.16.10 to 1.16.11 [iceberg-python]

kevinjqliu commented on PR #2044: URL: https://github.com/apache/iceberg-python/pull/2044#issuecomment-2913912625 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Build: Bump pydantic from 2.11.4 to 2.11.5 [iceberg-python]

kevinjqliu commented on PR #2045: URL: https://github.com/apache/iceberg-python/pull/2045#issuecomment-2913912513 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Build: Bump thrift from 0.21.0 to 0.22.0 [iceberg-python]

kevinjqliu commented on PR #2043: URL: https://github.com/apache/iceberg-python/pull/2043#issuecomment-2913912711 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Build: Bump coverage from 7.8.0 to 7.8.2 [iceberg-python]

kevinjqliu commented on PR #2047: URL: https://github.com/apache/iceberg-python/pull/2047#issuecomment-2913912350 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Build: Bump mkdocs-autorefs from 1.4.1 to 1.4.2 [iceberg-python]

kevinjqliu commented on PR #2048: URL: https://github.com/apache/iceberg-python/pull/2048#issuecomment-2913912223 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] SPARK: Remove dependency on hadoop's filesystem class from remove orphan files [iceberg]

RussellSpitzer commented on code in PR #12254: URL: https://github.com/apache/iceberg/pull/12254#discussion_r2110086157 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/actions/TestRemoveOrphanFilesAction.java: ## @@ -107,9 +115,12 @@ public abstract class TestRemoveOr

Re: [PR] Fix the CI [iceberg-python]

kevinjqliu commented on PR #2049: URL: https://github.com/apache/iceberg-python/pull/2049#issuecomment-2913913562 rebased dependabot PRs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] Build: Bump moto from 5.1.4 to 5.1.5 [iceberg-python]

kevinjqliu commented on PR #2039: URL: https://github.com/apache/iceberg-python/pull/2039#issuecomment-2913912991 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Build: Bump pytest-mock from 3.14.0 to 3.14.1 [iceberg-python]

kevinjqliu commented on PR #2040: URL: https://github.com/apache/iceberg-python/pull/2040#issuecomment-2913912920 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

  1   2   >