Re: [PR] Spec: Add implementation note on `current-snapshot-id` [iceberg]

2025-03-05 Thread via GitHub
Fokko commented on PR #12334: URL: https://github.com/apache/iceberg/pull/12334#issuecomment-2703056127 See https://lists.apache.org/thread/54r4nm7qmr4vxhdpwmbx5rntynspskl7 Thanks everyone! -- This is an automated message from the Apache Git Service. To respond to the message, pleas

Re: [PR] Avro: Add variant readers and writers [iceberg]

2025-03-05 Thread via GitHub
aihuaxu commented on code in PR #12457: URL: https://github.com/apache/iceberg/pull/12457#discussion_r1982799286 ## core/src/main/java/org/apache/iceberg/avro/ValueReaders.java: ## @@ -141,6 +144,13 @@ public static ValueReader decimalBytesReader(Schema schema) { } }

Re: [I] Serialize `null` for `current-snapshot-id` when there is no current snapshot for ≥V3 [iceberg]

2025-03-05 Thread via GitHub
Fokko closed issue #12310: Serialize `null` for `current-snapshot-id` when there is no current snapshot for ≥V3 URL: https://github.com/apache/iceberg/issues/12310 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] Core: Write `null` for `current-snapshot-id` for V3+ [iceberg]

2025-03-05 Thread via GitHub
Fokko commented on PR #12335: URL: https://github.com/apache/iceberg/pull/12335#issuecomment-2703016795 Moving this forward, thanks @nastra and @RussellSpitzer -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] Build: Bump jinja2 from 3.1.5 to 3.1.6 [iceberg-python]

2025-03-05 Thread via GitHub
Fokko merged PR #1769: URL: https://github.com/apache/iceberg-python/pull/1769 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] feat: support delete if empty for parquet writer [iceberg-rust]

2025-03-05 Thread via GitHub
liurenjie1024 merged PR #838: URL: https://github.com/apache/iceberg-rust/pull/838 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [I] Replace parquet metadata thrift version with in memory version. [iceberg-rust]

2025-03-05 Thread via GitHub
jonathanc-n commented on issue #1004: URL: https://github.com/apache/iceberg-rust/issues/1004#issuecomment-2702917222 Yes, I'll look to submit an issue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Spark: Support singular form of years, months, days, and hours functions [iceberg]

2025-03-05 Thread via GitHub
wypoon commented on PR #12117: URL: https://github.com/apache/iceberg/pull/12117#issuecomment-2702853084 @nastra can you take a look at this? @Fokko seems to have disappeared. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [PR] Reduce code duplication in VectorizedParquetDefinitionLevelReader [iceberg]

2025-03-05 Thread via GitHub
wypoon commented on PR #11661: URL: https://github.com/apache/iceberg/pull/11661#issuecomment-2702853897 @amogh-jahagirdar can you please take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Spark: Structured Streaming read limit support follow-up [iceberg]

2025-03-05 Thread via GitHub
wypoon commented on PR #12260: URL: https://github.com/apache/iceberg/pull/12260#issuecomment-2702852279 @szehon-ho @aokolnychyi would you mind reviewing this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[I] --output json not supported? [iceberg-go]

2025-03-05 Thread via GitHub
knoguchi opened a new issue, #325: URL: https://github.com/apache/iceberg-go/issues/325 ### Apache Iceberg version None ### Please describe the bug 🐞 --ourput json prints "unimplemented output type". text works. ``` ❯ iceberg --uri http://127.0.0.1:8181 --output js

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-05 Thread via GitHub
hsiang-c commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1982675279 ## core/src/test/java/org/apache/iceberg/util/TestGeometryUtil.java: ## @@ -0,0 +1,145 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

[I] config files from yaml file is ignored when docopt has default values [iceberg-go]

2025-03-05 Thread via GitHub
knoguchi opened a new issue, #324: URL: https://github.com/apache/iceberg-go/issues/324 ### Apache Iceberg version None ### Please describe the bug 🐞 Does this really work? https://github.com/apache/iceberg-go/blob/ba883bdb1d26cc8ea0505e3910ffee0f9c74f315/cmd/iceberg

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-05 Thread via GitHub
hsiang-c commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1982674696 ## api/src/main/java/org/apache/iceberg/types/Types.java: ## @@ -70,6 +74,20 @@ public static Type fromTypeName(String typeString) { return TYPES.get(lowerTyp

Re: [PR] Spark-3.5: Add procedure to compute partition stats [iceberg]

2025-03-05 Thread via GitHub
ajantha-bhat commented on code in PR #12451: URL: https://github.com/apache/iceberg/pull/12451#discussion_r1982665100 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/ComputePartitionStatsSparkAction.java: ## @@ -0,0 +1,88 @@ +/* + * Licensed to the Apache Soft

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-05 Thread via GitHub
Kontinuation commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1982624089 ## api/src/main/java/org/apache/iceberg/types/Types.java: ## @@ -543,6 +561,120 @@ public int hashCode() { } } + public static class GeometryType exte

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-05 Thread via GitHub
Kontinuation commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1982617958 ## core/src/main/java/org/apache/iceberg/util/GeometryUtil.java: ## @@ -0,0 +1,75 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-05 Thread via GitHub
Kontinuation commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1982625703 ## core/src/test/java/org/apache/iceberg/util/TestGeometryUtil.java: ## @@ -0,0 +1,145 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-05 Thread via GitHub
Kontinuation commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1982617958 ## core/src/main/java/org/apache/iceberg/util/GeometryUtil.java: ## @@ -0,0 +1,75 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-05 Thread via GitHub
Kontinuation commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1982617958 ## core/src/main/java/org/apache/iceberg/util/GeometryUtil.java: ## @@ -0,0 +1,75 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-05 Thread via GitHub
Kontinuation commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1982612715 ## api/src/main/java/org/apache/iceberg/types/Types.java: ## @@ -70,6 +74,20 @@ public static Type fromTypeName(String typeString) { return TYPES.get(lowe

Re: [I] Consolidate methods of converting parquet file to data file builder. [iceberg-rust]

2025-03-05 Thread via GitHub
liurenjie1024 commented on issue #1033: URL: https://github.com/apache/iceberg-rust/issues/1033#issuecomment-2702624277 > Hi [@liurenjie1024](https://github.com/liurenjie1024), I would like to work on this. Thanks ! -- This is an automated message from the Apache Git Service. To re

Re: [I] Replace parquet metadata thrift version with in memory version. [iceberg-rust]

2025-03-05 Thread via GitHub
liurenjie1024 commented on issue #1004: URL: https://github.com/apache/iceberg-rust/issues/1004#issuecomment-2702617699 > Probably not, so should we cahnge the AsyncFileWriter to return the in memory representation? Yes, but it seems there is no built no approach to do that? We may ne

Re: [I] Replace parquet metadata thrift version with in memory version. [iceberg-rust]

2025-03-05 Thread via GitHub
jonathanc-n commented on issue #1004: URL: https://github.com/apache/iceberg-rust/issues/1004#issuecomment-2702597751 Probably not, so should we cahnge the AsyncFileWriter to return the in memory representation? -- This is an automated message from the Apache Git Service. To respond to th

Re: [I] Replace `FileMetadata` in parquet writer with in memory representation. [iceberg-rust]

2025-03-05 Thread via GitHub
liurenjie1024 commented on issue #1004: URL: https://github.com/apache/iceberg-rust/issues/1004#issuecomment-2702526583 > [@liurenjie1024](https://github.com/liurenjie1024) I think the current problem with this is that `ArrowFileReader` (reader) returns `ParquetMetadata` and `AsyncFileWrite

Re: [PR] support create table like in flink catalog [iceberg]

2025-03-05 Thread via GitHub
swapna267 commented on PR #12199: URL: https://github.com/apache/iceberg/pull/12199#issuecomment-2702446281 Thanks @stevenzwu and @pvary for the review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] RewriteDataFiles maintenance action never converges [iceberg]

2025-03-05 Thread via GitHub
github-actions[bot] commented on issue #6669: URL: https://github.com/apache/iceberg/issues/6669#issuecomment-2702375800 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [PR] Spark : Derive Stats From Manifest on the Fly [iceberg]

2025-03-05 Thread via GitHub
github-actions[bot] closed pull request #11615: Spark : Derive Stats From Manifest on the Fly URL: https://github.com/apache/iceberg/pull/11615 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [PR] Spark : Derive Stats From Manifest on the Fly [iceberg]

2025-03-05 Thread via GitHub
github-actions[bot] commented on PR #11615: URL: https://github.com/apache/iceberg/pull/11615#issuecomment-2702375969 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [I] RewriteDataFiles maintenance action never converges [iceberg]

2025-03-05 Thread via GitHub
github-actions[bot] closed issue #6669: RewriteDataFiles maintenance action never converges URL: https://github.com/apache/iceberg/issues/6669 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-05 Thread via GitHub
hsiang-c commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1982341442 ## core/src/test/java/org/apache/iceberg/util/TestGeometryUtil.java: ## @@ -0,0 +1,145 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-05 Thread via GitHub
hsiang-c commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1982335637 ## core/src/main/java/org/apache/iceberg/util/GeometryUtil.java: ## @@ -0,0 +1,75 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more c

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-05 Thread via GitHub
hsiang-c commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1982335637 ## core/src/main/java/org/apache/iceberg/util/GeometryUtil.java: ## @@ -0,0 +1,75 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more c

[PR] Parquet: Support unknown and timestamp(9) in generics and internal model [iceberg]

2025-03-05 Thread via GitHub
rdblue opened a new pull request, #12463: URL: https://github.com/apache/iceberg/pull/12463 This adds Parquet readers and writers for unknown and nanosecond timestamp types. It supports Iceberg generics and the internal object model. Spark and Flink support will be added in following PRs.

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-05 Thread via GitHub
hsiang-c commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1982319469 ## api/src/main/java/org/apache/iceberg/types/Types.java: ## @@ -543,6 +561,120 @@ public int hashCode() { } } + public static class GeometryType extends

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-05 Thread via GitHub
hsiang-c commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1982315740 ## api/src/main/java/org/apache/iceberg/transforms/Identity.java: ## @@ -19,16 +19,21 @@ package org.apache.iceberg.transforms; import java.io.ObjectStreamExcept

Re: [I] [Feature] Add Support for Distributed Write [iceberg-python]

2025-03-05 Thread via GitHub
andormarkus commented on issue #1751: URL: https://github.com/apache/iceberg-python/issues/1751#issuecomment-2702284211 Hi @Fokko Thank you soo much for the code snippet. I have extended the test and run into the following problem with partitioned tables (non partitioned tabl

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-05 Thread via GitHub
hsiang-c commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1982300109 ## api/src/main/java/org/apache/iceberg/types/Types.java: ## @@ -70,6 +74,20 @@ public static Type fromTypeName(String typeString) { return TYPES.get(lowerTyp

Re: [PR] Parquet: Implement Variant writers [iceberg]

2025-03-05 Thread via GitHub
rdblue commented on PR #12323: URL: https://github.com/apache/iceberg/pull/12323#issuecomment-2702257228 Thanks for the reviews, @danielcweeks and @aihuaxu! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[PR] Build: Bump jinja2 from 3.1.5 to 3.1.6 [iceberg-python]

2025-03-05 Thread via GitHub
dependabot[bot] opened a new pull request, #1769: URL: https://github.com/apache/iceberg-python/pull/1769 Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.5 to 3.1.6. Release notes Sourced from https://github.com/pallets/jinja/releases";>jinja2's releases. 3.1.6 T

Re: [PR] Parquet: Implement Variant writers [iceberg]

2025-03-05 Thread via GitHub
danielcweeks merged PR #12323: URL: https://github.com/apache/iceberg/pull/12323 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceb

Re: [PR] Build: Bump getdaft from 0.4.4 to 0.4.6 [iceberg-python]

2025-03-05 Thread via GitHub
Fokko commented on PR #1758: URL: https://github.com/apache/iceberg-python/pull/1758#issuecomment-2702117438 Nice, thanks for linking that @kevinjqliu! It would be nice if it also includes https://github.com/Eventual-Inc/Daft/pull/3917 😏 -- This is an automated message from the Apache Gi

Re: [PR] support create table like in flink catalog [iceberg]

2025-03-05 Thread via GitHub
stevenzwu commented on PR #12199: URL: https://github.com/apache/iceberg/pull/12199#issuecomment-2702112881 thanks @swapna267 for the contribution and @pvary for the review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[PR] Make records purely position based [iceberg-python]

2025-03-05 Thread via GitHub
Fokko opened a new pull request, #1768: URL: https://github.com/apache/iceberg-python/pull/1768 This aligns the implementation with Java. We had the keywords there mostly for the tests, but they should not be used, and it seems like that's already the case :'( I was undecided if the c

Re: [I] unable to install pyicerberg [iceberg-python]

2025-03-05 Thread via GitHub
bhargav-bayer9 commented on issue #1763: URL: https://github.com/apache/iceberg-python/issues/1763#issuecomment-2702031811 Hi @Fokko . This message is seen when I am importing pyiceberg in my AWS Lambda. I created a pyiceberg layer for Lambda to access the pakage. Below is the error messag

Re: [I] unable to install pyicerberg [iceberg-python]

2025-03-05 Thread via GitHub
Fokko commented on issue #1763: URL: https://github.com/apache/iceberg-python/issues/1763#issuecomment-2702023660 Hey @bhargav-bayer9 Can you share the full stack trace? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [I] Delete Files in Table Scans [iceberg-rust]

2025-03-05 Thread via GitHub
sdd commented on issue #630: URL: https://github.com/apache/iceberg-rust/issues/630#issuecomment-2702008341 Update: The implementation is ready for review, split across a series of PRs: * https://github.com/apache/iceberg-rust/pull/652 * https://github.com/apache/iceberg-rust/pull/9

Re: [PR] Scan Delete Support Part 2: introduce `DeleteFileManager` skeleton. Use in `ArrowReader` [iceberg-rust]

2025-03-05 Thread via GitHub
sdd commented on code in PR #950: URL: https://github.com/apache/iceberg-rust/pull/950#discussion_r1982076981 ## crates/iceberg/src/arrow/reader.rs: ## @@ -162,30 +165,24 @@ impl ArrowReader { file_io: FileIO, row_group_filtering_enabled: bool, row_sel

Re: [PR] support create table like in flink catalog [iceberg]

2025-03-05 Thread via GitHub
stevenzwu merged PR #12199: URL: https://github.com/apache/iceberg/pull/12199 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [I] Broken build: Transitive dependency `twox-hash` is failing the security_audit step [iceberg-rust]

2025-03-05 Thread via GitHub
sdd closed issue #1041: Broken build: Transitive dependency `twox-hash` is failing the security_audit step URL: https://github.com/apache/iceberg-rust/issues/1041 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [I] Broken build: Transitive dependency `twox-hash` is failing the security_audit step [iceberg-rust]

2025-03-05 Thread via GitHub
sdd commented on issue #1041: URL: https://github.com/apache/iceberg-rust/issues/1041#issuecomment-2701894382 Looks to be fixed for me: https://github.com/apache/iceberg-rust/actions/runs/13684039718/job/38263063603?pr=950 Thanks @xxchan! -- This is an automated message from the Ap

Re: [PR] Scan Delete Support Part 2: introduce `DeleteFileManager` skeleton. Use in `ArrowReader` [iceberg-rust]

2025-03-05 Thread via GitHub
sdd commented on code in PR #950: URL: https://github.com/apache/iceberg-rust/pull/950#discussion_r1982076981 ## crates/iceberg/src/arrow/reader.rs: ## @@ -162,30 +165,24 @@ impl ArrowReader { file_io: FileIO, row_group_filtering_enabled: bool, row_sel

[PR] feat(table): make Table aware of its catalog [iceberg-go]

2025-03-05 Thread via GitHub
zeroshade opened a new pull request, #323: URL: https://github.com/apache/iceberg-go/pull/323 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

Re: [PR] Wrap variant in PrimitiveHoder so serialization can result same instance [iceberg]

2025-03-05 Thread via GitHub
aihuaxu commented on PR #12317: URL: https://github.com/apache/iceberg/pull/12317#issuecomment-2701622145 Thanks @flyrain -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] Core: Provide access to format-version of metadata table [iceberg]

2025-03-05 Thread via GitHub
nastra commented on code in PR #12462: URL: https://github.com/apache/iceberg/pull/12462#discussion_r1981853741 ## core/src/main/java/org/apache/iceberg/SerializableTable.java: ## @@ -178,10 +178,9 @@ public int formatVersion() { } private int formatVersion(Table table)

[PR] refactor: move Difference to internal [iceberg-go]

2025-03-05 Thread via GitHub
zeroshade opened a new pull request, #322: URL: https://github.com/apache/iceberg-go/pull/322 Let's not expose this helper function, keep it as an internal util so we don't have it as part of the public API -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [PR] Core: Provide access to format-version of metadata table [iceberg]

2025-03-05 Thread via GitHub
nastra commented on code in PR #12462: URL: https://github.com/apache/iceberg/pull/12462#discussion_r1981853741 ## core/src/main/java/org/apache/iceberg/SerializableTable.java: ## @@ -178,10 +178,9 @@ public int formatVersion() { } private int formatVersion(Table table)

Re: [I] unable to install pyicerberg [iceberg-python]

2025-03-05 Thread via GitHub
bhargav-bayer9 commented on issue #1763: URL: https://github.com/apache/iceberg-python/issues/1763#issuecomment-2701601502 I am sorry to extend this. and have very less time to build my module. Any idea why I am getting this error when I use pyiceberg in my Lambda function ' No module name

[PR] Core: Provide access to format-version of metadata table [iceberg]

2025-03-05 Thread via GitHub
nastra opened a new pull request, #12462: URL: https://github.com/apache/iceberg/pull/12462 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

Re: [PR] Core: Extended header support for RESTClient implementations [iceberg]

2025-03-05 Thread via GitHub
gaborkaszab commented on PR #12194: URL: https://github.com/apache/iceberg/pull/12194#issuecomment-2701567647 Thanks for taking a look, @1raghavmahajan! This is the proposal doc the freshness-aware table loading just in case: https://docs.google.com/document/d/1rnVSP_iv2I47giwfAe-Z3DY

[PR] feat(internal): Adding BinPacker [iceberg-go]

2025-03-05 Thread via GitHub
zeroshade opened a new pull request, #321: URL: https://github.com/apache/iceberg-go/pull/321 Needed for adding manifest and data file writers -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[PR] feat(table): add GetType for requirements [iceberg-go]

2025-03-05 Thread via GitHub
zeroshade opened a new pull request, #320: URL: https://github.com/apache/iceberg-go/pull/320 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

Re: [PR] Wrap variant in PrimitiveHoder so serialization can result same instance [iceberg]

2025-03-05 Thread via GitHub
flyrain commented on PR #12317: URL: https://github.com/apache/iceberg/pull/12317#issuecomment-2701552364 Some Flink after-merge [pipeline failed](https://github.com/apache/iceberg/actions/runs/13670147899/job/38218648591), but it looks like not related. Error messages: ``` Caused

Re: [PR] Core: Ensure current and newly added view versions are retained in ViewMetadata build [iceberg]

2025-03-05 Thread via GitHub
amogh-jahagirdar commented on code in PR #12401: URL: https://github.com/apache/iceberg/pull/12401#discussion_r1981745209 ## core/src/main/java/org/apache/iceberg/view/ViewMetadata.java: ## @@ -462,9 +463,18 @@ public ViewMetadata build() { ViewProperties.VERSION_HIST

Re: [I] unable to install pyicerberg [iceberg-python]

2025-03-05 Thread via GitHub
kevinjqliu commented on issue #1763: URL: https://github.com/apache/iceberg-python/issues/1763#issuecomment-2701436273 great! closing this issue :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [I] unable to install pyicerberg [iceberg-python]

2025-03-05 Thread via GitHub
kevinjqliu closed issue #1763: unable to install pyicerberg URL: https://github.com/apache/iceberg-python/issues/1763 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

Re: [PR] Spark: Rewrite V2 deletes to V3 DVs [iceberg]

2025-03-05 Thread via GitHub
danielcweeks commented on code in PR #12250: URL: https://github.com/apache/iceberg/pull/12250#discussion_r1981751859 ## core/src/main/java/org/apache/iceberg/TableUtil.java: ## @@ -30,6 +30,9 @@ public static int formatVersion(Table table) { if (table instanceof Serializab

Re: [I] unable to install pyicerberg [iceberg-python]

2025-03-05 Thread via GitHub
bhargav-bayer9 commented on issue #1763: URL: https://github.com/apache/iceberg-python/issues/1763#issuecomment-2701392349 Yeah, I reached out to my networks team and they resolved the issue. Now I am able to install the package using proxy -- This is an automated message from the Apache

Re: [PR] Core: Ensure current and newly added view versions are retained in ViewMetadata build [iceberg]

2025-03-05 Thread via GitHub
amogh-jahagirdar merged PR #12401: URL: https://github.com/apache/iceberg/pull/12401 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [I] ViewMetadataBuilder does not always retain changes in current build [iceberg]

2025-03-05 Thread via GitHub
amogh-jahagirdar closed issue #12113: ViewMetadataBuilder does not always retain changes in current build URL: https://github.com/apache/iceberg/issues/12113 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [PR] Core: Ensure current and newly added view versions are retained in ViewMetadata build [iceberg]

2025-03-05 Thread via GitHub
amogh-jahagirdar commented on PR #12401: URL: https://github.com/apache/iceberg/pull/12401#issuecomment-2701426771 Thanks @lliangyu-lin for the fix, @nastra @gaborkaszab for reviewing. I'll go ahead and merge! -- This is an automated message from the Apache Git Service. To respond to the

Re: [I] unable to install pyicerberg [iceberg-python]

2025-03-05 Thread via GitHub
kevinjqliu commented on issue #1763: URL: https://github.com/apache/iceberg-python/issues/1763#issuecomment-2701353499 hey @bhargav-bayer9 this seems like an issue with the python/pip setup and not related to pyiceberg. Specifically, > Failed to establish a new connection: [Errno 11

Re: [I] Column projection: filling in default values is broken for tables that don't have exactly 1 row [iceberg-python]

2025-03-05 Thread via GitHub
kevinjqliu closed issue #1766: Column projection: filling in default values is broken for tables that don't have exactly 1 row URL: https://github.com/apache/iceberg-python/issues/1766 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [I] Column projection: filling in default values is broken for tables that don't have exactly 1 row [iceberg-python]

2025-03-05 Thread via GitHub
kevinjqliu closed issue #1766: Column projection: filling in default values is broken for tables that don't have exactly 1 row URL: https://github.com/apache/iceberg-python/issues/1766 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [PR] Inject columns of proper length for missing fields [iceberg-python]

2025-03-05 Thread via GitHub
kevinjqliu merged PR #1767: URL: https://github.com/apache/iceberg-python/pull/1767 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

Re: [PR] Scan Delete Support Part 2: introduce `DeleteFileManager` skeleton. Use in `ArrowReader` [iceberg-rust]

2025-03-05 Thread via GitHub
Fokko commented on code in PR #950: URL: https://github.com/apache/iceberg-rust/pull/950#discussion_r1980293409 ## crates/iceberg/src/arrow/reader.rs: ## @@ -162,30 +165,24 @@ impl ArrowReader { file_io: FileIO, row_group_filtering_enabled: bool, row_s

Re: [I] S3FileIO does not raise exception when failing to drop/purge data [iceberg]

2025-03-05 Thread via GitHub
steveloughran commented on issue #12420: URL: https://github.com/apache/iceberg/issues/12420#issuecomment-2701093250 AWS SDK team do not test the SDK with third party stores, it is left to downstream projects. S3a project does has a [qualifying an SDK update](https://hadoop.apache.org/docs/

Re: [PR] Issue 1045 - Allow downcasting of parquet timestamp[ns] to timestamp[us] for Table.add_files() [iceberg-python]

2025-03-05 Thread via GitHub
lloyd-EA closed pull request #1569: Issue 1045 - Allow downcasting of parquet timestamp[ns] to timestamp[us] for Table.add_files() URL: https://github.com/apache/iceberg-python/pull/1569 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

Re: [PR] Auth Manager API part 6: API enablement [iceberg]

2025-03-05 Thread via GitHub
adutra commented on code in PR #12197: URL: https://github.com/apache/iceberg/pull/12197#discussion_r1981429970 ## aws/src/main/java/org/apache/iceberg/aws/s3/signer/S3V4RestSignerClient.java: ## @@ -138,148 +133,42 @@ boolean keepTokenRefreshed() { OAuth2Properties.TOK

Re: [PR] Expose ref_name parameter for table scans [iceberg-python]

2025-03-05 Thread via GitHub
b-phi commented on code in PR #1765: URL: https://github.com/apache/iceberg-python/pull/1765#discussion_r1981389638 ## pyiceberg/table/__init__.py: ## @@ -965,6 +969,10 @@ def scan( options=options, limit=limit, ) +if ref_name is not No

Re: [PR] Core: Extended header support for RESTClient implementations [iceberg]

2025-03-05 Thread via GitHub
1raghavmahajan commented on PR #12194: URL: https://github.com/apache/iceberg/pull/12194#issuecomment-2700949695 To make things simpler from a user perspective we could also add a default [CachedHttpClient](https://github.com/apache/httpcomponents-client/blob/37c80064726f0e4ae547c14ad7312948

[PR] HIVE-28801 Iceberg: Refactor HMS table parameter setting to be able to reuse [iceberg]

2025-03-05 Thread via GitHub
zratkai opened a new pull request, #12461: URL: https://github.com/apache/iceberg/pull/12461 HIVE-28801 Iceberg: Refactor HMS table parameter setting to be able to reuse -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Core: Ensure current and newly added view versions are retained in ViewMetadata build [iceberg]

2025-03-05 Thread via GitHub
gaborkaszab commented on code in PR #12401: URL: https://github.com/apache/iceberg/pull/12401#discussion_r1981338121 ## core/src/main/java/org/apache/iceberg/view/ViewMetadata.java: ## @@ -462,9 +463,18 @@ public ViewMetadata build() { ViewProperties.VERSION_HISTORY_S

Re: [I] Broken build: Transitive dependency `twox-hash` is failing the security_audit step [iceberg-rust]

2025-03-05 Thread via GitHub
Xuanwo commented on issue #1041: URL: https://github.com/apache/iceberg-rust/issues/1041#issuecomment-2700794759 Should be fixed by https://github.com/apache/iceberg-rust/pull/1042 I guess. -- This is an automated message from the Apache Git Service. To respond to the message, please log o

Re: [PR] fix(ci): fix audit break due to toolchain [iceberg-rust]

2025-03-05 Thread via GitHub
Xuanwo merged PR #1042: URL: https://github.com/apache/iceberg-rust/pull/1042 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] feat: make output file name of write task consistent with java api [iceberg-python]

2025-03-05 Thread via GitHub
sharkdtu commented on PR #1720: URL: https://github.com/apache/iceberg-python/pull/1720#issuecomment-2700025292 > @sharkdtu Thanks for the added context. Still, I don't think this is the right place to add this. > > Would each of the Ray workers call `_dataframe_to_data_files`? In the

Re: [I] Issue during Upsert [iceberg-python]

2025-03-05 Thread via GitHub
heman026 commented on issue #1759: URL: https://github.com/apache/iceberg-python/issues/1759#issuecomment-2700632648 > can you share more on what code you ran? specifically what is `table`, `pyarrow_table`, and `join_cols` ``` iceberg_table = 'test.table1' table = catalog.load_

Re: [I] Incorrect Output from `list_namespaces` in `SqlCatalog` [iceberg-python]

2025-03-05 Thread via GitHub
Fokko closed issue #1627: Incorrect Output from `list_namespaces` in `SqlCatalog` URL: https://github.com/apache/iceberg-python/issues/1627 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] Core: Ensure current and newly added view versions are retained in ViewMetadata build [iceberg]

2025-03-05 Thread via GitHub
nastra commented on code in PR #12401: URL: https://github.com/apache/iceberg/pull/12401#discussion_r1981161877 ## core/src/main/java/org/apache/iceberg/view/ViewMetadata.java: ## @@ -462,9 +463,18 @@ public ViewMetadata build() { ViewProperties.VERSION_HISTORY_SIZE,

Re: [PR] Core: Ensure current and newly added view versions are retained in ViewMetadata build [iceberg]

2025-03-05 Thread via GitHub
amogh-jahagirdar commented on code in PR #12401: URL: https://github.com/apache/iceberg/pull/12401#discussion_r1981126359 ## core/src/main/java/org/apache/iceberg/view/ViewMetadata.java: ## @@ -462,9 +463,18 @@ public ViewMetadata build() { ViewProperties.VERSION_HIST

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-05 Thread via GitHub
Kontinuation commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1981048435 ## api/src/main/java/org/apache/iceberg/types/TypeUtil.java: ## @@ -535,6 +535,8 @@ private static int estimateSize(Type type) { return ((Types.FixedTyp

Re: [PR] feat: add construct_ref for table_metadata [iceberg-rust]

2025-03-05 Thread via GitHub
ZENOTME commented on PR #1043: URL: https://github.com/apache/iceberg-rust/pull/1043#issuecomment-2700349589 cc @liurenjie1024 @Xuanwo @Fokko -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[PR] feat: add construct_ref for table_metadata [iceberg-rust]

2025-03-05 Thread via GitHub
ZENOTME opened a new pull request, #1043: URL: https://github.com/apache/iceberg-rust/pull/1043 ## Which issue does this PR close? ## What changes are included in this PR? This PR add a construct_ref to insert the main branch if it's not found in refs. ref f

Re: [PR] position_deletes metadata table [iceberg-python]

2025-03-05 Thread via GitHub
amitgilad3 commented on PR #1615: URL: https://github.com/apache/iceberg-python/pull/1615#issuecomment-2700156660 Ready for review :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] fix(ci): fix audit break due to toolchain [iceberg-rust]

2025-03-05 Thread via GitHub
ZENOTME commented on PR #1042: URL: https://github.com/apache/iceberg-rust/pull/1042#issuecomment-2700334028 Thanks @xxchan for this fix! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] Flink: Dynamic Iceberg Sink Contribution [iceberg]

2025-03-05 Thread via GitHub
mxm commented on code in PR #12424: URL: https://github.com/apache/iceberg/pull/12424#discussion_r1980978055 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/PartitionSpecAdjustment.java: ## @@ -0,0 +1,45 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-05 Thread via GitHub
szehon-ho commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1980949031 ## api/src/main/java/org/apache/iceberg/types/TypeUtil.java: ## @@ -535,6 +535,8 @@ private static int estimateSize(Type type) { return ((Types.FixedType)

Re: [I] Flaky Spark tests due to initializationError [iceberg]

2025-03-05 Thread via GitHub
nastra commented on issue #12009: URL: https://github.com/apache/iceberg/issues/12009#issuecomment-2700244737 This is probably because https://github.com/apache/iceberg/blob/a52afdc484f9e63477b96d0182586692daac8eab/spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestBaseWithCatalog.j

Re: [I] Incorrect Output from `list_namespaces` in `SqlCatalog` [iceberg-python]

2025-03-05 Thread via GitHub
Fokko closed issue #1627: Incorrect Output from `list_namespaces` in `SqlCatalog` URL: https://github.com/apache/iceberg-python/issues/1627 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif