Re: [I] Broken build: Transitive dependency `twox-hash` is failing the security_audit step [iceberg-rust]

2025-03-04 Thread via GitHub
sdd commented on issue #1041: URL: https://github.com/apache/iceberg-rust/issues/1041#issuecomment-2700142051 Looks like rustsec/audit-check or cargo-audit depend on twox-hash 2.1.0. I'm unable to look at this now until this evening - if someone else wants to pick it up before then, be my g

Re: [I] Broken build: Transitive dependency `twox-hash` is failing the security_audit step [iceberg-rust]

2025-03-04 Thread via GitHub
sdd commented on issue #1041: URL: https://github.com/apache/iceberg-rust/issues/1041#issuecomment-2700129905 Oh wait, it's the `cargo install cargo-audit` step that fails. We just happen to depend on twox-hash as well. -- This is an automated message from the Apache Git Service. To respo

Re: [PR] Fix: `SqlCatalog` list_namespaces() should return only sub-namespaces [iceberg-python]

2025-03-04 Thread via GitHub
Fokko merged PR #1629: URL: https://github.com/apache/iceberg-python/pull/1629 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Expose ref_name parameter for table scans [iceberg-python]

2025-03-04 Thread via GitHub
Fokko commented on code in PR #1765: URL: https://github.com/apache/iceberg-python/pull/1765#discussion_r1980868016 ## pyiceberg/table/__init__.py: ## @@ -965,6 +969,10 @@ def scan( options=options, limit=limit, ) +if ref_name is not No

Re: [I] Broken build: Transitive dependency `twox-hash` is failing the security_audit step [iceberg-rust]

2025-03-04 Thread via GitHub
sdd commented on issue #1041: URL: https://github.com/apache/iceberg-rust/issues/1041#issuecomment-2700121993 I'm unsure why this is happening, since we have twox-hash v1.6.3 in our Cargo.lock but cargo-audit is erroring on twox-hash v2.1.0 -- This is an automated message from the Apache

[I] Column projection: filling in default values is broken for tables that don't have exactly 1 row [iceberg-python]

2025-03-04 Thread via GitHub
lidavidm opened a new issue, #1766: URL: https://github.com/apache/iceberg-python/issues/1766 ### Apache Iceberg version main (development) ### Please describe the bug 🐞 **This happens on pyiceberg 0.9.0** https://github.com/apache/iceberg-python/blob/e3a5c3b9bd4a

[I] Broken build: Transitive dependency `twox-hash` is failing the security_audit step [iceberg-rust]

2025-03-04 Thread via GitHub
sdd opened a new issue, #1041: URL: https://github.com/apache/iceberg-rust/issues/1041 ### Apache Iceberg Rust version None ### Describe the bug https://github.com/user-attachments/assets/9d9903dd-d58b-4da8-80eb-8b6950e12013"; /> See https://github.com/apache/ice

Re: [PR] Wrap variant in PrimitiveHoder so serialization can result same instance [iceberg]

2025-03-04 Thread via GitHub
flyrain commented on PR #12317: URL: https://github.com/apache/iceberg/pull/12317#issuecomment-2700023335 Thanks @aihuaxu for the PR. Thanks @XBaith for the review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Partition Writer Support Part 1: add partition splitter [iceberg-rust]

2025-03-04 Thread via GitHub
ZENOTME commented on PR #1040: URL: https://github.com/apache/iceberg-rust/pull/1040#issuecomment-2700083148 This PR may conflict with #1014. But I'm not sure whether #1014 can be reviewed and merge recently. I'm ok to merge any one first and I will fix the conflict later. cc @liurenjie1024

Re: [PR] ci: use officail rustsec/audit-check action [iceberg-rust]

2025-03-04 Thread via GitHub
ZENOTME commented on PR #843: URL: https://github.com/apache/iceberg-rust/pull/843#issuecomment-2700081365 It looks like in the method, we can't control the version of cargo-audit and it will upgrade the recent version automatically. When our compile version is outdated, the ci will fail.🥵

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-04 Thread via GitHub
Kontinuation commented on PR #12346: URL: https://github.com/apache/iceberg/pull/12346#issuecomment-2700076284 I have removed dependency to JTS from iceberg-api, now the underlying Java type of Geometry and Geography are `ByteBuffer`. Now JTS is a implementation dependency of iceberg-core.

[PR] Partition Writer Support Part 1: add partition splitter [iceberg-rust]

2025-03-04 Thread via GitHub
ZENOTME opened a new pull request, #1040: URL: https://github.com/apache/iceberg-rust/pull/1040 ## Which issue does this PR close? This PR is part 1 to close #342. ## What changes are included in this PR? The partition writer support will be separate into

Re: [PR] Wrap variant in PrimitiveHoder so serialization can result same instance [iceberg]

2025-03-04 Thread via GitHub
flyrain merged PR #12317: URL: https://github.com/apache/iceberg/pull/12317 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [I] Update multiple rows of Iceberg Table using table.overwrite method [iceberg-python]

2025-03-04 Thread via GitHub
heman026 commented on issue #1760: URL: https://github.com/apache/iceberg-python/issues/1760#issuecomment-2699786840 Will check Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [I] discuss: Split iceberg-rust high level API into a mini engine instead [iceberg-rust]

2025-03-04 Thread via GitHub
Xuanwo commented on issue #1036: URL: https://github.com/apache/iceberg-rust/issues/1036#issuecomment-279899 Thank you all for the discussion. It seems we have a consensus that it's beneficial to have such an engine. I will start working on this. -- This is an automated message from t

Re: [PR] Add unit test for AddFilesProcedure to check invalid column in partition filter [iceberg]

2025-03-04 Thread via GitHub
bharos commented on code in PR #12456: URL: https://github.com/apache/iceberg/pull/12456#discussion_r1980770878 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestAddFilesProcedure.java: ## @@ -830,6 +830,44 @@ public void invalidDataImportParti

[PR] Avoid Avro recursive schema for Variant schema. [iceberg]

2025-03-04 Thread via GitHub
aihuaxu opened a new pull request, #12459: URL: https://github.com/apache/iceberg/pull/12459 Minor change to follow up https://github.com/apache/iceberg/pull/12238: - Detect Avro recursive schema definition for Variant schema in AvroSchemaVisitor and AvroCustomOrderSchemaVisitor.

Re: [PR] Add unit test for AddFilesProcedure to check invalid column in partition filter [iceberg]

2025-03-04 Thread via GitHub
ebyhr commented on code in PR #12456: URL: https://github.com/apache/iceberg/pull/12456#discussion_r1980741814 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestAddFilesProcedure.java: ## @@ -830,6 +830,44 @@ public void invalidDataImportPartit

[I] Spark returns incorrect results when reading Parquet bloom filters created by Trino [iceberg]

2025-03-04 Thread via GitHub
ebyhr opened a new issue, #12458: URL: https://github.com/apache/iceberg/issues/12458 ### Apache Iceberg version 1.8.1 (latest release) ### Query engine Spark ### Please describe the bug 🐞 Trino ```sql CREATE TABLE test (testinteger INTEGER, testlong B

Re: [PR] feat: Add unknown type [iceberg-python]

2025-03-04 Thread via GitHub
kaushiksrini commented on PR #1681: URL: https://github.com/apache/iceberg-python/pull/1681#issuecomment-2699807325 thanks @Fokko! Made the changes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-04 Thread via GitHub
Kontinuation commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1980666883 ## api/src/main/java/org/apache/iceberg/expressions/Literal.java: ## @@ -71,6 +73,14 @@ static Literal of(BigDecimal value) { return new Literals.DecimalLit

Re: [I] Update multiple rows of Iceberg Table using table.overwrite method [iceberg-python]

2025-03-04 Thread via GitHub
heman026 closed issue #1760: Update multiple rows of Iceberg Table using table.overwrite method URL: https://github.com/apache/iceberg-python/issues/1760 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-04 Thread via GitHub
Kontinuation commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1980664840 ## api/src/main/java/org/apache/iceberg/Geography.java: ## @@ -0,0 +1,127 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contr

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-04 Thread via GitHub
Kontinuation commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1980656298 ## core/src/main/java/org/apache/iceberg/SchemaParser.java: ## @@ -141,11 +143,34 @@ static void toJson(Types.MapType map, JsonGenerator generator) throws IOExc

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-04 Thread via GitHub
Kontinuation commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1980662689 ## api/src/main/java/org/apache/iceberg/types/TypeUtil.java: ## @@ -535,6 +535,8 @@ private static int estimateSize(Type type) { return ((Types.FixedTyp

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-04 Thread via GitHub
Kontinuation commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1980660047 ## core/src/main/java/org/apache/iceberg/SchemaParser.java: ## @@ -141,11 +143,34 @@ static void toJson(Types.MapType map, JsonGenerator generator) throws IOExc

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-04 Thread via GitHub
Kontinuation commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1980659522 ## api/src/main/java/org/apache/iceberg/types/Types.java: ## @@ -543,6 +562,123 @@ public int hashCode() { } } + public static class GeometryType exte

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-04 Thread via GitHub
Kontinuation commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1980658315 ## api/src/main/java/org/apache/iceberg/types/Types.java: ## @@ -70,6 +75,20 @@ public static Type fromTypeName(String typeString) { return TYPES.get(lowe

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-04 Thread via GitHub
Kontinuation commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1980656298 ## core/src/main/java/org/apache/iceberg/SchemaParser.java: ## @@ -141,11 +143,34 @@ static void toJson(Types.MapType map, JsonGenerator generator) throws IOExc

Re: [I] unable to install pyicerberg [iceberg-python]

2025-03-04 Thread via GitHub
bhargav-bayer9 commented on issue #1763: URL: https://github.com/apache/iceberg-python/issues/1763#issuecomment-2699749664 Thank you @CincyBC for the above. I downloaded the .tar.gaz file and tried installing, but no luck (iceberg_env) \pyiceberg_install>py -m pip install "\Download

Re: [I] unable to install pyicerberg [iceberg-python]

2025-03-04 Thread via GitHub
CincyBC commented on issue #1763: URL: https://github.com/apache/iceberg-python/issues/1763#issuecomment-2699695295 @bhargav-bayer9 Your stack trace says you're having trouble reaching Pypi. Check out this [StackOverflow](https://stackoverflow.com/questions/49192842/error-to-install-packag

Re: [PR] Parquet: Implement Variant writers [iceberg]

2025-03-04 Thread via GitHub
rdblue commented on code in PR #12323: URL: https://github.com/apache/iceberg/pull/12323#discussion_r1980618098 ## parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java: ## @@ -160,9 +161,10 @@ public static class WriteBuilder implements InternalData.WriteBuilder {

Re: [PR] Retry on NoSuchNamespaceException not found in rename table for rest catalog [iceberg]

2025-03-04 Thread via GitHub
huan233usc commented on PR #12159: URL: https://github.com/apache/iceberg/pull/12159#issuecomment-2699673586 Done https://github.com/user-attachments/assets/a608dd59-e67c-4971-a66d-4bac3fe95fbb"; /> -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] Small docstring fix [iceberg-python]

2025-03-04 Thread via GitHub
Fokko merged PR #1762: URL: https://github.com/apache/iceberg-python/pull/1762 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [I] Add option to provide partition spec in spark ADD_FILES procedure [iceberg]

2025-03-04 Thread via GitHub
RussellSpitzer closed issue #12325: Add option to provide partition spec in spark ADD_FILES procedure URL: https://github.com/apache/iceberg/issues/12325 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Avro: Support timestamp(9) and unknown types [iceberg]

2025-03-04 Thread via GitHub
rdblue commented on PR #12455: URL: https://github.com/apache/iceberg/pull/12455#issuecomment-2699511325 Thanks for reviewing, @danielcweeks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Avro: Support timestamp(9) and unknown types [iceberg]

2025-03-04 Thread via GitHub
rdblue merged PR #12455: URL: https://github.com/apache/iceberg/pull/12455 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [I] unable to install pyicerberg [iceberg-python]

2025-03-04 Thread via GitHub
bhargav-bayer9 commented on issue #1763: URL: https://github.com/apache/iceberg-python/issues/1763#issuecomment-2699505801 Dear @Fokko , tried installing in python 3.9 and I am still getting the same error. below [are](url) the command prompt logs. C:\Users\user>cd \pyiceberg_install

[PR] Add unit test for AddFilesProcedure to check invalid column in partition filter [iceberg]

2025-03-04 Thread via GitHub
bharos opened a new pull request, #12456: URL: https://github.com/apache/iceberg/pull/12456 …ion filter -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-ma

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-04 Thread via GitHub
szehon-ho commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1980148439 ## build.gradle: ## @@ -292,6 +292,7 @@ project(':iceberg-api') { dependencies { implementation project(path: ':iceberg-bundled-guava', configuration: 's

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-04 Thread via GitHub
szehon-ho commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1980491237 ## api/src/main/java/org/apache/iceberg/types/Types.java: ## @@ -543,6 +562,123 @@ public int hashCode() { } } + public static class GeometryType extends

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-04 Thread via GitHub
szehon-ho commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1980149907 ## core/src/main/java/org/apache/iceberg/SchemaParser.java: ## @@ -141,11 +143,34 @@ static void toJson(Types.MapType map, JsonGenerator generator) throws IOExcept

Re: [PR] Spark: Infer partition spec in ADD_FILES procedure for FileTables than taking latest table spec [iceberg]

2025-03-04 Thread via GitHub
bharos commented on PR #12327: URL: https://github.com/apache/iceberg/pull/12327#issuecomment-2699333702 Thanks @RussellSpitzer for the review and guidance to impement this change -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] Flink: Replace use of deprecated methods [iceberg]

2025-03-04 Thread via GitHub
github-actions[bot] commented on PR #11658: URL: https://github.com/apache/iceberg/pull/11658#issuecomment-2699325135 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Adding new rewrite manifest spark action to accept custom partition o… [iceberg]

2025-03-04 Thread via GitHub
github-actions[bot] commented on PR #11881: URL: https://github.com/apache/iceberg/pull/11881#issuecomment-2699325202 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] deadlock when spark call delete row postition [iceberg]

2025-03-04 Thread via GitHub
github-actions[bot] commented on issue #10987: URL: https://github.com/apache/iceberg/issues/10987#issuecomment-2699325005 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] deadlock when spark call delete row postition [iceberg]

2025-03-04 Thread via GitHub
github-actions[bot] closed issue #10987: deadlock when spark call delete row postition URL: https://github.com/apache/iceberg/issues/10987 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Spark: Infer partition spec in ADD_FILES procedure for FileTables than taking latest table spec [iceberg]

2025-03-04 Thread via GitHub
RussellSpitzer commented on PR #12327: URL: https://github.com/apache/iceberg/pull/12327#issuecomment-2699316005 Thanks @bharos for the PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [PR] Spark: Infer partition spec in ADD_FILES procedure for FileTables than taking latest table spec [iceberg]

2025-03-04 Thread via GitHub
RussellSpitzer merged PR #12327: URL: https://github.com/apache/iceberg/pull/12327 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Avro: Support timestamp(9) and unknown types [iceberg]

2025-03-04 Thread via GitHub
rdblue commented on PR #12455: URL: https://github.com/apache/iceberg/pull/12455#issuecomment-2699284354 > The only thing I note is that we're changing the terminology a little (option -> optional). While I agree with the using `optional` as it is less confusing, we're slightly inconsistent

Re: [PR] Arrow: Infer the types when reading [iceberg-python]

2025-03-04 Thread via GitHub
Fokko commented on code in PR #1669: URL: https://github.com/apache/iceberg-python/pull/1669#discussion_r1980345139 ## pyiceberg/io/pyarrow.py: ## @@ -1655,19 +1646,16 @@ class ArrowProjectionVisitor(SchemaWithPartnerVisitor[pa.Array, Optional[pa.Arra _file_schema: Schema

Re: [PR] Core: Ensure current and newly added view versions are retained in ViewMetadata build [iceberg]

2025-03-04 Thread via GitHub
lliangyu-lin commented on PR #12401: URL: https://github.com/apache/iceberg/pull/12401#issuecomment-2699225101 cc: @Fokko @amogh-jahagirdar PR pending for final review and merge, could you help? -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] ci(golangci-lint) :Add golangci linter [iceberg-go]

2025-03-04 Thread via GitHub
zeroshade merged PR #315: URL: https://github.com/apache/iceberg-go/pull/315 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Small docstring fix [iceberg-python]

2025-03-04 Thread via GitHub
Fokko commented on PR #1762: URL: https://github.com/apache/iceberg-python/pull/1762#issuecomment-2699136255 Thanks @b-phi 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] support create table like in flink catalog [iceberg]

2025-03-04 Thread via GitHub
stevenzwu commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1980372419 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalog.java: ## @@ -332,7 +335,33 @@ public List listTables(String databaseName) public Catalog

Re: [PR] support create table like in flink catalog [iceberg]

2025-03-04 Thread via GitHub
stevenzwu commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1980369112 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalog.java: ## @@ -332,7 +335,33 @@ public List listTables(String databaseName) public Catalog

Re: [PR] Core: Add Variant logical type for Avro [iceberg]

2025-03-04 Thread via GitHub
aihuaxu commented on code in PR #12238: URL: https://github.com/apache/iceberg/pull/12238#discussion_r1980343887 ## core/src/test/java/org/apache/iceberg/avro/TestPruneColumns.java: ## @@ -0,0 +1,114 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

Re: [PR] CI: Limit parallelism [iceberg-python]

2025-03-04 Thread via GitHub
kevinjqliu merged PR #1764: URL: https://github.com/apache/iceberg-python/pull/1764 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

[PR] Avro: Support timestamp(9) and unknown types [iceberg]

2025-03-04 Thread via GitHub
rdblue opened a new pull request, #12455: URL: https://github.com/apache/iceberg/pull/12455 This updates Avro to support nanosecond timestamps and unknown. This also cleans up some of the tests: * Moves `DataTest` to core so it can be used for Avro, rather than duplicating in `Avro

[I] unable to install pyicerberg [iceberg-python]

2025-03-04 Thread via GitHub
bhargav-bayer9 opened a new issue, #1763: URL: https://github.com/apache/iceberg-python/issues/1763 ### Apache Iceberg version 0.8.0 ### Please describe the bug 🐞 I am trying to install pyiceberg using pip. pip install pyiceberg. But I get the error 'ERROR: Could not fin

Re: [PR] support create table like in flink catalog [iceberg]

2025-03-04 Thread via GitHub
stevenzwu commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1980347728 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalog.java: ## @@ -404,10 +437,15 @@ void createIcebergTable(ObjectPath tablePath, ResolvedCatal

Re: [PR] support create table like in flink catalog [iceberg]

2025-03-04 Thread via GitHub
stevenzwu commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1980349222 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/FlinkCreateTableOptions.java: ## @@ -0,0 +1,115 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] support create table like in flink catalog [iceberg]

2025-03-04 Thread via GitHub
stevenzwu commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1980345060 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalog.java: ## @@ -97,17 +97,20 @@ public class FlinkCatalog extends AbstractCatalog { private

Re: [I] unable to install pyicerberg [iceberg-python]

2025-03-04 Thread via GitHub
bhargav-bayer9 commented on issue #1763: URL: https://github.com/apache/iceberg-python/issues/1763#issuecomment-2699058797 Hey @Fokko . Thanks for the quick reply. Yeah, pip version is 24.x. Then which version of python suits better? I can see from PyPi python version >=3.9 is supported. C

Re: [PR] Spark-3.5: Add procedure to compute partition stats [iceberg]

2025-03-04 Thread via GitHub
singhpk234 commented on code in PR #12451: URL: https://github.com/apache/iceberg/pull/12451#discussion_r1980148259 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/ComputePartitionStatsSparkAction.java: ## @@ -0,0 +1,88 @@ +/* + * Licensed to the Apache Softwa

Re: [I] unable to install pyicerberg [iceberg-python]

2025-03-04 Thread via GitHub
bhargav-bayer9 commented on issue #1763: URL: https://github.com/apache/iceberg-python/issues/1763#issuecomment-2699095895 sure @Fokko . Let me try that and get back to you -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Flink: Dynamic Iceberg Sink Contribution [iceberg]

2025-03-04 Thread via GitHub
stevenzwu commented on code in PR #12424: URL: https://github.com/apache/iceberg/pull/12424#discussion_r1980330718 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/PartitionSpecAdjustment.java: ## @@ -0,0 +1,45 @@ +/* + * Licensed to the Apache Software F

Re: [PR] Arrow: Infer the types when reading [iceberg-python]

2025-03-04 Thread via GitHub
Fokko commented on code in PR #1669: URL: https://github.com/apache/iceberg-python/pull/1669#discussion_r1980331024 ## pyiceberg/io/pyarrow.py: ## @@ -1655,19 +1646,16 @@ class ArrowProjectionVisitor(SchemaWithPartnerVisitor[pa.Array, Optional[pa.Arra _file_schema: Schema

Re: [PR] ci(golangci-lint) :Add golangci linter [iceberg-go]

2025-03-04 Thread via GitHub
dttung2905 commented on PR #315: URL: https://github.com/apache/iceberg-go/pull/315#issuecomment-2699083863 Hi @zeroshade the conflicts have been resolved. Feel free to merge it when you have the time :pray: -- This is an automated message from the Apache Git Service. To respond to the m

[PR] CI: Limit parallelism [iceberg-python]

2025-03-04 Thread via GitHub
Fokko opened a new pull request, #1764: URL: https://github.com/apache/iceberg-python/pull/1764 For the tests, we want to limit parallelism to avoid creating 1-row Parquet files. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [I] unable to install pyicerberg [iceberg-python]

2025-03-04 Thread via GitHub
Fokko commented on issue #1763: URL: https://github.com/apache/iceberg-python/issues/1763#issuecomment-2699072219 @bhargav-bayer9 Can you try anything between 3.9 and 3.11? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Core: Ensure current and newly added view versions are retained in ViewMetadata build [iceberg]

2025-03-04 Thread via GitHub
lliangyu-lin commented on code in PR #12401: URL: https://github.com/apache/iceberg/pull/12401#discussion_r1980313203 ## core/src/main/java/org/apache/iceberg/view/ViewMetadata.java: ## @@ -462,9 +463,15 @@ public ViewMetadata build() { ViewProperties.VERSION_HISTORY_

Re: [I] unable to install pyicerberg [iceberg-python]

2025-03-04 Thread via GitHub
Fokko commented on issue #1763: URL: https://github.com/apache/iceberg-python/issues/1763#issuecomment-2699010785 Hey @bhargav-bayer9 Officially, we don't yet support 3.12, and we also don't publish any wheels for it. This means that it will try to compile it from the source. I assume the

Re: [PR] Auth Manager API part 6: API enablement [iceberg]

2025-03-04 Thread via GitHub
danielcweeks commented on code in PR #12197: URL: https://github.com/apache/iceberg/pull/12197#discussion_r1980175050 ## aws/src/main/java/org/apache/iceberg/aws/s3/signer/S3V4RestSignerClient.java: ## @@ -138,148 +133,42 @@ boolean keepTokenRefreshed() { OAuth2Properti

Re: [PR] Parquet: Implement Variant writers [iceberg]

2025-03-04 Thread via GitHub
rdblue commented on code in PR #12323: URL: https://github.com/apache/iceberg/pull/12323#discussion_r1980265501 ## parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java: ## @@ -160,9 +161,10 @@ public static class WriteBuilder implements InternalData.WriteBuilder {

Re: [PR] Scan Delete Support Part 3: `ArrowReader::build_deletes_row_selection` implementation [iceberg-rust]

2025-03-04 Thread via GitHub
jonathanc-n commented on code in PR #951: URL: https://github.com/apache/iceberg-rust/pull/951#discussion_r1980039486 ## crates/iceberg/src/arrow/reader.rs: ## @@ -265,6 +317,132 @@ impl ArrowReader { Ok(Box::pin(record_batch_stream) as ArrowRecordBatchStream) }

[PR] Small docstring fix [iceberg-python]

2025-03-04 Thread via GitHub
b-phi opened a new pull request, #1762: URL: https://github.com/apache/iceberg-python/pull/1762 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] Parquet: Implement Variant writers [iceberg]

2025-03-04 Thread via GitHub
rdblue commented on code in PR #12323: URL: https://github.com/apache/iceberg/pull/12323#discussion_r1980243819 ## parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java: ## @@ -160,9 +161,10 @@ public static class WriteBuilder implements InternalData.WriteBuilder {

Re: [PR] Core: Add Variant logical type for Avro [iceberg]

2025-03-04 Thread via GitHub
rdblue merged PR #12238: URL: https://github.com/apache/iceberg/pull/12238 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Core: Add Variant logical type for Avro [iceberg]

2025-03-04 Thread via GitHub
rdblue commented on PR #12238: URL: https://github.com/apache/iceberg/pull/12238#issuecomment-2698900687 Thanks for working on this, @aihuaxu! And thanks for reviewing, @XBaith! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Core: Add Variant logical type for Avro [iceberg]

2025-03-04 Thread via GitHub
rdblue commented on code in PR #12238: URL: https://github.com/apache/iceberg/pull/12238#discussion_r1980220343 ## core/src/test/java/org/apache/iceberg/avro/TestPruneColumns.java: ## @@ -0,0 +1,114 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mor

Re: [PR] Core: Add Variant logical type for Avro [iceberg]

2025-03-04 Thread via GitHub
aihuaxu commented on code in PR #12238: URL: https://github.com/apache/iceberg/pull/12238#discussion_r1979981102 ## core/src/main/java/org/apache/iceberg/avro/AvroCustomOrderSchemaVisitor.java: ## @@ -35,20 +38,29 @@ public static T visit(Schema schema, AvroCustomOrderSchemaVi

Re: [PR] Core: Add Variant logical type for Avro [iceberg]

2025-03-04 Thread via GitHub
rdblue commented on code in PR #12238: URL: https://github.com/apache/iceberg/pull/12238#discussion_r1980206785 ## core/src/main/java/org/apache/iceberg/avro/AvroSchemaVisitor.java: ## @@ -33,21 +36,30 @@ public static T visit(Schema schema, AvroSchemaVisitor visitor) {

Re: [PR] Parquet: Implement Variant writers [iceberg]

2025-03-04 Thread via GitHub
danielcweeks commented on code in PR #12323: URL: https://github.com/apache/iceberg/pull/12323#discussion_r1980214339 ## parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java: ## @@ -192,6 +194,11 @@ public WriteBuilder schema(Schema newSchema) { return this;

Re: [PR] Parquet: Implement Variant writers [iceberg]

2025-03-04 Thread via GitHub
danielcweeks commented on code in PR #12323: URL: https://github.com/apache/iceberg/pull/12323#discussion_r1980212065 ## parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java: ## @@ -160,9 +161,10 @@ public static class WriteBuilder implements InternalData.WriteBuilder

Re: [PR] Core: Add Variant logical type for Avro [iceberg]

2025-03-04 Thread via GitHub
rdblue commented on code in PR #12238: URL: https://github.com/apache/iceberg/pull/12238#discussion_r1980206785 ## core/src/main/java/org/apache/iceberg/avro/AvroSchemaVisitor.java: ## @@ -33,21 +36,30 @@ public static T visit(Schema schema, AvroSchemaVisitor visitor) {

Re: [PR] Core: Add Variant logical type for Avro [iceberg]

2025-03-04 Thread via GitHub
rdblue commented on code in PR #12238: URL: https://github.com/apache/iceberg/pull/12238#discussion_r1980203427 ## core/src/main/java/org/apache/iceberg/avro/AvroCustomOrderSchemaVisitor.java: ## @@ -35,20 +38,29 @@ public static T visit(Schema schema, AvroCustomOrderSchemaVis

Re: [PR] Spark-3.5: Add procedure to compute partition stats [iceberg]

2025-03-04 Thread via GitHub
singhpk234 commented on code in PR #12451: URL: https://github.com/apache/iceberg/pull/12451#discussion_r1980150585 ## api/src/main/java/org/apache/iceberg/actions/ComputePartitionStats.java: ## @@ -0,0 +1,40 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] Spark-3.5: Add procedure to compute partition stats [iceberg]

2025-03-04 Thread via GitHub
singhpk234 commented on code in PR #12451: URL: https://github.com/apache/iceberg/pull/12451#discussion_r1980148259 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/ComputePartitionStatsSparkAction.java: ## @@ -0,0 +1,88 @@ +/* + * Licensed to the Apache Softwa

Re: [PR] added more logs [iceberg]

2025-03-04 Thread via GitHub
fenil25 closed pull request #12454: added more logs URL: https://github.com/apache/iceberg/pull/12454 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: is

Re: [PR] Support Distinct Counts in Manifest [iceberg-python]

2025-03-04 Thread via GitHub
jpugliesi commented on PR #1613: URL: https://github.com/apache/iceberg-python/pull/1613#issuecomment-2698679289 All good, glad we've sorted this out. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [I] Can not read data when there is an required filed under an optional stuct [iceberg]

2025-03-04 Thread via GitHub
hangc0276 commented on issue #12441: URL: https://github.com/apache/iceberg/issues/12441#issuecomment-2698690088 Looks like this bug is introduce by https://github.com/apache/iceberg/pull/11785 -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Auth Manager API part 6: API enablement [iceberg]

2025-03-04 Thread via GitHub
danielcweeks commented on code in PR #12197: URL: https://github.com/apache/iceberg/pull/12197#discussion_r1979920103 ## aws/src/main/java/org/apache/iceberg/aws/s3/signer/S3V4RestSignerClient.java: ## @@ -81,13 +76,13 @@ public abstract class S3V4RestSignerClient private sta

Re: [PR] Core: Add Variant logical type for Avro [iceberg]

2025-03-04 Thread via GitHub
rdblue commented on code in PR #12238: URL: https://github.com/apache/iceberg/pull/12238#discussion_r1979942118 ## core/src/main/java/org/apache/iceberg/avro/AvroSchemaVisitor.java: ## @@ -33,21 +36,30 @@ public static T visit(Schema schema, AvroSchemaVisitor visitor) {

Re: [PR] Core: Add Variant logical type for Avro [iceberg]

2025-03-04 Thread via GitHub
rdblue commented on code in PR #12238: URL: https://github.com/apache/iceberg/pull/12238#discussion_r1979940954 ## core/src/main/java/org/apache/iceberg/avro/AvroCustomOrderSchemaVisitor.java: ## @@ -35,20 +38,29 @@ public static T visit(Schema schema, AvroCustomOrderSchemaVis

Re: [PR] Auth Manager API part 6: API enablement [iceberg]

2025-03-04 Thread via GitHub
adutra commented on code in PR #12197: URL: https://github.com/apache/iceberg/pull/12197#discussion_r1979951530 ## aws/src/main/java/org/apache/iceberg/aws/s3/signer/S3V4RestSignerClient.java: ## @@ -81,13 +76,13 @@ public abstract class S3V4RestSignerClient private static fi

Re: [I] How to get rid of the warning [iceberg-python]

2025-03-04 Thread via GitHub
lk-1984 commented on issue #1336: URL: https://github.com/apache/iceberg-python/issues/1336#issuecomment-2698443830 > yes those warning lines are removed in the new 0.9.0 release If you are still using 0.8.x, you can manually suppress those warnings https://docs.python.org/3/library/warnin

Re: [PR] Auth Manager API part 6: API enablement [iceberg]

2025-03-04 Thread via GitHub
danielcweeks commented on code in PR #12197: URL: https://github.com/apache/iceberg/pull/12197#discussion_r1979920103 ## aws/src/main/java/org/apache/iceberg/aws/s3/signer/S3V4RestSignerClient.java: ## @@ -81,13 +76,13 @@ public abstract class S3V4RestSignerClient private sta

Re: [PR] Auth Manager API part 6: API enablement [iceberg]

2025-03-04 Thread via GitHub
danielcweeks commented on code in PR #12197: URL: https://github.com/apache/iceberg/pull/12197#discussion_r1979889796 ## aws/src/main/java/org/apache/iceberg/aws/s3/signer/S3V4RestSignerClient.java: ## @@ -138,148 +133,42 @@ boolean keepTokenRefreshed() { OAuth2Properti

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-04 Thread via GitHub
szehon-ho commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1979873731 ## api/src/main/java/org/apache/iceberg/expressions/BoundLiteralPredicate.java: ## @@ -88,11 +91,64 @@ public boolean test(T value) { return String.valueOf

  1   2   >