Re: [I] Add support to FileIO for Azure and GCS [iceberg-go]

xuhui-lu commented on issue #65: URL: https://github.com/apache/iceberg-go/issues/65#issuecomment-2633144713 I saw the azure is not added yet. Could I have a try? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Core: Add KLL Datasketch and Hive ColumnStatisticsObj as standard blo… [iceberg]

simhadri-g commented on PR #8202: URL: https://github.com/apache/iceberg/pull/8202#issuecomment-2633115925 thanks @nastra and @deniskuzZ ! I will resolve the merge conflicts and update this PR. :) -- This is an automated message from the Apache Git Service. To respond to the me

Re: [PR] API: Null check for auto-unboxed field id in builder [iceberg]

nastra commented on code in PR #12165: URL: https://github.com/apache/iceberg/pull/12165#discussion_r1940646332 ## api/src/test/java/org/apache/iceberg/types/TestTypes.java: ## @@ -47,4 +49,15 @@ public void fromPrimitiveString() { .isThrownBy(() -> Types.fromPrimitiveS

Re: [PR] Core: Add KLL Datasketch and Hive ColumnStatisticsObj as standard blo… [iceberg]

deniskuzZ commented on PR #8202: URL: https://github.com/apache/iceberg/pull/8202#issuecomment-2633109513 thanks @nastra! I would really help if we could get this in. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [PR] API: Null check for auto-unboxed field id in builder [iceberg]

nastra commented on code in PR #12165: URL: https://github.com/apache/iceberg/pull/12165#discussion_r1940646018 ## api/src/test/java/org/apache/iceberg/types/TestTypes.java: ## @@ -47,4 +49,15 @@ public void fromPrimitiveString() { .isThrownBy(() -> Types.fromPrimitiveS

Re: [PR] API: Null check for auto-unboxed field id in builder [iceberg]

nastra commented on code in PR #12165: URL: https://github.com/apache/iceberg/pull/12165#discussion_r1940645769 ## api/src/main/java/org/apache/iceberg/types/Types.java: ## @@ -609,7 +609,8 @@ public Builder withWriteDefault(Object fieldWriteDefault) { } public N

Re: [PR] Core: Add KLL Datasketch and Hive ColumnStatisticsObj as standard blo… [iceberg]

deniskuzZ commented on PR #8202: URL: https://github.com/apache/iceberg/pull/8202#issuecomment-2633097073 > Hi Denys! I think , I don't have the github permissions to reopen a closed PR in iceberg repo. Should i raise a new PR? hey @simhadri-g, if you'd like to pursue this I would rec

Re: [PR] Core: Add KLL Datasketch and Hive ColumnStatisticsObj as standard blo… [iceberg]

simhadri-g commented on PR #8202: URL: https://github.com/apache/iceberg/pull/8202#issuecomment-2633089866 Hi Denys! I think i don't have permission to reopen a closed PR in iceberg. Should i raise a new PR? -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Fix NOTICE and LICENSE in the spark-runtime jar [iceberg]

jbonofre commented on code in PR #12160: URL: https://github.com/apache/iceberg/pull/12160#discussion_r1940626997 ## spark/v3.5/spark-runtime/LICENSE: ## @@ -379,14 +334,6 @@ License: http://www.apache.org/licenses/LICENSE-2.0

Re: [PR] Fix NOTICE and LICENSE in the spark-runtime jar [iceberg]

jbonofre commented on code in PR #12160: URL: https://github.com/apache/iceberg/pull/12160#discussion_r1939846155 ## spark/v3.5/spark-runtime/LICENSE: ## @@ -379,14 +334,6 @@ License: http://www.apache.org/licenses/LICENSE-2.0

Re: [PR] chore: use shared containers for integration tests [iceberg-rust]

gruuya commented on PR #924: URL: https://github.com/apache/iceberg-rust/pull/924#issuecomment-2633053094 Hey @ZENOTME @Fokko, can you also take a look at this proposal? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [I] PartitionStatsUtil#computeStats returns incomplete stats in case of partition evolution [iceberg]

deniskuzZ commented on issue #12168: URL: https://github.com/apache/iceberg/issues/12168#issuecomment-2633017046 https://github.com/apache/iceberg/pull/12137 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[I] PartitionStatsUtil#computeStats returns incomplete stats in case of partition evolution [iceberg]

deniskuzZ opened a new issue, #12168: URL: https://github.com/apache/iceberg/issues/12168 ### Apache Iceberg version main (development) ### Query engine None ### Please describe the bug 🐞 Description: PartitionMap creates internal wrappers based on table

Re: [PR] Make field repr explicit about optionality, add exception type [iceberg-cpp]

Fokko commented on code in PR #43: URL: https://github.com/apache/iceberg-cpp/pull/43#discussion_r1940581911 ## src/iceberg/schema_field.cc: ## @@ -52,8 +52,8 @@ const std::shared_ptr& SchemaField::type() const { return type_; } bool SchemaField::optional() const { return opti

Re: [PR] Add data type/schema field/schema [iceberg-cpp]

Fokko commented on code in PR #31: URL: https://github.com/apache/iceberg-cpp/pull/31#discussion_r1940580970 ## src/iceberg/type.cc: ## @@ -0,0 +1,312 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTIC

Re: [PR] Add data type/schema field/schema [iceberg-cpp]

Fokko commented on code in PR #31: URL: https://github.com/apache/iceberg-cpp/pull/31#discussion_r1940578441 ## src/iceberg/type.h: ## @@ -0,0 +1,411 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE

Re: [PR] Data: Add partition stats writer and reader [iceberg]

ajantha-bhat commented on code in PR #11216: URL: https://github.com/apache/iceberg/pull/11216#discussion_r1940558710 ## core/src/test/java/org/apache/iceberg/TestTables.java: ## @@ -93,6 +93,26 @@ public static TestTable create( return new TestTable(ops, name, reporter);

Re: [PR] Spec: Update partition stats for V3 [iceberg]

aokolnychyi commented on code in PR #12098: URL: https://github.com/apache/iceberg/pull/12098#discussion_r1940551771 ## format/spec.md: ## @@ -927,20 +927,21 @@ These rows must be sorted (in ascending manner with NULL FIRST) by `partition` f The schema of the partition stati

Re: [PR] Add data type/schema field/schema [iceberg-cpp]

lidavidm commented on code in PR #31: URL: https://github.com/apache/iceberg-cpp/pull/31#discussion_r1940550748 ## src/iceberg/type.cc: ## @@ -0,0 +1,312 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NO

Re: [PR] Data: Add partition stats writer and reader [iceberg]

ajantha-bhat commented on PR #11216: URL: https://github.com/apache/iceberg/pull/11216#issuecomment-2632944341 thanks @pvary for the review. I have addressed the comments except (https://github.com/apache/iceberg/pull/11216#discussion_r1939040414), It is because I wanted to keep the sig

Re: [PR] Add data type/schema field/schema [iceberg-cpp]

Fokko commented on code in PR #31: URL: https://github.com/apache/iceberg-cpp/pull/31#discussion_r1940544261 ## src/iceberg/type.cc: ## @@ -0,0 +1,312 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTIC

Re: [PR] Data: Add partition stats writer and reader [iceberg]

ajantha-bhat commented on code in PR #11216: URL: https://github.com/apache/iceberg/pull/11216#discussion_r1940544107 ## data/src/main/java/org/apache/iceberg/data/PartitionStatsHandler.java: ## @@ -0,0 +1,273 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] Data: Add partition stats writer and reader [iceberg]

ajantha-bhat commented on code in PR #11216: URL: https://github.com/apache/iceberg/pull/11216#discussion_r1940542661 ## data/src/test/java/org/apache/iceberg/data/TestPartitionStatsHandler.java: ## @@ -0,0 +1,582 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] Data: Add partition stats writer and reader [iceberg]

ajantha-bhat commented on code in PR #11216: URL: https://github.com/apache/iceberg/pull/11216#discussion_r1940542661 ## data/src/test/java/org/apache/iceberg/data/TestPartitionStatsHandler.java: ## @@ -0,0 +1,582 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] Implement column projection [iceberg-python]

Fokko commented on code in PR #1443: URL: https://github.com/apache/iceberg-python/pull/1443#discussion_r1940542160 ## pyiceberg/io/pyarrow.py: ## @@ -1216,6 +1218,45 @@ def _field_id(self, field: pa.Field) -> int: return -1 +def _get_column_projection_values( +

Re: [PR] Data: Add partition stats writer and reader [iceberg]

ajantha-bhat commented on code in PR #11216: URL: https://github.com/apache/iceberg/pull/11216#discussion_r1940541813 ## data/src/main/java/org/apache/iceberg/data/PartitionStatsHandler.java: ## @@ -0,0 +1,273 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] Data: Add partition stats writer and reader [iceberg]

ajantha-bhat commented on code in PR #11216: URL: https://github.com/apache/iceberg/pull/11216#discussion_r1940535942 ## data/src/main/java/org/apache/iceberg/data/PartitionStatsHandler.java: ## @@ -0,0 +1,273 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] Data: Add partition stats writer and reader [iceberg]

ajantha-bhat commented on code in PR #11216: URL: https://github.com/apache/iceberg/pull/11216#discussion_r1940531115 ## data/src/main/java/org/apache/iceberg/data/PartitionStatsHandler.java: ## @@ -0,0 +1,273 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] Data: Add partition stats writer and reader [iceberg]

ajantha-bhat commented on code in PR #11216: URL: https://github.com/apache/iceberg/pull/11216#discussion_r1940531115 ## data/src/main/java/org/apache/iceberg/data/PartitionStatsHandler.java: ## @@ -0,0 +1,273 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] Reduce code duplication in VectorizedParquetDefinitionLevelReader [iceberg]

wypoon commented on PR #11661: URL: https://github.com/apache/iceberg/pull/11661#issuecomment-2632893053 @nastra any chance of moving this forward? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Data: Add partition stats writer and reader [iceberg]

ajantha-bhat commented on code in PR #11216: URL: https://github.com/apache/iceberg/pull/11216#discussion_r1940512306 ## data/src/main/java/org/apache/iceberg/data/PartitionStatsHandler.java: ## @@ -0,0 +1,273 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] Data: Add partition stats writer and reader [iceberg]

ajantha-bhat commented on code in PR #11216: URL: https://github.com/apache/iceberg/pull/11216#discussion_r1940510235 ## data/src/main/java/org/apache/iceberg/data/PartitionStatsHandler.java: ## @@ -0,0 +1,273 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] Data: Add partition stats writer and reader [iceberg]

ajantha-bhat commented on code in PR #11216: URL: https://github.com/apache/iceberg/pull/11216#discussion_r1940509287 ## core/src/main/java/org/apache/iceberg/PartitionStatsUtil.java: ## @@ -133,4 +134,27 @@ private static Collection mergeStats( return statsMap.values();

Re: [PR] Data: Add partition stats writer and reader [iceberg]

ajantha-bhat commented on code in PR #11216: URL: https://github.com/apache/iceberg/pull/11216#discussion_r1940508650 ## data/src/test/java/org/apache/iceberg/data/TestPartitionStatsHandler.java: ## @@ -0,0 +1,582 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] Kafka Connect: Add kerberos authentication option [iceberg]

Dawnpool commented on PR #12119: URL: https://github.com/apache/iceberg/pull/12119#issuecomment-2632874357 Thanks @bryanck Could you also mention someone who can review this feature and share some thoughts if you know anyone? -- This is an automated message from the Apache Git Service

Re: [PR] Spec: Update partition stats for V3 [iceberg]

ajantha-bhat commented on code in PR #12098: URL: https://github.com/apache/iceberg/pull/12098#discussion_r1940462763 ## format/spec.md: ## @@ -927,20 +927,21 @@ These rows must be sorted (in ascending manner with NULL FIRST) by `partition` f The schema of the partition stat

Re: [PR] Spec: Update partition stats for V3 [iceberg]

ajantha-bhat commented on code in PR #12098: URL: https://github.com/apache/iceberg/pull/12098#discussion_r1940460701 ## format/spec.md: ## @@ -927,20 +927,21 @@ These rows must be sorted (in ascending manner with NULL FIRST) by `partition` f The schema of the partition stat

Re: [PR] Spec: Update partition stats for V3 [iceberg]

aokolnychyi commented on code in PR #12098: URL: https://github.com/apache/iceberg/pull/12098#discussion_r1940457899 ## format/spec.md: ## @@ -927,20 +927,21 @@ These rows must be sorted (in ascending manner with NULL FIRST) by `partition` f The schema of the partition stati

Re: [PR] Spec: Update partition stats for V3 [iceberg]

aokolnychyi commented on code in PR #12098: URL: https://github.com/apache/iceberg/pull/12098#discussion_r1940457488 ## format/spec.md: ## @@ -927,20 +927,21 @@ These rows must be sorted (in ascending manner with NULL FIRST) by `partition` f The schema of the partition stati

Re: [PR] Spec: Update partition stats for V3 [iceberg]

aokolnychyi commented on code in PR #12098: URL: https://github.com/apache/iceberg/pull/12098#discussion_r1940456356 ## format/spec.md: ## @@ -927,20 +927,21 @@ These rows must be sorted (in ascending manner with NULL FIRST) by `partition` f The schema of the partition stati

Re: [PR] Core, Spark: Refactor FileRewriter interface to separate planning and execution [iceberg]

stevenzwu commented on code in PR #11513: URL: https://github.com/apache/iceberg/pull/11513#discussion_r1934818356 ## core/src/main/java/org/apache/iceberg/actions/RewriteFilePlan.java: ## @@ -0,0 +1,47 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] Core: Refactor variants to enable moving interfaces to API module [iceberg]

amogh-jahagirdar commented on code in PR #12167: URL: https://github.com/apache/iceberg/pull/12167#discussion_r1940434089 ## core/src/main/java/org/apache/iceberg/variants/VariantMetadata.java: ## @@ -34,4 +35,12 @@ public interface VariantMetadata extends Variants.Serialized {

Re: [PR] Add data type/schema field/schema [iceberg-cpp]

lidavidm commented on code in PR #31: URL: https://github.com/apache/iceberg-cpp/pull/31#discussion_r1940417078 ## src/iceberg/type.cc: ## @@ -0,0 +1,312 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NO

Re: [I] Kafka Connect Sporadic Commit Delay [iceberg]

fenil25 commented on issue #11796: URL: https://github.com/apache/iceberg/issues/11796#issuecomment-2632703032 Hey 👋 I am encountering something similar. Did you figure this out? Thank you! 🙇 -- This is an automated message from the Apache Git Service. To respond to the message, pleas

[I] Snowflake managed Open Catalog and Azure ADLS2 [iceberg-python]

martinseifertprojuventute opened a new issue, #1606: URL: https://github.com/apache/iceberg-python/issues/1606 ### Apache Iceberg version 0.8.1 (latest release) ### Please describe the bug 🐞 I have an external volume in Snowflake pointing to an Azure ADLS2: ```

[PR] try datafusion 45 rc1 [iceberg-rust]

kevinjqliu opened a new pull request, #943: URL: https://github.com/apache/iceberg-rust/pull/943 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

Re: [PR] feat: `SnapshotSummary` outline [iceberg-rust]

kevinjqliu commented on PR #939: URL: https://github.com/apache/iceberg-rust/pull/939#issuecomment-2632559861 From [CI ](https://github.com/apache/iceberg-rust/actions/runs/13121462209/job/36608163616?pr=939) ``` test test_scan_all_type ... FAILED failures: test_scan

Re: [PR] feat: `SnapshotSummary` outline [iceberg-rust]

jonathanc-n commented on PR #939: URL: https://github.com/apache/iceberg-rust/pull/939#issuecomment-2632534043 Is someone able to take a look at why the integration tests are failing? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[PR] Core: Refactor variants to enable moving interfaces to API module [iceberg]

rdblue opened a new pull request, #12167: URL: https://github.com/apache/iceberg/pull/12167 This refactors the existing Variant class structure to avoid breaking changes when the Variant interfaces are moved to core. This requries: * `PhysicalType` should not be a static class in `Var

Re: [PR] backport c++23 std::expected [iceberg-cpp]

lidavidm commented on code in PR #40: URL: https://github.com/apache/iceberg-cpp/pull/40#discussion_r1940318960 ## src/iceberg/expected.h: ## @@ -0,0 +1,2333 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See th

Re: [I] write.wap.enabled / spark.wap.branch / spark.wap.id behavior isn't really documented [iceberg]

rodrigc commented on issue #11528: URL: https://github.com/apache/iceberg/issues/11528#issuecomment-2632499136 @wimlewis-amazon my understanding is that: 1. `write.wap.enabled` is a table property, and must be set on the table in order to do WAP 2. `spark.wap.branch` is a SQL conf pr

Re: [PR] Add data type/schema field/schema [iceberg-cpp]

lidavidm commented on code in PR #31: URL: https://github.com/apache/iceberg-cpp/pull/31#discussion_r1940311010 ## src/iceberg/type.cc: ## @@ -0,0 +1,312 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NO

Re: [PR] Add data type/schema field/schema [iceberg-cpp]

lidavidm commented on PR #31: URL: https://github.com/apache/iceberg-cpp/pull/31#issuecomment-2632488206 Thanks @Fokko I'll follow up to correct those things -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] Add data type/schema field/schema [iceberg-cpp]

lidavidm commented on code in PR #31: URL: https://github.com/apache/iceberg-cpp/pull/31#discussion_r1940311892 ## src/iceberg/type.cc: ## @@ -0,0 +1,312 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NO

Re: [PR] Core: Remove `TableMetadata::Builder::resetMainBranch` [iceberg]

amogh-jahagirdar merged PR #12149: URL: https://github.com/apache/iceberg/pull/12149 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] Spark: Support singular form of years, months, days, and hours functions [iceberg]

wypoon commented on code in PR #12117: URL: https://github.com/apache/iceberg/pull/12117#discussion_r1940307458 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/sql/TestSparkMonthsFunction.java: ## @@ -76,13 +85,34 @@ public void testTimestampNtz() { assertThat(sc

Re: [PR] Spark: Support singular form of years, months, days, and hours functions [iceberg]

wypoon commented on code in PR #12117: URL: https://github.com/apache/iceberg/pull/12117#discussion_r1940307001 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/sql/TestSparkDaysFunction.java: ## @@ -76,12 +85,32 @@ public void testTimestampNtz() { assertThat(scal

Re: [PR] Flink: Add null check to writers to prevent resurrecting null values [iceberg]

stevenzwu commented on code in PR #12049: URL: https://github.com/apache/iceberg/pull/12049#discussion_r1940304992 ## data/src/test/java/org/apache/iceberg/data/DataTest.java: ## @@ -59,6 +63,10 @@ protected boolean supportsDefaultValues() { return false; } + protecte

Re: [PR] Flink: Add null check to writers to prevent resurrecting null values [iceberg]

stevenzwu commented on code in PR #12049: URL: https://github.com/apache/iceberg/pull/12049#discussion_r1940303839 ## data/src/test/java/org/apache/iceberg/data/orc/TestGenericData.java: ## @@ -54,13 +54,23 @@ public class TestGenericData extends DataTest { + @Override +

Re: [I] Parquet write option 'write.parquet.row-group-limit' issue [iceberg-python]

github-actions[bot] commented on issue #1012: URL: https://github.com/apache/iceberg-python/issues/1012#issuecomment-2632470282 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity

Re: [PR] Flink: Add null check to writers to prevent resurrecting null values [iceberg]

stevenzwu commented on code in PR #12049: URL: https://github.com/apache/iceberg/pull/12049#discussion_r1940302515 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/FlinkRowData.java: ## @@ -0,0 +1,42 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under on

Re: [I] write.wap.enabled / spark.wap.branch / spark.wap.id behavior isn't really documented [iceberg]

wimlewis-amazon commented on issue #11528: URL: https://github.com/apache/iceberg/issues/11528#issuecomment-2632467756 Hmm, from those code snippets, it looks like `enabled` is a _table_ property which enables the behavior, and `branch` is a _spark session_ property that indicates which bra

Re: [I] Some schema updates do not support dots inside a field name [iceberg]

github-actions[bot] commented on issue #10875: URL: https://github.com/apache/iceberg/issues/10875#issuecomment-2632467092 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Core: check catalog and schema for JdbcCatalog initialization [iceberg]

github-actions[bot] closed pull request #11864: Core: check catalog and schema for JdbcCatalog initialization URL: https://github.com/apache/iceberg/pull/11864 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Core: check catalog and schema for JdbcCatalog initialization [iceberg]

github-actions[bot] commented on PR #11864: URL: https://github.com/apache/iceberg/pull/11864#issuecomment-2632467251 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [I] MERGE INTO requires sorting in already sorted iceberg tables [iceberg]

github-actions[bot] commented on issue #10891: URL: https://github.com/apache/iceberg/issues/10891#issuecomment-2632467131 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] Rest catalog: write.metadata.delete-after-commit set true not deleting expired metadata files [iceberg]

github-actions[bot] commented on issue #10894: URL: https://github.com/apache/iceberg/issues/10894#issuecomment-2632467155 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Flink: Add null check to writers to prevent resurrecting null values [iceberg]

stevenzwu commented on code in PR #12049: URL: https://github.com/apache/iceberg/pull/12049#discussion_r1940300922 ## data/src/test/java/org/apache/iceberg/data/DataTest.java: ## @@ -59,6 +63,10 @@ protected boolean supportsDefaultValues() { return false; } + protecte

Re: [I] Will there be any problems migrating a hive table with 3 million partitions to Iceberg [iceberg]

github-actions[bot] commented on issue #10768: URL: https://github.com/apache/iceberg/issues/10768#issuecomment-2632467005 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] uppercase table name not supported [iceberg]

github-actions[bot] commented on issue #10758: URL: https://github.com/apache/iceberg/issues/10758#issuecomment-2632466940 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] Will there be any problems migrating a hive table with 3 million partitions to Iceberg [iceberg]

github-actions[bot] closed issue #10768: Will there be any problems migrating a hive table with 3 million partitions to Iceberg URL: https://github.com/apache/iceberg/issues/10768 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [I] Support for Default Values [iceberg]

github-actions[bot] closed issue #10761: Support for Default Values URL: https://github.com/apache/iceberg/issues/10761 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

Re: [I] Support for Default Values [iceberg]

github-actions[bot] commented on issue #10761: URL: https://github.com/apache/iceberg/issues/10761#issuecomment-2632466964 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] uppercase table name not supported [iceberg]

github-actions[bot] closed issue #10758: uppercase table name not supported URL: https://github.com/apache/iceberg/issues/10758 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] MergeSchema doesn't work if missing columns is used for Write Ordering. [iceberg]

github-actions[bot] commented on issue #10751: URL: https://github.com/apache/iceberg/issues/10751#issuecomment-2632466901 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] MergeSchema doesn't work if missing columns is used for Write Ordering. [iceberg]

github-actions[bot] closed issue #10751: MergeSchema doesn't work if missing columns is used for Write Ordering. URL: https://github.com/apache/iceberg/issues/10751 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] `add_files` procedure allows importing NULL on NOT NULL columns [iceberg]

github-actions[bot] commented on issue #10742: URL: https://github.com/apache/iceberg/issues/10742#issuecomment-2632466865 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] `add_files` procedure allows importing NULL on NOT NULL columns [iceberg]

github-actions[bot] closed issue #10742: `add_files` procedure allows importing NULL on NOT NULL columns URL: https://github.com/apache/iceberg/issues/10742 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] Apache Flink not committing new snapshots to Iceberg Table [iceberg]

github-actions[bot] commented on issue #9089: URL: https://github.com/apache/iceberg/issues/9089#issuecomment-2632466694 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Spark: Support singular form of years, months, days, and hours functions [iceberg]

wypoon commented on code in PR #12117: URL: https://github.com/apache/iceberg/pull/12117#discussion_r1940296342 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/sql/TestSparkHoursFunction.java: ## @@ -61,13 +68,28 @@ public void testTimestampsNtz() { assertThat(sc

Re: [PR] Build: Bump mkdocstrings from 0.27.0 to 0.28.0 [iceberg-python]

kevinjqliu commented on PR #1604: URL: https://github.com/apache/iceberg-python/pull/1604#issuecomment-2632454664 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Build: Bump mkdocs-material from 9.6.1 to 9.6.2 [iceberg-python]

kevinjqliu merged PR #1605: URL: https://github.com/apache/iceberg-python/pull/1605 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

Re: [PR] Spec: Support geo type [iceberg]

szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1940286775 ## format/spec.md: ## @@ -1480,6 +1497,9 @@ This serialization scheme is for storing single values as individual binary valu | **`struct`** | Not

Re: [PR] Spec: Support geo type [iceberg]

szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1940286775 ## format/spec.md: ## @@ -1480,6 +1497,9 @@ This serialization scheme is for storing single values as individual binary valu | **`struct`** | Not

Re: [PR] Spark: Support singular form of years, months, days, and hours functions [iceberg]

wypoon commented on code in PR #12117: URL: https://github.com/apache/iceberg/pull/12117#discussion_r1940285154 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkV2Filters.java: ## @@ -70,7 +70,8 @@ public class SparkV2Filters { public static final Set SUPPOR

Re: [PR] Spec: Support geo type [iceberg]

rdblue commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1940273461 ## format/spec.md: ## @@ -1480,6 +1497,9 @@ This serialization scheme is for storing single values as individual binary valu | **`struct`** | Not sup

Re: [PR] Spec: Support geo type [iceberg]

rdblue commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1940272745 ## format/spec.md: ## @@ -205,15 +205,40 @@ Supported primitive types are defined in the table below. Primitive types added | | **`uuid`** |

Re: [PR] Spec additions for encryption [iceberg]

rdblue commented on code in PR #12162: URL: https://github.com/apache/iceberg/pull/12162#discussion_r1940256313 ## format/spec.md: ## @@ -889,6 +892,9 @@ Table metadata consists of the following fields: | _optional_ | _optional_ | _optional_ | **`partition-statistics`** | A li

Re: [PR] Spec additions for encryption [iceberg]

rdblue commented on code in PR #12162: URL: https://github.com/apache/iceberg/pull/12162#discussion_r1940256987 ## format/spec.md: ## @@ -685,6 +686,8 @@ A snapshot consists of the following fields: | _optional_ | _optional_ | _optional_ | **`schema-id`** | ID of

[I] Java doc link is not working [iceberg]

ranup opened a new issue, #12166: URL: https://github.com/apache/iceberg/issues/12166 https://iceberg.apache.org/javadoc/nightly linked in iceberg doc main page https://iceberg.apache.org/docs/nightly/ is not working. Seeing below error Not Found The requested URL was not

Re: [PR] Build: Bump moto from 5.0.27 to 5.0.28 [iceberg-python]

kevinjqliu merged PR #1603: URL: https://github.com/apache/iceberg-python/pull/1603 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

tscottcoombes1 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1940230399 ## tests/table/test_merge_rows.py: ## @@ -0,0 +1,397 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agree

[PR] Build: Bump mkdocs-material from 9.6.1 to 9.6.2 [iceberg-python]

dependabot[bot] opened a new pull request, #1605: URL: https://github.com/apache/iceberg-python/pull/1605 Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.6.1 to 9.6.2. Release notes Sourced from https://github.com/squidfunk/mkdocs-material/releases";>mk

[PR] Build: Bump mkdocstrings from 0.27.0 to 0.28.0 [iceberg-python]

dependabot[bot] opened a new pull request, #1604: URL: https://github.com/apache/iceberg-python/pull/1604 Bumps [mkdocstrings](https://github.com/mkdocstrings/mkdocstrings) from 0.27.0 to 0.28.0. Release notes Sourced from https://github.com/mkdocstrings/mkdocstrings/releases";>mkd

[PR] Build: Bump moto from 5.0.27 to 5.0.28 [iceberg-python]

dependabot[bot] opened a new pull request, #1603: URL: https://github.com/apache/iceberg-python/pull/1603 Bumps [moto](https://github.com/getmoto/moto) from 5.0.27 to 5.0.28. Changelog Sourced from https://github.com/getmoto/moto/blob/master/CHANGELOG.md";>moto's changelog.

Re: [PR] Spec: Support geo type [iceberg]

jiayuasu commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1940158718 ## format/spec.md: ## @@ -1480,6 +1497,9 @@ This serialization scheme is for storing single values as individual binary valu | **`struct`** | Not s

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

tscottcoombes1 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1940112272 ## pyiceberg/table/__init__.py: ## @@ -1064,6 +1065,110 @@ def name_mapping(self) -> Optional[NameMapping]: """Return the table's field-id NameMa

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

tscottcoombes1 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1940141552 ## pyproject.toml: ## @@ -842,6 +851,5026 @@ ignore_missing_imports = true module = "daft.*" ignore_missing_imports = true +[[tool.mypy.overrides]] +m

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

tscottcoombes1 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1940129806 ## tests/table/test_merge_rows.py: ## @@ -0,0 +1,397 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agree

Re: [I] Create table format version constants [iceberg-python]

iyad-f commented on issue #851: URL: https://github.com/apache/iceberg-python/issues/851#issuecomment-263242 just a quick search for format_version in the library tells me it would need to be changed almost everywhere from tests to actual library code. If you have decided on whether to

  1   2   3   >