[PR] Core: Add metadataFileLocation in TableUtil [iceberg]

dramaticlly opened a new pull request, #12082: URL: https://github.com/apache/iceberg/pull/12082 Context: https://github.com/apache/iceberg/pull/11931#discussion_r1927795585 - Add a new static helper method in TableUtil to expose metadata file location for a given table - also adde

Re: [PR] ci: add clang-tidy Checks [iceberg-cpp]

lidavidm commented on PR #32: URL: https://github.com/apache/iceberg-cpp/pull/32#issuecomment-2611877332 Ah, interesting. (I am used to the std::expected backport which doesn't do this, so I always put it by hand everywhere...) Anyways, I don't feel strongly about this. If you want to

Re: [PR] Core, Spark: Scan only live entries in RewriteTablePathUtil [iceberg]

szehon-ho commented on code in PR #12006: URL: https://github.com/apache/iceberg/pull/12006#discussion_r1928243065 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/actions/TestRewriteTablePathsAction.java: ## @@ -923,16 +1005,20 @@ protected void checkFileNum(

Re: [PR] ci: add clang-tidy Checks [iceberg-cpp]

wgtmac commented on PR #32: URL: https://github.com/apache/iceberg-cpp/pull/32#issuecomment-2611769691 > If we are going to use a Result/Status object, then I think we should have `[[nodiscard]]` enforcement. If we are going to use exceptions then I don't feel so strongly. Then we ca

Re: [PR] Spark 3.5: Procedure to rewrite table path [iceberg]

dramaticlly commented on code in PR #11931: URL: https://github.com/apache/iceberg/pull/11931#discussion_r1928176085 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRewriteTablePathProcedure.java: ## @@ -0,0 +1,174 @@ +/* + * Licensed to the

Re: [PR] Spark: Disable rewriting position deletes for V3 tables [iceberg]

nastra commented on code in PR #12048: URL: https://github.com/apache/iceberg/pull/12048#discussion_r1928166943 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/actions/TestRewritePositionDeleteFilesAction.java: ## @@ -739,23 +749,31 @@ private Table createTablePartiti

Re: [PR] ci: add clang-tidy Checks [iceberg-cpp]

lidavidm commented on PR #32: URL: https://github.com/apache/iceberg-cpp/pull/32#issuecomment-2611736128 If we are going to use a Result/Status object, then I think we should have `[[nodiscard]]` enforcement. If we are going to use exceptions then I don't feel so strongly. -- This is an

Re: [PR] Core, Spark: Scan only live entries in RewriteTablePathUtil [iceberg]

dramaticlly commented on PR #12006: URL: https://github.com/apache/iceberg/pull/12006#issuecomment-2611724920 > Yes, thanks for fixing the issue (found by our internal usage). > > I wonder, because the deleted entry may be important for CDC (to mark that this file at some point existe

Re: [PR] Spark: Disable rewriting position deletes for V3 tables [iceberg]

nastra commented on PR #12048: URL: https://github.com/apache/iceberg/pull/12048#issuecomment-2611729505 thanks for the reviews @RussellSpitzer @aokolnychyi. I'll go ahead and merge this for 1.8.0 since I don't have the PR ready yet where we would support minor compactions for DVs -- Th

Re: [PR] Core, Spark: Include content offset/size in PositionDeletesTable [iceberg]

nastra merged PR #11808: URL: https://github.com/apache/iceberg/pull/11808 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Core, Spark: Include content offset/size in PositionDeletesTable [iceberg]

nastra commented on PR #11808: URL: https://github.com/apache/iceberg/pull/11808#issuecomment-2611721970 thanks for the reviews @amogh-jahagirdar @aokolnychyi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [PR] Core, Spark: Rewrite data files with high delete ratio [iceberg]

nastra merged PR #11825: URL: https://github.com/apache/iceberg/pull/11825 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Core, Spark: Rewrite data files with high delete ratio [iceberg]

nastra commented on code in PR #11825: URL: https://github.com/apache/iceberg/pull/11825#discussion_r1928162975 ## core/src/main/java/org/apache/iceberg/actions/SizeBasedDataRewriter.java: ## @@ -45,6 +47,7 @@ public abstract class SizeBasedDataRewriter extends SizeBasedFileRew

[I] Make DELETE_RATIO_THRESHOLD configurable in SizeBasedDataRewriter [iceberg]

nastra opened a new issue, #12081: URL: https://github.com/apache/iceberg/issues/12081 ### Feature Request / Improvement The feature was introduced by https://github.com/apache/iceberg/pull/11825 and it would be good to make the ratio configurable in Spark ### Query engine

Re: [I] Add additional configurations for ApacheHttpClientBuilder [iceberg]

amogh-jahagirdar commented on issue #12077: URL: https://github.com/apache/iceberg/issues/12077#issuecomment-2611712610 I think this is AI generated (also referencing past user history/other issues such as https://github.com/apache/pinot/issues/14867). I'm going to close this. Apolog

Re: [I] Add additional configurations for ApacheHttpClientBuilder [iceberg]

amogh-jahagirdar closed issue #12077: Add additional configurations for ApacheHttpClientBuilder URL: https://github.com/apache/iceberg/issues/12077 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] Core, API, Spec: Metadata Row Lineage [iceberg]

amogh-jahagirdar commented on code in PR #11948: URL: https://github.com/apache/iceberg/pull/11948#discussion_r1928151867 ## core/src/main/java/org/apache/iceberg/SnapshotProducer.java: ## @@ -290,7 +298,27 @@ public Snapshot apply() { operation(), summary(base

Re: [PR] ci: add clang-tidy Checks [iceberg-cpp]

zhjwpku commented on PR #32: URL: https://github.com/apache/iceberg-cpp/pull/32#issuecomment-2611699882 > TBH, I'm not in favor of adding `[[nodiscard]]` every where. It is boring to type it for every function and make the function signature too verbose. How about only adding `[[nodiscard]]

Re: [I] Drop table failed when metadata.json file is missing [iceberg]

ebyhr commented on issue #12062: URL: https://github.com/apache/iceberg/issues/12062#issuecomment-2611698373 How about adding a new procedure that drops a table without "load"? Trino Iceberg connector has [unregister_table](https://trino.io/docs/current/connector/iceberg.html#unregister-

Re: [PR] Core, API, Spec: Metadata Row Lineage [iceberg]

amogh-jahagirdar commented on code in PR #11948: URL: https://github.com/apache/iceberg/pull/11948#discussion_r1928121891 ## core/src/main/java/org/apache/iceberg/SnapshotProducer.java: ## @@ -282,6 +283,13 @@ public Snapshot apply() { throw new RuntimeIOException(e, "Fai

Re: [PR] Core, API, Spec: Metadata Row Lineage [iceberg]

amogh-jahagirdar commented on code in PR #11948: URL: https://github.com/apache/iceberg/pull/11948#discussion_r1928121891 ## core/src/main/java/org/apache/iceberg/SnapshotProducer.java: ## @@ -282,6 +283,13 @@ public Snapshot apply() { throw new RuntimeIOException(e, "Fai

Re: [PR] Add data type/schema field/schema [iceberg-cpp]

wgtmac commented on code in PR #31: URL: https://github.com/apache/iceberg-cpp/pull/31#discussion_r1928124052 ## src/iceberg/schema_field.h: ## @@ -0,0 +1,87 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See th

Re: [PR] Core, API, Spec: Metadata Row Lineage [iceberg]

amogh-jahagirdar commented on code in PR #11948: URL: https://github.com/apache/iceberg/pull/11948#discussion_r1928099165 ## core/src/test/java/org/apache/iceberg/TestRowLineageMetadata.java: ## @@ -0,0 +1,328 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] ci: add clang-tidy Checks [iceberg-cpp]

wgtmac commented on PR #32: URL: https://github.com/apache/iceberg-cpp/pull/32#issuecomment-2611657586 TBH, I'm not in favor of adding `[[nodiscard]]` every where. It is boring to type it for every function and make the function signature too verbose. How about only adding `[[nodiscard]]` t

Re: [I] question on using iceberg with KMS through IAM role [iceberg]

joshua-yeung-mox commented on issue #3886: URL: https://github.com/apache/iceberg/issues/3886#issuecomment-2611649241 Hi @sgedward , how did you solve this issue? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] ci: add initial collaborators [iceberg-cpp]

wgtmac commented on PR #38: URL: https://github.com/apache/iceberg-cpp/pull/38#issuecomment-2611625868 Thank you @Xuanwo! cc @gaborkaszab @lidavidm @pitrou @raulcd @zhjwpku to get aware of this. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] ci: add initial collaborators [iceberg-cpp]

Xuanwo merged PR #38: URL: https://github.com/apache/iceberg-cpp/pull/38 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] ci: add initial collaborators [iceberg-cpp]

wgtmac commented on PR #38: URL: https://github.com/apache/iceberg-cpp/pull/38#issuecomment-2611621557 @Xuanwo @Fokko Does this make sense to you? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] refactor: fix a type in manifest_entries field name [iceberg-rust]

Xuanwo merged PR #911: URL: https://github.com/apache/iceberg-rust/pull/911 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] Spark 3.5: Fix Javadoc in ColumnarBatchUtil [iceberg]

huaxingao commented on PR #12058: URL: https://github.com/apache/iceberg/pull/12058#issuecomment-2611571288 Thanks @aokolnychyi @dramaticlly -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Core, Spark: Rewrite data files with high delete ratio [iceberg]

aokolnychyi commented on code in PR #11825: URL: https://github.com/apache/iceberg/pull/11825#discussion_r1928080074 ## core/src/main/java/org/apache/iceberg/actions/SizeBasedDataRewriter.java: ## @@ -45,6 +47,7 @@ public abstract class SizeBasedDataRewriter extends SizeBasedFi

Re: [PR] Spark 3.5: Make ColumnVectorWithFilter generic and refactor batch load [iceberg]

aokolnychyi commented on PR #12056: URL: https://github.com/apache/iceberg/pull/12056#issuecomment-2611561075 @nastra @Fokko, could you check, please? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Spark 3.4: Refactor delete logic in batch reading [iceberg]

aokolnychyi commented on PR #12061: URL: https://github.com/apache/iceberg/pull/12061#issuecomment-2611559995 @huaxingao, looks like there are some CI failures. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] Fix Reader leak by removing useless copy [iceberg]

aokolnychyi commented on PR #12079: URL: https://github.com/apache/iceberg/pull/12079#issuecomment-2611557916 cc @Fokko @amogh-jahagirdar @nastra @huaxingao -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Core: add variant type support [iceberg]

aihuaxu commented on code in PR #11831: URL: https://github.com/apache/iceberg/pull/11831#discussion_r1928072103 ## api/src/main/java/org/apache/iceberg/types/TypeUtil.java: ## @@ -709,6 +709,10 @@ public T map(Types.MapType map, Supplier keyResult, Supplier valueResult)

[PR] Fix Reader leak by removing useless copy [iceberg]

zizon opened a new pull request, #12079: URL: https://github.com/apache/iceberg/pull/12079 The ReadConf copy constructor will nullify the reader of source, leaving the reader of original unclosed -- This is an automated message from the Apache Git Service. To respond to the message, pleas

Re: [I] write.wap.enabled / spark.wap.branch behavior isn't really documented [iceberg]

rodrigc commented on issue #11528: URL: https://github.com/apache/iceberg/issues/11528#issuecomment-2611484268 I also did not fully understand `write.wap.enabled`. Looking at the code, it seems like an additional configuration knob that can turn off WAP even if `spark.wap.branch` is set

Re: [PR] Flink: Avoid RANGE mode broken chain when write parallelism changes [iceberg]

huyuanfeng2018 commented on PR #11702: URL: https://github.com/apache/iceberg/pull/11702#issuecomment-2611465925 Thanks @Stevenzwu Ping, I will open a PR later. ~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[I] set tblproperties, spark action expireSnapshots is not work. [iceberg]

cosen-wu opened a new issue, #12078: URL: https://github.com/apache/iceberg/issues/12078 ### Query engine hive,flink,spark ### Question > create iceberg table in hive: `create table test.iceberg_v1( a int, b string, c string, d string ) partitioned b

Re: [PR] Parquet: Add readers and writers for the internal object model [iceberg]

ajantha-bhat commented on code in PR #11904: URL: https://github.com/apache/iceberg/pull/11904#discussion_r1928003465 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetValueReaders.java: ## @@ -412,6 +446,41 @@ public byte[] read(byte[] ignored) { } } + priva

Re: [PR] Parquet: Add readers and writers for the internal object model [iceberg]

ajantha-bhat commented on code in PR #11904: URL: https://github.com/apache/iceberg/pull/11904#discussion_r1928001378 ## parquet/src/main/java/org/apache/iceberg/data/parquet/InternalReader.java: ## @@ -0,0 +1,88 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] Core: add variant builder implementation [iceberg]

rdblue commented on code in PR #11857: URL: https://github.com/apache/iceberg/pull/11857#discussion_r1927947934 ## core/src/main/java/org/apache/iceberg/variants/VariantBuilderBase.java: ## @@ -0,0 +1,504 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

Re: [PR] test: replace `assert!( == )` by `assert_eq!(, )` in some tests [iceberg-rust]

liurenjie1024 merged PR #910: URL: https://github.com/apache/iceberg-rust/pull/910 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Core: add variant builder implementation [iceberg]

rdblue commented on code in PR #11857: URL: https://github.com/apache/iceberg/pull/11857#discussion_r1927944759 ## core/src/main/java/org/apache/iceberg/variants/VariantBuilder.java: ## @@ -0,0 +1,178 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

Re: [PR] Core: add variant builder implementation [iceberg]

rdblue commented on code in PR #11857: URL: https://github.com/apache/iceberg/pull/11857#discussion_r1927947249 ## core/src/main/java/org/apache/iceberg/variants/VariantBuilderBase.java: ## @@ -0,0 +1,504 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

Re: [PR] Core: add variant builder implementation [iceberg]

rdblue commented on code in PR #11857: URL: https://github.com/apache/iceberg/pull/11857#discussion_r1927946229 ## core/src/main/java/org/apache/iceberg/variants/VariantBuilderBase.java: ## @@ -0,0 +1,504 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

Re: [PR] Core: add variant type support [iceberg]

rdblue commented on code in PR #11831: URL: https://github.com/apache/iceberg/pull/11831#discussion_r1927936907 ## core/src/test/java/org/apache/iceberg/avro/TestBuildAvroProjection.java: ## @@ -401,4 +402,32 @@ public void projectMapWithLessFieldInValueSchema() { .as("

Re: [PR] Core: add variant builder implementation [iceberg]

rdblue commented on code in PR #11857: URL: https://github.com/apache/iceberg/pull/11857#discussion_r1927943877 ## core/src/main/java/org/apache/iceberg/variants/VariantBuilder.java: ## @@ -0,0 +1,178 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

Re: [PR] Core: add variant builder implementation [iceberg]

rdblue commented on code in PR #11857: URL: https://github.com/apache/iceberg/pull/11857#discussion_r1927940347 ## core/src/main/java/org/apache/iceberg/variants/VariantConstants.java: ## @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] Core: add variant builder implementation [iceberg]

rdblue commented on code in PR #11857: URL: https://github.com/apache/iceberg/pull/11857#discussion_r1927940032 ## core/src/main/java/org/apache/iceberg/variants/VariantConstants.java: ## @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] Core: add variant builder implementation [iceberg]

rdblue commented on code in PR #11857: URL: https://github.com/apache/iceberg/pull/11857#discussion_r1927939333 ## core/src/main/java/org/apache/iceberg/variants/VariantImpl.java: ## @@ -0,0 +1,73 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] Core: add variant builder implementation [iceberg]

rdblue commented on code in PR #11857: URL: https://github.com/apache/iceberg/pull/11857#discussion_r1927938774 ## core/src/main/java/org/apache/iceberg/variants/VariantSizeLimitException.java: ## @@ -0,0 +1,26 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under on

Re: [PR] Core: add variant builder implementation [iceberg]

rdblue commented on code in PR #11857: URL: https://github.com/apache/iceberg/pull/11857#discussion_r1927938004 ## core/src/main/java/org/apache/iceberg/variants/VariantArray.java: ## @@ -20,6 +20,10 @@ /** An variant array value. */ public interface VariantArray extends Var

Re: [PR] Core: add variant type support [iceberg]

rdblue commented on code in PR #11831: URL: https://github.com/apache/iceberg/pull/11831#discussion_r1927936162 ## core/src/test/java/org/apache/iceberg/TestMetadataUpdateParser.java: ## @@ -108,19 +121,19 @@ public void testUpgradeFormatVersionFromJson() { } /** AddSche

Re: [PR] Core: add variant type support [iceberg]

rdblue commented on code in PR #11831: URL: https://github.com/apache/iceberg/pull/11831#discussion_r1927935593 ## core/src/test/java/org/apache/iceberg/TestMetadataUpdateParser.java: ## @@ -52,6 +56,15 @@ public class TestMetadataUpdateParser { Types.NestedField.requ

Re: [PR] Core: add variant type support [iceberg]

rdblue commented on code in PR #11831: URL: https://github.com/apache/iceberg/pull/11831#discussion_r1927935200 ## core/src/main/java/org/apache/iceberg/avro/TypeToSchema.java: ## @@ -49,6 +49,15 @@ abstract class TypeToSchema extends TypeUtil.SchemaVisitor { private static

Re: [PR] Core: add variant type support [iceberg]

rdblue commented on code in PR #11831: URL: https://github.com/apache/iceberg/pull/11831#discussion_r1927933385 ## core/src/main/java/org/apache/iceberg/avro/BuildAvroProjection.java: ## @@ -56,6 +56,10 @@ class BuildAvroProjection extends AvroCustomOrderSchemaVisitor names, I

Re: [PR] Core: add variant type support [iceberg]

rdblue commented on code in PR #11831: URL: https://github.com/apache/iceberg/pull/11831#discussion_r1927930058 ## api/src/main/java/org/apache/iceberg/types/Types.java: ## @@ -61,6 +61,14 @@ private Types() {} private static final Pattern DECIMAL = Pattern.compile("de

Re: [PR] Core: add variant type support [iceberg]

rdblue commented on code in PR #11831: URL: https://github.com/apache/iceberg/pull/11831#discussion_r1927930058 ## api/src/main/java/org/apache/iceberg/types/Types.java: ## @@ -61,6 +61,14 @@ private Types() {} private static final Pattern DECIMAL = Pattern.compile("de

Re: [PR] add iceberg_avro interface [iceberg-cpp]

zhjwpku commented on PR #34: URL: https://github.com/apache/iceberg-cpp/pull/34#issuecomment-2611302409 > I have created a PR against your repo: [zhjwpku#3](https://github.com/zhjwpku/iceberg-cpp/pull/3). The CIs are all green on my side: [wgtmac#2](https://github.com/wgtmac/iceberg-cpp/pul

Re: [PR] Core: add variant type support [iceberg]

rdblue commented on code in PR #11831: URL: https://github.com/apache/iceberg/pull/11831#discussion_r1927927230 ## core/src/main/java/org/apache/iceberg/SchemaParser.java: ## @@ -132,6 +133,8 @@ static void toJson(Type.PrimitiveType primitive, JsonGenerator generator) throws

Re: [PR] Core: add variant type support [iceberg]

rdblue commented on code in PR #11831: URL: https://github.com/apache/iceberg/pull/11831#discussion_r1927926739 ## api/src/test/java/org/apache/iceberg/types/TestTypeUtil.java: ## @@ -85,33 +89,33 @@ public void testAssignIncreasingFreshIdWithIdentifier() { .isEqualTo(e

Re: [PR] Core: add variant type support [iceberg]

rdblue commented on code in PR #11831: URL: https://github.com/apache/iceberg/pull/11831#discussion_r1927926305 ## api/src/test/java/org/apache/iceberg/types/TestTypeUtil.java: ## @@ -85,33 +89,33 @@ public void testAssignIncreasingFreshIdWithIdentifier() { .isEqualTo(e

Re: [PR] Core: add variant type support [iceberg]

rdblue commented on code in PR #11831: URL: https://github.com/apache/iceberg/pull/11831#discussion_r1927926035 ## api/src/test/java/org/apache/iceberg/types/TestTypeUtil.java: ## @@ -24,38 +24,43 @@ import static org.assertj.core.api.Assertions.assertThatThrownBy; import ja

Re: [PR] Core: add variant type support [iceberg]

rdblue commented on code in PR #11831: URL: https://github.com/apache/iceberg/pull/11831#discussion_r1927925802 ## api/src/test/java/org/apache/iceberg/types/TestTypeUtil.java: ## @@ -24,38 +24,43 @@ import static org.assertj.core.api.Assertions.assertThatThrownBy; import ja

Re: [PR] Core: add variant type support [iceberg]

rdblue commented on code in PR #11831: URL: https://github.com/apache/iceberg/pull/11831#discussion_r1927920618 ## api/src/main/java/org/apache/iceberg/types/Types.java: ## @@ -61,6 +61,14 @@ private Types() {} private static final Pattern DECIMAL = Pattern.compile("de

Re: [PR] Core: add variant type support [iceberg]

rdblue commented on code in PR #11831: URL: https://github.com/apache/iceberg/pull/11831#discussion_r1927919640 ## api/src/main/java/org/apache/iceberg/types/TypeUtil.java: ## @@ -709,6 +709,10 @@ public T map(Types.MapType map, Supplier keyResult, Supplier valueResult)

[I] Add additional configurations for ApacheHttpClientBuilder [iceberg]

Mohammed-Karim226 opened a new issue, #12077: URL: https://github.com/apache/iceberg/issues/12077 ### Feature Request / Improvement ### *Description* This feature request proposes adding additional configurations for ApacheHttpClientBuilder to enhance its functionality and cu

Re: [PR] Parquet: Correctly prune nested columns [iceberg]

github-actions[bot] commented on PR #11373: URL: https://github.com/apache/iceberg/pull/11373#issuecomment-2611274214 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Core: Prevent dropping column which is referenced by active partition specs [iceberg]

github-actions[bot] commented on PR #11842: URL: https://github.com/apache/iceberg/pull/11842#issuecomment-2611274274 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] add iceberg_avro interface [iceberg-cpp]

wgtmac commented on PR #34: URL: https://github.com/apache/iceberg-cpp/pull/34#issuecomment-2611262437 I have created a PR against your repo: https://github.com/zhjwpku/iceberg-cpp/pull/3. The CIs are all green on my side: https://github.com/wgtmac/iceberg-cpp/pull/2. Please take a look and

Re: [I] `partial-progress.max-failed-commits` Incorrectly compare the failureCommit value [iceberg]

ruotianwang commented on issue #12076: URL: https://github.com/apache/iceberg/issues/12076#issuecomment-2611246998 cc: @manuzhang the original owner for visibility -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[I] `partial-progress.max-failed-commits` Incorrectly compare the failureCommit value [iceberg]

ruotianwang opened a new issue, #12076: URL: https://github.com/apache/iceberg/issues/12076 ### Apache Iceberg version 1.7.1 (latest release) ### Query engine Spark ### Please describe the bug 🐞 During the usage of `partial-progress.max-failed-commits`, we'v

Re: [PR] Spark 3.5: Procedure to rewrite table path [iceberg]

amogh-jahagirdar commented on code in PR #11931: URL: https://github.com/apache/iceberg/pull/11931#discussion_r1927795585 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRewriteTablePathProcedure.java: ## @@ -0,0 +1,174 @@ +/* + * Licensed to

Re: [PR] Core, API, Spec: Metadata Row Lineage [iceberg]

RussellSpitzer commented on PR #11948: URL: https://github.com/apache/iceberg/pull/11948#issuecomment-2611196839 Checks for equality deletes - https://github.com/apache/iceberg/pull/12075 Separate PR for now but can merge it here if we want to discuss that. I think we may have a conversatio

[PR] Core: Checks for Equality Delete when Row LIneage is Enabled - Using Snapshot Summary [iceberg]

RussellSpitzer opened a new pull request, #12075: URL: https://github.com/apache/iceberg/pull/12075 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

Re: [PR] Do not set Content-type header in doPost [iceberg-go]

zeroshade merged PR #268: URL: https://github.com/apache/iceberg-go/pull/268 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] HadoopFileIO to support bulk delete through the Hadoop Filesystem APIs [iceberg]

rdblue commented on code in PR #10233: URL: https://github.com/apache/iceberg/pull/10233#discussion_r1927789250 ## core/src/main/java/org/apache/iceberg/hadoop/wrappedio/BindingUtils.java: ## @@ -0,0 +1,202 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

Re: [PR] HadoopFileIO to support bulk delete through the Hadoop Filesystem APIs [iceberg]

rdblue commented on code in PR #10233: URL: https://github.com/apache/iceberg/pull/10233#discussion_r1927789250 ## core/src/main/java/org/apache/iceberg/hadoop/wrappedio/BindingUtils.java: ## @@ -0,0 +1,202 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[PR] Make s3.request_timeout configurable [iceberg-python]

metadaddy opened a new pull request, #1568: URL: https://github.com/apache/iceberg-python/pull/1568 Similarly to #218, we see occasional timeout errors when writing data to S3-compatible object storage: ``` When uploading part for key 'drivestats/data/date_month=2014-08/0-0-9c

Re: [I] software.amazon.awssdk.services.s3.model.S3Exception: The bucket you are attempting to access must be addressed using the specified endpoint. [iceberg]

petartushev commented on issue #11997: URL: https://github.com/apache/iceberg/issues/11997#issuecomment-2611147731 @punyashah yes that was an issue, but if I was submitting jobs to that spark cluster. In the above example I was running the application on it's standalone cluster. Nonetheless

Re: [I] 123123 [iceberg]

emmettd04 closed issue #12074: 123123 URL: https://github.com/apache/iceberg/issues/12074 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubsc

Re: [I] 123 [iceberg]

emmettd04 closed issue #12073: 123 URL: https://github.com/apache/iceberg/issues/12073 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr..

Re: [PR] Implement update for `remove-snapshots` action [iceberg-python]

grihabor commented on PR #1561: URL: https://github.com/apache/iceberg-python/pull/1561#issuecomment-2611088714 Hey @kevinjqliu, ready for another review round. I had to cherry pick the changes from https://github.com/apache/iceberg-python/pull/822 to reuse the code that removes refs --

[PR] Do not set Content-type header in doPost [iceberg-go]

curtisr7 opened a new pull request, #268: URL: https://github.com/apache/iceberg-go/pull/268 This change shouldn't really be necessary. but I just spent the better part of an afternoon tracking it down. I've been testing against snowflake opencatalog and it appears that they're

Re: [PR] Parquet: Add readers and writers for the internal object model [iceberg]

rdblue commented on code in PR #11904: URL: https://github.com/apache/iceberg/pull/11904#discussion_r1927713133 ## parquet/src/main/java/org/apache/iceberg/data/parquet/InternalReader.java: ## @@ -0,0 +1,88 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

Re: [PR] Remove `slf4j-api` reference in `LICENSE` (#12052) [iceberg]

RussellSpitzer merged PR #12068: URL: https://github.com/apache/iceberg/pull/12068 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Remove `slf4j-api` reference in `LICENSE` (#12052) [iceberg]

rdblue commented on PR #12068: URL: https://github.com/apache/iceberg/pull/12068#issuecomment-2611053977 Thanks, @Fokko and @jbonofre! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Spec, OpenAPI: Adds EnableRowLineage Metadata Update [iceberg]

RussellSpitzer commented on code in PR #12050: URL: https://github.com/apache/iceberg/pull/12050#discussion_r1927701544 ## open-api/rest-catalog-open-api.yaml: ## @@ -2945,6 +2946,14 @@ components: items: type: integer +EnableRowLineageUpdate: +

Re: [PR] Remove `slf4j-api` reference in `LICENSE` (#12052) [iceberg]

RussellSpitzer commented on PR #12068: URL: https://github.com/apache/iceberg/pull/12068#issuecomment-2611045376 Thanks @Fokko and @jbonofre for cleaning all this up! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [I] [DISCUSS] Exceptions vs status codes [iceberg-cpp]

ormandi commented on issue #14: URL: https://github.com/apache/iceberg-cpp/issues/14#issuecomment-2611038063 > Because any consumer of libiceberg would have to be careful about API/ABI conflicts with other Abseil-using libraries. Thank you for the response! Though, I think API-wise `a

Re: [PR] Flink: Backport default values support in Parquet reader on Flink v1.18 and v1.19 [iceberg]

rdblue commented on PR #12072: URL: https://github.com/apache/iceberg/pull/12072#issuecomment-2611030153 Thanks, @jbonofre! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] API: Add `UnknownType` [iceberg]

RussellSpitzer commented on code in PR #12012: URL: https://github.com/apache/iceberg/pull/12012#discussion_r1927677945 ## core/src/test/java/org/apache/iceberg/TestSortOrder.java: ## @@ -342,6 +343,13 @@ public void testVariantUnsupported() { .hasMessage("Unsupported t

Re: [PR] Flink: Backport default values support in Parquet reader on Flink v1.18 and v1.19 [iceberg]

amogh-jahagirdar merged PR #12072: URL: https://github.com/apache/iceberg/pull/12072 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] API: Add `UnknownType` [iceberg]

Fokko commented on code in PR #12012: URL: https://github.com/apache/iceberg/pull/12012#discussion_r1927674078 ## core/src/test/java/org/apache/iceberg/TestSortOrder.java: ## @@ -342,6 +343,13 @@ public void testVariantUnsupported() { .hasMessage("Unsupported type for i

Re: [PR] Flink: Avoid RANGE mode broken chain when write parallelism changes [iceberg]

stevenzwu commented on PR #11702: URL: https://github.com/apache/iceberg/pull/11702#issuecomment-2610528451 @huyuanfeng2018 can you create a back port for this fix? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] Implement update for `remove-snapshots` action [iceberg-python]

grihabor commented on code in PR #1561: URL: https://github.com/apache/iceberg-python/pull/1561#discussion_r1927678717 ## pyiceberg/table/update/__init__.py: ## @@ -455,6 +455,19 @@ def _(update: SetSnapshotRefUpdate, base_metadata: TableMetadata, context: _Tabl return bas

Re: [PR] API: Add `UnknownType` [iceberg]

Fokko commented on code in PR #12012: URL: https://github.com/apache/iceberg/pull/12012#discussion_r1927669099 ## api/src/main/java/org/apache/iceberg/expressions/ExpressionUtil.java: ## @@ -539,7 +539,8 @@ private static String sanitize(Type type, Object value, long now, int t

Re: [PR] Parquet: Add readers and writers for the internal object model [iceberg]

rdblue commented on code in PR #11904: URL: https://github.com/apache/iceberg/pull/11904#discussion_r1927639914 ## parquet/src/main/java/org/apache/iceberg/data/parquet/InternalReader.java: ## @@ -0,0 +1,88 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

Re: [PR] Parquet: Add readers and writers for the internal object model [iceberg]

rdblue commented on code in PR #11904: URL: https://github.com/apache/iceberg/pull/11904#discussion_r1927639914 ## parquet/src/main/java/org/apache/iceberg/data/parquet/InternalReader.java: ## @@ -0,0 +1,88 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

  1   2   3   >