Re: [PR] #9073 Junit 4 tests switched to JUnit 5 [iceberg]

nastra commented on code in PR #9793: URL: https://github.com/apache/iceberg/pull/9793#discussion_r1555319681 ## data/src/test/java/org/apache/iceberg/RecordWrapperTest.java: ## @@ -103,11 +103,11 @@ public void testNestedSchema() { } private void generateAndValidate(Sch

Re: [PR] [1.5.x] Core: Mark 502 and 504 failures as retryable to the exponential retry strategy [iceberg]

nastra merged PR #10113: URL: https://github.com/apache/iceberg/pull/10113 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] [1.5.x] API: Fix default FileIO#newInputFile ManifestFile, DataFile and DeleteFile implementations [iceberg]

nastra merged PR #10114: URL: https://github.com/apache/iceberg/pull/10114 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] [WIP] Core: Fix JDBC Catalog table commit when migrating from schema V0 to V1 [iceberg]

jbonofre commented on code in PR #10111: URL: https://github.com/apache/iceberg/pull/10111#discussion_r1558816450 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcUtil.java: ## @@ -54,7 +54,29 @@ enum SchemaVersion { static final String TABLE_RECORD_TYPE = "TABLE"; static

Re: [I] Incorrect Metrics Calculation for Iceberg Table Due to Column Name Transformation with Special Characters [iceberg]

lintingbin commented on issue #10115: URL: https://github.com/apache/iceberg/issues/10115#issuecomment-2046488163 To resolve this issue, there are two potential solutions: Utilize fieldId instead of fieldName to determine MetricsMode in the ParquetUtil.java file. By doing so, the corr

[I] Incorrect Metrics Calculation for Iceberg Table Due to Column Name Transformation with Special Characters [iceberg]

lintingbin opened a new issue, #10115: URL: https://github.com/apache/iceberg/issues/10115 ### Apache Iceberg version 1.3.1 ### Query engine Spark ### Please describe the bug 🐞 ``` CREATE TABLE tmp.iceberg_test3 ( `log_type.string` STRING, `even

Re: [I] read from Iceberg table throw java.lang.ArrayIndexOutOfBoundsException: 3 [iceberg]

jiantao-vungle commented on issue #10103: URL: https://github.com/apache/iceberg/issues/10103#issuecomment-2046375216 thank @nastra and @amogh-jahagirdar , it's need to evaluate to share the parquet file, i did some explores,like `parquet dictionary 00339-809-290052b9-087e-4bda-b9a6-716f

Re: [PR] Add Pagination To List Apis [iceberg]

rahil-c commented on code in PR #9782: URL: https://github.com/apache/iceberg/pull/9782#discussion_r1558712520 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -278,14 +286,26 @@ public void setConf(Object newConf) { @Override public List listTa

Re: [PR] Add Pagination To List Apis [iceberg]

rahil-c commented on code in PR #9782: URL: https://github.com/apache/iceberg/pull/9782#discussion_r1558712520 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -278,14 +286,26 @@ public void setConf(Object newConf) { @Override public List listTa

Re: [I] Spark query pushdown error in SQL query optimization phase [iceberg]

wForget commented on issue #9881: URL: https://github.com/apache/iceberg/issues/9881#issuecomment-2046330916 > Could you share the query as well as the table schema? @RussellSpitzer I reported a similar issue: https://issues.apache.org/jira/browse/SPARK-47463, which contains reproduce

Re: [PR] Add Pagination To List Apis [iceberg]

rahil-c commented on code in PR #9782: URL: https://github.com/apache/iceberg/pull/9782#discussion_r1558704222 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -278,14 +286,26 @@ public void setConf(Object newConf) { @Override public List listTa

Re: [I] NPE During RewriteDataFiles Action with Nessie [iceberg]

ajantha-bhat commented on issue #10110: URL: https://github.com/apache/iceberg/issues/10110#issuecomment-2046317631 Ideally we should use the same version. Can you try once with the same version? This NPE (that too in a debug log) is little odd to me (never seen before). Nessie does

Re: [PR] Core: Lazily compute & cache hashCode in CharSequenceWrapper [iceberg]

amogh-jahagirdar commented on code in PR #10023: URL: https://github.com/apache/iceberg/pull/10023#discussion_r1558676985 ## api/src/main/java/org/apache/iceberg/util/CharSequenceWrapper.java: ## @@ -29,13 +29,16 @@ public static CharSequenceWrapper wrap(CharSequence seq) { }

Re: [PR] Core: Lazily compute & cache hashCode in CharSequenceWrapper [iceberg]

amogh-jahagirdar commented on code in PR #10023: URL: https://github.com/apache/iceberg/pull/10023#discussion_r1558672872 ## api/src/main/java/org/apache/iceberg/util/CharSequenceWrapper.java: ## @@ -29,13 +29,16 @@ public static CharSequenceWrapper wrap(CharSequence seq) { }

Re: [PR] [WIP] Core: Fix JDBC Catalog table commit when migrating from schema V0 to V1 [iceberg]

ajantha-bhat commented on code in PR #10111: URL: https://github.com/apache/iceberg/pull/10111#discussion_r1558665711 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcUtil.java: ## @@ -54,7 +54,29 @@ enum SchemaVersion { static final String TABLE_RECORD_TYPE = "TABLE"; st

Re: [I] NPE During RewriteDataFiles Action with Nessie [iceberg]

pseudomuto commented on issue #10110: URL: https://github.com/apache/iceberg/issues/10110#issuecomment-2046306547 In case it's relevant, the spark extensions jar is version 0.78.0 (Iceberg 1.4.3) -- This is an automated message from the Apache Git Service. To respond to the message, pleas

Re: [I] NPE During RewriteDataFiles Action with Nessie [iceberg]

pseudomuto commented on issue #10110: URL: https://github.com/apache/iceberg/issues/10110#issuecomment-2046298594 Hey @ajantha-bhat! I'm running 0.76.3 deployed with the Helm chart -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

Re: [I] NPE During RewriteDataFiles Action with Nessie [iceberg]

ajantha-bhat commented on issue #10110: URL: https://github.com/apache/iceberg/issues/10110#issuecomment-2046290081 @pseudomuto: Which version of the Nessie server are you using and how it is deployed? -- This is an automated message from the Apache Git Service. To respond to the messag

Re: [I] NPE During RewriteDataFiles Action with Nessie [iceberg]

amogh-jahagirdar commented on issue #10110: URL: https://github.com/apache/iceberg/issues/10110#issuecomment-2046273223 @ajantha-bhat @jbonofre Would you be able to help out on this issue? -- This is an automated message from the Apache Git Service. To respond to the message, please log o

Re: [PR] Add Pagination To List Apis [iceberg]

danielcweeks commented on code in PR #9782: URL: https://github.com/apache/iceberg/pull/9782#discussion_r1558551506 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -278,14 +286,26 @@ public void setConf(Object newConf) { @Override public List l

Re: [PR] Add Pagination To List Apis [iceberg]

danielcweeks commented on code in PR #9782: URL: https://github.com/apache/iceberg/pull/9782#discussion_r1558549404 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -278,14 +286,26 @@ public void setConf(Object newConf) { @Override public List l

Re: [I] Hive: Check the increased usage of HMSClients in TestHiveIcebergStorageHandlerWithEngine tests [iceberg]

github-actions[bot] commented on issue #2474: URL: https://github.com/apache/iceberg/issues/2474#issuecomment-2046229680 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Add support for type promotion in the ArrowReader [iceberg]

github-actions[bot] commented on issue #2483: URL: https://github.com/apache/iceberg/issues/2483#issuecomment-2046229702 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] NPE while processing data in flink job [iceberg]

github-actions[bot] commented on issue #2471: URL: https://github.com/apache/iceberg/issues/2471#issuecomment-2046229656 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Kevinjqliu/special character column parquet [iceberg-python]

gwindes commented on code in PR #590: URL: https://github.com/apache/iceberg-python/pull/590#discussion_r1558403558 ## column_name_test.py: ## @@ -0,0 +1,87 @@ +import os.path + +from pyiceberg.catalog.sql import SqlCatalog +import pyarrow as pa +import pandas as pd + + +def san

Re: [PR] Kevinjqliu/special character column parquet [iceberg-python]

gwindes commented on code in PR #590: URL: https://github.com/apache/iceberg-python/pull/590#discussion_r1558403270 ## column_name_test.py: ## Review Comment: Should this file live under the `tests/` folder instead of project root? -- This is an automated message from t

Re: [PR] Kevinjqliu/special character column parquet [iceberg-python]

gwindes commented on code in PR #590: URL: https://github.com/apache/iceberg-python/pull/590#discussion_r1558403974 ## column_name_test.py: ## @@ -0,0 +1,87 @@ +import os.path + +from pyiceberg.catalog.sql import SqlCatalog +import pyarrow as pa +import pandas as pd + + +def san

Re: [PR] Parquet: Use native getRowIndexOffset support instead of calculating it [iceberg]

wypoon commented on PR #10107: URL: https://github.com/apache/iceberg/pull/10107#issuecomment-2046147418 @flyrain I have updated your PR. Please review. cc @chenjunjiedada @Fokko. One thing I'd like to add is a test that would fail with Parquet 1.12.3 (which has the bug [PARQUET-2161](ht

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

rdblue commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1558327213 ## api/src/main/java/org/apache/iceberg/transforms/SortOrderVisitor.java: ## @@ -84,22 +84,16 @@ static List visit(SortOrder sortOrder, SortOrderVisitor visitor) {

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

rdblue commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1558325800 ## api/src/test/java/org/apache/iceberg/expressions/TestMiscLiteralConversions.java: ## @@ -99,8 +117,10 @@ public void testInvalidBooleanConversions() { Types.

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

rdblue commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1558325098 ## api/src/test/java/org/apache/iceberg/expressions/TestMiscLiteralConversions.java: ## @@ -62,6 +64,22 @@ public void testIdentityConversions() { } } + @Test

Re: [PR] Add Pagination To List Apis [iceberg]

rahil-c commented on code in PR #9782: URL: https://github.com/apache/iceberg/pull/9782#discussion_r1558307800 ## core/src/test/java/org/apache/iceberg/rest/responses/TestListNamespacesResponse.java: ## @@ -83,6 +83,23 @@ public void testBuilderDoesNotCreateInvalidObjects() {

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

rdblue commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1558277643 ## api/src/main/java/org/apache/iceberg/transforms/Timestamps.java: ## @@ -31,54 +32,117 @@ import org.apache.iceberg.util.DateTimeUtil; import org.apache.iceberg.util

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

rdblue commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1558270044 ## api/src/main/java/org/apache/iceberg/transforms/TransformUtil.java: ## @@ -62,6 +62,14 @@ static String humanTimestampWithoutZone(Long timestampMicros) { return

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

rdblue commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1558269287 ## api/src/main/java/org/apache/iceberg/transforms/Timestamps.java: ## @@ -184,13 +252,25 @@ public String toHumanString(Type outputType, Integer value) { case H

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

rdblue commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1558268249 ## api/src/main/java/org/apache/iceberg/transforms/Timestamps.java: ## @@ -112,11 +180,11 @@ public boolean satisfiesOrderOf(Transform other) { } if (other i

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

rdblue commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1558267994 ## api/src/main/java/org/apache/iceberg/transforms/Timestamps.java: ## @@ -89,17 +153,21 @@ public SerializableFunction bind(Type type) { @Override public boole

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

rdblue commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1558262073 ## api/src/main/java/org/apache/iceberg/expressions/ExpressionUtil.java: ## @@ -600,6 +612,12 @@ private static String sanitizeString(CharSequence value, long now, int

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

rdblue commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1558261406 ## api/src/main/java/org/apache/iceberg/transforms/Bucket.java: ## @@ -54,6 +54,7 @@ static & SerializableFunction> B get( return (B) new BucketInteger(numBuck

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

rdblue commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1558253135 ## api/src/main/java/org/apache/iceberg/expressions/Literals.java: ## @@ -515,6 +548,21 @@ public Literal to(Type type) { return (Literal) new TimestampLit

[PR] Flink: Adds support for 1.19 version [iceberg]

rodmeneses opened a new pull request, #10112: URL: https://github.com/apache/iceberg/pull/10112 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

rdblue commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1558237579 ## api/src/main/java/org/apache/iceberg/expressions/ExpressionUtil.java: ## @@ -564,6 +575,7 @@ private static String sanitizeDate(int days, int today) { return "(d

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

rdblue commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1558234765 ## api/src/main/java/org/apache/iceberg/expressions/ExpressionUtil.java: ## @@ -600,6 +612,12 @@ private static String sanitizeString(CharSequence value, long now, int

Re: [I] unpartitioned write produce objects containing extraneous slash using S3FileIO causing other system not able to read [iceberg]

puchengy commented on issue #8166: URL: https://github.com/apache/iceberg/issues/8166#issuecomment-2045919148 fixed by https://github.com/apache/iceberg/pull/7685 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] unpartitioned write produce objects containing extraneous slash using S3FileIO causing other system not able to read [iceberg]

puchengy closed issue #8166: unpartitioned write produce objects containing extraneous slash using S3FileIO causing other system not able to read URL: https://github.com/apache/iceberg/issues/8166 -- This is an automated message from the Apache Git Service. To respond to the message, please l

Re: [PR] Add Pagination To List Apis [iceberg]

rahil-c commented on code in PR #9782: URL: https://github.com/apache/iceberg/pull/9782#discussion_r1558181883 ## core/src/test/java/org/apache/iceberg/rest/TestRESTCatalog.java: ## @@ -2288,6 +2289,34 @@ public void multipleDiffsAgainstMultipleTablesLastFails() { assertTh

Re: [PR] [WIP] Core: Fix JDBC Catalog table commit when migrating from schema V0 to V1 [iceberg]

jbonofre commented on PR #10111: URL: https://github.com/apache/iceberg/pull/10111#issuecomment-2045915251 I'm preparing tests to illustrate the original problem and demonstrate this fix works. -- This is an automated message from the Apache Git Service. To respond to the message, please

[PR] [WIP] Core: Fix JDBC Catalog table commit when migrating from schema V0 to V1 [iceberg]

jbonofre opened a new pull request, #10111: URL: https://github.com/apache/iceberg/pull/10111 This PR fixes table commit on the JDBC Catalog. When upgrading JDBC Catalog SQL schema from `V0` to `V1`, the `iceberg_type` column is added with `NULL` value by default. However, the SQL

Re: [PR] [draft] Use Parquet's getRowIndexOffset support instead of calculating it [iceberg]

wypoon commented on code in PR #10107: URL: https://github.com/apache/iceberg/pull/10107#discussion_r1558177377 ## .palantir/revapi.yml: ## @@ -1018,6 +1018,43 @@ acceptedBreaks: old: "method void org.apache.iceberg.PositionDeletesTable.PositionDeletesBatchScan::(org.apa

Re: [PR] [draft] Use Parquet's getRowIndexOffset support instead of calculating it [iceberg]

wypoon commented on code in PR #10107: URL: https://github.com/apache/iceberg/pull/10107#discussion_r1558177377 ## .palantir/revapi.yml: ## @@ -1018,6 +1018,43 @@ acceptedBreaks: old: "method void org.apache.iceberg.PositionDeletesTable.PositionDeletesBatchScan::(org.apa

Re: [PR] feat: init iceberg writer [iceberg-rust]

sdd commented on code in PR #275: URL: https://github.com/apache/iceberg-rust/pull/275#discussion_r1558106190 ## crates/iceberg/src/writer/base_writer/data_file_writer.rs: ## @@ -0,0 +1,323 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor

[I] [feat request] Add `metadata_log_entries` metadata table [iceberg-python]

kevinjqliu opened a new issue, #594: URL: https://github.com/apache/iceberg-python/issues/594 ### Feature Request / Improvement Similar to the Spark metadata table https://iceberg.apache.org/docs/nightly/spark-queries/#metadata-log-entries which uses the `metadata-log` field

Re: [PR] Validate overwrite filter [iceberg-python]

jqin61 commented on PR #582: URL: https://github.com/apache/iceberg-python/pull/582#issuecomment-2045791559 A delete + append will do the static overwrite and most of the validation will be done by delete itself. Closing the pr and will open a static overwrite PR when DELETE and MERGE_APPEN

Re: [PR] Validate overwrite filter [iceberg-python]

jqin61 closed pull request #582: Validate overwrite filter URL: https://github.com/apache/iceberg-python/pull/582 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

Re: [I] [BUG] Valid column characters fail on to_arrow() or to_pandas() ArrowInvalid: No match for FieldRef.Name [iceberg-python]

Fokko commented on issue #584: URL: https://github.com/apache/iceberg-python/issues/584#issuecomment-2045785832 > Yeah me too. But I think Java Iceberg doesn't support this since parquet files with ABC-GG-1-A column will be read as Iceberg column ABC_x2DGG_x2D1_x2DA. I think it's worth open

[I] NPE During RewriteDataFiles Action with Nessie [iceberg]

pseudomuto opened a new issue, #10110: URL: https://github.com/apache/iceberg/issues/10110 ### Apache Iceberg version 1.4.3 ### Query engine Spark ### Please describe the bug 🐞 I'm having trouble running the RewriteDataFiles action in Spark. I have a table

Re: [PR] Build: Bump net.snowflake:snowflake-jdbc from 3.14.5 to 3.15.1 [iceberg]

sfc-gh-dhuo commented on PR #10095: URL: https://github.com/apache/iceberg/pull/10095#issuecomment-2045602040 @Fokko Thanks for the heads up! No concerns, LGTM. Spot tested a local build with this update successfully. -- This is an automated message from the Apache Git Service. To respond

Re: [I] Improve read times and reduce size of metadata.json by storing schemas in external files [iceberg]

syun64 commented on issue #9734: URL: https://github.com/apache/iceberg/issues/9734#issuecomment-2045599198 Yeah, I agree that it would definitely need a change to the spec and maybe this is something we could discuss for the [V3 Spec milestone](https://github.com/apache/iceberg/milestone/4

Re: [PR] feat: init iceberg writer [iceberg-rust]

Xuanwo commented on code in PR #275: URL: https://github.com/apache/iceberg-rust/pull/275#discussion_r1557943458 ## crates/iceberg/src/writer/base_writer/data_file_writer.rs: ## @@ -0,0 +1,323 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contribu

Re: [PR] feat: init iceberg writer [iceberg-rust]

Xuanwo commented on code in PR #275: URL: https://github.com/apache/iceberg-rust/pull/275#discussion_r1557940399 ## crates/iceberg/src/writer/base_writer/data_file_writer.rs: ## @@ -0,0 +1,323 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contribu

Re: [PR] feat: init iceberg writer [iceberg-rust]

Xuanwo commented on code in PR #275: URL: https://github.com/apache/iceberg-rust/pull/275#discussion_r1557940399 ## crates/iceberg/src/writer/base_writer/data_file_writer.rs: ## @@ -0,0 +1,323 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contribu

Re: [PR] Core: Add EnvironmentContext to commit summary [iceberg]

amogh-jahagirdar commented on PR #9273: URL: https://github.com/apache/iceberg/pull/9273#issuecomment-2045466272 Sorry for the delay on reviewing this @manuzhang thanks for adding this, I think the environment details in the summary will be quite helpful. I'll go ahead and merge. Thanks @na

Re: [PR] Core: Add EnvironmentContext to commit summary [iceberg]

amogh-jahagirdar merged PR #9273: URL: https://github.com/apache/iceberg/pull/9273 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Add Pagination To List Apis [iceberg]

nastra commented on code in PR #9782: URL: https://github.com/apache/iceberg/pull/9782#discussion_r1557807369 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -224,6 +226,7 @@ public void initialize(String name, Map unresolved) { clien

Re: [PR] Add Pagination To List Apis [iceberg]

nastra commented on code in PR #9782: URL: https://github.com/apache/iceberg/pull/9782#discussion_r1557801099 ## core/src/test/java/org/apache/iceberg/rest/responses/TestListTablesResponse.java: ## @@ -105,6 +105,23 @@ public void testBuilderDoesNotCreateInvalidObjects() {

Re: [PR] Add Pagination To List Apis [iceberg]

nastra commented on code in PR #9782: URL: https://github.com/apache/iceberg/pull/9782#discussion_r1557800399 ## core/src/test/java/org/apache/iceberg/rest/responses/TestListNamespacesResponse.java: ## @@ -83,6 +83,23 @@ public void testBuilderDoesNotCreateInvalidObjects() {

Re: [I] Improve read times and reduce size of metadata.json by storing schemas in external files [iceberg]

syun64 commented on issue #9734: URL: https://github.com/apache/iceberg/issues/9734#issuecomment-2045357481 +1 to this idea as well. I think this proposal will be a significant improvement to use cases where: 1. There are many schema changes 2. The schema is already large (think

[I] Pushdown of aggregate functions in where [iceberg]

clintf1982 opened a new issue, #10109: URL: https://github.com/apache/iceberg/issues/10109 ### Query engine Spark ### Question Table X has a partition that uses id. **I perform the following query:** select count(1) from X where id > (select max(id) from X)

Re: [I] [JDBC Catalog] Table commit fails if iceberg_type field is NULL [iceberg]

jbonofre commented on issue #10046: URL: https://github.com/apache/iceberg/issues/10046#issuecomment-2045252857 I found the problem: it's in the `V1_DO_COMMIT_SQL` statement. It only tests `RECORD_TYPE = ?`. It should also test for `NULL`. I'm adding a unit test and fixing the SQL statement

Re: [PR] Core: Pass input file into iterators to get the file name [iceberg]

javsanbel2 commented on PR #9691: URL: https://github.com/apache/iceberg/pull/9691#issuecomment-2044695626 I tried to find a test to check this new functionality, but I didn't find any =L, I also tried to create a corrupted file myself, but I never got into the error... any ideas? -- Thi

Re: [PR] File size in bytes tracking with deleted files in expire snapshots [iceberg]

nastra commented on code in PR #10036: URL: https://github.com/apache/iceberg/pull/10036#discussion_r1557409774 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RichFileInfo.java: ## @@ -0,0 +1,36 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

Re: [PR] Docs: Fix On-screen display issues and minor expressions on Branching and Tagging DDL [iceberg]

nastra merged PR #10091: URL: https://github.com/apache/iceberg/pull/10091 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] [draft] Use Parquet's getRowIndexOffset support instead of calculating it [iceberg]

nastra commented on code in PR #10107: URL: https://github.com/apache/iceberg/pull/10107#discussion_r1557175353 ## .palantir/revapi.yml: ## @@ -1018,6 +1018,43 @@ acceptedBreaks: old: "method void org.apache.iceberg.PositionDeletesTable.PositionDeletesBatchScan::(org.apa

Re: [PR] Add Pagination To List Apis [iceberg]

nastra commented on code in PR #9782: URL: https://github.com/apache/iceberg/pull/9782#discussion_r1557169245 ## core/src/test/java/org/apache/iceberg/rest/TestRESTCatalog.java: ## @@ -2288,6 +2289,34 @@ public void multipleDiffsAgainstMultipleTablesLastFails() { assertTha

[I] MERGE INTO pushdown filter is not applied to a table defined with partition transform [iceberg]

bk-mz opened a new issue, #10108: URL: https://github.com/apache/iceberg/issues/10108 ### Apache Iceberg version 1.4.2 ### Query engine Spark ### Please describe the bug 🐞 Table is defined with `partitioned by (hour(data_load_ts))` where data_load_ts is tim

Re: [PR] Docs: Update releases.md for Spark scala versions [iceberg]

nastra merged PR #10104: URL: https://github.com/apache/iceberg/pull/10104 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

[I] Pyiceberg scan with custom filters [iceberg-python]

bigluck opened a new issue, #593: URL: https://github.com/apache/iceberg-python/issues/593 ### Feature Request / Improvement Ciao all, I'm looking for a way to bypass the limited number of supported filters on Pyiceberg without raising an out-of-memory error on my running insta

Re: [PR] Docs: Update releases.md for Spark scala versions [iceberg]

nastra commented on PR #10104: URL: https://github.com/apache/iceberg/pull/10104#issuecomment-2044373818 thanks for improving this @liko9 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] Docs: Added Upsolver to vendor list [iceberg]

nastra merged PR #10096: URL: https://github.com/apache/iceberg/pull/10096 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap