Re: [PR] docs: Add release guide for iceberg-rust [iceberg-rust]

Fokko commented on code in PR #147: URL: https://github.com/apache/iceberg-rust/pull/147#discussion_r1442596925 ## website/src/reference/setup_gpg.md: ## @@ -0,0 +1,161 @@ + + +# Setup GPG key + +> This section is a brief from the [Cryptography with OpenPGP](https://infra.apach

Re: [I] Flink import debezium cdc record(delete type) to iceberg(0.13.2+) got IndexOutOfBoundsException [iceberg]

zinking commented on issue #5043: URL: https://github.com/apache/iceberg/issues/5043#issuecomment-1878282717 incorrect data type mapping between mysql and iceberg could also lead to this. for example tinyint -> boolean -- This is an automated message from the Apache Git Service. To respon

Re: [PR] docs: Add release guide for iceberg-rust [iceberg-rust]

Fokko commented on code in PR #147: URL: https://github.com/apache/iceberg-rust/pull/147#discussion_r1442598483 ## website/src/release.md: ## @@ -0,0 +1,383 @@ + + +This document mainly introduces how the release manager releases a new version in accordance with the Apache requ

Re: [PR] docs: Add release guide for iceberg-rust [iceberg-rust]

Fokko commented on code in PR #147: URL: https://github.com/apache/iceberg-rust/pull/147#discussion_r1442599426 ## website/src/release.md: ## @@ -0,0 +1,383 @@ + + +This document mainly introduces how the release manager releases a new version in accordance with the Apache requ

Re: [PR] docs: Add release guide for iceberg-rust [iceberg-rust]

Fokko commented on code in PR #147: URL: https://github.com/apache/iceberg-rust/pull/147#discussion_r1442598860 ## website/src/release.md: ## @@ -0,0 +1,383 @@ + + +This document mainly introduces how the release manager releases a new version in accordance with the Apache requ

Re: [PR] docs: Add release guide for iceberg-rust [iceberg-rust]

Fokko commented on code in PR #147: URL: https://github.com/apache/iceberg-rust/pull/147#discussion_r1442599810 ## website/src/release.md: ## @@ -0,0 +1,383 @@ + + +This document mainly introduces how the release manager releases a new version in accordance with the Apache requ

Re: [PR] docs: Add release guide for iceberg-rust [iceberg-rust]

Fokko commented on code in PR #147: URL: https://github.com/apache/iceberg-rust/pull/147#discussion_r1442600155 ## website/src/release.md: ## @@ -0,0 +1,383 @@ + + +This document mainly introduces how the release manager releases a new version in accordance with the Apache requ

Re: [PR] docs: Add release guide for iceberg-rust [iceberg-rust]

Fokko commented on code in PR #147: URL: https://github.com/apache/iceberg-rust/pull/147#discussion_r1442602143 ## website/src/release.md: ## @@ -0,0 +1,383 @@ + + +This document mainly introduces how the release manager releases a new version in accordance with the Apache requ

Re: [PR] Spark 3.5: Migrate tests that depend on SparkDistributedDataScanTestBase to JUnit5 [iceberg]

chinmay-bhat commented on PR #9416: URL: https://github.com/apache/iceberg/pull/9416#issuecomment-1878298683 Based on the test failures, older spark versions depend on `core` files like `DeleteFileIndexTestBase`, `ParameterizedTableTestBase`, `DataTableScanTestBase`,`ScanPlanningAndReportin

Re: [PR] Spark: Support renaming views [iceberg]

nastra commented on code in PR #9343: URL: https://github.com/apache/iceberg/pull/9343#discussion_r1442615912 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveViews.scala: ## @@ -53,6 +57,17 @@ case class ResolveViews(spark: SparkSessio

Re: [PR] docs: Add release guide for iceberg-rust [iceberg-rust]

liurenjie1024 commented on code in PR #147: URL: https://github.com/apache/iceberg-rust/pull/147#discussion_r1442622182 ## website/src/release.md: ## @@ -0,0 +1,383 @@ + + +This document mainly introduces how the release manager releases a new version in accordance with the Apa

Re: [PR] Core: Remove deprecated method from BaseMetadataTable [iceberg]

ajantha-bhat commented on code in PR #9298: URL: https://github.com/apache/iceberg/pull/9298#discussion_r1442638780 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/BaseFileRewriteCoordinator.java: ## @@ -72,18 +70,12 @@ public void clearRewrite(Table table, String fil

Re: [PR] Flink: Watermark read options [iceberg]

pvary commented on code in PR #9346: URL: https://github.com/apache/iceberg/pull/9346#discussion_r1442661021 ## flink/v1.18/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceSql.java: ## @@ -39,4 +62,76 @@ public void before() throws IOException { .ge

Re: [PR] Flink: Watermark read options [iceberg]

pvary commented on code in PR #9346: URL: https://github.com/apache/iceberg/pull/9346#discussion_r1442664555 ## flink/v1.18/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceSql.java: ## @@ -39,4 +62,76 @@ public void before() throws IOException { .ge

[I] why Iceberg Spark add_files procedure does not support parquet file with parquet-mr 1.10 version? [iceberg]

hbpeng0115 opened a new issue, #9418: URL: https://github.com/apache/iceberg/issues/9418 ### Query engine Spark ### Question Hello, we use Spark 3.2 + Iceberg 1.4.3 call procedure to add S3 parquet files to existed Iceberg table. However, we find parquet files with parqu

Re: [I] Pruning partitions using filter when reading file. [iceberg-rust]

liurenjie1024 commented on issue #126: URL: https://github.com/apache/iceberg-rust/issues/126#issuecomment-1878483283 Blocked by #153 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Flink: Create JUnit5 version of TestFlinkScan [iceberg]

pvary commented on PR #9185: URL: https://github.com/apache/iceberg/pull/9185#issuecomment-1878512144 > > Do we have a way to check if the new tests were running on the CI? With the correct parameters? > > I don't think so, but I was verifying them all locally In this case, I a

[PR] API, Core: Fix errorprone warnings [iceberg]

ajantha-bhat opened a new pull request, #9419: URL: https://github.com/apache/iceberg/pull/9419 `./gradlew clean build -x test` is reporting some warnings as seen below. https://github.com/apache/iceberg/assets/5889404/e64c76c4-29d3-42cf-b3e0-c1d9f847200b";> After this PR, `./g

Re: [PR] API, Core: Fix errorprone warnings [iceberg]

ajantha-bhat commented on PR #9419: URL: https://github.com/apache/iceberg/pull/9419#issuecomment-1878529981 cc: @aokolnychyi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] API, Core: Fix errorprone warnings [iceberg]

ajantha-bhat commented on code in PR #9419: URL: https://github.com/apache/iceberg/pull/9419#discussion_r1442771024 ## api/src/main/java/org/apache/iceberg/util/CharSequenceSet.java: ## @@ -166,6 +166,7 @@ public void clear() { wrapperSet.clear(); } + @SuppressWarning

Re: [PR] open-api: Use openapi-generator-gradle-plugin for validating specification [iceberg]

Fokko commented on code in PR #9344: URL: https://github.com/apache/iceberg/pull/9344#discussion_r1442774264 ## open-api/rest-catalog-open-api.yaml: ## @@ -2568,26 +2568,26 @@ components: propertyName: type mapping: assert-view-uuid: '#/components/sc

Re: [I] Allow snapshotting iceberg table (create new table based on certain Iceberg snapshot) [iceberg]

YehorKrivokon commented on issue #2481: URL: https://github.com/apache/iceberg/issues/2481#issuecomment-1878533755 Hi all, I'm trying to use Iceberg with Spark and SparkCatalog and I've a repro of this issue. Is there any workaround to work with snapshots? Thank you. -- This

[PR] AWS: Add Option to don't write non current columns in glue schema closes #7584 [iceberg]

Raphael-Vignes opened a new pull request, #9420: URL: https://github.com/apache/iceberg/pull/9420 This PR aims to close this [issue](https://github.com/apache/iceberg/issues/7584) (and would resolve this [issue](https://github.com/apache/iceberg/issues/6340) too. We want to provide a

Re: [PR] Spark: Support dropping views [iceberg]

nastra commented on code in PR #9421: URL: https://github.com/apache/iceberg/pull/9421#discussion_r1442789176 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveViews.scala: ## @@ -111,8 +116,8 @@ case class ResolveViews(spark: SparkSessi

Re: [PR] Build: Bump jetty from 9.4.53.v20231009 to 11.0.19 [iceberg]

nastra closed pull request #9370: Build: Bump jetty from 9.4.53.v20231009 to 11.0.19 URL: https://github.com/apache/iceberg/pull/9370 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] Build: Bump jetty from 9.4.53.v20231009 to 11.0.19 [iceberg]

dependabot[bot] commented on PR #9370: URL: https://github.com/apache/iceberg/pull/9370#issuecomment-1878554815 OK, I won't notify you again about this release, but will get in touch when a new version is available. You can also ignore all major, minor, or patch releases for a dependency by

Re: [PR] Build: Bump com.google.errorprone:error_prone_annotations from 2.23.0 to 2.24.0 [iceberg]

nastra merged PR #9369: URL: https://github.com/apache/iceberg/pull/9369 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Build: Bump spring-boot from 2.5.4 to 3.2.1 [iceberg]

nastra commented on PR #9371: URL: https://github.com/apache/iceberg/pull/9371#issuecomment-1878558251 Looks like `iceberg-aliyun` uses this for testing. I don't think we can upgrade to the newer version, since that requires JDK 11 -- This is an automated message from the Apache Git Servi

Re: [PR] Build: Bump spring-boot from 2.5.4 to 3.2.1 [iceberg]

dependabot[bot] commented on PR #9371: URL: https://github.com/apache/iceberg/pull/9371#issuecomment-1878558298 OK, I won't notify you again about this release, but will get in touch when a new version is available. You can also ignore all major, minor, or patch releases for a dependency by

Re: [PR] Build: Bump spring-boot from 2.5.4 to 3.2.1 [iceberg]

nastra closed pull request #9371: Build: Bump spring-boot from 2.5.4 to 3.2.1 URL: https://github.com/apache/iceberg/pull/9371 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] Flink: Create JUnit5 version of TestFlinkScan [iceberg]

nastra commented on PR #9185: URL: https://github.com/apache/iceberg/pull/9185#issuecomment-1878559904 @cgpoh can you please port these changes to other Flink versions? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [PR] Flink 1.17: Create JUnit5 version of TestFlinkScan [iceberg]

nastra merged PR #9185: URL: https://github.com/apache/iceberg/pull/9185 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [I] Create JUnit5-version of TestFlinkScan [iceberg]

nastra closed issue #9077: Create JUnit5-version of TestFlinkScan URL: https://github.com/apache/iceberg/issues/9077 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

Re: [PR] Nessie: Strip trailing slash for warehouse location [iceberg]

nastra merged PR #9415: URL: https://github.com/apache/iceberg/pull/9415 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Adding Junit 5 conversion and AssertJ style for TestFlinkCatalogTable… [iceberg]

nastra commented on PR #9381: URL: https://github.com/apache/iceberg/pull/9381#issuecomment-1878626062 @vinitpatni the issue with `GenericAppenderHelper` has been addressed in https://github.com/apache/iceberg/pull/9185, so this class should now be usable with JUnit4 + JUni5 -- This is a

Re: [PR] Build: Bump com.palantir.baseline:gradle-baseline-java from 4.42.0 to 5.31.0 [iceberg]

dependabot[bot] commented on PR #9041: URL: https://github.com/apache/iceberg/pull/9041#issuecomment-1878626952 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let

Re: [PR] Build: Bump com.palantir.baseline:gradle-baseline-java from 4.42.0 to 5.31.0 [iceberg]

nastra closed pull request #9041: Build: Bump com.palantir.baseline:gradle-baseline-java from 4.42.0 to 5.31.0 URL: https://github.com/apache/iceberg/pull/9041 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Spark 3.5: Remaining tests migrated to Junit5 [iceberg]

nastra commented on code in PR #9417: URL: https://github.com/apache/iceberg/pull/9417#discussion_r1442849817 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestChangelogIterator.java: ## @@ -196,10 +196,10 @@ public void testUpdatedRowsWithDuplication() { Itera

[PR] Arrow: Use case instead of wrapping a map/list [iceberg-python]

Fokko opened a new pull request, #252: URL: https://github.com/apache/iceberg-python/pull/252 Wrapping the list seems to introduce an odd behavior where `null` values are converted to an empty list `[]`. Resolves #251 -- This is an automated message from the Apache Git Service. To

Re: [PR] Spark 3.5: Remaining tests migrated to Junit5 [iceberg]

nastra commented on code in PR #9417: URL: https://github.com/apache/iceberg/pull/9417#discussion_r1442860998 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/TestTableSerialization.java: ## @@ -36,29 +39,23 @@ import org.apache.iceberg.relocated.com.google.common.collect.L

Re: [PR] Spark 3.5: Remaining tests migrated to Junit5 [iceberg]

nastra commented on code in PR #9417: URL: https://github.com/apache/iceberg/pull/9417#discussion_r1442861281 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/TestTableSerialization.java: ## @@ -36,29 +39,23 @@ import org.apache.iceberg.relocated.com.google.common.collect.L

Re: [I] Failed to assign splits due to the serialized split size [iceberg]

javrasya commented on issue #9410: URL: https://github.com/apache/iceberg/issues/9410#issuecomment-1878646665 No idea tbh since I run into this on production and there I don't have the ability to go deep and debug. I wouldn't say it is a wide table in terms of number of columns and I

Re: [PR] Spark 3.5: Remaining tests migrated to Junit5 [iceberg]

nastra commented on code in PR #9417: URL: https://github.com/apache/iceberg/pull/9417#discussion_r1442867841 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestFunctionCatalog.java: ## @@ -27,107 +30,106 @@ import org.apache.spark.sql.connector.catalog.FunctionCata

Re: [PR] Spark 3.5: Remaining tests migrated to Junit5 [iceberg]

nastra commented on code in PR #9417: URL: https://github.com/apache/iceberg/pull/9417#discussion_r1442872510 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestFunctionCatalog.java: ## @@ -27,107 +30,106 @@ import org.apache.spark.sql.connector.catalog.FunctionCata

Re: [PR] Spark 3.5: Remaining tests migrated to Junit5 [iceberg]

nastra commented on PR #9417: URL: https://github.com/apache/iceberg/pull/9417#issuecomment-1878660248 @chinmay-bhat this is almost ready to go, just a few minor things to address -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] Spark 3.5: Migrate tests in SQL directory to JUnit5 [iceberg]

nastra commented on code in PR #9401: URL: https://github.com/apache/iceberg/pull/9401#discussion_r1442885901 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/sql/PartitionedWritesTestBase.java: ## @@ -18,53 +18,49 @@ */ package org.apache.iceberg.spark.sql; +impo

Re: [PR] Spark 3.5: Set log level to WARN for rewrite task failure with partial progress [iceberg]

manuzhang commented on code in PR #9400: URL: https://github.com/apache/iceberg/pull/9400#discussion_r1442954829 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RewriteDataFilesSparkAction.java: ## @@ -345,7 +345,7 @@ private Result doExecuteWithPartialProgres

[PR] Fix parquet default compression codec [iceberg-docs]

manuzhang opened a new pull request, #305: URL: https://github.com/apache/iceberg-docs/pull/305 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] Spark 3.5: Remaining tests migrated to Junit5 [iceberg]

chinmay-bhat commented on PR #9417: URL: https://github.com/apache/iceberg/pull/9417#issuecomment-1878814101 Thanks for the review, I've updated the PR with the suggestions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] Failed to assign splits due to the serialized split size [iceberg]

pvary commented on issue #9410: URL: https://github.com/apache/iceberg/issues/9410#issuecomment-1878817631 @javrasya: This is table should not be too wide, and the statistics should be limited as well (unless you did some specific tweaking there). My best guess is your first suggestion:

Re: [PR] Spark 3.5: Migrate tests in SQL directory to JUnit5 [iceberg]

chinmay-bhat commented on code in PR #9401: URL: https://github.com/apache/iceberg/pull/9401#discussion_r1442989187 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/sql/PartitionedWritesTestBase.java: ## @@ -18,53 +18,49 @@ */ package org.apache.iceberg.spark.sql;

Re: [PR] Flink 1.17: Create JUnit5 version of TestFlinkScan [iceberg]

cgpoh commented on PR #9185: URL: https://github.com/apache/iceberg/pull/9185#issuecomment-1878844200 > @cgpoh can you please port these changes to other Flink versions? Sure. Will try to port these changes next week. -- This is an automated message from the Apache Git Service. To r

Re: [PR] Parquet: Support reading INT96 column in row group filter [iceberg]

nastra commented on code in PR #8988: URL: https://github.com/apache/iceberg/pull/8988#discussion_r1443018102 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestIcebergSourceTablesBase.java: ## @@ -2181,20 +2181,28 @@ public void testTableWithInt96Timestamp()

Re: [PR] Core: Create JUnit5 version of TableTestBase [iceberg]

nastra commented on code in PR #9217: URL: https://github.com/apache/iceberg/pull/9217#discussion_r1443020124 ## core/src/test/java/org/apache/iceberg/TestCreateSnapshotEvent.java: ## @@ -40,76 +41,76 @@ public TestCreateSnapshotEvent(int formatVersion) { Listeners.register

Re: [PR] Parquet: Support reading INT96 column in row group filter [iceberg]

manuzhang commented on code in PR #8988: URL: https://github.com/apache/iceberg/pull/8988#discussion_r1443027055 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestIcebergSourceTablesBase.java: ## @@ -2181,20 +2181,28 @@ public void testTableWithInt96Timestamp

Re: [PR] Core: Create JUnit5 version of TableTestBase [iceberg]

nastra commented on code in PR #9217: URL: https://github.com/apache/iceberg/pull/9217#discussion_r1443033477 ## core/src/test/java/org/apache/iceberg/TestCreateSnapshotEvent.java: ## @@ -40,76 +41,76 @@ public TestCreateSnapshotEvent(int formatVersion) { Listeners.register

Re: [PR] Adding Junit 5 conversion and AssertJ style for TestFlinkCatalogTable… [iceberg]

vinitpatni commented on PR #9381: URL: https://github.com/apache/iceberg/pull/9381#issuecomment-1878891716 I have done changes in TestStreamScanSql locally but running testcases on my local machine is taking lot of time. It is running indefinitely for this class. @nastra Do you have any ide

Re: [PR] Spark 3.5: Remaining tests migrated to Junit5 [iceberg]

nastra merged PR #9417: URL: https://github.com/apache/iceberg/pull/9417 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Spark 3.4: Add support for reading Iceberg views [iceberg]

amogh-jahagirdar commented on PR #9422: URL: https://github.com/apache/iceberg/pull/9422#issuecomment-1878930402 Since this was a clean backport. I'll go ahead and merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Spark 3.4: Add support for reading Iceberg views [iceberg]

amogh-jahagirdar merged PR #9422: URL: https://github.com/apache/iceberg/pull/9422 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Parquet: Support reading INT96 column in row group filter [iceberg]

manuzhang commented on code in PR #8988: URL: https://github.com/apache/iceberg/pull/8988#discussion_r1443085007 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestIcebergSourceTablesBase.java: ## @@ -2181,20 +2181,28 @@ public void testTableWithInt96Timestamp

Re: [PR] Parquet: Support reading INT96 column in row group filter [iceberg]

manuzhang commented on code in PR #8988: URL: https://github.com/apache/iceberg/pull/8988#discussion_r1443085007 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestIcebergSourceTablesBase.java: ## @@ -2181,20 +2181,28 @@ public void testTableWithInt96Timestamp

Re: [PR] Core: Create JUnit5 version of TableTestBase [iceberg]

nastra commented on code in PR #9217: URL: https://github.com/apache/iceberg/pull/9217#discussion_r1443105597 ## core/src/test/java/org/apache/iceberg/TestManifestReader.java: ## @@ -32,17 +32,15 @@ import org.apache.iceberg.types.Types; import org.assertj.core.api.Assertions;

[PR] Core: Add JUnit5 version of TableTestBase [iceberg]

nastra opened a new pull request, #9424: URL: https://github.com/apache/iceberg/pull/9424 This fixes https://github.com/apache/iceberg/issues/9073 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Core: Create JUnit5 version of TableTestBase [iceberg]

nastra commented on code in PR #9217: URL: https://github.com/apache/iceberg/pull/9217#discussion_r1443109201 ## core/src/test/java/org/apache/iceberg/TestManifestReader.java: ## @@ -32,17 +32,15 @@ import org.apache.iceberg.types.Types; import org.assertj.core.api.Assertions;

Re: [PR] API, Core: Add Schema#withUpdatedDoc and View#updateColumnDoc APIs [iceberg]

rdblue commented on PR #9414: URL: https://github.com/apache/iceberg/pull/9414#issuecomment-1878993066 @amogh-jahagirdar, I agree with not adding a `with` method to modify a schema. If we can avoid making that kind of API addition I would prefer it. I'm also not sure that we want to m

Re: [PR] API, Core: Add Schema#withUpdatedDoc and View#updateColumnDoc APIs [iceberg]

rdblue closed pull request #9414: API, Core: Add Schema#withUpdatedDoc and View#updateColumnDoc APIs URL: https://github.com/apache/iceberg/pull/9414 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Flink: Watermark read options [iceberg]

stevenzwu commented on code in PR #9346: URL: https://github.com/apache/iceberg/pull/9346#discussion_r1443084956 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/FlinkReadOptions.java: ## @@ -109,4 +110,13 @@ private FlinkReadOptions() {} public static final String

Re: [PR] API, Core: Add Schema#withUpdatedDoc and View#updateColumnDoc APIs [iceberg]

rdblue commented on PR #9414: URL: https://github.com/apache/iceberg/pull/9414#issuecomment-1879017514 Oops, I didn't intend to close this! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] Parquet: Move to ValueReader generation to a visitor [iceberg]

amogh-jahagirdar commented on code in PR #9063: URL: https://github.com/apache/iceberg/pull/9063#discussion_r1443148176 ## parquet/src/main/java/org/apache/iceberg/data/parquet/BaseParquetReaders.java: ## @@ -108,6 +110,113 @@ public ParquetValueReader struct( } } + p

Re: [PR] Spark: Support dropping views [iceberg]

rdblue commented on code in PR #9421: URL: https://github.com/apache/iceberg/pull/9421#discussion_r1443152002 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/parser/extensions/IcebergSparkSqlExtensionsParser.scala: ## @@ -122,7 +128,27 @@ class Iceber

Re: [PR] Spark: Support dropping views [iceberg]

rdblue commented on code in PR #9421: URL: https://github.com/apache/iceberg/pull/9421#discussion_r1443153917 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/parser/extensions/IcebergSparkSqlExtensionsParser.scala: ## @@ -122,7 +128,27 @@ class Iceber

Re: [PR] Spark: Support dropping views [iceberg]

rdblue commented on code in PR #9421: URL: https://github.com/apache/iceberg/pull/9421#discussion_r1443156571 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DropV2ViewExec.scala: ## @@ -0,0 +1,49 @@ +/* + * Licensed to the Apache Soft

Re: [PR] Parquet: Move to ValueReader generation to a visitor [iceberg]

amogh-jahagirdar commented on code in PR #9063: URL: https://github.com/apache/iceberg/pull/9063#discussion_r1443148385 ## parquet/src/main/java/org/apache/iceberg/data/parquet/BaseParquetReaders.java: ## @@ -108,6 +110,113 @@ public ParquetValueReader struct( } } + p

Re: [I] Spark: Session level Iceberg table config defaults [iceberg]

kuldeepsinghchauhan commented on issue #2270: URL: https://github.com/apache/iceberg/issues/2270#issuecomment-1879091855 This is a very critical feature, In an actual production scenario where we are planning to rollout the iceberg format for operations and analytics user. we are currently

Re: [PR] Spark: Support dropping views [iceberg]

rdblue commented on code in PR #9421: URL: https://github.com/apache/iceberg/pull/9421#discussion_r1443226670 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestViews.java: ## @@ -635,6 +633,51 @@ private Catalog tableCatalog() { return Spa

Re: [PR] Spark: Support dropping views [iceberg]

rdblue commented on code in PR #9421: URL: https://github.com/apache/iceberg/pull/9421#discussion_r1443227383 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestViews.java: ## @@ -635,6 +633,51 @@ private Catalog tableCatalog() { return Spa

Re: [PR] Spark: Support dropping views [iceberg]

rdblue commented on code in PR #9421: URL: https://github.com/apache/iceberg/pull/9421#discussion_r1443275368 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveViews.scala: ## @@ -45,14 +47,17 @@ case class ResolveViews(spark: SparkSessi

Re: [I] Failed to assign splits due to the serialized split size [iceberg]

javrasya commented on issue #9410: URL: https://github.com/apache/iceberg/issues/9410#issuecomment-1879135064 @pvary I wasn't aware of `splitOpenFileCost`, thank you for sharing that. The way how I did it is that I introduced my own SplitAssignerFactory and SplitAssigner and pass tha

Re: [PR] Fix parquet default compression codec in 1.4.0 [iceberg-docs]

Fokko merged PR #301: URL: https://github.com/apache/iceberg-docs/pull/301 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Fix parquet default compression codec in 1.4.1 [iceberg-docs]

Fokko merged PR #302: URL: https://github.com/apache/iceberg-docs/pull/302 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Fix parquet default compression codec in 1.4.3 [iceberg-docs]

Fokko merged PR #304: URL: https://github.com/apache/iceberg-docs/pull/304 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Fix parquet default compression codec in 1.4.2 [iceberg-docs]

Fokko merged PR #303: URL: https://github.com/apache/iceberg-docs/pull/303 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Fix parquet default compression codec [iceberg-docs]

Fokko merged PR #305: URL: https://github.com/apache/iceberg-docs/pull/305 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Fix PR link in 1.4.3 release notes [iceberg-docs]

Fokko merged PR #299: URL: https://github.com/apache/iceberg-docs/pull/299 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] docs: Add link for iceberg rust [iceberg-docs]

Fokko merged PR #300: URL: https://github.com/apache/iceberg-docs/pull/300 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] API: Fix day partition transform result type [iceberg]

tdcmeehan commented on PR #9345: URL: https://github.com/apache/iceberg/pull/9345#issuecomment-1879158054 It seems the consensus is it's more straightforward to keep this as `int`, so I will close this. Thanks folks! -- This is an automated message from the Apache Git Service. To respond

Re: [PR] API: Fix day partition transform result type [iceberg]

tdcmeehan closed pull request #9345: API: Fix day partition transform result type URL: https://github.com/apache/iceberg/pull/9345 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] Adding Snowflake's public documentation [iceberg-docs]

Fokko commented on PR #297: URL: https://github.com/apache/iceberg-docs/pull/297#issuecomment-1879159046 @scottteal sorry for the late reply here, could you fix the conflicts once more? -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

[I] How can we build self-defined iceberg-spark-runtime jar package? [iceberg]

gaoshihang opened a new issue, #9425: URL: https://github.com/apache/iceberg/issues/9425 ### Query engine Spark ### Question How can we build self-defined iceberg-spark-runtime jar package? -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Adding Snowflake's public documentation [iceberg-docs]

scottteal commented on PR #297: URL: https://github.com/apache/iceberg-docs/pull/297#issuecomment-1879185914 @Fokko I think I've now resolved the conflicts. I tried to preserve the ordering to match the current website as much as possible, although it's still unclear to me what the ordering

Re: [PR] Parquet: Move to ValueReader generation to a visitor [iceberg]

Fokko commented on code in PR #9063: URL: https://github.com/apache/iceberg/pull/9063#discussion_r1443316711 ## parquet/src/main/java/org/apache/iceberg/data/parquet/BaseParquetReaders.java: ## @@ -108,6 +110,113 @@ public ParquetValueReader struct( } } + private clas

[I] Add the homepage to the `Cargo.toml` [iceberg-rust]

Fokko opened a new issue, #154: URL: https://github.com/apache/iceberg-rust/issues/154 Looks like we can add the homepage as well: https://doc.rust-lang.org/cargo/reference/manifest.html#the-homepage-field To our newly created https://rust.iceberg.apache.org/ -- This is an auto

Re: [PR] Replace black by Ruff Formatter [iceberg-python]

Fokko merged PR #127: URL: https://github.com/apache/iceberg-python/pull/127 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Shift site build to use monorepo and gh-pages [iceberg]

Fokko commented on code in PR #8919: URL: https://github.com/apache/iceberg/pull/8919#discussion_r1443340509 ## site/docs/community.md: ## @@ -40,13 +40,13 @@ Issues are tracked in GitHub: ## Slack -We use the [Apache Iceberg workspace](https://apache-iceberg.slack.com/) on

Re: [PR] Shift site build to use monorepo and gh-pages [iceberg]

bitsondatadev commented on code in PR #8919: URL: https://github.com/apache/iceberg/pull/8919#discussion_r1443356487 ## site/docs/community.md: ## @@ -40,13 +40,13 @@ Issues are tracked in GitHub: ## Slack -We use the [Apache Iceberg workspace](https://apache-iceberg.slack.

Re: [PR] API, Core: Add Schema#withUpdatedDoc and View#updateColumnDoc APIs [iceberg]

amogh-jahagirdar commented on PR #9414: URL: https://github.com/apache/iceberg/pull/9414#issuecomment-1879243149 > That line of reasoning makes me think that any change (even comments) should just produce a new version of the view. Yes, that's the approach in this PR currently. A new

Re: [PR] Shift site build to use monorepo and gh-pages [iceberg]

bitsondatadev commented on code in PR #8919: URL: https://github.com/apache/iceberg/pull/8919#discussion_r1443359408 ## site/nav.yml: ## @@ -0,0 +1,49 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file

Re: [PR] Shift site build to use monorepo and gh-pages [iceberg]

bitsondatadev commented on code in PR #8919: URL: https://github.com/apache/iceberg/pull/8919#discussion_r1443361277 ## site/docs/community.md: ## @@ -40,13 +40,13 @@ Issues are tracked in GitHub: ## Slack -We use the [Apache Iceberg workspace](https://apache-iceberg.slack.

Re: [I] How can we build self-defined iceberg-spark-runtime jar package? [iceberg]

gaoshihang closed issue #9425: How can we build self-defined iceberg-spark-runtime jar package? URL: https://github.com/apache/iceberg/issues/9425 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

  1   2   >