Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

nastra commented on code in PR #9342: URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432357336 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestSparkReadProjection.java: ## @@ -110,7 +110,7 @@ public static void stopSpark() { @Override p

Re: [I] Java calls API error [iceberg]

nastra commented on issue #9339: URL: https://github.com/apache/iceberg/issues/9339#issuecomment-1863991199 `GenericParquetWriter` is in `iceberg-data`, so you would have to add that dependency. -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [I] Java calls API error [iceberg]

smileyboy2019 commented on issue #9339: URL: https://github.com/apache/iceberg/issues/9339#issuecomment-1863977161  Version 1.4.2, using Java API developed by iceberg+minio -- This is an aut

Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

chinmay-bhat commented on PR #9342: URL: https://github.com/apache/iceberg/pull/9342#issuecomment-1863976054 Thanks for the review. Yes, I assumed the convention was the other way. Will update all the files. -- This is an automated message from the Apache Git Service. To respond to the me

Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

chinmay-bhat commented on code in PR #9342: URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432347207 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestSparkReadProjection.java: ## @@ -110,7 +110,7 @@ public static void stopSpark() { @Overrid

Re: [PR] Flink: implement pause and resume in IcebergSourceSplitReader [iceberg]

nastra commented on code in PR #9308: URL: https://github.com/apache/iceberg/pull/9308#discussion_r1432345821 ## flink/v1.18/flink/src/test/java/org/apache/iceberg/flink/source/reader/TestIcebergSourceSplitReader.java: ## @@ -0,0 +1,170 @@ +/* + * Licensed to the Apache Software

Re: [PR] Flink: implement pause and resume in IcebergSourceSplitReader [iceberg]

nastra commented on code in PR #9308: URL: https://github.com/apache/iceberg/pull/9308#discussion_r1432345821 ## flink/v1.18/flink/src/test/java/org/apache/iceberg/flink/source/reader/TestIcebergSourceSplitReader.java: ## @@ -0,0 +1,170 @@ +/* + * Licensed to the Apache Software

Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

nastra commented on code in PR #9342: URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432341279 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestStreamingOffset.java: ## @@ -35,10 +36,10 @@ public void testJsonConversion() { new Stre

Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

nastra commented on code in PR #9342: URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432340649 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestSparkStagedScan.java: ## @@ -111,7 +111,9 @@ public void testTaskSetPlanning() throws NoSuchTableE

Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

nastra commented on code in PR #9342: URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432340253 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestSparkReadProjection.java: ## @@ -110,7 +110,7 @@ public static void stopSpark() { @Override p

Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

chinmay-bhat commented on code in PR #9342: URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432338113 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestDataSourceOptions.java: ## @@ -297,7 +299,7 @@ public void testIncrementalScanOptions() throw

Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

nastra commented on code in PR #9342: URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432336655 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestReadProjection.java: ## @@ -64,12 +65,13 @@ public void testFullProjection() throws Exception {

Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

nastra commented on code in PR #9342: URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432337147 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestSparkAggregates.java: ## @@ -50,27 +51,34 @@ public void testAggregates() { Max max = ne

Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

nastra commented on code in PR #9342: URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432336282 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestPathIdentifier.java: ## @@ -75,12 +74,11 @@ public void testPathIdentifier() throws TableAlreadyEx

Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

nastra commented on code in PR #9342: URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432334842 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestIcebergSpark.java: ## @@ -59,83 +60,78 @@ public static void stopSpark() { public void testRegis

Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

nastra commented on code in PR #9342: URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432327788 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestDataSourceOptions.java: ## @@ -297,7 +299,7 @@ public void testIncrementalScanOptions() throws IOEx

Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

nastra commented on code in PR #9342: URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432326205 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestDataSourceOptions.java: ## @@ -219,12 +219,14 @@ public void testSplitOptionsOverridesTableProperti

Re: [I] iceberg-data: Switch tests to JUnit5 + AssertJ-style assertions [iceberg]

igoradulian commented on issue #9082: URL: https://github.com/apache/iceberg/issues/9082#issuecomment-1863915544 Hi @nastra, can I be assigned to this issue? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] Flink: implement range partitioner for map data statistics [iceberg]

stevenzwu commented on code in PR #9321: URL: https://github.com/apache/iceberg/pull/9321#discussion_r1432236943 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/MapRangePartitioner.java: ## @@ -0,0 +1,288 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] Spark: IN clause on system function is not pushed down [iceberg]

wypoon commented on PR #9192: URL: https://github.com/apache/iceberg/pull/9192#issuecomment-1863749851 @rdblue @szehon-ho @RussellSpitzer can you please review this? Also, can you please enable the CI for this PR? -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

chinmay-bhat commented on code in PR #9342: URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432169104 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestAvroScan.java: ## @@ -41,25 +43,24 @@ import org.apache.spark.sql.Dataset; import org.apach

Re: [I] Support Parquet v2 Spark vectorized read [iceberg]

wgtmac commented on issue #7162: URL: https://github.com/apache/iceberg/issues/7162#issuecomment-1863692315 @jackye1995 Do you have any ETA on this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] Flink: Create JUnit5 version of TestFlinkScan [iceberg]

cgpoh commented on PR #9185: URL: https://github.com/apache/iceberg/pull/9185#issuecomment-1863682637 > @cgpoh #9161 was just merged, can you please rebase this PR? @nastra Thanks, will do that. -- This is an automated message from the Apache Git Service. To respond to the message,

[PR] Core: Add ApplyNameMapping for Avro [iceberg]

rdblue opened a new pull request, #9347: URL: https://github.com/apache/iceberg/pull/9347 This separates the logic to apply a name mapping from the logic to prune columns in Avro. This is some cleanup since name mapping shouldn't be mixed with column pruning. -- This is an automated mess

Re: [I] 🐞Default table location may cause table to be lost [iceberg]

github-actions[bot] commented on issue #7052: URL: https://github.com/apache/iceberg/issues/7052#issuecomment-1863642452 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] 🐞Default table location may cause table to be lost [iceberg]

github-actions[bot] closed issue #7052: 🐞Default table location may cause table to be lost URL: https://github.com/apache/iceberg/issues/7052 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [I] Cannot delete column with nested field filter [iceberg]

github-actions[bot] commented on issue #7065: URL: https://github.com/apache/iceberg/issues/7065#issuecomment-1863642431 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Cannot delete column with nested field filter [iceberg]

github-actions[bot] closed issue #7065: Cannot delete column with nested field filter URL: https://github.com/apache/iceberg/issues/7065 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] WIP: first pass at `UnboundTransform` [iceberg-python]

jayceslesar commented on PR #209: URL: https://github.com/apache/iceberg-python/pull/209#issuecomment-1863579944 No worries on the speed of this one, I know you are heads down on getting write support -- This is an automated message from the Apache Git Service. To respond to the message,

[PR] Build: Bump cython from 3.0.6 to 3.0.7 [iceberg-python]

dependabot[bot] opened a new pull request, #228: URL: https://github.com/apache/iceberg-python/pull/228 Bumps [cython](https://github.com/cython/cython) from 3.0.6 to 3.0.7. Changelog Sourced from https://github.com/cython/cython/blob/master/CHANGES.rst";>cython's changelog.

Re: [PR] API, Core, Spark 3.5: Parallelize reading of deletes and cache them on executors [iceberg]

aokolnychyi commented on code in PR #8755: URL: https://github.com/apache/iceberg/pull/8755#discussion_r1431972974 ## core/src/main/java/org/apache/iceberg/SystemConfigs.java: ## @@ -43,14 +43,14 @@ private SystemConfigs() {} Integer::parseUnsignedInt); /** - *

Re: [PR] open-api: Use openapi-generator-gradle-plugin for validating specification [iceberg]

dramaticlly commented on PR #9344: URL: https://github.com/apache/iceberg/pull/9344#issuecomment-1863493846 @Fokko @nastra if you want to take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] File length in AES GCM Stream Spec [iceberg]

rdblue merged PR #9136: URL: https://github.com/apache/iceberg/pull/9136 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Flink: implement pause and resume in IcebergSourceSplitReader [iceberg]

mas-chen commented on code in PR #9308: URL: https://github.com/apache/iceberg/pull/9308#discussion_r1431884906 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergSourceSplitReader.java: ## @@ -112,8 +149,20 @@ public void handleSplitsChanges(Split

Re: [PR] Flink: implement pause and resume in IcebergSourceSplitReader [iceberg]

mas-chen commented on code in PR #9308: URL: https://github.com/apache/iceberg/pull/9308#discussion_r1431879530 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergSourceSplitReader.java: ## @@ -112,8 +149,20 @@ public void handleSplitsChanges(Split

Re: [PR] Spark: Add support for reading Iceberg views [iceberg]

rdblue commented on code in PR #9340: URL: https://github.com/apache/iceberg/pull/9340#discussion_r1431863233 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestViews.java: ## @@ -0,0 +1,152 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] Flink: implement pause and resume in IcebergSourceSplitReader [iceberg]

stevenzwu commented on code in PR #9308: URL: https://github.com/apache/iceberg/pull/9308#discussion_r1431861322 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergSourceSplitReader.java: ## @@ -46,6 +51,8 @@ class IcebergSourceSplitReader implement

Re: [PR] Spark: Add support for reading Iceberg views [iceberg]

rdblue commented on code in PR #9340: URL: https://github.com/apache/iceberg/pull/9340#discussion_r1431863233 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestViews.java: ## @@ -0,0 +1,152 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] Spark: Add support for reading Iceberg views [iceberg]

rdblue commented on code in PR #9340: URL: https://github.com/apache/iceberg/pull/9340#discussion_r1431861828 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/logicalOperators.scala: ## @@ -0,0 +1,131 @@ +/* + * Licensed to the Apache Sof

Re: [PR] Spark: Add support for reading Iceberg views [iceberg]

rdblue commented on code in PR #9340: URL: https://github.com/apache/iceberg/pull/9340#discussion_r1431861467 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/logicalOperators.scala: ## @@ -0,0 +1,131 @@ +/* + * Licensed to the Apache Sof

Re: [PR] Spark: IN clause on system function is not pushed down [iceberg]

wypoon commented on code in PR #9192: URL: https://github.com/apache/iceberg/pull/9192#discussion_r1431856870 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ReplaceStaticInvoke.scala: ## @@ -56,7 +69,6 @@ object ReplaceStaticInvoke extends

Re: [PR] Flink: implement pause and resume in IcebergSourceSplitReader [iceberg]

stevenzwu commented on code in PR #9308: URL: https://github.com/apache/iceberg/pull/9308#discussion_r1431856035 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergSourceSplitReader.java: ## @@ -80,6 +88,35 @@ public RecordsWithSplitIds> fetch() th

Re: [PR] Spark: Add support for reading Iceberg views [iceberg]

rdblue commented on code in PR #9340: URL: https://github.com/apache/iceberg/pull/9340#discussion_r1431854785 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestViews.java: ## @@ -0,0 +1,152 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] Spark: Add support for reading Iceberg views [iceberg]

rdblue commented on code in PR #9340: URL: https://github.com/apache/iceberg/pull/9340#discussion_r1431853526 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestSparkCatalog.java: ## @@ -31,8 +31,10 @@ import org.apache.spark.sql.connector.catalog.SupportsNam

Re: [PR] Spark: Add support for reading Iceberg views [iceberg]

rdblue commented on code in PR #9340: URL: https://github.com/apache/iceberg/pull/9340#discussion_r1431850234 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/SparkTestBaseWithCatalog.java: ## @@ -87,7 +87,7 @@ public SparkTestBaseWithCatalog( config.forEach(

Re: [PR] Spark: Add support for reading Iceberg views [iceberg]

rdblue commented on code in PR #9340: URL: https://github.com/apache/iceberg/pull/9340#discussion_r1431849001 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkView.java: ## @@ -0,0 +1,133 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] Spark: Add support for reading Iceberg views [iceberg]

rdblue commented on code in PR #9340: URL: https://github.com/apache/iceberg/pull/9340#discussion_r1431848699 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkView.java: ## @@ -0,0 +1,133 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] Spark: Add support for reading Iceberg views [iceberg]

rdblue commented on code in PR #9340: URL: https://github.com/apache/iceberg/pull/9340#discussion_r1431846400 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkView.java: ## @@ -0,0 +1,133 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] Spark: Add support for reading Iceberg views [iceberg]

rdblue commented on code in PR #9340: URL: https://github.com/apache/iceberg/pull/9340#discussion_r1431845174 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkView.java: ## @@ -0,0 +1,133 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[PR] Arrow: Set field-id with prefix [iceberg-python]

Fokko opened a new pull request, #227: URL: https://github.com/apache/iceberg-python/pull/227 While working on the write support, I started to understand the meaning of the Arrow metadata. The `PARQUET:` prefix means that it is specific to Parquet, and setting `PARQUET:field_id` will actual

Re: [PR] Spark: Add support for reading Iceberg views [iceberg]

rdblue commented on code in PR #9340: URL: https://github.com/apache/iceberg/pull/9340#discussion_r1431846020 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkView.java: ## @@ -0,0 +1,133 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] Spark: Add support for reading Iceberg views [iceberg]

rdblue commented on code in PR #9340: URL: https://github.com/apache/iceberg/pull/9340#discussion_r1431842806 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkView.java: ## @@ -0,0 +1,133 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] Flink: Support watermark alignment of source splits [iceberg]

pvary commented on code in PR #9308: URL: https://github.com/apache/iceberg/pull/9308#discussion_r1431841694 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergSourceSplitReader.java: ## @@ -80,6 +88,35 @@ public RecordsWithSplitIds> fetch() throws

Re: [PR] Spark: Add support for reading Iceberg views [iceberg]

rdblue commented on code in PR #9340: URL: https://github.com/apache/iceberg/pull/9340#discussion_r1431839691 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveViews.scala: ## @@ -0,0 +1,156 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] Spark: Add support for reading Iceberg views [iceberg]

rdblue commented on code in PR #9340: URL: https://github.com/apache/iceberg/pull/9340#discussion_r1431834246 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveViews.scala: ## @@ -0,0 +1,156 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] Spark: Add support for reading Iceberg views [iceberg]

rdblue commented on code in PR #9340: URL: https://github.com/apache/iceberg/pull/9340#discussion_r1431833263 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveViews.scala: ## @@ -0,0 +1,156 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] Spark: Add support for reading Iceberg views [iceberg]

rdblue commented on code in PR #9340: URL: https://github.com/apache/iceberg/pull/9340#discussion_r1431831525 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveViews.scala: ## @@ -0,0 +1,156 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] Flink: Support watermark alignment of source splits [iceberg]

pvary commented on code in PR #9308: URL: https://github.com/apache/iceberg/pull/9308#discussion_r1431830538 ## flink/v1.18/flink/src/test/java/org/apache/iceberg/flink/source/reader/TestIcebergSourceSplitReader.java: ## @@ -0,0 +1,167 @@ +/* + * Licensed to the Apache Software

Re: [PR] Spark: Add support for reading Iceberg views [iceberg]

rdblue commented on code in PR #9340: URL: https://github.com/apache/iceberg/pull/9340#discussion_r1431830077 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveViews.scala: ## @@ -0,0 +1,156 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] Apply Name mapping [iceberg-python]

syun64 commented on PR #219: URL: https://github.com/apache/iceberg-python/pull/219#issuecomment-1863321308 I've discussed extensively with @fokko regarding how we'd like to handle the edge cases, and here's the summary of the logic that I've implemented in the current version: 1. Ha

Re: [PR] Spark: Add support for reading Iceberg views [iceberg]

rdblue commented on code in PR #9340: URL: https://github.com/apache/iceberg/pull/9340#discussion_r1431824437 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveViews.scala: ## @@ -0,0 +1,156 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] Spark: Add support for reading Iceberg views [iceberg]

rdblue commented on code in PR #9340: URL: https://github.com/apache/iceberg/pull/9340#discussion_r1431823108 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveViews.scala: ## @@ -0,0 +1,156 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] Spark: Add support for reading Iceberg views [iceberg]

rdblue commented on code in PR #9340: URL: https://github.com/apache/iceberg/pull/9340#discussion_r1431821372 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/V2ViewDescription.scala: ## @@ -0,0 +1,56 @@ +/* + * Licensed to the Apache Software

Re: [PR] Spark: Add support for reading Iceberg views [iceberg]

rdblue commented on code in PR #9340: URL: https://github.com/apache/iceberg/pull/9340#discussion_r1431819566 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/logicalOperators.scala: ## @@ -0,0 +1,131 @@ +/* + * Licensed to the Apache Sof

Re: [PR] Spark: Add support for reading Iceberg views [iceberg]

rdblue commented on code in PR #9340: URL: https://github.com/apache/iceberg/pull/9340#discussion_r1431818650 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/logicalOperators.scala: ## @@ -0,0 +1,131 @@ +/* + * Licensed to the Apache Sof

Re: [PR] Spark: Add support for reading Iceberg views [iceberg]

rdblue commented on code in PR #9340: URL: https://github.com/apache/iceberg/pull/9340#discussion_r1431816934 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveViews.scala: ## @@ -0,0 +1,156 @@ +/* + * Licensed to the Apache Software Fou

[PR] open-api: Use openapi-generator-gradle-plugin for validating specification [iceberg]

dramaticlly opened a new pull request, #9344: URL: https://github.com/apache/iceberg/pull/9344 Today we have open-api specification validated as part of github actions/workflow when PR was submitted (or by author running make task) This PR tried to use [openapi-generator-gradle-plugi

Re: [PR] Flink: Support watermark alignment of source splits [iceberg]

stevenzwu commented on code in PR #9308: URL: https://github.com/apache/iceberg/pull/9308#discussion_r1431802868 ## flink/v1.18/flink/src/test/java/org/apache/iceberg/flink/source/reader/TestIcebergSourceSplitReader.java: ## @@ -0,0 +1,167 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

nastra commented on PR #9342: URL: https://github.com/apache/iceberg/pull/9342#issuecomment-1863235505 can you please run `./gradlew spotlessApply` to fix any formatting? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Flink: Support watermark alignment of source splits [iceberg]

stevenzwu commented on code in PR #9308: URL: https://github.com/apache/iceberg/pull/9308#discussion_r1431758647 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergSourceSplitReader.java: ## @@ -80,6 +88,35 @@ public RecordsWithSplitIds> fetch() th

Re: [PR] Flink: Support watermark alignment of source splits [iceberg]

stevenzwu commented on code in PR #9308: URL: https://github.com/apache/iceberg/pull/9308#discussion_r1431754664 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergSourceSplitReader.java: ## @@ -46,6 +51,8 @@ class IcebergSourceSplitReader implement

Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

nastra commented on code in PR #9342: URL: https://github.com/apache/iceberg/pull/9342#discussion_r1431752607 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestBaseReader.java: ## @@ -47,14 +49,13 @@ import org.apache.iceberg.relocated.com.google.common.coll

Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

nastra commented on code in PR #9342: URL: https://github.com/apache/iceberg/pull/9342#discussion_r1431751123 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestAvroScan.java: ## @@ -41,25 +43,24 @@ import org.apache.spark.sql.Dataset; import org.apache.spar

Re: [PR] Flink: Support watermark alignment of source splits [iceberg]

stevenzwu commented on code in PR #9308: URL: https://github.com/apache/iceberg/pull/9308#discussion_r1431726956 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergSourceSplitReader.java: ## @@ -80,6 +88,35 @@ public RecordsWithSplitIds> fetch() th

Re: [PR] Flink: Support watermark alignment of source splits [iceberg]

pvary commented on code in PR #9308: URL: https://github.com/apache/iceberg/pull/9308#discussion_r1431743938 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergSourceSplitReader.java: ## @@ -80,6 +88,35 @@ public RecordsWithSplitIds> fetch() throws

Re: [PR] Flink: Support watermark alignment of source splits [iceberg]

pvary commented on code in PR #9308: URL: https://github.com/apache/iceberg/pull/9308#discussion_r1431742741 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergSourceSplitReader.java: ## @@ -46,6 +51,8 @@ class IcebergSourceSplitReader implements S

Re: [PR] Flink: Support watermark alignment of source splits [iceberg]

stevenzwu commented on code in PR #9308: URL: https://github.com/apache/iceberg/pull/9308#discussion_r1431740597 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergSourceSplitReader.java: ## @@ -46,6 +51,8 @@ class IcebergSourceSplitReader implement

Re: [PR] Flink: Support watermark alignment of source splits [iceberg]

pvary commented on code in PR #9308: URL: https://github.com/apache/iceberg/pull/9308#discussion_r1431735881 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergSourceSplitReader.java: ## @@ -80,6 +88,35 @@ public RecordsWithSplitIds> fetch() throws

Re: [PR] Flink: Support watermark alignment of source splits [iceberg]

stevenzwu commented on code in PR #9308: URL: https://github.com/apache/iceberg/pull/9308#discussion_r1431736659 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergSourceSplitReader.java: ## @@ -80,6 +88,35 @@ public RecordsWithSplitIds> fetch() th

Re: [PR] Flink: Support watermark alignment of source splits [iceberg]

pvary commented on code in PR #9308: URL: https://github.com/apache/iceberg/pull/9308#discussion_r1431731166 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergSourceSplitReader.java: ## @@ -80,6 +88,35 @@ public RecordsWithSplitIds> fetch() throws

Re: [PR] Flink: Support watermark alignment of source splits [iceberg]

stevenzwu commented on code in PR #9308: URL: https://github.com/apache/iceberg/pull/9308#discussion_r1431726956 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergSourceSplitReader.java: ## @@ -80,6 +88,35 @@ public RecordsWithSplitIds> fetch() th

Re: [PR] Flink: Support watermark alignment of source splits [iceberg]

stevenzwu commented on code in PR #9308: URL: https://github.com/apache/iceberg/pull/9308#discussion_r1431716141 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergSourceSplitReader.java: ## @@ -80,6 +88,35 @@ public RecordsWithSplitIds> fetch() th

Re: [PR] Flink: Support watermark alignment of source splits [iceberg]

stevenzwu commented on code in PR #9308: URL: https://github.com/apache/iceberg/pull/9308#discussion_r1431716141 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergSourceSplitReader.java: ## @@ -80,6 +88,35 @@ public RecordsWithSplitIds> fetch() th

Re: [PR] Spark 3.5: Remove UnresolvedIcebergTable [iceberg]

aokolnychyi merged PR #9338: URL: https://github.com/apache/iceberg/pull/9338 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] [1.4.x] Core: Expired Snapshot files in a transaction should be deleted [iceberg]

amogh-jahagirdar merged PR #9223: URL: https://github.com/apache/iceberg/pull/9223 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Flink: Support watermark alignment of source splits [iceberg]

stevenzwu commented on code in PR #9308: URL: https://github.com/apache/iceberg/pull/9308#discussion_r1431702572 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergSourceSplitReader.java: ## @@ -80,6 +88,35 @@ public RecordsWithSplitIds> fetch() th

Re: [PR] [1.4.x] JDBC Catalog: Fix namespaceExists check with special characters (#8340) [iceberg]

nastra merged PR #9291: URL: https://github.com/apache/iceberg/pull/9291 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [I] Remove MockAWSResponse and pyiceberg.io.fsspec.FsspecFileIO from testing [iceberg-python]

Fokko closed issue #222: Remove MockAWSResponse and pyiceberg.io.fsspec.FsspecFileIO from testing URL: https://github.com/apache/iceberg-python/issues/222 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [I] Remove MockAWSResponse and pyiceberg.io.fsspec.FsspecFileIO from testing [iceberg-python]

Fokko commented on issue #222: URL: https://github.com/apache/iceberg-python/issues/222#issuecomment-1863126161 Closing this one since the PR is in, thanks for working on this @sebpretzer 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] Flink: Support watermark alignment of source splits [iceberg]

mxm commented on code in PR #9308: URL: https://github.com/apache/iceberg/pull/9308#discussion_r1431664838 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergSourceSplitReader.java: ## @@ -80,6 +88,35 @@ public RecordsWithSplitIds> fetch() throws I

Re: [PR] Glue catalog commit table [iceberg-python]

jackye1995 commented on code in PR #140: URL: https://github.com/apache/iceberg-python/pull/140#discussion_r1431672549 ## pyiceberg/catalog/__init__.py: ## @@ -587,8 +590,32 @@ def _write_metadata(metadata: TableMetadata, io: FileIO, metadata_path: str) -> ToOutputFile

[I] Saving Polars Dataframe as Parquet to Iceberg [iceberg-python]

asheeshgarg opened a new issue, #226: URL: https://github.com/apache/iceberg-python/issues/226 ### Feature Request / Improvement @Fokko I have a polars dataframe I have save it to parquet file using the and using the Java API DataFile dataFile = DataFiles.builder(table.spe

Re: [PR] API: New API For sequential / streaming updates [iceberg]

jasonf20 commented on PR #9323: URL: https://github.com/apache/iceberg/pull/9323#issuecomment-1863068142 @rdblue Correct, we need multiple sequence (new) sequence numbers since each batch has deletes that need to apply to prior batches, but not newer batches. Committing more than once wo

[PR] Spark: Support renaming views [iceberg]

nastra opened a new pull request, #9343: URL: https://github.com/apache/iceberg/pull/9343 This is based on https://github.com/apache/iceberg/pull/9340 and adds support for resolving and renaming views -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] Build: Bump ray from 2.7.1 to 2.8.1 [iceberg-python]

Fokko commented on PR #174: URL: https://github.com/apache/iceberg-python/pull/174#issuecomment-1863016154 @dependabot recreate -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] Build: Bump ray from 2.7.1 to 2.8.1 [iceberg-python]

dependabot[bot] commented on PR #174: URL: https://github.com/apache/iceberg-python/pull/174#issuecomment-1863011538 Looks like this PR has been edited by someone other than Dependabot. That means Dependabot can't rebase it - sorry! If you're happy for Dependabot to recreate it from s

Re: [PR] Build: Bump ray from 2.7.1 to 2.8.1 [iceberg-python]

Fokko commented on PR #174: URL: https://github.com/apache/iceberg-python/pull/174#issuecomment-1863011421 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Build: Bump aiohttp from 3.8.6 to 3.9.0 [iceberg-python]

Fokko merged PR #168: URL: https://github.com/apache/iceberg-python/pull/168 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Spark: Add support for Iceberg views [iceberg]

nastra commented on code in PR #9332: URL: https://github.com/apache/iceberg/pull/9332#discussion_r1431576583 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/views/ResolvedV2View.scala: ## @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache

Re: [PR] Make connect_timeout configurable in IO [iceberg-python]

Fokko merged PR #218: URL: https://github.com/apache/iceberg-python/pull/218 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

  1   2   >