Re: [PR] Build: Bump com.google.errorprone:error_prone_annotations from 2.30.0 to 2.31.0 [iceberg]

Fokko merged PR #11055: URL: https://github.com/apache/iceberg/pull/11055 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Make `commit_table` public [iceberg-python]

Fokko commented on code in PR #1112: URL: https://github.com/apache/iceberg-python/pull/1112#discussion_r1739974152 ## pyiceberg/catalog/sql.py: ## @@ -407,20 +412,18 @@ def _commit_table(self, table_request: CommitTableRequest) -> CommitTableRespons NoSuchTableErr

Re: [PR] Make `commit_table` public [iceberg-python]

Fokko commented on code in PR #1112: URL: https://github.com/apache/iceberg-python/pull/1112#discussion_r1739973689 ## pyiceberg/table/__init__.py: ## @@ -1673,12 +1673,8 @@ def refs(self) -> Dict[str, SnapshotRef]: return self.metadata.refs def _do_commit(self,

Re: [PR] Make `commit_table` public [iceberg-python]

Fokko commented on code in PR #1112: URL: https://github.com/apache/iceberg-python/pull/1112#discussion_r1739973551 ## pyiceberg/catalog/rest.py: ## @@ -773,6 +771,17 @@ def _commit_table(self, table_request: CommitTableRequest) -> CommitTableRespons ) ret

Re: [PR] Bump python-snappy from 0.7.2 to 0.7.3 [iceberg-python]

Fokko merged PR #1115: URL: https://github.com/apache/iceberg-python/pull/1115 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Use `markdownlint` instead of `mdformat` [iceberg-python]

Fokko commented on code in PR #1118: URL: https://github.com/apache/iceberg-python/pull/1118#discussion_r1739972576 ## .pre-commit-config.yaml: ## @@ -46,17 +46,11 @@ repos: hooks: - id: pycln args: [--config=pyproject.toml] - - repo: https://github.com/exe

Re: [PR] Bump mkdocstrings-python from 1.10.8 to 1.10.9 [iceberg-python]

Fokko merged PR #1116: URL: https://github.com/apache/iceberg-python/pull/1116 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Bump mkdocs from 1.6.0 to 1.6.1 [iceberg-python]

Fokko merged PR #1117: URL: https://github.com/apache/iceberg-python/pull/1117 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Build: Bump mkdocs-material from 9.5.33 to 9.5.34 [iceberg]

Fokko merged PR #11062: URL: https://github.com/apache/iceberg/pull/11062 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

[PR] Build: Bump mkdocs-material from 9.5.33 to 9.5.34 [iceberg]

dependabot[bot] opened a new pull request, #11062: URL: https://github.com/apache/iceberg/pull/11062 Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.5.33 to 9.5.34. Release notes Sourced from https://github.com/squidfunk/mkdocs-material/releases";>mkdoc

[PR] Build: Bump org.apache.hadoop.thirdparty:hadoop-shaded-guava from 1.2.0 to 1.3.0 [iceberg]

dependabot[bot] opened a new pull request, #11061: URL: https://github.com/apache/iceberg/pull/11061 Bumps org.apache.hadoop.thirdparty:hadoop-shaded-guava from 1.2.0 to 1.3.0. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?de

[PR] Build: Bump com.google.cloud:libraries-bom from 26.44.0 to 26.45.0 [iceberg]

dependabot[bot] opened a new pull request, #11060: URL: https://github.com/apache/iceberg/pull/11060 Bumps [com.google.cloud:libraries-bom](https://github.com/googleapis/java-cloud-bom) from 26.44.0 to 26.45.0. Release notes Sourced from https://github.com/googleapis/java-cloud-bo

[PR] Build: Bump junit-platform from 1.10.3 to 1.11.0 [iceberg]

dependabot[bot] opened a new pull request, #11059: URL: https://github.com/apache/iceberg/pull/11059 Bumps `junit-platform` from 1.10.3 to 1.11.0. Updates `org.junit.platform:junit-platform-suite-api` from 1.10.3 to 1.11.0 Commits See full diff in https://github.com/junit-tea

Re: [PR] Build: Bump com.azure:azure-sdk-bom from 1.2.25 to 1.2.26 [iceberg]

dependabot[bot] commented on PR #10870: URL: https://github.com/apache/iceberg/pull/10870#issuecomment-2323156827 Superseded by #11058. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Build: Bump com.azure:azure-sdk-bom from 1.2.25 to 1.2.26 [iceberg]

dependabot[bot] closed pull request #10870: Build: Bump com.azure:azure-sdk-bom from 1.2.25 to 1.2.26 URL: https://github.com/apache/iceberg/pull/10870 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[PR] Build: Bump com.azure:azure-sdk-bom from 1.2.25 to 1.2.27 [iceberg]

dependabot[bot] opened a new pull request, #11058: URL: https://github.com/apache/iceberg/pull/11058 Bumps [com.azure:azure-sdk-bom](https://github.com/azure/azure-sdk-for-java) from 1.2.25 to 1.2.27. Release notes Sourced from https://github.com/azure/azure-sdk-for-java/releases";

[PR] Build: Bump net.snowflake:snowflake-jdbc from 3.18.0 to 3.19.0 [iceberg]

dependabot[bot] opened a new pull request, #11057: URL: https://github.com/apache/iceberg/pull/11057 Bumps [net.snowflake:snowflake-jdbc](https://github.com/snowflakedb/snowflake-jdbc) from 3.18.0 to 3.19.0. Release notes Sourced from https://github.com/snowflakedb/snowflake-jdbc/

[PR] Build: Bump software.amazon.awssdk:bom from 2.27.12 to 2.27.17 [iceberg]

dependabot[bot] opened a new pull request, #11056: URL: https://github.com/apache/iceberg/pull/11056 Bumps software.amazon.awssdk:bom from 2.27.12 to 2.27.17. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=soft

[PR] Build: Bump com.google.errorprone:error_prone_annotations from 2.30.0 to 2.31.0 [iceberg]

dependabot[bot] opened a new pull request, #11055: URL: https://github.com/apache/iceberg/pull/11055 Bumps [com.google.errorprone:error_prone_annotations](https://github.com/google/error-prone) from 2.30.0 to 2.31.0. Release notes Sourced from https://github.com/google/error-prone

[PR] Build: Bump parquet from 1.13.1 to 1.14.2 [iceberg]

dependabot[bot] opened a new pull request, #11054: URL: https://github.com/apache/iceberg/pull/11054 Bumps `parquet` from 1.13.1 to 1.14.2. Updates `org.apache.parquet:parquet-avro` from 1.13.1 to 1.14.2 Release notes Sourced from https://github.com/apache/parquet-mr/releases";>o

Re: [I] The Orc file (via iceberg)because large than Orc file(only via spark) ? [iceberg]

github-actions[bot] commented on issue #7775: URL: https://github.com/apache/iceberg/issues/7775#issuecomment-2323082748 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Docs: Include references GCP libraries [iceberg]

github-actions[bot] closed issue #7787: Docs: Include references GCP libraries URL: https://github.com/apache/iceberg/issues/7787 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] Iceberg Schema moving inner column after adding new StructType doesn't work. [iceberg]

github-actions[bot] commented on issue #7786: URL: https://github.com/apache/iceberg/issues/7786#issuecomment-2323082762 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Iceberg Schema moving inner column after adding new StructType doesn't work. [iceberg]

github-actions[bot] closed issue #7786: Iceberg Schema moving inner column after adding new StructType doesn't work. URL: https://github.com/apache/iceberg/issues/7786 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [I] How to enable iceberg upsert operation!??? [iceberg]

github-actions[bot] commented on issue #7769: URL: https://github.com/apache/iceberg/issues/7769#issuecomment-2323082726 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] How to enable iceberg upsert operation!??? [iceberg]

github-actions[bot] closed issue #7769: How to enable iceberg upsert operation!??? URL: https://github.com/apache/iceberg/issues/7769 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [I] Scope the annotations so that they don't show up as a dependency in the pom.xml for org.apache.iceberg:iceberg-{api|core} [iceberg]

github-actions[bot] commented on issue #7756: URL: https://github.com/apache/iceberg/issues/7756#issuecomment-2323082711 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Scope the annotations so that they don't show up as a dependency in the pom.xml for org.apache.iceberg:iceberg-{api|core} [iceberg]

github-actions[bot] closed issue #7756: Scope the annotations so that they don't show up as a dependency in the pom.xml for org.apache.iceberg:iceberg-{api|core} URL: https://github.com/apache/iceberg/issues/7756 -- This is an automated message from the Apache Git Service. To respond to the

Re: [I] Docs: Include references GCP libraries [iceberg]

github-actions[bot] commented on issue #7787: URL: https://github.com/apache/iceberg/issues/7787#issuecomment-2323082769 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Failed to check if LessThan(status,2) can be pushed down: Cannot find field 'status' in struct: struct<> [iceberg]

github-actions[bot] commented on issue #7774: URL: https://github.com/apache/iceberg/issues/7774#issuecomment-2323082735 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Flink: Replace Flink Tableschema To Resolvedschema [iceberg]

github-actions[bot] closed pull request #7008: Flink: Replace Flink Tableschema To Resolvedschema URL: https://github.com/apache/iceberg/pull/7008 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] Flink: Replace Flink Tableschema To Resolvedschema [iceberg]

github-actions[bot] commented on PR #7008: URL: https://github.com/apache/iceberg/pull/7008#issuecomment-2323082563 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Integration tests for Snowflake catalog [iceberg]

github-actions[bot] closed pull request #7004: Integration tests for Snowflake catalog URL: https://github.com/apache/iceberg/pull/7004 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Integration tests for Snowflake catalog [iceberg]

github-actions[bot] commented on PR #7004: URL: https://github.com/apache/iceberg/pull/7004#issuecomment-2323082556 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Docs: Document 'write.parquet.row-group-check-min-record-count' and '… [iceberg]

github-actions[bot] closed pull request #6905: Docs: Document 'write.parquet.row-group-check-min-record-count' and '… URL: https://github.com/apache/iceberg/pull/6905 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] Docs: Document 'write.parquet.row-group-check-min-record-count' and '… [iceberg]

github-actions[bot] commented on PR #6905: URL: https://github.com/apache/iceberg/pull/6905#issuecomment-2323082521 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Fix StructCopy [iceberg]

github-actions[bot] commented on PR #6894: URL: https://github.com/apache/iceberg/pull/6894#issuecomment-2323082506 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Fix StructCopy [iceberg]

github-actions[bot] closed pull request #6894: Fix StructCopy URL: https://github.com/apache/iceberg/pull/6894 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

Re: [PR] Spark: allows catalog.warehouse for Spark Hive Catalogs #6863 [iceberg]

github-actions[bot] closed pull request #6886: Spark: allows catalog.warehouse for Spark Hive Catalogs #6863 URL: https://github.com/apache/iceberg/pull/6886 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [PR] Spark: allows catalog.warehouse for Spark Hive Catalogs #6863 [iceberg]

github-actions[bot] commented on PR #6886: URL: https://github.com/apache/iceberg/pull/6886#issuecomment-2323082481 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] cache delete files for repeat reading [iceberg]

github-actions[bot] closed pull request #6866: cache delete files for repeat reading URL: https://github.com/apache/iceberg/pull/6866 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] cache delete files for repeat reading [iceberg]

github-actions[bot] commented on PR #6866: URL: https://github.com/apache/iceberg/pull/6866#issuecomment-2323082468 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Core: Fix for - Do not use overwrite avro appender for manifests with non Hadoop io [iceberg]

github-actions[bot] commented on PR #6839: URL: https://github.com/apache/iceberg/pull/6839#issuecomment-2323082456 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Core: Fix for - Do not use overwrite avro appender for manifests with non Hadoop io [iceberg]

github-actions[bot] closed pull request #6839: Core: Fix for - Do not use overwrite avro appender for manifests with non Hadoop io URL: https://github.com/apache/iceberg/pull/6839 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [PR] Parquet: Optimize dictionary filter evaluation on notIn and notEq [iceberg]

github-actions[bot] closed pull request #6836: Parquet: Optimize dictionary filter evaluation on notIn and notEq URL: https://github.com/apache/iceberg/pull/6836 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] Parquet: Optimize dictionary filter evaluation on notIn and notEq [iceberg]

github-actions[bot] commented on PR #6836: URL: https://github.com/apache/iceberg/pull/6836#issuecomment-2323082446 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Parquet: fix conversion of enums to strings [iceberg]

github-actions[bot] closed pull request #6804: Parquet: fix conversion of enums to strings URL: https://github.com/apache/iceberg/pull/6804 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] Parquet: fix conversion of enums to strings [iceberg]

github-actions[bot] commented on PR #6804: URL: https://github.com/apache/iceberg/pull/6804#issuecomment-2323082433 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Core: Use avro compression properties from table properties when writing manifests and manifest lists [iceberg]

github-actions[bot] closed pull request #6799: Core: Use avro compression properties from table properties when writing manifests and manifest lists URL: https://github.com/apache/iceberg/pull/6799 -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Core: Use avro compression properties from table properties when writing manifests and manifest lists [iceberg]

github-actions[bot] commented on PR #6799: URL: https://github.com/apache/iceberg/pull/6799#issuecomment-2323082423 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Spark-3.3: Support unregister table procedure [iceberg]

github-actions[bot] closed pull request #6786: Spark-3.3: Support unregister table procedure URL: https://github.com/apache/iceberg/pull/6786 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] Spark-3.3: Support unregister table procedure [iceberg]

github-actions[bot] commented on PR #6786: URL: https://github.com/apache/iceberg/pull/6786#issuecomment-2323082414 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Core: enforce writing POSIX compatible paths for data location [iceberg]

github-actions[bot] commented on PR #6772: URL: https://github.com/apache/iceberg/pull/6772#issuecomment-2323082393 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Extend data files abstraction in the table manifest with the modification time [iceberg]

github-actions[bot] closed pull request #6773: Extend data files abstraction in the table manifest with the modification time URL: https://github.com/apache/iceberg/pull/6773 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

Re: [PR] Extend data files abstraction in the table manifest with the modification time [iceberg]

github-actions[bot] commented on PR #6773: URL: https://github.com/apache/iceberg/pull/6773#issuecomment-2323082403 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Core: enforce writing POSIX compatible paths for data location [iceberg]

github-actions[bot] closed pull request #6772: Core: enforce writing POSIX compatible paths for data location URL: https://github.com/apache/iceberg/pull/6772 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] [Spark] add extraSnapshotMetadata using sql conf [iceberg]

github-actions[bot] closed pull request #6755: [Spark] add extraSnapshotMetadata using sql conf URL: https://github.com/apache/iceberg/pull/6755 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] [Spark] add extraSnapshotMetadata using sql conf [iceberg]

github-actions[bot] commented on PR #6755: URL: https://github.com/apache/iceberg/pull/6755#issuecomment-2323082380 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [I] show table extended not supported for v2 table. [iceberg]

dargmuesli commented on issue #5782: URL: https://github.com/apache/iceberg/issues/5782#issuecomment-2323073302 The PR https://github.com/apache/spark/pull/37588 is now merged for quite some time, but I still get this error. Is there a change necessary to iceberg now or what would be the co

Re: [PR] Spark 3.5: Skip sort for incomparable data types in CreateChangelogViewProcedure [iceberg]

flyrain commented on PR #11045: URL: https://github.com/apache/iceberg/pull/11045#issuecomment-2323003542 Thanks @dramaticlly for the PR. We will need to look a bit deeper to see how it works. The `ChangeIterator` relies on the partition and sort. I didn't see a way to skip a column without

Re: [PR] Spark 3.5: Skip sort for incomparable data types in CreateChangelogViewProcedure [iceberg]

flyrain commented on code in PR #11045: URL: https://github.com/apache/iceberg/pull/11045#discussion_r1739891387 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/procedures/CreateChangelogViewProcedure.java: ## @@ -183,21 +185,26 @@ private boolean shouldComputeUpdateI

Re: [PR] Spark 3.5: Skip sort for incomparable data types in CreateChangelogViewProcedure [iceberg]

flyrain commented on code in PR #11045: URL: https://github.com/apache/iceberg/pull/11045#discussion_r1739891387 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/procedures/CreateChangelogViewProcedure.java: ## @@ -183,21 +185,26 @@ private boolean shouldComputeUpdateI

Re: [PR] Spark 3.5: Skip sort for incomparable data types in CreateChangelogViewProcedure [iceberg]

flyrain commented on code in PR #11045: URL: https://github.com/apache/iceberg/pull/11045#discussion_r1739888471 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/procedures/CreateChangelogViewProcedure.java: ## @@ -183,21 +185,26 @@ private boolean shouldComputeUpdateI

Re: [PR] Spark 3.5: Skip sort for incomparable data types in CreateChangelogViewProcedure [iceberg]

flyrain commented on code in PR #11045: URL: https://github.com/apache/iceberg/pull/11045#discussion_r1739882108 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestCreateChangelogViewProcedure.java: ## @@ -242,6 +249,34 @@ public void testUpdate

Re: [PR] Build: Upgrade google-java-format to latest version [iceberg]

findepi commented on code in PR #11050: URL: https://github.com/apache/iceberg/pull/11050#discussion_r1739866418 ## baseline.gradle: ## @@ -50,16 +50,7 @@ subprojects { t.setDuplicatesStrategy(DuplicatesStrategy.WARN); }); apply plugin: 'com.palantir.baseline-exact-de

Re: [PR] feat (datafusion integration): convert datafusion expr filters to Iceberg Predicate [iceberg-rust]

a-agmon commented on code in PR #588: URL: https://github.com/apache/iceberg-rust/pull/588#discussion_r1739863150 ## crates/integrations/datafusion/src/physical_plan/scan.rs: ## @@ -138,3 +150,231 @@ async fn get_batch_stream( Ok(Box::pin(stream)) } + +/// convert DataFu

Re: [PR] Add drop_view to the rest catalog [iceberg-python]

sungwy commented on PR #820: URL: https://github.com/apache/iceberg-python/pull/820#issuecomment-2322977610 LGTM! Thank you for adding this new API @ndrluis -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [I] Implement `TableProviderFactory` for a `IcebergTableFactory` [iceberg-rust]

matthewmturner commented on issue #586: URL: https://github.com/apache/iceberg-rust/issues/586#issuecomment-2322972430 @yukkit I am actually only familiar with the `ObjectStore` abstraction and not `FileIO` - ill need to look into that more. Prior to knowing that I had naively expected to

Re: [I] Merge into / Upsert [iceberg-python]

Minfante377 commented on issue #402: URL: https://github.com/apache/iceberg-python/issues/402#issuecomment-2322949632 Any updates on this one? I'm good with overwrite + overwrite filters for now but for tables where columns are populated by different sources it would be awesome to have full

Re: [I] Implement `TableProviderFactory` for a `IcebergTableFactory` [iceberg-rust]

yukkit commented on issue #586: URL: https://github.com/apache/iceberg-rust/issues/586#issuecomment-2322940734 Here are a few questions that need clarification: 1. Is there a need to support specifying a version? @matthewmturner 2. Where should the parameters for object storage, suc

[I] [bug] `table.inspect.partitions()` does not respect partition evolution [iceberg-python]

kevinjqliu opened a new issue, #1120: URL: https://github.com/apache/iceberg-python/issues/1120 ### Apache Iceberg version main (development) ### Please describe the bug 🐞 Expectation: The table is partitioned by the `timestamp ` field. After evolving the partition

Re: [I] PyIceberg is not respecting `token` in the load table response [iceberg-python]

creechy commented on issue #1113: URL: https://github.com/apache/iceberg-python/issues/1113#issuecomment-2322930679 @kevinjqliu > Do you have an example to reproduce this issue? I provided an example with a Tabular config to @Fokko, who confirmed that PyIceberg does not appea

Re: [I] Issue when overwriting data with row filter [iceberg-python]

kevinjqliu commented on issue #1108: URL: https://github.com/apache/iceberg-python/issues/1108#issuecomment-2322911398 Thanks for the example @JasperHG90 Here's a notebook to help with debugging. https://gist.github.com/kevinjqliu/bc0b6457b27a89e3628720896fb24195 Something I n

[PR] Core: Generalize Util::blockLocations [iceberg]

okumin opened a new pull request, #11053: URL: https://github.com/apache/iceberg/pull/11053 The Apache Hive community is trying to implement optimizations, such as Bucket Map Join, using partition transform specs. We presume `Util::blockLocations` should accept not `CombinedScanTask` but `

Re: [PR] feat (datafusion integration): convert datafusion expr filters to Iceberg Predicate [iceberg-rust]

sdd commented on code in PR #588: URL: https://github.com/apache/iceberg-rust/pull/588#discussion_r1739722426 ## crates/integrations/datafusion/src/physical_plan/scan.rs: ## @@ -138,3 +150,231 @@ async fn get_batch_stream( Ok(Box::pin(stream)) } + +/// convert DataFusion

Re: [I] PyIceberg is not respecting `token` in the load table response [iceberg-python]

kevinjqliu commented on issue #1113: URL: https://github.com/apache/iceberg-python/issues/1113#issuecomment-2322904152 Thanks for reporting this @creechy Do you have an example to reproduce this issue? For now, I found some more docs on token https://github.com/apache/iceberg

Re: [I] TypeError when `operation` field is missing in `summary`. [iceberg-python]

kevinjqliu commented on issue #1106: URL: https://github.com/apache/iceberg-python/issues/1106#issuecomment-2322902850 > Java is interestingly more graceful in parsing the operation tag (and it probably should not be) @sungwy does this mean that the current JAVA implementation does n

Re: [I] Implement `TableProviderFactory` for a `IcebergTableFactory` [iceberg-rust]

yukkit commented on issue #586: URL: https://github.com/apache/iceberg-rust/issues/586#issuecomment-2322902965 Please assign it to me -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] Do not deprecate Botocore Session in upcoming release (0.8) [iceberg-python]

kevinjqliu commented on issue #1104: URL: https://github.com/apache/iceberg-python/issues/1104#issuecomment-2322901352 Thanks for raising this issue @BTheunissen > botocore_session is helpful to make available to override in order to support automatically refreshable credentials for

Re: [I] Fast Avro Decoder not included in Conda Deployment of pyiceberg [iceberg-python]

kevinjqliu commented on issue #1093: URL: https://github.com/apache/iceberg-python/issues/1093#issuecomment-2322898924 Im not sure how condo deals with Cython extensions but here's the relevant code https://github.com/apache/iceberg-python/blob/e4c1748fee220076f04e35ab2f182dd51ca2

Re: [I] tbl.append(df): schema validation of tbl & df during compares the order & data types [iceberg-python]

kevinjqliu commented on issue #1088: URL: https://github.com/apache/iceberg-python/issues/1088#issuecomment-2322898180 We improved `_check_schema_compatible` since 0.6.1 (see #921) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [I] Concurrent writes failures [iceberg-python]

kevinjqliu commented on issue #1084: URL: https://github.com/apache/iceberg-python/issues/1084#issuecomment-2322896488 > currently using sqlite + local fs FYI, according to the docs, "SQLite is not built for concurrency, you should use this catalog for exploratory or development purp

Re: [PR] Use `markdownlint` instead of `mdformat` [iceberg-python]

kevinjqliu commented on code in PR #1118: URL: https://github.com/apache/iceberg-python/pull/1118#discussion_r1739713693 ## mkdocs/docs/verify-release.md: ## @@ -117,8 +117,10 @@ This will spin up Docker containers to faciliate running test coverage. Votes are cast by replyi

Re: [PR] Core: Fix the behavior of IncrementalFileCleanup when expire a snapshot [iceberg]

hantangwangd commented on code in PR #10983: URL: https://github.com/apache/iceberg/pull/10983#discussion_r1739706862 ## core/src/test/java/org/apache/iceberg/TestRemoveSnapshots.java: ## @@ -370,7 +370,7 @@ public void testRetainLastWithExpireById() { } // Retain la

Re: [I] AWS: provide option to hide old fields in Glue table [iceberg]

tcassou commented on issue #7584: URL: https://github.com/apache/iceberg/issues/7584#issuecomment-2322852130 Hi there! This is still an issue, and the only workaround we found is to build a custom Iceberg jar without the faulty commit which is not really sustainable of course. Any ch