Re: [PR] fix: bump parquet minor version [iceberg-rust]

Xuanwo merged PR #684: URL: https://github.com/apache/iceberg-rust/pull/684 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [I] iceberg-spark-runtime: Sources jar does not contain any sources [iceberg]

manuzhang commented on issue #11418: URL: https://github.com/apache/iceberg/issues/11418#issuecomment-2449159549 https://github.com/apache/iceberg/tree/main/spark/v3.5/spark-runtime doesn't contain any source. It's intended for [runtime](https://iceberg.apache.org/multi-engine-support/#runt

Re: [PR] [WIP] Core: Prototype for DVs in V3 [iceberg]

nastra commented on code in PR #11302: URL: https://github.com/apache/iceberg/pull/11302#discussion_r1823839780 ## .palantir/revapi.yml: ## @@ -1100,6 +1100,15 @@ acceptedBreaks: - code: "java.class.removed" old: "enum org.apache.iceberg.BaseMetastoreTableOperations.

Re: [PR] [WIP] Core: Prototype for DVs in V3 [iceberg]

nastra commented on code in PR #11302: URL: https://github.com/apache/iceberg/pull/11302#discussion_r1823830578 ## core/src/main/java/org/apache/iceberg/metrics/CommitMetricsResult.java: ## @@ -75,6 +77,9 @@ public interface CommitMetricsResult { @Nullable CounterResult ad

[PR] Core: Add validation for table commit properties [iceberg]

dramaticlly opened a new pull request, #11437: URL: https://github.com/apache/iceberg/pull/11437 close #11435 Total of 2 Changes proposed here 1. Add validation for creating new table where commit related table properties need to have value type checked (as integer) and value

Re: [PR] fix: list_tables method in glue catalog now only return tables. [iceberg-python]

omkenge commented on PR #1258: URL: https://github.com/apache/iceberg-python/pull/1258#issuecomment-2449064604 HI @HonahX Changes are in ... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] Add REST Catalog tests to Spark 3.5 integration test [iceberg]

nastra commented on code in PR #11093: URL: https://github.com/apache/iceberg/pull/11093#discussion_r1823802445 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestBaseWithCatalog.java: ## @@ -36,17 +41,37 @@ import org.apache.iceberg.catalog.SupportsNamespaces; imp

Re: [PR] Add REST Catalog tests to Spark 3.5 integration test [iceberg]

nastra commented on code in PR #11093: URL: https://github.com/apache/iceberg/pull/11093#discussion_r1823797130 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestBaseWithCatalog.java: ## @@ -36,17 +41,37 @@ import org.apache.iceberg.catalog.SupportsNamespaces; imp

Re: [PR] Add REST Catalog tests to Spark 3.5 integration test [iceberg]

nastra commented on code in PR #11093: URL: https://github.com/apache/iceberg/pull/11093#discussion_r1823798343 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestBaseWithCatalog.java: ## @@ -59,18 +84,30 @@ protected static Object[][] parameters() { } @Befor

Re: [PR] Add REST Catalog tests to Spark 3.5 integration test [iceberg]

nastra commented on code in PR #11093: URL: https://github.com/apache/iceberg/pull/11093#discussion_r1823798127 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestBaseWithCatalog.java: ## @@ -36,17 +41,37 @@ import org.apache.iceberg.catalog.SupportsNamespaces; imp

Re: [PR] Add REST Catalog tests to Spark 3.5 integration test [iceberg]

nastra commented on code in PR #11093: URL: https://github.com/apache/iceberg/pull/11093#discussion_r1823793249 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestBaseWithCatalog.java: ## @@ -59,18 +94,28 @@ protected static Object[][] parameters() { } @Befor

Re: [PR] Add REST Catalog tests to Spark 3.5 integration test [iceberg]

nastra commented on code in PR #11093: URL: https://github.com/apache/iceberg/pull/11093#discussion_r1823795448 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestBaseWithCatalog.java: ## @@ -36,17 +41,38 @@ import org.apache.iceberg.catalog.SupportsNamespaces; imp

Re: [PR] Add REST Catalog tests to Spark 3.5 integration test [iceberg]

nastra commented on code in PR #11093: URL: https://github.com/apache/iceberg/pull/11093#discussion_r1823794984 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestBaseWithCatalog.java: ## @@ -36,16 +41,46 @@ import org.apache.iceberg.catalog.SupportsNamespaces; imp

Re: [PR] Add REST Catalog tests to Spark 3.5 integration test [iceberg]

nastra commented on code in PR #11093: URL: https://github.com/apache/iceberg/pull/11093#discussion_r1823794365 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestBaseWithCatalog.java: ## @@ -36,16 +41,46 @@ import org.apache.iceberg.catalog.SupportsNamespaces; imp

Re: [PR] Add REST Catalog tests to Spark 3.5 integration test [iceberg]

nastra commented on code in PR #11093: URL: https://github.com/apache/iceberg/pull/11093#discussion_r1823794001 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestBaseWithCatalog.java: ## @@ -36,16 +41,46 @@ import org.apache.iceberg.catalog.SupportsNamespaces; imp

Re: [PR] Add REST Catalog tests to Spark 3.5 integration test [iceberg]

nastra commented on code in PR #11093: URL: https://github.com/apache/iceberg/pull/11093#discussion_r1823793249 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestBaseWithCatalog.java: ## @@ -59,18 +94,28 @@ protected static Object[][] parameters() { } @Befor

Re: [PR] Add REST Catalog tests to Spark 3.5 integration test [iceberg]

nastra commented on code in PR #11093: URL: https://github.com/apache/iceberg/pull/11093#discussion_r1823793865 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestBaseWithCatalog.java: ## @@ -59,18 +94,28 @@ protected static Object[][] parameters() { } @Befor

Re: [PR] Add REST Catalog tests to Spark 3.5 integration test [iceberg]

nastra commented on code in PR #11093: URL: https://github.com/apache/iceberg/pull/11093#discussion_r1823793733 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestBaseWithCatalog.java: ## @@ -59,18 +94,28 @@ protected static Object[][] parameters() { } @Befor

Re: [I] Flink: Maintenance - Add support for more kinds of scheduling [iceberg]

netvl commented on issue #11246: URL: https://github.com/apache/iceberg/issues/11246#issuecomment-2449037988 @pvary sorry for the delayed response, missed it somehow :( Yes, I think it is totally all right (and in fact, it is even *desirable*) to wait for the current maintenance run t

[I] Add support for multiple table DataStream FlinkSink [iceberg]

adasari opened a new issue, #11436: URL: https://github.com/apache/iceberg/issues/11436 ### Feature Request / Improvement We're currently using FlinkCDC to create an Iceberg database from a source PostgreSQL database, which necessitates a one-to-one mapping between source tables and

[PR] Bump rich from 13.8.1 to 13.9.3 [iceberg-python]

dependabot[bot] opened a new pull request, #1273: URL: https://github.com/apache/iceberg-python/pull/1273 Bumps [rich](https://github.com/Textualize/rich) from 13.8.1 to 13.9.3. Release notes Sourced from https://github.com/Textualize/rich/releases";>rich's releases. The irre

Re: [I] Iceberg Spark Extensions conflict with Paimon [iceberg]

github-actions[bot] commented on issue #10143: URL: https://github.com/apache/iceberg/issues/10143#issuecomment-2448718930 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

[PR] fix: bump parquet minor version [iceberg-rust]

xxchan opened a new pull request, #684: URL: https://github.com/apache/iceberg-rust/pull/684 We met compile error with parquet 53.0. This API was added in 53.1: https://github.com/apache/arrow-rs/blob/master/CHANGELOG-old.md#5310-2024-10-02 ``` error[E0432]: unresolved import `parquet

Re: [PR] fix: list_tables method in glue catalog now only return tables. [iceberg-python]

HonahX commented on code in PR #1258: URL: https://github.com/apache/iceberg-python/pull/1258#discussion_r1823751559 ## pyiceberg/catalog/glue.py: ## @@ -784,4 +784,4 @@ def drop_view(self, identifier: Union[str, Identifier]) -> None: @staticmethod def __is_iceberg_

Re: [PR] Bump rich from 13.8.1 to 13.9.3 [iceberg-python]

HonahX merged PR #1273: URL: https://github.com/apache/iceberg-python/pull/1273 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@icebe

Re: [PR] Deprecate for 0.8.0 release [iceberg-python]

HonahX commented on code in PR #1269: URL: https://github.com/apache/iceberg-python/pull/1269#discussion_r1823739460 ## mkdocs/docs/configuration.md: ## @@ -341,7 +341,7 @@ catalog: !!! warning "Deprecated Properties" -`profile_name`, `region_name`, `botocore_session`,

Re: [PR] AWS: Refresh vended credentials [iceberg]

singhpk234 commented on code in PR #11389: URL: https://github.com/apache/iceberg/pull/11389#discussion_r1823648657 ## aws/src/main/java/org/apache/iceberg/aws/s3/VendedCredentialsProvider.java: ## @@ -0,0 +1,138 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [I] Bad Table Properties cause commit failure [iceberg]

singhpk234 commented on issue #11435: URL: https://github.com/apache/iceberg/issues/11435#issuecomment-2448956350 Nice catch ! Sounds great looking forward to the change ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] Iceberg Spark Extensions conflict with Paimon [iceberg]

wForget closed issue #10143: Iceberg Spark Extensions conflict with Paimon URL: https://github.com/apache/iceberg/issues/10143 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] AWS: Refresh vended credentials [iceberg]

danielcweeks commented on code in PR #11389: URL: https://github.com/apache/iceberg/pull/11389#discussion_r1823626747 ## aws/src/main/java/org/apache/iceberg/aws/s3/VendedCredentialsProvider.java: ## @@ -0,0 +1,138 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

Re: [I] Implement `TableProviderFactory` for a `IcebergTableFactory` [iceberg-rust]

yukkit commented on issue #586: URL: https://github.com/apache/iceberg-rust/issues/586#issuecomment-2448916773 > Will this enable `insert into` as well for iceberg? This feature is implemented solely to enable reading Iceberg data as an external table in DataFusion. -- This is an a

Re: [I] Runtime jars are not including module's license and notice [iceberg]

ajantha-bhat commented on issue #11431: URL: https://github.com/apache/iceberg/issues/11431#issuecomment-2448893435 I don't have solution yet (not a Gradle expert). I am figuring out. But let me know if you find something obvious. Can refer any of the runtime shadow jar task like

Re: [I] Runtime jars are not including module's license and notice [iceberg]

ajantha-bhat commented on issue #11431: URL: https://github.com/apache/iceberg/issues/11431#issuecomment-2448887845 Same thing observed for spark runtime and flink runtime also. Also, checked old release like 1.5.0 and it also has the same problem. -- This is an automated message

Re: [PR] Flink: FlinkFileIO implementation [iceberg]

github-actions[bot] commented on PR #10151: URL: https://github.com/apache/iceberg/pull/10151#issuecomment-2448719011 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Procedure to compute table stats [iceberg]

szehon-ho commented on code in PR #10986: URL: https://github.com/apache/iceberg/pull/10986#discussion_r1823666819 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/procedures/ComputeTableStatsProcedure.java: ## @@ -0,0 +1,121 @@ +/* + * Licensed to the Apache Software

Re: [I] The hour partition transform overflows for large dates [iceberg]

github-actions[bot] commented on issue #9713: URL: https://github.com/apache/iceberg/issues/9713#issuecomment-2448717960 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Iceberg may occur data duplication when use flink to write data to iceberg and commit failed [iceberg]

github-actions[bot] commented on issue #10165: URL: https://github.com/apache/iceberg/issues/10165#issuecomment-2448719080 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] AWS: Refresh vended credentials [iceberg]

singhpk234 commented on code in PR #11389: URL: https://github.com/apache/iceberg/pull/11389#discussion_r1823648657 ## aws/src/main/java/org/apache/iceberg/aws/s3/VendedCredentialsProvider.java: ## @@ -0,0 +1,138 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] Core: Uncached files not be materialized [iceberg]

github-actions[bot] commented on PR #10041: URL: https://github.com/apache/iceberg/pull/10041#issuecomment-2448718756 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [I] TestDataFrameWrites#testFaultToleranceOnWrite failed due to exception in cleaning up temporary directory [iceberg]

github-actions[bot] commented on issue #10172: URL: https://github.com/apache/iceberg/issues/10172#issuecomment-2448719109 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] The hour partition transform overflows for large dates [iceberg]

github-actions[bot] closed issue #9713: The hour partition transform overflows for large dates URL: https://github.com/apache/iceberg/issues/9713 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] Cannot insert table created by spark temp into iceberg table [iceberg]

github-actions[bot] commented on issue #10164: URL: https://github.com/apache/iceberg/issues/10164#issuecomment-2448719057 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] [feature request] Improve integration test reliance on docker [iceberg-python]

github-actions[bot] commented on issue #637: URL: https://github.com/apache/iceberg-python/issues/637#issuecomment-2448721295 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity oc

Re: [I] Changes in describe behaviour of a table break partition info? [iceberg]

github-actions[bot] commented on issue #10174: URL: https://github.com/apache/iceberg/issues/10174#issuecomment-2448719123 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Aliyun: Add loading oss file into memory option to OSS client properties [iceberg]

github-actions[bot] closed pull request #10062: Aliyun: Add loading oss file into memory option to OSS client properties URL: https://github.com/apache/iceberg/pull/10062 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] Logs: More precise error message for Iceberg Partition Parsing [iceberg]

github-actions[bot] commented on PR #10047: URL: https://github.com/apache/iceberg/pull/10047#issuecomment-2448718787 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] Spark: Hive-View Integration Tests [iceberg]

github-actions[bot] commented on PR #10088: URL: https://github.com/apache/iceberg/pull/10088#issuecomment-2448718844 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [I] Support for writing Parquet files from the Iceberg Java API without the Hadoop Configuration class [iceberg]

github-actions[bot] commented on issue #10180: URL: https://github.com/apache/iceberg/issues/10180#issuecomment-2448719170 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] spark.table() raises warn: Unclosed S3FileIO instance in HadoopTableOperations [iceberg]

github-actions[bot] commented on issue #10145: URL: https://github.com/apache/iceberg/issues/10145#issuecomment-2448718964 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] Does the FlushOnEveryBlock feature in Avro affect Iceberg data integrity? [iceberg]

github-actions[bot] commented on issue #10142: URL: https://github.com/apache/iceberg/issues/10142#issuecomment-2448718918 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Drop table purge issue for parquet tables with SparkSessionCatalog [iceberg]

github-actions[bot] commented on PR #10158: URL: https://github.com/apache/iceberg/pull/10158#issuecomment-2448719030 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] truncate partition transform width greater than long errors out [iceberg]

github-actions[bot] closed issue #9715: truncate partition transform width greater than long errors out URL: https://github.com/apache/iceberg/issues/9715 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [I] Restrict generated locations to URI syntax [iceberg]

github-actions[bot] commented on issue #10168: URL: https://github.com/apache/iceberg/issues/10168#issuecomment-2448719097 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] Date partition transforms are underspecified [iceberg]

github-actions[bot] closed issue #9714: Date partition transforms are underspecified URL: https://github.com/apache/iceberg/issues/9714 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Logs: More precise error message for Iceberg Partition Parsing [iceberg]

github-actions[bot] closed pull request #10047: Logs: More precise error message for Iceberg Partition Parsing URL: https://github.com/apache/iceberg/pull/10047 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [I] Newly generated Positional Delete file has lowerbound & upperbound values as empty after running rewrite_position_delete_files spark procedure [iceberg]

github-actions[bot] commented on issue #10146: URL: https://github.com/apache/iceberg/issues/10146#issuecomment-2448718984 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] spark.table() raises warn: Unclosed S3FileIO instance in NessieTableOperations [iceberg]

github-actions[bot] commented on issue #10144: URL: https://github.com/apache/iceberg/issues/10144#issuecomment-2448718945 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Core: Uncached files not be materialized [iceberg]

github-actions[bot] closed pull request #10041: Core: Uncached files not be materialized URL: https://github.com/apache/iceberg/pull/10041 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Spark: Hive-View Integration Tests [iceberg]

github-actions[bot] closed pull request #10088: Spark: Hive-View Integration Tests URL: https://github.com/apache/iceberg/pull/10088 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [I] Truncate does not support long widths [iceberg]

github-actions[bot] commented on issue #9716: URL: https://github.com/apache/iceberg/issues/9716#issuecomment-2448718228 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [PR] Aliyun: Add loading oss file into memory option to OSS client properties [iceberg]

github-actions[bot] commented on PR #10062: URL: https://github.com/apache/iceberg/pull/10062#issuecomment-2448718811 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [I] Truncate does not support long widths [iceberg]

github-actions[bot] closed issue #9716: Truncate does not support long widths URL: https://github.com/apache/iceberg/issues/9716 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] Date partition transforms are underspecified [iceberg]

github-actions[bot] commented on issue #9714: URL: https://github.com/apache/iceberg/issues/9714#issuecomment-2448718030 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [PR] (AWS) Docs: List all AWS S3 properties from all language impl. [iceberg]

danielcweeks commented on PR #11383: URL: https://github.com/apache/iceberg/pull/11383#issuecomment-2448671764 @Neuw84 it looks like we're duplicating what should be EMR documentation here. We already link off to the EMR docs, so I don't feel this is the right place for putting specific co

Re: [PR] (AWS) Docs: List all AWS S3 properties from all language impl. [iceberg]

danielcweeks commented on code in PR #11383: URL: https://github.com/apache/iceberg/pull/11383#discussion_r1823593819 ## docs/docs/aws.md: ## @@ -669,6 +669,13 @@ Users can use catalog properties to override the defaults. For example, to confi --conf spark.sql.catalog.my_catal

Re: [PR] (AWS) Docs: List all AWS S3 properties from all language impl. [iceberg]

danielcweeks commented on code in PR #11383: URL: https://github.com/apache/iceberg/pull/11383#discussion_r1823584270 ## docs/docs/aws.md: ## @@ -717,13 +724,21 @@ install_dependencies () { install_dependencies $LIB_PATH $ICEBERG_MAVEN_URL $ICEBERG_VERSION "${ICEBERG_PACKAGES[

Re: [I] Iceberg does not work with Spark's default hive metastore (embedded Derby database) [iceberg]

RussellSpitzer commented on issue #7847: URL: https://github.com/apache/iceberg/issues/7847#issuecomment-2448632538 > Hi all, has anyone figured out how to get this working nicely when used with pyspark + pytest? The solution proposed in this [comment](https://github.com/apache/iceberg/issu

[PR] Bump coverage from 7.6.1 to 7.6.4 [iceberg-python]

dependabot[bot] opened a new pull request, #1275: URL: https://github.com/apache/iceberg-python/pull/1275 Bumps [coverage](https://github.com/nedbat/coveragepy) from 7.6.1 to 7.6.4. Changelog Sourced from https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst";>coverage's cha

[PR] Bump pyparsing from 3.1.4 to 3.2.0 [iceberg-python]

dependabot[bot] opened a new pull request, #1274: URL: https://github.com/apache/iceberg-python/pull/1274 Bumps [pyparsing](https://github.com/pyparsing/pyparsing) from 3.1.4 to 3.2.0. Changelog Sourced from https://github.com/pyparsing/pyparsing/blob/master/CHANGES";>pyparsing's

Re: [PR] Build: Bump com.aliyun.oss:aliyun-sdk-oss from 3.10.2 to 3.18.1 [iceberg]

Fokko commented on PR #10799: URL: https://github.com/apache/iceberg/pull/10799#issuecomment-2447283567 @xingbowu Should we update this dependency? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] Iceberg does not work with Spark's default hive metastore (embedded Derby database) [iceberg]

JeroenSchmidt commented on issue #7847: URL: https://github.com/apache/iceberg/issues/7847#issuecomment-2448565183 Hi all, has anyone figured out how to get this working nicely when used with pyspark + pytest? The solution proposed in this [comment](https://github.com/apache/iceberg/issu

Re: [PR] API: Add compatibility checks for Schemas with default values [iceberg]

RussellSpitzer commented on PR #11434: URL: https://github.com/apache/iceberg/pull/11434#issuecomment-2448510107 Thanks everyone - @rdblue (for original fix), @Fokko , @anuragmantri , @kevinjqliu , @singhpk234 , @amogh-jahagirdar For reviews! -- This is an automated message from the Apac

Re: [PR] API: Add compatibility checks for Schemas with default values [iceberg]

RussellSpitzer merged PR #11434: URL: https://github.com/apache/iceberg/pull/11434 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [I] Remove `numpy` as a dependency [iceberg-python]

corleyma commented on issue #1259: URL: https://github.com/apache/iceberg-python/issues/1259#issuecomment-2448429340 Arrow's release cycles are... not quick, but I have no doubt that logic will land there at some point. -- This is an automated message from the Apache Git Service. To resp

Re: [I] Remove `numpy` as a dependency [iceberg-python]

kevinjqliu commented on issue #1259: URL: https://github.com/apache/iceberg-python/issues/1259#issuecomment-2448411602 supercool! As a project, I don't want to maintain an implementation of the `range` function 😄 Is there a way to contribute `create_arrow_range` to Arrow? -- This is an

Re: [PR] Remove numpy as a hard dependency [iceberg-python]

corleyma commented on PR #1270: URL: https://github.com/apache/iceberg-python/pull/1270#issuecomment-2448396256 @kevinjqliu curious what y'all think of my comment here: https://github.com/apache/iceberg-python/issues/1259#issuecomment-2448393527 -- This is an automated message from the Ap

Re: [I] Remove `numpy` as a dependency [iceberg-python]

corleyma commented on issue #1259: URL: https://github.com/apache/iceberg-python/issues/1259#issuecomment-2448393527 > It looks like we're spending a lot of time on generating the sequence. I think it would be good to add this to Arrow itself: https://github.com/apache/arrow/issues/44583 I

Re: [I] optimize `_combine_positional_deletes` [iceberg-python]

omkenge commented on issue #1271: URL: https://github.com/apache/iceberg-python/issues/1271#issuecomment-2448320956 Hi @kevinjqliu Can we rewrite `_combine_positional_deletes` function by using a `set-based approach` instead of the previous `NumPy` method. The set method significantly

Re: [PR] Bump PyArrow to 18.0.0 [iceberg-python]

Fokko merged PR #1256: URL: https://github.com/apache/iceberg-python/pull/1256 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [I] pyarrow 18 regression: ValueError: type(schema)= [iceberg-python]

grihabor commented on issue #1265: URL: https://github.com/apache/iceberg-python/issues/1265#issuecomment-2448322124 Thanks for verifying, I've caught this in CI on dependabot update and reproduced it locally, I'll try to reproduce the problem in a docker container -- This is an automate

Re: [PR] Remove numpy as a hard dependency [iceberg-python]

Fokko merged PR #1270: URL: https://github.com/apache/iceberg-python/pull/1270 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [I] NumPy Hardpin 1.26 issue [iceberg-python]

Fokko closed issue #1198: NumPy Hardpin 1.26 issue URL: https://github.com/apache/iceberg-python/issues/1198 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

Re: [I] Remove `numpy` as a dependency [iceberg-python]

Fokko closed issue #1259: Remove `numpy` as a dependency URL: https://github.com/apache/iceberg-python/issues/1259 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

Re: [I] pyarrow 18 regression: ValueError: type(schema)= [iceberg-python]

Fokko closed issue #1265: pyarrow 18 regression: ValueError: type(schema)= URL: https://github.com/apache/iceberg-python/issues/1265 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] API: Add compatibility checks for Schemas with default values [iceberg]

RussellSpitzer commented on code in PR #11434: URL: https://github.com/apache/iceberg/pull/11434#discussion_r1823318178 ## api/src/main/java/org/apache/iceberg/Schema.java: ## @@ -586,16 +587,37 @@ private List reassignIds(List columns, TypeUtil.GetID * @param formatVersion

Re: [PR] API: Add compatibility checks for Schemas with default values [iceberg]

RussellSpitzer commented on code in PR #11434: URL: https://github.com/apache/iceberg/pull/11434#discussion_r1823317138 ## api/src/main/java/org/apache/iceberg/Schema.java: ## @@ -586,16 +587,37 @@ private List reassignIds(List columns, TypeUtil.GetID * @param formatVersion

Re: [PR] API: Add compatibility checks for Schemas with default values [iceberg]

singhpk234 commented on code in PR #11434: URL: https://github.com/apache/iceberg/pull/11434#discussion_r1823309114 ## api/src/main/java/org/apache/iceberg/Schema.java: ## @@ -586,16 +587,37 @@ private List reassignIds(List columns, TypeUtil.GetID * @param formatVersion tab

Re: [PR] API: Add compatibility checks for Schemas with default values [iceberg]

Fokko commented on code in PR #11434: URL: https://github.com/apache/iceberg/pull/11434#discussion_r1823307130 ## api/src/main/java/org/apache/iceberg/Schema.java: ## @@ -54,6 +54,7 @@ public class Schema implements Serializable { private static final Joiner NEWLINE = Joiner.

Re: [I] Implement `TableProviderFactory` for a `IcebergTableFactory` [iceberg-rust]

mkarbo commented on issue #586: URL: https://github.com/apache/iceberg-rust/issues/586#issuecomment-2448189813 Will this enable `insert into` as well for iceberg? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Build: Bump pytest from 7.4.4 to 8.0.0 [iceberg-python]

Fokko commented on PR #393: URL: https://github.com/apache/iceberg-python/pull/393#issuecomment-2448142367 @hussein-awala It looks like there is a fork [that's compatible with `pytest>=7.0.0`](https://github.com/dev-petrov/pytest-lazy-fixtures/blob/e23be56fb14f3452763309ad6fac1646009b7156/py

Re: [I] Block writing to sorted tables [iceberg-python]

kevinjqliu commented on issue #1247: URL: https://github.com/apache/iceberg-python/issues/1247#issuecomment-2448132613 According to #271, "We fail when we see a sort order". -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [I] Support partitioned writes [iceberg-python]

Fokko commented on issue #208: URL: https://github.com/apache/iceberg-python/issues/208#issuecomment-2448147351 @RLashofRegas Sorry for the long wait, @sungwy has been working on adding a rust extension to efficiently run the bucketing transform 🥳 We're blocked on a release on the rust sid

Re: [PR] Bump mkdocs-material from 9.5.36 to 9.5.42 [iceberg-python]

Fokko merged PR #1266: URL: https://github.com/apache/iceberg-python/pull/1266 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [I] Support writing to a table with sort-order [iceberg-python]

Fokko commented on issue #271: URL: https://github.com/apache/iceberg-python/issues/271#issuecomment-2448131475 Let's pass this to the next release when we have all the transforms implemented using the Rust extension. cc @sungwy -- This is an automated message from the Apache Git Service

Re: [I] Block writing to sorted tables [iceberg-python]

kevinjqliu commented on issue #1247: URL: https://github.com/apache/iceberg-python/issues/1247#issuecomment-2448078653 This might be a correctness issue. Maybe we can disable writing to a table with sort order for now, similar to what we do with equality deletes https://github.com/a

Re: [I] Block writing to sorted tables [iceberg-python]

kevinjqliu commented on issue #1247: URL: https://github.com/apache/iceberg-python/issues/1247#issuecomment-2448076726 > From the [docs](https://iceberg.apache.org/docs/latest/spark-ddl/#alter-table-write-ordered-by) I'm understanding that sort order is purely suggestive, and it is up to t

Re: [PR] API: Add compatibility checks for Schemas with default values [iceberg]

rdblue commented on PR #11434: URL: https://github.com/apache/iceberg/pull/11434#issuecomment-2448072176 Updated. Thanks, @RussellSpitzer! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] API: Add compatibility checks for Schemas with default values [iceberg]

RussellSpitzer commented on code in PR #11434: URL: https://github.com/apache/iceberg/pull/11434#discussion_r1823174316 ## api/src/main/java/org/apache/iceberg/Schema.java: ## @@ -586,16 +588,37 @@ private List reassignIds(List columns, TypeUtil.GetID * @param formatVersion

Re: [PR] Remove numpy as a hard dependency [iceberg-python]

Fokko commented on PR #1270: URL: https://github.com/apache/iceberg-python/pull/1270#issuecomment-2448010464 @kevinjqliu That's an excellent suggestion, I only created one on the Arrow side, thanks! -- This is an automated message from the Apache Git Service. To respond to the message, pl

Re: [PR] Remove numpy as a hard dependency [iceberg-python]

Fokko commented on code in PR #1270: URL: https://github.com/apache/iceberg-python/pull/1270#discussion_r1823175129 ## pyiceberg/io/pyarrow.py: ## @@ -812,7 +811,15 @@ def _combine_positional_deletes(positional_deletes: List[pa.ChunkedArray], start all_chunks = positio

  1   2   >