Re: [PR] REST: Refactor docker image files [iceberg]

ajantha-bhat commented on code in PR #11676: URL: https://github.com/apache/iceberg/pull/11676#discussion_r1864021521 ## .github/workflows/publish-iceberg-rest-fixture-docker.yml: ## @@ -28,6 +28,7 @@ on: env: DOCKER_IMAGE_TAG: iceberg-rest-fixture DOCKER_IMAGE_VERSION: l

Re: [PR] (AWS) Docs: List all AWS S3 properties from all language impl. [iceberg]

github-actions[bot] closed pull request #11321: (AWS) Docs: List all AWS S3 properties from all language impl. URL: https://github.com/apache/iceberg/pull/11321 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] OpenAPI: Define REST Catalog models for Snapshot Production [iceberg]

github-actions[bot] closed pull request #11287: OpenAPI: Define REST Catalog models for Snapshot Production URL: https://github.com/apache/iceberg/pull/11287 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [PR] (AWS) Docs: List all AWS S3 properties from all language impl. [iceberg]

github-actions[bot] commented on PR #11321: URL: https://github.com/apache/iceberg/pull/11321#issuecomment-2508748365 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] [WIP] API, Core, Spark: Proof of concept implementation of the geo support proposal [iceberg]

github-actions[bot] commented on PR #11293: URL: https://github.com/apache/iceberg/pull/11293#issuecomment-2508748342 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] `rewrite_data_files` does not respect table sort order [iceberg]

github-actions[bot] closed issue #10346: `rewrite_data_files` does not respect table sort order URL: https://github.com/apache/iceberg/issues/10346 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] Build: Bump com.aliyun.oss:aliyun-sdk-oss from 3.10.2 to 3.18.1 [iceberg]

github-actions[bot] commented on PR #10799: URL: https://github.com/apache/iceberg/pull/10799#issuecomment-2508748289 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Support convert orc timestamptz [iceberg]

github-actions[bot] closed pull request #9905: Support convert orc timestamptz URL: https://github.com/apache/iceberg/pull/9905 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Support convert orc timestamptz [iceberg]

github-actions[bot] commented on PR #9905: URL: https://github.com/apache/iceberg/pull/9905#issuecomment-2508748226 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Core: Add list/map block sizes [iceberg]

github-actions[bot] commented on PR #10973: URL: https://github.com/apache/iceberg/pull/10973#issuecomment-2508748312 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] OpenAPI: Define REST Catalog models for Snapshot Production [iceberg]

github-actions[bot] commented on PR #11287: URL: https://github.com/apache/iceberg/pull/11287#issuecomment-2508748334 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] Core: Suppress exceptions in case of dropTableData [iceberg]

github-actions[bot] commented on PR #9184: URL: https://github.com/apache/iceberg/pull/9184#issuecomment-2508748199 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [I] `rewrite_data_files` does not respect table sort order [iceberg]

github-actions[bot] commented on issue #10346: URL: https://github.com/apache/iceberg/issues/10346#issuecomment-2508748242 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [PR] WIP: Testing out using Polaris Docker image in Integration Test Suite [iceberg-python]

sungwy closed pull request #1252: WIP: Testing out using Polaris Docker image in Integration Test Suite URL: https://github.com/apache/iceberg-python/pull/1252 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] WIP: Testing out using Polaris Docker image in Integration Test Suite [iceberg-python]

sungwy commented on PR #1252: URL: https://github.com/apache/iceberg-python/pull/1252#issuecomment-2508644198 Closed in favor of: https://github.com/apache/iceberg-python/pull/1389 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[PR] TEST: adopt new rest catalog image and enable tableExists tests [iceberg-python]

sungwy opened a new pull request, #1389: URL: https://github.com/apache/iceberg-python/pull/1389 the test is failing because tableExists endpoint isn't exposed on the adapter -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Spark: Remove extra columns for ColumnBatch [iceberg]

flyrain commented on code in PR #11551: URL: https://github.com/apache/iceberg/pull/11551#discussion_r1863894343 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/ColumnarBatchReader.java: ## @@ -245,5 +259,16 @@ void applyEqDelete(ColumnarBatch columnar

Re: [PR] REST: Refactor docker image files [iceberg]

kevinjqliu commented on code in PR #11676: URL: https://github.com/apache/iceberg/pull/11676#discussion_r1863874986 ## .github/workflows/publish-iceberg-rest-fixture-docker.yml: ## @@ -28,6 +28,7 @@ on: env: DOCKER_IMAGE_TAG: iceberg-rest-fixture DOCKER_IMAGE_VERSION: lat

Re: [PR] REST: Refactor docker image files [iceberg]

kevinjqliu commented on code in PR #11676: URL: https://github.com/apache/iceberg/pull/11676#discussion_r1863872835 ## docker/iceberg-rest-fixture/README.md: ## @@ -32,7 +32,7 @@ When making changes to the local files and test them out, you can build the imag ./gradlew :iceber

Re: [PR] REST: Refactor docker image files [iceberg]

sungwy commented on code in PR #11676: URL: https://github.com/apache/iceberg/pull/11676#discussion_r1863869770 ## docker/iceberg-rest-fixture/README.md: ## @@ -32,7 +32,7 @@ When making changes to the local files and test them out, you can build the imag ./gradlew :iceberg-op

Re: [PR] Create publish-docker.yml [iceberg]

kevinjqliu commented on PR #11632: URL: https://github.com/apache/iceberg/pull/11632#issuecomment-2508360201 I saw that this action was run manually https://github.com/apache/iceberg/actions/runs/12082416551 but I cant find the image on docker https://hub.docker.com/search?q=iceberg-rest

Re: [I] Iceberg rest catalog with DynamodbCatalog implementation is throwing an exception with *dynamodb local* build. [iceberg]

krisroops commented on issue #11674: URL: https://github.com/apache/iceberg/issues/11674#issuecomment-2508110710 # Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional inform

Re: [I] Iceberg rest catalog with DynamodbCatalog implementation is throwing an exception with *dynamodb local* build. [iceberg]

krisroops commented on issue #11674: URL: https://github.com/apache/iceberg/issues/11674#issuecomment-2508111320 Somehow I am not able to attach the compose.yml, so pasted the contents above. -- This is an automated message from the Apache Git Service. To respond to the message, please l

Re: [PR] Spark: Write DVs for V3 MoR tables [iceberg]

amogh-jahagirdar commented on code in PR #11561: URL: https://github.com/apache/iceberg/pull/11561#discussion_r1863716254 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkPositionDeltaWrite.java: ## @@ -483,6 +493,14 @@ protected PartitioningWriter, Delete

Re: [PR] Spark: Write DVs for V3 MoR tables [iceberg]

amogh-jahagirdar commented on code in PR #11561: URL: https://github.com/apache/iceberg/pull/11561#discussion_r1863714851 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkPositionDeltaWrite.java: ## @@ -483,6 +493,14 @@ protected PartitioningWriter, Delete

Re: [PR] Spark: Write DVs for V3 MoR tables [iceberg]

amogh-jahagirdar commented on code in PR #11561: URL: https://github.com/apache/iceberg/pull/11561#discussion_r1863707596 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkBatchQueryScan.java: ## @@ -168,7 +168,7 @@ protected Map rewritableDeletes() { f

Re: [I] Iceberg rest catalog with DynamodbCatalog implementation is throwing an exception with *dynamodb local* build. [iceberg]

nastra commented on issue #11674: URL: https://github.com/apache/iceberg/issues/11674#issuecomment-2508057237 Note that ``` AWS_ACCESS_KEY_ID=TempKeyId123 AWS_SECRET_ACCESS_KEY=TempSecretAccessKey123 AWS_REGION=us-east-1 ``` are also required for Minio, so make sure al

Re: [PR] Docs: Default value of table level distribution-mode should be not set [iceberg]

nastra commented on code in PR #11663: URL: https://github.com/apache/iceberg/pull/11663#discussion_r1863697062 ## docs/docs/configuration.md: ## @@ -67,10 +67,10 @@ Iceberg tables support table properties to configure table behavior, like the de | write.metadata.metrics.colum

Re: [I] Iceberg rest catalog with DynamodbCatalog implementation is throwing an exception with *dynamodb local* build. [iceberg]

nastra commented on issue #11674: URL: https://github.com/apache/iceberg/issues/11674#issuecomment-2507995468 @krisroops I don't think this issue is directly related to Iceberg. The error `The security token included in the request is invalid` indicates that the security token is invalid

Re: [PR] Replace wrong order of assertion [iceberg]

nastra merged PR #11677: URL: https://github.com/apache/iceberg/pull/11677 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Bump pyarrow from 18.0.0 to 18.1.0 [iceberg-python]

Fokko merged PR #1381: URL: https://github.com/apache/iceberg-python/pull/1381 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Count rows as a metadata only operation [iceberg-python]

Fokko commented on code in PR #1388: URL: https://github.com/apache/iceberg-python/pull/1388#discussion_r1863537090 ## pyiceberg/table/__init__.py: ## @@ -1493,6 +1496,13 @@ def to_ray(self) -> ray.data.dataset.Dataset: return ray.data.from_arrow(self.to_arrow()) +

Re: [PR] Count rows as a metadata only operation [iceberg-python]

tusharchou commented on code in PR #1388: URL: https://github.com/apache/iceberg-python/pull/1388#discussion_r1863507446 ## pyiceberg/table/__init__.py: ## @@ -1493,6 +1496,13 @@ def to_ray(self) -> ray.data.dataset.Dataset: return ray.data.from_arrow(self.to_arrow())

Re: [I] Update Table Error: UPDATE TABLE is not supported temporarily. [iceberg]

Syed-bug commented on issue #9960: URL: https://github.com/apache/iceberg/issues/9960#issuecomment-2507785079 @jurossiar Thanks a lot for all the information...I have fixed the issue by changing the jars.. -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [I] Tracking issues of iceberg rust v0.4.0 Release [iceberg-rust]

Xuanwo commented on issue #739: URL: https://github.com/apache/iceberg-rust/issues/739#issuecomment-2507782914 I believe we need something like https://github.com/apache/opendal/blob/main/scripts/release.py to help generate ASF source tarballs for core and python binding. -- This is an a

Re: [I] Update Table Error: UPDATE TABLE is not supported temporarily. [iceberg]

jurossiar commented on issue #9960: URL: https://github.com/apache/iceberg/issues/9960#issuecomment-2507769859 @Syed-bug The issue is with spark-version: 3.5. I though you had the issue and wanted to be able to update tables. :) Refers to my comments: - https://github.com/apache/

Re: [PR] Flink: Fix range distribution npe when value is null [iceberg]

Guosmilesmile commented on PR #11662: URL: https://github.com/apache/iceberg/pull/11662#issuecomment-2507769750 Hi @mxm! Thank you very much for your suggestions. Following your advice, I added a version number to SortKeySerializer through restoreSerializer, implementing different serializa

Re: [I] Generation of Snapshot Summaries [iceberg-rust]

barronw commented on issue #724: URL: https://github.com/apache/iceberg-rust/issues/724#issuecomment-2507766282 Can I pick this up? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] REST: Refactor docker image files [iceberg]

Fokko commented on code in PR #11676: URL: https://github.com/apache/iceberg/pull/11676#discussion_r1863459375 ## .github/workflows/publish-iceberg-rest-fixture-docker.yml: ## @@ -42,14 +42,14 @@ jobs: run: ./gradlew :iceberg-open-api:shadowJar - name: Login to Docke

[PR] REST: Refactor docker image files [iceberg]

ajantha-bhat opened a new pull request, #11676: URL: https://github.com/apache/iceberg/pull/11676 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

[PR] Core, Spark: Include content offset/size in PositionDeletesTable [iceberg]

nastra opened a new pull request, #11675: URL: https://github.com/apache/iceberg/pull/11675 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

[I] Iceberg rest catalog with DynamodbCatalog implementation is throwing an exception with *dynamodb local* build. [iceberg]

krisroops opened a new issue, #11674: URL: https://github.com/apache/iceberg/issues/11674 ### Apache Iceberg version None ### Query engine Spark ### Please describe the bug 🐞 I'm setting up a spark iceberg cluster on local environment by spinning up contai

Re: [PR] REST: Use apache/iceberg-rest-fixture docker image [iceberg]

Fokko commented on PR #11673: URL: https://github.com/apache/iceberg/pull/11673#issuecomment-2507538087 @ajantha-bhat Should we split out the renaming into a separate PR? I think it would be good to get that cleaned up ASAP. We can combine the rename with decoupling the user and the reposit

Re: [PR] Spark: Read DVs when reading from .position_deletes table [iceberg]

nastra commented on code in PR #11657: URL: https://github.com/apache/iceberg/pull/11657#discussion_r1863303958 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/DVIterable.java: ## @@ -0,0 +1,158 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] Flink: Fix range distribution npe when value is null [iceberg]

mxm commented on code in PR #11662: URL: https://github.com/apache/iceberg/pull/11662#discussion_r1863289530 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/SortKeySerializer.java: ## @@ -310,10 +324,16 @@ public void writeSnapshot(DataOutputView out) th

Re: [PR] Flink: Fix range distribution npe when value is null [iceberg]

mxm commented on code in PR #11662: URL: https://github.com/apache/iceberg/pull/11662#discussion_r1863289530 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/SortKeySerializer.java: ## @@ -310,10 +324,16 @@ public void writeSnapshot(DataOutputView out) th

Re: [PR] REST: Use apache/iceberg-rest-fixture docker image [iceberg]

ajantha-bhat commented on PR #11673: URL: https://github.com/apache/iceberg/pull/11673#issuecomment-2507462690 waiting to publish image first to the dockerhub -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[PR] refine: refine writer interface [iceberg-rust]

ZENOTME opened a new pull request, #741: URL: https://github.com/apache/iceberg-rust/pull/741 Hi, I'm working on writer recently and find some weaknesses of our original writer interface design: 1. Our IcebergWriter Builder interface looks like following: ``` trait IcebergWriterBuil

[I] [Arrow]Incorrect argument passed to setInitialCapacity in allocateVectorBasedOnTypeName method (should be count of values, not bytes) [iceberg]

doki23 opened a new issue, #11672: URL: https://github.com/apache/iceberg/issues/11672 ### Apache Iceberg version main (development) ### Query engine None ### Please describe the bug 🐞 In the `allocateVectorBasedOnTypeName` method of `VectorizedArrowReader`,

Re: [PR] Create publish-docker.yml [iceberg]

Fokko merged PR #11632: URL: https://github.com/apache/iceberg/pull/11632 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Create publish-docker.yml [iceberg]

Fokko commented on PR #11632: URL: https://github.com/apache/iceberg/pull/11632#issuecomment-2507427785 @sungwy Thanks for raising this and the thorough testing. I also did some tests against my personal Dockerhub, so let's push this forward. -- This is an automated message from the Apach

Re: [I] Gradle task to update LICENSE and NOTICE on every build for runtime jars [iceberg]

Fokko commented on issue #11559: URL: https://github.com/apache/iceberg/issues/11559#issuecomment-2507425500 Thanks for raising this issue @ajantha-bhat. I think this is pretty important and happy to help where possible. -- This is an automated message from the Apache Git Service. To resp

Re: [PR] Docs: Add guidelines for contributors to become committers [iceberg]

Fokko commented on code in PR #11670: URL: https://github.com/apache/iceberg/pull/11670#discussion_r1863168496 ## site/docs/community.md: ## @@ -76,7 +76,9 @@ Iceberg has four mailing lists: The Apache Iceberg community is built on the principles described in the [Apache Way