Re: [PR] Bump pyspark from 3.5.3 to 3.5.4 [iceberg-python]

Fokko commented on PR #1461: URL: https://github.com/apache/iceberg-python/pull/1461#issuecomment-2572498850 Thanks @hussein-awala, we're blocked on https://github.com/apache/iceberg/pull/11731 which should be part of the 1.7.2 release -- This is an automated message from the Apache Git

Re: [I] Move mkdocs action/workflow into `docs` group [iceberg-python]

Fokko closed issue #923: Move mkdocs action/workflow into `docs` group URL: https://github.com/apache/iceberg-python/issues/923 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [infra] Move `mkdocs` to use poetry as `docs` group dep [iceberg-python]

Fokko merged PR #1486: URL: https://github.com/apache/iceberg-python/pull/1486 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [I] [Java API] Rough edges when partitioning by time types [iceberg]

Gezi-lzq commented on issue #11899: URL: https://github.com/apache/iceberg/issues/11899#issuecomment-2572232239 > I think the issue here is that the Copy constructor for GenericRecord does not do type checking. The accessor is failing because the Generic record has an illegal object in it.

Re: [PR] feat(puffin): Parse Puffin FileMetadata [iceberg-rust]

fqaiser94 commented on code in PR #765: URL: https://github.com/apache/iceberg-rust/pull/765#discussion_r1903607534 ## crates/iceberg/src/puffin/metadata.rs: ## @@ -0,0 +1,797 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agree

Re: [PR] feat(puffin): Parse Puffin FileMetadata [iceberg-rust]

fqaiser94 commented on code in PR #765: URL: https://github.com/apache/iceberg-rust/pull/765#discussion_r1903607534 ## crates/iceberg/src/puffin/metadata.rs: ## @@ -0,0 +1,797 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agree

Re: [PR] feat(puffin): Parse Puffin FileMetadata [iceberg-rust]

fqaiser94 commented on code in PR #765: URL: https://github.com/apache/iceberg-rust/pull/765#discussion_r1903608079 ## crates/iceberg/src/puffin/metadata.rs: ## @@ -0,0 +1,797 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agree

Re: [I] How to apply partition/bloom filter to old data? Does rewrite_data_files/rewrite_manifests procedure work? [iceberg]

madeirak commented on issue #11878: URL: https://github.com/apache/iceberg/issues/11878#issuecomment-2572205430 > by the way, ALTER TABLE prod.db.sample SET TBLPROPERTIES . Do you meet any other problems? After adding the bloom filter related table properties, the data written will h

Re: [PR] Backport #11557 to FLink1.19 and 1.18 [iceberg]

huyuanfeng2018 commented on PR #11834: URL: https://github.com/apache/iceberg/pull/11834#issuecomment-2572203776 @stevenzwu This PR is ready. Please take a look at it if you have time. Thanks~ -- This is an automated message from the Apache Git Service. To respond to the messag

Re: [PR] feat(puffin): Parse Puffin FileMetadata [iceberg-rust]

fqaiser94 commented on code in PR #765: URL: https://github.com/apache/iceberg-rust/pull/765#discussion_r1903610687 ## crates/iceberg/src/puffin/metadata.rs: ## @@ -0,0 +1,797 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agree

Re: [PR] feat(puffin): Parse Puffin FileMetadata [iceberg-rust]

fqaiser94 commented on code in PR #765: URL: https://github.com/apache/iceberg-rust/pull/765#discussion_r1903609625 ## crates/iceberg/src/puffin/metadata.rs: ## @@ -0,0 +1,797 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agree

Re: [I] HiveTableOperations may incorrectly consider a successful commit as failed [iceberg]

lirui-apache closed issue #11866: HiveTableOperations may incorrectly consider a successful commit as failed URL: https://github.com/apache/iceberg/issues/11866 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [I] HiveTableOperations may incorrectly consider a successful commit as failed [iceberg]

lirui-apache commented on issue #11866: URL: https://github.com/apache/iceberg/issues/11866#issuecomment-2572200605 @sauliusvl @pvary Thanks for the pointer, closing this one as duplicate -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] feat(puffin): Parse Puffin FileMetadata [iceberg-rust]

fqaiser94 commented on code in PR #765: URL: https://github.com/apache/iceberg-rust/pull/765#discussion_r1903608079 ## crates/iceberg/src/puffin/metadata.rs: ## @@ -0,0 +1,797 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agree

Re: [PR] feat(puffin): Parse Puffin FileMetadata [iceberg-rust]

fqaiser94 commented on code in PR #765: URL: https://github.com/apache/iceberg-rust/pull/765#discussion_r1903607534 ## crates/iceberg/src/puffin/metadata.rs: ## @@ -0,0 +1,797 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agree

Re: [I] When MAP and ARRAY are next to each other, changing the field name inside the nested field will cause an ERROR [iceberg]

madeirak closed issue #11872: When MAP and ARRAY are next to each other, changing the field name inside the nested field will cause an ERROR URL: https://github.com/apache/iceberg/issues/11872 -- This is an automated message from the Apache Git Service. To respond to the message, please log o

Re: [I] When MAP and ARRAY are next to each other, changing the field name inside the nested field will cause an ERROR [iceberg]

madeirak commented on issue #11872: URL: https://github.com/apache/iceberg/issues/11872#issuecomment-2572194150 > @madeirak this seems to be working on iceberg runtime 1.5.0 and spark 3.5.1. > > Here is what I did: > > ``` > # Setup demo infra > ## use docker-compose.yaml

Re: [PR] Update maintenance.md [iceberg]

hgeraldino closed pull request #11917: Update maintenance.md URL: https://github.com/apache/iceberg/pull/11917 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

Re: [PR] Fix read from multiple s3 regions [iceberg-python]

kevinjqliu commented on code in PR #1453: URL: https://github.com/apache/iceberg-python/pull/1453#discussion_r1903446304 ## pyiceberg/io/pyarrow.py: ## @@ -351,77 +351,141 @@ def parse_location(location: str) -> Tuple[str, str, str]: return uri.scheme, uri.netloc,

Re: [PR] feat(datafusion): support metadata tables for Datafusion [iceberg-rust]

xxchan commented on PR #879: URL: https://github.com/apache/iceberg-rust/pull/879#issuecomment-2572114662 cc @liurenjie1024 @Xuanwo @rshkv @flaneur2020 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] chore(deps): Bump tempfile from 3.14.0 to 3.15.0 [iceberg-rust]

liurenjie1024 merged PR #878: URL: https://github.com/apache/iceberg-rust/pull/878 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] chore(deps): Bump async-trait from 0.1.83 to 0.1.84 [iceberg-rust]

liurenjie1024 merged PR #877: URL: https://github.com/apache/iceberg-rust/pull/877 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] chore(deps): Bump moka from 0.12.8 to 0.12.9 [iceberg-rust]

liurenjie1024 merged PR #876: URL: https://github.com/apache/iceberg-rust/pull/876 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] chore(deps): Bump aws-sdk-s3tables from 1.2.0 to 1.3.0 [iceberg-rust]

liurenjie1024 merged PR #874: URL: https://github.com/apache/iceberg-rust/pull/874 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] chore(deps): Bump reqwest from 0.12.11 to 0.12.12 [iceberg-rust]

liurenjie1024 merged PR #875: URL: https://github.com/apache/iceberg-rust/pull/875 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] chore(deps): Bump crate-ci/typos from 1.28.4 to 1.29.4 [iceberg-rust]

liurenjie1024 commented on PR #873: URL: https://github.com/apache/iceberg-rust/pull/873#issuecomment-2571979021 Thanks @kevinjqliu for review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] chore(deps): Bump crate-ci/typos from 1.28.4 to 1.29.4 [iceberg-rust]

liurenjie1024 merged PR #873: URL: https://github.com/apache/iceberg-rust/pull/873 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [I] Flaky test `TestCopyOnWriteDelete > testDeleteWithSnapshotIsolation()` [iceberg]

ebyhr commented on issue #11651: URL: https://github.com/apache/iceberg/issues/11651#issuecomment-2571827269 https://github.com/apache/iceberg/actions/runs/12616617874/job/35158039585?pr=11908 -- This is an automated message from the Apache Git Service. To respond to the message, please l

Re: [PR] Build: Bump org.assertj:assertj-core from 3.27.0 to 3.27.2 [iceberg]

ebyhr commented on PR #11908: URL: https://github.com/apache/iceberg/pull/11908#issuecomment-2571827180 CI hit https://github.com/apache/iceberg/issues/11651 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] [infra] Move `mkdocs` to use poetry as `docs` group dep [iceberg-python]

kevinjqliu commented on PR #1486: URL: https://github.com/apache/iceberg-python/pull/1486#issuecomment-2571808770 moved to poetry as `docs` group dep, i like this more -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [I] [DeleteManifest] Making file validation optional [iceberg]

github-actions[bot] commented on issue #10535: URL: https://github.com/apache/iceberg/issues/10535#issuecomment-2571807148 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] Spark rewrite Files Action OOM [iceberg]

github-actions[bot] commented on issue #10054: URL: https://github.com/apache/iceberg/issues/10054#issuecomment-2571807122 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] API: Make the Field-ID required on the `UnboundPartitionSpec` [iceberg]

github-actions[bot] commented on PR #11708: URL: https://github.com/apache/iceberg/pull/11708#issuecomment-2571807259 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Spark: 4.0 snapshot support [iceberg]

github-actions[bot] closed pull request #11583: Spark: 4.0 snapshot support URL: https://github.com/apache/iceberg/pull/11583 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Spark: 4.0 snapshot support [iceberg]

github-actions[bot] commented on PR #11583: URL: https://github.com/apache/iceberg/pull/11583#issuecomment-2571807236 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [I] Iceberg bucket partitioning issue [iceberg]

github-actions[bot] closed issue #10559: Iceberg bucket partitioning issue URL: https://github.com/apache/iceberg/issues/10559 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [I] Iceberg bucket partitioning issue [iceberg]

github-actions[bot] commented on issue #10559: URL: https://github.com/apache/iceberg/issues/10559#issuecomment-2571807154 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] [DeleteManifest] Making file validation optional [iceberg]

github-actions[bot] closed issue #10535: [DeleteManifest] Making file validation optional URL: https://github.com/apache/iceberg/issues/10535 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] [infra] Move mkdocs to use poetry [iceberg-python]

kevinjqliu commented on code in PR #1486: URL: https://github.com/apache/iceberg-python/pull/1486#discussion_r1903366107 ## .github/workflows/python-ci-docs.yml: ## @@ -40,8 +40,6 @@ jobs: with: python-version: 3.12 - name: Install -working-dir

Re: [PR] Fix read from multiple s3 regions [iceberg-python]

jiakai-li commented on code in PR #1453: URL: https://github.com/apache/iceberg-python/pull/1453#discussion_r1903376358 ## pyiceberg/io/pyarrow.py: ## @@ -351,76 +344,146 @@ def parse_location(location: str) -> Tuple[str, str, str]: return uri.scheme, uri.netloc, f

[I] improve repo linter rules [iceberg-python]

kevinjqliu opened a new issue, #1487: URL: https://github.com/apache/iceberg-python/issues/1487 ### Feature Request / Improvement In #1485, we added `"UP035"` and `"UP006"` to the linter ignore list. Let's see if we can resolve the underlying issue instead. Check if we can res

Re: [PR] [infra] Move mkdocs to use poetry [iceberg-python]

kevinjqliu commented on code in PR #1486: URL: https://github.com/apache/iceberg-python/pull/1486#discussion_r1903366107 ## .github/workflows/python-ci-docs.yml: ## @@ -40,8 +40,6 @@ jobs: with: python-version: 3.12 - name: Install -working-dir

Re: [PR] [infra] Move mkdocs to use poetry [iceberg-python]

kevinjqliu commented on code in PR #1486: URL: https://github.com/apache/iceberg-python/pull/1486#discussion_r1903364245 ## .github/workflows/python-ci-docs.yml: ## @@ -40,8 +40,6 @@ jobs: with: python-version: 3.12 - name: Install -working-dir

Re: [PR] Fix read from multiple s3 regions [iceberg-python]

jiakai-li commented on code in PR #1453: URL: https://github.com/apache/iceberg-python/pull/1453#discussion_r1903363474 ## pyiceberg/io/pyarrow.py: ## @@ -351,76 +344,146 @@ def parse_location(location: str) -> Tuple[str, str, str]: return uri.scheme, uri.netloc, f

Re: [PR] [infra] replace `pycln` with `ruff` [iceberg-python]

kevinjqliu merged PR #1485: URL: https://github.com/apache/iceberg-python/pull/1485 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

Re: [PR] [infra] replace `pycln` with `ruff` [iceberg-python]

kevinjqliu commented on code in PR #1485: URL: https://github.com/apache/iceberg-python/pull/1485#discussion_r1903359225 ## .pre-commit-config.yaml: ## @@ -28,26 +28,19 @@ repos: - id: check-yaml - id: check-ast - repo: https://github.com/astral-sh/ruff-pre-comm

Re: [PR] [infra] replace `pycln` with `ruff` [iceberg-python]

Fokko commented on code in PR #1485: URL: https://github.com/apache/iceberg-python/pull/1485#discussion_r1903357651 ## poetry.lock: ## @@ -1896,8 +1896,6 @@ optional = false python-versions = "*" files = [ {file = "jsonpath-ng-1.7.0.tar.gz", hash = "sha256:f6f5f7fd4e5ff7

Re: [PR] [infra] Move mkdocs to use poetry [iceberg-python]

Fokko commented on code in PR #1486: URL: https://github.com/apache/iceberg-python/pull/1486#discussion_r1903357418 ## .github/workflows/python-ci-docs.yml: ## @@ -40,8 +40,6 @@ jobs: with: python-version: 3.12 - name: Install -working-director

Re: [PR] [infra] Move mkdocs to use poetry [iceberg-python]

Fokko commented on code in PR #1486: URL: https://github.com/apache/iceberg-python/pull/1486#discussion_r1903357326 ## pyproject.toml: ## @@ -95,6 +95,17 @@ pyspark = "3.5.3" cython = "3.0.11" deptry = ">=0.14,<0.22" docutils = "!=0.21.post1" # https://github.com/python-po

Re: [PR] [infra] Move mkdocs to use poetry [iceberg-python]

Fokko commented on code in PR #1486: URL: https://github.com/apache/iceberg-python/pull/1486#discussion_r1903357269 ## Makefile: ## @@ -97,3 +97,9 @@ clean: ## Clean up the project Python working environment @find . -name "*.pyd" -exec echo Deleting {} \; -delete

Re: [PR] [infra] replace `pycln` with `ruff` [iceberg-python]

Fokko commented on code in PR #1485: URL: https://github.com/apache/iceberg-python/pull/1485#discussion_r1903356918 ## ruff.toml: ## @@ -58,7 +58,7 @@ select = [ "I", # isort "UP", # pyupgrade ] -ignore = ["E501","E203","B024","B028","UP037"] +ignore = ["E501","E203",

Re: [PR] [infra] replace `pycln` with `ruff` [iceberg-python]

Fokko commented on code in PR #1485: URL: https://github.com/apache/iceberg-python/pull/1485#discussion_r1903356546 ## .pre-commit-config.yaml: ## @@ -28,26 +28,19 @@ repos: - id: check-yaml - id: check-ast - repo: https://github.com/astral-sh/ruff-pre-commit -

Re: [PR] [infra] replace `pycln` with `ruff` [iceberg-python]

Fokko commented on code in PR #1485: URL: https://github.com/apache/iceberg-python/pull/1485#discussion_r1903356460 ## .pre-commit-config.yaml: ## @@ -28,26 +28,19 @@ repos: - id: check-yaml - id: check-ast - repo: https://github.com/astral-sh/ruff-pre-commit -

Re: [PR] [infra] replace `pycln` with `ruff` [iceberg-python]

jiakai-li commented on code in PR #1485: URL: https://github.com/apache/iceberg-python/pull/1485#discussion_r1903356378 ## pyproject.toml: ## @@ -859,6 +860,310 @@ ignore_missing_imports = true module = "tenacity.*" ignore_missing_imports = true +[[tool.mypy.overrides]] Rev

Re: [PR] [infra] replace `pycln` with `ruff` [iceberg-python]

Fokko commented on code in PR #1485: URL: https://github.com/apache/iceberg-python/pull/1485#discussion_r1903356286 ## pyproject.toml: ## @@ -95,6 +95,7 @@ pyspark = "3.5.3" cython = "3.0.11" deptry = ">=0.14,<0.22" docutils = "!=0.21.post1" # https://github.com/python-poe

Re: [PR] Fix read from multiple s3 regions [iceberg-python]

Fokko commented on code in PR #1453: URL: https://github.com/apache/iceberg-python/pull/1453#discussion_r1903352592 ## pyiceberg/io/pyarrow.py: ## @@ -190,13 +190,6 @@ T = TypeVar("T") -class PyArrowLocalFileSystem(pyarrow.fs.LocalFileSystem): Review Comment: People cou

Re: [PR] [infra] replace `pycln` with `ruff` [iceberg-python]

kevinjqliu commented on code in PR #1485: URL: https://github.com/apache/iceberg-python/pull/1485#discussion_r1903355022 ## pyproject.toml: ## @@ -95,6 +95,7 @@ pyspark = "3.5.3" cython = "3.0.11" deptry = ">=0.14,<0.22" docutils = "!=0.21.post1" # https://github.com/pytho

Re: [PR] [infra] Move mkdocs to use poetry [iceberg-python]

kevinjqliu commented on PR #1486: URL: https://github.com/apache/iceberg-python/pull/1486#issuecomment-2571779747 blocked on #1485 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] [infra] replace `pycln` with `ruff` [iceberg-python]

kevinjqliu commented on code in PR #1485: URL: https://github.com/apache/iceberg-python/pull/1485#discussion_r1903355778 ## pyproject.toml: ## @@ -859,6 +860,310 @@ ignore_missing_imports = true module = "tenacity.*" ignore_missing_imports = true +[[tool.mypy.overrides]] Re

Re: [PR] [infra] replace `pycln` with `ruff` [iceberg-python]

kevinjqliu commented on code in PR #1485: URL: https://github.com/apache/iceberg-python/pull/1485#discussion_r1903354935 ## pyproject.toml: ## @@ -859,6 +860,310 @@ ignore_missing_imports = true module = "tenacity.*" ignore_missing_imports = true +[[tool.mypy.overrides]] Re

Re: [PR] [infra] replace `pycln` with `ruff` [iceberg-python]

Fokko commented on code in PR #1485: URL: https://github.com/apache/iceberg-python/pull/1485#discussion_r1903352105 ## .pre-commit-config.yaml: ## @@ -28,26 +28,22 @@ repos: - id: check-yaml - id: check-ast - repo: https://github.com/astral-sh/ruff-pre-commit -

Re: [PR] [infra] replace `pycln` with `ruff` [iceberg-python]

jiakai-li commented on code in PR #1485: URL: https://github.com/apache/iceberg-python/pull/1485#discussion_r190900 ## pyproject.toml: ## @@ -859,6 +860,310 @@ ignore_missing_imports = true module = "tenacity.*" ignore_missing_imports = true +[[tool.mypy.overrides]] Rev

Re: [PR] [infra] replace `pycln` with `ruff` [iceberg-python]

Fokko commented on code in PR #1485: URL: https://github.com/apache/iceberg-python/pull/1485#discussion_r1903351934 ## pyproject.toml: ## @@ -95,6 +95,7 @@ pyspark = "3.5.3" cython = "3.0.11" deptry = ">=0.14,<0.22" docutils = "!=0.21.post1" # https://github.com/python-poe

Re: [PR] [infra] replace `pycln` with `ruff` [iceberg-python]

Fokko commented on code in PR #1485: URL: https://github.com/apache/iceberg-python/pull/1485#discussion_r1903351654 ## .pre-commit-config.yaml: ## @@ -28,26 +28,22 @@ repos: - id: check-yaml - id: check-ast - repo: https://github.com/astral-sh/ruff-pre-commit -

Re: [I] [SPJ] Skweded partitions harm merge performances [iceberg]

aiss93 commented on issue #11800: URL: https://github.com/apache/iceberg/issues/11800#issuecomment-2571751817 Thank you for your reply @szehon-ho I actually get your point regarding the example 2) The idea I was suggesting is to centralize the vision of each replicated task : For e

Re: [PR] feat: support S3 Table Buckets with S3TablesCatalog [iceberg-python]

kevinjqliu commented on code in PR #1429: URL: https://github.com/apache/iceberg-python/pull/1429#discussion_r1903336998 ## pyiceberg/catalog/s3tables.py: ## @@ -0,0 +1,322 @@ +import re +from typing import TYPE_CHECKING, List, Optional, Set, Tuple, Union + +import boto3 + +from

Re: [PR] feat: support S3 Table Buckets with S3TablesCatalog [iceberg-python]

kevinjqliu commented on code in PR #1429: URL: https://github.com/apache/iceberg-python/pull/1429#discussion_r1903334237 ## tests/catalog/test_s3tables.py: ## @@ -0,0 +1,173 @@ +import pytest + +from pyiceberg.catalog.s3tables import S3TableCatalog +from pyiceberg.exceptions imp

Re: [PR] Fix read from multiple s3 regions [iceberg-python]

kevinjqliu commented on PR #1453: URL: https://github.com/apache/iceberg-python/pull/1453#issuecomment-2571735953 https://github.com/apache/iceberg-python/pull/1485 to replace `pycln` with ruff linter -- This is an automated message from the Apache Git Service. To respond to the message,

[PR] [infra] replace `pycln` with `ruff` [iceberg-python]

kevinjqliu opened a new pull request, #1485: URL: https://github.com/apache/iceberg-python/pull/1485 `pycln` has an [issue in CI ](https://github.com/apache/iceberg-python/actions/runs/12612627362/job/35170475290?pr=1453), also see https://github.com/hadialqattan/pycln/issues/249 Sin

Re: [PR] Add pyiceberg DataFusion e2e test [iceberg-rust]

kevinjqliu commented on code in PR #825: URL: https://github.com/apache/iceberg-rust/pull/825#discussion_r1903328953 ## crates/integration_tests/tests/datafusion.rs: ## @@ -0,0 +1,150 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor licen

[PR] AWS: Add support for enabling access to S3 Requester Pays bucket [iceberg]

blitzmohit opened a new pull request, #11915: URL: https://github.com/apache/iceberg/pull/11915 Took a stab at resolving #11912 Note will also need to document the new parameter in aws.md, hoping to get some feedback on the PR before that esp. since this is my first time working with

Re: [PR] feat(datafusion): support metadata tables for Datafusion [iceberg-rust]

xxchan commented on code in PR #879: URL: https://github.com/apache/iceberg-rust/pull/879#discussion_r1903316029 ## crates/integrations/datafusion/tests/integration_datafusion_test.rs: ## @@ -295,3 +303,129 @@ async fn test_table_predict_pushdown() -> Result<()> { assert!(s

Re: [PR] feat(datafusion): support metadata tables for Datafusion [iceberg-rust]

xxchan commented on code in PR #879: URL: https://github.com/apache/iceberg-rust/pull/879#discussion_r1903315532 ## crates/integrations/datafusion/src/schema.rs: ## @@ -89,14 +87,43 @@ impl SchemaProvider for IcebergSchemaProvider { } fn table_names(&self) -> Vec { -

Re: [PR] feat(datafusion): support metadata tables for Datafusion [iceberg-rust]

xxchan commented on code in PR #879: URL: https://github.com/apache/iceberg-rust/pull/879#discussion_r1903315532 ## crates/integrations/datafusion/src/schema.rs: ## @@ -89,14 +87,43 @@ impl SchemaProvider for IcebergSchemaProvider { } fn table_names(&self) -> Vec { -

Re: [PR] Split metadata tables into separate modules [iceberg-rust]

rshkv commented on PR #872: URL: https://github.com/apache/iceberg-rust/pull/872#issuecomment-2571704099 cc @liurenjie1024 @Xuanwo -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[PR] feat(datafusion): support metadata tables for Datafusion [iceberg-rust]

xxchan opened a new pull request, #879: URL: https://github.com/apache/iceberg-rust/pull/879 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-

Re: [PR] feat: support S3 Table Buckets with S3TablesCatalog [iceberg-python]

felixscherz commented on PR #1429: URL: https://github.com/apache/iceberg-python/pull/1429#issuecomment-2571680772 Thank you for the review! I removed tests related to boto3 and set the AWS region explicitly for the test run. I agree with you that we should not merge this as long a

Re: [PR] feat: support S3 Table Buckets with S3TablesCatalog [iceberg-python]

felixscherz commented on code in PR #1429: URL: https://github.com/apache/iceberg-python/pull/1429#discussion_r1903300198 ## pyiceberg/catalog/s3tables.py: ## @@ -0,0 +1,318 @@ +import re +from typing import TYPE_CHECKING, List, Optional, Set, Tuple, Union + +import boto3 + +fro

Re: [PR] feat: support S3 Table Buckets with S3TablesCatalog [iceberg-python]

felixscherz commented on code in PR #1429: URL: https://github.com/apache/iceberg-python/pull/1429#discussion_r1903299209 ## pyiceberg/catalog/s3tables.py: ## @@ -0,0 +1,318 @@ +import re +from typing import TYPE_CHECKING, List, Optional, Set, Tuple, Union + +import boto3 + +fro

Re: [PR] feat: support S3 Table Buckets with S3TablesCatalog [iceberg-python]

felixscherz commented on code in PR #1429: URL: https://github.com/apache/iceberg-python/pull/1429#discussion_r1903299080 ## pyiceberg/catalog/s3tables.py: ## @@ -0,0 +1,318 @@ +import re +from typing import TYPE_CHECKING, List, Optional, Set, Tuple, Union + +import boto3 + +fro

Re: [PR] feat: support S3 Table Buckets with S3TablesCatalog [iceberg-python]

felixscherz commented on code in PR #1429: URL: https://github.com/apache/iceberg-python/pull/1429#discussion_r1903297270 ## pyiceberg/catalog/s3tables.py: ## @@ -0,0 +1,318 @@ +import re +from typing import TYPE_CHECKING, List, Optional, Set, Tuple, Union + +import boto3 + +fro

Re: [PR] feat: support S3 Table Buckets with S3TablesCatalog [iceberg-python]

felixscherz commented on code in PR #1429: URL: https://github.com/apache/iceberg-python/pull/1429#discussion_r1903296990 ## tests/catalog/test_s3tables.py: ## @@ -0,0 +1,180 @@ +import uuid + +import boto3 +import pytest + +from pyiceberg.catalog.s3tables import S3TableCatalog

Re: [PR] Spark 3.5: Implement RewriteTablePath [iceberg]

szehon-ho commented on code in PR #11555: URL: https://github.com/apache/iceberg/pull/11555#discussion_r1903293655 ## core/src/main/java/org/apache/iceberg/io/DeleteSchemaUtil.java: ## @@ -43,4 +43,15 @@ public static Schema pathPosSchema() { public static Schema posDeleteSch

Re: [PR] Spark 3.5: Implement RewriteTablePath [iceberg]

szehon-ho commented on code in PR #11555: URL: https://github.com/apache/iceberg/pull/11555#discussion_r1903293655 ## core/src/main/java/org/apache/iceberg/io/DeleteSchemaUtil.java: ## @@ -43,4 +43,15 @@ public static Schema pathPosSchema() { public static Schema posDeleteSch

Re: [PR] Spark 3.5: Implement RewriteTablePath [iceberg]

szehon-ho commented on code in PR #11555: URL: https://github.com/apache/iceberg/pull/11555#discussion_r1903293655 ## core/src/main/java/org/apache/iceberg/io/DeleteSchemaUtil.java: ## @@ -43,4 +43,15 @@ public static Schema pathPosSchema() { public static Schema posDeleteSch

Re: [PR] Spark 3.5: Implement RewriteTablePath [iceberg]

szehon-ho commented on code in PR #11555: URL: https://github.com/apache/iceberg/pull/11555#discussion_r1903293186 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RewriteTablePathSparkAction.java: ## @@ -0,0 +1,731 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] Spark 3.5: Implement RewriteTablePath [iceberg]

szehon-ho commented on code in PR #11555: URL: https://github.com/apache/iceberg/pull/11555#discussion_r1903293655 ## core/src/main/java/org/apache/iceberg/io/DeleteSchemaUtil.java: ## @@ -43,4 +43,15 @@ public static Schema pathPosSchema() { public static Schema posDeleteSch

Re: [PR] Spark 3.5: Implement RewriteTablePath [iceberg]

szehon-ho commented on code in PR #11555: URL: https://github.com/apache/iceberg/pull/11555#discussion_r1903293186 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RewriteTablePathSparkAction.java: ## @@ -0,0 +1,731 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] Spark 3.5: Implement RewriteTablePath [iceberg]

szehon-ho commented on code in PR #11555: URL: https://github.com/apache/iceberg/pull/11555#discussion_r1903293655 ## core/src/main/java/org/apache/iceberg/io/DeleteSchemaUtil.java: ## @@ -43,4 +43,15 @@ public static Schema pathPosSchema() { public static Schema posDeleteSch

Re: [PR] Spark 3.5: Implement RewriteTablePath [iceberg]

szehon-ho commented on code in PR #11555: URL: https://github.com/apache/iceberg/pull/11555#discussion_r1903293655 ## core/src/main/java/org/apache/iceberg/io/DeleteSchemaUtil.java: ## @@ -43,4 +43,15 @@ public static Schema pathPosSchema() { public static Schema posDeleteSch

Re: [PR] Spark 3.5: Implement RewriteTablePath [iceberg]

szehon-ho commented on code in PR #11555: URL: https://github.com/apache/iceberg/pull/11555#discussion_r1903293655 ## core/src/main/java/org/apache/iceberg/io/DeleteSchemaUtil.java: ## @@ -43,4 +43,15 @@ public static Schema pathPosSchema() { public static Schema posDeleteSch

Re: [PR] Spark 3.5: Implement RewriteTablePath [iceberg]

szehon-ho commented on code in PR #11555: URL: https://github.com/apache/iceberg/pull/11555#discussion_r1903293655 ## core/src/main/java/org/apache/iceberg/io/DeleteSchemaUtil.java: ## @@ -43,4 +43,15 @@ public static Schema pathPosSchema() { public static Schema posDeleteSch

Re: [PR] Spark 3.5: Implement RewriteTablePath [iceberg]

szehon-ho commented on code in PR #11555: URL: https://github.com/apache/iceberg/pull/11555#discussion_r1903293186 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RewriteTablePathSparkAction.java: ## @@ -0,0 +1,731 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] Use SupportsPrefixOperations for Remove OrphanFile Procedure [iceberg]

ismailsimsek commented on code in PR #11906: URL: https://github.com/apache/iceberg/pull/11906#discussion_r1903264071 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/actions/TestRemoveOrphanFilesAction.java: ## @@ -610,9 +613,12 @@ public void testHiddenPathsStarting

[PR] chore(deps): Bump tempfile from 3.14.0 to 3.15.0 [iceberg-rust]

dependabot[bot] opened a new pull request, #878: URL: https://github.com/apache/iceberg-rust/pull/878 Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.14.0 to 3.15.0. Changelog Sourced from https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md";>tempfile's c

[PR] chore(deps): Bump async-trait from 0.1.83 to 0.1.84 [iceberg-rust]

dependabot[bot] opened a new pull request, #877: URL: https://github.com/apache/iceberg-rust/pull/877 Bumps [async-trait](https://github.com/dtolnay/async-trait) from 0.1.83 to 0.1.84. Release notes Sourced from https://github.com/dtolnay/async-trait/releases";>async-trait's relea

[PR] chore(deps): Bump moka from 0.12.8 to 0.12.9 [iceberg-rust]

dependabot[bot] opened a new pull request, #876: URL: https://github.com/apache/iceberg-rust/pull/876 Bumps [moka](https://github.com/moka-rs/moka) from 0.12.8 to 0.12.9. Changelog Sourced from https://github.com/moka-rs/moka/blob/main/CHANGELOG.md";>moka's changelog. Versio

[PR] chore(deps): Bump reqwest from 0.12.11 to 0.12.12 [iceberg-rust]

dependabot[bot] opened a new pull request, #875: URL: https://github.com/apache/iceberg-rust/pull/875 Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.12.11 to 0.12.12. Changelog Sourced from https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md";>reqwest's

[PR] chore(deps): Bump aws-sdk-s3tables from 1.2.0 to 1.3.0 [iceberg-rust]

dependabot[bot] opened a new pull request, #874: URL: https://github.com/apache/iceberg-rust/pull/874 Bumps [aws-sdk-s3tables](https://github.com/awslabs/aws-sdk-rust) from 1.2.0 to 1.3.0. Commits See full diff in https://github.com/awslabs/aws-sdk-rust/commits";>compare view

Re: [PR] Use SupportsPrefixOperations for Remove OrphanFile Procedure [iceberg]

ismailsimsek commented on code in PR #11906: URL: https://github.com/apache/iceberg/pull/11906#discussion_r1903264071 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/actions/TestRemoveOrphanFilesAction.java: ## @@ -610,9 +613,12 @@ public void testHiddenPathsStarting

  1   2   >