Re: [PR] Retry on NoSuchNamespaceException not found in rename table for rest catalog [iceberg]

2025-02-17 Thread via GitHub
nastra commented on code in PR #12159: URL: https://github.com/apache/iceberg/pull/12159#discussion_r1959225795 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -428,7 +429,19 @@ public void renameTable(SessionContext context, TableIdentifier from, T

Re: [PR] Core: BugFix: PartitionStatsUtil#computeStats returns incomplete stats in case of partition evolution [iceberg]

2025-02-17 Thread via GitHub
ajantha-bhat commented on code in PR #12137: URL: https://github.com/apache/iceberg/pull/12137#discussion_r1959224967 ## core/src/test/java/org/apache/iceberg/TestPartitionStatsUtil.java: ## @@ -370,16 +371,167 @@ public void testPartitionStatsWithSchemaEvolution() throws Excep

Re: [PR] Retry on NoSuchNamespaceException not found in rename table for rest catalog [iceberg]

2025-02-17 Thread via GitHub
nastra commented on code in PR #12159: URL: https://github.com/apache/iceberg/pull/12159#discussion_r1959224821 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -428,7 +429,19 @@ public void renameTable(SessionContext context, TableIdentifier from, T

Re: [PR] Retry on NoSuchNamespaceException not found in rename table for rest catalog [iceberg]

2025-02-17 Thread via GitHub
nastra commented on code in PR #12159: URL: https://github.com/apache/iceberg/pull/12159#discussion_r1959224261 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -428,7 +429,19 @@ public void renameTable(SessionContext context, TableIdentifier from, T

Re: [PR] AWS, AZURE: Move docker-based tests to integration test source [iceberg]

2025-02-17 Thread via GitHub
anuragmantri commented on PR #12274: URL: https://github.com/apache/iceberg/pull/12274#issuecomment-2664824667 > so adding check.dependsOn integrationTest on iceberg-aws module would break CI Another option to still run the `S3FileIO` tests is to create a new source set called `docke

Re: [PR] backport c++23 std::expected [iceberg-cpp]

2025-02-17 Thread via GitHub
wgtmac commented on PR #40: URL: https://github.com/apache/iceberg-cpp/pull/40#issuecomment-2664807118 @pitrou @raulcd Do you have any comment on this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [PR] chore: datafusion 45 upgrade [iceberg-rust]

2025-02-17 Thread via GitHub
liurenjie1024 merged PR #943: URL: https://github.com/apache/iceberg-rust/pull/943 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Make field repr explicit about optionality, add exception type [iceberg-cpp]

2025-02-17 Thread via GitHub
wgtmac commented on PR #43: URL: https://github.com/apache/iceberg-cpp/pull/43#issuecomment-2664805968 @Fokko @Xuanwo Could you help merge this? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] AWS, AZURE: Move docker-based tests to integration test source [iceberg]

2025-02-17 Thread via GitHub
nastra commented on PR #12274: URL: https://github.com/apache/iceberg/pull/12274#issuecomment-2664797488 @Fokko can you also take a look please? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] Fix IndexOutOfBounds exception in FileFormat#fromFileName [iceberg]

2025-02-17 Thread via GitHub
nastra commented on code in PR #12301: URL: https://github.com/apache/iceberg/pull/12301#discussion_r1959182084 ## api/src/test/java/org/apache/iceberg/FileFormatTest.java: ## @@ -0,0 +1,86 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contrib

Re: [PR] [infra] nightly pypi build for `pyiceberg_core` [iceberg-rust]

2025-02-17 Thread via GitHub
Xuanwo commented on code in PR #948: URL: https://github.com/apache/iceberg-rust/pull/948#discussion_r1959179801 ## .github/workflows/release_python_nightly.yml: ## @@ -0,0 +1,113 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agr

Re: [PR] [infra] nightly pypi build for `pyiceberg_core` [iceberg-rust]

2025-02-17 Thread via GitHub
Xuanwo commented on code in PR #948: URL: https://github.com/apache/iceberg-rust/pull/948#discussion_r1959178229 ## .github/workflows/release_python_nightly.yml: ## @@ -0,0 +1,113 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agr

Re: [PR] Parquet: Implement Variant readers [iceberg]

2025-02-17 Thread via GitHub
aihuaxu commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1958863116 ## core/src/test/java/org/apache/iceberg/variants/VariantTestUtil.java: ## @@ -27,10 +29,55 @@ import java.util.stream.Collectors; import java.util.stream.Stream;

Re: [PR] feat: Introduce C FFI for iceberg rust [iceberg-rust]

2025-02-17 Thread via GitHub
Xuanwo commented on code in PR #966: URL: https://github.com/apache/iceberg-rust/pull/966#discussion_r1959177072 ## bindings/c/README.md: ## @@ -0,0 +1,22 @@ + + +# Iceberg C FFI + +This project is used to build an FFI for iceberg-rust. Review Comment: Added. ## b

Re: [PR] feat: Introduce C FFI for iceberg rust [iceberg-rust]

2025-02-17 Thread via GitHub
Xuanwo commented on code in PR #966: URL: https://github.com/apache/iceberg-rust/pull/966#discussion_r1959174275 ## bindings/c/Cargo.toml: ## @@ -0,0 +1,37 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE

Re: [PR] feat: Introduce C FFI for iceberg rust [iceberg-rust]

2025-02-17 Thread via GitHub
Xuanwo commented on code in PR #966: URL: https://github.com/apache/iceberg-rust/pull/966#discussion_r1959173058 ## bindings/c/.gitignore: ## @@ -0,0 +1,18 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE

Re: [PR] feat: Introduce C FFI for iceberg rust [iceberg-rust]

2025-02-17 Thread via GitHub
Xuanwo commented on code in PR #966: URL: https://github.com/apache/iceberg-rust/pull/966#discussion_r1959174508 ## bindings/c/README.md: ## @@ -0,0 +1,22 @@ + + +# Iceberg C FFI + +This project is used to build an FFI for iceberg-rust. Review Comment: Makes sense, will do.

Re: [PR] feat: Introduce C FFI for iceberg rust [iceberg-rust]

2025-02-17 Thread via GitHub
manuzhang commented on code in PR #966: URL: https://github.com/apache/iceberg-rust/pull/966#discussion_r1959155416 ## bindings/c/README.md: ## @@ -0,0 +1,22 @@ + + +# Iceberg C FFI + +This project is used to build an FFI for iceberg-rust. Review Comment: Add a build guide?

Re: [PR] feat: Introduce C FFI for iceberg rust [iceberg-rust]

2025-02-17 Thread via GitHub
manuzhang commented on code in PR #966: URL: https://github.com/apache/iceberg-rust/pull/966#discussion_r1959155090 ## bindings/c/Cargo.toml: ## @@ -0,0 +1,37 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOT

Re: [PR] feat: Introduce C FFI for iceberg rust [iceberg-rust]

2025-02-17 Thread via GitHub
manuzhang commented on code in PR #966: URL: https://github.com/apache/iceberg-rust/pull/966#discussion_r1959156862 ## bindings/c/cbindgen.toml: ## @@ -0,0 +1,53 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the

Re: [PR] feat: Introduce C FFI for iceberg rust [iceberg-rust]

2025-02-17 Thread via GitHub
manuzhang commented on code in PR #966: URL: https://github.com/apache/iceberg-rust/pull/966#discussion_r1959152456 ## bindings/c/.gitignore: ## @@ -0,0 +1,18 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOT

Re: [PR] feat: Introduce C FFI for iceberg rust [iceberg-rust]

2025-02-17 Thread via GitHub
manuzhang commented on PR #966: URL: https://github.com/apache/iceberg-rust/pull/966#issuecomment-2664744562 I'd suggest notifying the dev list on this feature and open early discussion with iceberg-cpp folks for collaboration. -- This is an automated message from the Apache Git Service.

Re: [PR] Docs: add apache amoro(incubating) with iceberg (#11965) [iceberg]

2025-02-17 Thread via GitHub
czy006 commented on PR #11966: URL: https://github.com/apache/iceberg/pull/11966#issuecomment-2664728995 > @czy006 IMHO, the doc looks too verbose, and it might be better linking to the detailed description on the Amoro website while only keeping a brief introduction here for each feature.

Re: [PR] feat: Introduce C FFI for iceberg rust [iceberg-rust]

2025-02-17 Thread via GitHub
Xuanwo commented on PR #966: URL: https://github.com/apache/iceberg-rust/pull/966#issuecomment-2664596679 > Just curious how would this interface work with [iceberg-cpp](https://github.com/apache/iceberg-cpp?rgh-link-date=2025-02-18T04%3A37%3A15Z)? Will there be collaboration or duplication

Re: [PR] chore: datafusion 45 upgrade [iceberg-rust]

2025-02-17 Thread via GitHub
kevinjqliu commented on code in PR #943: URL: https://github.com/apache/iceberg-rust/pull/943#discussion_r1959038726 ## Cargo.toml: ## @@ -96,7 +96,7 @@ serde_with = "3.4" tempfile = "3.16" tokio = { version = "1.36", default-features = false } typed-builder = "0.20" -url = "

Re: [PR] feat: Introduce C FFI for iceberg rust [iceberg-rust]

2025-02-17 Thread via GitHub
manuzhang commented on PR #966: URL: https://github.com/apache/iceberg-rust/pull/966#issuecomment-2664590010 @Xuanwo Just curious how would this interface work with [iceberg-cpp](https://github.com/apache/iceberg-cpp)? Will there be collaboration or duplication? -- This is an automated m

Re: [PR] Docs: Refactor site navigation bar [iceberg]

2025-02-17 Thread via GitHub
manuzhang commented on PR #12289: URL: https://github.com/apache/iceberg/pull/12289#issuecomment-2664583070 Screenshots of the website after the change ### Homepage ![CleanShot 2025-02-18 at 12 27 41@2x](https://github.com/user-attachments/assets/1ebc6bf9-949a-4e4b-97ca-42fba2ef93

Re: [PR] chore: datafusion 45 upgrade [iceberg-rust]

2025-02-17 Thread via GitHub
kevinjqliu commented on code in PR #943: URL: https://github.com/apache/iceberg-rust/pull/943#discussion_r1959038726 ## Cargo.toml: ## @@ -96,7 +96,7 @@ serde_with = "3.4" tempfile = "3.16" tokio = { version = "1.36", default-features = false } typed-builder = "0.20" -url = "

Re: [PR] Docs: add apache amoro(incubating) with iceberg (#11965) [iceberg]

2025-02-17 Thread via GitHub
manuzhang commented on PR #11966: URL: https://github.com/apache/iceberg/pull/11966#issuecomment-2664570228 @czy006 IMHO, the doc looks too verbose, and it might be better linking to the detailed description on the Amoro website while only keeping a brief introduction here for each feature.

Re: [I] Enhance iceberg-go to Support Nessie API for All Catalog Operations [iceberg-go]

2025-02-17 Thread via GitHub
shubham-tomar commented on issue #291: URL: https://github.com/apache/iceberg-go/issues/291#issuecomment-2664568826 Thank you @zeroshade , i have shared my docker-compose file above in the comments, let me know if you would need any other info for reproducing this Meanwhile i will also lo

Re: [PR] Docs: add apache amoro(incubating) with iceberg (#11965) [iceberg]

2025-02-17 Thread via GitHub
manuzhang commented on code in PR #11966: URL: https://github.com/apache/iceberg/pull/11966#discussion_r1959031682 ## docs/docs/amoro.md: ## @@ -0,0 +1,89 @@ +--- +title: "Apache Amoro" +--- + + +# Apache Amoro With Iceberg + +**[Apache Amoro(incubating)](https://amoro.apache.or

Re: [PR] Docs: add apache amoro(incubating) with iceberg (#11965) [iceberg]

2025-02-17 Thread via GitHub
manuzhang commented on code in PR #11966: URL: https://github.com/apache/iceberg/pull/11966#discussion_r1959029226 ## docs/docs/amoro.md: ## @@ -0,0 +1,89 @@ +--- +title: "Apache Amoro" +--- + + +# Apache Amoro With Iceberg + +**[Apache Amoro(incubating)](https://amoro.apache.or

Re: [PR] Docs: add apache amoro(incubating) with iceberg (#11965) [iceberg]

2025-02-17 Thread via GitHub
manuzhang commented on code in PR #11966: URL: https://github.com/apache/iceberg/pull/11966#discussion_r1959028139 ## docs/docs/amoro.md: ## @@ -0,0 +1,89 @@ +--- +title: "Apache Amoro" +--- + + +# Apache Amoro With Iceberg + +**[Apache Amoro(incubating)](https://amoro.apache.or

Re: [PR] Docs: add apache amoro(incubating) with iceberg (#11965) [iceberg]

2025-02-17 Thread via GitHub
manuzhang commented on code in PR #11966: URL: https://github.com/apache/iceberg/pull/11966#discussion_r1959027363 ## docs/docs/amoro.md: ## @@ -0,0 +1,89 @@ +--- +title: "Apache Amoro" +--- + + +# Apache Amoro With Iceberg + +**[Apache Amoro(incubating)](https://amoro.apache.or

Re: [PR] Docs: add apache amoro(incubating) with iceberg (#11965) [iceberg]

2025-02-17 Thread via GitHub
manuzhang commented on code in PR #11966: URL: https://github.com/apache/iceberg/pull/11966#discussion_r1959027222 ## docs/docs/amoro.md: ## @@ -0,0 +1,89 @@ +--- +title: "Apache Amoro" +--- + + +# Apache Amoro With Iceberg + +**[Apache Amoro(incubating)](https://amoro.apache.or

Re: [PR] [infra] nightly pypi build for `pyiceberg_core` [iceberg-rust]

2025-02-17 Thread via GitHub
kevinjqliu commented on code in PR #948: URL: https://github.com/apache/iceberg-rust/pull/948#discussion_r1958953694 ## .github/workflows/release_python_nightly.yml: ## @@ -0,0 +1,113 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license

Re: [PR] Data: Handle case where partition location is missing for `TableMigrationUtil` [iceberg]

2025-02-17 Thread via GitHub
jshmchenxi commented on PR #12212: URL: https://github.com/apache/iceberg/pull/12212#issuecomment-2664324934 Kindly ping @manuzhang @RussellSpitzer @stevenzwu -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [I] Explore potential issue with `scan` returning the incorrect results [iceberg-python]

2025-02-17 Thread via GitHub
dongsupkim-onepredict commented on issue #1506: URL: https://github.com/apache/iceberg-python/issues/1506#issuecomment-2664281635 > What is the row_filter youre using? - I'm using literal row_filter somthing like `row_filter = " int_column = 13"` >Are the manifest files the same in te

Re: [PR] Docs: Refactor site navigation bar [iceberg]

2025-02-17 Thread via GitHub
stevenzwu commented on code in PR #12289: URL: https://github.com/apache/iceberg/pull/12289#discussion_r1958882598 ## site/docs/terms.md: ## @@ -20,39 +20,69 @@ title: "Terms" # Terms -### Snapshot +## Catalog Review Comment: agree that catalog is a term, which is good

Re: [PR] Fix IndexOutOfBounds exception in FileFormat#fromFileName [iceberg]

2025-02-17 Thread via GitHub
ebyhr commented on code in PR #12301: URL: https://github.com/apache/iceberg/pull/12301#discussion_r1958874825 ## api/src/test/java/org/apache/iceberg/FileFormatTest.java: ## @@ -0,0 +1,86 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contribu

Re: [PR] Purge RCK test entries in `afterEach` instead of `beforeEach` [iceberg]

2025-02-17 Thread via GitHub
github-actions[bot] commented on PR #11699: URL: https://github.com/apache/iceberg/pull/11699#issuecomment-2664265109 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] Parquet write option 'write.parquet.row-group-limit' issue [iceberg-python]

2025-02-17 Thread via GitHub
github-actions[bot] commented on issue #1012: URL: https://github.com/apache/iceberg-python/issues/1012#issuecomment-2664267387 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the A

Re: [I] It's not possible to readStream from an Iceberg table as source when its snapshots expire [iceberg]

2025-02-17 Thread via GitHub
github-actions[bot] commented on issue #9504: URL: https://github.com/apache/iceberg/issues/9504#issuecomment-2664264993 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Parquet write option 'write.parquet.row-group-limit' issue [iceberg-python]

2025-02-17 Thread via GitHub
github-actions[bot] closed issue #1012: Parquet write option 'write.parquet.row-group-limit' issue URL: https://github.com/apache/iceberg-python/issues/1012 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] Apache Flink not committing new snapshots to Iceberg Table [iceberg]

2025-02-17 Thread via GitHub
github-actions[bot] closed issue #9089: Apache Flink not committing new snapshots to Iceberg Table URL: https://github.com/apache/iceberg/issues/9089 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Fix comment on `WRITE_OBJECT_STORE_PARTITIONED_PATHS` table property [iceberg]

2025-02-17 Thread via GitHub
github-actions[bot] commented on PR #11798: URL: https://github.com/apache/iceberg/pull/11798#issuecomment-2664265135 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] Apache Flink not committing new snapshots to Iceberg Table [iceberg]

2025-02-17 Thread via GitHub
github-actions[bot] commented on issue #9089: URL: https://github.com/apache/iceberg/issues/9089#issuecomment-2664264977 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] UpdateSchema.add_column doesn't support adding parent and child in the same transaction [iceberg]

2025-02-17 Thread via GitHub
singhpk234 commented on issue #12223: URL: https://github.com/apache/iceberg/issues/12223#issuecomment-2664228233 > At this point, do you think this bug report is not valid or the use-case of the issue should still work? Once can make a case that this is a multi-statement transaction

Re: [PR] Docs: Refactor site navigation bar [iceberg]

2025-02-17 Thread via GitHub
manuzhang commented on code in PR #12289: URL: https://github.com/apache/iceberg/pull/12289#discussion_r1958850913 ## site/docs/terms.md: ## @@ -20,39 +20,69 @@ title: "Terms" # Terms -### Snapshot +## Catalog Review Comment: I agree catalogs deserve more details but ha

Re: [PR] Fix #305: ManifestEntry partition field schema should be dynamically … [iceberg-go]

2025-02-17 Thread via GitHub
zeroshade commented on code in PR #307: URL: https://github.com/apache/iceberg-go/pull/307#discussion_r1958843788 ## utils.go: ## @@ -238,3 +239,68 @@ func avroEncode[T any](key string, version int, vals []T, out io.Writer) error { return enc.Close() } + +func struct

[PR] Build: Bump mkdocstrings-python from 1.15.0 to 1.16.0 [iceberg-python]

2025-02-17 Thread via GitHub
dependabot[bot] opened a new pull request, #1675: URL: https://github.com/apache/iceberg-python/pull/1675 Bumps [mkdocstrings-python](https://github.com/mkdocstrings/python) from 1.15.0 to 1.16.0. Release notes Sourced from https://github.com/mkdocstrings/python/releases";>mkdocstr

[PR] Build: Bump mkdocstrings from 0.28.0 to 0.28.1 [iceberg-python]

2025-02-17 Thread via GitHub
dependabot[bot] opened a new pull request, #1676: URL: https://github.com/apache/iceberg-python/pull/1676 Bumps [mkdocstrings](https://github.com/mkdocstrings/mkdocstrings) from 0.28.0 to 0.28.1. Release notes Sourced from https://github.com/mkdocstrings/mkdocstrings/releases";>mkd

Re: [PR] List data and metadata directories instead of table root [iceberg]

2025-02-17 Thread via GitHub
dramaticlly commented on PR #12278: URL: https://github.com/apache/iceberg/pull/12278#issuecomment-2664203151 > How about introducing the UUID in the table path while creating table? This way two tables cannot share a same location. I think that's what trino did with `iceberg.unique-t

Re: [I] Enhance iceberg-go to Support Nessie API for All Catalog Operations [iceberg-go]

2025-02-17 Thread via GitHub
zeroshade commented on issue #291: URL: https://github.com/apache/iceberg-go/issues/291#issuecomment-2664201267 I'll see if i can find some time this week to spin up a nessie catalog myself and see if I can figure out what's going wrong -- This is an automated message from the Apache Git

Re: [PR] List data and metadata directories instead of table root [iceberg]

2025-02-17 Thread via GitHub
dramaticlly commented on code in PR #12278: URL: https://github.com/apache/iceberg/pull/12278#discussion_r1958833411 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/DeleteOrphanFilesSparkAction.java: ## @@ -84,11 +86,11 @@ * comparing the actual files in tha

Re: [PR] List data and metadata directories instead of table root [iceberg]

2025-02-17 Thread via GitHub
dramaticlly commented on code in PR #12278: URL: https://github.com/apache/iceberg/pull/12278#discussion_r1958833411 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/DeleteOrphanFilesSparkAction.java: ## @@ -84,11 +86,11 @@ * comparing the actual files in tha

Re: [PR] List data and metadata directories instead of table root [iceberg]

2025-02-17 Thread via GitHub
ajantha-bhat commented on code in PR #12278: URL: https://github.com/apache/iceberg/pull/12278#discussion_r1958839422 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/DeleteOrphanFilesSparkAction.java: ## @@ -301,24 +303,34 @@ private Dataset actualFileIdentDS(

Re: [PR] List data and metadata directories instead of table root [iceberg]

2025-02-17 Thread via GitHub
karuppayya commented on code in PR #12278: URL: https://github.com/apache/iceberg/pull/12278#discussion_r1958822862 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/DeleteOrphanFilesSparkAction.java: ## @@ -335,6 +347,21 @@ private Dataset listedFileDS() {

[PR] Add instruction to install shell plugin to doc [iceberg-python]

2025-02-17 Thread via GitHub
gabeiglio opened a new pull request, #1674: URL: https://github.com/apache/iceberg-python/pull/1674 More context [here](https://github.com/apache/iceberg-python/pull/1673) @kevinjqliu -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] Fix IndexOutOfBounds exception in FileFormat#fromFileName [iceberg]

2025-02-17 Thread via GitHub
rshkv commented on code in PR #12301: URL: https://github.com/apache/iceberg/pull/12301#discussion_r1958818328 ## api/src/main/java/org/apache/iceberg/FileFormat.java: ## @@ -60,9 +60,10 @@ public String addExtension(String filename) { public static FileFormat fromFileName(Ch

Re: [PR] Fix IndexOutOfBounds exception in FileFormat#fromFileName [iceberg]

2025-02-17 Thread via GitHub
rshkv commented on code in PR #12301: URL: https://github.com/apache/iceberg/pull/12301#discussion_r1958812097 ## api/src/test/java/org/apache/iceberg/FileFormatTest.java: ## @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contribu

Re: [I] Java doc link is not working [iceberg]

2025-02-17 Thread via GitHub
ranup commented on issue #12166: URL: https://github.com/apache/iceberg/issues/12166#issuecomment-2664143215 Thanks Manu. Verified it is working now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [I] [feature] Table Scan should take into account the table's sort order [iceberg-python]

2025-02-17 Thread via GitHub
kevinjqliu commented on issue #1637: URL: https://github.com/apache/iceberg-python/issues/1637#issuecomment-2664143272 sure @gabeiglio feel free to tag me for review :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [I] Partition spec mismatch when 'compatibility.snapshot-id-inheritance.enabled' is true [iceberg]

2025-02-17 Thread via GitHub
RussellSpitzer commented on issue #12273: URL: https://github.com/apache/iceberg/issues/12273#issuecomment-2664127254 So I tracked this down it is a combination of some fun issues. TL;DR - We write manifests using a `partition spec` which we create whole cloth from the Spark Tabl

Re: [PR] AWS, AZURE: Move docker-based tests to integration test source [iceberg]

2025-02-17 Thread via GitHub
anuragmantri commented on PR #12274: URL: https://github.com/apache/iceberg/pull/12274#issuecomment-2664067160 I turns out, we don't run AWS CI workflow at all. See https://github.com/apache/iceberg/pull/4855 and https://github.com/apache/iceberg/pull/5826 so adding `check.dependsOn integr

[PR] API: Move variant to API and add extract expression [iceberg]

2025-02-17 Thread via GitHub
rdblue opened a new pull request, #12304: URL: https://github.com/apache/iceberg/pull/12304 This adds `Expressions.extract` to extract a value from a variant in Iceberg filters. The new method, `Expressions.extract(column, path, type)`, accepts a column name, a JSON path, and a type.

Re: [PR] Core: add variant type support [iceberg]

2025-02-17 Thread via GitHub
aihuaxu commented on code in PR #11831: URL: https://github.com/apache/iceberg/pull/11831#discussion_r1958783694 ## api/src/main/java/org/apache/iceberg/types/PrimitiveHolder.java: ## @@ -33,6 +33,6 @@ class PrimitiveHolder implements Serializable { } Object readResolve(

Re: [PR] Core: add variant type support [iceberg]

2025-02-17 Thread via GitHub
aihuaxu commented on code in PR #11831: URL: https://github.com/apache/iceberg/pull/11831#discussion_r1958783694 ## api/src/main/java/org/apache/iceberg/types/PrimitiveHolder.java: ## @@ -33,6 +33,6 @@ class PrimitiveHolder implements Serializable { } Object readResolve(

Re: [PR] Install poetry-shell-plugin [iceberg-python]

2025-02-17 Thread via GitHub
gabeiglio closed pull request #1673: Install poetry-shell-plugin URL: https://github.com/apache/iceberg-python/pull/1673 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

Re: [PR] Install poetry-shell-plugin [iceberg-python]

2025-02-17 Thread via GitHub
gabeiglio commented on PR #1673: URL: https://github.com/apache/iceberg-python/pull/1673#issuecomment-2664094562 Sounds good! Ill close this pr and open another one to update the docs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

Re: [PR] Core: Fix failure when reading files table with branch [iceberg]

2025-02-17 Thread via GitHub
dramaticlly commented on code in PR #11719: URL: https://github.com/apache/iceberg/pull/11719#discussion_r1958778016 ## core/src/main/java/org/apache/iceberg/FilesTable.java: ## @@ -51,6 +52,20 @@ public static class FilesTableScan extends BaseFilesTableScan { super(tabl

Re: [PR] Core: Fix failure when reading files table with branch [iceberg]

2025-02-17 Thread via GitHub
dramaticlly commented on code in PR #11719: URL: https://github.com/apache/iceberg/pull/11719#discussion_r1958778016 ## core/src/main/java/org/apache/iceberg/FilesTable.java: ## @@ -51,6 +52,20 @@ public static class FilesTableScan extends BaseFilesTableScan { super(tabl

Re: [PR] Install poetry-shell-plugin [iceberg-python]

2025-02-17 Thread via GitHub
kevinjqliu commented on PR #1673: URL: https://github.com/apache/iceberg-python/pull/1673#issuecomment-2664081621 oh interesting. yea i think its good to update the docs to optionally install `poetry shell`. I've always just used `poetry run ...` maybe we can add that to the docs too --

Re: [PR] Install poetry-shell-plugin [iceberg-python]

2025-02-17 Thread via GitHub
gabeiglio commented on PR #1673: URL: https://github.com/apache/iceberg-python/pull/1673#issuecomment-2664064539 I usually follow this instruction in the docs to setup/activate the venv: > If you want to install the library on the host, you can simply run pip3 install -e .. If you wish to

Re: [PR] Core,REST: extend httpClient builder to support tls factory [iceberg]

2025-02-17 Thread via GitHub
nastra commented on PR #11979: URL: https://github.com/apache/iceberg/pull/11979#issuecomment-2663464772 @danielcweeks @amogh-jahagirdar can you guys review this as well please? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] support all_entries in pyiceberg [iceberg-python]

2025-02-17 Thread via GitHub
amitgilad3 commented on code in PR #1608: URL: https://github.com/apache/iceberg-python/pull/1608#discussion_r1958750651 ## pyiceberg/table/inspect.py: ## @@ -157,74 +158,96 @@ def _readable_metrics_struct(bound_type: PrimitiveType) -> pa.StructType: pa.field("

Re: [PR] Core: Fix failure when reading files table with branch [iceberg]

2025-02-17 Thread via GitHub
dramaticlly commented on code in PR #11719: URL: https://github.com/apache/iceberg/pull/11719#discussion_r1958742193 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestMetadataTables.java: ## @@ -671,6 +671,76 @@ public void testFilesTableTimeTr

[PR] [INFRA] Install poetry-shell-plugin [iceberg-python]

2025-02-17 Thread via GitHub
gabeiglio opened a new pull request, #1673: URL: https://github.com/apache/iceberg-python/pull/1673 With the release of poetry version 2.0.1 the shell command will only be available through installing the poetry-plugin-shell: [see](https://python-poetry.org/docs/cli#shell) This PR ma

Re: [PR] Replace usages of Aws4Signer with AwsV4HttpSigner in REST SigV4 [iceberg]

2025-02-17 Thread via GitHub
mayankvadariya commented on PR #12295: URL: https://github.com/apache/iceberg/pull/12295#issuecomment-2663998192 @jackye1995 could you advise if this change has been attempted appropriately. -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] Materialized View Spec [iceberg]

2025-02-17 Thread via GitHub
stevenzwu commented on code in PR #11041: URL: https://github.com/apache/iceberg/pull/11041#discussion_r1958726223 ## format/view-spec.md: ## @@ -160,6 +179,56 @@ Each entry in `version-log` is a struct with the following fields: | _required_ | `timestamp-ms` | Timestamp when

Re: [PR] Materialized View Spec [iceberg]

2025-02-17 Thread via GitHub
stevenzwu commented on code in PR #11041: URL: https://github.com/apache/iceberg/pull/11041#discussion_r1958712529 ## format/view-spec.md: ## @@ -160,6 +179,56 @@ Each entry in `version-log` is a struct with the following fields: | _required_ | `timestamp-ms` | Timestamp when

Re: [PR] Materialized View Spec [iceberg]

2025-02-17 Thread via GitHub
stevenzwu commented on code in PR #11041: URL: https://github.com/apache/iceberg/pull/11041#discussion_r1958712529 ## format/view-spec.md: ## @@ -160,6 +179,56 @@ Each entry in `version-log` is a struct with the following fields: | _required_ | `timestamp-ms` | Timestamp when

Re: [PR] Materialized View Spec [iceberg]

2025-02-17 Thread via GitHub
stevenzwu commented on code in PR #11041: URL: https://github.com/apache/iceberg/pull/11041#discussion_r1958709902 ## format/view-spec.md: ## @@ -160,6 +179,56 @@ Each entry in `version-log` is a struct with the following fields: | _required_ | `timestamp-ms` | Timestamp when

Re: [PR] Materialized View Spec [iceberg]

2025-02-17 Thread via GitHub
stevenzwu commented on code in PR #11041: URL: https://github.com/apache/iceberg/pull/11041#discussion_r1958703593 ## format/view-spec.md: ## @@ -160,6 +179,56 @@ Each entry in `version-log` is a struct with the following fields: | _required_ | `timestamp-ms` | Timestamp when

Re: [PR] Materialized View Spec [iceberg]

2025-02-17 Thread via GitHub
stevenzwu commented on code in PR #11041: URL: https://github.com/apache/iceberg/pull/11041#discussion_r1958702676 ## format/view-spec.md: ## @@ -160,6 +179,56 @@ Each entry in `version-log` is a struct with the following fields: | _required_ | `timestamp-ms` | Timestamp when

Re: [PR] Fix IndexOutOfBounds exception in FileFormat#fromFileName [iceberg]

2025-02-17 Thread via GitHub
RussellSpitzer commented on code in PR #12301: URL: https://github.com/apache/iceberg/pull/12301#discussion_r1958699822 ## api/src/main/java/org/apache/iceberg/FileFormat.java: ## @@ -60,9 +60,10 @@ public String addExtension(String filename) { public static FileFormat fromFi

Re: [PR] Fix IndexOutOfBounds exception in FileFormat#fromFileName [iceberg]

2025-02-17 Thread via GitHub
RussellSpitzer commented on code in PR #12301: URL: https://github.com/apache/iceberg/pull/12301#discussion_r1958697008 ## api/src/test/java/org/apache/iceberg/FileFormatTest.java: ## @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] Fix IndexOutOfBounds exception in FileFormat#fromFileName [iceberg]

2025-02-17 Thread via GitHub
RussellSpitzer commented on code in PR #12301: URL: https://github.com/apache/iceberg/pull/12301#discussion_r1958693758 ## api/src/test/java/org/apache/iceberg/FileFormatTest.java: ## @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [I] IndexOutOfBounds in FileFormat#fromFileName [iceberg]

2025-02-17 Thread via GitHub
RussellSpitzer commented on issue #12300: URL: https://github.com/apache/iceberg/issues/12300#issuecomment-2663908052 Good catch, so I guess we need to just not do the check if the start is negative and not check in that case? -- This is an automated message from the Apache Git Service. T

Re: [I] previous eq deletes handling on new write [iceberg]

2025-02-17 Thread via GitHub
ismailsimsek commented on issue #12280: URL: https://github.com/apache/iceberg/issues/12280#issuecomment-2663901082 `eq deletes are partition scoped,` AFAIK this is correct. Would it be more performant partitioning the table based on Primary key (`id`) ? In that case equality delete compact

Re: [PR] Implement update for `remove-snapshots` action [iceberg-python]

2025-02-17 Thread via GitHub
kevinjqliu merged PR #1561: URL: https://github.com/apache/iceberg-python/pull/1561 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

Re: [PR] Spark: Rewrite V2 deletes to V3 DVs [iceberg]

2025-02-17 Thread via GitHub
danielcweeks commented on code in PR #12250: URL: https://github.com/apache/iceberg/pull/12250#discussion_r1958620415 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RewritePositionDeleteFilesSparkAction.java: ## @@ -140,6 +148,33 @@ public RewritePositionDele

[PR] Fix IndexOutOfBounds exception in FileFormat#fromFileName [iceberg]

2025-02-17 Thread via GitHub
rshkv opened a new pull request, #12301: URL: https://github.com/apache/iceberg/pull/12301 Closes #12300. `FileFormat#fromFileName` should account for file names that are shorter than a file format extension. -- This is an automated message from the Apache Git Service. To respond t

Re: [PR] Core: add variant type support [iceberg]

2025-02-17 Thread via GitHub
rdblue commented on code in PR #11831: URL: https://github.com/apache/iceberg/pull/11831#discussion_r1958675667 ## core/src/main/java/org/apache/iceberg/mapping/MappingUtil.java: ## @@ -302,6 +302,11 @@ public MappedFields map(Types.MapType map, MappedFields keyResult, MappedFi

Re: [I] Support for Shallow Clone / Zero Copy Cloning in Apache Iceberg [iceberg]

2025-02-17 Thread via GitHub
RussellSpitzer commented on issue #12263: URL: https://github.com/apache/iceberg/issues/12263#issuecomment-2663873430 I don't follow these points > * Branching creates an isolated version of a table's metadata that can diverge over time, while shallow cloning creates a new table refer

Re: [PR] Core: add variant type support [iceberg]

2025-02-17 Thread via GitHub
rdblue commented on code in PR #11831: URL: https://github.com/apache/iceberg/pull/11831#discussion_r1955196818 ## api/src/main/java/org/apache/iceberg/types/TypeUtil.java: ## @@ -712,6 +712,10 @@ public T map(Types.MapType map, Supplier keyResult, Supplier valueResult)

Re: [PR] Core: add variant type support [iceberg]

2025-02-17 Thread via GitHub
rdblue commented on code in PR #11831: URL: https://github.com/apache/iceberg/pull/11831#discussion_r1958673483 ## api/src/main/java/org/apache/iceberg/types/PrimitiveHolder.java: ## @@ -33,6 +33,6 @@ class PrimitiveHolder implements Serializable { } Object readResolve()

Re: [PR] fix: SqlCatalog namespace_exists should work with multi-level namespaces [iceberg-python]

2025-02-17 Thread via GitHub
kevinjqliu merged PR #1671: URL: https://github.com/apache/iceberg-python/pull/1671 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

[PR] AWS: Integrate S3 analytics accelerator library [iceberg]

2025-02-17 Thread via GitHub
SanjayMarreddi opened a new pull request, #12299: URL: https://github.com/apache/iceberg/pull/12299 Description - This is an initial PR for integrating [analytics-accelerator-s3](https://github.com/awslabs/analytics-accelerator-s3) into `iceberg/aws` - In short, our library a

Re: [PR] Core: add variant type support [iceberg]

2025-02-17 Thread via GitHub
rdblue commented on code in PR #11831: URL: https://github.com/apache/iceberg/pull/11831#discussion_r1958670627 ## api/src/test/java/org/apache/iceberg/types/TestReadabilityChecks.java: ## @@ -112,6 +114,34 @@ private void testDisallowPrimitiveToStruct(PrimitiveType from, Schem

  1   2   >