Re: [PR] Parquet: Use native getRowIndexOffset support instead of calculating it [iceberg]

2024-11-10 Thread via GitHub
huaxingao commented on PR #10107: URL: https://github.com/apache/iceberg/pull/10107#issuecomment-2467426867 @wypoon Would you like to reopen this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [I] Javadoc issues [iceberg]

2024-11-10 Thread via GitHub
manuzhang commented on issue #10378: URL: https://github.com/apache/iceberg/issues/10378#issuecomment-2467401515 @jbonofre Have you got a chance to fix javadoc (issue No.1)? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] feat: Expose length of Iterators [iceberg-rust]

2024-11-10 Thread via GitHub
liurenjie1024 merged PR #692: URL: https://github.com/apache/iceberg-rust/pull/692 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] feat: Expose length of Iterators [iceberg-rust]

2024-11-10 Thread via GitHub
liurenjie1024 commented on PR #692: URL: https://github.com/apache/iceberg-rust/pull/692#issuecomment-2467364199 > Alternatively we could expose the Hashmaps directly? Those would provide a few more accessors than any proxy type I'm not a big fan of exposing concrete underlying data s

Re: [PR] chore(deps): Bump crate-ci/typos from 1.27.0 to 1.27.3 [iceberg-rust]

2024-11-10 Thread via GitHub
liurenjie1024 merged PR #693: URL: https://github.com/apache/iceberg-rust/pull/693 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Support WASB scheme in ADLSFileIO [iceberg]

2024-11-10 Thread via GitHub
jbonofre commented on code in PR #11504: URL: https://github.com/apache/iceberg/pull/11504#discussion_r1836010627 ## azure/src/main/java/org/apache/iceberg/azure/AzureProperties.java: ## @@ -93,7 +93,7 @@ public void applyClientConfiguration(String account, DataLakeFileSystemCl

Re: [PR] Build: Bump parquet from 1.13.1 to 1.14.3 [iceberg]

2024-11-10 Thread via GitHub
jbonofre commented on PR #11507: URL: https://github.com/apache/iceberg/pull/11507#issuecomment-2467327259 This PR should not be merged: better to use the one from @Fokko -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

Re: [PR] Ignore schema merge updates from long -> int [iceberg]

2024-11-10 Thread via GitHub
huaxingao commented on code in PR #11419: URL: https://github.com/apache/iceberg/pull/11419#discussion_r1836006381 ## core/src/main/java/org/apache/iceberg/schema/UnionByNameVisitor.java: ## @@ -163,8 +164,7 @@ private void updateColumn(Types.NestedField field, Types.NestedFiel

Re: [I] Adjust the "table_exists" behavior in the REST Catalog [iceberg-python]

2024-11-10 Thread via GitHub
djouallah commented on issue #1018: URL: https://github.com/apache/iceberg-python/issues/1018#issuecomment-2467190119 still getting errors with with pyiceberg 0.8 rc1 ``` HTTPError Traceback (most recent call last) [/usr/local/lib/python3.10/dist

Re: [PR] Spec: Add context query parameter for all REST APIs [iceberg]

2024-11-10 Thread via GitHub
github-actions[bot] commented on PR #10359: URL: https://github.com/apache/iceberg/pull/10359#issuecomment-2467006389 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] doc: Add statement for contributors to avoid force push. [iceberg]

2024-11-10 Thread via GitHub
github-actions[bot] commented on PR #10939: URL: https://github.com/apache/iceberg/pull/10939#issuecomment-2467006691 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Test: Add rowDelete test in TestChangeLogReader [iceberg]

2024-11-10 Thread via GitHub
github-actions[bot] commented on PR #10924: URL: https://github.com/apache/iceberg/pull/10924#issuecomment-2467006657 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Core/RewriteFiles: Duplicate Data Bug - Fixed dropping delete files that are still required [iceberg]

2024-11-10 Thread via GitHub
github-actions[bot] commented on PR #10962: URL: https://github.com/apache/iceberg/pull/10962#issuecomment-2467006719 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] [DRAFT] Build: remove hadoop 2 support [iceberg]

2024-11-10 Thread via GitHub
github-actions[bot] commented on PR #10932: URL: https://github.com/apache/iceberg/pull/10932#issuecomment-2467006675 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] Pushdown of aggregate functions in where [iceberg]

2024-11-10 Thread via GitHub
github-actions[bot] commented on issue #10109: URL: https://github.com/apache/iceberg/issues/10109#issuecomment-2467006297 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [PR] View: add property to describe advisory read mode [iceberg]

2024-11-10 Thread via GitHub
github-actions[bot] closed pull request #10362: View: add property to describe advisory read mode URL: https://github.com/apache/iceberg/pull/10362 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] OpenAPI: Add query param to control namespace separator [iceberg]

2024-11-10 Thread via GitHub
github-actions[bot] commented on PR #10904: URL: https://github.com/apache/iceberg/pull/10904#issuecomment-2467006601 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Handled case where struct has fewer elements than the sink table [iceberg]

2024-11-10 Thread via GitHub
github-actions[bot] commented on PR #10951: URL: https://github.com/apache/iceberg/pull/10951#issuecomment-2467006705 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Core: RESTTableOperations commit add deleteRemovedMetadataFiles methods [iceberg]

2024-11-10 Thread via GitHub
github-actions[bot] commented on PR #10895: URL: https://github.com/apache/iceberg/pull/10895#issuecomment-2467006588 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Spec: Add context query parameter for all REST APIs [iceberg]

2024-11-10 Thread via GitHub
github-actions[bot] closed pull request #10359: Spec: Add context query parameter for all REST APIs URL: https://github.com/apache/iceberg/pull/10359 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] DRAFT: DO NOT MERGE Create a reader for missing column in parquet file [iceberg]

2024-11-10 Thread via GitHub
github-actions[bot] commented on PR #10922: URL: https://github.com/apache/iceberg/pull/10922#issuecomment-2467006626 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Core: Pass namespace separator via query param [iceberg]

2024-11-10 Thread via GitHub
github-actions[bot] commented on PR #10905: URL: https://github.com/apache/iceberg/pull/10905#issuecomment-2467006613 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Api: Support setting docs for nested fields in ListType and MapType [iceberg]

2024-11-10 Thread via GitHub
github-actions[bot] commented on PR #10887: URL: https://github.com/apache/iceberg/pull/10887#issuecomment-2467006573 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] View: add property to describe advisory read mode [iceberg]

2024-11-10 Thread via GitHub
github-actions[bot] commented on PR #10362: URL: https://github.com/apache/iceberg/pull/10362#issuecomment-2467006408 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] Core: Prevent dropping column which is referenced by active partition… [iceberg]

2024-11-10 Thread via GitHub
github-actions[bot] closed pull request #10352: Core: Prevent dropping column which is referenced by active partition… URL: https://github.com/apache/iceberg/pull/10352 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [I] Pushdown of aggregate functions in where [iceberg]

2024-11-10 Thread via GitHub
github-actions[bot] closed issue #10109: Pushdown of aggregate functions in where URL: https://github.com/apache/iceberg/issues/10109 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] Core: Prevent dropping column which is referenced by active partition… [iceberg]

2024-11-10 Thread via GitHub
github-actions[bot] commented on PR #10352: URL: https://github.com/apache/iceberg/pull/10352#issuecomment-2467006368 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] Add table statistics [iceberg-python]

2024-11-10 Thread via GitHub
ndrluis commented on PR #1285: URL: https://github.com/apache/iceberg-python/pull/1285#issuecomment-2466990871 @kevinjqliu could you please review it once more? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [I] Add `@typing.override` to functions [iceberg-python]

2024-11-10 Thread via GitHub
cosmastech commented on issue #1310: URL: https://github.com/apache/iceberg-python/issues/1310#issuecomment-2466949191 Would you recommend something like this: ```py from typing import TYPE_CHECKING if TYPE_CHECKING and sys.version_info >= (3, 12): from typing import

Re: [PR] [Views] Update view spec with table identifier requirements [iceberg]

2024-11-10 Thread via GitHub
stevenzwu commented on code in PR #11365: URL: https://github.com/apache/iceberg/pull/11365#discussion_r1835757854 ## format/view-spec.md: ## @@ -97,7 +97,10 @@ Summary is a string to string map of metadata about a view version. Common metad View definitions can be represent

Re: [PR] Ignore schema merge updates from long -> int [iceberg]

2024-11-10 Thread via GitHub
rocco408 commented on PR #11419: URL: https://github.com/apache/iceberg/pull/11419#issuecomment-2466841180 I may be missing something, but after several variations I don't see a way to cleanly use `TypeUtil.isPromotionAllowed` from `UnionByNameVisitor`. Even after adding the long-to-int and

Re: [PR] Feature: Write to branches [iceberg-python]

2024-11-10 Thread via GitHub
kevinjqliu commented on code in PR #941: URL: https://github.com/apache/iceberg-python/pull/941#discussion_r1835751569 ## tests/integration/test_writes/test_writes.py: ## @@ -1448,3 +1450,100 @@ def test_rewrite_manifest_after_partition_evolution(session_catalog: Catalog) ->

Re: [I] The source artifacts provided with GitHub releases are not buildable. [iceberg]

2024-11-10 Thread via GitHub
AnatolyPopov commented on issue #11505: URL: https://github.com/apache/iceberg/issues/11505#issuecomment-2466749038 Thanks for a quick reply @Fokko! First of all, thanks, building from the tarball you linked above works as expected. However I have several points that I think still

[PR] chore(deps): Bump crate-ci/typos from 1.27.0 to 1.27.3 [iceberg-rust]

2024-11-10 Thread via GitHub
dependabot[bot] opened a new pull request, #693: URL: https://github.com/apache/iceberg-rust/pull/693 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.27.0 to 1.27.3. Release notes Sourced from https://github.com/crate-ci/typos/releases";>crate-ci/typos's releases.

Re: [I] [bug] read from multiple s3 regions [iceberg-python]

2024-11-10 Thread via GitHub
danhphan commented on issue #1279: URL: https://github.com/apache/iceberg-python/issues/1279#issuecomment-2466676408 Thanks @kevinjqliu , I'm reading the code base. Can you please give me an example of expected unit-tests for the feature if possible? For instance, if we create the fo

[PR] build(deps): bump golang.org/x/sync from 0.8.0 to 0.9.0 [iceberg-go]

2024-11-10 Thread via GitHub
dependabot[bot] opened a new pull request, #197: URL: https://github.com/apache/iceberg-go/pull/197 Bumps [golang.org/x/sync](https://github.com/golang/sync) from 0.8.0 to 0.9.0. Commits https://github.com/golang/sync/commit/151027eb5637d626749783009eab9f0a7cfb446a";>151027e R

[PR] build(deps): bump github.com/aws/aws-sdk-go-v2 from 1.32.3 to 1.32.4 [iceberg-go]

2024-11-10 Thread via GitHub
dependabot[bot] opened a new pull request, #196: URL: https://github.com/apache/iceberg-go/pull/196 Bumps [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) from 1.32.3 to 1.32.4. Commits https://github.com/aws/aws-sdk-go-v2/commit/70eb57ac775f782db6856c73f1ca

[PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.66.2 to 1.66.3 [iceberg-go]

2024-11-10 Thread via GitHub
dependabot[bot] opened a new pull request, #193: URL: https://github.com/apache/iceberg-go/pull/193 Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.66.2 to 1.66.3. Commits https://github.com/aws/aws-sdk-go-v2/commit/70eb57ac775f782d

[PR] build(deps): bump github.com/substrait-io/substrait-go from 1.1.0 to 1.2.0 [iceberg-go]

2024-11-10 Thread via GitHub
dependabot[bot] opened a new pull request, #195: URL: https://github.com/apache/iceberg-go/pull/195 Bumps [github.com/substrait-io/substrait-go](https://github.com/substrait-io/substrait-go) from 1.1.0 to 1.2.0. Release notes Sourced from https://github.com/substrait-io/substrait-

[PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/glue from 1.101.0 to 1.101.2 [iceberg-go]

2024-11-10 Thread via GitHub
dependabot[bot] opened a new pull request, #194: URL: https://github.com/apache/iceberg-go/pull/194 Bumps [github.com/aws/aws-sdk-go-v2/service/glue](https://github.com/aws/aws-sdk-go-v2) from 1.101.0 to 1.101.2. Commits https://github.com/aws/aws-sdk-go-v2/commit/2f70834c690f