Re: [PR] DRAFT: DO NOT MERGE - create a NullVector instance as the dummy holder for null values [iceberg]

2024-11-17 Thread via GitHub
github-actions[bot] closed pull request #10923: DRAFT: DO NOT MERGE - create a NullVector instance as the dummy holder for null values URL: https://github.com/apache/iceberg/pull/10923 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [I] check-ordering enablement for flink config [iceberg]

2024-11-17 Thread via GitHub
github-actions[bot] commented on issue #10360: URL: https://github.com/apache/iceberg/issues/10360#issuecomment-2481696487 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Remove Hive 2 [iceberg]

2024-11-17 Thread via GitHub
pvary commented on PR #10996: URL: https://github.com/apache/iceberg/pull/10996#issuecomment-2482012167 > @pvary @nastra @Fokko Given Hive 3.1 is already broken on JDK 11+ before this PR, how about skipping these failed tests not to block removing Hive 2? I think upgrading to Hive 4 is a bi

Re: [PR] Parquet: Use native getRowIndexOffset support instead of calculating it [iceberg]

2024-11-17 Thread via GitHub
wypoon commented on PR #11520: URL: https://github.com/apache/iceberg/pull/11520#issuecomment-2481943013 @Fokko can you help merge this if you have no further feedback? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [PR] Support changelog scan for table with delete files [iceberg]

2024-11-17 Thread via GitHub
wypoon commented on PR #10935: URL: https://github.com/apache/iceberg/pull/10935#issuecomment-2481942281 @aokolnychyi I agree that we should stick to existing changelog tasks and always resolve historical deletes to produce the changelog. Have you thought of any optimizations for processing

[I] Handling Updates on Partition Columns in Iceberg with Flink CDC [iceberg]

2024-11-17 Thread via GitHub
a8356555 opened a new issue, #11573: URL: https://github.com/apache/iceberg/issues/11573 ### Apache Iceberg version 1.5.2 ### Query engine Athena ### Please describe the bug 🐞 Hi, I'm using MySQL Flink CDC with Iceberg 1.5.2 and Flink 1.16. I have a t

Re: [I] Consider Using object_store as IO Abstraction [iceberg-rust]

2024-11-17 Thread via GitHub
liurenjie1024 commented on issue #172: URL: https://github.com/apache/iceberg-rust/issues/172#issuecomment-2481838237 > Sounds like a good compromise, did you have any thoughts on how this might integrate with the existing Datafusion machinery? I'm mainly thinking for configuration, so user

Re: [PR] Add @override [iceberg-python]

2024-11-17 Thread via GitHub
cosmastech commented on PR #1312: URL: https://github.com/apache/iceberg-python/pull/1312#issuecomment-2481198369 > thanks for the PR! Looks like i was wrong and we need to do something like this in order to make the other python versions happy > > [#1310 (comment)](https://github.co

Re: [PR] [DRAFT] Build: remove hadoop 2 support [iceberg]

2024-11-17 Thread via GitHub
github-actions[bot] closed pull request #10932: [DRAFT] Build: remove hadoop 2 support URL: https://github.com/apache/iceberg/pull/10932 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Remove Hive 2 [iceberg]

2024-11-17 Thread via GitHub
manuzhang commented on PR #10996: URL: https://github.com/apache/iceberg/pull/10996#issuecomment-2481719211 @pvary @nastra @Fokko Given Hive 3.1 is already broken on JDK 11+ before this PR, how about skipping these failed tests not to block removing Hive 2? I think upgrading to Hive 4 is

Re: [PR] get snapshots info to be expired [iceberg]

2024-11-17 Thread via GitHub
github-actions[bot] commented on PR #11343: URL: https://github.com/apache/iceberg/pull/11343#issuecomment-2481696799 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Core: Pass namespace separator via query param [iceberg]

2024-11-17 Thread via GitHub
github-actions[bot] commented on PR #10905: URL: https://github.com/apache/iceberg/pull/10905#issuecomment-2481696577 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] OpenAPI: Add query param to control namespace separator [iceberg]

2024-11-17 Thread via GitHub
github-actions[bot] commented on PR #10904: URL: https://github.com/apache/iceberg/pull/10904#issuecomment-2481696561 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [I] Spark: Schema evolution is not reflected on branches [iceberg]

2024-11-17 Thread via GitHub
github-actions[bot] closed issue #10274: Spark: Schema evolution is not reflected on branches URL: https://github.com/apache/iceberg/issues/10274 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] doc: Add statement for contributors to avoid force push. [iceberg]

2024-11-17 Thread via GitHub
github-actions[bot] closed pull request #10939: doc: Add statement for contributors to avoid force push. URL: https://github.com/apache/iceberg/pull/10939 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] Test: Add rowDelete test in TestChangeLogReader [iceberg]

2024-11-17 Thread via GitHub
github-actions[bot] closed pull request #10924: Test: Add rowDelete test in TestChangeLogReader URL: https://github.com/apache/iceberg/pull/10924 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] DRAFT: DO NOT MERGE Create a reader for missing column in parquet file [iceberg]

2024-11-17 Thread via GitHub
github-actions[bot] closed pull request #10922: DRAFT: DO NOT MERGE Create a reader for missing column in parquet file URL: https://github.com/apache/iceberg/pull/10922 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] OpenAPI: Add query param to control namespace separator [iceberg]

2024-11-17 Thread via GitHub
github-actions[bot] closed pull request #10904: OpenAPI: Add query param to control namespace separator URL: https://github.com/apache/iceberg/pull/10904 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] DRAFT: DO NOT MERGE Create a reader for missing column in parquet file [iceberg]

2024-11-17 Thread via GitHub
github-actions[bot] commented on PR #10922: URL: https://github.com/apache/iceberg/pull/10922#issuecomment-2481696598 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] Core: RESTTableOperations commit add deleteRemovedMetadataFiles methods [iceberg]

2024-11-17 Thread via GitHub
github-actions[bot] closed pull request #10895: Core: RESTTableOperations commit add deleteRemovedMetadataFiles methods URL: https://github.com/apache/iceberg/pull/10895 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] Handled case where struct has fewer elements than the sink table [iceberg]

2024-11-17 Thread via GitHub
github-actions[bot] closed pull request #10951: Handled case where struct has fewer elements than the sink table URL: https://github.com/apache/iceberg/pull/10951 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [I] Spark: Schema evolution is not reflected on branches [iceberg]

2024-11-17 Thread via GitHub
github-actions[bot] commented on issue #10274: URL: https://github.com/apache/iceberg/issues/10274#issuecomment-2481696443 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [PR] Core: RESTTableOperations commit add deleteRemovedMetadataFiles methods [iceberg]

2024-11-17 Thread via GitHub
github-actions[bot] commented on PR #10895: URL: https://github.com/apache/iceberg/pull/10895#issuecomment-2481696541 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] Handled case where struct has fewer elements than the sink table [iceberg]

2024-11-17 Thread via GitHub
github-actions[bot] commented on PR #10951: URL: https://github.com/apache/iceberg/pull/10951#issuecomment-2481696688 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] doc: Add statement for contributors to avoid force push. [iceberg]

2024-11-17 Thread via GitHub
github-actions[bot] commented on PR #10939: URL: https://github.com/apache/iceberg/pull/10939#issuecomment-2481696670 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] [DRAFT] Build: remove hadoop 2 support [iceberg]

2024-11-17 Thread via GitHub
github-actions[bot] commented on PR #10932: URL: https://github.com/apache/iceberg/pull/10932#issuecomment-2481696655 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] Test: Add rowDelete test in TestChangeLogReader [iceberg]

2024-11-17 Thread via GitHub
github-actions[bot] commented on PR #10924: URL: https://github.com/apache/iceberg/pull/10924#issuecomment-2481696637 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] Core: Pass namespace separator via query param [iceberg]

2024-11-17 Thread via GitHub
github-actions[bot] closed pull request #10905: Core: Pass namespace separator via query param URL: https://github.com/apache/iceberg/pull/10905 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Api: Support setting docs for nested fields in ListType and MapType [iceberg]

2024-11-17 Thread via GitHub
github-actions[bot] commented on PR #10887: URL: https://github.com/apache/iceberg/pull/10887#issuecomment-2481696524 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] Api: Support setting docs for nested fields in ListType and MapType [iceberg]

2024-11-17 Thread via GitHub
github-actions[bot] closed pull request #10887: Api: Support setting docs for nested fields in ListType and MapType URL: https://github.com/apache/iceberg/pull/10887 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] Remove Dependency on Hadoop's Filesystem Class from Remove Orphan Files [iceberg]

2024-11-17 Thread via GitHub
RussellSpitzer commented on issue #11541: URL: https://github.com/apache/iceberg/issues/11541#issuecomment-2481634096 > I have a few questions to help get started. > > 1. Is [TestRemoveOrphanFilesProcedure](https://github.com/apache/iceberg/blob/acd7cc1126b192ccb53ad8198bda37e983aa4c6

Re: [PR] Spark 3.4: IcebergSource extends SessionConfigSupport [iceberg]

2024-11-17 Thread via GitHub
szehon-ho commented on code in PR #7732: URL: https://github.com/apache/iceberg/pull/7732#discussion_r1845559383 ## spark/v3.4/spark/src/test/java/org/apache/iceberg/spark/source/TestIcebergSourceTablesBase.java: ## @@ -1953,6 +1953,65 @@ public void testTableWithInt96Timestamp(

Re: [I] Remove Dependency on Hadoop's Filesystem Class from Remove Orphan Files [iceberg]

2024-11-17 Thread via GitHub
rocco408 commented on issue #11541: URL: https://github.com/apache/iceberg/issues/11541#issuecomment-2481603847 I have a few questions to help get started. 1. Is [TestRemoveOrphanFilesProcedure](https://github.com/apache/iceberg/blob/acd7cc1126b192ccb53ad8198bda37e983aa4c6c/spark/

[PR] Spark 3.4: IcebergSource extends SessionConfigSupport [iceberg]

2024-11-17 Thread via GitHub
pan3793 opened a new pull request, #7732: URL: https://github.com/apache/iceberg/pull/7732 This PR aims to make `IcebergSource extends SessionConfigSupport` to improve the Spark DataSource v2 API coverage. ``` /** * A mix-in interface for {@link TableProvider}. Data sources can imp

Re: [PR] Data, Flink, Spark: Test deletes with format-version=3 [iceberg]

2024-11-17 Thread via GitHub
nastra merged PR #11538: URL: https://github.com/apache/iceberg/pull/11538 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [I] Consider Using object_store as IO Abstraction [iceberg-rust]

2024-11-17 Thread via GitHub
tustvold commented on issue #172: URL: https://github.com/apache/iceberg-rust/issues/172#issuecomment-2481271106 Sounds like a good compromise, did you have any thoughts on how this might integrate with the existing Datafusion machinery? I'm mainly thinking for configuration, so users get a

Re: [I] Consider Using object_store as IO Abstraction [iceberg-rust]

2024-11-17 Thread via GitHub
liurenjie1024 commented on issue #172: URL: https://github.com/apache/iceberg-rust/issues/172#issuecomment-2481267603 Thanks for everyone joining the discussion here. I think we have reached some conclusions here: 1. We need to support different storages, like s3, google cloud storage.

Re: [I] Support to optimize, analyze tables and expire snapshots, remove orphan files [iceberg-python]

2024-11-17 Thread via GitHub
ndrluis commented on issue #31: URL: https://github.com/apache/iceberg-python/issues/31#issuecomment-2481246404 Hello @eedduuar @Samreay, the recommendation is to use Spark, Trino, or another engine that provides support. There is ongoing work on expiring snapshots, but there is no ETA yet.

Re: [PR] Build: Bump parquet from 1.13.1 to 1.14.4 [iceberg]

2024-11-17 Thread via GitHub
dependabot[bot] commented on PR #11570: URL: https://github.com/apache/iceberg/pull/11570#issuecomment-2480989749 OK, I won't notify you again about this release, but will get in touch when a new version is available. You can also ignore all major, minor, or patch releases for a dependency

Re: [PR] Core: Add DataFiles builder API to enable users to specify their own custom conversion logic for string partition values [iceberg]

2024-11-17 Thread via GitHub
github-actions[bot] commented on PR #10724: URL: https://github.com/apache/iceberg/pull/10724#issuecomment-2480862570 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] Add row-level operation benchmarks [iceberg]

2024-11-17 Thread via GitHub
github-actions[bot] closed pull request #10687: Add row-level operation benchmarks URL: https://github.com/apache/iceberg/pull/10687 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[I] A more robust way to deprecate our APIs [iceberg-python]

2024-11-17 Thread via GitHub
ndrluis opened a new issue, #1330: URL: https://github.com/apache/iceberg-python/issues/1330 ### Feature Request / Improvement I was studying Python libraries and how they handle deprecation. Specifically, I was exploring constant deprecation in the context of issue #1217. I found th

[PR] build(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.28.1 to 1.28.4 [iceberg-go]

2024-11-17 Thread via GitHub
dependabot[bot] opened a new pull request, #210: URL: https://github.com/apache/iceberg-go/pull/210 Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.28.1 to 1.28.4. Commits https://github.com/aws/aws-sdk-go-v2/commit/f0fcf5955d8b5db77815

[PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/glue from 1.101.2 to 1.101.3 [iceberg-go]

2024-11-17 Thread via GitHub
dependabot[bot] opened a new pull request, #209: URL: https://github.com/apache/iceberg-go/pull/209 Bumps [github.com/aws/aws-sdk-go-v2/service/glue](https://github.com/aws/aws-sdk-go-v2) from 1.101.2 to 1.101.3. Commits https://github.com/aws/aws-sdk-go-v2/commit/27326538a1c0

[PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.66.3 to 1.67.0 [iceberg-go]

2024-11-17 Thread via GitHub
dependabot[bot] opened a new pull request, #208: URL: https://github.com/apache/iceberg-go/pull/208 Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.66.3 to 1.67.0. Commits https://github.com/aws/aws-sdk-go-v2/commit/f0fcf5955d8b5db7

[PR] build(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.17.42 to 1.17.45 [iceberg-go]

2024-11-17 Thread via GitHub
dependabot[bot] opened a new pull request, #207: URL: https://github.com/apache/iceberg-go/pull/207 Bumps [github.com/aws/aws-sdk-go-v2/credentials](https://github.com/aws/aws-sdk-go-v2) from 1.17.42 to 1.17.45. Commits https://github.com/aws/aws-sdk-go-v2/commit/f0fcf5955d8b5

[PR] build(deps): bump github.com/aws/smithy-go from 1.22.0 to 1.22.1 [iceberg-go]

2024-11-17 Thread via GitHub
dependabot[bot] opened a new pull request, #206: URL: https://github.com/apache/iceberg-go/pull/206 Bumps [github.com/aws/smithy-go](https://github.com/aws/smithy-go) from 1.22.0 to 1.22.1. Changelog Sourced from https://github.com/aws/smithy-go/blob/main/CHANGELOG.md";>github.com/

Re: [PR] Build: Bump parquet from 1.13.1 to 1.14.4 [iceberg]

2024-11-17 Thread via GitHub
Fokko commented on PR #11570: URL: https://github.com/apache/iceberg/pull/11570#issuecomment-2480989741 Needs some changes in tests, as in https://github.com/apache/iceberg/pull/11502 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] Build: Bump parquet from 1.13.1 to 1.14.4 [iceberg]

2024-11-17 Thread via GitHub
Fokko closed pull request #11570: Build: Bump parquet from 1.13.1 to 1.14.4 URL: https://github.com/apache/iceberg/pull/11570 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Build: Bump io.netty:netty-buffer from 4.1.114.Final to 4.1.115.Final [iceberg]

2024-11-17 Thread via GitHub
Fokko merged PR #11569: URL: https://github.com/apache/iceberg/pull/11569 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa