Re: [PR] feat: implement IcebergTableProviderFactory for datafusion [iceberg-rust]

2024-09-07 Thread via GitHub
yukkit commented on code in PR #600: URL: https://github.com/apache/iceberg-rust/pull/600#discussion_r1747989388 ## crates/integrations/datafusion/src/table/table_provider_factory.rs: ## @@ -0,0 +1,312 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more

Re: [I] Schema: Allow field name `foo.bar` even if struct foo->bar is present [iceberg-rust]

2024-09-07 Thread via GitHub
yukkit commented on issue #591: URL: https://github.com/apache/iceberg-rust/issues/591#issuecomment-2335128308 FWIW. Can we introduce a structure similar to `TableIdent` to represent field paths, named 'FieldPath'? 1. We could modify the elements of selected `TableScan::column_names` fro

Re: [PR] refactor: Store DataFile in FileScanTask instead [iceberg-rust]

2024-09-07 Thread via GitHub
Xuanwo commented on PR #607: URL: https://github.com/apache/iceberg-rust/pull/607#issuecomment-2335160095 Great, let's me follow the option 1 first. We can discuss about option 2 while needed. -- This is an automated message from the Apache Git Service. To respond to the message, please l

[PR] feat: Add more fields in FileScanTask [iceberg-rust]

2024-09-07 Thread via GitHub
Xuanwo opened a new pull request, #609: URL: https://github.com/apache/iceberg-rust/pull/609 Follow up of https://github.com/apache/iceberg-rust/pull/607 Add format, content, recound_count in `FileScanTask` so users can handle those tasks without extra metadata to fetch. -- This is

Re: [PR] feat: Add more fields in FileScanTask [iceberg-rust]

2024-09-07 Thread via GitHub
Xuanwo commented on PR #609: URL: https://github.com/apache/iceberg-rust/pull/609#issuecomment-2335162784 Also inviting @SDD and @ZENOTME for review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[I] Spark Streaming Job with multiple queries MERGE INTO the same target table [iceberg]

2024-09-07 Thread via GitHub
cccs-jc opened a new issue, #11094: URL: https://github.com/apache/iceberg/issues/11094 ### Query engine - Spark 3.5 - Iceberg 1.5.2 - Azure storage account ABFSS ### Question I’m trying to use the `MERGE INTO` statement within a Spark streaming application that ru

Re: [PR] Bump mkdocstrings from 0.26.0 to 0.26.1 [iceberg-python]

2024-09-07 Thread via GitHub
sungwy merged PR #1143: URL: https://github.com/apache/iceberg-python/pull/1143 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@icebe

[PR] feat: SQL Catalog - Tables [iceberg-rust]

2024-09-07 Thread via GitHub
callum-ryan opened a new pull request, #610: URL: https://github.com/apache/iceberg-rust/pull/610 this is a WIP. Pulling tests in from @fqaiser94's original recommended test set as I go. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [I] Iceberg to Redshift load [iceberg]

2024-09-07 Thread via GitHub
github-actions[bot] commented on issue #6841: URL: https://github.com/apache/iceberg/issues/6841#issuecomment-2336490282 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Iceberg to Redshift load [iceberg]

2024-09-07 Thread via GitHub
github-actions[bot] closed issue #6841: Iceberg to Redshift load URL: https://github.com/apache/iceberg/issues/6841 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

Re: [I] Reset data file's storage location after change partition [iceberg]

2024-09-07 Thread via GitHub
github-actions[bot] commented on issue #8110: URL: https://github.com/apache/iceberg/issues/8110#issuecomment-2336490411 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Core, API: Add non-nullable constraint checks to support making optional columns… [iceberg]

2024-09-07 Thread via GitHub
github-actions[bot] commented on PR #8128: URL: https://github.com/apache/iceberg/pull/8128#issuecomment-2336490508 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] Spark 3.4: Remove deprecated AssertHelpers [iceberg]

2024-09-07 Thread via GitHub
github-actions[bot] commented on PR #8129: URL: https://github.com/apache/iceberg/pull/8129#issuecomment-2336490518 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [I] iceberg materialized views [iceberg]

2024-09-07 Thread via GitHub
github-actions[bot] commented on issue #8143: URL: https://github.com/apache/iceberg/issues/8143#issuecomment-2336490528 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Docs: document the compareWithFileList parameter [iceberg]

2024-09-07 Thread via GitHub
github-actions[bot] commented on issue #8155: URL: https://github.com/apache/iceberg/issues/8155#issuecomment-2336490536 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] feat: Add more fields in FileScanTask [iceberg-rust]

2024-09-07 Thread via GitHub
liurenjie1024 merged PR #609: URL: https://github.com/apache/iceberg-rust/pull/609 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] feat (datafusion integration): convert datafusion expr filters to Iceberg Predicate [iceberg-rust]

2024-09-07 Thread via GitHub
liurenjie1024 commented on code in PR #588: URL: https://github.com/apache/iceberg-rust/pull/588#discussion_r1749056511 ## crates/integrations/datafusion/src/physical_plan/expr_to_predicate.rs: ## @@ -0,0 +1,312 @@ +// Licensed to the Apache Software Foundation (ASF) under one +

Re: [I] Add apply interface in transaction [iceberg-rust]

2024-09-07 Thread via GitHub
liurenjie1024 commented on issue #596: URL: https://github.com/apache/iceberg-rust/issues/596#issuecomment-2336534759 We have [check](https://github.com/apache/iceberg-rust/blob/a1ec0fa98113fc02b2479d81759fccd9dab10378/crates/iceberg/src/transaction.rs#L50) to avoid such duplicated case. Fo

[PR] Build: Bump jetty from 11.0.23 to 11.0.24 [iceberg]

2024-09-07 Thread via GitHub
dependabot[bot] opened a new pull request, #11096: URL: https://github.com/apache/iceberg/pull/11096 Bumps `jetty` from 11.0.23 to 11.0.24. Updates `org.eclipse.jetty:jetty-server` from 11.0.23 to 11.0.24 Updates `org.eclipse.jetty:jetty-servlet` from 11.0.23 to 11.0.24 D

[PR] Build: Bump software.amazon.awssdk:bom from 2.27.12 to 2.27.21 [iceberg]

2024-09-07 Thread via GitHub
dependabot[bot] opened a new pull request, #11098: URL: https://github.com/apache/iceberg/pull/11098 Bumps software.amazon.awssdk:bom from 2.27.12 to 2.27.21. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=soft

Re: [PR] Build: Bump software.amazon.awssdk:bom from 2.27.12 to 2.27.17 [iceberg]

2024-09-07 Thread via GitHub
dependabot[bot] commented on PR #11056: URL: https://github.com/apache/iceberg/pull/11056#issuecomment-2336543031 Superseded by #11098. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi