Re: [PR] feat: Glue Catalog - table operations (3/3) [iceberg-rust]

2024-04-21 Thread via GitHub
liurenjie1024 merged PR #314: URL: https://github.com/apache/iceberg-rust/pull/314 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [I] Implement Glue Catalog [iceberg-rust]

2024-04-21 Thread via GitHub
liurenjie1024 closed issue #249: Implement Glue Catalog URL: https://github.com/apache/iceberg-rust/issues/249 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

Re: [PR] feat: Glue Catalog - table operations (3/3) [iceberg-rust]

2024-04-21 Thread via GitHub
liurenjie1024 commented on PR #314: URL: https://github.com/apache/iceberg-rust/pull/314#issuecomment-2068035928 Thanks @marvinlanhenke for this pr! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] chore: update roadmap [iceberg-rust]

2024-04-21 Thread via GitHub
liurenjie1024 merged PR #336: URL: https://github.com/apache/iceberg-rust/pull/336 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] chore: update roadmap [iceberg-rust]

2024-04-21 Thread via GitHub
liurenjie1024 commented on PR #336: URL: https://github.com/apache/iceberg-rust/pull/336#issuecomment-2068036193 Thanks @marvinlanhenke for this pr, and @sdd for review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Add `ManifestEvaluator`, used to filter manifests in table scans [iceberg-rust]

2024-04-21 Thread via GitHub
liurenjie1024 commented on code in PR #322: URL: https://github.com/apache/iceberg-rust/pull/322#discussion_r1573750563 ## crates/iceberg/src/expr/visitors/manifest_evaluator.rs: ## @@ -0,0 +1,459 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more cont

Re: [PR] Add `ManifestEvaluator`, used to filter manifests in table scans [iceberg-rust]

2024-04-21 Thread via GitHub
liurenjie1024 commented on PR #322: URL: https://github.com/apache/iceberg-rust/pull/322#issuecomment-2068041808 Let's wait to see if others have comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Add `ManifestEvaluator`, used to filter manifests in table scans [iceberg-rust]

2024-04-21 Thread via GitHub
liurenjie1024 commented on PR #322: URL: https://github.com/apache/iceberg-rust/pull/322#issuecomment-2068041946 cc @Xuanwo @Fokko @marvinlanhenke PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [I] Update Roadmap / Close old Issues [iceberg-rust]

2024-04-21 Thread via GitHub
liurenjie1024 commented on issue #330: URL: https://github.com/apache/iceberg-rust/issues/330#issuecomment-2068046354 cc @marvinlanhenke Could we close this now? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Add Files metadata table [iceberg-python]

2024-04-21 Thread via GitHub
Gowthami03B commented on code in PR #614: URL: https://github.com/apache/iceberg-python/pull/614#discussion_r1573765866 ## pyiceberg/table/__init__.py: ## @@ -3537,6 +3537,58 @@ def update_partitions_map( schema=table_schema, ) +def files(self) -> "pa

Re: [PR] REST catalog: add UnsupportedOperationResponse to namespace interfaces [iceberg]

2024-04-21 Thread via GitHub
FANNG1 closed pull request #8506: REST catalog: add UnsupportedOperationResponse to namespace interfaces URL: https://github.com/apache/iceberg/pull/8506 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Add Files metadata table [iceberg-python]

2024-04-21 Thread via GitHub
Gowthami03B commented on code in PR #614: URL: https://github.com/apache/iceberg-python/pull/614#discussion_r1573767911 ## pyiceberg/table/__init__.py: ## @@ -3537,6 +3537,58 @@ def update_partitions_map( schema=table_schema, ) +def files(self) -> "pa

Re: [PR] Build: Bump software.amazon.awssdk:bom from 2.25.31 to 2.25.35 [iceberg]

2024-04-21 Thread via GitHub
Fokko merged PR #10192: URL: https://github.com/apache/iceberg/pull/10192 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Build: Bump io.netty:netty-buffer from 4.1.108.Final to 4.1.109.Final [iceberg]

2024-04-21 Thread via GitHub
Fokko merged PR #10191: URL: https://github.com/apache/iceberg/pull/10191 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Build: Bump org.roaringbitmap:RoaringBitmap from 1.0.5 to 1.0.6 [iceberg]

2024-04-21 Thread via GitHub
Fokko merged PR #10190: URL: https://github.com/apache/iceberg/pull/10190 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Build: Bump mkdocs-material from 9.5.17 to 9.5.18 [iceberg]

2024-04-21 Thread via GitHub
Fokko merged PR #10189: URL: https://github.com/apache/iceberg/pull/10189 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Build: Bump gradle.plugin.io.morethan.jmhreport:gradle-jmh-report from 0.9.0 to 0.9.6 [iceberg]

2024-04-21 Thread via GitHub
Fokko merged PR #10193: URL: https://github.com/apache/iceberg/pull/10193 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Iceberg/Comet integration POC [iceberg]

2024-04-21 Thread via GitHub
huaxingao commented on code in PR #9841: URL: https://github.com/apache/iceberg/pull/9841#discussion_r1573864929 ## spark/v3.4/build.gradle: ## @@ -70,8 +70,11 @@ project(":iceberg-spark:iceberg-spark-${sparkMajorVersion}_${scalaVersion}") { exclude group: 'io.netty', mo

Re: [PR] Iceberg/Comet integration POC [iceberg]

2024-04-21 Thread via GitHub
huaxingao commented on code in PR #9841: URL: https://github.com/apache/iceberg/pull/9841#discussion_r1573865006 ## spark/v3.4/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/BaseColumnBatchLoader.java: ## @@ -0,0 +1,199 @@ +/* + * Licensed to the Apache Software Fo

Re: [I] Integration tests performance degradation [iceberg-python]

2024-04-21 Thread via GitHub
kevinjqliu commented on issue #604: URL: https://github.com/apache/iceberg-python/issues/604#issuecomment-2068146191 Making all the tests run in parallel requires lots and lots of changes. For the short term, the best way forward is to look at individual tests. `pytest`'s `--duration

Re: [I] Update Roadmap / Close old Issues [iceberg-rust]

2024-04-21 Thread via GitHub
marvinlanhenke commented on issue #330: URL: https://github.com/apache/iceberg-rust/issues/330#issuecomment-2068157972 @liurenjie1024 regarding the roadmap i think we can close this, however perhaps we want to do some grooming and close some old issues as well - or maybe duplicates. I th

Re: [PR] Add `ManifestEvaluator`, used to filter manifests in table scans [iceberg-rust]

2024-04-21 Thread via GitHub
marvinlanhenke commented on PR #322: URL: https://github.com/apache/iceberg-rust/pull/322#issuecomment-2068159189 @sdd Thanks a lot. LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-04-21 Thread via GitHub
rahil-c commented on PR #9695: URL: https://github.com/apache/iceberg/pull/9695#issuecomment-2068261225 @rdblue @danielcweeks @jackye1995 Was wondering if you guys can take a look at this pr when you get a chance? -- This is an automated message from the Apache Git Service. To respond to

Re: [I] master branch - flink sql create hive catalog error [iceberg]

2024-04-21 Thread via GitHub
github-actions[bot] closed issue #2468: master branch - flink sql create hive catalog error URL: https://github.com/apache/iceberg/issues/2468 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [I] master branch - flink sql create hive catalog error [iceberg]

2024-04-21 Thread via GitHub
github-actions[bot] commented on issue #2468: URL: https://github.com/apache/iceberg/issues/2468#issuecomment-2068278380 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] How do I fetch the latest ROADMAP [iceberg]

2024-04-21 Thread via GitHub
github-actions[bot] closed issue #2469: How do I fetch the latest ROADMAP URL: https://github.com/apache/iceberg/issues/2469 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [I] How do I fetch the latest ROADMAP [iceberg]

2024-04-21 Thread via GitHub
github-actions[bot] commented on issue #2469: URL: https://github.com/apache/iceberg/issues/2469#issuecomment-2068278392 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Flink: Can not see hive udf in Flink when use iceberg hive catalog [iceberg]

2024-04-21 Thread via GitHub
github-actions[bot] commented on issue #2653: URL: https://github.com/apache/iceberg/issues/2653#issuecomment-2068278541 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

[I] does iceberg can run on k8s? [iceberg]

2024-04-21 Thread via GitHub
JunseoChoJJ opened a new issue, #10196: URL: https://github.com/apache/iceberg/issues/10196 ### Query engine I am using spark ### Question hi I'm a newbee to iceberg. and it might be stupid question but please tell me does iceberg can run on k8s? My think was s

Re: [PR] [WIP] Integration with Datafusion [iceberg-rust]

2024-04-21 Thread via GitHub
ZENOTME commented on PR #324: URL: https://github.com/apache/iceberg-rust/pull/324#issuecomment-2068383273 Thanks! Sorry for replying late. I think this is a good start for the integration work. And I have completed https://github.com/apache/iceberg-rust/pull/277. Maybe we can convert this

Re: [PR] feat: init iceberg writer [iceberg-rust]

2024-04-21 Thread via GitHub
ZENOTME commented on PR #275: URL: https://github.com/apache/iceberg-rust/pull/275#issuecomment-2068385611 I think this PR is ready to move. PTAL. I modified the interface a bit to make it easier for error handling. Feel free to let me there is still @Xuanwo @liurenjie1024 @Fokko @sdd -

Re: [PR] Spark 3.3: Set useCommitCoordinator to false in batch writes [iceberg]

2024-04-21 Thread via GitHub
manuzhang closed pull request #10197: Spark 3.3: Set useCommitCoordinator to false in batch writes URL: https://github.com/apache/iceberg/pull/10197 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] Add `ManifestEvaluator`, used to filter manifests in table scans [iceberg-rust]

2024-04-21 Thread via GitHub
ZENOTME commented on code in PR #322: URL: https://github.com/apache/iceberg-rust/pull/322#discussion_r1574157028 ## crates/iceberg/src/expr/visitors/manifest_evaluator.rs: ## @@ -0,0 +1,459 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributo

Re: [PR] Add `ManifestEvaluator`, used to filter manifests in table scans [iceberg-rust]

2024-04-21 Thread via GitHub
sdd commented on code in PR #322: URL: https://github.com/apache/iceberg-rust/pull/322#discussion_r1574182261 ## crates/iceberg/src/expr/visitors/manifest_evaluator.rs: ## @@ -0,0 +1,459 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor li

Re: [PR] Add `ManifestEvaluator`, used to filter manifests in table scans [iceberg-rust]

2024-04-21 Thread via GitHub
sdd commented on code in PR #322: URL: https://github.com/apache/iceberg-rust/pull/322#discussion_r1574191566 ## crates/iceberg/src/expr/visitors/manifest_evaluator.rs: ## @@ -0,0 +1,459 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor li

Re: [PR] Add `ManifestEvaluator`, used to filter manifests in table scans [iceberg-rust]

2024-04-21 Thread via GitHub
ZENOTME commented on code in PR #322: URL: https://github.com/apache/iceberg-rust/pull/322#discussion_r1574193520 ## crates/iceberg/src/expr/visitors/manifest_evaluator.rs: ## @@ -0,0 +1,459 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributo

Re: [PR] Add `ManifestEvaluator`, used to filter manifests in table scans [iceberg-rust]

2024-04-21 Thread via GitHub
sdd commented on code in PR #322: URL: https://github.com/apache/iceberg-rust/pull/322#discussion_r1574198729 ## crates/iceberg/src/expr/visitors/manifest_evaluator.rs: ## @@ -0,0 +1,459 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor li

Re: [PR] Core: Lazily compute & cache hashCode in CharSequenceWrapper [iceberg]

2024-04-21 Thread via GitHub
nastra commented on code in PR #10023: URL: https://github.com/apache/iceberg/pull/10023#discussion_r1574204162 ## api/src/main/java/org/apache/iceberg/util/CharSequenceWrapper.java: ## @@ -29,13 +29,16 @@ public static CharSequenceWrapper wrap(CharSequence seq) { } priv