Re: [PR] Implement BoundPredicateVisitor trait for ManifestFilterVisitor [iceberg-rust]

2024-05-21 Thread via GitHub
sdd commented on code in PR #367: URL: https://github.com/apache/iceberg-rust/pull/367#discussion_r1609384502 ## crates/iceberg/src/expr/visitors/manifest_evaluator.rs: ## @@ -103,98 +106,245 @@ impl BoundPredicateVisitor for ManifestFilterVisitor<'_> { reference: &Bou

Re: [PR] Implement BoundPredicateVisitor trait for ManifestFilterVisitor [iceberg-rust]

2024-05-21 Thread via GitHub
sdd commented on code in PR #367: URL: https://github.com/apache/iceberg-rust/pull/367#discussion_r1609384502 ## crates/iceberg/src/expr/visitors/manifest_evaluator.rs: ## @@ -103,98 +106,245 @@ impl BoundPredicateVisitor for ManifestFilterVisitor<'_> { reference: &Bou

Re: [PR] feat: Add equality delete writer [iceberg-rust]

2024-05-21 Thread via GitHub
sdd commented on code in PR #372: URL: https://github.com/apache/iceberg-rust/pull/372#discussion_r1609377510 ## crates/iceberg/src/writer/base_writer/equality_delete_writer.rs: ## @@ -0,0 +1,438 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contr

Re: [PR] feat: Add equality delete writer [iceberg-rust]

2024-05-21 Thread via GitHub
sdd commented on code in PR #372: URL: https://github.com/apache/iceberg-rust/pull/372#discussion_r1609375064 ## crates/iceberg/src/writer/base_writer/equality_delete_writer.rs: ## @@ -0,0 +1,438 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contr

Re: [PR] feat: Add equality delete writer [iceberg-rust]

2024-05-21 Thread via GitHub
sdd commented on code in PR #372: URL: https://github.com/apache/iceberg-rust/pull/372#discussion_r1609372918 ## crates/iceberg/src/writer/base_writer/equality_delete_writer.rs: ## @@ -0,0 +1,438 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contr

Re: [PR] feat: Add equality delete writer [iceberg-rust]

2024-05-21 Thread via GitHub
sdd commented on code in PR #372: URL: https://github.com/apache/iceberg-rust/pull/372#discussion_r1609372170 ## crates/iceberg/src/writer/base_writer/equality_delete_writer.rs: ## @@ -0,0 +1,438 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contr

Re: [PR] Concurrent table scans [iceberg-rust]

2024-05-21 Thread via GitHub
sdd commented on code in PR #373: URL: https://github.com/apache/iceberg-rust/pull/373#discussion_r1609368600 ## crates/iceberg/src/scan.rs: ## @@ -189,66 +195,20 @@ impl TableScan { self.case_sensitive, )?; -let mut partition_filter_cache = Parti

Re: [PR] Spark 3.5: Add validation to SparkConfParser [iceberg]

2024-05-21 Thread via GitHub
jerqi commented on code in PR #10315: URL: https://github.com/apache/iceberg/pull/10315#discussion_r1609368411 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkConfParser.java: ## @@ -264,7 +261,26 @@ public ThisT tableProperty(String name) { return self();

Re: [PR] Spark 3.5: Add validation to SparkConfParser [iceberg]

2024-05-21 Thread via GitHub
jerqi commented on code in PR #10315: URL: https://github.com/apache/iceberg/pull/10315#discussion_r1609365261 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkConfParser.java: ## @@ -233,19 +229,20 @@ public EnumConfParser defaultValue(String value) { }

Re: [PR] Spark 3.5: Add validation to SparkConfParser [iceberg]

2024-05-21 Thread via GitHub
jerqi commented on code in PR #10315: URL: https://github.com/apache/iceberg/pull/10315#discussion_r1609365261 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkConfParser.java: ## @@ -233,19 +229,20 @@ public EnumConfParser defaultValue(String value) { }

Re: [PR] feat: make file scan task serializable [iceberg-rust]

2024-05-21 Thread via GitHub
ZENOTME commented on PR #377: URL: https://github.com/apache/iceberg-rust/pull/377#issuecomment-2123974633 > This seems reasonable, but perhaps we might want to consider having this as a separate method to the existing `plan_files` though so that anyone who is using the existing stream of f

Re: [PR] feat: make file scan task serializable [iceberg-rust]

2024-05-21 Thread via GitHub
sdd commented on PR #377: URL: https://github.com/apache/iceberg-rust/pull/377#issuecomment-2123920187 This seems reasonable, but perhaps we might want to consider having this as a separate method to the existing `plan_files` though so that anyone who is using the existing stream of file pl

Re: [I] remove orphan file question [iceberg]

2024-05-21 Thread via GitHub
manuzhang commented on issue #10363: URL: https://github.com/apache/iceberg/issues/10363#issuecomment-2123824636 It usually happens when snapshots are removed while their referenced files are not somehow. For example, in `expire_snapshots` procedure, expired file are removed after their sn

Re: [PR] Spark 3.5: Only traverse ancestors of current snapshot when building changelog scan [iceberg]

2024-05-21 Thread via GitHub
manuzhang commented on PR #10252: URL: https://github.com/apache/iceberg/pull/10252#issuecomment-2123810484 Gentle ping @flyrain @aokolnychyi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Migrate HadoopCatalog related tests in Flink [iceberg]

2024-05-21 Thread via GitHub
tomtongue commented on PR #10358: URL: https://github.com/apache/iceberg/pull/10358#issuecomment-2123786092 @nastra Sorry for the delay. Could you review this PR when you have time? If it's fine, I will add another versions to this PR. -- This is an automated message from the Apache Git S

Re: [PR] Support creating tags by adding `set_ref_snapshot` API [iceberg-python]

2024-05-21 Thread via GitHub
HonahX commented on PR #728: URL: https://github.com/apache/iceberg-python/pull/728#issuecomment-2123764123 Thanks @chinmay-bhat for taking this and @syun64 for reviewing! Linking [my comment](https://github.com/apache/iceberg-python/issues/737) to here, I actually prefer making this an int

Re: [I] Support Snapshot Management Operations [iceberg-python]

2024-05-21 Thread via GitHub
HonahX commented on issue #737: URL: https://github.com/apache/iceberg-python/issues/737#issuecomment-2123759077 Hi @syun64 @chinmay-bhat. Thank you for driving these features. I saw some PRs raised by @chinmay-bhat : #748, #750, #758. Those are great! I would like to discuss what AP

Re: [I] The decimal data type is transformed after the data is inserted. [iceberg-python]

2024-05-21 Thread via GitHub
as10128 commented on issue #751: URL: https://github.com/apache/iceberg-python/issues/751#issuecomment-2123742680 @ndrluis I wrote a test code, I used minio and hive catalog, you need to modify the aws_access_key, aws_secret_key, s3_endpoint, hive_thrift_uri, warehouse_uri parameters.The re

Re: [PR] [RAY] ray support Process concurrent [iceberg-python]

2024-05-21 Thread via GitHub
KnightChess commented on code in PR #753: URL: https://github.com/apache/iceberg-python/pull/753#discussion_r1609166388 ## pyiceberg/table/__init__.py: ## @@ -1774,8 +1774,19 @@ def to_duckdb(self, table_name: str, connection: Optional[DuckDBPyConnection] = def to_ray(se

Re: [PR] Improve CLI Text by Adding Verbose Text for Commands [iceberg-go]

2024-05-21 Thread via GitHub
rtadepalli commented on code in PR #68: URL: https://github.com/apache/iceberg-go/pull/68#discussion_r1609138974 ## cmd/iceberg/main.go: ## @@ -34,16 +34,21 @@ import ( const usage = `iceberg. Usage: - iceberg list [options] [PARENT] - iceberg describe [options] [namespace

Re: [I] Create Table in SQL Catalog fails if location is not mentioned but write.data.path and write.metadata.path are [iceberg-python]

2024-05-21 Thread via GitHub
github-actions[bot] commented on issue #161: URL: https://github.com/apache/iceberg-python/issues/161#issuecomment-2123624559 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity oc

[PR] Bump mypy-boto3-glue from 1.34.88 to 1.34.110 [iceberg-python]

2024-05-21 Thread via GitHub
dependabot[bot] opened a new pull request, #761: URL: https://github.com/apache/iceberg-python/pull/761 Bumps [mypy-boto3-glue](https://github.com/youtype/mypy_boto3_builder) from 1.34.88 to 1.34.110. Commits See full diff in https://github.com/youtype/mypy_boto3_builder/commit

[PR] Bump griffe from 0.45.0 to 0.45.1 [iceberg-python]

2024-05-21 Thread via GitHub
dependabot[bot] opened a new pull request, #760: URL: https://github.com/apache/iceberg-python/pull/760 Bumps [griffe](https://github.com/mkdocstrings/griffe) from 0.45.0 to 0.45.1. Release notes Sourced from https://github.com/mkdocstrings/griffe/releases";>griffe's releases.

[PR] Bump requests from 2.32.1 to 2.32.2 [iceberg-python]

2024-05-21 Thread via GitHub
dependabot[bot] opened a new pull request, #759: URL: https://github.com/apache/iceberg-python/pull/759 Bumps [requests](https://github.com/psf/requests) from 2.32.1 to 2.32.2. Release notes Sourced from https://github.com/psf/requests/releases";>requests's releases. v2.32.2

Re: [PR] Spark 3.5: Add validation to SparkConfParser [iceberg]

2024-05-21 Thread via GitHub
szehon-ho commented on code in PR #10315: URL: https://github.com/apache/iceberg/pull/10315#discussion_r1605423430 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkConfParser.java: ## @@ -264,7 +261,26 @@ public ThisT tableProperty(String name) { return sel

Re: [PR] View: add property to describe advisory read mode [iceberg]

2024-05-21 Thread via GitHub
rdblue commented on PR #10362: URL: https://github.com/apache/iceberg/pull/10362#issuecomment-2123427489 Can you give an example? I can't think of any pushdown operation that exposes data to user code. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] [FEAT]register table using iceberg metadata file via pyiceberg [iceberg-python]

2024-05-21 Thread via GitHub
MehulBatra commented on code in PR #711: URL: https://github.com/apache/iceberg-python/pull/711#discussion_r1608874056 ## tests/catalog/integration_test_glue.py: ## @@ -564,3 +564,13 @@ def test_table_exists(test_catalog: Catalog, table_schema_nested: Schema, table_ test_c

Re: [PR] [FEAT]register table using iceberg metadata file via pyiceberg [iceberg-python]

2024-05-21 Thread via GitHub
MehulBatra commented on code in PR #711: URL: https://github.com/apache/iceberg-python/pull/711#discussion_r1608868910 ## tests/catalog/test_glue.py: ## @@ -848,3 +848,17 @@ def test_table_exists( assert test_catalog.table_exists(identifier) is True # Act and Assert fo

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

2024-05-21 Thread via GitHub
epgif commented on PR #9008: URL: https://github.com/apache/iceberg/pull/9008#issuecomment-2123288831 This should be ready to go. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] View: add property to describe advisory read mode [iceberg]

2024-05-21 Thread via GitHub
jackye1995 commented on PR #10362: URL: https://github.com/apache/iceberg/pull/10362#issuecomment-2123261221 This allows an engine to distinguish if a view is a secure view, and act accordingly. As described in similar systems like https://docs.snowflake.com/en/user-guide/views-secure:

Re: [PR] View: add property to describe advisory read mode [iceberg]

2024-05-21 Thread via GitHub
rdblue commented on PR #10362: URL: https://github.com/apache/iceberg/pull/10362#issuecomment-2123228246 @jackye1995 can you help me understand the case that this is protecting against? I'm not sure I understand the loophole that this is attempting to close. -- This is an automated messa

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-21 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1608734772 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MonitorSource.java: ## @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-21 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1608734772 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MonitorSource.java: ## @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-21 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1608725056 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/SingleThreadedIteratorSource.java: ## @@ -0,0 +1,196 @@ +/* + * Licensed to the Apache

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-21 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1608717408 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MonitorSource.java: ## @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-21 Thread via GitHub
stevenzwu commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1608709287 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MonitorSource.java: ## @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-21 Thread via GitHub
stevenzwu commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1608678483 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MonitorSource.java: ## @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-21 Thread via GitHub
stevenzwu commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1608691840 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/SingleThreadedIteratorSource.java: ## @@ -0,0 +1,196 @@ +/* + * Licensed to the Apa

Re: [PR] [Spec] Add Iceberg Materialized View Spec [iceberg]

2024-05-21 Thread via GitHub
szehon-ho commented on code in PR #10280: URL: https://github.com/apache/iceberg/pull/10280#discussion_r1607015368 ## format/materialized-view-spec.md: ## @@ -0,0 +1,132 @@ + + +# Iceberg Materialized View Spec + +## Background and Motivation +Iceberg views are a powerful tool t

Re: [PR] [Spec] Add Iceberg Materialized View Spec [iceberg]

2024-05-21 Thread via GitHub
szehon-ho commented on code in PR #10280: URL: https://github.com/apache/iceberg/pull/10280#discussion_r1607015368 ## format/materialized-view-spec.md: ## @@ -0,0 +1,132 @@ + + +# Iceberg Materialized View Spec + +## Background and Motivation +Iceberg views are a powerful tool t

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-21 Thread via GitHub
stevenzwu commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1608678483 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MonitorSource.java: ## @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] Build: Bump pypa/cibuildwheel from 2.18.0 to 2.18.1 [iceberg-python]

2024-05-21 Thread via GitHub
HonahX merged PR #756: URL: https://github.com/apache/iceberg-python/pull/756 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Build: Bump requests from 2.31.0 to 2.32.1 [iceberg-python]

2024-05-21 Thread via GitHub
HonahX merged PR #755: URL: https://github.com/apache/iceberg-python/pull/755 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Build: Bump mkdocs-material from 9.5.22 to 9.5.24 [iceberg-python]

2024-05-21 Thread via GitHub
HonahX merged PR #754: URL: https://github.com/apache/iceberg-python/pull/754 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Build: Bump boto3 from 1.34.69 to 1.34.106 [iceberg-python]

2024-05-21 Thread via GitHub
HonahX merged PR #749: URL: https://github.com/apache/iceberg-python/pull/749 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Build: Bump mkdocstrings-python from 1.10.1 to 1.10.2 [iceberg-python]

2024-05-21 Thread via GitHub
HonahX merged PR #746: URL: https://github.com/apache/iceberg-python/pull/746 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] [FEAT]register table using iceberg metadata file via pyiceberg [iceberg-python]

2024-05-21 Thread via GitHub
HonahX commented on code in PR #711: URL: https://github.com/apache/iceberg-python/pull/711#discussion_r1608625978 ## tests/catalog/test_glue.py: ## @@ -848,3 +848,17 @@ def test_table_exists( assert test_catalog.table_exists(identifier) is True # Act and Assert for a

Re: [PR] [FEAT]register table using iceberg metadata file via pyiceberg [iceberg-python]

2024-05-21 Thread via GitHub
HonahX commented on code in PR #711: URL: https://github.com/apache/iceberg-python/pull/711#discussion_r1608637860 ## tests/catalog/integration_test_glue.py: ## @@ -564,3 +564,13 @@ def test_table_exists(test_catalog: Catalog, table_schema_nested: Schema, table_ test_catal

Re: [PR] [RAY] ray support Process concurrent [iceberg-python]

2024-05-21 Thread via GitHub
corleyma commented on code in PR #753: URL: https://github.com/apache/iceberg-python/pull/753#discussion_r1608626491 ## pyiceberg/table/__init__.py: ## @@ -1774,8 +1774,19 @@ def to_duckdb(self, table_name: str, connection: Optional[DuckDBPyConnection] = def to_ray(self)

Re: [PR] [FEAT]register table using iceberg metadata file via pyiceberg [iceberg-python]

2024-05-21 Thread via GitHub
HonahX commented on code in PR #711: URL: https://github.com/apache/iceberg-python/pull/711#discussion_r1608625978 ## tests/catalog/test_glue.py: ## @@ -848,3 +848,17 @@ def test_table_exists( assert test_catalog.table_exists(identifier) is True # Act and Assert for a

Re: [PR] AWS: add retry logic to S3InputStream [iceberg]

2024-05-21 Thread via GitHub
puchengy commented on PR #4912: URL: https://github.com/apache/iceberg/pull/4912#issuecomment-2123007781 thank you @danielcweeks and @amogh-jahagirdar ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Data: Make GenericFileWriterFactory public [iceberg]

2024-05-21 Thread via GitHub
jbonofre commented on PR #10365: URL: https://github.com/apache/iceberg/pull/10365#issuecomment-2122902157 @danielcweeks just verified and it's OK using `OutputFileFactory` and `GenericAppenderFactory`. Thanks and sorry for the noise 😄 -- This is an automated message from the Apache Git

Re: [PR] Data: Make GenericFileWriterFactory public [iceberg]

2024-05-21 Thread via GitHub
jbonofre closed pull request #10365: Data: Make GenericFileWriterFactory public URL: https://github.com/apache/iceberg/pull/10365 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Data: Make GenericFileWriterFactory public [iceberg]

2024-05-21 Thread via GitHub
jbonofre commented on PR #10365: URL: https://github.com/apache/iceberg/pull/10365#issuecomment-2122886388 @danielcweeks let me check, that should do the trick 😄 Thanks ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Data: Make GenericFileWriterFactory public [iceberg]

2024-05-21 Thread via GitHub
danielcweeks commented on PR #10365: URL: https://github.com/apache/iceberg/pull/10365#issuecomment-2122881259 @jbonofre other projects (like kafka connect) are using the [GenericAppenderFactory](https://github.com/apache/iceberg/blob/1288eb8e33304e2c5dbf7ea78de9f51f422cfad1/data/src/main/ja

Re: [I] check-ordering enablement for flink config [iceberg]

2024-05-21 Thread via GitHub
pvary commented on issue #10360: URL: https://github.com/apache/iceberg/issues/10360#issuecomment-2122772197 This is non-trivial, as currently the schema is compiled to the job graph. We need to decouple the schema either using external schema registry, or switching to less performant seria

Re: [I] The decimal data type is transformed after the data is inserted. [iceberg-python]

2024-05-21 Thread via GitHub
ndrluis commented on issue #751: URL: https://github.com/apache/iceberg-python/issues/751#issuecomment-2122738484 Hello @as10128, can you share with us a minimal code example that reproduces this error? -- This is an automated message from the Apache Git Service. To respond to the message

Re: [PR] OpenAPI: Express server capabilities via /config endpoint [iceberg]

2024-05-21 Thread via GitHub
jackye1995 commented on PR #9940: URL: https://github.com/apache/iceberg/pull/9940#issuecomment-2122735443 How is this tag-based approach backwards compatible? Suppose someone does not have a `view` tag but we support all view APIs because they have the catalog running right now, it seems l

Re: [PR] Spec: Add context query parameter for all REST APIs [iceberg]

2024-05-21 Thread via GitHub
jackye1995 commented on PR #10359: URL: https://github.com/apache/iceberg/pull/10359#issuecomment-2122678361 Thanks for the quick feedback Dan! I think this is a similar discussion to the MV one for properties vs new metadata, I think at least in this case the property-based approach fits t

Re: [PR] Fix ParallelIterable memory leak because queue continues to be added even if iterator exited [iceberg]

2024-05-21 Thread via GitHub
etra commented on PR #9402: URL: https://github.com/apache/iceberg/pull/9402#issuecomment-2122642037 Any plans for when this will be released? Thank you -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [PR] Data: Make GenericFileWriterFactory public [iceberg]

2024-05-21 Thread via GitHub
jbonofre commented on PR #10365: URL: https://github.com/apache/iceberg/pull/10365#issuecomment-2122551297 @aokolnychyi @nastra thoughts ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] Kafka Connect: Commit coordination [iceberg]

2024-05-21 Thread via GitHub
ajantha-bhat commented on code in PR #10351: URL: https://github.com/apache/iceberg/pull/10351#discussion_r1607813508 ## kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/CatalogUtils.java: ## @@ -0,0 +1,98 @@ +/* + * Licensed to the Apache Software Foundation

Re: [I] unable to show views [iceberg]

2024-05-21 Thread via GitHub
Shantanu0701 commented on issue #10364: URL: https://github.com/apache/iceberg/issues/10364#issuecomment-2122264569 All let me know the proper syntax to create a view and show all views as per above catalog name, database name and table name. I have below record stored in SELECT * FRO

[I] unable to show views [iceberg]

2024-05-21 Thread via GitHub
Shantanu0701 opened a new issue, #10364: URL: https://github.com/apache/iceberg/issues/10364 As I have created a view as seen below but when I am printing its showing error. So can't even say if the views are created successfully or not. import pyspark from pyspark.sql import S

Re: [PR] feat: make file scan task serializable [iceberg-rust]

2024-05-21 Thread via GitHub
ZENOTME commented on PR #377: URL: https://github.com/apache/iceberg-rust/pull/377#issuecomment-2122049316 cc @liurenjie1024 @Fokko @Xuanwo @sdd @viirya -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[PR] feat: make file scan task serializable [iceberg-rust]

2024-05-21 Thread via GitHub
ZENOTME opened a new pull request, #377: URL: https://github.com/apache/iceberg-rust/pull/377 There is a user case of file scan task for the compute engine: 1. compute the file scan task and shuffle them to the compute node 2. The compute node does the scan work in parallel In

Re: [PR] [WIP] Migrate HadoopCatalog related tests in Flink [iceberg]

2024-05-21 Thread via GitHub
tomtongue commented on PR #10358: URL: https://github.com/apache/iceberg/pull/10358#issuecomment-2121947382 Other `HadoopTableResource` related files will be migrated in another PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [PR] [FEAT]register table using iceberg metadata file via pyiceberg [iceberg-python]

2024-05-21 Thread via GitHub
MehulBatra commented on code in PR #711: URL: https://github.com/apache/iceberg-python/pull/711#discussion_r1607758533 ## tests/catalog/test_glue.py: ## @@ -848,3 +848,17 @@ def test_table_exists( assert test_catalog.table_exists(identifier) is True # Act and Assert fo

Re: [I] feat: Implement data file metrics evaluator to prune data files using filter. [iceberg-rust]

2024-05-21 Thread via GitHub
sdd commented on issue #152: URL: https://github.com/apache/iceberg-rust/issues/152#issuecomment-2121901733 @liurenjie1024 this is now done, since https://github.com/apache/iceberg-rust/pull/347 got merged. -- This is an automated message from the Apache Git Service. To respond to the mes