Re: [PR] Spark SystemFunctions are not pushed down during MERGE [iceberg]

tmnd1991 commented on PR #9233: URL: https://github.com/apache/iceberg/pull/9233#issuecomment-1844912767 Hi @ConeyLiu, this still needs some refinement (mostly wrt testing) but do you think the change make sense? I would avoid to work more on it if I'm way off ;) Thanks 🙏 -- This is

[I] Add partition evolution [iceberg-python]

nicor88 opened a new issue, #193: URL: https://github.com/apache/iceberg-python/issues/193 ### Feature Request / Improvement ## Contex One of the advantage of Iceberg is that the original "partition" design can be changed over time. This is often the case, as a table is originally

Re: [PR] Arrow: Allow missing field-ids from Schema [iceberg-python]

HonahX commented on code in PR #183: URL: https://github.com/apache/iceberg-python/pull/183#discussion_r1418516560 ## pyiceberg/io/pyarrow.py: ## @@ -713,28 +714,50 @@ def primitive(self, primitive: pa.DataType) -> Optional[T]: """Visit a primitive type.""" -def _g

Re: [I] The "Status" paragraph in the readme seems very outdated [iceberg]

ronkorving commented on issue #9127: URL: https://github.com/apache/iceberg/issues/9127#issuecomment-1844981085 @Fokko Thanks for the engagement :) Would you agree that perhaps the readme is not the best place to the "what we're currently working on" information, and perhaps it's better t

Re: [PR] Spark: IN clause on system function is not pushed down [iceberg]

tmnd1991 commented on PR #9192: URL: https://github.com/apache/iceberg/pull/9192#issuecomment-1845066093 Can I have a feedback on this one @ConeyLiu, too? Thanks a lot -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] Nessie: Support views for NessieCatalog [iceberg]

ajantha-bhat commented on PR #8909: URL: https://github.com/apache/iceberg/pull/8909#issuecomment-1845096677 @snazy, @nastra: If this PR is ok. Can I get an approval? If not, what more changes are expected? I thought of waiting for https://github.com/apache/iceberg/pull/8857/ an

Re: [PR] Update Slack Links [iceberg-docs]

nastra commented on PR #295: URL: https://github.com/apache/iceberg-docs/pull/295#issuecomment-1845144252 @bitsondatadev I think you'd need to update this on the `latest` branch as well to properly show up on the website -- This is an automated message from the Apache Git Service. To resp

Re: [PR] Fix branching and tagging images on 1.2.1 branch [iceberg-docs]

nastra commented on PR #231: URL: https://github.com/apache/iceberg-docs/pull/231#issuecomment-1845144982 @amogh-jahagirdar is this still relevant? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Docs: Document reading in Spark using branch and tag identifiers [iceberg]

nastra merged PR #9238: URL: https://github.com/apache/iceberg/pull/9238 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] feat: support UnboundPartitionSpec [iceberg-rust]

Fokko commented on code in PR #106: URL: https://github.com/apache/iceberg-rust/pull/106#discussion_r1418802658 ## crates/iceberg/src/catalog/mod.rs: ## @@ -758,23 +760,19 @@ mod tests { { "action": "add-spec", "spec": { -"spec-id": 1, "fields": [

Re: [PR] test: Add integration tests for rest catalog. [iceberg-rust]

Fokko commented on code in PR #109: URL: https://github.com/apache/iceberg-rust/pull/109#discussion_r1418811263 ## crates/catalog/rest/tests/rest_catalog_test.rs: ## @@ -0,0 +1,376 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license

[PR] Docs: Update default format version to 2 [iceberg]

zhongyujiang opened a new pull request, #9239: URL: https://github.com/apache/iceberg/pull/9239 Correct the default format version in documentation, we already switched to V2 by default since #8381. -- This is an automated message from the Apache Git Service. To respond to the message, pl

Re: [PR] chore: Add cargo build and build guide [iceberg-rust]

Fokko merged PR #111: URL: https://github.com/apache/iceberg-rust/pull/111 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] feat: Add hms catalog layout [iceberg-rust]

Fokko merged PR #112: URL: https://github.com/apache/iceberg-rust/pull/112 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] feat: support UnboundPartitionSpec [iceberg-rust]

liurenjie1024 commented on code in PR #106: URL: https://github.com/apache/iceberg-rust/pull/106#discussion_r1418814937 ## crates/iceberg/src/catalog/mod.rs: ## @@ -758,23 +760,19 @@ mod tests { { "action": "add-spec", "spec": { -"spec-id": 1, "fields

Re: [PR] test: Add integration tests for rest catalog. [iceberg-rust]

Fokko commented on code in PR #109: URL: https://github.com/apache/iceberg-rust/pull/109#discussion_r1418813623 ## crates/test_utils/src/docker.rs: ## @@ -0,0 +1,102 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. Se

Re: [PR] feat: support UnboundPartitionSpec [iceberg-rust]

Fokko commented on code in PR #106: URL: https://github.com/apache/iceberg-rust/pull/106#discussion_r1418820203 ## crates/iceberg/src/catalog/mod.rs: ## @@ -758,23 +760,19 @@ mod tests { { "action": "add-spec", "spec": { -"spec-id": 1, "fields": [

Re: [PR] Nessie: reimplement namespace operations [iceberg]

nastra merged PR #8857: URL: https://github.com/apache/iceberg/pull/8857 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] feat: support UnboundPartitionSpec [iceberg-rust]

Fokko commented on code in PR #106: URL: https://github.com/apache/iceberg-rust/pull/106#discussion_r1418824322 ## crates/iceberg/src/catalog/mod.rs: ## @@ -758,23 +760,19 @@ mod tests { { "action": "add-spec", "spec": { -"spec-id": 1, "fields": [

Re: [PR] feat: support UnboundPartitionSpec [iceberg-rust]

Fokko commented on code in PR #106: URL: https://github.com/apache/iceberg-rust/pull/106#discussion_r1418823234 ## crates/iceberg/src/catalog/mod.rs: ## @@ -758,23 +760,19 @@ mod tests { { "action": "add-spec", "spec": { -"spec-id": 1, "fields": [

Re: [PR] Parquet: Add a table property to control the Parquet row-group size of position delete files [iceberg]

zhongyujiang commented on PR #9177: URL: https://github.com/apache/iceberg/pull/9177#issuecomment-1845189563 @rdblue @RussellSpitzer would you mind take a look at this? Thanks in advance! -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] Typo? [iceberg-docs]

Fokko commented on PR #270: URL: https://github.com/apache/iceberg-docs/pull/270#issuecomment-1845241457 Thanks @lorenzwalthert for taking the time to fix this 👍 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] The "Status" paragraph in the readme seems very outdated [iceberg]

Fokko commented on issue #9127: URL: https://github.com/apache/iceberg/issues/9127#issuecomment-1845240644 Yes, I think that makes a lot of sense. @bitsondatadev is also working on updating the [roadmap](https://iceberg.apache.org/roadmap/), it might be a good idea to link to that so we hav

Re: [PR] Typo? [iceberg-docs]

Fokko merged PR #270: URL: https://github.com/apache/iceberg-docs/pull/270 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Fix spelling [iceberg-docs]

Fokko merged PR #271: URL: https://github.com/apache/iceberg-docs/pull/271 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Update vendors.md [iceberg-docs]

Fokko merged PR #275: URL: https://github.com/apache/iceberg-docs/pull/275 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Nessie: Support views for NessieCatalog [iceberg]

ajantha-bhat commented on PR #8909: URL: https://github.com/apache/iceberg/pull/8909#issuecomment-1845247046 Note: I am squashing the commits into one (since I am having hard time rebasing with 8 commits). -- This is an automated message from the Apache Git Service. To respond to the mes

[PR] Open-API: Refactor updates with discriminator [iceberg]

Fokko opened a new pull request, #9240: URL: https://github.com/apache/iceberg/pull/9240 This generates nicer code -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

Re: [PR] feat: support UnboundPartitionSpec [iceberg-rust]

Fokko commented on code in PR #106: URL: https://github.com/apache/iceberg-rust/pull/106#discussion_r141983 ## crates/iceberg/src/catalog/mod.rs: ## @@ -758,23 +760,19 @@ mod tests { { "action": "add-spec", "spec": { -"spec-id": 1, "fields": [

Re: [PR] feat: support UnboundPartitionSpec [iceberg-rust]

Fokko merged PR #106: URL: https://github.com/apache/iceberg-rust/pull/106 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [I] feat: Add support for `UnboundPartitionSpec`. [iceberg-rust]

Fokko closed issue #98: feat: Add support for `UnboundPartitionSpec`. URL: https://github.com/apache/iceberg-rust/issues/98 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

Re: [PR] feat: support UnboundPartitionSpec [iceberg-rust]

Fokko commented on PR #106: URL: https://github.com/apache/iceberg-rust/pull/106#issuecomment-1845257956 Thanks @my-vegetable-has-exploded for working on this, and @liurenjie1024 and @Xuanwo for reviewing this 👍 -- This is an automated message from the Apache Git Service. To respond to t

Re: [PR] Build: Bump actions/setup-python from 4 to 5 [iceberg-python]

Fokko merged PR #191: URL: https://github.com/apache/iceberg-python/pull/191 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Build: Bump fastavro from 1.9.0 to 1.9.1 [iceberg-python]

Fokko merged PR #190: URL: https://github.com/apache/iceberg-python/pull/190 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Core: Fixed certain operations failing to add new data files during retries [iceberg]

nastra commented on PR #9230: URL: https://github.com/apache/iceberg/pull/9230#issuecomment-1845262257 @ConeyLiu could you also please take a look at this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Build: Bump griffe from 0.38.0 to 0.38.1 [iceberg-python]

Fokko merged PR #189: URL: https://github.com/apache/iceberg-python/pull/189 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] maint(transforms): replace `type()` calls with `isinstance()` [iceberg-python]

Fokko commented on PR #188: URL: https://github.com/apache/iceberg-python/pull/188#issuecomment-1845273979 @jayceslesar looks like `black` still wants to reformat some code. Can you run `make lint`? Thanks! -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [PR] Automatically create the tables for the `SqlCatalog` [iceberg-python]

Fokko commented on PR #186: URL: https://github.com/apache/iceberg-python/pull/186#issuecomment-1845276186 This is beautiful @cosmastech, in this same PR, can you remove these lines: https://github.com/apache/iceberg-python/blob/5337504963dec7a410ba762da08064578976bd7a/mkdocs/docs/api.md?pla

Re: [PR] Add Snapshot logic and Summary generation [iceberg-python]

Fokko commented on code in PR #61: URL: https://github.com/apache/iceberg-python/pull/61#discussion_r1418929287 ## pyiceberg/table/snapshots.py: ## @@ -116,3 +144,199 @@ class MetadataLogEntry(IcebergBaseModel): class SnapshotLogEntry(IcebergBaseModel): snapshot_id: int =

Re: [PR] Add Snapshot logic and Summary generation [iceberg-python]

Fokko commented on code in PR #61: URL: https://github.com/apache/iceberg-python/pull/61#discussion_r1418931669 ## pyiceberg/table/snapshots.py: ## @@ -19,15 +19,40 @@ Any, Dict, List, +Mapping, Optional, ) from pydantic import Field, PrivateAttr, mod

Re: [PR] Add Snapshot logic and Summary generation [iceberg-python]

Fokko commented on code in PR #61: URL: https://github.com/apache/iceberg-python/pull/61#discussion_r1418932283 ## pyiceberg/table/snapshots.py: ## @@ -19,15 +19,40 @@ Any, Dict, List, +Mapping, Optional, ) from pydantic import Field, PrivateAttr, mod

Re: [PR] Add Snapshot logic and Summary generation [iceberg-python]

Fokko commented on code in PR #61: URL: https://github.com/apache/iceberg-python/pull/61#discussion_r1418934519 ## pyiceberg/table/snapshots.py: ## @@ -65,6 +90,25 @@ def __init__(self, operation: Operation, **data: Any) -> None: super().__init__(operation=operation, *

Re: [PR] Add Snapshot logic and Summary generation [iceberg-python]

Fokko commented on code in PR #61: URL: https://github.com/apache/iceberg-python/pull/61#discussion_r1418935983 ## pyiceberg/table/snapshots.py: ## @@ -116,3 +168,193 @@ class MetadataLogEntry(IcebergBaseModel): class SnapshotLogEntry(IcebergBaseModel): snapshot_id: int =

Re: [PR] Automatically create the tables for the `SqlCatalog` [iceberg-python]

cosmastech commented on PR #186: URL: https://github.com/apache/iceberg-python/pull/186#issuecomment-1845323941 > This is beautiful @cosmastech, in this same PR, can you remove these lines: > > https://github.com/apache/iceberg-python/blob/5337504963dec7a410ba762da08064578976bd7a/mkdo

Re: [PR] Core: Fixed certain operations failing to add new data files during retries [iceberg]

nastra commented on code in PR #9230: URL: https://github.com/apache/iceberg/pull/9230#discussion_r1418954500 ## core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java: ## @@ -895,7 +895,7 @@ private void cleanUncommittedAppends(Set committed) { } }

Re: [PR] Docs: Update default format version to 2 [iceberg]

nastra commented on PR #9239: URL: https://github.com/apache/iceberg/pull/9239#issuecomment-1845334573 thanks @zhongyujiang for adding this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] Docs: Update default format version to 2 [iceberg]

nastra merged PR #9239: URL: https://github.com/apache/iceberg/pull/9239 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] API: Replace special case of deprecated RuntimeIOException with FileHandlingException [iceberg]

nastra commented on code in PR #6887: URL: https://github.com/apache/iceberg/pull/6887#discussion_r1418963810 ## core/src/main/java/org/apache/iceberg/avro/AvroFileAppender.java: ## @@ -78,14 +78,13 @@ public Metrics metrics() { @Override public long length() { -if (

Re: [PR] API: Replace special case of deprecated RuntimeIOException with FileHandlingException [iceberg]

nastra commented on code in PR #6887: URL: https://github.com/apache/iceberg/pull/6887#discussion_r1418968003 ## core/src/main/java/org/apache/iceberg/avro/AvroFileAppender.java: ## @@ -78,14 +78,13 @@ public Metrics metrics() { @Override public long length() { -if (

Re: [PR] API: Replace special case of deprecated RuntimeIOException with FileHandlingException [iceberg]

nastra commented on code in PR #6887: URL: https://github.com/apache/iceberg/pull/6887#discussion_r1418968329 ## core/src/main/java/org/apache/iceberg/avro/AvroIO.java: ## @@ -178,7 +178,7 @@ static long findStartingRowPos(Supplier open, long start) { SYNC_READER.rea

Re: [PR] API: Replace special case of deprecated RuntimeIOException with FileHandlingException [iceberg]

nastra commented on code in PR #6887: URL: https://github.com/apache/iceberg/pull/6887#discussion_r1418968329 ## core/src/main/java/org/apache/iceberg/avro/AvroIO.java: ## @@ -178,7 +178,7 @@ static long findStartingRowPos(Supplier open, long start) { SYNC_READER.rea

Re: [PR] JDBC catalog fix namespaceExists check [iceberg]

nastra commented on code in PR #8340: URL: https://github.com/apache/iceberg/pull/8340#discussion_r1418975543 ## core/src/test/java/org/apache/iceberg/jdbc/TestJdbcCatalog.java: ## @@ -601,15 +601,76 @@ public void testDropNamespace() { public void testCreateNamespace() {

Re: [PR] API: Replace special case of deprecated RuntimeIOException [iceberg]

ajantha-bhat commented on code in PR #6887: URL: https://github.com/apache/iceberg/pull/6887#discussion_r1418996270 ## core/src/main/java/org/apache/iceberg/avro/AvroFileAppender.java: ## @@ -78,14 +78,13 @@ public Metrics metrics() { @Override public long length() { -

Re: [PR] API: Replace special case of deprecated RuntimeIOException [iceberg]

ajantha-bhat commented on code in PR #6887: URL: https://github.com/apache/iceberg/pull/6887#discussion_r1418998188 ## core/src/main/java/org/apache/iceberg/avro/AvroFileAppender.java: ## @@ -78,14 +78,13 @@ public Metrics metrics() { @Override public long length() { -

Re: [PR] API: Replace special case of deprecated RuntimeIOException [iceberg]

ajantha-bhat commented on code in PR #6887: URL: https://github.com/apache/iceberg/pull/6887#discussion_r1418998188 ## core/src/main/java/org/apache/iceberg/avro/AvroFileAppender.java: ## @@ -78,14 +78,13 @@ public Metrics metrics() { @Override public long length() { -

Re: [PR] API: Replace special case of deprecated RuntimeIOException [iceberg]

ajantha-bhat commented on code in PR #6887: URL: https://github.com/apache/iceberg/pull/6887#discussion_r1419002428 ## core/src/main/java/org/apache/iceberg/avro/AvroFileAppender.java: ## @@ -78,14 +78,13 @@ public Metrics metrics() { @Override public long length() { -

Re: [PR] API: Replace special case of deprecated RuntimeIOException [iceberg]

ajantha-bhat commented on code in PR #6887: URL: https://github.com/apache/iceberg/pull/6887#discussion_r1419010809 ## core/src/main/java/org/apache/iceberg/avro/AvroFileAppender.java: ## @@ -78,14 +78,13 @@ public Metrics metrics() { @Override public long length() { -

Re: [PR] API: Replace special case of deprecated RuntimeIOException [iceberg]

ajantha-bhat commented on PR #6887: URL: https://github.com/apache/iceberg/pull/6887#issuecomment-1845411543 @nastra: Thanks for the review. I have replied to comments. I think no new code modifications needed. Please check again. Thanks. -- This is an automated message from the

Re: [PR] API: Replace special case of deprecated RuntimeIOException [iceberg]

nastra commented on code in PR #6887: URL: https://github.com/apache/iceberg/pull/6887#discussion_r1419046382 ## core/src/main/java/org/apache/iceberg/avro/AvroFileAppender.java: ## @@ -78,14 +78,13 @@ public Metrics metrics() { @Override public long length() { -if (

Re: [PR] Nessie: Support views for NessieCatalog [iceberg]

nastra commented on code in PR #8909: URL: https://github.com/apache/iceberg/pull/8909#discussion_r1419059002 ## nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java: ## @@ -448,33 +473,82 @@ public void renameTable(TableIdentifier from, TableIdentifier to) {

Re: [PR] Add Snapshot logic and Summary generation [iceberg-python]

Fokko commented on code in PR #61: URL: https://github.com/apache/iceberg-python/pull/61#discussion_r1419062626 ## pyiceberg/table/snapshots.py: ## @@ -116,3 +168,193 @@ class MetadataLogEntry(IcebergBaseModel): class SnapshotLogEntry(IcebergBaseModel): snapshot_id: int =

Re: [PR] Core: Fixed certain operations failing to add new data files during retries [iceberg]

ConeyLiu commented on code in PR #9230: URL: https://github.com/apache/iceberg/pull/9230#discussion_r1419062959 ## core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java: ## @@ -895,7 +895,7 @@ private void cleanUncommittedAppends(Set committed) { }

Re: [PR] Nessie: Support views for NessieCatalog [iceberg]

nastra commented on code in PR #8909: URL: https://github.com/apache/iceberg/pull/8909#discussion_r1419065698 ## nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java: ## @@ -141,16 +147,26 @@ private UpdateableReference loadReference(String requestedRef, Stri

Re: [PR] Spark: IN clause on system function is not pushed down [iceberg]

ConeyLiu commented on code in PR #9192: URL: https://github.com/apache/iceberg/pull/9192#discussion_r1419067449 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ReplaceStaticInvoke.scala: ## @@ -40,14 +37,20 @@ import org.apache.spark.sql.typ

Re: [PR] Spark: Use Awaitility instead of Thread.sleep [iceberg]

yyy1000 commented on PR #9224: URL: https://github.com/apache/iceberg/pull/9224#issuecomment-1845461829 > > Hi, @nk1506 It seems that my code can't pass the unit tests. Is it because of the default waiting time is not enough? Do you have any ideas? > > Please refer [here](https://git

Re: [PR] Spark: IN clause on system function is not pushed down [iceberg]

ConeyLiu commented on PR #9192: URL: https://github.com/apache/iceberg/pull/9192#issuecomment-1845465132 @tmnd1991 thanks for your contribution. I think this is a valid fix. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Nessie: Support views for NessieCatalog [iceberg]

nastra commented on code in PR #8909: URL: https://github.com/apache/iceberg/pull/8909#discussion_r1419074476 ## nessie/src/test/java/org/apache/iceberg/nessie/BaseTestIceberg.java: ## @@ -180,6 +188,32 @@ protected Table createTable(TableIdentifier tableIdentifier, Schema sche

Re: [PR] Nessie: Support views for NessieCatalog [iceberg]

nastra commented on code in PR #8909: URL: https://github.com/apache/iceberg/pull/8909#discussion_r1419075510 ## nessie/src/test/java/org/apache/iceberg/nessie/BaseTestIceberg.java: ## @@ -180,6 +188,32 @@ protected Table createTable(TableIdentifier tableIdentifier, Schema sche

Re: [PR] Nessie: Support views for NessieCatalog [iceberg]

nastra commented on code in PR #8909: URL: https://github.com/apache/iceberg/pull/8909#discussion_r1419073781 ## nessie/src/test/java/org/apache/iceberg/nessie/BaseTestIceberg.java: ## @@ -267,4 +305,23 @@ static DataFile makeDataFile(Table icebergTable, String fileLocation) {

Re: [PR] Nessie: Support views for NessieCatalog [iceberg]

nastra commented on code in PR #8909: URL: https://github.com/apache/iceberg/pull/8909#discussion_r1419078008 ## nessie/src/test/java/org/apache/iceberg/nessie/TestNessieViewCatalog.java: ## @@ -0,0 +1,217 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

Re: [PR] Nessie: Support views for NessieCatalog [iceberg]

nastra commented on code in PR #8909: URL: https://github.com/apache/iceberg/pull/8909#discussion_r1419081069 ## nessie/src/test/java/org/apache/iceberg/nessie/TestNessieViewCatalog.java: ## @@ -0,0 +1,217 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

Re: [PR] Nessie: Support views for NessieCatalog [iceberg]

nastra commented on code in PR #8909: URL: https://github.com/apache/iceberg/pull/8909#discussion_r1419080356 ## nessie/src/test/java/org/apache/iceberg/nessie/TestNessieViewCatalog.java: ## @@ -0,0 +1,222 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

Re: [PR] Arrow: Allow missing field-ids from Schema [iceberg-python]

Fokko commented on code in PR #183: URL: https://github.com/apache/iceberg-python/pull/183#discussion_r1419083232 ## pyiceberg/io/pyarrow.py: ## @@ -713,28 +714,50 @@ def primitive(self, primitive: pa.DataType) -> Optional[T]: """Visit a primitive type.""" -def _ge

Re: [PR] Nessie: Support views for NessieCatalog [iceberg]

nastra commented on code in PR #8909: URL: https://github.com/apache/iceberg/pull/8909#discussion_r1419087100 ## nessie/src/test/java/org/apache/iceberg/nessie/TestNessieView.java: ## @@ -0,0 +1,337 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mor

Re: [PR] Nessie: Support views for NessieCatalog [iceberg]

nastra commented on code in PR #8909: URL: https://github.com/apache/iceberg/pull/8909#discussion_r1419083813 ## nessie/src/test/java/org/apache/iceberg/nessie/TestNessieView.java: ## @@ -0,0 +1,337 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mor

Re: [PR] Nessie: Support views for NessieCatalog [iceberg]

nastra commented on code in PR #8909: URL: https://github.com/apache/iceberg/pull/8909#discussion_r1419086031 ## nessie/src/test/java/org/apache/iceberg/nessie/TestNessieView.java: ## @@ -0,0 +1,337 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mor

Re: [PR] Nessie: Support views for NessieCatalog [iceberg]

nastra commented on code in PR #8909: URL: https://github.com/apache/iceberg/pull/8909#discussion_r1419087834 ## nessie/src/test/java/org/apache/iceberg/nessie/TestNessieView.java: ## @@ -0,0 +1,337 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mor

Re: [PR] Arrow: Allow missing field-ids from Schema [iceberg-python]

Fokko commented on code in PR #183: URL: https://github.com/apache/iceberg-python/pull/183#discussion_r1419090834 ## pyiceberg/io/pyarrow.py: ## @@ -713,28 +714,50 @@ def primitive(self, primitive: pa.DataType) -> Optional[T]: """Visit a primitive type.""" -def _ge

Re: [PR] Automatically create the tables for the `SqlCatalog` [iceberg-python]

Fokko merged PR #186: URL: https://github.com/apache/iceberg-python/pull/186 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Nessie: Support views for NessieCatalog [iceberg]

nastra commented on code in PR #8909: URL: https://github.com/apache/iceberg/pull/8909#discussion_r1419093588 ## nessie/src/test/java/org/apache/iceberg/nessie/TestNessieView.java: ## @@ -0,0 +1,337 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mor

Re: [PR] Nessie: Support views for NessieCatalog [iceberg]

nastra commented on code in PR #8909: URL: https://github.com/apache/iceberg/pull/8909#discussion_r1419090042 ## nessie/src/test/java/org/apache/iceberg/nessie/TestNessieView.java: ## @@ -0,0 +1,337 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mor

Re: [PR] Nessie: Support views for NessieCatalog [iceberg]

nastra commented on code in PR #8909: URL: https://github.com/apache/iceberg/pull/8909#discussion_r1419091171 ## nessie/src/test/java/org/apache/iceberg/nessie/TestNessieView.java: ## @@ -0,0 +1,337 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mor

Re: [PR] Nessie: Support views for NessieCatalog [iceberg]

nastra commented on code in PR #8909: URL: https://github.com/apache/iceberg/pull/8909#discussion_r1419092132 ## nessie/src/test/java/org/apache/iceberg/nessie/TestNessieView.java: ## @@ -0,0 +1,337 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mor

Re: [PR] Nessie: Support views for NessieCatalog [iceberg]

nastra commented on code in PR #8909: URL: https://github.com/apache/iceberg/pull/8909#discussion_r1419095071 ## nessie/src/test/java/org/apache/iceberg/nessie/TestNessieView.java: ## @@ -0,0 +1,337 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mor

Re: [I] Automatically create the tables for the `SqlCatalog` [iceberg-python]

Fokko closed issue #184: Automatically create the tables for the `SqlCatalog` URL: https://github.com/apache/iceberg-python/issues/184 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] Nessie: Support views for NessieCatalog [iceberg]

nastra commented on code in PR #8909: URL: https://github.com/apache/iceberg/pull/8909#discussion_r1419093010 ## nessie/src/test/java/org/apache/iceberg/nessie/TestNessieView.java: ## @@ -0,0 +1,337 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mor

Re: [I] When will the 0.6.0 version be released? [iceberg-python]

Fokko commented on issue #192: URL: https://github.com/apache/iceberg-python/issues/192#issuecomment-1845492573 @1taoze The goal is this year, and I'm quite confident that it is realistic. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] Spark SystemFunctions are not pushed down during JOIN [iceberg]

ConeyLiu commented on code in PR #9233: URL: https://github.com/apache/iceberg/pull/9233#discussion_r1419107868 ## spark/v3.4/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ReplaceStaticInvoke.scala: ## @@ -39,22 +41,31 @@ import org.apache.spark.sql.typ

Re: [PR] maint(transforms): replace `type()` calls with `isinstance()` [iceberg-python]

Fokko merged PR #188: URL: https://github.com/apache/iceberg-python/pull/188 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] maint(transforms): replace `type()` calls with `isinstance()` [iceberg-python]

Fokko commented on PR #188: URL: https://github.com/apache/iceberg-python/pull/188#issuecomment-1845509371 Thanks for fixing this @jayceslesar 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] Nessie: Support views for NessieCatalog [iceberg]

nastra commented on code in PR #8909: URL: https://github.com/apache/iceberg/pull/8909#discussion_r1419122432 ## nessie/src/main/java/org/apache/iceberg/nessie/NessieViewOperations.java: ## @@ -0,0 +1,137 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

Re: [I] The "Status" paragraph in the readme seems very outdated [iceberg]

bitsondatadev commented on issue #9127: URL: https://github.com/apache/iceberg/issues/9127#issuecomment-1845537180 @ronkorving Let me know if you'd like to raise the PR, or I can do it as well. For now pointing to the roadmap page will be the best. Eventually, I'd like there to be something

Re: [PR] Core: Optimize manifest evaluation for super wide tables [iceberg]

irshadcc commented on PR #9147: URL: https://github.com/apache/iceberg/pull/9147#issuecomment-1845560669 > Thanks for raising this @irshadcc, this looks good to me. I've left two small comments, could you take a peek at those? Thanks for fixing this! 🙌 I've added the Javadoc and remov

Re: [PR] Support parameterized tests at class-level with JUnit5 [iceberg]

nastra commented on code in PR #9161: URL: https://github.com/apache/iceberg/pull/9161#discussion_r1419205681 ## api/src/test/java/org/apache/iceberg/ParameterizedTestExtension.java: ## @@ -0,0 +1,245 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

Re: [PR] Core: Iceberg streaming streaming-skip-overwrite-snapshots SparkMicroBatchStream only skips over one file per trigger [iceberg]

cccs-jc commented on PR #8980: URL: https://github.com/apache/iceberg/pull/8980#issuecomment-1845659813 > @cccs-jc i mean let's have changes for 3.5 with it's test only in 3.5 and we can backport the change with it's test in lower spark version like 3.4 and 3.3, 3.4 test failures are expect

Re: [PR] Spark SystemFunctions are not pushed down during JOIN [iceberg]

tmnd1991 commented on code in PR #9233: URL: https://github.com/apache/iceberg/pull/9233#discussion_r1419309041 ## spark/v3.4/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ReplaceStaticInvoke.scala: ## @@ -39,22 +41,31 @@ import org.apache.spark.sql.typ

Re: [PR] Spark: IN clause on system function is not pushed down [iceberg]

tmnd1991 commented on code in PR #9192: URL: https://github.com/apache/iceberg/pull/9192#discussion_r1419314263 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ReplaceStaticInvoke.scala: ## @@ -40,14 +37,20 @@ import org.apache.spark.sql.typ

[PR] Switch to junit5 for mr [iceberg]

lschetanrao opened a new pull request, #9241: URL: https://github.com/apache/iceberg/pull/9241 ### Description This PR fixes https://github.com/apache/iceberg/issues/9083 The goal here is to switch all imports to Junit5 and to use AssertJ-style assertions. ### Implementation

Re: [PR] Spark: IN clause on system function is not pushed down [iceberg]

tmnd1991 commented on PR #9192: URL: https://github.com/apache/iceberg/pull/9192#issuecomment-1845761731 cc @nastra @dramaticlly @advancedxy for review thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Spark SystemFunctions are not pushed down during JOIN [iceberg]

tmnd1991 commented on PR #9233: URL: https://github.com/apache/iceberg/pull/9233#issuecomment-1845762590 cc @nastra @dramaticlly @advancedxy for review thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

  1   2   >