Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2 from 1.21.0 to 1.21.1 [iceberg-go]

2023-10-10 Thread via GitHub
nastra commented on PR #14: URL: https://github.com/apache/iceberg-go/pull/14#issuecomment-1756970884 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.40.0 to 1.40.1 [iceberg-go]

2023-10-10 Thread via GitHub
nastra merged PR #13: URL: https://github.com/apache/iceberg-go/pull/13 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [PR] Infra: Cleanup labeler.yml [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on code in PR #8795: URL: https://github.com/apache/iceberg/pull/8795#discussion_r1354240275 ## .github/labeler.yml: ## @@ -55,16 +55,12 @@ HIVE: - hive3/**/* - hive-metastore/**/* - hive-runtime/**/* + - hive3-orc-bundle/**/* DATA: - data/**

Re: [I] Iceberg does not work with other types of data lakes [iceberg]

2023-10-10 Thread via GitHub
BsoBird commented on issue #8624: URL: https://github.com/apache/iceberg/issues/8624#issuecomment-1756922456 @amogh-jahagirdar Hi. Anything new on this subject? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Infra: Cleanup labeler.yml [iceberg]

2023-10-10 Thread via GitHub
Fokko commented on code in PR #8795: URL: https://github.com/apache/iceberg/pull/8795#discussion_r1354179965 ## .github/labeler.yml: ## @@ -55,16 +55,12 @@ HIVE: - hive3/**/* - hive-metastore/**/* - hive-runtime/**/* + - hive3-orc-bundle/**/* DATA: - data/**/* SPA

Re: [PR] OpenAPI: Add description for AssignUUID [iceberg]

2023-10-10 Thread via GitHub
nastra merged PR #8753: URL: https://github.com/apache/iceberg/pull/8753 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] OpenAPI: Add description for AssignUUID [iceberg]

2023-10-10 Thread via GitHub
nastra commented on code in PR #8753: URL: https://github.com/apache/iceberg/pull/8753#discussion_r1354167117 ## open-api/rest-catalog-open-api.py: ## @@ -230,6 +230,10 @@ class BaseUpdate(BaseModel): class AssignUUIDUpdate(BaseUpdate): +""" +Assigning a UUID to a t

Re: [PR] OpenAPI: Add description for AssignUUID [iceberg]

2023-10-10 Thread via GitHub
nastra commented on PR #8753: URL: https://github.com/apache/iceberg/pull/8753#issuecomment-1756891176 thanks everyone for reviewing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Nessie: Remove deprecated usage of Operation.Put.of() [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on PR #8796: URL: https://github.com/apache/iceberg/pull/8796#issuecomment-1756885657 cc: @dimas-b , @snazy -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[PR] Nessie: Remove deprecated usage of Operation.Put.of() [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat opened a new pull request, #8796: URL: https://github.com/apache/iceberg/pull/8796 Follow up from https://github.com/apache/iceberg/pull/8763#discussion_r1352824568 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[PR] Infra: Cleanup labeler.yml [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat opened a new pull request, #8795: URL: https://github.com/apache/iceberg/pull/8795 Old folders does not exist anymore. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [I] Investigate why dependabot didn't detect upgrades [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on issue #8764: URL: https://github.com/apache/iceberg/issues/8764#issuecomment-1756844390 Awesome. I can see the PR for Arrow now 👍 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[PR] Build: Bump com.azure:azure-sdk-bom from 1.2.16 to 1.2.17 [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] opened a new pull request, #8794: URL: https://github.com/apache/iceberg/pull/8794 Bumps [com.azure:azure-sdk-bom](https://github.com/azure/azure-sdk-for-java) from 1.2.16 to 1.2.17. Release notes Sourced from https://github.com/azure/azure-sdk-for-java/releases";>c

[PR] Build: Bump com.esotericsoftware:kryo from 4.0.2 to 5.5.0 [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] opened a new pull request, #8793: URL: https://github.com/apache/iceberg/pull/8793 Bumps [com.esotericsoftware:kryo](https://github.com/EsotericSoftware/kryo) from 4.0.2 to 5.5.0. Release notes Sourced from https://github.com/EsotericSoftware/kryo/releases";>com.eso

[PR] Build: Bump org.roaringbitmap:RoaringBitmap from 0.9.47 to 1.0.0 [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] opened a new pull request, #8792: URL: https://github.com/apache/iceberg/pull/8792 Bumps [org.roaringbitmap:RoaringBitmap](https://github.com/RoaringBitmap/RoaringBitmap) from 0.9.47 to 1.0.0. Release notes Sourced from https://github.com/RoaringBitmap/RoaringBitma

[PR] Build: Bump jakarta.el:jakarta.el-api from 3.0.3 to 5.0.1 [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] opened a new pull request, #8791: URL: https://github.com/apache/iceberg/pull/8791 Bumps [jakarta.el:jakarta.el-api](https://github.com/eclipse-ee4j/el-ri) from 3.0.3 to 5.0.1. Commits See full diff in https://github.com/eclipse-ee4j/el-ri/commits";>compare view

[PR] Build: Bump net.snowflake:snowflake-jdbc from 3.13.30 to 3.14.2 [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] opened a new pull request, #8790: URL: https://github.com/apache/iceberg/pull/8790 Bumps [net.snowflake:snowflake-jdbc](https://github.com/snowflakedb/snowflake-jdbc) from 3.13.30 to 3.14.2. Release notes Sourced from https://github.com/snowflakedb/snowflake-jdbc/r

[PR] Build: Bump com.fasterxml.jackson.dataformat:jackson-dataformat-xml from 2.9.9 to 2.15.2 [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] opened a new pull request, #8789: URL: https://github.com/apache/iceberg/pull/8789 Bumps [com.fasterxml.jackson.dataformat:jackson-dataformat-xml](https://github.com/FasterXML/jackson-dataformat-xml) from 2.9.9 to 2.15.2. Commits https://github.com/FasterXML/ja

[PR] Build: Bump jetty from 9.4.43.v20210629 to 11.0.17 [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] opened a new pull request, #8788: URL: https://github.com/apache/iceberg/pull/8788 Bumps `jetty` from 9.4.43.v20210629 to 11.0.17. Updates `org.eclipse.jetty:jetty-server` from 9.4.43.v20210629 to 11.0.17 Release notes Sourced from https://github.com/eclipse/jetty

[PR] Build: Bump spring-boot from 2.5.4 to 3.1.4 [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] opened a new pull request, #8787: URL: https://github.com/apache/iceberg/pull/8787 Bumps `spring-boot` from 2.5.4 to 3.1.4. Updates `org.springframework.boot:spring-boot-starter-jetty` from 2.5.4 to 3.1.4 Release notes Sourced from https://github.com/spring-proje

Re: [PR] Build: Bump spring-boot from 2.5.4 to 3.1.3 [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] closed pull request #8471: Build: Bump spring-boot from 2.5.4 to 3.1.3 URL: https://github.com/apache/iceberg/pull/8471 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Build: Bump spring-boot from 2.5.4 to 3.1.3 [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] commented on PR #8471: URL: https://github.com/apache/iceberg/pull/8471#issuecomment-1756835809 Superseded by #8787. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[PR] Build: Bump com.github.ben-manes.caffeine:caffeine from 2.9.3 to 3.1.8 [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] opened a new pull request, #8784: URL: https://github.com/apache/iceberg/pull/8784 Bumps [com.github.ben-manes.caffeine:caffeine](https://github.com/ben-manes/caffeine) from 2.9.3 to 3.1.8. Release notes Sourced from https://github.com/ben-manes/caffeine/releases";

[PR] Build: Bump org.glassfish.jaxb:jaxb-runtime from 2.3.3 to 4.0.3 [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] opened a new pull request, #8782: URL: https://github.com/apache/iceberg/pull/8782 Bumps org.glassfish.jaxb:jaxb-runtime from 2.3.3 to 4.0.3. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=org.g

[PR] Build: Bump com.aliyun.oss:aliyun-sdk-oss from 3.10.2 to 3.17.1 [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] opened a new pull request, #8781: URL: https://github.com/apache/iceberg/pull/8781 Bumps com.aliyun.oss:aliyun-sdk-oss from 3.10.2 to 3.17.1. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=com.a

[PR] Build: Bump io.delta:delta-core_2.12 from 2.2.0 to 2.4.0 [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] opened a new pull request, #8786: URL: https://github.com/apache/iceberg/pull/8786 Bumps [io.delta:delta-core_2.12](https://github.com/delta-io/delta) from 2.2.0 to 2.4.0. Release notes Sourced from https://github.com/delta-io/delta/releases";>io.delta:delta-core_2.

[PR] Build: Bump arrow from 12.0.1 to 13.0.0 [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] opened a new pull request, #8785: URL: https://github.com/apache/iceberg/pull/8785 Bumps `arrow` from 12.0.1 to 13.0.0. Updates `org.apache.arrow:arrow-memory-netty` from 12.0.1 to 13.0.0 Updates `org.apache.arrow:arrow-vector` from 12.0.1 to 13.0.0 Commits

[PR] Build: Bump io.netty:netty-buffer from 4.1.68.Final to 4.1.100.Final [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] opened a new pull request, #8783: URL: https://github.com/apache/iceberg/pull/8783 Bumps [io.netty:netty-buffer](https://github.com/netty/netty) from 4.1.68.Final to 4.1.100.Final. Commits https://github.com/netty/netty/commit/58df783eb4fc50f95a1061dc4274020d680

[PR] Build: Bump org.testcontainers:testcontainers from 1.17.6 to 1.19.1 [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] opened a new pull request, #8780: URL: https://github.com/apache/iceberg/pull/8780 Bumps [org.testcontainers:testcontainers](https://github.com/testcontainers/testcontainers-java) from 1.17.6 to 1.19.1. Release notes Sourced from https://github.com/testcontainers/t

[PR] Build: Bump com.adobe.testing:s3mock-junit5 from 2.11.0 to 3.1.0 [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] opened a new pull request, #8779: URL: https://github.com/apache/iceberg/pull/8779 Bumps com.adobe.testing:s3mock-junit5 from 2.11.0 to 3.1.0. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=com.

[PR] Build: Bump com.palantir.baseline:gradle-baseline-java from 4.42.0 to 5.22.0 [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] opened a new pull request, #8778: URL: https://github.com/apache/iceberg/pull/8778 Bumps [com.palantir.baseline:gradle-baseline-java](https://github.com/palantir/gradle-baseline) from 4.42.0 to 5.22.0. Release notes Sourced from https://github.com/palantir/gradle-b

[PR] Build: Bump guava from 32.1.1-jre to 32.1.3-jre [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] opened a new pull request, #8777: URL: https://github.com/apache/iceberg/pull/8777 Bumps `guava` from 32.1.1-jre to 32.1.3-jre. Updates `com.google.guava:guava` from 32.1.1-jre to 32.1.3-jre Release notes Sourced from https://github.com/google/guava/releases";>com

[PR] Build: Bump com.google.errorprone:error_prone_annotations from 2.3.3 to 2.22.0 [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] opened a new pull request, #8776: URL: https://github.com/apache/iceberg/pull/8776 Bumps [com.google.errorprone:error_prone_annotations](https://github.com/google/error-prone) from 2.3.3 to 2.22.0. Release notes Sourced from https://github.com/google/error-prone/re

[PR] Build: Bump org.xerial:sqlite-jdbc from 3.42.0.0 to 3.43.0.0 [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] opened a new pull request, #8775: URL: https://github.com/apache/iceberg/pull/8775 Bumps [org.xerial:sqlite-jdbc](https://github.com/xerial/sqlite-jdbc) from 3.42.0.0 to 3.43.0.0. Release notes Sourced from https://github.com/xerial/sqlite-jdbc/releases";>org.xerial

[PR] Build: Bump org.apache.pig:pig from 0.14.0 to 0.17.0 [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] opened a new pull request, #8774: URL: https://github.com/apache/iceberg/pull/8774 Bumps org.apache.pig:pig from 0.14.0 to 0.17.0. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=org.apache.pig:p

[PR] Build: Bump software.amazon.awssdk:bom from 2.20.131 to 2.20.162 [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] opened a new pull request, #8773: URL: https://github.com/apache/iceberg/pull/8773 Bumps software.amazon.awssdk:bom from 2.20.131 to 2.20.162. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=soft

[PR] Build: Bump antlr from 4.9.3 to 4.13.1 [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] opened a new pull request, #8772: URL: https://github.com/apache/iceberg/pull/8772 Bumps `antlr` from 4.9.3 to 4.13.1. Updates `org.antlr:antlr4` from 4.9.3 to 4.13.1 Release notes Sourced from https://github.com/antlr/antlr4/releases";>org.antlr:antlr4's release

[PR] Build: Bump nessie from 0.71.0 to 0.71.1 [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] opened a new pull request, #8771: URL: https://github.com/apache/iceberg/pull/8771 Bumps `nessie` from 0.71.0 to 0.71.1. Updates `org.projectnessie.nessie:nessie-client` from 0.71.0 to 0.71.1 Updates `org.projectnessie.nessie:nessie-jaxrs-testextension` from 0.71.0 t

[PR] Build: Bump hadoop2 from 2.7.3 to 3.3.6 [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] opened a new pull request, #8770: URL: https://github.com/apache/iceberg/pull/8770 Bumps `hadoop2` from 2.7.3 to 3.3.6. Updates `org.apache.hadoop:hadoop-client` from 2.7.3 to 3.3.6 Updates `org.apache.hadoop:hadoop-common` from 2.7.3 to 3.3.6 Updates `org.apa

[PR] Build: Bump org.apache.avro:avro from 1.11.1 to 1.11.3 [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] opened a new pull request, #8769: URL: https://github.com/apache/iceberg/pull/8769 Bumps org.apache.avro:avro from 1.11.1 to 1.11.3. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=org.apache.avr

Re: [I] Investigate why dependabot didn't detect upgrades [iceberg]

2023-10-10 Thread via GitHub
Fokko closed issue #8764: Investigate why dependabot didn't detect upgrades URL: https://github.com/apache/iceberg/issues/8764 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] Build: increase open-pull-requests-limit to 50 [iceberg]

2023-10-10 Thread via GitHub
Fokko merged PR #8768: URL: https://github.com/apache/iceberg/pull/8768 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [PR] [WIP] API, Core, Spark 3.5: Parallelize reading of deletes and cache them on executors [iceberg]

2023-10-10 Thread via GitHub
aokolnychyi commented on code in PR #8755: URL: https://github.com/apache/iceberg/pull/8755#discussion_r1353979495 ## core/src/main/java/org/apache/iceberg/SystemConfigs.java: ## @@ -42,6 +42,13 @@ private SystemConfigs() {} Math.max(2, Runtime.getRuntime().availableP

Re: [PR] push down min/max/count to iceberg [iceberg]

2023-10-10 Thread via GitHub
huaxingao commented on PR #6252: URL: https://github.com/apache/iceberg/pull/6252#issuecomment-1756724970 If you have filters on the aggregated columns, e.g. SELECT MAX(col) FROM table WHERE col > 1 && col < 10, then push down is not supported. On Tue, Oct 10, 2023 at 9:52 AM Atif *

Re: [PR] feat: manifest list writer [iceberg-rust]

2023-10-10 Thread via GitHub
barronw commented on code in PR #76: URL: https://github.com/apache/iceberg-rust/pull/76#discussion_r1353723342 ## crates/iceberg/src/spec/manifest_list.rs: ## @@ -69,6 +73,25 @@ impl ManifestList { &self.entries } +/// Get the v1 schema of the manifest list

[PR] feat: manifest list writer [iceberg-rust]

2023-10-10 Thread via GitHub
barronw opened a new pull request, #76: URL: https://github.com/apache/iceberg-rust/pull/76 This adds a manifest list writer based on the discussion in #72. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [I] Flink: filter expression should be a single expression (not a list) [iceberg]

2023-10-10 Thread via GitHub
github-actions[bot] commented on issue #7335: URL: https://github.com/apache/iceberg/issues/7335#issuecomment-1756489166 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Improve error reporting when streaming snapshot ID is no longer available [iceberg]

2023-10-10 Thread via GitHub
github-actions[bot] commented on issue #7340: URL: https://github.com/apache/iceberg/issues/7340#issuecomment-1756489143 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] [WIP] API, Core, Spark 3.5: Parallelize reading of deletes and cache them on executors [iceberg]

2023-10-10 Thread via GitHub
parthchandra commented on code in PR #8755: URL: https://github.com/apache/iceberg/pull/8755#discussion_r1353486486 ## core/src/main/java/org/apache/iceberg/SystemConfigs.java: ## @@ -42,6 +42,13 @@ private SystemConfigs() {} Math.max(2, Runtime.getRuntime().available

Re: [PR] Docs: Fix missing semicolons in SQL snippets. [iceberg]

2023-10-10 Thread via GitHub
Priyansh121096 commented on PR #8748: URL: https://github.com/apache/iceberg/pull/8748#issuecomment-1756235225 Thanks @Fokko! This was my first contribution to apache/iceberg. Hope to make many more in the future. -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] Fix Iceberg to Avro Schema Conversion: Fixed, Decimal, UUID [iceberg-python]

2023-10-10 Thread via GitHub
rdblue commented on code in PR #53: URL: https://github.com/apache/iceberg-python/pull/53#discussion_r1353376710 ## pyiceberg/utils/schema_conversion.py: ## @@ -565,10 +566,17 @@ def map(self, map_type: MapType, key_result: AvroType, value_result: AvroType) - }

Re: [PR] Docs: Fix missing semicolons in SQL snippets. [iceberg]

2023-10-10 Thread via GitHub
Priyansh121096 commented on code in PR #8748: URL: https://github.com/apache/iceberg/pull/8748#discussion_r1353371980 ## docs/spark-getting-started.md: ## @@ -69,7 +69,7 @@ To create your first Iceberg table in Spark, use the `spark-sql` shell or `spark ```sql -- local is t

Re: [PR] Construct a writer tree [iceberg-python]

2023-10-10 Thread via GitHub
rdblue commented on PR #40: URL: https://github.com/apache/iceberg-python/pull/40#issuecomment-1756223971 I think overall this is ready. There's one bug where a missing record field produces a double `OptionWriter`, but it wouldn't actually result in bad data because the outer option will a

Re: [PR] Construct a writer tree [iceberg-python]

2023-10-10 Thread via GitHub
rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1353347354 ## tests/avro/test_resolver.py: ## @@ -293,11 +306,103 @@ def test_resolver_initial_value() -> None: schema_id=2, ) -assert resolve(write_schema,

Re: [PR] Construct a writer tree [iceberg-python]

2023-10-10 Thread via GitHub
rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1353340562 ## pyiceberg/avro/resolver.py: ## @@ -192,7 +194,28 @@ def visit_binary(self, binary_type: BinaryType) -> Writer: return BinaryWriter() -def resolve( +CO

Re: [PR] Construct a writer tree [iceberg-python]

2023-10-10 Thread via GitHub
rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r135477 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +255,107 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVi

Re: [PR] Construct a writer tree [iceberg-python]

2023-10-10 Thread via GitHub
rdblue commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1353328176 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +256,95 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVis

Re: [PR] Data: Support reading default values from generic Avro readers [iceberg]

2023-10-10 Thread via GitHub
wmoustafa commented on PR #6004: URL: https://github.com/apache/iceberg/pull/6004#issuecomment-1756154580 > @wmoustafa, looks like there are test failures. Can you take a look? Fixed. -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] Data: Support reading default values from generic Avro readers [iceberg]

2023-10-10 Thread via GitHub
wmoustafa commented on code in PR #6004: URL: https://github.com/apache/iceberg/pull/6004#discussion_r1353272532 ## .palantir/revapi.yml: ## @@ -451,6 +451,15 @@ acceptedBreaks: - code: "java.field.removedWithConstant" old: "field org.apache.iceberg.TableProperties.H

Re: [PR] Avro: Add Avro-assisted name mapping [iceberg]

2023-10-10 Thread via GitHub
wmoustafa commented on PR #7392: URL: https://github.com/apache/iceberg/pull/7392#issuecomment-1756153921 > I think this is ready. Just a few minor updates needed; mostly https://github.com/apache/iceberg/pull/7392/files#r1224853756. Addressed. -- This is an automated message from

Re: [PR] Avro: Add Avro-assisted name mapping [iceberg]

2023-10-10 Thread via GitHub
wmoustafa commented on code in PR #7392: URL: https://github.com/apache/iceberg/pull/7392#discussion_r1353271608 ## core/src/main/java/org/apache/iceberg/avro/AvroWithPartnerByStructureVisitor.java: ## @@ -93,14 +94,23 @@ private static T visitRecord( private static T visit

[PR] Fix column rename doc example to reflect correct API [iceberg-python]

2023-10-10 Thread via GitHub
cabhishek opened a new pull request, #59: URL: https://github.com/apache/iceberg-python/pull/59 * Rename column example in [this](https://py.iceberg.apache.org/api/#rename-column) doc is incorrect. * This PR updates the example to use `update.rename_column(...)` instead of `update.renam

Re: [PR] Spec: Add partition stats spec [iceberg]

2023-10-10 Thread via GitHub
aokolnychyi commented on PR #7105: URL: https://github.com/apache/iceberg/pull/7105#issuecomment-1756079392 I added this PR to our community sync. I am not sure I will be there this week but I'll sync with Russell and Yufei afterwards. -- This is an automated message from the Apache Git S

Re: [PR] Spec: Add partition stats spec [iceberg]

2023-10-10 Thread via GitHub
aokolnychyi commented on code in PR #7105: URL: https://github.com/apache/iceberg/pull/7105#discussion_r1353203145 ## format/spec.md: ## @@ -702,6 +703,58 @@ Blob metadata is a struct with the following fields: | _optional_ | _optional_ | **`properties`** | `map` | Additional

Re: [PR] push down min/max/count to iceberg [iceberg]

2023-10-10 Thread via GitHub
atifiu commented on PR #6252: URL: https://github.com/apache/iceberg/pull/6252#issuecomment-1755857764 @huaxingao I was executing max/count query on iceberg table version 1.3.0 and Spark3.3.1 but unable to see aggregate pushdown i.e. LocalTableScan Cc: @RussellSpitzer `spark.s

Re: [I] Unable to write to iceberg table using spark [iceberg]

2023-10-10 Thread via GitHub
di2mot commented on issue #8419: URL: https://github.com/apache/iceberg/issues/8419#issuecomment-1755796639 This works for me in general it works: ``` ("spark.jars.packages", "org.apache.iceberg:iceberg-spark3:0.11.0"), ("spark.sql.extensions", "org.apache.iceberg.spark.extensions.I

Re: [PR] Disable merge-commit and enforce linear history [iceberg-python]

2023-10-10 Thread via GitHub
rdblue merged PR #57: URL: https://github.com/apache/iceberg-python/pull/57 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] Construct a writer tree [iceberg-python]

2023-10-10 Thread via GitHub
Fokko commented on PR #40: URL: https://github.com/apache/iceberg-python/pull/40#issuecomment-1755767706 Forgot to push, just pushed the latest changes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [I] Document all metadata tables [iceberg]

2023-10-10 Thread via GitHub
nastra closed issue #757: Document all metadata tables URL: https://github.com/apache/iceberg/issues/757 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Docs: Document all metadata tables. [iceberg]

2023-10-10 Thread via GitHub
nastra merged PR #8709: URL: https://github.com/apache/iceberg/pull/8709 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Build: increase open-pull-requests-limit to 50 [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on PR #8768: URL: https://github.com/apache/iceberg/pull/8768#issuecomment-1755762122 > @ajantha-bhat maybe we have 5 pending PRs not closed/merged, so blocking any new PR. Yeah, anyways this change will definitely give a clarity if that was the problem. So

Re: [PR] Build: increase open-pull-requests-limit to 50 [iceberg]

2023-10-10 Thread via GitHub
jbonofre commented on PR #8768: URL: https://github.com/apache/iceberg/pull/8768#issuecomment-1755758543 @ajantha-bhat maybe we have 5 pending PRs not closed/merged, so blocking any new PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [I] Upsert support for keyless Apache Flink tables [iceberg]

2023-10-10 Thread via GitHub
Ge commented on issue #8719: URL: https://github.com/apache/iceberg/issues/8719#issuecomment-1755752149 `SELECT word, COUNT(*) FROM word_table GROUP BY word;` is the retract stream: ``` Flink SQL> SELECT word, COUNT(*) FROM word_table GROUP BY word; ++

Re: [PR] Build: Fix compiler warnings [iceberg]

2023-10-10 Thread via GitHub
dimas-b commented on code in PR #8763: URL: https://github.com/apache/iceberg/pull/8763#discussion_r1352835064 ## nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java: ## @@ -477,7 +477,7 @@ public void commitTable( Branch branch = getApi()

Re: [PR] Build: Upgrade to gradle 8.4 [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on PR #8486: URL: https://github.com/apache/iceberg/pull/8486#issuecomment-1755743335 > Unfortunately gradle-revapi-plugin doesn't seem super active (https://github.com/palantir/gradle-revapi). I think it's important to be up to date in regards of Gradle. I will pro

Re: [PR] Build: Fix compiler warnings [iceberg]

2023-10-10 Thread via GitHub
dimas-b commented on code in PR #8763: URL: https://github.com/apache/iceberg/pull/8763#discussion_r1352831841 ## nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java: ## @@ -477,7 +477,7 @@ public void commitTable( Branch branch = getApi()

Re: [PR] Build: Fix compiler warnings [iceberg]

2023-10-10 Thread via GitHub
dimas-b commented on code in PR #8763: URL: https://github.com/apache/iceberg/pull/8763#discussion_r1352831841 ## nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java: ## @@ -477,7 +477,7 @@ public void commitTable( Branch branch = getApi()

Re: [PR] Build: Fix compiler warnings [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on code in PR #8763: URL: https://github.com/apache/iceberg/pull/8763#discussion_r1352827194 ## nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java: ## @@ -477,7 +477,7 @@ public void commitTable( Branch branch = getApi()

Re: [PR] Build: Fix compiler warnings [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on code in PR #8763: URL: https://github.com/apache/iceberg/pull/8763#discussion_r1352827194 ## nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java: ## @@ -477,7 +477,7 @@ public void commitTable( Branch branch = getApi()

Re: [PR] Build: Fix compiler warnings [iceberg]

2023-10-10 Thread via GitHub
dimas-b commented on code in PR #8763: URL: https://github.com/apache/iceberg/pull/8763#discussion_r1352824568 ## nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java: ## @@ -477,7 +477,7 @@ public void commitTable( Branch branch = getApi()

Re: [PR] Build: Fix compiler warnings [iceberg]

2023-10-10 Thread via GitHub
dimas-b commented on code in PR #8763: URL: https://github.com/apache/iceberg/pull/8763#discussion_r1352823020 ## nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java: ## @@ -477,7 +477,7 @@ public void commitTable( Branch branch = getApi()

Re: [PR] Build: Upgrade to gradle 8.4 [iceberg]

2023-10-10 Thread via GitHub
jbonofre commented on PR #8486: URL: https://github.com/apache/iceberg/pull/8486#issuecomment-1755722580 Unfortunately `gradle-revapi-plugin` doesn't seem super active (https://github.com/palantir/gradle-revapi). I think it's important to be up to date in regards of Gradle. I will pro

Re: [PR] Build: Bump slf4j from 1.7.36 to 2.0.9 [iceberg]

2023-10-10 Thread via GitHub
nastra commented on PR #8737: URL: https://github.com/apache/iceberg/pull/8737#issuecomment-1755713705 https://github.com/apache/iceberg/actions/runs/6455472887/job/17523030796?pr=8737 contains a CI run with failures -- This is an automated message from the Apache Git Service. To respond

Re: [PR] Build: increase open-pull-requests-limit to 50 [iceberg]

2023-10-10 Thread via GitHub
jbonofre commented on code in PR #8768: URL: https://github.com/apache/iceberg/pull/8768#discussion_r1352811489 ## .github/dependabot.yml: ## @@ -28,6 +28,6 @@ updates: directory: "/" schedule: interval: "weekly" - day: "sunday" -open-pull-requests-limi

Re: [PR] Build: increase open-pull-requests-limit to 50 [iceberg]

2023-10-10 Thread via GitHub
Fokko commented on code in PR #8768: URL: https://github.com/apache/iceberg/pull/8768#discussion_r1352807655 ## .github/dependabot.yml: ## @@ -28,6 +28,6 @@ updates: directory: "/" schedule: interval: "weekly" - day: "sunday" -open-pull-requests-limit:

Re: [PR] Build: Add note about running tests/itests on MacOS [iceberg]

2023-10-10 Thread via GitHub
jbonofre commented on PR #8766: URL: https://github.com/apache/iceberg/pull/8766#issuecomment-1755703566 @nastra actually it's the workaround I have to do on my Mac :) I'm using MacOS (tested both on 13 & 14 with Docker Desktop) on M1. -- This is an automated message from the Apache Git S

Re: [PR] Build: Add note about running tests/itests on MacOS [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on PR #8766: URL: https://github.com/apache/iceberg/pull/8766#issuecomment-1755702793 > LGTM, but would be great if somebody with OSX could confirm this I can confirm. I use MAC. Not just for Iceberg project, any project that uses `TestContainers` on MAC will th

Re: [PR] Build: Bump slf4j from 1.7.36 to 2.0.9 [iceberg]

2023-10-10 Thread via GitHub
jbonofre commented on PR #8737: URL: https://github.com/apache/iceberg/pull/8737#issuecomment-1755700578 @nastra on a specific Spark version or any ? I can take a look if you want :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

Re: [PR] Build: Fix compiler warnings [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on code in PR #8763: URL: https://github.com/apache/iceberg/pull/8763#discussion_r1352798678 ## nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java: ## @@ -477,7 +477,7 @@ public void commitTable( Branch branch = getApi()

Re: [PR] Build: Fix compiler warnings [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on code in PR #8763: URL: https://github.com/apache/iceberg/pull/8763#discussion_r1352796153 ## nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java: ## @@ -477,7 +477,7 @@ public void commitTable( Branch branch = getApi()

[I] Parquet.write to S3 with GlueCatalog requires commit [iceberg]

2023-10-10 Thread via GitHub
djchapm opened a new issue, #8767: URL: https://github.com/apache/iceberg/issues/8767 ### Feature Request / Improvement Hi, writing this in an effort to improve documentation - I spent a crazy amount of time writing to glue catalog and parquet-avro files in S3 with Iceberg, but could

Re: [PR] Build: Bump slf4j from 1.7.36 to 2.0.9 [iceberg]

2023-10-10 Thread via GitHub
nastra commented on PR #8737: URL: https://github.com/apache/iceberg/pull/8737#issuecomment-1755681588 @jbonofre there's an issue with Spark that needs some investigation -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Core: Use more permissive check when registering existing table [iceberg]

2023-10-10 Thread via GitHub
Fokko merged PR #8759: URL: https://github.com/apache/iceberg/pull/8759 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [PR] Docs: Fix missing semicolons in SQL snippets. [iceberg]

2023-10-10 Thread via GitHub
Fokko merged PR #8748: URL: https://github.com/apache/iceberg/pull/8748 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [PR] Docs: Fix missing semicolons in SQL snippets. [iceberg]

2023-10-10 Thread via GitHub
Fokko commented on PR #8748: URL: https://github.com/apache/iceberg/pull/8748#issuecomment-1755637911 Great work @Priyansh121096 If we find more we can create a new PR. (I also noticed that some blocks start with: ``` ```SQL ``` For consistency it would be nice to have everything

Re: [I] Unable to write to iceberg table using spark [iceberg]

2023-10-10 Thread via GitHub
RussellSpitzer commented on issue #8419: URL: https://github.com/apache/iceberg/issues/8419#issuecomment-1755633933 Pyspark I think has some issues with setting "packages" in the Spark conf since the py4j execution means that the Spark Context has to be started a bit weirdly. I would try us

Re: [PR] Docs: Fix missing semicolons in SQL snippets. [iceberg]

2023-10-10 Thread via GitHub
Fokko commented on code in PR #8748: URL: https://github.com/apache/iceberg/pull/8748#discussion_r1352755607 ## docs/spark-getting-started.md: ## @@ -69,7 +69,7 @@ To create your first Iceberg table in Spark, use the `spark-sql` shell or `spark ```sql -- local is the path-b

Re: [I] Investigate why dependabot didn't detect upgrades [iceberg]

2023-10-10 Thread via GitHub
jbonofre commented on issue #8764: URL: https://github.com/apache/iceberg/issues/8764#issuecomment-1755627197 @Fokko yes, I think we use the 5 PRs pool. +1 to upgrade. I'm doing it in a PR attached to this issue. -- This is an automated message from the Apache Git Service. To respond to t

[PR] Open-API: Make error required [iceberg]

2023-10-10 Thread via GitHub
Fokko opened a new pull request, #8765: URL: https://github.com/apache/iceberg/pull/8765 I think we want to make `error` required, otherwise it would just be an empty document `{}`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

Re: [PR] Core: Allow missing object in ErrorResponse [iceberg]

2023-10-10 Thread via GitHub
Fokko closed pull request #8760: Core: Allow missing object in ErrorResponse URL: https://github.com/apache/iceberg/pull/8760 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

  1   2   >