Re: [PR] refactor: some more small refactors [iceberg-go]

2024-08-20 Thread via GitHub
nastra merged PR #130: URL: https://github.com/apache/iceberg-go/pull/130 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2 from 1.27.0 to 1.30.4 [iceberg-go]

2024-08-20 Thread via GitHub
nastra merged PR #128: URL: https://github.com/apache/iceberg-go/pull/128 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.17.16 to 1.17.28 [iceberg-go]

2024-08-20 Thread via GitHub
nastra commented on PR #129: URL: https://github.com/apache/iceberg-go/pull/129#issuecomment-2298122330 @zeroshade you should be able to tell dependabot to rebase the PR as well via `@dependabot rebase` -- This is an automated message from the Apache Git Service. To respond to the message

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.17.16 to 1.17.28 [iceberg-go]

2024-08-20 Thread via GitHub
nastra commented on PR #129: URL: https://github.com/apache/iceberg-go/pull/129#issuecomment-2298122483 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] Data: Add a util to read write partition stats [iceberg]

2024-08-20 Thread via GitHub
ajantha-bhat commented on code in PR #10176: URL: https://github.com/apache/iceberg/pull/10176#discussion_r1722827850 ## data/src/jmh/java/org/apache/iceberg/PartitionStatsGeneratorBenchmark.java: ## @@ -0,0 +1,131 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

Re: [PR] Data: Add a util to read write partition stats [iceberg]

2024-08-20 Thread via GitHub
ajantha-bhat commented on code in PR #10176: URL: https://github.com/apache/iceberg/pull/10176#discussion_r1722735848 ## data/src/jmh/java/org/apache/iceberg/PartitionStatsGeneratorBenchmark.java: ## @@ -0,0 +1,131 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

Re: [PR] AWS: Don't complete upload on finalize for S3OutputStream [iceberg]

2024-08-20 Thread via GitHub
jasonf20 commented on PR #10874: URL: https://github.com/apache/iceberg/pull/10874#issuecomment-2298237237 @amogh-jahagirdar any way to move this forward? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-20 Thread via GitHub
Fokko commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1722881230 ## pyiceberg/io/pyarrow.py: ## @@ -1304,6 +1305,195 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_per_fi

Re: [I] [Bug] Cannot use PyIceberg with multiple FS [iceberg-python]

2024-08-20 Thread via GitHub
Fokko commented on issue #1041: URL: https://github.com/apache/iceberg-python/issues/1041#issuecomment-2298248947 This is a good point, I've heard that folks store their metadata on HDFS, and the data itself on S3. I don't think the example with the add-files is the best, it would be

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-20 Thread via GitHub
Fokko merged PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Bump mypy-boto3-glue from 1.34.160 to 1.35.0 [iceberg-python]

2024-08-20 Thread via GitHub
Fokko merged PR #1070: URL: https://github.com/apache/iceberg-python/pull/1070 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Data: Add a util to read write partition stats [iceberg]

2024-08-20 Thread via GitHub
ajantha-bhat commented on code in PR #10176: URL: https://github.com/apache/iceberg/pull/10176#discussion_r1722931839 ## core/src/main/java/org/apache/iceberg/PartitionStatsUtil.java: ## @@ -0,0 +1,218 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

[PR] Infra, Docs: Publish Apache Iceberg 1.6.1 release [iceberg]

2024-08-20 Thread via GitHub
findepi opened a new pull request, #10966: URL: https://github.com/apache/iceberg/pull/10966 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-

Re: [PR] add aliyun bundle jar [iceberg]

2024-08-20 Thread via GitHub
FANNG1 commented on PR #10971: URL: https://github.com/apache/iceberg/pull/10971#issuecomment-2298487583 Help needed about `LICENCE` and `NOTICE` files, should I add all dependency licenses to the LICECE file? -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Build: Prevent implicit default locale/charset usage [iceberg]

2024-08-20 Thread via GitHub
Fokko commented on PR #10969: URL: https://github.com/apache/iceberg/pull/10969#issuecomment-2298665879 Looks good, thanks for fixing this @findepi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] Build: Prevent implicit default locale/charset usage [iceberg]

2024-08-20 Thread via GitHub
Fokko merged PR #10969: URL: https://github.com/apache/iceberg/pull/10969 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] DOCS: Exclude signature files from twine upload [iceberg-python]

2024-08-20 Thread via GitHub
Fokko merged PR #1071: URL: https://github.com/apache/iceberg-python/pull/1071 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Spark Action to Analyze table [iceberg]

2024-08-20 Thread via GitHub
guykhazma commented on code in PR #10288: URL: https://github.com/apache/iceberg/pull/10288#discussion_r1723214648 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/AnalyzeTableSparkAction.java: ## @@ -0,0 +1,150 @@ +/* + * Licensed to the Apache Software Founda

Re: [I] [feat] Unify implementation of `to_arrow` and `to_arrow_batch_reader` [iceberg-python]

2024-08-20 Thread via GitHub
sungwy closed issue #1039: [feat] Unify implementation of `to_arrow` and `to_arrow_batch_reader` URL: https://github.com/apache/iceberg-python/issues/1039 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [I] [feat] Unify implementation of `to_arrow` and `to_arrow_batch_reader` [iceberg-python]

2024-08-20 Thread via GitHub
sungwy commented on issue #1039: URL: https://github.com/apache/iceberg-python/issues/1039#issuecomment-2298797090 closed through: https://github.com/apache/iceberg-python/pull/1043 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

Re: [PR] Core: Add ManifestWrite benchmark [iceberg]

2024-08-20 Thread via GitHub
Fokko merged PR #8637: URL: https://github.com/apache/iceberg/pull/8637 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

[PR] Core: Add list/map block sizes [iceberg]

2024-08-20 Thread via GitHub
Fokko opened a new pull request, #10973: URL: https://github.com/apache/iceberg/pull/10973 This PR switches the encoder to the `BlockingDirectBinaryEncoder` that actually encodes the size of list and map objects in the Avro file. When doing projected reads, it can skip of the fields t

Re: [PR] Core/RewriteFiles: Fixed dropping delete files that are still required [iceberg]

2024-08-20 Thread via GitHub
findepi commented on PR #10962: URL: https://github.com/apache/iceberg/pull/10962#issuecomment-2298921650 Flink jobs all failed with timeout after 6h (https://github.com/apache/iceberg/actions/runs/10442521766/job/28914906512?pr=10962) I did restart them. -- This is an automated messag

Re: [PR] Table Scan: Add Row Group Skipping [iceberg-rust]

2024-08-20 Thread via GitHub
liurenjie1024 commented on code in PR #558: URL: https://github.com/apache/iceberg-rust/pull/558#discussion_r1721853381 ## crates/iceberg/src/expr/visitors/row_group_metrics_evaluator.rs: ## @@ -0,0 +1,523 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.17.16 to 1.17.28 [iceberg-go]

2024-08-20 Thread via GitHub
nastra merged PR #129: URL: https://github.com/apache/iceberg-go/pull/129 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Spark 3.5: Fix incorrect catalog loaded in TestCreateActions [iceberg]

2024-08-20 Thread via GitHub
nastra commented on code in PR #10952: URL: https://github.com/apache/iceberg/pull/10952#discussion_r1723442916 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/actions/TestCreateActions.java: ## @@ -186,6 +186,8 @@ public void before() { public void after() throws

Re: [PR] AWS: Don't complete multipart upload on finalize for S3OutputStream [iceberg]

2024-08-20 Thread via GitHub
amogh-jahagirdar commented on PR #10874: URL: https://github.com/apache/iceberg/pull/10874#issuecomment-2299127203 @jasonf20 Thanks for your patience I think we can move forward as is. @Fokko Thank you for the review! -- This is an automated message from the Apache Git Service. To respon

Re: [PR] AWS: Don't complete multipart upload on finalize for S3OutputStream [iceberg]

2024-08-20 Thread via GitHub
amogh-jahagirdar merged PR #10874: URL: https://github.com/apache/iceberg/pull/10874 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.17.16 to 1.17.28 [iceberg-go]

2024-08-20 Thread via GitHub
zeroshade commented on PR #129: URL: https://github.com/apache/iceberg-go/pull/129#issuecomment-2299147836 @nastra I tried that earlier, I'm not able to tell dependabot to rebase because I don't have push permission on here (https://github.com/apache/iceberg-go/pull/127#issuecomment-2296739

Re: [PR] Data: Add a util to read write partition stats [iceberg]

2024-08-20 Thread via GitHub
RussellSpitzer commented on code in PR #10176: URL: https://github.com/apache/iceberg/pull/10176#discussion_r1723533548 ## data/src/jmh/java/org/apache/iceberg/PartitionStatsGeneratorBenchmark.java: ## @@ -0,0 +1,131 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

Re: [PR] Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction [iceberg]

2024-08-20 Thread via GitHub
rodmeneses commented on code in PR #10179: URL: https://github.com/apache/iceberg/pull/10179#discussion_r1723543103 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -0,0 +1,740 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

Re: [PR] Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction [iceberg]

2024-08-20 Thread via GitHub
rodmeneses commented on code in PR #10179: URL: https://github.com/apache/iceberg/pull/10179#discussion_r1723543103 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -0,0 +1,740 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

Re: [PR] Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction [iceberg]

2024-08-20 Thread via GitHub
rodmeneses commented on code in PR #10179: URL: https://github.com/apache/iceberg/pull/10179#discussion_r1723543103 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -0,0 +1,740 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

Re: [PR] Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction [iceberg]

2024-08-20 Thread via GitHub
rodmeneses commented on code in PR #10179: URL: https://github.com/apache/iceberg/pull/10179#discussion_r1723555383 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -0,0 +1,740 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

[I] Support Merge-on-Read mode for Deletes [iceberg-python]

2024-08-20 Thread via GitHub
sungwy opened a new issue, #1078: URL: https://github.com/apache/iceberg-python/issues/1078 ### Feature Request / Improvement Similar to Spark, we would like to implement Merge-on-Read mode of deleting, by creating delete files that encode position and equality delete markers that ca

[PR] Drop ParallelIterable's queue low water mark [iceberg]

2024-08-20 Thread via GitHub
findepi opened a new pull request, #10978: URL: https://github.com/apache/iceberg/pull/10978 As part of the change in commit 7831a8dfc3a2de546ca069f4fc1e7afd03777554, queue low water mark was introduced. However, it resulted in increased number of manifests being read when planning LIMIT

[PR] [1.6] Core: Drop ParallelIterable's queue low water mark [iceberg]

2024-08-20 Thread via GitHub
findepi opened a new pull request, #10979: URL: https://github.com/apache/iceberg/pull/10979 As part of the change in commit 7831a8dfc3a2de546ca069f4fc1e7afd03777554, queue low water mark was introduced. However, it resulted in increased number of manifests being read when planning LIMIT

Re: [PR] Build: Enable UnusedMethod error-prone check [iceberg]

2024-08-20 Thread via GitHub
findepi merged PR #10968: URL: https://github.com/apache/iceberg/pull/10968 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [I] Upgrade to Gradle 8.10 [iceberg]

2024-08-20 Thread via GitHub
amogh-jahagirdar closed issue #10975: Upgrade to Gradle 8.10 URL: https://github.com/apache/iceberg/issues/10975 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] #10668 - Support case-insensitivity for column names in PartitionSpec [iceberg]

2024-08-20 Thread via GitHub
rdblue commented on code in PR #10678: URL: https://github.com/apache/iceberg/pull/10678#discussion_r1723721641 ## core/src/test/java/org/apache/iceberg/TestPartitionSpecBuilderCaseSensitivity.java: ## @@ -0,0 +1,993 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

Re: [PR] Upgrade to Gradle 8.10 [iceberg]

2024-08-20 Thread via GitHub
amogh-jahagirdar merged PR #10976: URL: https://github.com/apache/iceberg/pull/10976 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] #10668 - Support case-insensitivity for column names in PartitionSpec [iceberg]

2024-08-20 Thread via GitHub
rdblue commented on PR #10678: URL: https://github.com/apache/iceberg/pull/10678#issuecomment-2299429899 Looks good and I don't see any other open comments so I'll merge. Thanks, @sl255051! -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] #10668 - Support case-insensitivity for column names in PartitionSpec [iceberg]

2024-08-20 Thread via GitHub
rdblue merged PR #10678: URL: https://github.com/apache/iceberg/pull/10678 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.17.16 to 1.17.28 [iceberg-go]

2024-08-20 Thread via GitHub
nastra commented on PR #129: URL: https://github.com/apache/iceberg-go/pull/129#issuecomment-2299434096 > @nastra I tried that earlier, I'm not able to tell dependabot to rebase because I don't have push permission on here ([#127 (comment)](https://github.com/apache/iceberg-go/pull/127#issu

Re: [PR] Build: Bump com.azure:azure-sdk-bom from 1.2.25 to 1.2.26 [iceberg]

2024-08-20 Thread via GitHub
Fokko commented on PR #10870: URL: https://github.com/apache/iceberg/pull/10870#issuecomment-2299440980 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] Build: Bump junit from 5.10.1 to 5.10.2 [iceberg]

2024-08-20 Thread via GitHub
Fokko commented on PR #9699: URL: https://github.com/apache/iceberg/pull/9699#issuecomment-2299441652 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Build: Bump calcite from 1.10.0 to 1.36.0 [iceberg]

2024-08-20 Thread via GitHub
Fokko commented on PR #9042: URL: https://github.com/apache/iceberg/pull/9042#issuecomment-2299442087 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Build: Bump antlr from 4.9.3 to 4.13.1 [iceberg]

2024-08-20 Thread via GitHub
Fokko commented on PR #8772: URL: https://github.com/apache/iceberg/pull/8772#issuecomment-2299442355 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Build: Bump calcite from 1.10.0 to 1.36.0 [iceberg]

2024-08-20 Thread via GitHub
dependabot[bot] commented on PR #9042: URL: https://github.com/apache/iceberg/pull/9042#issuecomment-2299443530 OK, I won't notify you again about this release, but will get in touch when a new version is available. You can also ignore all major, minor, or patch releases for a dependency by

Re: [PR] Build: Bump software.amazon.s3.accessgrants:aws-s3-accessgrants-java-plugin from 2.0.0 to 2.0.1 [iceberg]

2024-08-20 Thread via GitHub
Fokko commented on PR #9705: URL: https://github.com/apache/iceberg/pull/9705#issuecomment-2299442773 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Build: Bump calcite from 1.10.0 to 1.36.0 [iceberg]

2024-08-20 Thread via GitHub
Fokko closed pull request #9042: Build: Bump calcite from 1.10.0 to 1.36.0 URL: https://github.com/apache/iceberg/pull/9042 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

Re: [PR] Build: Bump calcite from 1.10.0 to 1.36.0 [iceberg]

2024-08-20 Thread via GitHub
Fokko commented on PR #9042: URL: https://github.com/apache/iceberg/pull/9042#issuecomment-2299443475 https://github.com/apache/iceberg/pull/10317 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Build: Bump antlr from 4.9.3 to 4.13.1 [iceberg]

2024-08-20 Thread via GitHub
Fokko closed pull request #8772: Build: Bump antlr from 4.9.3 to 4.13.1 URL: https://github.com/apache/iceberg/pull/8772 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

Re: [PR] Build: Bump antlr from 4.9.3 to 4.13.1 [iceberg]

2024-08-20 Thread via GitHub
Fokko commented on PR #8772: URL: https://github.com/apache/iceberg/pull/8772#issuecomment-2299445059 https://github.com/apache/iceberg/pull/10867 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Build: Bump antlr from 4.9.3 to 4.13.1 [iceberg]

2024-08-20 Thread via GitHub
dependabot[bot] commented on PR #8772: URL: https://github.com/apache/iceberg/pull/8772#issuecomment-2299445123 OK, I won't notify you again about this release, but will get in touch when a new version is available. You can also ignore all major, minor, or patch releases for a dependency by

Re: [PR] Spark Action to Analyze table [iceberg]

2024-08-20 Thread via GitHub
aokolnychyi commented on code in PR #10288: URL: https://github.com/apache/iceberg/pull/10288#discussion_r1723732386 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/ComputeTableStatsSparkAction.java: ## @@ -0,0 +1,177 @@ +/* + * Licensed to the Apache Software

Re: [PR] Access delegation [iceberg-python]

2024-08-20 Thread via GitHub
guitcastro commented on code in PR #1033: URL: https://github.com/apache/iceberg-python/pull/1033#discussion_r1723770469 ## pyiceberg/catalog/rest.py: ## @@ -532,7 +534,7 @@ def _config_headers(self, session: Session) -> None: session.headers["Content-type"] = "applicat

Re: [PR] Data: Add a util to read write partition stats [iceberg]

2024-08-20 Thread via GitHub
RussellSpitzer commented on code in PR #10176: URL: https://github.com/apache/iceberg/pull/10176#discussion_r1723873509 ## data/src/main/java/org/apache/iceberg/data/PartitionStatsGenerator.java: ## @@ -0,0 +1,148 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [I] [Bug] Allow users to drop column description in `update_column` [iceberg-python]

2024-08-20 Thread via GitHub
corleyma commented on issue #1079: URL: https://github.com/apache/iceberg-python/issues/1079#issuecomment-2299646746 This proposed change made me wonder if I may have been implicitly relying on this behavior (bootstrapping tables with a schema, but then ingesting new data that may evolve t

Re: [PR] Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction [iceberg]

2024-08-20 Thread via GitHub
rodmeneses commented on code in PR #10179: URL: https://github.com/apache/iceberg/pull/10179#discussion_r1723920526 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -0,0 +1,740 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

Re: [PR] Add REST Compatibility Kit [iceberg]

2024-08-20 Thread via GitHub
danielcweeks commented on code in PR #10908: URL: https://github.com/apache/iceberg/pull/10908#discussion_r1723957789 ## build.gradle: ## @@ -952,6 +952,36 @@ project(':iceberg-snowflake') { } project(':iceberg-open-api') { + apply plugin: 'java-test-fixtures' Review Comme

Re: [PR] Spec: Support geo type [iceberg]

2024-08-20 Thread via GitHub
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1723950848 ## format/spec.md: ## @@ -373,6 +378,13 @@ Notes: 3. Strings are truncated to a valid UTF-8 string with no more than `L` code points. 4. In contrast to strings,

Re: [PR] Spec: Support geo type [iceberg]

2024-08-20 Thread via GitHub
szehon-ho commented on PR #10981: URL: https://github.com/apache/iceberg/pull/10981#issuecomment-2299761105 Also this is based closely on the decisions taken in the Parquet proposal for the same : https://github.com/apache/parquet-format/pull/240 -- This is an automated message from the

Re: [PR] Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction [iceberg]

2024-08-20 Thread via GitHub
rodmeneses commented on code in PR #10179: URL: https://github.com/apache/iceberg/pull/10179#discussion_r1723984264 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -0,0 +1,740 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

Re: [I] [Bug] Allow users to drop column description in `update_column` [iceberg-python]

2024-08-20 Thread via GitHub
sungwy commented on issue #1079: URL: https://github.com/apache/iceberg-python/issues/1079#issuecomment-2299781716 Hi @corleyma I think you are right! I think both of these code paths have a bug... it would be great to get this fixed together this time around with additional tests demonstr

Re: [PR] Spec: Support geo type [iceberg]

2024-08-20 Thread via GitHub
rdblue commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1724004137 ## format/spec.md: ## @@ -444,28 +456,28 @@ The schema of a manifest file is a struct called `manifest_entry` with the follo `data_file` is a struct with the follo

Re: [PR] Spec: Support geo type [iceberg]

2024-08-20 Thread via GitHub
rdblue commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1724005075 ## format/spec.md: ## @@ -576,12 +588,12 @@ Manifest list files store `manifest_file`, a struct with the following fields: `field_summary` is a struct with the fol

Re: [I] Add geometry type to iceberg [iceberg]

2024-08-20 Thread via GitHub
gregorywaynepower commented on issue #2586: URL: https://github.com/apache/iceberg/issues/2586#issuecomment-2299811087 Just want to make sure folks who care about Geospatial Support are forwarded to https://github.com/apache/iceberg/issues/10260 -- This is an automated message from the Ap

Re: [PR] Add REST Compatibility Kit [iceberg]

2024-08-20 Thread via GitHub
danielcweeks commented on code in PR #10908: URL: https://github.com/apache/iceberg/pull/10908#discussion_r1724006687 ## open-api/src/test/java/org/apache/iceberg/rest/RESTCompatibilityKitCatalogTests.java: ## @@ -0,0 +1,77 @@ +/* + * Licensed to the Apache Software Foundation (

Re: [PR] Spec: Support geo type [iceberg]

2024-08-20 Thread via GitHub
rdblue commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1724006108 ## format/spec.md: ## @@ -323,16 +327,17 @@ Partition field IDs must be reused if an existing partition spec contains an equ Partition Transforms -| Transfo

Re: [PR] Spec: Support geo type [iceberg]

2024-08-20 Thread via GitHub
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1724008688 ## format/spec.md: ## @@ -444,28 +456,28 @@ The schema of a manifest file is a struct called `manifest_entry` with the follo `data_file` is a struct with the fo

Re: [PR] Spec: Support geo type [iceberg]

2024-08-20 Thread via GitHub
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1724009250 ## format/spec.md: ## @@ -576,12 +588,12 @@ Manifest list files store `manifest_file`, a struct with the following fields: `field_summary` is a struct with the

Re: [PR] Spark Action to Analyze table [iceberg]

2024-08-20 Thread via GitHub
karuppayya commented on code in PR #10288: URL: https://github.com/apache/iceberg/pull/10288#discussion_r1723948256 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/actions/TestComputeTableStatsAction.java: ## @@ -0,0 +1,427 @@ +/* + * Licensed to the Apache Software F

Re: [PR] Spec: Support geo type [iceberg]

2024-08-20 Thread via GitHub
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1724008688 ## format/spec.md: ## @@ -444,28 +456,28 @@ The schema of a manifest file is a struct called `manifest_entry` with the follo `data_file` is a struct with the fo

Re: [PR] Spark Action to Analyze table [iceberg]

2024-08-20 Thread via GitHub
karuppayya commented on code in PR #10288: URL: https://github.com/apache/iceberg/pull/10288#discussion_r1724030187 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/AnalyzeTableSparkAction.java: ## @@ -0,0 +1,150 @@ +/* + * Licensed to the Apache Software Found

Re: [PR] Spec: Support geo type [iceberg]

2024-08-20 Thread via GitHub
hsiang-c commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1724053865 ## format/spec.md: ## @@ -1085,7 +1106,8 @@ Notes: Hash results are not dependent on decimal scale, which is part of the type, not the data value. 3. UUIDs are en

Re: [PR] Spec: Support geo type [iceberg]

2024-08-20 Thread via GitHub
flyrain commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1724059560 ## format/spec.md: ## @@ -444,28 +456,28 @@ The schema of a manifest file is a struct called `manifest_entry` with the follo `data_file` is a struct with the foll

Re: [PR] Core, Spark: Remove dangling deletes as part of RewriteDataFilesAction [iceberg]

2024-08-20 Thread via GitHub
szehon-ho commented on code in PR #9724: URL: https://github.com/apache/iceberg/pull/9724#discussion_r1724047273 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RemoveDanglingDeletesSparkAction.java: ## @@ -0,0 +1,179 @@ +/* + * Licensed to the Apache Software

Re: [PR] Core, Spark: Remove dangling deletes as part of RewriteDataFilesAction [iceberg]

2024-08-20 Thread via GitHub
szehon-ho commented on code in PR #9724: URL: https://github.com/apache/iceberg/pull/9724#discussion_r1724047895 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RemoveDanglingDeletesSparkAction.java: ## @@ -0,0 +1,179 @@ +/* + * Licensed to the Apache Software

Re: [PR] Core, Spark: Remove dangling deletes as part of RewriteDataFilesAction [iceberg]

2024-08-20 Thread via GitHub
szehon-ho commented on code in PR #9724: URL: https://github.com/apache/iceberg/pull/9724#discussion_r1724046486 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RemoveDanglingDeletesSparkAction.java: ## @@ -0,0 +1,179 @@ +/* + * Licensed to the Apache Software

Re: [PR] Spec: Support geo type [iceberg]

2024-08-20 Thread via GitHub
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1724070294 ## format/spec.md: ## @@ -1085,7 +1106,8 @@ Notes: Hash results are not dependent on decimal scale, which is part of the type, not the data value. 3. UUIDs are e

Re: [PR] Core, Spark: Remove dangling deletes as part of RewriteDataFilesAction [iceberg]

2024-08-20 Thread via GitHub
dramaticlly commented on code in PR #9724: URL: https://github.com/apache/iceberg/pull/9724#discussion_r1724070479 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RemoveDanglingDeletesSparkAction.java: ## @@ -0,0 +1,179 @@ +/* + * Licensed to the Apache Softwa

Re: [PR] Core, Spark: Remove dangling deletes as part of RewriteDataFilesAction [iceberg]

2024-08-20 Thread via GitHub
dramaticlly commented on code in PR #9724: URL: https://github.com/apache/iceberg/pull/9724#discussion_r1724071313 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RewriteDataFilesSparkAction.java: ## @@ -175,11 +179,21 @@ public RewriteDataFiles.Result execute

Re: [PR] Spec: Support geo type [iceberg]

2024-08-20 Thread via GitHub
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1724073282 ## format/spec.md: ## @@ -323,16 +327,17 @@ Partition field IDs must be reused if an existing partition spec contains an equ Partition Transforms -| Tran

Re: [PR] Spec: Support geo type [iceberg]

2024-08-20 Thread via GitHub
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1724074287 ## format/spec.md: ## @@ -444,28 +456,28 @@ The schema of a manifest file is a struct called `manifest_entry` with the follo `data_file` is a struct with the fo

[PR] Bump getdaft from 0.2.33 to 0.3.0 [iceberg-python]

2024-08-20 Thread via GitHub
dependabot[bot] opened a new pull request, #1080: URL: https://github.com/apache/iceberg-python/pull/1080 Bumps [getdaft](https://github.com/Eventual-Inc/Daft) from 0.2.33 to 0.3.0. Release notes Sourced from https://github.com/Eventual-Inc/Daft/releases";>getdaft's releases.

[PR] Bump griffe from 1.1.0 to 1.1.1 [iceberg-python]

2024-08-20 Thread via GitHub
dependabot[bot] opened a new pull request, #1081: URL: https://github.com/apache/iceberg-python/pull/1081 Bumps [griffe](https://github.com/mkdocstrings/griffe) from 1.1.0 to 1.1.1. Release notes Sourced from https://github.com/mkdocstrings/griffe/releases";>griffe's releases.

[PR] Bump mkdocs-autorefs from 1.0.1 to 1.1.0 [iceberg-python]

2024-08-20 Thread via GitHub
dependabot[bot] opened a new pull request, #1082: URL: https://github.com/apache/iceberg-python/pull/1082 Bumps [mkdocs-autorefs](https://github.com/mkdocstrings/autorefs) from 1.0.1 to 1.1.0. Release notes Sourced from https://github.com/mkdocstrings/autorefs/releases";>mkdocs-aut

Re: [PR] Core, Spark: Remove dangling deletes as part of RewriteDataFilesAction [iceberg]

2024-08-20 Thread via GitHub
dramaticlly commented on code in PR #9724: URL: https://github.com/apache/iceberg/pull/9724#discussion_r1724083371 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RemoveDanglingDeletesSparkAction.java: ## @@ -0,0 +1,179 @@ +/* + * Licensed to the Apache Softwa

Re: [PR] Spec: Support geo type [iceberg]

2024-08-20 Thread via GitHub
flyrain commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1724083771 ## format/spec.md: ## @@ -323,16 +327,17 @@ Partition field IDs must be reused if an existing partition spec contains an equ Partition Transforms -| Transf

Re: [PR] Spec: Support geo type [iceberg]

2024-08-20 Thread via GitHub
flyrain commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1724084062 ## format/spec.md: ## @@ -323,16 +327,17 @@ Partition field IDs must be reused if an existing partition spec contains an equ Partition Transforms -| Transf

Re: [PR] Core, Spark: Remove dangling deletes as part of RewriteDataFilesAction [iceberg]

2024-08-20 Thread via GitHub
dramaticlly commented on code in PR #9724: URL: https://github.com/apache/iceberg/pull/9724#discussion_r1724083371 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RemoveDanglingDeletesSparkAction.java: ## @@ -0,0 +1,179 @@ +/* + * Licensed to the Apache Softwa

Re: [PR] Spec: Support geo type [iceberg]

2024-08-20 Thread via GitHub
jiayuasu commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1724114033 ## format/spec.md: ## @@ -198,6 +199,9 @@ Notes: - Timestamp values _with time zone_ represent a point in time: values are stored as UTC and do not retain a so

Re: [PR] Spec: Support geo type [iceberg]

2024-08-20 Thread via GitHub
jiayuasu commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1724119576 ## format/spec.md: ## @@ -444,28 +456,28 @@ The schema of a manifest file is a struct called `manifest_entry` with the follo `data_file` is a struct with the fol

Re: [I] Missing Types.UUIDType in SUPPORTED_PRIMITIVES [iceberg]

2024-08-20 Thread via GitHub
github-actions[bot] commented on issue #1302: URL: https://github.com/apache/iceberg/issues/1302#issuecomment-2299972974 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Flink: IcebergTableSink to write data into multiple iceberg tables [iceberg]

2024-08-20 Thread via GitHub
github-actions[bot] closed issue #2208: Flink: IcebergTableSink to write data into multiple iceberg tables URL: https://github.com/apache/iceberg/issues/2208 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [I] Flink: IcebergTableSink to write data into multiple iceberg tables [iceberg]

2024-08-20 Thread via GitHub
github-actions[bot] commented on issue #2208: URL: https://github.com/apache/iceberg/issues/2208#issuecomment-2299972994 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Add metadata tables tests to make sure they don't break when reading different versions of tables [iceberg]

2024-08-20 Thread via GitHub
github-actions[bot] commented on issue #2532: URL: https://github.com/apache/iceberg/issues/2532#issuecomment-2299972999 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Spark: Added ability to add uuid suffix to the table location in Hive catalog [iceberg]

2024-08-20 Thread via GitHub
github-actions[bot] commented on PR #2850: URL: https://github.com/apache/iceberg/pull/2850#issuecomment-2299973016 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] Fixes RemoveOrphanFiles delete files unexpected [iceberg]

2024-08-20 Thread via GitHub
github-actions[bot] commented on PR #2890: URL: https://github.com/apache/iceberg/pull/2890#issuecomment-2299973036 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

  1   2   3   >