Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/glue from 1.73.1 to 1.93.0 [iceberg-go]

2024-08-18 Thread via GitHub
dependabot[bot] closed pull request #121: build(deps): bump github.com/aws/aws-sdk-go-v2/service/glue from 1.73.1 to 1.93.0 URL: https://github.com/apache/iceberg-go/pull/121 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

[PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/glue from 1.73.1 to 1.94.1 [iceberg-go]

2024-08-18 Thread via GitHub
dependabot[bot] opened a new pull request, #125: URL: https://github.com/apache/iceberg-go/pull/125 Bumps [github.com/aws/aws-sdk-go-v2/service/glue](https://github.com/aws/aws-sdk-go-v2) from 1.73.1 to 1.94.1. Commits https://github.com/aws/aws-sdk-go-v2/commit/1d44177adda2c5

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/glue from 1.73.1 to 1.93.0 [iceberg-go]

2024-08-18 Thread via GitHub
dependabot[bot] commented on PR #121: URL: https://github.com/apache/iceberg-go/pull/121#issuecomment-2295174211 Superseded by #125. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[PR] build(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.27.16 to 1.27.28 [iceberg-go]

2024-08-18 Thread via GitHub
dependabot[bot] opened a new pull request, #126: URL: https://github.com/apache/iceberg-go/pull/126 Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.27.16 to 1.27.28. Commits https://github.com/aws/aws-sdk-go-v2/commit/1d44177adda2c58035

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.27.16 to 1.27.27 [iceberg-go]

2024-08-18 Thread via GitHub
dependabot[bot] closed pull request #117: build(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.27.16 to 1.27.27 URL: https://github.com/apache/iceberg-go/pull/117 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.27.16 to 1.27.27 [iceberg-go]

2024-08-18 Thread via GitHub
dependabot[bot] commented on PR #117: URL: https://github.com/apache/iceberg-go/pull/117#issuecomment-2295174413 Superseded by #126. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.48.0 to 1.59.0 [iceberg-go]

2024-08-18 Thread via GitHub
dependabot[bot] opened a new pull request, #127: URL: https://github.com/apache/iceberg-go/pull/127 Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.48.0 to 1.59.0. Commits https://github.com/aws/aws-sdk-go-v2/commit/1d44177adda2c580

[PR] build(deps): bump github.com/aws/aws-sdk-go-v2 from 1.27.0 to 1.30.4 [iceberg-go]

2024-08-18 Thread via GitHub
dependabot[bot] opened a new pull request, #128: URL: https://github.com/apache/iceberg-go/pull/128 Bumps [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) from 1.27.0 to 1.30.4. Commits https://github.com/aws/aws-sdk-go-v2/commit/1d44177adda2c58035abe5f2e878

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2 from 1.27.0 to 1.30.3 [iceberg-go]

2024-08-18 Thread via GitHub
dependabot[bot] commented on PR #113: URL: https://github.com/apache/iceberg-go/pull/113#issuecomment-2295174737 Superseded by #128. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2 from 1.27.0 to 1.30.3 [iceberg-go]

2024-08-18 Thread via GitHub
dependabot[bot] closed pull request #113: build(deps): bump github.com/aws/aws-sdk-go-v2 from 1.27.0 to 1.30.3 URL: https://github.com/apache/iceberg-go/pull/113 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.17.16 to 1.17.27 [iceberg-go]

2024-08-18 Thread via GitHub
dependabot[bot] closed pull request #119: build(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.17.16 to 1.17.27 URL: https://github.com/apache/iceberg-go/pull/119 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.48.0 to 1.58.3 [iceberg-go]

2024-08-18 Thread via GitHub
dependabot[bot] commented on PR #120: URL: https://github.com/apache/iceberg-go/pull/120#issuecomment-2295174588 Superseded by #127. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.48.0 to 1.58.3 [iceberg-go]

2024-08-18 Thread via GitHub
dependabot[bot] closed pull request #120: build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.48.0 to 1.58.3 URL: https://github.com/apache/iceberg-go/pull/120 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[PR] build(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.17.16 to 1.17.28 [iceberg-go]

2024-08-18 Thread via GitHub
dependabot[bot] opened a new pull request, #129: URL: https://github.com/apache/iceberg-go/pull/129 Bumps [github.com/aws/aws-sdk-go-v2/credentials](https://github.com/aws/aws-sdk-go-v2) from 1.17.16 to 1.17.28. Commits https://github.com/aws/aws-sdk-go-v2/commit/1d44177adda2c

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.17.16 to 1.17.27 [iceberg-go]

2024-08-18 Thread via GitHub
dependabot[bot] commented on PR #119: URL: https://github.com/apache/iceberg-go/pull/119#issuecomment-2295174875 Superseded by #129. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Table Scan: Add Row Group Skipping and Row Selection Filtering [iceberg-rust]

2024-08-18 Thread via GitHub
liurenjie1024 commented on PR #558: URL: https://github.com/apache/iceberg-rust/pull/558#issuecomment-2295199077 Thanks @sdd for this pr. I just skimmed through it and thought that it's great! My suggestion is to split this pr into two smaller ones: one for row group and one for page index.

[I] bug: Main branch build failed. [iceberg-rust]

2024-08-18 Thread via GitHub
liurenjie1024 opened a new issue, #563: URL: https://github.com/apache/iceberg-rust/issues/563 Seems that the deploy to github pages workflow failed. cc @Xuanwo -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] refactor(evaluators): shift evaluator code into the table package [iceberg-go]

2024-08-18 Thread via GitHub
nastra merged PR #123: URL: https://github.com/apache/iceberg-go/pull/123 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] chore(s3): update to remove deprecated s3 sdk [iceberg-go]

2024-08-18 Thread via GitHub
nastra merged PR #122: URL: https://github.com/apache/iceberg-go/pull/122 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Core: Make namespace separator configurable [iceberg]

2024-08-18 Thread via GitHub
nastra commented on code in PR #10877: URL: https://github.com/apache/iceberg/pull/10877#discussion_r1720944857 ## core/src/main/java/org/apache/iceberg/rest/RESTUtil.java: ## @@ -194,15 +192,34 @@ public static String decodeString(String encoded) { * @return UTF-8 encoded s

Re: [PR] Core: Make namespace separator configurable [iceberg]

2024-08-18 Thread via GitHub
nastra commented on code in PR #10877: URL: https://github.com/apache/iceberg/pull/10877#discussion_r1720947546 ## core/src/main/java/org/apache/iceberg/rest/RESTUtil.java: ## @@ -194,15 +192,34 @@ public static String decodeString(String encoded) { * @return UTF-8 encoded s

Re: [PR] Build: Bump guava from 33.2.1-jre to 33.3.0-jre [iceberg]

2024-08-18 Thread via GitHub
nastra merged PR #10960: URL: https://github.com/apache/iceberg/pull/10960 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Build: Bump org.springframework:spring-web from 5.3.37 to 5.3.39 [iceberg]

2024-08-18 Thread via GitHub
nastra merged PR #10959: URL: https://github.com/apache/iceberg/pull/10959 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Flink-1.19: Fix the file offset mismatch when Flink reader first seek… [iceberg]

2024-08-18 Thread via GitHub
zhongyujiang commented on PR #10567: URL: https://github.com/apache/iceberg/pull/10567#issuecomment-2295217325 > What if we test the reader operator recovery by using OneInputStreamOperatorTestHarness? We can store the state of the operator, and later use it to restore the reader Hi

Re: [I] Iceberg Spark streaming skips rows of data [iceberg]

2024-08-18 Thread via GitHub
cccs-jc commented on issue #10156: URL: https://github.com/apache/iceberg/issues/10156#issuecomment-2295262592 @singhpk234 are you able to reproduce the issue ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [I] The ColumnarToRow Spark optimization is not applied when using nested fields from an Iceberg table [iceberg]

2024-08-18 Thread via GitHub
cccs-jc commented on issue #10828: URL: https://github.com/apache/iceberg/issues/10828#issuecomment-2295262915 @RussellSpitzer Is this something scheduled to be implemented in the next version of Iceberg ? -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [I] bug: Main branch build failed. [iceberg-rust]

2024-08-18 Thread via GitHub
Xuanwo commented on issue #563: URL: https://github.com/apache/iceberg-rust/issues/563#issuecomment-2295264778 It seems the infra team has changed "Workflow Permissions" in the Apache org. I have contact with them. This should have been fixed now, please try agaib. -- This is an automate

Re: [I] bug: Main branch build failed. [iceberg-rust]

2024-08-18 Thread via GitHub
Xuanwo commented on issue #563: URL: https://github.com/apache/iceberg-rust/issues/563#issuecomment-2295267087 > It seems the infra team has changed "Workflow Permissions" in the Apache org. I have contact with them. This should have been fixed now, please try again. Not fixed by the

Re: [PR] Flink: deprecate ReaderFunction with a new Converter interface to simplify user experience [iceberg]

2024-08-18 Thread via GitHub
stevenzwu commented on code in PR #10956: URL: https://github.com/apache/iceberg/pull/10956#discussion_r1721011163 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/source/reader/RowDataConverter.java: ## @@ -0,0 +1,50 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] Flink: deprecate ReaderFunction with a new Converter interface to simplify user experience [iceberg]

2024-08-18 Thread via GitHub
stevenzwu commented on code in PR #10956: URL: https://github.com/apache/iceberg/pull/10956#discussion_r1721011848 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/source/reader/ConverterReaderFunction.java: ## @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software

Re: [PR] Flink: deprecate ReaderFunction with a new Converter interface to simplify user experience [iceberg]

2024-08-18 Thread via GitHub
stevenzwu commented on code in PR #10956: URL: https://github.com/apache/iceberg/pull/10956#discussion_r1721011163 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/source/reader/RowDataConverter.java: ## @@ -0,0 +1,50 @@ +/* + * Licensed to the Apache Software Foundat

[PR] Core/RewriteFiles: Fixed dropping delete files that are still required [iceberg]

2024-08-18 Thread via GitHub
jasonf20 opened a new pull request, #10962: URL: https://github.com/apache/iceberg/pull/10962 This fixes a bug where delete files that are still required by the re-written file could be deleted prematurely. This happened because `MergingSnapshotProducer` calculated `minDataSequenceNu

Re: [PR] Add Support for Dynamic Overwrite [iceberg-python]

2024-08-18 Thread via GitHub
jqin61 commented on code in PR #931: URL: https://github.com/apache/iceberg-python/pull/931#discussion_r1721043893 ## pyiceberg/table/__init__.py: ## @@ -1551,6 +1627,17 @@ def append(self, df: pa.Table, snapshot_properties: Dict[str, str] = EMPTY_DICT) with self.trans

[PR] DOCS: Exclude signature files from twine upload [iceberg-python]

2024-08-18 Thread via GitHub
sungwy opened a new pull request, #1071: URL: https://github.com/apache/iceberg-python/pull/1071 The current `twine upload` command in the docs results in `Unknown distribution format` error due to the signature files. https://discuss.python.org/t/gpg-signature-support-removed-from-py

Re: [PR] Add Support for Dynamic Overwrite [iceberg-python]

2024-08-18 Thread via GitHub
jqin61 commented on code in PR #931: URL: https://github.com/apache/iceberg-python/pull/931#discussion_r1721056249 ## pyiceberg/table/__init__.py: ## @@ -3542,14 +3629,22 @@ def __init__(self, transaction: Transaction, io: FileIO, snapshot_properties: Di self._io = io

Re: [PR] Spark partial limit push down [iceberg]

2024-08-18 Thread via GitHub
huaxingao commented on code in PR #10943: URL: https://github.com/apache/iceberg/pull/10943#discussion_r1721078438 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkScanBuilder.java: ## @@ -407,14 +422,35 @@ public Scan build() { private Scan buildBatch

Re: [PR] Bump griffe from 0.49.0 to 1.0.0 [iceberg-python]

2024-08-18 Thread via GitHub
sungwy merged PR #1069: URL: https://github.com/apache/iceberg-python/pull/1069 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@icebe

Re: [PR] Flink: new sink base on the unified sink API [iceberg]

2024-08-18 Thread via GitHub
github-actions[bot] closed pull request #4904: Flink: new sink base on the unified sink API URL: https://github.com/apache/iceberg/pull/4904 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] Flink: new sink base on the unified sink API [iceberg]

2024-08-18 Thread via GitHub
github-actions[bot] commented on PR #4904: URL: https://github.com/apache/iceberg/pull/4904#issuecomment-2295449806 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] AWS: use pre-created IAM role for AssumeRole related integration tests [iceberg]

2024-08-18 Thread via GitHub
github-actions[bot] closed pull request #4909: AWS: use pre-created IAM role for AssumeRole related integration tests URL: https://github.com/apache/iceberg/pull/4909 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] Core, API: Add support for renaming tags [iceberg]

2024-08-18 Thread via GitHub
github-actions[bot] commented on PR #4936: URL: https://github.com/apache/iceberg/pull/4936#issuecomment-2295449836 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] AWS: use pre-created IAM role for AssumeRole related integration tests [iceberg]

2024-08-18 Thread via GitHub
github-actions[bot] commented on PR #4909: URL: https://github.com/apache/iceberg/pull/4909#issuecomment-2295449820 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Core, API: Add support for renaming tags [iceberg]

2024-08-18 Thread via GitHub
github-actions[bot] closed pull request #4936: Core, API: Add support for renaming tags URL: https://github.com/apache/iceberg/pull/4936 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Core:Collect filesizeinbytes to the manifest [iceberg]

2024-08-18 Thread via GitHub
github-actions[bot] commented on PR #5712: URL: https://github.com/apache/iceberg/pull/5712#issuecomment-2295450117 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] Docs: Fix docs for writing with partition clustering [iceberg]

2024-08-18 Thread via GitHub
github-actions[bot] commented on PR #5713: URL: https://github.com/apache/iceberg/pull/5713#issuecomment-2295450130 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [I] Registering BucketUDF on PySpark [iceberg]

2024-08-18 Thread via GitHub
github-actions[bot] commented on issue #5721: URL: https://github.com/apache/iceberg/issues/5721#issuecomment-2295450147 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] docs: Document how to register bucket function in pyspark [iceberg]

2024-08-18 Thread via GitHub
github-actions[bot] commented on PR #5722: URL: https://github.com/apache/iceberg/pull/5722#issuecomment-2295450159 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] Core/Spark: Fix memory leaks in vectorized reader for parquet [iceberg]

2024-08-18 Thread via GitHub
github-actions[bot] commented on PR #5735: URL: https://github.com/apache/iceberg/pull/5735#issuecomment-2295450171 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] Spark: Demonstrate reading Avro files with default value using a mock… [iceberg]

2024-08-18 Thread via GitHub
github-actions[bot] commented on PR #5738: URL: https://github.com/apache/iceberg/pull/5738#issuecomment-2295450190 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] Spark: Test custom metric for number of deletes applied, in code path that use streaming delete filter [iceberg]

2024-08-18 Thread via GitHub
github-actions[bot] commented on PR #5742: URL: https://github.com/apache/iceberg/pull/5742#issuecomment-2295450208 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] Spark: Simplify RewriteFiles.commit [iceberg]

2024-08-18 Thread via GitHub
github-actions[bot] commented on PR #5748: URL: https://github.com/apache/iceberg/pull/5748#issuecomment-2295450223 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] Core: Add minimum data sequence number to ManifestEntry [iceberg]

2024-08-18 Thread via GitHub
github-actions[bot] commented on PR #5760: URL: https://github.com/apache/iceberg/pull/5760#issuecomment-2295450241 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [I] show table extended not supported for v2 table. [iceberg]

2024-08-18 Thread via GitHub
github-actions[bot] commented on issue #5782: URL: https://github.com/apache/iceberg/issues/5782#issuecomment-2295450251 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Spark: Support call sort order/zorder procedure when having multiple spark extensions [iceberg]

2024-08-18 Thread via GitHub
github-actions[bot] commented on PR #5823: URL: https://github.com/apache/iceberg/pull/5823#issuecomment-2295450280 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] Spark: support hilbert curve when rewrite [iceberg]

2024-08-18 Thread via GitHub
github-actions[bot] commented on PR #5824: URL: https://github.com/apache/iceberg/pull/5824#issuecomment-2295450295 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] API,Core: Introduce metrics for data files by file format [iceberg]

2024-08-18 Thread via GitHub
github-actions[bot] commented on PR #5837: URL: https://github.com/apache/iceberg/pull/5837#issuecomment-2295450317 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [I] bug: Main branch build failed. [iceberg-rust]

2024-08-18 Thread via GitHub
liurenjie1024 closed issue #563: bug: Main branch build failed. URL: https://github.com/apache/iceberg-rust/issues/563 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

Re: [PR] docs: Fix build website permission changed [iceberg-rust]

2024-08-18 Thread via GitHub
liurenjie1024 merged PR #564: URL: https://github.com/apache/iceberg-rust/pull/564 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Support force option on RegisterTable procedure [iceberg]

2024-08-18 Thread via GitHub
yabola closed pull request #5327: Support force option on RegisterTable procedure URL: https://github.com/apache/iceberg/pull/5327 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] Expose Bucket Transform to Python Binding [iceberg-rust]

2024-08-18 Thread via GitHub
Xuanwo commented on code in PR #556: URL: https://github.com/apache/iceberg-rust/pull/556#discussion_r1721162878 ## bindings/python/src/transform.rs: ## @@ -0,0 +1,142 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

Re: [PR] Spark 3.5: Skip hadoop based catalog in migration test [iceberg]

2024-08-18 Thread via GitHub
manuzhang closed pull request #10952: Spark 3.5: Skip hadoop based catalog in migration test URL: https://github.com/apache/iceberg/pull/10952 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [I] Specify in lower/upper bounds in data_file struct are exact [iceberg]

2024-08-18 Thread via GitHub
osscm commented on issue #10930: URL: https://github.com/apache/iceberg/issues/10930#issuecomment-2295688832 > Before committing to such spec change, it would be great to have good understanding on how much real-world improvement it would bring, Based on my observations, in addition

Re: [I] Specify in lower/upper bounds in data_file struct are exact [iceberg]

2024-08-18 Thread via GitHub
findepi commented on issue #10930: URL: https://github.com/apache/iceberg/issues/10930#issuecomment-2295721068 i do you mean min/max and count from the Iceberg metadata are already used for query planning? though none of them used for the agg pushdown? @osscm Correct. Basic idea

Re: [PR] Object Cache: caches parsed Manifests and ManifestLists for performance [iceberg-rust]

2024-08-18 Thread via GitHub
sdd commented on code in PR #512: URL: https://github.com/apache/iceberg-rust/pull/512#discussion_r1721274453 ## crates/iceberg/src/io/object_cache.rs: ## @@ -0,0 +1,396 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

Re: [PR] Object Cache: caches parsed Manifests and ManifestLists for performance [iceberg-rust]

2024-08-18 Thread via GitHub
sdd commented on code in PR #512: URL: https://github.com/apache/iceberg-rust/pull/512#discussion_r1721276668 ## crates/iceberg/src/io/object_cache.rs: ## @@ -0,0 +1,396 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

Re: [PR] Object Cache: caches parsed Manifests and ManifestLists for performance [iceberg-rust]

2024-08-18 Thread via GitHub
sdd commented on code in PR #512: URL: https://github.com/apache/iceberg-rust/pull/512#discussion_r1721277607 ## crates/iceberg/src/io/object_cache.rs: ## @@ -0,0 +1,396 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

Re: [I] Support row filter & column masking in REST spec [iceberg]

2024-08-18 Thread via GitHub
nqvuong1998 commented on issue #10909: URL: https://github.com/apache/iceberg/issues/10909#issuecomment-2295781165 cc @nastra -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] Object Cache: caches parsed Manifests and ManifestLists for performance [iceberg-rust]

2024-08-18 Thread via GitHub
sdd commented on code in PR #512: URL: https://github.com/apache/iceberg-rust/pull/512#discussion_r1721285997 ## crates/iceberg/src/io/object_cache.rs: ## @@ -0,0 +1,396 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.