Re: [PR] Relax client.credentials-provider requirements [iceberg]

2024-12-16 Thread via GitHub
github-actions[bot] commented on PR #10604: URL: https://github.com/apache/iceberg/pull/10604#issuecomment-2547212972 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [I] Custom s3 endpoint: Unable to execute HTTP request: Remote host terminated the handshake [iceberg]

2024-12-16 Thread via GitHub
github-actions[bot] commented on issue #10490: URL: https://github.com/apache/iceberg/issues/10490#issuecomment-2547212745 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Flink-1.19: Fix the file offset mismatch when Flink reader first seek… [iceberg]

2024-12-16 Thread via GitHub
github-actions[bot] closed pull request #10567: Flink-1.19: Fix the file offset mismatch when Flink reader first seek… URL: https://github.com/apache/iceberg/pull/10567 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Relax client.credentials-provider requirements [iceberg]

2024-12-16 Thread via GitHub
github-actions[bot] closed pull request #10604: Relax client.credentials-provider requirements URL: https://github.com/apache/iceberg/pull/10604 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [I] Crash when writing map type with unsigned types [iceberg-python]

2024-12-16 Thread via GitHub
github-actions[bot] commented on issue #837: URL: https://github.com/apache/iceberg-python/issues/837#issuecomment-2547218078 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity oc

Re: [PR] Support WASB scheme in ADLSFileIO [iceberg]

2024-12-16 Thread via GitHub
ajreid21 commented on PR #11504: URL: https://github.com/apache/iceberg/pull/11504#issuecomment-2547124997 This unfortunately is/was a breaking change 😞 - related to the concern @jbonofre brought up. -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [PR] Prep 0.4.0 release [iceberg-rust]

2024-12-16 Thread via GitHub
Xuanwo merged PR #809: URL: https://github.com/apache/iceberg-rust/pull/809 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] feat: Add RemovePartitionSpecs table update [iceberg-rust]

2024-12-16 Thread via GitHub
Xuanwo merged PR #804: URL: https://github.com/apache/iceberg-rust/pull/804 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] Use compatible column name to set Parquet bloom filter [iceberg]

2024-12-16 Thread via GitHub
singhpk234 commented on code in PR #11799: URL: https://github.com/apache/iceberg/pull/11799#discussion_r1887780848 ## parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java: ## @@ -266,6 +272,43 @@ private WriteBuilder createContextFunc( return this; } +

Re: [PR] Use compatible column name to set Parquet bloom filter [iceberg]

2024-12-16 Thread via GitHub
huaxingao commented on code in PR #11799: URL: https://github.com/apache/iceberg/pull/11799#discussion_r1887787105 ## parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java: ## @@ -266,6 +272,43 @@ private WriteBuilder createContextFunc( return this; } +

Re: [PR] feat: Store file io props to allow re-build it [iceberg-rust]

2024-12-16 Thread via GitHub
Xuanwo commented on code in PR #802: URL: https://github.com/apache/iceberg-rust/pull/802#discussion_r1887841721 ## crates/iceberg/src/io/file_io.rs: ## @@ -165,7 +175,7 @@ impl FileIOBuilder { /// Fetch the scheme string. /// /// The scheme_str will be empty if i

Re: [PR] feat: Store file io props to allow re-build it [iceberg-rust]

2024-12-16 Thread via GitHub
liurenjie1024 merged PR #802: URL: https://github.com/apache/iceberg-rust/pull/802 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] feat: Store file io props to allow re-build it [iceberg-rust]

2024-12-16 Thread via GitHub
liurenjie1024 commented on code in PR #802: URL: https://github.com/apache/iceberg-rust/pull/802#discussion_r1887835700 ## crates/iceberg/src/io/file_io.rs: ## @@ -165,7 +175,7 @@ impl FileIOBuilder { /// Fetch the scheme string. /// /// The scheme_str will be emp

Re: [PR] Add license checker [iceberg-cpp]

2024-12-16 Thread via GitHub
zhjwpku commented on code in PR #10: URL: https://github.com/apache/iceberg-cpp/pull/10#discussion_r1887940523 ## .github/workflows/license_check.yml: ## @@ -0,0 +1,33 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. Se

Re: [PR] Add plan tasks for TableScan [iceberg-python]

2024-12-16 Thread via GitHub
ConeyLiu commented on code in PR #1427: URL: https://github.com/apache/iceberg-python/pull/1427#discussion_r1887950445 ## pyiceberg/table/__init__.py: ## @@ -1423,6 +1451,66 @@ def plan_files(self) -> Iterable[FileScanTask]: for data_entry in data_entries ]

Re: [PR] Doc: Add staus page for different implementations. [iceberg]

2024-12-16 Thread via GitHub
manuzhang commented on code in PR #11772: URL: https://github.com/apache/iceberg/pull/11772#discussion_r1887989451 ## site/nav.yml: ## @@ -49,6 +49,7 @@ nav: - Multi-engine support: multi-engine-support.md - How to release: how-to-release.md - Terms: terms.md +

Re: [PR] Support WASB scheme in ADLSFileIO [iceberg]

2024-12-16 Thread via GitHub
ajreid21 commented on PR #11504: URL: https://github.com/apache/iceberg/pull/11504#issuecomment-2547163579 AzureProperties builds a map of account -> sasToken here when you create ADLSFileIO using `adls.sas-token.` as the credential mechanism. Prior to this change, the account passed

[PR] Use compatible column name to set Parquet bloom filter [iceberg]

2024-12-16 Thread via GitHub
huaxingao opened a new pull request, #11799: URL: https://github.com/apache/iceberg/pull/11799 When writing a Parquet file, if a column name contains special characters, e.g. `-`, Iceberg converts it to a compatible format. However, the bloom filter is still set using the original column na

Re: [PR] Materialized View Spec [iceberg]

2024-12-16 Thread via GitHub
stevenzwu commented on code in PR #11041: URL: https://github.com/apache/iceberg/pull/11041#discussion_r1885417678 ## format/view-spec.md: ## @@ -160,6 +179,56 @@ Each entry in `version-log` is a struct with the following fields: | _required_ | `timestamp-ms` | Timestamp when

Re: [PR] Use compatible column name to set Parquet bloom filter [iceberg]

2024-12-16 Thread via GitHub
huaxingao commented on PR #11799: URL: https://github.com/apache/iceberg/pull/11799#issuecomment-2547289098 cc @szehon-ho -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[I] Performance issue with range queries over a partitioned table. [iceberg-rust]

2024-12-16 Thread via GitHub
ryzhyk opened a new issue, #811: URL: https://github.com/apache/iceberg-rust/issues/811 I ran into a performance issue querying an Iceberg table in S3 via the datafusion provider. The table was created using pyiceberg with the following schema: ```python schema = Schema( N

Re: [PR] Support WASB scheme in ADLSFileIO [iceberg]

2024-12-16 Thread via GitHub
bryanck commented on PR #11504: URL: https://github.com/apache/iceberg/pull/11504#issuecomment-2547272339 In retrospect this is probably a feature change we shouldn't have added in a patch release. We should probably revert this and have another patch release. -- This is an automated mess

Re: [I] Storage Partitioned Join (SPJ) fails when >2 tables are joined [iceberg]

2024-12-16 Thread via GitHub
mrbrahman commented on issue #10450: URL: https://github.com/apache/iceberg/issues/10450#issuecomment-2547277838 Issue is still there. Commenting to keep it open -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Doc: Add staus page for different implementations. [iceberg]

2024-12-16 Thread via GitHub
manuzhang commented on code in PR #11772: URL: https://github.com/apache/iceberg/pull/11772#discussion_r1887776540 ## site/nav.yml: ## @@ -49,6 +49,7 @@ nav: - Multi-engine support: multi-engine-support.md - How to release: how-to-release.md - Terms: terms.md +

Re: [PR] add Status data structure [iceberg-cpp]

2024-12-16 Thread via GitHub
zhjwpku commented on PR #8: URL: https://github.com/apache/iceberg-cpp/pull/8#issuecomment-2547351804 > I agree with @gaborkaszab that it would be better to discuss a concrete API design (e.g. Table, FileIO, etc.) before introducing a full-functional status implementation. If we decide to g

Re: [PR] Doc: Add staus page for different implementations. [iceberg]

2024-12-16 Thread via GitHub
sungwy commented on code in PR #11772: URL: https://github.com/apache/iceberg/pull/11772#discussion_r1885084187 ## site/docs/status.md: ## @@ -0,0 +1,358 @@ +--- +title: "Implementation Status" +--- + + +# Implementations Status + +Apache iceberg's spec is implemented in multipl

Re: [PR] Prep 0.4.0 release [iceberg-rust]

2024-12-16 Thread via GitHub
sungwy commented on PR #809: URL: https://github.com/apache/iceberg-rust/pull/809#issuecomment-2547355292 Thank you @kevinjqliu and @Xuanwo ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Core: Fix numeric overflow of timestamp nano literal [iceberg]

2024-12-16 Thread via GitHub
amogh-jahagirdar commented on code in PR #11775: URL: https://github.com/apache/iceberg/pull/11775#discussion_r1887748229 ## api/src/main/java/org/apache/iceberg/expressions/Literals.java: ## @@ -300,8 +300,7 @@ public Literal to(Type type) { case TIMESTAMP:

Re: [I] Support for S3 catalog to work with S3 Tables [iceberg-python]

2024-12-16 Thread via GitHub
jamesbornholt commented on issue #1404: URL: https://github.com/apache/iceberg-python/issues/1404#issuecomment-2547387263 @felixscherz thanks for catching this (and thanks to everyone who's interested in building S3 Tables support for PyIceberg!). We're working on an S3-side fix for the `x

Re: [I] [DISCUSS] Exceptions vs status codes [iceberg-cpp]

2024-12-16 Thread via GitHub
zhjwpku commented on issue #14: URL: https://github.com/apache/iceberg-cpp/issues/14#issuecomment-2547406513 I came across this `expected`[0] class which seems conform to the idea of Status/Result, but it's in c++23. [0] https://en.cppreference.com/w/cpp/utility/expected -- This is

Re: [PR] refactor(puffin): Move puffin crate contents inside iceberg crate [iceberg-rust]

2024-12-16 Thread via GitHub
liurenjie1024 commented on PR #789: URL: https://github.com/apache/iceberg-rust/pull/789#issuecomment-2547437282 The reason why I suggest moving puffin into core crate is to avoid circulate dependency problem, thinking about the case we put it outside: 1. Puffer Read/Writer -> FileIO -> c

Re: [I] [DISCUSS] Exceptions vs status codes [iceberg-cpp]

2024-12-16 Thread via GitHub
zhjwpku commented on issue #14: URL: https://github.com/apache/iceberg-cpp/issues/14#issuecomment-2547432101 Facebook's Folly has a Expected class [0], Google's Abseil has a StatusOr class [1] all for the same purpose. [0] https://github.com/facebook/folly/blob/main/folly/Expected

Re: [PR] Doc: Add staus page for different implementations. [iceberg]

2024-12-16 Thread via GitHub
liurenjie1024 commented on code in PR #11772: URL: https://github.com/apache/iceberg/pull/11772#discussion_r1888025551 ## site/nav.yml: ## @@ -49,6 +49,7 @@ nav: - Multi-engine support: multi-engine-support.md - How to release: how-to-release.md - Terms: terms.md

Re: [PR] Doc: Do Not Modify the Source Data Table During MergeIntoCommand Exec… [iceberg]

2024-12-16 Thread via GitHub
Fokko commented on code in PR #11787: URL: https://github.com/apache/iceberg/pull/11787#discussion_r1888049453 ## docs/docs/spark-writes.md: ## @@ -101,6 +101,9 @@ Spark 3.5 added support for `WHEN NOT MATCHED BY SOURCE ... THEN ...` to update WHEN NOT MATCHED BY SOURCE THEN U

Re: [PR] Spark 3.4: Add view support to SparkSessionCatalog [iceberg]

2024-12-16 Thread via GitHub
nastra commented on code in PR #11797: URL: https://github.com/apache/iceberg/pull/11797#discussion_r1888046849 ## spark/v3.4/spark-runtime/src/integration/java/org/apache/iceberg/spark/SmokeTest.java: ## @@ -169,9 +169,9 @@ public void testCreateTable() { @Test public v

<    1   2   3