Re: [PR] Doc: Do Not Modify the Source Data Table During MergeIntoCommand Exec… [iceberg]

2024-12-16 Thread via GitHub
Fokko commented on code in PR #11787: URL: https://github.com/apache/iceberg/pull/11787#discussion_r1888049453 ## docs/docs/spark-writes.md: ## @@ -101,6 +101,9 @@ Spark 3.5 added support for `WHEN NOT MATCHED BY SOURCE ... THEN ...` to update WHEN NOT MATCHED BY SOURCE THEN U

Re: [PR] Spark 3.4: Add view support to SparkSessionCatalog [iceberg]

2024-12-16 Thread via GitHub
nastra commented on code in PR #11797: URL: https://github.com/apache/iceberg/pull/11797#discussion_r1888046849 ## spark/v3.4/spark-runtime/src/integration/java/org/apache/iceberg/spark/SmokeTest.java: ## @@ -169,9 +169,9 @@ public void testCreateTable() { @Test public v

Re: [PR] Doc: Add staus page for different implementations. [iceberg]

2024-12-16 Thread via GitHub
liurenjie1024 commented on code in PR #11772: URL: https://github.com/apache/iceberg/pull/11772#discussion_r1888025551 ## site/nav.yml: ## @@ -49,6 +49,7 @@ nav: - Multi-engine support: multi-engine-support.md - How to release: how-to-release.md - Terms: terms.md

Re: [PR] Doc: Add staus page for different implementations. [iceberg]

2024-12-16 Thread via GitHub
manuzhang commented on code in PR #11772: URL: https://github.com/apache/iceberg/pull/11772#discussion_r1887989451 ## site/nav.yml: ## @@ -49,6 +49,7 @@ nav: - Multi-engine support: multi-engine-support.md - How to release: how-to-release.md - Terms: terms.md +

Re: [PR] Add plan tasks for TableScan [iceberg-python]

2024-12-16 Thread via GitHub
ConeyLiu commented on code in PR #1427: URL: https://github.com/apache/iceberg-python/pull/1427#discussion_r1887950445 ## pyiceberg/table/__init__.py: ## @@ -1423,6 +1451,66 @@ def plan_files(self) -> Iterable[FileScanTask]: for data_entry in data_entries ]

Re: [PR] Add license checker [iceberg-cpp]

2024-12-16 Thread via GitHub
zhjwpku commented on code in PR #10: URL: https://github.com/apache/iceberg-cpp/pull/10#discussion_r1887940523 ## .github/workflows/license_check.yml: ## @@ -0,0 +1,33 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. Se

Re: [PR] Add license checker [iceberg-cpp]

2024-12-16 Thread via GitHub
zhjwpku commented on code in PR #10: URL: https://github.com/apache/iceberg-cpp/pull/10#discussion_r1887940523 ## .github/workflows/license_check.yml: ## @@ -0,0 +1,33 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. Se

Re: [PR] Add license checker [iceberg-cpp]

2024-12-16 Thread via GitHub
zhjwpku commented on code in PR #10: URL: https://github.com/apache/iceberg-cpp/pull/10#discussion_r1887922484 ## .github/.licenserc.yaml: ## @@ -0,0 +1,15 @@ +header: + license: +spdx-id: Apache-2.0 +copyright-owner: Apache Software Foundation + + paths: +- '**' +

Re: [PR] Doc: Add staus page for different implementations. [iceberg]

2024-12-16 Thread via GitHub
liurenjie1024 commented on code in PR #11772: URL: https://github.com/apache/iceberg/pull/11772#discussion_r1887930360 ## site/nav.yml: ## @@ -49,6 +49,7 @@ nav: - Multi-engine support: multi-engine-support.md - How to release: how-to-release.md - Terms: terms.md

Re: [I] [DISCUSS] Exceptions vs status codes [iceberg-cpp]

2024-12-16 Thread via GitHub
mapleFU commented on issue #14: URL: https://github.com/apache/iceberg-cpp/issues/14#issuecomment-2547519056 Personally I think if iceberg-cpp is just for parsing the metadata, all is ok for me. If it's also able to handle the dataset layer, I think exception might making maintaining the in

Re: [PR] Spec: Support geo type [iceberg]

2024-12-16 Thread via GitHub
jiayuasu commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1887892961 ## format/spec.md: ## @@ -584,8 +589,8 @@ The schema of a manifest file is a struct called `manifest_entry` with the follo | _optional_ | _optional_ | _optional_ |

Re: [PR] Spec: Support geo type [iceberg]

2024-12-16 Thread via GitHub
jiayuasu commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1887892961 ## format/spec.md: ## @@ -584,8 +589,8 @@ The schema of a manifest file is a struct called `manifest_entry` with the follo | _optional_ | _optional_ | _optional_ |

Re: [PR] Fix comment on `WRITE_OBJECT_STORE_PARTITIONED_PATHS` table property [iceberg]

2024-12-16 Thread via GitHub
amogh-jahagirdar commented on code in PR #11798: URL: https://github.com/apache/iceberg/pull/11798#discussion_r1887882188 ## core/src/main/java/org/apache/iceberg/TableProperties.java: ## @@ -244,7 +244,7 @@ private TableProperties() {} public static final String OBJECT_STORE

Re: [PR] Fix comment on `WRITE_OBJECT_STORE_PARTITIONED_PATHS` table property [iceberg]

2024-12-16 Thread via GitHub
amogh-jahagirdar commented on code in PR #11798: URL: https://github.com/apache/iceberg/pull/11798#discussion_r1887882188 ## core/src/main/java/org/apache/iceberg/TableProperties.java: ## @@ -244,7 +244,7 @@ private TableProperties() {} public static final String OBJECT_STORE

Re: [PR] chore: Generate Changelog Dependencies for 0.4.0 release [iceberg-rust]

2024-12-16 Thread via GitHub
Xuanwo merged PR #812: URL: https://github.com/apache/iceberg-rust/pull/812 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] refactor(puffin): Move puffin crate contents inside iceberg crate [iceberg-rust]

2024-12-16 Thread via GitHub
liurenjie1024 commented on PR #789: URL: https://github.com/apache/iceberg-rust/pull/789#issuecomment-2547437282 The reason why I suggest moving puffin into core crate is to avoid circulate dependency problem, thinking about the case we put it outside: 1. Puffer Read/Writer -> FileIO -> c

Re: [I] [DISCUSS] Exceptions vs status codes [iceberg-cpp]

2024-12-16 Thread via GitHub
zhjwpku commented on issue #14: URL: https://github.com/apache/iceberg-cpp/issues/14#issuecomment-2547432101 Facebook's Folly has a Expected class [0], Google's Abseil has a StatusOr class [1] all for the same purpose. [0] https://github.com/facebook/folly/blob/main/folly/Expected

Re: [PR] feat: Store file io props to allow re-build it [iceberg-rust]

2024-12-16 Thread via GitHub
Xuanwo commented on code in PR #802: URL: https://github.com/apache/iceberg-rust/pull/802#discussion_r1887841721 ## crates/iceberg/src/io/file_io.rs: ## @@ -165,7 +175,7 @@ impl FileIOBuilder { /// Fetch the scheme string. /// /// The scheme_str will be empty if i

Re: [PR] feat: Store file io props to allow re-build it [iceberg-rust]

2024-12-16 Thread via GitHub
liurenjie1024 merged PR #802: URL: https://github.com/apache/iceberg-rust/pull/802 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] feat: Store file io props to allow re-build it [iceberg-rust]

2024-12-16 Thread via GitHub
liurenjie1024 commented on code in PR #802: URL: https://github.com/apache/iceberg-rust/pull/802#discussion_r1887835700 ## crates/iceberg/src/io/file_io.rs: ## @@ -165,7 +175,7 @@ impl FileIOBuilder { /// Fetch the scheme string. /// /// The scheme_str will be emp

Re: [I] [DISCUSS] Exceptions vs status codes [iceberg-cpp]

2024-12-16 Thread via GitHub
zhjwpku commented on issue #14: URL: https://github.com/apache/iceberg-cpp/issues/14#issuecomment-2547406513 I came across this `expected`[0] class which seems conform to the idea of Status/Result, but it's in c++23. [0] https://en.cppreference.com/w/cpp/utility/expected -- This is

Re: [I] Support for S3 catalog to work with S3 Tables [iceberg-python]

2024-12-16 Thread via GitHub
jamesbornholt commented on issue #1404: URL: https://github.com/apache/iceberg-python/issues/1404#issuecomment-2547387263 @felixscherz thanks for catching this (and thanks to everyone who's interested in building S3 Tables support for PyIceberg!). We're working on an S3-side fix for the `x

Re: [PR] Doc: Add staus page for different implementations. [iceberg]

2024-12-16 Thread via GitHub
sungwy commented on code in PR #11772: URL: https://github.com/apache/iceberg/pull/11772#discussion_r1885084187 ## site/docs/status.md: ## @@ -0,0 +1,358 @@ +--- +title: "Implementation Status" +--- + + +# Implementations Status + +Apache iceberg's spec is implemented in multipl

Re: [PR] Prep 0.4.0 release [iceberg-rust]

2024-12-16 Thread via GitHub
sungwy commented on PR #809: URL: https://github.com/apache/iceberg-rust/pull/809#issuecomment-2547355292 Thank you @kevinjqliu and @Xuanwo ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] add Status data structure [iceberg-cpp]

2024-12-16 Thread via GitHub
zhjwpku commented on PR #8: URL: https://github.com/apache/iceberg-cpp/pull/8#issuecomment-2547351804 > I agree with @gaborkaszab that it would be better to discuss a concrete API design (e.g. Table, FileIO, etc.) before introducing a full-functional status implementation. If we decide to g

Re: [PR] Use compatible column name to set Parquet bloom filter [iceberg]

2024-12-16 Thread via GitHub
huaxingao commented on code in PR #11799: URL: https://github.com/apache/iceberg/pull/11799#discussion_r1887787105 ## parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java: ## @@ -266,6 +272,43 @@ private WriteBuilder createContextFunc( return this; } +

Re: [PR] feat: Add RemovePartitionSpecs table update [iceberg-rust]

2024-12-16 Thread via GitHub
Xuanwo merged PR #804: URL: https://github.com/apache/iceberg-rust/pull/804 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] Prep 0.4.0 release [iceberg-rust]

2024-12-16 Thread via GitHub
Xuanwo merged PR #809: URL: https://github.com/apache/iceberg-rust/pull/809 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] Use compatible column name to set Parquet bloom filter [iceberg]

2024-12-16 Thread via GitHub
singhpk234 commented on code in PR #11799: URL: https://github.com/apache/iceberg/pull/11799#discussion_r1887780848 ## parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java: ## @@ -266,6 +272,43 @@ private WriteBuilder createContextFunc( return this; } +

Re: [PR] Doc: Add staus page for different implementations. [iceberg]

2024-12-16 Thread via GitHub
manuzhang commented on code in PR #11772: URL: https://github.com/apache/iceberg/pull/11772#discussion_r1887776540 ## site/nav.yml: ## @@ -49,6 +49,7 @@ nav: - Multi-engine support: multi-engine-support.md - How to release: how-to-release.md - Terms: terms.md +

Re: [PR] Use compatible column name to set Parquet bloom filter [iceberg]

2024-12-16 Thread via GitHub
huaxingao commented on PR #11799: URL: https://github.com/apache/iceberg/pull/11799#issuecomment-2547289098 cc @szehon-ho -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [I] Storage Partitioned Join (SPJ) fails when >2 tables are joined [iceberg]

2024-12-16 Thread via GitHub
mrbrahman commented on issue #10450: URL: https://github.com/apache/iceberg/issues/10450#issuecomment-2547277838 Issue is still there. Commenting to keep it open -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Support WASB scheme in ADLSFileIO [iceberg]

2024-12-16 Thread via GitHub
bryanck commented on PR #11504: URL: https://github.com/apache/iceberg/pull/11504#issuecomment-2547272339 In retrospect this is probably a feature change we shouldn't have added in a patch release. We should probably revert this and have another patch release. -- This is an automated mess

Re: [PR] Core: Fix numeric overflow of timestamp nano literal [iceberg]

2024-12-16 Thread via GitHub
amogh-jahagirdar commented on code in PR #11775: URL: https://github.com/apache/iceberg/pull/11775#discussion_r1887748229 ## api/src/main/java/org/apache/iceberg/expressions/Literals.java: ## @@ -300,8 +300,7 @@ public Literal to(Type type) { case TIMESTAMP:

Re: [PR] Materialized View Spec [iceberg]

2024-12-16 Thread via GitHub
stevenzwu commented on code in PR #11041: URL: https://github.com/apache/iceberg/pull/11041#discussion_r1885417678 ## format/view-spec.md: ## @@ -160,6 +179,56 @@ Each entry in `version-log` is a struct with the following fields: | _required_ | `timestamp-ms` | Timestamp when

Re: [I] [DISCUSSION] Project Goal [iceberg-cpp]

2024-12-16 Thread via GitHub
GregoryKimball commented on issue #2: URL: https://github.com/apache/iceberg-cpp/issues/2#issuecomment-2547239172 Thank you for this discussion. > I would almost rather not depend on Arrow C++ if possible (what if I want to use the cuDF parquet reader, or OpenDAL for S3 access?)

Re: [I] Crash when writing map type with unsigned types [iceberg-python]

2024-12-16 Thread via GitHub
github-actions[bot] commented on issue #837: URL: https://github.com/apache/iceberg-python/issues/837#issuecomment-2547218078 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity oc

Re: [PR] Relax client.credentials-provider requirements [iceberg]

2024-12-16 Thread via GitHub
github-actions[bot] closed pull request #10604: Relax client.credentials-provider requirements URL: https://github.com/apache/iceberg/pull/10604 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Flink-1.19: Fix the file offset mismatch when Flink reader first seek… [iceberg]

2024-12-16 Thread via GitHub
github-actions[bot] closed pull request #10567: Flink-1.19: Fix the file offset mismatch when Flink reader first seek… URL: https://github.com/apache/iceberg/pull/10567 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Relax client.credentials-provider requirements [iceberg]

2024-12-16 Thread via GitHub
github-actions[bot] commented on PR #10604: URL: https://github.com/apache/iceberg/pull/10604#issuecomment-2547212972 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] Flink-1.19: Fix the file offset mismatch when Flink reader first seek… [iceberg]

2024-12-16 Thread via GitHub
github-actions[bot] commented on PR #10567: URL: https://github.com/apache/iceberg/pull/10567#issuecomment-2547212923 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [I] DeleteOrphanFilesSparkAction.listDirRecursively - No FileSystem for scheme "s3" [iceberg]

2024-12-16 Thread via GitHub
github-actions[bot] commented on issue #10539: URL: https://github.com/apache/iceberg/issues/10539#issuecomment-2547212864 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] Custom s3 endpoint: Unable to execute HTTP request: Remote host terminated the handshake [iceberg]

2024-12-16 Thread via GitHub
github-actions[bot] commented on issue #10490: URL: https://github.com/apache/iceberg/issues/10490#issuecomment-2547212745 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Support WASB scheme in ADLSFileIO [iceberg]

2024-12-16 Thread via GitHub
ajreid21 commented on PR #11504: URL: https://github.com/apache/iceberg/pull/11504#issuecomment-2547163579 AzureProperties builds a map of account -> sasToken here when you create ADLSFileIO using `adls.sas-token.` as the credential mechanism. Prior to this change, the account passed

Re: [PR] chore(docs): Update Readme - Lakekeeper repository moved [iceberg-rust]

2024-12-16 Thread via GitHub
liurenjie1024 merged PR #810: URL: https://github.com/apache/iceberg-rust/pull/810 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Support WASB scheme in ADLSFileIO [iceberg]

2024-12-16 Thread via GitHub
ajreid21 commented on PR #11504: URL: https://github.com/apache/iceberg/pull/11504#issuecomment-2547124997 This unfortunately is/was a breaking change 😞 - related to the concern @jbonofre brought up. -- This is an automated message from the Apache Git Service. To respond to the message, p

[PR] Use compatible column name to set Parquet bloom filter [iceberg]

2024-12-16 Thread via GitHub
huaxingao opened a new pull request, #11799: URL: https://github.com/apache/iceberg/pull/11799 When writing a Parquet file, if a column name contains special characters, e.g. `-`, Iceberg converts it to a compatible format. However, the bloom filter is still set using the original column na

[I] Performance issue with range queries over a partitioned table. [iceberg-rust]

2024-12-16 Thread via GitHub
ryzhyk opened a new issue, #811: URL: https://github.com/apache/iceberg-rust/issues/811 I ran into a performance issue querying an Iceberg table in S3 via the datafusion provider. The table was created using pyiceberg with the following schema: ```python schema = Schema( N

Re: [PR] Core: Fix numeric overflow of timestamp nano literal [iceberg]

2024-12-16 Thread via GitHub
ebyhr commented on code in PR #11775: URL: https://github.com/apache/iceberg/pull/11775#discussion_r1887709351 ## api/src/test/java/org/apache/iceberg/types/TestConversions.java: ## @@ -111,9 +111,9 @@ public void testByteBufferConversions() { assertConversion( 400

Re: [PR] Core: Allow adding files to multiple partition specs in FastAppend [iceberg]

2024-12-16 Thread via GitHub
anuragmantri commented on code in PR #11771: URL: https://github.com/apache/iceberg/pull/11771#discussion_r1887709101 ## core/src/test/java/org/apache/iceberg/catalog/CatalogTests.java: ## @@ -1590,13 +1590,15 @@ public void testCompleteCreateTransactionMultipleSchemas() {

Re: [PR] Fix comment on `WRITE_OBJECT_STORE_PARTITIONED_PATHS` table property [iceberg]

2024-12-16 Thread via GitHub
ebyhr commented on PR #11798: URL: https://github.com/apache/iceberg/pull/11798#issuecomment-2547049649 I believe this change is correct. The usage is: https://github.com/apache/iceberg/blob/b9b61b1d72ebb192d5e90453ff7030ece73d2603/core/src/main/java/org/apache/iceberg/LocationProviders.

Re: [PR] Docs: Change to Flink directory for instructions [iceberg]

2024-12-16 Thread via GitHub
szehon-ho commented on PR #11031: URL: https://github.com/apache/iceberg/pull/11031#issuecomment-2546995771 Whoops sorry, I must have missed this. I think it makes sense to me. cc @pvary @stevenzwu -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] Avro: Support default values for generic data [iceberg]

2024-12-16 Thread via GitHub
rdblue commented on PR #11786: URL: https://github.com/apache/iceberg/pull/11786#issuecomment-2546993293 Thanks for the review, @Fokko! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Avro: Support default values for generic data [iceberg]

2024-12-16 Thread via GitHub
rdblue merged PR #11786: URL: https://github.com/apache/iceberg/pull/11786 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Avro: Support default values for generic data [iceberg]

2024-12-16 Thread via GitHub
rdblue commented on code in PR #11786: URL: https://github.com/apache/iceberg/pull/11786#discussion_r1887662465 ## core/src/main/java/org/apache/iceberg/data/avro/PlannedDataReader.java: ## @@ -0,0 +1,187 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

Re: [PR] fix: field id in name mapping should be optional [iceberg-python]

2024-12-16 Thread via GitHub
barronw commented on code in PR #1426: URL: https://github.com/apache/iceberg-python/pull/1426#discussion_r1887662401 ## pyiceberg/table/name_mapping.py: ## @@ -333,8 +334,8 @@ def struct(self, struct: StructType, struct_partner: Optional[MappedField], fiel return Stru

[PR] Bump mkdocs-material from 9.5.48 to 9.5.49 [iceberg-python]

2024-12-16 Thread via GitHub
dependabot[bot] opened a new pull request, #1437: URL: https://github.com/apache/iceberg-python/pull/1437 Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.5.48 to 9.5.49. Release notes Sourced from https://github.com/squidfunk/mkdocs-material/releases";>

[PR] Bump adlfs from 2024.7.0 to 2024.12.0 [iceberg-python]

2024-12-16 Thread via GitHub
dependabot[bot] opened a new pull request, #1436: URL: https://github.com/apache/iceberg-python/pull/1436 Bumps [adlfs](https://github.com/fsspec/adlfs) from 2024.7.0 to 2024.12.0. Changelog Sourced from https://github.com/fsspec/adlfs/blob/main/CHANGELOG.md";>adlfs's changelog.

[PR] Bump moto from 5.0.22 to 5.0.23 [iceberg-python]

2024-12-16 Thread via GitHub
dependabot[bot] opened a new pull request, #1435: URL: https://github.com/apache/iceberg-python/pull/1435 Bumps [moto](https://github.com/getmoto/moto) from 5.0.22 to 5.0.23. Changelog Sourced from https://github.com/getmoto/moto/blob/master/CHANGELOG.md";>moto's changelog.

[PR] chore(docs): Update Readme - Lakekeeper repository moved [iceberg-rust]

2024-12-16 Thread via GitHub
c-thiel opened a new pull request, #810: URL: https://github.com/apache/iceberg-rust/pull/810 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

Re: [PR] Fix ParallelIterable deadlock [iceberg]

2024-12-16 Thread via GitHub
sopel39 commented on code in PR #11781: URL: https://github.com/apache/iceberg/pull/11781#discussion_r1887631464 ## core/src/main/java/org/apache/iceberg/util/ParallelIterable.java: ## @@ -257,17 +257,17 @@ private static class Task implements Supplier>>, Closeable { @Over

Re: [PR] Spec: Support geo type [iceberg]

2024-12-16 Thread via GitHub
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1887625905 ## format/spec.md: ## @@ -584,8 +589,8 @@ The schema of a manifest file is a struct called `manifest_entry` with the follo | _optional_ | _optional_ | _optional_

Re: [PR] Spec: Support geo type [iceberg]

2024-12-16 Thread via GitHub
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1887625905 ## format/spec.md: ## @@ -584,8 +589,8 @@ The schema of a manifest file is a struct called `manifest_entry` with the follo | _optional_ | _optional_ | _optional_

Re: [PR] Avro: Support default values for generic data [iceberg]

2024-12-16 Thread via GitHub
Fokko commented on code in PR #11786: URL: https://github.com/apache/iceberg/pull/11786#discussion_r1887585686 ## core/src/main/java/org/apache/iceberg/data/avro/PlannedDataReader.java: ## @@ -0,0 +1,181 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * o

[PR] Fix comment on `WRITE_OBJECT_STORE_PARTITIONED_PATHS` table property [iceberg]

2024-12-16 Thread via GitHub
smaheshwar-pltr opened a new pull request, #11798: URL: https://github.com/apache/iceberg/pull/11798 The code comment above the `WRITE_OBJECT_STORE_PARTITIONED_PATHS` constant in `TableProperties` was incorrect - partition values are excluded when this property is set to *false*, not true,

Re: [PR] Implementing namespace_exists function on the REST Catalog [iceberg-python]

2024-12-16 Thread via GitHub
sungwy commented on PR #1434: URL: https://github.com/apache/iceberg-python/pull/1434#issuecomment-2546756342 Hi @AhmedNader42 - thank you very much for picking up this issue and getting a working solution up already! I'm in agreement with @kevinjqliu 's comment, that it would be grea

Re: [PR] Docs: add note for `day` transform [iceberg]

2024-12-16 Thread via GitHub
kevinjqliu commented on code in PR #11749: URL: https://github.com/apache/iceberg/pull/11749#discussion_r1887580518 ## format/spec.md: ## @@ -454,7 +454,7 @@ Partition field IDs must be reused if an existing partition spec contains an equ | **`truncate[W]`** | Value truncated

Re: [PR] Prep 0.4.0 release [iceberg-rust]

2024-12-16 Thread via GitHub
sungwy commented on code in PR #809: URL: https://github.com/apache/iceberg-rust/pull/809#discussion_r1887569858 ## crates/iceberg/src/spec/table_metadata.rs: ## @@ -398,33 +397,6 @@ impl TableMetadata { self.partition_statistics.get(&snapshot_id) } -/// Appe

Re: [PR] Avro: Support default values for generic data [iceberg]

2024-12-16 Thread via GitHub
rdblue commented on code in PR #11786: URL: https://github.com/apache/iceberg/pull/11786#discussion_r1887573521 ## core/src/main/java/org/apache/iceberg/data/avro/DataReader.java: ## @@ -36,6 +36,10 @@ import org.apache.iceberg.types.Type; import org.apache.iceberg.types.Types

Re: [PR] fix: gurantee the deserialize order of struct is same as the struct type [iceberg-rust]

2024-12-16 Thread via GitHub
Fokko commented on code in PR #795: URL: https://github.com/apache/iceberg-rust/pull/795#discussion_r1887570509 ## crates/iceberg/src/spec/values.rs: ## @@ -3604,4 +3608,29 @@ mod tests { assert_eq!(result, expected); } + +#[test] +fn test_record_ser_de()

Re: [PR] Parquet: Implement defaults for generic data [iceberg]

2024-12-16 Thread via GitHub
rdblue commented on PR #11785: URL: https://github.com/apache/iceberg/pull/11785#issuecomment-2546711840 Merging this. Thanks for the review, @Fokko! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Parquet: Implement defaults for generic data [iceberg]

2024-12-16 Thread via GitHub
rdblue merged PR #11785: URL: https://github.com/apache/iceberg/pull/11785 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Spec: Support geo type [iceberg]

2024-12-16 Thread via GitHub
jiayuasu commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1887549063 ## format/spec.md: ## @@ -584,8 +589,8 @@ The schema of a manifest file is a struct called `manifest_entry` with the follo | _optional_ | _optional_ | _optional_ |

Re: [PR] Prep 0.4.0 release [iceberg-rust]

2024-12-16 Thread via GitHub
kevinjqliu commented on code in PR #809: URL: https://github.com/apache/iceberg-rust/pull/809#discussion_r1887539719 ## crates/iceberg/src/spec/table_metadata.rs: ## @@ -398,33 +397,6 @@ impl TableMetadata { self.partition_statistics.get(&snapshot_id) } -///

Re: [PR] Prep 0.4.0 release [iceberg-rust]

2024-12-16 Thread via GitHub
kevinjqliu commented on code in PR #809: URL: https://github.com/apache/iceberg-rust/pull/809#discussion_r1887538305 ## crates/iceberg/src/spec/snapshot.rs: ## @@ -192,13 +191,6 @@ impl Snapshot { partition_type_provider, ) } - -pub(crate) fn log(&

Re: [PR] Feat: support aliyun oss backend. [iceberg-go]

2024-12-16 Thread via GitHub
zeroshade commented on PR #216: URL: https://github.com/apache/iceberg-go/pull/216#issuecomment-2546599302 the Java iceberg impl has some mocking and test setups for Aliyun as seen [here](https://github.com/apache/iceberg/tree/main/aliyun/src/test/java/org/apache/iceberg/aliyun/oss) would i

Re: [PR] Flink: make `StatisticsOrRecord` to be correctly serialized and deser… [iceberg]

2024-12-16 Thread via GitHub
stevenzwu commented on code in PR #11557: URL: https://github.com/apache/iceberg/pull/11557#discussion_r1887477818 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/StatisticsOrRecordTypeInformation.java: ## @@ -0,0 +1,116 @@ +/* + * Licensed to the Apache

Re: [PR] Deserialize NestedField initial-default and write-default Attributes [iceberg-python]

2024-12-16 Thread via GitHub
paulcichonski commented on code in PR #1432: URL: https://github.com/apache/iceberg-python/pull/1432#discussion_r1887418287 ## tests/conftest.py: ## @@ -149,6 +149,35 @@ def table_schema_simple() -> Schema: ) +@pytest.fixture(scope="session") Review Comment: There w

Re: [I] Table scan doesn't respect casesensitive argument [iceberg-python]

2024-12-16 Thread via GitHub
jiakai-li closed issue #1421: Table scan doesn't respect casesensitive argument URL: https://github.com/apache/iceberg-python/issues/1421 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Fix `Table.scan` to enable case sensitive argument [iceberg-python]

2024-12-16 Thread via GitHub
jiakai-li commented on code in PR #1423: URL: https://github.com/apache/iceberg-python/pull/1423#discussion_r1887376637 ## pyiceberg/table/__init__.py: ## @@ -268,12 +268,10 @@ def _apply(self, updates: Tuple[TableUpdate, ...], requirements: Tuple[TableRequ return se

Re: [PR] Auth Manager API part 1: HTTPRequest, HTTPHeader [iceberg]

2024-12-16 Thread via GitHub
nastra commented on code in PR #11769: URL: https://github.com/apache/iceberg/pull/11769#discussion_r1887361072 ## core/src/main/java/org/apache/iceberg/rest/HTTPHeaders.java: ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more con

[PR] Spark 3.4: Add view support to SparkSessionCatalog [iceberg]

2024-12-16 Thread via GitHub
nastra opened a new pull request, #11797: URL: https://github.com/apache/iceberg/pull/11797 This backports #11388 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

Re: [I] Tracking issues of iceberg rust v0.4.0 Release [iceberg-rust]

2024-12-16 Thread via GitHub
sungwy commented on issue #739: URL: https://github.com/apache/iceberg-rust/issues/739#issuecomment-2546410059 Let's get this party started~! 🎈 Here's the PR to remove deprecated functions, and bump the version of iceberg-rust for review: https://github.com/apache/iceberg-rust/pull/8

Re: [PR] Prep 0.4.0 release [iceberg-rust]

2024-12-16 Thread via GitHub
sungwy commented on code in PR #809: URL: https://github.com/apache/iceberg-rust/pull/809#discussion_r1887327473 ## README.md: ## @@ -23,8 +23,6 @@ Rust implementation of [Apache Iceberg™](https://iceberg.apache.org/). -Working on [v0.3.0 Release Milestone](https://github.

[PR] Prep 0.4.0 release [iceberg-rust]

2024-12-16 Thread via GitHub
sungwy opened a new pull request, #809: URL: https://github.com/apache/iceberg-rust/pull/809 - Deprecated functions - Bump version of `iceberg-rust` to 0.4.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] Spark 3.4: Add REST catalog to Spark integration tests [iceberg]

2024-12-16 Thread via GitHub
danielcweeks merged PR #11698: URL: https://github.com/apache/iceberg/pull/11698 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceb

Re: [PR] Deserialize NestedField initial-default and write-default Attributes [iceberg-python]

2024-12-16 Thread via GitHub
paulcichonski commented on code in PR #1432: URL: https://github.com/apache/iceberg-python/pull/1432#discussion_r1887299466 ## pyiceberg/types.py: ## @@ -328,8 +328,8 @@ def __init__( data["type"] = data["type"] if "type" in data else field_type data["required"

Re: [PR] Auth Manager API part 1: HTTPRequest, HTTPHeader [iceberg]

2024-12-16 Thread via GitHub
danielcweeks commented on code in PR #11769: URL: https://github.com/apache/iceberg/pull/11769#discussion_r1887289371 ## core/src/test/java/org/apache/iceberg/rest/TestHTTPHeaders.java: ## @@ -0,0 +1,119 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * o

Re: [PR] Auth Manager API part 1: HTTPRequest, HTTPHeader [iceberg]

2024-12-16 Thread via GitHub
adutra commented on code in PR #11769: URL: https://github.com/apache/iceberg/pull/11769#discussion_r1887288109 ## core/src/main/java/org/apache/iceberg/rest/HTTPHeaders.java: ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more con

Re: [PR] Auth Manager API part 1: HTTPRequest, HTTPHeader [iceberg]

2024-12-16 Thread via GitHub
danielcweeks commented on code in PR #11769: URL: https://github.com/apache/iceberg/pull/11769#discussion_r1887285432 ## core/src/main/java/org/apache/iceberg/rest/HTTPHeaders.java: ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

Re: [PR] Auth Manager API part 1: HTTPRequest, HTTPHeader [iceberg]

2024-12-16 Thread via GitHub
danielcweeks commented on code in PR #11769: URL: https://github.com/apache/iceberg/pull/11769#discussion_r1887271043 ## core/src/main/java/org/apache/iceberg/rest/HTTPHeaders.java: ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

Re: [PR] Auth Manager API part 1: HTTPRequest, HTTPHeader [iceberg]

2024-12-16 Thread via GitHub
danielcweeks commented on code in PR #11769: URL: https://github.com/apache/iceberg/pull/11769#discussion_r1887281686 ## core/src/main/java/org/apache/iceberg/rest/HTTPHeaders.java: ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

Re: [PR] Auth Manager API part 1: HTTPRequest, HTTPHeader [iceberg]

2024-12-16 Thread via GitHub
danielcweeks commented on code in PR #11769: URL: https://github.com/apache/iceberg/pull/11769#discussion_r1887278512 ## core/src/main/java/org/apache/iceberg/rest/HTTPHeaders.java: ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

Re: [PR] Auth Manager API part 1: HTTPRequest, HTTPHeader [iceberg]

2024-12-16 Thread via GitHub
danielcweeks commented on code in PR #11769: URL: https://github.com/apache/iceberg/pull/11769#discussion_r1887276438 ## core/src/main/java/org/apache/iceberg/rest/HTTPHeaders.java: ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

Re: [PR] Auth Manager API part 1: HTTPRequest, HTTPHeader [iceberg]

2024-12-16 Thread via GitHub
danielcweeks commented on code in PR #11769: URL: https://github.com/apache/iceberg/pull/11769#discussion_r1887274849 ## core/src/main/java/org/apache/iceberg/rest/HTTPHeaders.java: ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

Re: [PR] Auth Manager API part 1: HTTPRequest, HTTPHeader [iceberg]

2024-12-16 Thread via GitHub
danielcweeks commented on code in PR #11769: URL: https://github.com/apache/iceberg/pull/11769#discussion_r1887271278 ## core/src/main/java/org/apache/iceberg/rest/HTTPHeaders.java: ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

Re: [PR] Auth Manager API part 1: HTTPRequest, HTTPHeader [iceberg]

2024-12-16 Thread via GitHub
danielcweeks commented on code in PR #11769: URL: https://github.com/apache/iceberg/pull/11769#discussion_r1887271043 ## core/src/main/java/org/apache/iceberg/rest/HTTPHeaders.java: ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

Re: [PR] Core: Add Variant implementation to read serialized objects [iceberg]

2024-12-16 Thread via GitHub
rdblue commented on code in PR #11415: URL: https://github.com/apache/iceberg/pull/11415#discussion_r1887270705 ## core/src/main/java/org/apache/iceberg/variants/Variants.java: ## @@ -0,0 +1,276 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more co

Re: [PR] Core: Add Variant implementation to read serialized objects [iceberg]

2024-12-16 Thread via GitHub
rdblue commented on code in PR #11415: URL: https://github.com/apache/iceberg/pull/11415#discussion_r1887268226 ## core/src/main/java/org/apache/iceberg/variants/PrimitiveWrapper.java: ## @@ -0,0 +1,206 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] Core: Add Variant implementation to read serialized objects [iceberg]

2024-12-16 Thread via GitHub
aihuaxu commented on code in PR #11415: URL: https://github.com/apache/iceberg/pull/11415#discussion_r1887235550 ## core/src/main/java/org/apache/iceberg/variants/PrimitiveWrapper.java: ## @@ -0,0 +1,206 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * o

  1   2   3   >