Re: [PR] Bump cpp-linter/cpp-linter-action from 2.13.3 to 2.13.4 [iceberg-cpp]

2025-01-09 Thread via GitHub
wgtmac commented on PR #22: URL: https://github.com/apache/iceberg-cpp/pull/22#issuecomment-2581965020 @dependabot ignore this minor version -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [PR] Bump cpp-linter/cpp-linter-action from 2.13.3 to 2.13.4 [iceberg-cpp]

2025-01-09 Thread via GitHub
dependabot[bot] commented on PR #22: URL: https://github.com/apache/iceberg-cpp/pull/22#issuecomment-2581965070 Sorry, only users with push access can use that command. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[PR] Bump cpp-linter/cpp-linter-action from 2.13.3 to 2.13.4 [iceberg-cpp]

2025-01-09 Thread via GitHub
dependabot[bot] opened a new pull request, #22: URL: https://github.com/apache/iceberg-cpp/pull/22 Bumps [cpp-linter/cpp-linter-action](https://github.com/cpp-linter/cpp-linter-action) from 2.13.3 to 2.13.4. Release notes Sourced from https://github.com/cpp-linter/cpp-linter-actio

Re: [PR] Spec: Support geo type [iceberg]

2025-01-09 Thread via GitHub
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1909918007 ## format/spec.md: ## @@ -205,13 +205,18 @@ Supported primitive types are defined in the table below. Primitive types added | | **`uuid`**

Re: [PR] Spec: Support geo type [iceberg]

2025-01-09 Thread via GitHub
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1909917123 ## format/spec.md: ## @@ -205,13 +205,18 @@ Supported primitive types are defined in the table below. Primitive types added | | **`uuid`**

Re: [PR] Spec: Support geo type [iceberg]

2025-01-09 Thread via GitHub
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1909915990 ## format/spec.md: ## @@ -205,13 +205,18 @@ Supported primitive types are defined in the table below. Primitive types added | | **`uuid`**

Re: [PR] Spec: Support geo type [iceberg]

2025-01-09 Thread via GitHub
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1909913853 ## format/spec.md: ## @@ -603,8 +608,9 @@ Notes: 4. Position delete metadata can use `referenced_data_file` when all deletes tracked by the entry are in a single

Re: [PR] Spec: Support geo type [iceberg]

2025-01-09 Thread via GitHub
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1909913296 ## format/spec.md: ## @@ -940,9 +946,7 @@ Note that partition data tuple's schema is based on the partition spec output us The unified partition type is a struct

Re: [PR] Spec: Support geo type [iceberg]

2025-01-09 Thread via GitHub
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1909911607 ## format/spec.md: ## @@ -1480,6 +1494,9 @@ This serialization scheme is for storing single values as individual binary valu | **`struct`** | Not

Re: [PR] Spec: Support geo type [iceberg]

2025-01-09 Thread via GitHub
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1909910700 ## format/spec.md: ## @@ -1633,3 +1652,27 @@ might indicate different snapshot IDs for a specific timestamp. The discrepancie When processing point in time quer

Re: [PR] Auth Manager API part 3: OAuth2 Manager [iceberg]

2025-01-09 Thread via GitHub
nastra commented on code in PR #11844: URL: https://github.com/apache/iceberg/pull/11844#discussion_r1909181281 ## core/src/main/java/org/apache/iceberg/rest/auth/RefreshingAuthManager.java: ## @@ -0,0 +1,88 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[I] Atomic operation when writing to Icebergtables via flink [iceberg]

2025-01-09 Thread via GitHub
adityakumarsinha opened a new issue, #11944: URL: https://github.com/apache/iceberg/issues/11944 ### Query engine Flink ### Question I have a requirement to write to multiple Iceberg tables based on single source event. I like to know whether multiple iceberg sinks / com

Re: [I] [BUG] pyiceberg hanging on multiprocessing [iceberg-python]

2025-01-09 Thread via GitHub
frankliee commented on issue #1488: URL: https://github.com/apache/iceberg-python/issues/1488#issuecomment-2581774240 I use strace on the worker process, there are `FUTEX_WAIT_BITSET_PRIVATE`, and I not sure it was dead locking caused by process forking in pyarrow. Then I find that

Re: [I] cannot load table thru glue catalog [iceberg-python]

2025-01-09 Thread via GitHub
xpj01 commented on issue #1501: URL: https://github.com/apache/iceberg-python/issues/1501#issuecomment-2581656109 Hi @kevinjqliu , I used the same credential to upload and download file to the same bucket with S3Hooks which is working. The permission settings is different from pyice

Re: [I] [DISCUSS] Exceptions vs status codes [iceberg-cpp]

2025-01-09 Thread via GitHub
zhjwpku commented on issue #14: URL: https://github.com/apache/iceberg-cpp/issues/14#issuecomment-2581605188 > > I came across this `expected`[0] class which seems conform to the idea of Status/Result, but it's in c++23. > > [0] https://en.cppreference.com/w/cpp/utility/expected >

Re: [I] [DISCUSSION] Project Goal [iceberg-cpp]

2025-01-09 Thread via GitHub
MisterRaindrop commented on issue #2: URL: https://github.com/apache/iceberg-cpp/issues/2#issuecomment-2581588814 @GregoryKimball I am also interested in accessing different cloud providers through libcurl. We can implement the corresponding interfaces based on libcurl. Different cloud

Re: [PR] Avro: Add internal writer [iceberg]

2025-01-09 Thread via GitHub
ajantha-bhat commented on PR #11919: URL: https://github.com/apache/iceberg/pull/11919#issuecomment-2581584910 Rebased as I hit the flaky test https://github.com/apache/iceberg/issues/11833 -- This is an automated message from the Apache Git Service. To respond to the message, please l

Re: [PR] add Status data structure [iceberg-cpp]

2025-01-09 Thread via GitHub
lidavidm commented on PR #8: URL: https://github.com/apache/iceberg-cpp/pull/8#issuecomment-2581584623 I have used this before: https://github.com/TartanLlama/expected I think my main complaint (this applies to Arrow too) is that the structure is a bit opaque in a debugger. Arrow ende

Re: [I] Flaky test `TestFlinkTableSink > testInsertFromSourceTable` [iceberg]

2025-01-09 Thread via GitHub
ajantha-bhat commented on issue #11833: URL: https://github.com/apache/iceberg/issues/11833#issuecomment-2581584085 Another flaky test from the same class https://github.com/apache/iceberg/actions/runs/12701536583/job/35406205488?pr=11919 `TestFlinkTableSink > testReplacePartit

Re: [PR] add Status data structure [iceberg-cpp]

2025-01-09 Thread via GitHub
zhjwpku commented on PR #8: URL: https://github.com/apache/iceberg-cpp/pull/8#issuecomment-2581582802 > I'm a bit late here but even if we want to use Status/Result, it may be better to go with a backport of std::expected to at least try to align with the STL. > > Ah sorry, I see dis

Re: [PR] Use compatible column name to set Parquet bloom filter [iceberg]

2025-01-09 Thread via GitHub
huaxingao commented on PR #11799: URL: https://github.com/apache/iceberg/pull/11799#issuecomment-2581580455 I looked the code again, for the newly added method ``` private void setBloomFilterConfig( Context context, MessageType parquetSchema, BiCons

Re: [I] [DISCUSS] Exceptions vs status codes [iceberg-cpp]

2025-01-09 Thread via GitHub
wgtmac commented on issue #14: URL: https://github.com/apache/iceberg-cpp/issues/14#issuecomment-2581572706 > I came across this `expected`[0] class which seems conform to the idea of Status/Result, but it's in c++23. > > [0] https://en.cppreference.com/w/cpp/utility/expected I

Re: [PR] add Status data structure [iceberg-cpp]

2025-01-09 Thread via GitHub
wgtmac commented on PR #8: URL: https://github.com/apache/iceberg-cpp/pull/8#issuecomment-2581569168 @lidavidm I was about to reply to this thread to revive the discussion :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] [Discussion] googletest(gtest) or Catch2 [iceberg-cpp]

2025-01-09 Thread via GitHub
wgtmac commented on issue #12: URL: https://github.com/apache/iceberg-cpp/issues/12#issuecomment-258153 Should we move forward? I don't see any objection to googletest and https://github.com/apache/iceberg-cpp/pull/13 looks good now. cc @Fokko @Xuanwo -- This is an automated m

Re: [I] [DISCUSSION] Project Goal [iceberg-cpp]

2025-01-09 Thread via GitHub
wgtmac commented on issue #2: URL: https://github.com/apache/iceberg-cpp/issues/2#issuecomment-2581565411 Thanks @GregoryKimball for the input! I think `iceberg-core` should only contain a clean I/O interface (like `RemoteHandle` in your provided example) with a default implementation for l

Re: [PR] Avro: Add internal writer [iceberg]

2025-01-09 Thread via GitHub
ajantha-bhat commented on code in PR #11919: URL: https://github.com/apache/iceberg/pull/11919#discussion_r1909649535 ## core/src/test/java/org/apache/iceberg/avro/TestInternalAvro.java: ## @@ -0,0 +1,66 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * o

Re: [PR] add Status data structure [iceberg-cpp]

2025-01-09 Thread via GitHub
lidavidm commented on PR #8: URL: https://github.com/apache/iceberg-cpp/pull/8#issuecomment-2581550186 I'm a bit late here but even if we want to use Status/Result, it may be better to go with a backport of std::expected to at least try to align with the STL. -- This is an automated mess

Re: [PR] API, Core: Add scan planning apis to REST Catalog [iceberg]

2025-01-09 Thread via GitHub
rahil-c commented on PR #11180: URL: https://github.com/apache/iceberg/pull/11180#issuecomment-2581526854 > This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request

[PR] sanitize casting issue fix [iceberg]

2025-01-09 Thread via GitHub
dmgkeke opened a new pull request, #11943: URL: https://github.com/apache/iceberg/pull/11943 This is the fixed code that corrected the casting error that occurred in the issue below. I have also added one TestCode. #11932 -- This is an automated message from the Apache Git Servi

Re: [PR] API: Remove deprecated `apply()` [iceberg]

2025-01-09 Thread via GitHub
github-actions[bot] commented on PR #11691: URL: https://github.com/apache/iceberg/pull/11691#issuecomment-2581488452 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [I] [suggestion] Write path optimization [iceberg-python]

2025-01-09 Thread via GitHub
github-actions[bot] commented on issue #926: URL: https://github.com/apache/iceberg-python/issues/926#issuecomment-2581490674 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity oc

Re: [PR] API: Remove deprecated `apply()` [iceberg]

2025-01-09 Thread via GitHub
github-actions[bot] closed pull request #11691: API: Remove deprecated `apply()` URL: https://github.com/apache/iceberg/pull/11691 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] Core, Spark: Refactor FileRewriter interface to separate planning and execution [iceberg]

2025-01-09 Thread via GitHub
github-actions[bot] commented on PR #11513: URL: https://github.com/apache/iceberg/pull/11513#issuecomment-2581488381 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] GCP: Implement SupportsRecoveryOperations for GCSFileIO [iceberg]

2025-01-09 Thread via GitHub
github-actions[bot] commented on PR #11565: URL: https://github.com/apache/iceberg/pull/11565#issuecomment-2581488404 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Add scan planning api request and response models, parsers [iceberg]

2025-01-09 Thread via GitHub
github-actions[bot] commented on PR #11369: URL: https://github.com/apache/iceberg/pull/11369#issuecomment-2581488348 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] API, Core: Add scan planning apis to REST Catalog [iceberg]

2025-01-09 Thread via GitHub
github-actions[bot] commented on PR #11180: URL: https://github.com/apache/iceberg/pull/11180#issuecomment-2581488323 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Core: Fix caching table with metadata table names [iceberg]

2025-01-09 Thread via GitHub
github-actions[bot] commented on PR #11123: URL: https://github.com/apache/iceberg/pull/11123#issuecomment-2581488298 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] The unit test for class TestFlinkIcebergSink cannot be executed [iceberg]

2025-01-09 Thread via GitHub
github-actions[bot] commented on issue #10694: URL: https://github.com/apache/iceberg/issues/10694#issuecomment-2581488260 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] A casting error occurs when Sanitizing the expression value in a specific case. [iceberg]

2025-01-09 Thread via GitHub
dmgkeke commented on issue #11932: URL: https://github.com/apache/iceberg/issues/11932#issuecomment-2581476397 > need to update the code with above mentioned line of code? not enough code change I will change the code and push the PR -- This is an automated message from the Apach

Re: [PR] Auth Manager API part 3: OAuth2 Manager [iceberg]

2025-01-09 Thread via GitHub
danielcweeks commented on code in PR #11844: URL: https://github.com/apache/iceberg/pull/11844#discussion_r1909568642 ## core/src/main/java/org/apache/iceberg/rest/auth/AuthSessionCache.java: ## @@ -0,0 +1,127 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[PR] Build: Bump getdaft from 0.4.1 to 0.4.2 [iceberg-python]

2025-01-09 Thread via GitHub
dependabot[bot] opened a new pull request, #1503: URL: https://github.com/apache/iceberg-python/pull/1503 Bumps [getdaft](https://github.com/Eventual-Inc/Daft) from 0.4.1 to 0.4.2. Release notes Sourced from https://github.com/Eventual-Inc/Daft/releases";>getdaft's releases.

[PR] Build: Bump pydantic from 2.10.4 to 2.10.5 [iceberg-python]

2025-01-09 Thread via GitHub
dependabot[bot] opened a new pull request, #1504: URL: https://github.com/apache/iceberg-python/pull/1504 Bumps [pydantic](https://github.com/pydantic/pydantic) from 2.10.4 to 2.10.5. Changelog Sourced from https://github.com/pydantic/pydantic/blob/main/HISTORY.md";>pydantic's chan

[PR] Build: Bump sqlalchemy from 2.0.36 to 2.0.37 [iceberg-python]

2025-01-09 Thread via GitHub
dependabot[bot] opened a new pull request, #1502: URL: https://github.com/apache/iceberg-python/pull/1502 Bumps [sqlalchemy](https://github.com/sqlalchemy/sqlalchemy) from 2.0.36 to 2.0.37. Release notes Sourced from https://github.com/sqlalchemy/sqlalchemy/releases";>sqlalchemy's

Re: [PR] Nit fixes to URL-encoding of partition field names [iceberg-python]

2025-01-09 Thread via GitHub
smaheshwar-pltr commented on code in PR #1499: URL: https://github.com/apache/iceberg-python/pull/1499#discussion_r1909541302 ## tests/integration/test_partitioning_key.py: ## @@ -823,11 +789,6 @@ def test_partition_key( snapshot.manifests(iceberg_table.io)[0].fetc

Re: [PR] Nit fixes to URL-encoding of partition field names [iceberg-python]

2025-01-09 Thread via GitHub
smaheshwar-pltr commented on code in PR #1499: URL: https://github.com/apache/iceberg-python/pull/1499#discussion_r1909541206 ## tests/integration/test_partitioning_key.py: ## @@ -823,11 +789,6 @@ def test_partition_key( snapshot.manifests(iceberg_table.io)[0].fetc

Re: [PR] Auth Manager API part 3: OAuth2 Manager [iceberg]

2025-01-09 Thread via GitHub
danielcweeks commented on code in PR #11844: URL: https://github.com/apache/iceberg/pull/11844#discussion_r1909453326 ## core/src/main/java/org/apache/iceberg/rest/auth/AuthConfig.java: ## @@ -21,15 +21,16 @@ import java.util.Map; import javax.annotation.Nullable; import org.

Re: [PR] Modified exception objects being thrown when converting Pyarrow tables [iceberg-python]

2025-01-09 Thread via GitHub
DevChrisCross commented on PR #1498: URL: https://github.com/apache/iceberg-python/pull/1498#issuecomment-2581182631 @kevinjqliu thank you for the insight! I kind of somehow hesitated also at first in adding the `visit_pyarrow` for `pa.field`. I've committed the necessary changes :) -- T

Re: [PR] Support Location Providers [iceberg-python]

2025-01-09 Thread via GitHub
smaheshwar-pltr commented on PR #1452: URL: https://github.com/apache/iceberg-python/pull/1452#issuecomment-2581158650 > This matches the behavior of the [Java implementation](https://iceberg.apache.org/docs/nightly/custom-catalog/#custom-location-provider-implementation). However, if we're

Re: [PR] Call For Proposals Banner.html [iceberg]

2025-01-09 Thread via GitHub
RussellSpitzer commented on PR #11924: URL: https://github.com/apache/iceberg/pull/11924#issuecomment-2581157913 Merged slightly updated version https://github.com/apache/iceberg/pull/11942 since @Nhyi-streamlit is on vacation 🌴 -- This is an automated message from the Apache Git Service

Re: [PR] Call For Proposals Banner.html [iceberg]

2025-01-09 Thread via GitHub
RussellSpitzer closed pull request #11924: Call For Proposals Banner.html URL: https://github.com/apache/iceberg/pull/11924 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

Re: [PR] Put a CFP Banner on the Homepage [iceberg]

2025-01-09 Thread via GitHub
RussellSpitzer merged PR #11942: URL: https://github.com/apache/iceberg/pull/11942 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

[PR] Put a CFP Banner on the Homepage [iceberg]

2025-01-09 Thread via GitHub
RussellSpitzer opened a new pull request, #11942: URL: https://github.com/apache/iceberg/pull/11942 Follow up on [Nhyi-streamlit](https://github.com/apache/iceberg/issues?q=is%3Apr+is%3Aopen+author%3ANhyi-streamlit) since they are currently on vacation :) -- This is an automated

Re: [PR] Support Location Providers [iceberg-python]

2025-01-09 Thread via GitHub
smaheshwar-pltr commented on code in PR #1452: URL: https://github.com/apache/iceberg-python/pull/1452#discussion_r1909356132 ## tests/integration/test_writes/test_partitioned_writes.py: ## @@ -280,6 +280,43 @@ def test_query_filter_v1_v2_append_null( assert df.where(f"

Re: [PR] Support Location Providers [iceberg-python]

2025-01-09 Thread via GitHub
smaheshwar-pltr commented on code in PR #1452: URL: https://github.com/apache/iceberg-python/pull/1452#discussion_r1909309033 ## tests/integration/test_writes/test_writes.py: ## @@ -285,6 +285,33 @@ def test_data_files(spark: SparkSession, session_catalog: Catalog, arrow_table_

Re: [PR] Support Location Providers [iceberg-python]

2025-01-09 Thread via GitHub
smaheshwar-pltr commented on code in PR #1452: URL: https://github.com/apache/iceberg-python/pull/1452#discussion_r1909339914 ## pyiceberg/io/pyarrow.py: ## @@ -2234,7 +2235,9 @@ def data_file_statistics_from_parquet_metadata( ) -def write_file(io: FileIO, table_metadat

Re: [PR] Support Location Providers [iceberg-python]

2025-01-09 Thread via GitHub
smaheshwar-pltr commented on code in PR #1452: URL: https://github.com/apache/iceberg-python/pull/1452#discussion_r1909339914 ## pyiceberg/io/pyarrow.py: ## @@ -2234,7 +2235,9 @@ def data_file_statistics_from_parquet_metadata( ) -def write_file(io: FileIO, table_metadat

Re: [PR] Support Location Providers [iceberg-python]

2025-01-09 Thread via GitHub
smaheshwar-pltr commented on code in PR #1452: URL: https://github.com/apache/iceberg-python/pull/1452#discussion_r1909339914 ## pyiceberg/io/pyarrow.py: ## @@ -2234,7 +2235,9 @@ def data_file_statistics_from_parquet_metadata( ) -def write_file(io: FileIO, table_metadat

Re: [PR] Spark 3.5: Procedure to rewrite table path [iceberg]

2025-01-09 Thread via GitHub
karuppayya commented on code in PR #11931: URL: https://github.com/apache/iceberg/pull/11931#discussion_r1909323927 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRewriteTablePathProcedure.java: ## @@ -0,0 +1,141 @@ +/* + * Licensed to the A

Re: [PR] Support Location Providers [iceberg-python]

2025-01-09 Thread via GitHub
smaheshwar-pltr commented on code in PR #1452: URL: https://github.com/apache/iceberg-python/pull/1452#discussion_r1909330087 ## pyiceberg/table/locations.py: ## @@ -0,0 +1,82 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreeme

Re: [PR] Support Location Providers [iceberg-python]

2025-01-09 Thread via GitHub
smaheshwar-pltr commented on code in PR #1452: URL: https://github.com/apache/iceberg-python/pull/1452#discussion_r1909330087 ## pyiceberg/table/locations.py: ## @@ -0,0 +1,82 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreeme

Re: [I] [feature] Add support for `write.data.path` and `write.metadata.path` [iceberg-python]

2025-01-09 Thread via GitHub
jiakai-li commented on issue #1492: URL: https://github.com/apache/iceberg-python/issues/1492#issuecomment-2581073509 Sure, that makes sense to me, thank you @smaheshwar-pltr , :-) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [PR] Materialized View Spec [iceberg]

2025-01-09 Thread via GitHub
danielcweeks commented on code in PR #11041: URL: https://github.com/apache/iceberg/pull/11041#discussion_r1909320996 ## format/view-spec.md: ## @@ -82,9 +98,12 @@ Each version in `versions` is a struct with the following fields: | _required_ | `representations` | A list of

Re: [PR] Support Location Providers [iceberg-python]

2025-01-09 Thread via GitHub
smaheshwar-pltr commented on code in PR #1452: URL: https://github.com/apache/iceberg-python/pull/1452#discussion_r1909315124 ## pyiceberg/table/locations.py: ## @@ -0,0 +1,82 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreeme

Re: [PR] Support Location Providers [iceberg-python]

2025-01-09 Thread via GitHub
smaheshwar-pltr commented on code in PR #1452: URL: https://github.com/apache/iceberg-python/pull/1452#discussion_r1909315124 ## pyiceberg/table/locations.py: ## @@ -0,0 +1,82 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreeme

Re: [PR] Support Location Providers [iceberg-python]

2025-01-09 Thread via GitHub
smaheshwar-pltr commented on code in PR #1452: URL: https://github.com/apache/iceberg-python/pull/1452#discussion_r1909315124 ## pyiceberg/table/locations.py: ## @@ -0,0 +1,82 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreeme

Re: [PR] Support Location Providers [iceberg-python]

2025-01-09 Thread via GitHub
smaheshwar-pltr commented on code in PR #1452: URL: https://github.com/apache/iceberg-python/pull/1452#discussion_r1909309033 ## tests/integration/test_writes/test_writes.py: ## @@ -285,6 +285,33 @@ def test_data_files(spark: SparkSession, session_catalog: Catalog, arrow_table_

Re: [PR] Materialized View Spec [iceberg]

2025-01-09 Thread via GitHub
wmoustafa commented on PR #11041: URL: https://github.com/apache/iceberg/pull/11041#issuecomment-2581043683 > > > If i understand correctly @wmoustafa comment on the mailing list, then there is some ambiguity here for what to put, if the same table in expressed in the various forms (catalog

Re: [PR] Materialized View Spec [iceberg]

2025-01-09 Thread via GitHub
stevenzwu commented on code in PR #11041: URL: https://github.com/apache/iceberg/pull/11041#discussion_r1909298744 ## format/view-spec.md: ## @@ -82,9 +98,12 @@ Each version in `versions` is a struct with the following fields: | _required_ | `representations` | A list of [

Re: [PR] Support Location Providers [iceberg-python]

2025-01-09 Thread via GitHub
smaheshwar-pltr commented on code in PR #1452: URL: https://github.com/apache/iceberg-python/pull/1452#discussion_r1909299027 ## tests/table/test_locations.py: ## @@ -0,0 +1,123 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agree

Re: [PR] Support Location Providers [iceberg-python]

2025-01-09 Thread via GitHub
smaheshwar-pltr commented on code in PR #1452: URL: https://github.com/apache/iceberg-python/pull/1452#discussion_r1909297301 ## tests/table/test_locations.py: ## @@ -0,0 +1,123 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agree

Re: [PR] Support Location Providers [iceberg-python]

2025-01-09 Thread via GitHub
smaheshwar-pltr commented on code in PR #1452: URL: https://github.com/apache/iceberg-python/pull/1452#discussion_r1909284421 ## pyiceberg/table/locations.py: ## @@ -0,0 +1,82 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreeme

Re: [PR] Support Location Providers [iceberg-python]

2025-01-09 Thread via GitHub
smaheshwar-pltr commented on code in PR #1452: URL: https://github.com/apache/iceberg-python/pull/1452#discussion_r1909281754 ## pyiceberg/table/locations.py: ## @@ -0,0 +1,82 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreeme

Re: [PR] Support Location Providers [iceberg-python]

2025-01-09 Thread via GitHub
smaheshwar-pltr commented on code in PR #1452: URL: https://github.com/apache/iceberg-python/pull/1452#discussion_r1909263229 ## pyiceberg/table/locations.py: ## @@ -0,0 +1,81 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreeme

Re: [PR] Support Location Providers [iceberg-python]

2025-01-09 Thread via GitHub
kevinjqliu commented on code in PR #1452: URL: https://github.com/apache/iceberg-python/pull/1452#discussion_r1909263268 ## pyiceberg/io/pyarrow.py: ## @@ -2234,7 +2235,9 @@ def data_file_statistics_from_parquet_metadata( ) -def write_file(io: FileIO, table_metadata: Ta

Re: [PR] Change dot notation in add column documentation to tuple [iceberg-python]

2025-01-09 Thread via GitHub
kevinjqliu commented on PR #1433: URL: https://github.com/apache/iceberg-python/pull/1433#issuecomment-2580965961 Thanks for your help improving the docs @jeppe-dos! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Change dot notation in add column documentation to tuple [iceberg-python]

2025-01-09 Thread via GitHub
kevinjqliu merged PR #1433: URL: https://github.com/apache/iceberg-python/pull/1433 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

Re: [PR] Change dot notation in add column documentation to tuple [iceberg-python]

2025-01-09 Thread via GitHub
kevinjqliu commented on code in PR #1433: URL: https://github.com/apache/iceberg-python/pull/1433#discussion_r1909250917 ## mkdocs/docs/api.md: ## @@ -951,8 +951,12 @@ Using `add_column` you can add a column, without having to worry about the field with table.update_schema() a

Re: [PR] Avro: Add internal writer [iceberg]

2025-01-09 Thread via GitHub
rdblue commented on code in PR #11919: URL: https://github.com/apache/iceberg/pull/11919#discussion_r1909249879 ## core/src/test/java/org/apache/iceberg/avro/TestInternalAvro.java: ## @@ -0,0 +1,66 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] Avro: Add internal writer [iceberg]

2025-01-09 Thread via GitHub
rdblue commented on code in PR #11919: URL: https://github.com/apache/iceberg/pull/11919#discussion_r1909246082 ## api/src/test/java/org/apache/iceberg/util/RandomUtil.java: ## @@ -228,4 +235,54 @@ private static BigInteger randomUnscaled(int precision, Random random) {

[PR] API: Fix parameter of ScanTask [iceberg]

2025-01-09 Thread via GitHub
engraving-knife opened a new pull request, #11941: URL: https://github.com/apache/iceberg/pull/11941 close #11939 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

Re: [PR] API: Fix parameter of ScanTask [iceberg]

2025-01-09 Thread via GitHub
engraving-knife closed pull request #11940: API: Fix parameter of ScanTask URL: https://github.com/apache/iceberg/pull/11940 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[PR] API: Fix parameter of ScanTask [iceberg]

2025-01-09 Thread via GitHub
engraving-knife opened a new pull request, #11940: URL: https://github.com/apache/iceberg/pull/11940 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

[I] Default parameters error in ScanTask [iceberg]

2025-01-09 Thread via GitHub
engraving-knife opened a new issue, #11939: URL: https://github.com/apache/iceberg/issues/11939 ### Apache Iceberg version 1.7.1 (latest release) ### Query engine None ### Please describe the bug 🐞 I found that 1024 seemed to be written incorrectly as 1028 i

Re: [PR] Build: Bump mypy-boto3-glue from 1.35.87 to 1.35.93 [iceberg-python]

2025-01-09 Thread via GitHub
kevinjqliu merged PR #1496: URL: https://github.com/apache/iceberg-python/pull/1496 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

Re: [PR] Build: Bump boto3 from 1.35.88 to 1.35.93 [iceberg-python]

2025-01-09 Thread via GitHub
kevinjqliu merged PR #1495: URL: https://github.com/apache/iceberg-python/pull/1495 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

Re: [PR] Modified exception objects being thrown when converting Pyarrow tables [iceberg-python]

2025-01-09 Thread via GitHub
kevinjqliu commented on PR #1498: URL: https://github.com/apache/iceberg-python/pull/1498#issuecomment-2580930119 I think generally we want this kind of error message when calling the `pyarrow_to_schema` function, which is used in the create table path and the write path. ``` Column '

Re: [PR] Nit fixes to URL-encoding of partition field names [iceberg-python]

2025-01-09 Thread via GitHub
kevinjqliu commented on code in PR #1499: URL: https://github.com/apache/iceberg-python/pull/1499#discussion_r1907602687 ## pyiceberg/partitioning.py: ## @@ -237,8 +237,7 @@ def partition_to_path(self, data: Record, schema: Schema) -> str: value_str = quote_plus(va

Re: [I] cannot load table thru glue catalog [iceberg-python]

2025-01-09 Thread via GitHub
kevinjqliu commented on issue #1501: URL: https://github.com/apache/iceberg-python/issues/1501#issuecomment-2580902171 > OSError: When reading information for key 'iceberg_test/metadata/4-804bd1cf-09b6-48b4-8d8b-4e332d971b13.metadata.json' in bucket '': AWS Error ACCESS_DENIED during H

Re: [PR] Impl rest catalog + table updates & requirements [iceberg-go]

2025-01-09 Thread via GitHub
chil-pavn commented on PR #146: URL: https://github.com/apache/iceberg-go/pull/146#issuecomment-2580874842 @jwtryg yes, i am intending to finish off rest catalog operations first as i already have a draft for the same. Once it is done, i could extend to others too. @zeroshade sure, i

Re: [I] HiveCatalog Not Support Nanotimestamp [iceberg]

2025-01-09 Thread via GitHub
dfercode commented on issue #11937: URL: https://github.com/apache/iceberg/issues/11937#issuecomment-2580779525 > @dfercode: Are you using Hive engine, or just HiveCatalog? > @deniskuzZ: Are there any plans to support nanosecond timestamps with Hive? (#8657) Im using HiveCatalog in

[PR] Modified the Code block [iceberg]

2025-01-09 Thread via GitHub
richochetclementine1315 opened a new pull request, #11938: URL: https://github.com/apache/iceberg/pull/11938 Added a code block in the **ExpressionUtil.Java** to counter A casting error when Sanitizing the expression value in a specific case. ISSUE #11932 -- This is an automated mes

Re: [I] HiveCatalog Not Support Nanotimestamp [iceberg]

2025-01-09 Thread via GitHub
pvary commented on issue #11937: URL: https://github.com/apache/iceberg/issues/11937#issuecomment-2580741633 @dfercode: Are you using Hive engine, or just HiveCatalog? @deniskuzZ: Are there any plans to support nanosecond timestamps with Hive? (#8657) -- This is an automated message fr

Re: [I] A casting error occurs when Sanitizing the expression value in a specific case. [iceberg]

2025-01-09 Thread via GitHub
richochetclementine1315 commented on issue #11932: URL: https://github.com/apache/iceberg/issues/11932#issuecomment-2580724970 .take issue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] Tests: Set PySpark driver host to `localhost` [iceberg-python]

2025-01-09 Thread via GitHub
smaheshwar-pltr closed pull request #1466: Tests: Set PySpark driver host to `localhost` URL: https://github.com/apache/iceberg-python/pull/1466 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Tests: Set PySpark driver host to `localhost` [iceberg-python]

2025-01-09 Thread via GitHub
smaheshwar-pltr commented on PR #1466: URL: https://github.com/apache/iceberg-python/pull/1466#issuecomment-2580707479 Thanks so much folks for confirming that my local setup was the problem - I looked more carefully and found `25/01/09 16:13:06 WARN Utils: Set SPARK_LOCAL_IP if you need to

Re: [PR] Support Location Providers [iceberg-python]

2025-01-09 Thread via GitHub
smaheshwar-pltr commented on code in PR #1452: URL: https://github.com/apache/iceberg-python/pull/1452#discussion_r1909052866 ## pyiceberg/table/__init__.py: ## @@ -1627,6 +1632,67 @@ class AddFileTask: partition_field_value: Record +class LocationProvider(ABC): Review

Re: [PR] Support Location Providers [iceberg-python]

2025-01-09 Thread via GitHub
smaheshwar-pltr commented on code in PR #1452: URL: https://github.com/apache/iceberg-python/pull/1452#discussion_r1909052866 ## pyiceberg/table/__init__.py: ## @@ -1627,6 +1632,67 @@ class AddFileTask: partition_field_value: Record +class LocationProvider(ABC): Review

Re: [PR] Spec: Support geo type [iceberg]

2025-01-09 Thread via GitHub
mkaravel commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1909040124 ## format/spec.md: ## @@ -1633,3 +1652,27 @@ might indicate different snapshot IDs for a specific timestamp. The discrepancie When processing point in time queri

Re: [PR] Kafka Connect: Add SMTs for Debezium and AWS DMS [iceberg]

2025-01-09 Thread via GitHub
ismailsimsek commented on code in PR #11936: URL: https://github.com/apache/iceberg/pull/11936#discussion_r1909022386 ## kafka-connect/build.gradle: ## @@ -242,3 +243,19 @@ project(':iceberg-kafka-connect:iceberg-kafka-connect-runtime') { assemble.dependsOn distZip, hiveDi

Re: [PR] Parquet: Add readers and writers for the internal object model [iceberg]

2025-01-09 Thread via GitHub
ajantha-bhat commented on code in PR #11904: URL: https://github.com/apache/iceberg/pull/11904#discussion_r1908952177 ## .palantir/revapi.yml: ## @@ -1171,6 +1171,28 @@ acceptedBreaks: \ java.util.function.Function, org.apache.iceberg.io.CloseableIterable,\ \

  1   2   >