Re: [PR] OpenAPI: Express server capabilities via /config endpoint [iceberg]

nastra commented on code in PR #9940: URL: https://github.com/apache/iceberg/pull/9940#discussion_r1665217429 ## open-api/rest-catalog-open-api.yaml: ## @@ -61,12 +65,26 @@ security: - OAuth2: [catalog] - BearerAuth: [] +# Capability tags describe optional functionality

Re: [PR] AWS: Retain Glue Catalog column comment after updating Iceberg table [iceberg]

rahil-c commented on code in PR #10276: URL: https://github.com/apache/iceberg/pull/10276#discussion_r1665217122 ## aws/src/integration/java/org/apache/iceberg/aws/glue/TestGlueCatalogTable.java: ## @@ -221,6 +221,69 @@ public void testUpdateTable() { assertThat(response.ta

Re: [PR] OpenAPI: Express server capabilities via /config endpoint [iceberg]

nastra commented on code in PR #9940: URL: https://github.com/apache/iceberg/pull/9940#discussion_r1665211346 ## open-api/rest-catalog-open-api.yaml: ## @@ -28,6 +28,10 @@ info: description: Defines the specification for the first version of the REST Catalog API. Im

Re: [PR] Apply IntelliJ inspection findings to older Spark + Flink versions [iceberg]

nastra merged PR #10625: URL: https://github.com/apache/iceberg/pull/10625 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] AWS: Add Option to don't write non current columns in glue schema closes #7584 [iceberg]

mgmarino commented on PR #9420: URL: https://github.com/apache/iceberg/pull/9420#issuecomment-2208193970 @Raphael-Vignes this seems like a good solution to the quoted issues. Anything keeping you from opening this for review? One point: I expect that the maintainers will want to keep

Re: [PR] REST: disallow overriding "credential" in table sessions [iceberg]

nastra merged PR #10345: URL: https://github.com/apache/iceberg/pull/10345 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Spark Action to Analyze table [iceberg]

ajantha-bhat commented on code in PR #10288: URL: https://github.com/apache/iceberg/pull/10288#discussion_r1665179103 ## api/src/main/java/org/apache/iceberg/actions/ActionsProvider.java: ## @@ -70,4 +70,10 @@ default RewritePositionDeleteFiles rewritePositionDeletes(Table tabl

Re: [PR] Handle error when trying to drop a non-empty namespace on rest.py [iceberg-python]

HonahX merged PR #868: URL: https://github.com/apache/iceberg-python/pull/868 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Raise error on nonexistent namespace on sql.py [iceberg-python]

HonahX merged PR #865: URL: https://github.com/apache/iceberg-python/pull/865 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] REST: assume issued token type is access token [iceberg]

nastra commented on code in PR #10314: URL: https://github.com/apache/iceberg/pull/10314#discussion_r1665172068 ## core/src/main/java/org/apache/iceberg/rest/auth/OAuth2Util.java: ## @@ -738,13 +738,21 @@ private static AuthSession fromTokenResponse( long startTimeMilli

Re: [PR] fix: update all namespace properties in update_namespace_properties [iceberg-python]

HonahX merged PR #873: URL: https://github.com/apache/iceberg-python/pull/873 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [I] NullPointerException after deleting old partition column [iceberg]

mgmarino commented on issue #10626: URL: https://github.com/apache/iceberg/issues/10626#issuecomment-2208166944 @lurnagao-dahua thanks for linking that, I did not find it when searching. I see a few things possibly different here (though of course the underlying cause could be the same):

Re: [PR] feat: runtime module [iceberg-rust]

liurenjie1024 commented on code in PR #233: URL: https://github.com/apache/iceberg-rust/pull/233#discussion_r1665146620 ## crates/iceberg/Cargo.toml: ## @@ -82,3 +86,4 @@ iceberg_test_utils = { path = "../test_utils", features = ["tests"] } pretty_assertions = { workspace = tr

Re: [PR] Create rollback and set snapshot APIs [iceberg-python]

HonahX commented on code in PR #758: URL: https://github.com/apache/iceberg-python/pull/758#discussion_r1665133966 ## pyiceberg/table/__init__.py: ## @@ -1975,6 +1948,52 @@ def _commit(self) -> UpdatesAndRequirements: """Apply the pending changes and commit."""

Re: [PR] Don't refresh table in the for-loop [iceberg-python]

HonahX merged PR #874: URL: https://github.com/apache/iceberg-python/pull/874 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [I] Add metadata tables [iceberg-python]

HonahX commented on issue #511: URL: https://github.com/apache/iceberg-python/issues/511#issuecomment-2208120231 The last metadata table has been merged. Thanks everyone for all the great work and contributions. I think this issue can be closed -- This is an automated message from the Apa

Re: [I] Add metadata tables [iceberg-python]

HonahX closed issue #511: Add metadata tables URL: https://github.com/apache/iceberg-python/issues/511 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: i

Re: [PR] Add Files metadata table [iceberg-python]

HonahX merged PR #614: URL: https://github.com/apache/iceberg-python/pull/614 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Add Files metadata table [iceberg-python]

HonahX commented on PR #614: URL: https://github.com/apache/iceberg-python/pull/614#issuecomment-2208118906 LGTM too. @Gowthami03B Thanks for working on this! Thanks everyone for reviewing. Let's get this last metadata table in! -- This is an automated message from the Apache Git Service.

Re: [PR] refactor(catalogs/rest): Split user config and runtime config [iceberg-rust]

liurenjie1024 commented on code in PR #431: URL: https://github.com/apache/iceberg-rust/pull/431#discussion_r1665116676 ## crates/catalog/rest/Cargo.toml: ## @@ -32,13 +32,15 @@ keywords = ["iceberg", "rest", "catalog"] # async-trait = { workspace = true } async-trait = { work

Re: [I] idea(catalog/rest): Separate static and dynamic props [iceberg-rust]

liurenjie1024 closed issue #430: idea(catalog/rest): Separate static and dynamic props URL: https://github.com/apache/iceberg-rust/issues/430 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] refactor(catalogs/rest): Split user config and runtime config [iceberg-rust]

liurenjie1024 merged PR #431: URL: https://github.com/apache/iceberg-rust/pull/431 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Bump PyIceberg in CI to 0.6.1 [iceberg-python]

HonahX merged PR #879: URL: https://github.com/apache/iceberg-python/pull/879 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Cast 's', 'ms' and 'ns' PyArrow timestamp to 'us' precision on write [iceberg-python]

HonahX commented on code in PR #848: URL: https://github.com/apache/iceberg-python/pull/848#discussion_r1665113108 ## pyiceberg/io/pyarrow.py: ## @@ -918,11 +919,24 @@ def primitive(self, primitive: pa.DataType) -> PrimitiveType: return TimeType() elif pa.

Re: [PR] Flink: Pre-create fieldGetters to avoid constructing them for each row [iceberg]

pvary commented on code in PR #10565: URL: https://github.com/apache/iceberg/pull/10565#discussion_r1665110520 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/data/RowDataUtil.java: ## @@ -79,7 +79,11 @@ public static Object convertConstant(Type type, Object value)

Re: [PR] Flink: Pre-create fieldGetters to avoid constructing them for each row [iceberg]

pvary commented on code in PR #10565: URL: https://github.com/apache/iceberg/pull/10565#discussion_r1665112819 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/data/RowDataUtil.java: ## @@ -78,6 +78,37 @@ public static Object convertConstant(Type type, Object value)

Re: [PR] Use SupportsPrefixOperations for Remove OrphanFile Procedure [iceberg]

rahil-c commented on PR #7914: URL: https://github.com/apache/iceberg/pull/7914#issuecomment-2208107487 Hi all sorry for the delay on this issue, been engaged in many internal things at work so did not get time to revisit this. Originally when I encountered this issue it was a very s

Re: [PR] refactor(catalogs/rest): Split user config and runtime config [iceberg-rust]

Xuanwo commented on code in PR #431: URL: https://github.com/apache/iceberg-rust/pull/431#discussion_r1665089134 ## crates/catalog/rest/src/client.rs: ## @@ -15,25 +15,165 @@ // specific language governing permissions and limitations // under the License. +use crate::types::

Re: [PR] Flink: Pre-create fieldGetters to avoid constructing them for each row [iceberg]

fengjiajie commented on PR #10565: URL: https://github.com/apache/iceberg/pull/10565#issuecomment-2208086135 > Thanks for the detailed explanation and the PR @fengjiajie! > > Left a few small comments. > > As a general rule we create a PR for a single version. Usually the latest

Re: [PR] Flink: Pre-create fieldGetters to avoid constructing them for each row [iceberg]

fengjiajie commented on code in PR #10565: URL: https://github.com/apache/iceberg/pull/10565#discussion_r1665086491 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/data/RowDataUtil.java: ## @@ -79,7 +79,11 @@ public static Object convertConstant(Type type, Object va

Re: [PR] refactor(catalogs/rest): Split user config and runtime config [iceberg-rust]

Xuanwo commented on code in PR #431: URL: https://github.com/apache/iceberg-rust/pull/431#discussion_r1665084442 ## crates/catalog/rest/Cargo.toml: ## @@ -32,13 +32,15 @@ keywords = ["iceberg", "rest", "catalog"] # async-trait = { workspace = true } async-trait = { workspace =

Re: [PR] refactor(catalogs/rest): Split user config and runtime config [iceberg-rust]

Xuanwo commented on code in PR #431: URL: https://github.com/apache/iceberg-rust/pull/431#discussion_r1665083249 ## crates/catalog/rest/src/client.rs: ## @@ -15,25 +15,165 @@ // specific language governing permissions and limitations // under the License. +use crate::types::

Re: [PR] refactor(catalogs/rest): Split user config and runtime config [iceberg-rust]

Xuanwo commented on code in PR #431: URL: https://github.com/apache/iceberg-rust/pull/431#discussion_r1665083079 ## crates/catalog/rest/src/client.rs: ## @@ -15,25 +15,165 @@ // specific language governing permissions and limitations // under the License. +use crate::types::

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

epgif commented on PR #9008: URL: https://github.com/apache/iceberg/pull/9008#issuecomment-2208073422 Thanks @amogh-jahagirdar! I believe I've addressed all the comments. Please note that @jacobmarble and I are both in the US and will be out of the office Thursday and Friday. W

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

epgif commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1665079938 ## api/src/main/java/org/apache/iceberg/expressions/ExpressionUtil.java: ## @@ -594,6 +605,12 @@ private static String sanitizeString(CharSequence value, long now, int t

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

epgif commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1665079821 ## api/src/main/java/org/apache/iceberg/transforms/Timestamps.java: ## @@ -31,54 +33,160 @@ import org.apache.iceberg.util.DateTimeUtil; import org.apache.iceberg.util.

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

epgif commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1665080227 ## api/src/test/java/org/apache/iceberg/TestPartitionPaths.java: ## @@ -54,6 +54,25 @@ public void testPartitionPath() { .isEqualTo("ts_hour=2017-12-01-10/id_buc

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

epgif commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1665080053 ## api/src/main/java/org/apache/iceberg/types/Types.java: ## @@ -259,6 +261,60 @@ public int hashCode() { } } + public static class TimestampNanoType extends P

Re: [I] idea(catalog/rest): Separate static and dynamic props [iceberg-rust]

liurenjie1024 commented on issue #430: URL: https://github.com/apache/iceberg-rust/issues/430#issuecomment-2207930775 > BTW should we use a strong-typed Config/Props instead of a Hashmap? Do we have discussions before? It seems that there is no clear definition of all props. -- Thi

Re: [PR] refactor(catalogs/rest): Split user config and runtime config [iceberg-rust]

liurenjie1024 commented on code in PR #431: URL: https://github.com/apache/iceberg-rust/pull/431#discussion_r1664977067 ## crates/catalog/rest/Cargo.toml: ## @@ -32,13 +32,15 @@ keywords = ["iceberg", "rest", "catalog"] # async-trait = { workspace = true } async-trait = { work

Re: [PR] core:Refactor the code of HadoopTableOptions [iceberg]

BsoBird commented on PR #10623: URL: https://github.com/apache/iceberg/pull/10623#issuecomment-2207862734 @nastra @Fokko @RussellSpitzer @rdblue @pvary hello. If someone could help me review this PR, that would be great. Tks. -- This is an automated message from the Apache Git Service.

Re: [I] NullPointerException after deleting old partition column [iceberg]

lurnagao-dahua commented on issue #10626: URL: https://github.com/apache/iceberg/issues/10626#issuecomment-2207685799 Hi, same issues in [10234](https://github.com/apache/iceberg/issues/10234) -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [I] NullPointerException after deleting old partition column [iceberg]

dramaticlly commented on issue #10626: URL: https://github.com/apache/iceberg/issues/10626#issuecomment-2207648800 Based on the stacktrace, looks like partitions table need to evaluate all historical partition specs to build the partition column per #7551 , but since the column is alread

Re: [I] Spark3 DSv2: Handle Snapshots of Type OVERWRITE - while reading Stream from Iceberg table [iceberg]

github-actions[bot] commented on issue #2788: URL: https://github.com/apache/iceberg/issues/2788#issuecomment-2207610565 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Drop table doesn't clean up metadata and data files [iceberg]

github-actions[bot] commented on issue #2784: URL: https://github.com/apache/iceberg/issues/2784#issuecomment-2207610460 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Avro inferred mapping issues with nested struct [iceberg]

github-actions[bot] closed issue #2774: Avro inferred mapping issues with nested struct URL: https://github.com/apache/iceberg/issues/2774 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [I] Avro inferred mapping issues with nested struct [iceberg]

github-actions[bot] commented on issue #2774: URL: https://github.com/apache/iceberg/issues/2774#issuecomment-2207610352 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Allow snapshotting iceberg table (create new table based on certain Iceberg snapshot) [iceberg]

github-actions[bot] commented on issue #2481: URL: https://github.com/apache/iceberg/issues/2481#issuecomment-2207609767 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Fix some `long` casting issues [iceberg]

amogh-jahagirdar merged PR #10580: URL: https://github.com/apache/iceberg/pull/10580 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] Fix some `long` casting issues [iceberg]

amogh-jahagirdar commented on code in PR #10580: URL: https://github.com/apache/iceberg/pull/10580#discussion_r1664908939 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/data/FlinkParquetReaders.java: ## @@ -422,7 +422,7 @@ public TimestampData read(TimestampData ign

Re: [PR] REST: refactor OAuth logic into AuthManager Interface [iceberg]

amogh-jahagirdar commented on code in PR #10621: URL: https://github.com/apache/iceberg/pull/10621#discussion_r1664893504 ## core/src/main/java/org/apache/iceberg/rest/auth/AuthSession.java: ## @@ -0,0 +1,28 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

Re: [PR] Add pool_pre_ping param to SQLCatalog and fix echo parsing logic [iceberg-python]

cccs-eric commented on code in PR #886: URL: https://github.com/apache/iceberg-python/pull/886#discussion_r1664857704 ## mkdocs/docs/configuration.md: ## @@ -222,6 +222,12 @@ catalog: uri: sqlite:tmp/pyiceberg.db ``` +| Key | Example

Re: [PR] Use SupportsPrefixOperations for Remove OrphanFile Procedure [iceberg]

schobe commented on PR #7914: URL: https://github.com/apache/iceberg/pull/7914#issuecomment-2207301578 Hi , I am also facing the same issue while running orphan file clean up via Nessie REST. Auto-compaction and snapshot expiry works, but orphan file clean up procecure gives the same error.

Re: [PR] Add pool_pre_ping param to SQLCatalog and fix echo parsing logic [iceberg-python]

Fokko commented on code in PR #886: URL: https://github.com/apache/iceberg-python/pull/886#discussion_r1664804723 ## mkdocs/docs/configuration.md: ## @@ -222,6 +222,12 @@ catalog: uri: sqlite:tmp/pyiceberg.db ``` +| Key | Example

[I] NullPointerException after deleting old partition column [iceberg]

mgmarino opened a new issue, #10626: URL: https://github.com/apache/iceberg/issues/10626 ### Apache Iceberg version 1.5.2 (latest release) ### Query engine Spark ### Please describe the bug 🐞 We have an iceberg table where we have changed the partitioning, g

Re: [PR] Support for Flink's SpeculativeExecution in batch execution mode [iceberg]

pvary commented on PR #10548: URL: https://github.com/apache/iceberg/pull/10548#issuecomment-2207052299 @venkata91: How can we be sure that the tests are exercising the speculative execution code path? Does any of the tests reads some splits multiple times, and use the result of the

Re: [PR] Forward Compatible large_* type support: read as large, write as small [iceberg-python]

syun64 commented on PR #890: URL: https://github.com/apache/iceberg-python/pull/890#issuecomment-2207040653 This is almost ready - I'm going to introduce the flag to all read API as well, so that users can control which schema they are using to read their pyarrow tables, as they may be usin

Re: [PR] Flink: Pre-create fieldGetters to avoid constructing them for each row [iceberg]

pvary commented on PR #10565: URL: https://github.com/apache/iceberg/pull/10565#issuecomment-2207028521 Thanks for the detailed explanation and the PR @fengjiajie! Left a few small comments. As a general rule we create a PR for a single version. Usually the latest. So if there

Re: [PR] Flink: Pre-create fieldGetters to avoid constructing them for each row [iceberg]

pvary commented on code in PR #10565: URL: https://github.com/apache/iceberg/pull/10565#discussion_r1664641504 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/TestHelpers.java: ## @@ -87,7 +87,11 @@ public static RowData copyRowData(RowData from, RowType rowType) {

Re: [PR] Flink: Pre-create fieldGetters to avoid constructing them for each row [iceberg]

pvary commented on code in PR #10565: URL: https://github.com/apache/iceberg/pull/10565#discussion_r1664640913 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/RowDataRecordFactory.java: ## @@ -40,6 +42,14 @@ static TypeSerializer[] createFieldSerializer

Re: [PR] Flink: Pre-create fieldGetters to avoid constructing them for each row [iceberg]

pvary commented on code in PR #10565: URL: https://github.com/apache/iceberg/pull/10565#discussion_r1664639363 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/data/RowDataUtil.java: ## @@ -79,7 +79,11 @@ public static Object convertConstant(Type type, Object value)

Re: [PR] Bump PyIceberg in CI to 0.6.1 [iceberg-python]

Fokko commented on PR #879: URL: https://github.com/apache/iceberg-python/pull/879#issuecomment-2207011586 @kevinjqliu That's an excellent suggestion, just added it 👍 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] Bump mkdocs-material from 9.5.27 to 9.5.28 [iceberg-python]

Fokko merged PR #888: URL: https://github.com/apache/iceberg-python/pull/888 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] DO-NOT-MERGE: Jackson access issue [iceberg]

pvary commented on PR #10460: URL: https://github.com/apache/iceberg/pull/10460#issuecomment-2206994528 > @pvary can we disable CBO in Iceberg Hive test? otherwise, we can not upgrade Hive 2.3.10. Disabling the test would just hide the issue that Hive 2.3.10 is not supported with CBO

Re: [PR] Spark: support read of partition metadata column when table is over 1k [iceberg]

dramaticlly commented on code in PR #10547: URL: https://github.com/apache/iceberg/pull/10547#discussion_r1664606896 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkScanBuilder.java: ## @@ -366,15 +371,56 @@ public void pruneColumns(StructType requestedSch

Re: [PR] Kafka Connect: Commit coordination [iceberg]

fqaiser94 commented on code in PR #10351: URL: https://github.com/apache/iceberg/pull/10351#discussion_r1664602970 ## kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/channel/Coordinator.java: ## @@ -0,0 +1,294 @@ +/* + * Licensed to the Apache Software Found

Re: [PR] Core: Support appending files with different specs [iceberg]

fqaiser94 commented on PR #9860: URL: https://github.com/apache/iceberg/pull/9860#issuecomment-2206964353 Simplified the PR to just multi-partition-append use case. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Apply IntelliJ inspection findings to older Spark + Flink versions [iceberg]

snazy commented on PR #10625: URL: https://github.com/apache/iceberg/pull/10625#issuecomment-2206956499 /cc @nastra -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

Re: [PR] Spark: support read of partition metadata column when table is over 1k [iceberg]

szehon-ho commented on code in PR #10547: URL: https://github.com/apache/iceberg/pull/10547#discussion_r1664595395 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkScanBuilder.java: ## @@ -366,15 +371,56 @@ public void pruneColumns(StructType requestedSchem

Re: [PR] Spark: support read of partition metadata column when table is over 1k [iceberg]

szehon-ho commented on code in PR #10547: URL: https://github.com/apache/iceberg/pull/10547#discussion_r1664595395 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkScanBuilder.java: ## @@ -366,15 +371,56 @@ public void pruneColumns(StructType requestedSchem

Re: [PR] WIP: Initial Support for Spark 4.0 [iceberg]

amogh-jahagirdar commented on PR #10622: URL: https://github.com/apache/iceberg/pull/10622#issuecomment-2206939528 Updated the description to link #10497, this is great to see @huaxingao -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] Add pool_pre_ping param to SQLCatalog and fix echo parsing logic [iceberg-python]

kevinjqliu commented on code in PR #886: URL: https://github.com/apache/iceberg-python/pull/886#discussion_r1664577534 ## tests/catalog/test_sql.py: ## @@ -168,6 +168,39 @@ def test_creation_with_unsupported_uri(catalog_name: str) -> None: SqlCatalog(catalog_name, uri=

Re: [PR] REST: refactor OAuth logic into AuthManager Interface [iceberg]

adutra commented on code in PR #10621: URL: https://github.com/apache/iceberg/pull/10621#discussion_r1664568478 ## core/src/main/java/org/apache/iceberg/rest/auth/OAuth2Manager.java: ## @@ -0,0 +1,241 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

Re: [PR] REST: assume issued token type is access token [iceberg]

danielcweeks commented on PR #10314: URL: https://github.com/apache/iceberg/pull/10314#issuecomment-2206897363 @adutra > An RFC cannot modify another one's structs without officially superseding it. > . . . RFC 8693 as a general rewrite of RFC 6749 section 5.1 I'm not sugg

Re: [PR] Add pool_pre_ping param to SQLCatalog and fix echo parsing logic [iceberg-python]

cccs-eric commented on code in PR #886: URL: https://github.com/apache/iceberg-python/pull/886#discussion_r1664555741 ## tests/catalog/test_sql.py: ## @@ -168,6 +168,39 @@ def test_creation_with_unsupported_uri(catalog_name: str) -> None: SqlCatalog(catalog_name, uri="

Re: [PR] REST: refactor OAuth logic into AuthManager Interface [iceberg]

adutra commented on PR #10621: URL: https://github.com/apache/iceberg/pull/10621#issuecomment-2206891162 > Yes that's the thing I am planning to prototype today. My position is that, if it fits then it fits, if it does not let's not try to fit it. I explored a bit deeper today, includ

Re: [PR] Spark Action to Analyze table [iceberg]

karuppayya commented on code in PR #10288: URL: https://github.com/apache/iceberg/pull/10288#discussion_r1664540073 ## spark/v3.5/build.gradle: ## @@ -59,6 +59,7 @@ project(":iceberg-spark:iceberg-spark-${sparkMajorVersion}_${scalaVersion}") { implementation project(':iceb

Re: [PR] Docs: Update defaults for distribution mode [iceberg]

dramaticlly commented on code in PR #10575: URL: https://github.com/apache/iceberg/pull/10575#discussion_r1664551015 ## docs/docs/configuration.md: ## @@ -67,7 +67,7 @@ Iceberg tables support table properties to configure table behavior, like the de | write.metadata.metrics.co

Re: [PR] feat(visitors): Implement basic boolean expression visitors [iceberg-go]

zeroshade commented on PR #108: URL: https://github.com/apache/iceberg-go/pull/108#issuecomment-2206884654 CC @Fokko @nastra @wolfeidau -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Add pool_pre_ping param to SQLCatalog and fix echo parsing logic [iceberg-python]

cccs-eric commented on code in PR #886: URL: https://github.com/apache/iceberg-python/pull/886#discussion_r1664545780 ## pyiceberg/catalog/sql.py: ## @@ -110,8 +114,12 @@ def __init__(self, name: str, **properties: str): if not (uri_prop := self.properties.get("uri"))

Re: [PR] Add pool_pre_ping param to SQLCatalog and fix echo parsing logic [iceberg-python]

kevinjqliu commented on code in PR #886: URL: https://github.com/apache/iceberg-python/pull/886#discussion_r1664532945 ## pyiceberg/catalog/sql.py: ## @@ -110,8 +114,12 @@ def __init__(self, name: str, **properties: str): if not (uri_prop := self.properties.get("uri")

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

amogh-jahagirdar commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1664436827 ## api/src/main/java/org/apache/iceberg/transforms/Timestamps.java: ## @@ -31,54 +33,160 @@ import org.apache.iceberg.util.DateTimeUtil; import org.apache.ic

Re: [PR] REST: assume issued token type is access token [iceberg]

adutra commented on PR #10314: URL: https://github.com/apache/iceberg/pull/10314#issuecomment-2206868094 > I believe that the intent includes that a client credential exchange could return any of the enumerated token types defined in [section 3](https://www.rfc-editor.org/rfc/rfc8693.html#n

Re: [PR] Add pool_pre_ping param to SQLCatalog and fix echo parsing logic [iceberg-python]

cccs-eric commented on code in PR #886: URL: https://github.com/apache/iceberg-python/pull/886#discussion_r1664515537 ## pyiceberg/catalog/sql.py: ## @@ -110,8 +114,14 @@ def __init__(self, name: str, **properties: str): if not (uri_prop := self.properties.get("uri"))

Re: [PR] Add pool_pre_ping param to SQLCatalog and fix echo parsing logic [iceberg-python]

kevinjqliu commented on code in PR #886: URL: https://github.com/apache/iceberg-python/pull/886#discussion_r1664510351 ## pyiceberg/catalog/sql.py: ## @@ -110,8 +114,14 @@ def __init__(self, name: str, **properties: str): if not (uri_prop := self.properties.get("uri")

Re: [PR] REST: assume issued token type is access token [iceberg]

snazy commented on PR #10314: URL: https://github.com/apache/iceberg/pull/10314#issuecomment-2206790685 [@nastra requested to revert](https://github.com/apache/iceberg/pull/10314#discussion_r1601220154) @adutra 's fix for the client-credentials flow, only RFC 6749 applies here. This part o

Re: [PR] REST: assume issued token type is access token [iceberg]

danielcweeks commented on PR #10314: URL: https://github.com/apache/iceberg/pull/10314#issuecomment-2206759806 I think the application of extensions referenced in RFC 8693 are a little ambiguous due to the following: `RFC 6749` section 4.1 references the response described in [section

Re: [PR] OpenAPI: Express server capabilities via /config endpoint [iceberg]

jackye1995 commented on code in PR #9940: URL: https://github.com/apache/iceberg/pull/9940#discussion_r1664441868 ## open-api/rest-catalog-open-api.yaml: ## @@ -191,7 +220,8 @@ paths: get: tags: -- Catalog API +- tables +- views Review Comm

Re: [PR] OpenAPI: Express server capabilities via /config endpoint [iceberg]

jackye1995 commented on code in PR #9940: URL: https://github.com/apache/iceberg/pull/9940#discussion_r1664416515 ## open-api/rest-catalog-open-api.yaml: ## @@ -28,6 +28,10 @@ info: description: Defines the specification for the first version of the REST Catalog API.

Re: [PR] OpenAPI: Express server capabilities via /config endpoint [iceberg]

jackye1995 commented on code in PR #9940: URL: https://github.com/apache/iceberg/pull/9940#discussion_r1664421615 ## open-api/rest-catalog-open-api.yaml: ## @@ -191,7 +220,8 @@ paths: get: tags: -- Catalog API +- tables +- views Review Comm

Re: [PR] OpenAPI: Express server capabilities via /config endpoint [iceberg]

jackye1995 commented on code in PR #9940: URL: https://github.com/apache/iceberg/pull/9940#discussion_r1664416515 ## open-api/rest-catalog-open-api.yaml: ## @@ -28,6 +28,10 @@ info: description: Defines the specification for the first version of the REST Catalog API.

Re: [PR] OpenAPI: Express server capabilities via /config endpoint [iceberg]

jackye1995 commented on code in PR #9940: URL: https://github.com/apache/iceberg/pull/9940#discussion_r1664416515 ## open-api/rest-catalog-open-api.yaml: ## @@ -28,6 +28,10 @@ info: description: Defines the specification for the first version of the REST Catalog API.

Re: [PR] REST: refactor OAuth logic into AuthManager Interface [iceberg]

flyrain commented on PR #10621: URL: https://github.com/apache/iceberg/pull/10621#issuecomment-2206504324 Thanks @jackye1995 for pinging me. Will take a look today. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

nastra commented on PR #9008: URL: https://github.com/apache/iceberg/pull/9008#issuecomment-2206491863 Hey @jacobmarble thanks for being patient here. There's no particular concern but I've been busy with other things and it was difficult to find a big block of time to re-review this PR. I'

Re: [PR] REST: refactor OAuth logic into AuthManager Interface [iceberg]

jackye1995 commented on PR #10621: URL: https://github.com/apache/iceberg/pull/10621#issuecomment-2206480833 > How are you going to migrate RESTSigV4Signer? Yes that's the thing I am planning to prototype today. My position is that, if it fits then it fits, if it does not let's not tr

Re: [I] Spark: Support alter partition in V2 Catalog SparkCatalog Class [iceberg]

psilos commented on issue #3558: URL: https://github.com/apache/iceberg/issues/3558#issuecomment-2206288118 hi @huaxingao I was probably referring to the later one, adding it on the iceberg extended syntax. -- This is an automated message from the Apache Git Service. To respond to the me

Re: [PR] Kafka Connect: Commit coordination [iceberg]

fqaiser94 commented on code in PR #10351: URL: https://github.com/apache/iceberg/pull/10351#discussion_r1663397086 ## kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/channel/Worker.java: ## @@ -0,0 +1,127 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] Kafka Connect: Commit coordination [iceberg]

fqaiser94 commented on code in PR #10351: URL: https://github.com/apache/iceberg/pull/10351#discussion_r1664211281 ## kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/IcebergSinkTask.java: ## @@ -0,0 +1,103 @@ +/* + * Licensed to the Apache Software Foundatio

Re: [PR] Bump pypa/cibuildwheel from 2.19.1 to 2.19.2 [iceberg-python]

Fokko merged PR #889: URL: https://github.com/apache/iceberg-python/pull/889 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Flink: Pre-create fieldGetters to avoid constructing them for each row [iceberg]

Fokko commented on PR #10565: URL: https://github.com/apache/iceberg/pull/10565#issuecomment-2206068288 @fengjiajie Rerunning the failed tests 👍 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

  1   2   >