Re: [PR] Fix `Table.scan` to enable case sensitive argument [iceberg-python]

jiakai-li commented on PR #1423: URL: https://github.com/apache/iceberg-python/pull/1423#issuecomment-2542657647 Hey guys, thanks a lot for your kind guidance and great suggestions. I've updated the PR to: - Enable `Table.delete` and `Table.overwrite` operations to control case-sensitivi

Re: [PR] Fix `Table.scan` to enable case sensitive argument [iceberg-python]

jiakai-li commented on code in PR #1423: URL: https://github.com/apache/iceberg-python/pull/1423#discussion_r1884721249 ## tests/table/test_init.py: ## @@ -310,6 +310,19 @@ def test_table_scan_row_filter(table_v2: Table) -> None: assert scan.filter(EqualTo("x", 10)).filter(

Re: [PR] IO Implementation using Go CDK [iceberg-go]

dwilson1988 commented on PR #176: URL: https://github.com/apache/iceberg-go/pull/176#issuecomment-2542660861 @zeroshade - I'll take a look this weekend! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Spark: Change Delete granularity to file for Spark 3.5 [iceberg]

aokolnychyi commented on code in PR #11478: URL: https://github.com/apache/iceberg/pull/11478#discussion_r1884625538 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkWriteConf.java: ## @@ -719,7 +719,7 @@ public DeleteGranularity deleteGranularity() {

Re: [PR] GCS: Suppress JavaUtilDate in OAuth2RefreshCredentialsHandler [iceberg]

nastra merged PR #11773: URL: https://github.com/apache/iceberg/pull/11773 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Auth Manager API part 1: HTTPRequest, HTTPHeader [iceberg]

nastra commented on code in PR #11769: URL: https://github.com/apache/iceberg/pull/11769#discussion_r1883568018 ## core/src/test/java/org/apache/iceberg/rest/TestHTTPHeaders.java: ## @@ -0,0 +1,167 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] Auth Manager API part 1: HTTPRequest, HTTPHeader [iceberg]

nastra commented on code in PR #11769: URL: https://github.com/apache/iceberg/pull/11769#discussion_r1883571471 ## core/src/test/java/org/apache/iceberg/rest/TestHTTPHeaders.java: ## @@ -0,0 +1,167 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] Auth Manager API part 1: HTTPRequest, HTTPHeader [iceberg]

nastra commented on code in PR #11769: URL: https://github.com/apache/iceberg/pull/11769#discussion_r1883847591 ## core/src/main/java/org/apache/iceberg/rest/HTTPHeaders.java: ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more con

Re: [PR] Auth Manager API part 1: HTTPRequest, HTTPHeader [iceberg]

nastra commented on code in PR #11769: URL: https://github.com/apache/iceberg/pull/11769#discussion_r1883560954 ## core/src/main/java/org/apache/iceberg/rest/HTTPHeaders.java: ## @@ -0,0 +1,168 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more con

Re: [PR] Auth Manager API part 1: HTTPRequest, HTTPHeader [iceberg]

nastra commented on code in PR #11769: URL: https://github.com/apache/iceberg/pull/11769#discussion_r1883564062 ## core/src/main/java/org/apache/iceberg/rest/HTTPHeaders.java: ## @@ -0,0 +1,168 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more con

Re: [PR] Auth Manager API part 1: HTTPRequest, HTTPHeader [iceberg]

adutra commented on code in PR #11769: URL: https://github.com/apache/iceberg/pull/11769#discussion_r1883565088 ## core/src/main/java/org/apache/iceberg/rest/HTTPRequest.java: ## @@ -0,0 +1,91 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more cont

Re: [PR] Auth Manager API part 1: HTTPRequest, HTTPHeader [iceberg]

adutra commented on code in PR #11769: URL: https://github.com/apache/iceberg/pull/11769#discussion_r1883564166 ## core/src/main/java/org/apache/iceberg/rest/HTTPHeaders.java: ## @@ -0,0 +1,168 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more con

Re: [PR] Auth Manager API part 1: HTTPRequest, HTTPHeader [iceberg]

nastra commented on code in PR #11769: URL: https://github.com/apache/iceberg/pull/11769#discussion_r1883809126 ## core/src/test/java/org/apache/iceberg/rest/TestHTTPHeaders.java: ## @@ -95,6 +100,9 @@ void addIfAbsentHTTPHeader() { "header1", List.of("value1a",

Re: [PR] Auth Manager API part 1: HTTPRequest, HTTPHeader [iceberg]

adutra commented on code in PR #11769: URL: https://github.com/apache/iceberg/pull/11769#discussion_r1883811731 ## core/src/main/java/org/apache/iceberg/rest/HTTPHeaders.java: ## @@ -0,0 +1,168 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more con

Re: [PR] Auth Manager API part 1: HTTPRequest, HTTPHeader [iceberg]

adutra commented on code in PR #11769: URL: https://github.com/apache/iceberg/pull/11769#discussion_r1883813563 ## core/src/test/java/org/apache/iceberg/rest/TestHTTPHeaders.java: ## @@ -95,6 +100,9 @@ void addIfAbsentHTTPHeader() { "header1", List.of("value1a",

Re: [PR] Auth Manager API part 1: HTTPRequest, HTTPHeader [iceberg]

nastra commented on code in PR #11769: URL: https://github.com/apache/iceberg/pull/11769#discussion_r1883806469 ## core/src/main/java/org/apache/iceberg/rest/HTTPHeaders.java: ## @@ -0,0 +1,168 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more con

Re: [PR] Docs: add note for `day` transform [iceberg]

Fokko commented on code in PR #11749: URL: https://github.com/apache/iceberg/pull/11749#discussion_r1883817906 ## format/spec.md: ## @@ -454,7 +454,7 @@ Partition field IDs must be reused if an existing partition spec contains an equ | **`truncate[W]`** | Value truncated to wi

Re: [I] REST catalog doesn't return old history if we execute `CREATE OR REPLACE TABLE` statement [iceberg]

ebyhr commented on issue #11777: URL: https://github.com/apache/iceberg/issues/11777#issuecomment-2540941152 I think we need to add a new `MetadataUpdate` class (e.g. `ResetMainBranch`) or modify `RemoveSnapshotRef` to provide an option to clear snapshot-logs or not. -- This is an autom

Re: [PR] Auth Manager API part 1: HTTPRequest, HTTPHeader [iceberg]

adutra commented on code in PR #11769: URL: https://github.com/apache/iceberg/pull/11769#discussion_r1883626591 ## core/src/main/java/org/apache/iceberg/rest/HTTPHeaders.java: ## @@ -0,0 +1,168 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more con

Re: [PR] feat: eagerly project the arrow schema to scope out non-selected fields [iceberg-rust]

Xuanwo commented on code in PR #785: URL: https://github.com/apache/iceberg-rust/pull/785#discussion_r1883575270 ## crates/iceberg/src/arrow/reader.rs: ## @@ -1180,4 +1194,62 @@ mod tests { assert_eq!(visitor.field_ids, expected); } + +#[test] +fn test_ar

Re: [PR] refactor: avoid async_trait for FileRead and provide object safe dyn methods [iceberg-rust]

Xuanwo commented on PR #761: URL: https://github.com/apache/iceberg-rust/pull/761#issuecomment-2540901488 I have similar comments like https://github.com/apache/iceberg-rust/pull/760#issuecomment-2540446608 -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [PR] refactor: Move puffin crate contents inside iceberg crate [iceberg-rust]

Xuanwo merged PR #789: URL: https://github.com/apache/iceberg-rust/pull/789 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] Move puffin crate contents inside iceberg crate [iceberg-rust]

Xuanwo commented on PR #789: URL: https://github.com/apache/iceberg-rust/pull/789#issuecomment-2540906883 Thank you, @Fokko, for the feedback. Since neither of us has a strong opinion on this and the suggestion comes from @liurenjie1024 with agreement from @fqaiser94, I believe there’s no n

Re: [PR] Doc: Add staus page for different implementations. [iceberg]

Fokko commented on code in PR #11772: URL: https://github.com/apache/iceberg/pull/11772#discussion_r1883511101 ## site/docs/status.md: ## @@ -0,0 +1,362 @@ +--- +title: "Implementation Status" +--- + + +# Implementations Status + +Apache iceberg now has implementations of the ic

[I] Failed to read iceberg TPCH generated by snowflake [iceberg-rust]

Xuanwo opened a new issue, #790: URL: https://github.com/apache/iceberg-rust/issues/790 The table generated in this way: ```sql CREATE OR REPLACE ICEBERG TABLE lineitem ( l_orderkey BIGINT, l_partkeyBIGINT, l_suppkeyBIGINT, l_linenum

Re: [PR] Retry object store reads on temporary errors. [iceberg-rust]

ryzhyk commented on PR #788: URL: https://github.com/apache/iceberg-rust/pull/788#issuecomment-2540837170 > Comparing to add retry layer for every service, how about adding it here? > > https://github.com/apache/iceberg-rust/blob/e073e75bf51fd4e8999e9d99a9986a4380afd0bc/crates/iceberg

[PR] feat: Print debug source error instead [iceberg-rust]

Xuanwo opened a new pull request, #792: URL: https://github.com/apache/iceberg-rust/pull/792 This PR will print debug source error instead for better understanding what happened inside. Fix error like: ``` iceberg table scan plan: Unexpected => Failure in doing io operation

Re: [I] Geospatial Support [iceberg]

jaakla commented on issue #10260: URL: https://github.com/apache/iceberg/issues/10260#issuecomment-2541099197 IMHO the concept of SPATIAL_REF_SYS, (and the other geo metadata tables) is a bit legacy from the times when (1) storage was really expensive and (2) databases did not have proper w

Re: [PR] feat: eagerly project the arrow schema to scope out non-selected fields [iceberg-rust]

Xuanwo merged PR #785: URL: https://github.com/apache/iceberg-rust/pull/785 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] Flink: Avoid RANGE mode broken chain when write parallelism changes [iceberg]

pvary commented on PR #11702: URL: https://github.com/apache/iceberg/pull/11702#issuecomment-2541180202 @huyuanfeng2018: Could we have a unit test for this to avoid future regressions? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[PR] chore: Add more debug message inside error [iceberg-rust]

Xuanwo opened a new pull request, #793: URL: https://github.com/apache/iceberg-rust/pull/793 This PR will add more debug message inside error from rest catalog so we can avoid error message like: ``` 1107=> Iceberg catalog load failed: ReadTableDataError. Code: 1107, Text = Iceb

Re: [I] Eagerly project arrow schema when calculating the parquet `ProjectionMask` [iceberg-rust]

Xuanwo closed issue #784: Eagerly project arrow schema when calculating the parquet `ProjectionMask` URL: https://github.com/apache/iceberg-rust/issues/784 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[PR] add Status data structure [iceberg-cpp]

zhjwpku opened a new pull request, #8: URL: https://github.com/apache/iceberg-cpp/pull/8 This is adpated from Apache Arrow, this is a pre step before integret the Result data structure. -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

Re: [PR] Add basic CI to build iceberg and example [iceberg-cpp]

raulcd commented on code in PR #7: URL: https://github.com/apache/iceberg-cpp/pull/7#discussion_r1883773249 ## .github/workflows/test.yml: ## @@ -0,0 +1,95 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE

Re: [PR] Add basic CI to build iceberg and example [iceberg-cpp]

raulcd commented on PR #7: URL: https://github.com/apache/iceberg-cpp/pull/7#issuecomment-2541213585 Run is successful as seen here: https://github.com/raulcd/iceberg-cpp/actions/runs/12314556083 @gaborkaszab @wgtmac if you are :+1: with this it probably can be merged -- This is an

[I] Path/file URI handling [iceberg]

jdockerty opened a new issue, #11778: URL: https://github.com/apache/iceberg/issues/11778 ### Query engine N/A ### Question Are the path URIs intended to be fully addressable, such as within the `location` or `metadata-file` fields? From the example given in the

[I] Encountering `429 Too Many Requests` error every time when accessing Snowflake's Polaris catalog [iceberg-rust]

Xuanwo opened a new issue, #791: URL: https://github.com/apache/iceberg-rust/issues/791 I found iceberg-rust always raise `429 Too Many Requests` while calling Snowflake's Polaris catalog: ```rust 1107=> Iceberg catalog load failed: ReadTableDataError. Code: 1107, Text = Iceberg

Re: [PR] feat: Implement Decimal from/to bytes represents [iceberg-rust]

liurenjie1024 merged PR #665: URL: https://github.com/apache/iceberg-rust/pull/665 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] feat: eagerly project the arrow schema to scope out non-selected fields [iceberg-rust]

gruuya commented on code in PR #785: URL: https://github.com/apache/iceberg-rust/pull/785#discussion_r1883665954 ## crates/iceberg/src/arrow/reader.rs: ## @@ -1180,4 +1194,62 @@ mod tests { assert_eq!(visitor.field_ids, expected); } + +#[test] +fn test_ar

Re: [PR] feat: eagerly project the arrow schema to scope out non-selected fields [iceberg-rust]

gruuya commented on code in PR #785: URL: https://github.com/apache/iceberg-rust/pull/785#discussion_r1883721441 ## crates/iceberg/src/arrow/reader.rs: ## @@ -1180,4 +1194,62 @@ mod tests { assert_eq!(visitor.field_ids, expected); } + +#[test] +fn test_ar

Re: [I] Change the glue catalog doc ( `list_tables` method only return Iceberg Tables ) [iceberg-python]

omkenge closed issue #1291: Change the glue catalog doc ( `list_tables` method only return Iceberg Tables ) URL: https://github.com/apache/iceberg-python/issues/1291 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] feat: eagerly project the arrow schema to scope out non-selected fields [iceberg-rust]

gruuya commented on code in PR #785: URL: https://github.com/apache/iceberg-rust/pull/785#discussion_r1883721441 ## crates/iceberg/src/arrow/reader.rs: ## @@ -1180,4 +1194,62 @@ mod tests { assert_eq!(visitor.field_ids, expected); } + +#[test] +fn test_ar

Re: [PR] add Status data structure [iceberg-cpp]

pitrou commented on code in PR #8: URL: https://github.com/apache/iceberg-cpp/pull/8#discussion_r1883865597 ## CMakeLists.txt: ## @@ -56,6 +64,11 @@ add_subdirectory(api) add_subdirectory(src) if(ICEBERG_BUILD_TESTS) + fetchcontent_declare(googletest Review Comment: Thi

Re: [I] Failed to read iceberg TPCH generated by snowflake [iceberg-rust]

Fokko commented on issue #790: URL: https://github.com/apache/iceberg-rust/issues/790#issuecomment-2540844729 Yes, we should support this. I think https://github.com/apache/iceberg-rust/pull/786 fixes this. Looking at the schema, only the `decimal` ones are different. And we support

Re: [I] Failed to read iceberg TPCH generated by snowflake [iceberg-rust]

Xuanwo commented on issue #790: URL: https://github.com/apache/iceberg-rust/issues/790#issuecomment-2540849063 Thank you @Fokko for this. I will review the PR accordingly! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [PR] Core: Fix loading a table in CachingCatalog with metadata table name [iceberg]

gaborkaszab commented on PR #11738: URL: https://github.com/apache/iceberg/pull/11738#issuecomment-2540849458 I see you have made changes previously in CachingCatalog, @rdblue @aokolnychyi @RussellSpitzer . Would you mind taking a look at this PR? -- This is an automated message from the

Re: [PR] feat: Allow for schema evolution [iceberg-rust]

Xuanwo commented on PR #786: URL: https://github.com/apache/iceberg-rust/pull/786#issuecomment-2540855953 Hi @Fokko, I'm following up on issue #790. It appears this PR doesn't address the scenario I described in #790, where the underlying parquet schema differs from the declared one.

Re: [PR] Auth Manager API part 1: HTTPRequest, HTTPHeader [iceberg]

nastra commented on code in PR #11769: URL: https://github.com/apache/iceberg/pull/11769#discussion_r1883552348 ## core/src/main/java/org/apache/iceberg/rest/HTTPRequest.java: ## @@ -0,0 +1,91 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more cont

Re: [PR] feat: Allow for schema evolution [iceberg-rust]

Fokko commented on PR #786: URL: https://github.com/apache/iceberg-rust/pull/786#issuecomment-2540869512 This is pretty similar. With this PR we first write an int, and then read it as a long, which is a valid schema evolution. In the case of #790 we try to read a narrower decimal from the

Re: [PR] feat: Allow for schema evolution [iceberg-rust]

Xuanwo commented on PR #786: URL: https://github.com/apache/iceberg-rust/pull/786#issuecomment-2540873200 Thank you for the explanation! This PR is good enough for me. Let's proceed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

Re: [PR] feat: Allow for schema evolution [iceberg-rust]

Xuanwo merged PR #786: URL: https://github.com/apache/iceberg-rust/pull/786 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] feat: Implement Decimal from/to bytes represents [iceberg-rust]

Xuanwo commented on PR #665: URL: https://github.com/apache/iceberg-rust/pull/665#issuecomment-2540876657 cc @Fokko, would you like to take another? I believe it's good for merging now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

Re: [PR] Retry object store reads on temporary errors. [iceberg-rust]

Xuanwo merged PR #788: URL: https://github.com/apache/iceberg-rust/pull/788 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [I] Data loss bug in MergeIntoCommand [iceberg]

RussellSpitzer commented on issue #11765: URL: https://github.com/apache/iceberg/issues/11765#issuecomment-2541337509 You'll need to elaborate a bit more. What does "losing some of the increment new data writes" mean? Can you give an example? We can't really debug the generic case si

Re: [I] Kafka connect iceberg connector seems to stop after having processed all messages in Kafka topic [iceberg]

thjaeckle commented on issue #11703: URL: https://github.com/apache/iceberg/issues/11703#issuecomment-2541339588 I found out the configuration mistake I did.. Turns out that `iceberg.connect.group-id` **must** match the configured Kafka Connect `consumer.group.id`. By default, this

Re: [PR] Auth Manager API part 1: HTTPRequest, HTTPHeader [iceberg]

nastra commented on code in PR #11769: URL: https://github.com/apache/iceberg/pull/11769#discussion_r1883856330 ## core/src/test/java/org/apache/iceberg/rest/TestHTTPRequest.java: ## @@ -0,0 +1,138 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [I] Data loss bug in MergeIntoCommand [iceberg]

RussellSpitzer commented on issue #11765: URL: https://github.com/apache/iceberg/issues/11765#issuecomment-2541339227 It would also be very helpful to know how you are determining there is data loss -- This is an automated message from the Apache Git Service. To respond to the message, pl

Re: [I] Kafka connect iceberg connector seems to stop after having processed all messages in Kafka topic [iceberg]

thjaeckle closed issue #11703: Kafka connect iceberg connector seems to stop after having processed all messages in Kafka topic URL: https://github.com/apache/iceberg/issues/11703 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [PR] add Status data structure [iceberg-cpp]

pitrou commented on PR #8: URL: https://github.com/apache/iceberg-cpp/pull/8#issuecomment-2541348036 Also @zhjwpku can you make sure you proof-read your PR description? There are spell checkers that can help. -- This is an automated message from the Apache Git Service. To respond to the m

Re: [PR] API: Support removeUnusedSpecs in ExpireSnapshots [iceberg]

advancedxy commented on PR #10755: URL: https://github.com/apache/iceberg/pull/10755#issuecomment-2541354779 @danielcweeks would you mind to take a look at this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Fix ParallelIterable deadlock [iceberg]

osscm commented on code in PR #11781: URL: https://github.com/apache/iceberg/pull/11781#discussion_r1884308287 ## core/src/main/java/org/apache/iceberg/util/ParallelIterable.java: ## @@ -257,17 +257,17 @@ private static class Task implements Supplier>>, Closeable { @Overri

[I] HiveCatalog incorrectly uses FileIOTracker [iceberg]

tom-s-powell opened a new issue, #11783: URL: https://github.com/apache/iceberg/issues/11783 ### Apache Iceberg version 1.7.1 (latest release) ### Query engine Spark ### Please describe the bug 🐞 Encountering an issue with `HiveCatalog` and `S3FileIO`. I bel

Re: [PR] IO Implementation using Go CDK [iceberg-go]

zeroshade commented on PR #176: URL: https://github.com/apache/iceberg-go/pull/176#issuecomment-2542399559 @loicalleyne following pyiceberg's example, I've added an option to force virtual addressing. That work for you? -- This is an automated message from the Apache Git Service. To respo

Re: [PR] Feat: support aliyun oss backend. [iceberg-go]

zeroshade commented on PR #216: URL: https://github.com/apache/iceberg-go/pull/216#issuecomment-2542403956 This seems generally good to me. Does Aliyun have something similar to how MinIO works for S3 that can be added to the integration tests to have CI testing the backend? i.e. is there a

Re: [PR] Spark: Read DVs when reading from .position_deletes table [iceberg]

aokolnychyi commented on code in PR #11657: URL: https://github.com/apache/iceberg/pull/11657#discussion_r1884381201 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/DVIterator.java: ## @@ -0,0 +1,108 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

Re: [PR] Core: Add TableUtil to provide access to a table's format version [iceberg]

aokolnychyi commented on code in PR #11620: URL: https://github.com/apache/iceberg/pull/11620#discussion_r1884367833 ## core/src/main/java/org/apache/iceberg/TableUtil.java: ## @@ -0,0 +1,40 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contri

Re: [PR] Core: Add TableUtil to provide access to a table's format version [iceberg]

aokolnychyi commented on code in PR #11620: URL: https://github.com/apache/iceberg/pull/11620#discussion_r1884369789 ## core/src/main/java/org/apache/iceberg/SerializableTable.java: ## @@ -158,6 +160,21 @@ public Map properties() { return properties; } + public int fo

Re: [PR] feat: TableMetadata Statistic Files [iceberg-rust]

c-thiel commented on code in PR #799: URL: https://github.com/apache/iceberg-rust/pull/799#discussion_r1884298259 ## crates/iceberg/src/catalog/mod.rs: ## @@ -446,6 +446,30 @@ pub enum TableUpdate { /// Properties to remove removals: Vec, }, +/// Set s

Re: [PR] Hive: Add Hive 4 support and remove Hive runtime [iceberg]

rdblue commented on code in PR #11750: URL: https://github.com/apache/iceberg/pull/11750#discussion_r1884343692 ## gradle.properties: ## @@ -18,8 +18,8 @@ jmhJsonOutputPath=build/reports/jmh/results.json jmhIncludeRegex=.* systemProp.defaultFlinkVersions=1.20 systemProp.known

[PR] feat: TableMetadata Statistics [iceberg-rust]

c-thiel opened a new pull request, #799: URL: https://github.com/apache/iceberg-rust/pull/799 Adds `StatisticFile` and `PartitionStatisticsFile` to spec, builder and REST TableUpdate. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] IO Implementation using Go CDK [iceberg-go]

loicalleyne commented on PR #176: URL: https://github.com/apache/iceberg-go/pull/176#issuecomment-2542287393 Is it intended to not provide the choice between virtual hosted bucket addressing and path-style addressing? LGTM otherwise - the tests are passing :) -- This is an automated m

Re: [PR] Fix ParallelIterable deadlock [iceberg]

sopel39 commented on code in PR #11781: URL: https://github.com/apache/iceberg/pull/11781#discussion_r1884520727 ## core/src/main/java/org/apache/iceberg/util/ParallelIterable.java: ## @@ -257,17 +257,17 @@ private static class Task implements Supplier>>, Closeable { @Over

Re: [PR] Add plan tasks for TableScan [iceberg-python]

corleyma commented on code in PR #1427: URL: https://github.com/apache/iceberg-python/pull/1427#discussion_r1884523394 ## pyiceberg/table/__init__.py: ## @@ -1229,7 +1240,8 @@ def with_case_sensitive(self: S, case_sensitive: bool = True) -> S: class ScanTask(ABC): -pas

Re: [PR] Add plan tasks for TableScan [iceberg-python]

corleyma commented on code in PR #1427: URL: https://github.com/apache/iceberg-python/pull/1427#discussion_r1884524371 ## pyiceberg/table/__init__.py: ## @@ -1253,6 +1265,22 @@ def __init__( self.start = start or 0 self.length = length or data_file.file_size_in

Re: [PR] Fix ParallelIterable deadlock [iceberg]

RussellSpitzer commented on code in PR #11781: URL: https://github.com/apache/iceberg/pull/11781#discussion_r1884527019 ## core/src/main/java/org/apache/iceberg/util/ParallelIterable.java: ## @@ -257,17 +257,17 @@ private static class Task implements Supplier>>, Closeable {

Re: [I] Data loss bug in MergeIntoCommand [iceberg]

RussellSpitzer commented on issue #11765: URL: https://github.com/apache/iceberg/issues/11765#issuecomment-2542309392 > However, sir, I might have discovered some issues. When executing the COW-MERGE-INTO command, Spark needs to use the ods_table twice. The first time is to match data

Re: [PR] add .gitignore [iceberg-cpp]

pitrou commented on code in PR #9: URL: https://github.com/apache/iceberg-cpp/pull/9#discussion_r1884280319 ## .gitignore: ## @@ -0,0 +1,18 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distrib

Re: [PR] feat(puffin): Parse Puffin FileMetadata [iceberg-rust]

c-thiel commented on PR #765: URL: https://github.com/apache/iceberg-rust/pull/765#issuecomment-2541965685 @fqaiser94, just added the higher level statistic files in https://github.com/apache/iceberg-rust/pull/799 FYI. I would guess you would end up building those soon too. -- This is an

Re: [PR] Core: Add Variant implementation to read serialized objects [iceberg]

rdblue commented on PR #11415: URL: https://github.com/apache/iceberg/pull/11415#issuecomment-2542017239 The Spark failures are a port conflict. I think it's unrelated to these changes. We'll see the next time CI runs (I'm sure we'll have more changes to trigger them) -- This is an autom

Re: [PR] Add plan tasks for TableScan [iceberg-python]

corleyma commented on code in PR #1427: URL: https://github.com/apache/iceberg-python/pull/1427#discussion_r1884533676 ## pyiceberg/table/__init__.py: ## @@ -1423,6 +1451,66 @@ def plan_files(self) -> Iterable[FileScanTask]: for data_entry in data_entries ]

Re: [PR] Add plan tasks for TableScan [iceberg-python]

corleyma commented on code in PR #1427: URL: https://github.com/apache/iceberg-python/pull/1427#discussion_r1884536195 ## pyiceberg/table/__init__.py: ## @@ -1423,6 +1451,66 @@ def plan_files(self) -> Iterable[FileScanTask]: for data_entry in data_entries ]

Re: [PR] Fix ParallelIterable deadlock [iceberg]

sopel39 commented on code in PR #11781: URL: https://github.com/apache/iceberg/pull/11781#discussion_r1884535343 ## core/src/main/java/org/apache/iceberg/util/ParallelIterable.java: ## @@ -257,17 +257,17 @@ private static class Task implements Supplier>>, Closeable { @Over

Re: [PR] Add plan tasks for TableScan [iceberg-python]

corleyma commented on code in PR #1427: URL: https://github.com/apache/iceberg-python/pull/1427#discussion_r1884536195 ## pyiceberg/table/__init__.py: ## @@ -1423,6 +1451,66 @@ def plan_files(self) -> Iterable[FileScanTask]: for data_entry in data_entries ]

[I] Hivemetastore unable to create hive lock after upgrading from hivemetastore 3.1.3 to 4.0.0 during iceberg operations [iceberg]

mAlf1999 opened a new issue, #11784: URL: https://github.com/apache/iceberg/issues/11784 ### Apache Iceberg version 1.6.0 ### Query engine Spark ### Please describe the bug 🐞 We are currently using iceberg version 1.6.0 and have been successfully using it a

Re: [PR] IO Implementation using Go CDK [iceberg-go]

zeroshade commented on PR #176: URL: https://github.com/apache/iceberg-go/pull/176#issuecomment-2542213486 @loicalleyne can you take a look at the latest changes I made here? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Doc: Add staus page for different implementations. [iceberg]

RussellSpitzer commented on code in PR #11772: URL: https://github.com/apache/iceberg/pull/11772#discussion_r1884491976 ## site/docs/status.md: ## @@ -0,0 +1,362 @@ +--- +title: "Implementation Status" +--- + + +# Implementations Status + +Apache iceberg now has implementations

Re: [PR] Doc: Add staus page for different implementations. [iceberg]

RussellSpitzer commented on code in PR #11772: URL: https://github.com/apache/iceberg/pull/11772#discussion_r1884493485 ## site/docs/status.md: ## @@ -0,0 +1,362 @@ +--- +title: "Implementation Status" +--- + + +# Implementations Status + +Apache iceberg now has implementations

Re: [PR] Doc: Add staus page for different implementations. [iceberg]

RussellSpitzer commented on code in PR #11772: URL: https://github.com/apache/iceberg/pull/11772#discussion_r1884494396 ## site/docs/status.md: ## @@ -0,0 +1,362 @@ +--- +title: "Implementation Status" +--- + + +# Implementations Status + +Apache iceberg now has implementations

Re: [PR] Doc: Add staus page for different implementations. [iceberg]

RussellSpitzer commented on code in PR #11772: URL: https://github.com/apache/iceberg/pull/11772#discussion_r1884494856 ## site/docs/status.md: ## @@ -0,0 +1,362 @@ +--- +title: "Implementation Status" +--- + + +# Implementations Status + +Apache iceberg now has implementations

Re: [PR] ci(infra): Remove sha256 [iceberg-go]

zeroshade merged PR #226: URL: https://github.com/apache/iceberg-go/pull/226 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] IO Implementation using Go CDK [iceberg-go]

loicalleyne commented on PR #176: URL: https://github.com/apache/iceberg-go/pull/176#issuecomment-2542428675 LGTM 👍 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

Re: [PR] IO Implementation using Go CDK [iceberg-go]

zeroshade commented on PR #176: URL: https://github.com/apache/iceberg-go/pull/176#issuecomment-2542436725 @dwilson1988 When you get a chance, can you take a look at the changes I made here. I liked your thought on isolating things, but there was still a bunch of specific options for partic

Re: [PR] feat: TableMetadata Statistic Files [iceberg-rust]

c-thiel commented on code in PR #799: URL: https://github.com/apache/iceberg-rust/pull/799#discussion_r1884798642 ## crates/iceberg/src/spec/table_metadata.rs: ## @@ -158,11 +160,15 @@ pub struct TableMetadata { /// writers, but is not used when reading because reads use th

Re: [PR] fix: gurantee the deserialize order of struct is same as the struct type [iceberg-rust]

Xuanwo commented on code in PR #795: URL: https://github.com/apache/iceberg-rust/pull/795#discussion_r1884868240 ## crates/iceberg/src/spec/values.rs: ## @@ -3439,11 +3443,13 @@ mod tests { "bar".to_string(), ))), None, +

Re: [PR] fix: set key_metadata to Null by default [iceberg-rust]

feniljain commented on code in PR #800: URL: https://github.com/apache/iceberg-rust/pull/800#discussion_r1884869867 ## crates/iceberg/src/expr/visitors/expression_evaluator.rs: ## @@ -338,7 +338,7 @@ mod tests { nan_value_counts: HashMap::new(), lower_b

Re: [PR] fix: set key_metadata to Null by default [iceberg-rust]

Xuanwo commented on code in PR #800: URL: https://github.com/apache/iceberg-rust/pull/800#discussion_r1884868013 ## crates/iceberg/src/expr/visitors/expression_evaluator.rs: ## @@ -338,7 +338,7 @@ mod tests { nan_value_counts: HashMap::new(), lower_boun

Re: [PR] Doc: Add staus page for different implementations. [iceberg]

liurenjie1024 commented on code in PR #11772: URL: https://github.com/apache/iceberg/pull/11772#discussion_r1884790587 ## site/docs/status.md: ## @@ -0,0 +1,362 @@ +--- +title: "Implementation Status" +--- + + +# Implementations Status + +Apache iceberg now has implementations o

Re: [PR] fix: gurantee the deserialize order of struct is same as the struct type [iceberg-rust]

Xuanwo commented on code in PR #795: URL: https://github.com/apache/iceberg-rust/pull/795#discussion_r1884848752 ## crates/iceberg/src/spec/values.rs: ## @@ -3439,11 +3443,13 @@ mod tests { "bar".to_string(), ))), None, +

Re: [PR] feat: support to append delete type data file [iceberg-rust]

ZENOTME commented on PR #798: URL: https://github.com/apache/iceberg-rust/pull/798#issuecomment-2542902331 Sorry, I think I have some misunderstanding here since the action which support to append data file and delete file is RowDelta.🤔 So I guess what we need is right: 1. MergingSnapsho

Re: [PR] fix: gurantee the deserialize order of struct is same as the struct type [iceberg-rust]

ZENOTME commented on code in PR #795: URL: https://github.com/apache/iceberg-rust/pull/795#discussion_r1884853773 ## crates/iceberg/src/spec/values.rs: ## @@ -3439,11 +3443,13 @@ mod tests { "bar".to_string(), ))), None, +

  1   2   >