Re: [I] discuss: Split iceberg-rust high level API into a mini engine instead [iceberg-rust]

sdd commented on issue #1036: URL: https://github.com/apache/iceberg-rust/issues/1036#issuecomment-2696442323 Hi @Xuanwo. I find myself agreeing in part with @a-agmon - whilst conceptually I agree that re-architecting into an `iceberg-core` and an `iceberg-engine-lite` does provide the oppo

Re: [PR] support create table like in flink catalog [iceberg]

pvary commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1978740329 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalog.java: ## @@ -97,17 +97,20 @@ public class FlinkCatalog extends AbstractCatalog { private fin

Re: [PR] Spark-3.5: Add spark action to compute partition stats [iceberg]

ajantha-bhat commented on PR #12450: URL: https://github.com/apache/iceberg/pull/12450#issuecomment-2696345701 @huaxingao, @karuppayya, @flyrain, @aokolnychyi, @deniskuzZ, @gszadovszky, @pvary : Tagging the people who might be intersted in this feature and review. Will add a ca

Re: [PR] Retry on NoSuchNamespaceException not found in rename table for rest catalog [iceberg]

huan233usc commented on code in PR #12159: URL: https://github.com/apache/iceberg/pull/12159#discussion_r1978663899 ## core/src/test/java/org/apache/iceberg/rest/TestRESTCatalog.java: ## @@ -2506,6 +2506,29 @@ public void testNamespaceExistsViaHEADRequest() { any())

Re: [I] The identity partition path of timestamp type is inconsistent with java api [iceberg-python]

sharkdtu commented on issue #1735: URL: https://github.com/apache/iceberg-python/issues/1735#issuecomment-2696211969 > Hi [@sharkdtu](https://github.com/sharkdtu) thank you for the explanation! I think that's an interesting use of the location prefix. If I understand correctly, are you run

Re: [PR] Wrap variant in PrimitiveHoder so serialization can result same instance [iceberg]

aihuaxu commented on code in PR #12317: URL: https://github.com/apache/iceberg/pull/12317#discussion_r1978572547 ## api/src/main/java/org/apache/iceberg/types/PrimitiveHolder.java: ## @@ -21,7 +21,7 @@ import java.io.ObjectStreamException; import java.io.Serializable; -/** R

Re: [PR] Build: Bump pypa/cibuildwheel from 2.22.0 to 2.23.0 [iceberg-python]

kevinjqliu merged PR #1752: URL: https://github.com/apache/iceberg-python/pull/1752 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

Re: [I] Flaky test `TestHadoopCommits > testConcurrentFastAppends(File)` [iceberg]

kevinjqliu commented on issue #11047: URL: https://github.com/apache/iceberg/issues/11047#issuecomment-2696119907 thanks @manuzhang i've been experiencing this when i verify a release, only happens on my m1 mac -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] fix: fix http custom headers for rest catalog [iceberg-rust]

chenzl25 commented on code in PR #1010: URL: https://github.com/apache/iceberg-rust/pull/1010#discussion_r1978562210 ## crates/catalog/rest/src/catalog.rs: ## @@ -160,22 +160,28 @@ impl RestCatalogConfig { // The unwrap here is same since we are filtering the keys

Re: [PR] Flink: Dynamic Iceberg Sink Contribution [iceberg]

stevenzwu commented on code in PR #12424: URL: https://github.com/apache/iceberg/pull/12424#discussion_r1978022176 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/PartitionSpecAdjustment.java: ## @@ -0,0 +1,45 @@ +/* + * Licensed to the Apache Software F

Re: [PR] Flink: Dynamic Iceberg Sink Contribution [iceberg]

stevenzwu commented on code in PR #12424: URL: https://github.com/apache/iceberg/pull/12424#discussion_r1978022176 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/PartitionSpecAdjustment.java: ## @@ -0,0 +1,45 @@ +/* + * Licensed to the Apache Software F

Re: [PR] Flink: Dynamic Iceberg Sink Contribution [iceberg]

stevenzwu commented on code in PR #12424: URL: https://github.com/apache/iceberg/pull/12424#discussion_r1978022176 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/PartitionSpecAdjustment.java: ## @@ -0,0 +1,45 @@ +/* + * Licensed to the Apache Software F

Re: [PR] Support In and notIn operators in ParquetFilters.ConvertFilterToParquet [iceberg]

sririshindra commented on PR #12449: URL: https://github.com/apache/iceberg/pull/12449#issuecomment-2695990865 I originally created https://github.com/apache/iceberg/pull/10341 . It did not get traction at the time and it was closed. I would appreciate if the committers can take another loo

Re: [I] I cannot create iceberg tables using databrics. [iceberg]

manabian- commented on issue #10889: URL: https://github.com/apache/iceberg/issues/10889#issuecomment-2695930115 I resolved the following error on clusters configured with access mode set to "No isolation shared": > [[UC_DATASOURCE_NOT_SUPPORTED](https://docs.databricks.com/error-mes

Re: [PR] Auth Manager API part 6: API enablement [iceberg]

danielcweeks commented on code in PR #12197: URL: https://github.com/apache/iceberg/pull/12197#discussion_r1978340592 ## aws/src/main/java/org/apache/iceberg/aws/s3/signer/S3V4RestSignerClient.java: ## @@ -138,148 +133,42 @@ boolean keepTokenRefreshed() { OAuth2Properti

Re: [PR] support create table like in flink catalog [iceberg]

swapna267 commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1978409928 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/FlinkCreateTableOptions.java: ## @@ -0,0 +1,115 @@ +/* + * Licensed to the Apache Software Foundation (A

[I] [iceberg] use AWS IAM role with serviceAccount instated of IAM user [iceberg]

luismacosta opened a new issue, #12448: URL: https://github.com/apache/iceberg/issues/12448 ### Feature Request / Improvement Hello, We use iceberg connector in flink jobs. In order to access s3, the connector uses an IAM user. Our goal is to use IRSA, IAM role + serviceAcc

Re: [I] Implement partition writer [iceberg-rust]

ZENOTME commented on issue #342: URL: https://github.com/apache/iceberg-rust/issues/342#issuecomment-2695904343 > [@ZENOTME](https://github.com/ZENOTME) are you currently working on this? I can look to work on this. Thanks @jonathanc-n! I have completed it and will send PR later. Feel

Re: [PR] Spark: 4.0 snapshot support [iceberg]

szehon-ho commented on PR #11583: URL: https://github.com/apache/iceberg/pull/11583#issuecomment-2695877264 Hi @huaxingao can we rebase against the latest Spark 4.0 RC2 ? https://github.com/apache/spark/tree/v4.0.0-rc2 @danielcweeks wanted to see if we can start testing against thi

Re: [PR] Spark: 4.0 snapshot support [iceberg]

huaxingao commented on PR #11583: URL: https://github.com/apache/iceberg/pull/11583#issuecomment-269589 @szehon-ho will do. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] Auth Manager API part 6: API enablement [iceberg]

danielcweeks commented on code in PR #12197: URL: https://github.com/apache/iceberg/pull/12197#discussion_r1978418025 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -462,12 +375,15 @@ public boolean tableExists(SessionContext context, TableIdentifi

Re: [I] Regression in 0.7.0 due to type coercion from "string" to "large_string" [iceberg-python]

kevinjqliu closed issue #1128: Regression in 0.7.0 due to type coercion from "string" to "large_string" URL: https://github.com/apache/iceberg-python/issues/1128 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[PR] Spark: 4.0 snapshot support [iceberg]

huaxingao opened a new pull request, #11583: URL: https://github.com/apache/iceberg/pull/11583 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [I] Getting "offset overflow while concatenating arrays" Error when writing to iceberg [iceberg-python]

kevinjqliu commented on issue #1733: URL: https://github.com/apache/iceberg-python/issues/1733#issuecomment-2695871493 TY! 0.9.0 RC just passed, we'll upload the official version soon -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [I] AWS Glue Apache Iceberg Data Recovery [iceberg]

github-actions[bot] commented on issue #11077: URL: https://github.com/apache/iceberg/issues/11077#issuecomment-2695863142 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] add dataframes to data files [iceberg-python]

kevinjqliu commented on PR #1677: URL: https://github.com/apache/iceberg-python/pull/1677#issuecomment-2695869399 thanks @jimmyxie-figma do you think this PR is still necessary? if not, can i close it? -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] Auth Manager API part 6: API enablement [iceberg]

danielcweeks commented on code in PR #12197: URL: https://github.com/apache/iceberg/pull/12197#discussion_r1978418025 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -462,12 +375,15 @@ public boolean tableExists(SessionContext context, TableIdentifi

Re: [I] Regression in 0.7.0 due to type coercion from "string" to "large_string" [iceberg-python]

github-actions[bot] commented on issue #1128: URL: https://github.com/apache/iceberg-python/issues/1128#issuecomment-2695866191 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity

Re: [I] CLI list not working [iceberg-python]

github-actions[bot] commented on issue #1122: URL: https://github.com/apache/iceberg-python/issues/1122#issuecomment-2695866218 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity

Re: [I] It's not possible to readStream from an Iceberg table as source when its snapshots expire [iceberg]

github-actions[bot] closed issue #9504: It's not possible to readStream from an Iceberg table as source when its snapshots expire URL: https://github.com/apache/iceberg/issues/9504 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [I] It's not possible to readStream from an Iceberg table as source when its snapshots expire [iceberg]

github-actions[bot] commented on issue #9504: URL: https://github.com/apache/iceberg/issues/9504#issuecomment-2695862970 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [PR] Use SupportsPrefixOperations for Remove OrphanFile Procedure [iceberg]

github-actions[bot] closed pull request #11906: Use SupportsPrefixOperations for Remove OrphanFile Procedure URL: https://github.com/apache/iceberg/pull/11906 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Use SupportsPrefixOperations for Remove OrphanFile Procedure [iceberg]

github-actions[bot] commented on PR #11906: URL: https://github.com/apache/iceberg/pull/11906#issuecomment-2695863233 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [I] Cannot update map key [iceberg]

github-actions[bot] commented on issue #9163: URL: https://github.com/apache/iceberg/issues/9163#issuecomment-2695862941 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

[I] Clean up TODOs in the codebase [iceberg-python]

kevinjqliu opened a new issue, #1052: URL: https://github.com/apache/iceberg-python/issues/1052 ### Feature Request / Improvement Opening this to track a few todos I've seen around the codebase. For example, https://github.com/search?q=repo%3Aapache%2Ficeberg-python+github.com

Re: [PR] support create table like in flink catalog [iceberg]

swapna267 commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1978413720 ## flink/v1.20/flink/src/test/java/org/apache/iceberg/flink/TestFlinkCatalogTable.java: ## @@ -188,6 +188,37 @@ public void testCreateTableLike() throws TableNotEx

Re: [I] Delete orphan files [iceberg-python]

kevinjqliu commented on issue #1200: URL: https://github.com/apache/iceberg-python/issues/1200#issuecomment-2695859107 > But I do not know how to create s3 file system or how to support other file storage in same logic .Could you plz help me on this take a look at `load_file_io` and

Re: [PR] support create table like in flink catalog [iceberg]

swapna267 commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1978409403 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalog.java: ## @@ -404,10 +437,15 @@ void createIcebergTable(ObjectPath tablePath, ResolvedCatal

Re: [PR] support create table like in flink catalog [iceberg]

swapna267 commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1978407287 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalog.java: ## @@ -332,7 +335,33 @@ public List listTables(String databaseName) public Catalog

Re: [I] How to get rid of the warning [iceberg-python]

kevinjqliu commented on issue #1336: URL: https://github.com/apache/iceberg-python/issues/1336#issuecomment-2695842177 @lk-1984 it was cherry-picked and released as part of 0.8.1 https://github.com/apache/iceberg-python/compare/main...pyiceberg-0.8.x commit id f8cb1c0d75c5ec0d9b479446eaa

Re: [I] Support Arrow `string_view` and `bin_view` types. [iceberg-python]

kevinjqliu closed issue #1745: Support Arrow `string_view` and `bin_view` types. URL: https://github.com/apache/iceberg-python/issues/1745 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Support `string_view` and `binary_view` [iceberg-python]

kevinjqliu merged PR #1748: URL: https://github.com/apache/iceberg-python/pull/1748 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

Re: [PR] Build: Bump getdaft from 0.4.4 to 0.4.6 [iceberg-python]

kevinjqliu commented on PR #1758: URL: https://github.com/apache/iceberg-python/pull/1758#issuecomment-2695828302 fixed was merged recently, should be part of the next release https://github.com/Eventual-Inc/Daft/pull/3888 -- This is an automated message from the Apache Git Service. To re

Re: [PR] Build: Bump mkdocs-material from 9.6.5 to 9.6.7 [iceberg-python]

kevinjqliu commented on PR #1757: URL: https://github.com/apache/iceberg-python/pull/1757#issuecomment-2695827229 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Build: Bump griffe from 1.5.7 to 1.6.0 [iceberg-python]

kevinjqliu merged PR #1754: URL: https://github.com/apache/iceberg-python/pull/1754 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

Re: [PR] Build: Bump boto3 from 1.36.23 to 1.37.1 [iceberg-python]

kevinjqliu merged PR #1755: URL: https://github.com/apache/iceberg-python/pull/1755 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

Re: [PR] Build: Bump moto from 5.1.0 to 5.1.1 [iceberg-python]

kevinjqliu merged PR #1756: URL: https://github.com/apache/iceberg-python/pull/1756 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

Re: [PR] Build: Bump polars from 1.23.0 to 1.24.0 [iceberg-python]

kevinjqliu merged PR #1753: URL: https://github.com/apache/iceberg-python/pull/1753 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

[PR] Build: Bump getdaft from 0.4.4 to 0.4.6 [iceberg-python]

dependabot[bot] opened a new pull request, #1758: URL: https://github.com/apache/iceberg-python/pull/1758 Bumps [getdaft](https://github.com/Eventual-Inc/Daft) from 0.4.4 to 0.4.6. Release notes Sourced from https://github.com/Eventual-Inc/Daft/releases";>getdaft's releases.

Re: [PR] Build: Bump getdaft from 0.4.4 to 0.4.5 [iceberg-python]

dependabot[bot] closed pull request #1739: Build: Bump getdaft from 0.4.4 to 0.4.5 URL: https://github.com/apache/iceberg-python/pull/1739 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Build: Bump getdaft from 0.4.4 to 0.4.5 [iceberg-python]

dependabot[bot] commented on PR #1739: URL: https://github.com/apache/iceberg-python/pull/1739#issuecomment-2695737294 Superseded by #1758. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[PR] Build: Bump mkdocs-material from 9.6.5 to 9.6.7 [iceberg-python]

dependabot[bot] opened a new pull request, #1757: URL: https://github.com/apache/iceberg-python/pull/1757 Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.6.5 to 9.6.7. Release notes Sourced from https://github.com/squidfunk/mkdocs-material/releases";>mk

[PR] Build: Bump moto from 5.1.0 to 5.1.1 [iceberg-python]

dependabot[bot] opened a new pull request, #1756: URL: https://github.com/apache/iceberg-python/pull/1756 Bumps [moto](https://github.com/getmoto/moto) from 5.1.0 to 5.1.1. Changelog Sourced from https://github.com/getmoto/moto/blob/master/CHANGELOG.md";>moto's changelog. 5.

[PR] Build: Bump boto3 from 1.36.23 to 1.37.1 [iceberg-python]

dependabot[bot] opened a new pull request, #1755: URL: https://github.com/apache/iceberg-python/pull/1755 Bumps [boto3](https://github.com/boto/boto3) from 1.36.23 to 1.37.1. Commits https://github.com/boto/boto3/commit/11b849477946bb5f6eaf103d81b841354b3d63b9";>11b8494 Merge b

[PR] Build: Bump polars from 1.23.0 to 1.24.0 [iceberg-python]

dependabot[bot] opened a new pull request, #1753: URL: https://github.com/apache/iceberg-python/pull/1753 Bumps [polars](https://github.com/pola-rs/polars) from 1.23.0 to 1.24.0. Release notes Sourced from https://github.com/pola-rs/polars/releases";>polars's releases. Pytho

[PR] Build: Bump griffe from 1.5.7 to 1.6.0 [iceberg-python]

dependabot[bot] opened a new pull request, #1754: URL: https://github.com/apache/iceberg-python/pull/1754 Bumps [griffe](https://github.com/mkdocstrings/griffe) from 1.5.7 to 1.6.0. Release notes Sourced from https://github.com/mkdocstrings/griffe/releases";>griffe's releases.

Re: [PR] Auth Manager API part 6: API enablement [iceberg]

danielcweeks commented on code in PR #12197: URL: https://github.com/apache/iceberg/pull/12197#discussion_r1978340592 ## aws/src/main/java/org/apache/iceberg/aws/s3/signer/S3V4RestSignerClient.java: ## @@ -138,148 +133,42 @@ boolean keepTokenRefreshed() { OAuth2Properti

[PR] Build: Bump pypa/cibuildwheel from 2.22.0 to 2.23.0 [iceberg-python]

dependabot[bot] opened a new pull request, #1752: URL: https://github.com/apache/iceberg-python/pull/1752 Bumps [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel) from 2.22.0 to 2.23.0. Release notes Sourced from https://github.com/pypa/cibuildwheel/releases";>pypa/cibuildwh

Re: [PR] Auth Manager API part 6: API enablement [iceberg]

danielcweeks commented on PR #12197: URL: https://github.com/apache/iceberg/pull/12197#issuecomment-2695640318 @adutra A few comments, but I'm also running into issues with the SigV4 signer implementation while testing this. Trying to track down what the behavior difference is. -- This

Re: [PR] Spec: Add implementation note on `current-snapshot-id` [iceberg]

szehon-ho commented on code in PR #12334: URL: https://github.com/apache/iceberg/pull/12334#discussion_r1978283986 ## format/spec.md: ## @@ -1754,6 +1754,14 @@ Snapshot summary can include metrics fields to track numeric stats of the snapsh | **`engine-name`**| "spark"

Re: [PR] Spec: Add implementation note on `current-snapshot-id` [iceberg]

szehon-ho commented on code in PR #12334: URL: https://github.com/apache/iceberg/pull/12334#discussion_r1978283986 ## format/spec.md: ## @@ -1754,6 +1754,14 @@ Snapshot summary can include metrics fields to track numeric stats of the snapsh | **`engine-name`**| "spark"

Re: [PR] Spec: Add implementation note on `current-snapshot-id` [iceberg]

szehon-ho commented on code in PR #12334: URL: https://github.com/apache/iceberg/pull/12334#discussion_r1978283986 ## format/spec.md: ## @@ -1754,6 +1754,14 @@ Snapshot summary can include metrics fields to track numeric stats of the snapsh | **`engine-name`**| "spark"

Re: [PR] Spec: Add implementation note on `current-snapshot-id` [iceberg]

Fokko commented on code in PR #12334: URL: https://github.com/apache/iceberg/pull/12334#discussion_r1978270259 ## format/spec.md: ## @@ -1754,6 +1754,14 @@ Snapshot summary can include metrics fields to track numeric stats of the snapsh | **`engine-name`**| "spark"

Re: [PR] Spec: Add implementation note on `current-snapshot-id` [iceberg]

Fokko commented on code in PR #12334: URL: https://github.com/apache/iceberg/pull/12334#discussion_r1978271846 ## format/spec.md: ## @@ -1754,6 +1754,14 @@ Snapshot summary can include metrics fields to track numeric stats of the snapsh | **`engine-name`**| "spark"

[PR] API: Speed up Timestamps#toHumanString [iceberg]

suneet-s opened a new pull request, #12447: URL: https://github.com/apache/iceberg/pull/12447 String#format uses regexes internally which can be expensive when used repeatedly. This PR switches out the implementation for a char array implementation that pads zeros as needed. A simple

Re: [PR] Docs: Remove data_file.distinct_counts [iceberg]

Fokko closed pull request #12183: Docs: Remove data_file.distinct_counts URL: https://github.com/apache/iceberg/pull/12183 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [PR] Docs: Remove data_file.distinct_counts [iceberg]

Fokko commented on PR #12183: URL: https://github.com/apache/iceberg/pull/12183#issuecomment-2695596229 Since https://github.com/apache/iceberg/pull/12182 got merged, I'll go ahead and close this one for now. Thanks @jacobmarble for running this 🙌 -- This is an automated message from the

Re: [PR] Docs: Deprecate data_file.distinct_counts in v3 [iceberg]

Fokko merged PR #12182: URL: https://github.com/apache/iceberg/pull/12182 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Docs: Deprecate data_file.distinct_counts in v3 [iceberg]

Fokko commented on PR #12182: URL: https://github.com/apache/iceberg/pull/12182#issuecomment-2695594986 Thanks again for running this @jacobmarble 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Flink: Dynamic Iceberg Sink Contribution [iceberg]

stevenzwu commented on code in PR #12424: URL: https://github.com/apache/iceberg/pull/12424#discussion_r1978087339 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/PartitionSpecEvolver.java: ## @@ -0,0 +1,119 @@ +/* + * Licensed to the Apache Software Fou

Re: [I] [Feature] Add Support for Distributed Write [iceberg-python]

andormarkus commented on issue #1751: URL: https://github.com/apache/iceberg-python/issues/1751#issuecomment-2695366541 Hi @Fokko Based on the source code writing to manifest / Avro can be achieved like this ```python manifest_path = f"temp-manifest-{uuid.uuid4()}.avro"

Re: [PR] Flink: Dynamic Iceberg Sink Contribution [iceberg]

stevenzwu commented on code in PR #12424: URL: https://github.com/apache/iceberg/pull/12424#discussion_r1978022176 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/PartitionSpecAdjustment.java: ## @@ -0,0 +1,45 @@ +/* + * Licensed to the Apache Software F

Re: [PR] Auth Manager API part 6: API enablement [iceberg]

danielcweeks commented on code in PR #12197: URL: https://github.com/apache/iceberg/pull/12197#discussion_r1978070687 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -198,18 +167,33 @@ Map params() { } public RESTSessionCatalog() { -this(

Re: [PR] Auth Manager API part 6: API enablement [iceberg]

danielcweeks commented on code in PR #12197: URL: https://github.com/apache/iceberg/pull/12197#discussion_r1978070687 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -198,18 +167,33 @@ Map params() { } public RESTSessionCatalog() { -this(

Re: [I] Replace `FileMetadata` in parquet writer with in memory representation. [iceberg-rust]

jonathanc-n commented on issue #1004: URL: https://github.com/apache/iceberg-rust/issues/1004#issuecomment-2695342111 @liurenjie1024 I think the current problem with this is that `ArrowFileReader` (reader) returns `ParquetMetadata` and `AsyncFileWriter` (writer) returns the thrift definitio

Re: [I] Implement partition writer [iceberg-rust]

jonathanc-n commented on issue #342: URL: https://github.com/apache/iceberg-rust/issues/342#issuecomment-2695272119 @ZENOTME are you currently working on this? I can look to work on this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] Flink: Dynamic Iceberg Sink Contribution [iceberg]

stevenzwu commented on code in PR #12424: URL: https://github.com/apache/iceberg/pull/12424#discussion_r1978022176 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/PartitionSpecAdjustment.java: ## @@ -0,0 +1,45 @@ +/* + * Licensed to the Apache Software F

[I] Add more variants to `ErrorKind` [iceberg-rust]

connortsui20 opened a new issue, #1038: URL: https://github.com/apache/iceberg-rust/issues/1038 ### Is your feature request related to a problem or challenge? As of now, `ErrorKind` looks like this (doc comments removed): ```rust pub enum ErrorKind { Unexpected,

Re: [I] [Feature Request] Add Writer Support for Table-Compatible Parquet Files [iceberg-python]

andormarkus-alcd closed issue #1737: [Feature Request] Add Writer Support for Table-Compatible Parquet Files URL: https://github.com/apache/iceberg-python/issues/1737 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [I] [Feature Request] Add Writer Support for Table-Compatible Parquet Files [iceberg-python]

andormarkus-alcd commented on issue #1737: URL: https://github.com/apache/iceberg-python/issues/1737#issuecomment-2695296016 Superseded by #1751 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] Fix partitioning on timestamp field with year and month transformations [iceberg]

Fokko commented on PR #12443: URL: https://github.com/apache/iceberg/pull/12443#issuecomment-2695295036 @tousifanwar11 Can you provide more context on what you're trying to achieve? The partitioning schemes are redundant: ``` // When completion_date == 2025-01-03 year(completion

[I] [Feature] Add Support for Distributed Write [iceberg-python]

andormarkus opened a new issue, #1751: URL: https://github.com/apache/iceberg-python/issues/1751 ### Feature Request / Improvement ## Problem Statement A key problem in distributed Iceberg systems is that commit processes can block each other when multiple workers try to update tab

[I] Arrow to iceberg schema conversion does not preserve names [iceberg-rust]

feniljain opened a new issue, #1039: URL: https://github.com/apache/iceberg-rust/issues/1039 ### Apache Iceberg Rust version 0.4.0 (latest version) ### Describe the bug I was trying to debug a failing test I wrote for nan value count PR, and realized name of primitive co

Re: [PR] Flink: Dynamic Iceberg Sink Contribution [iceberg]

stevenzwu commented on code in PR #12424: URL: https://github.com/apache/iceberg/pull/12424#discussion_r1978022176 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/PartitionSpecAdjustment.java: ## @@ -0,0 +1,45 @@ +/* + * Licensed to the Apache Software F

Re: [PR] Add `write_parquet` API for writing Parquet files without committing [iceberg-python]

andormarkus commented on PR #1742: URL: https://github.com/apache/iceberg-python/pull/1742#issuecomment-2695267357 We want to avoid `streaming` and `Spark` on all costs. We have burned ourself with both. I have no problem with Avro (Manifestfile) combined with `append_data_file`.

Re: [PR] Add `write_parquet` API for writing Parquet files without committing [iceberg-python]

andormarkus closed pull request #1742: Add `write_parquet` API for writing Parquet files without committing URL: https://github.com/apache/iceberg-python/pull/1742 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [I] Consolidate methods of converting parquet file to data file builder. [iceberg-rust]

jonathanc-n commented on issue #1033: URL: https://github.com/apache/iceberg-rust/issues/1033#issuecomment-2695259012 @mnpw I can help try to resolve #1004 today and you can work on this after. -- This is an automated message from the Apache Git Service. To respond to the message, please l

Re: [PR] Auth Manager API part 6: API enablement [iceberg]

danielcweeks commented on code in PR #12197: URL: https://github.com/apache/iceberg/pull/12197#discussion_r1977989033 ## aws/src/main/java/org/apache/iceberg/aws/s3/signer/S3V4RestSignerClient.java: ## @@ -81,13 +76,13 @@ public abstract class S3V4RestSignerClient private sta

Re: [PR] support create table like in flink catalog [iceberg]

stevenzwu commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1977969893 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/FlinkCreateTableOptions.java: ## @@ -0,0 +1,115 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] support create table like in flink catalog [iceberg]

stevenzwu commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1977958925 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalog.java: ## @@ -404,10 +437,15 @@ void createIcebergTable(ObjectPath tablePath, ResolvedCatal

Re: [PR] refactor: REST `Catalog` implementation [iceberg-rust]

connortsui20 commented on code in PR #965: URL: https://github.com/apache/iceberg-rust/pull/965#discussion_r1977962499 ## crates/catalog/rest/src/catalog.rs: ## @@ -312,90 +317,107 @@ impl RestCatalog { } } +/// All requests and expected responses are derived from the RE

Re: [PR] support create table like in flink catalog [iceberg]

stevenzwu commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1977957696 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalog.java: ## @@ -404,10 +437,15 @@ void createIcebergTable(ObjectPath tablePath, ResolvedCatal

Re: [PR] support create table like in flink catalog [iceberg]

stevenzwu commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1977956857 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalog.java: ## @@ -332,7 +335,33 @@ public List listTables(String databaseName) public Catalog

Re: [PR] support create table like in flink catalog [iceberg]

stevenzwu commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1977940877 ## flink/v1.20/flink/src/test/java/org/apache/iceberg/flink/TestFlinkCatalogTable.java: ## @@ -188,6 +188,37 @@ public void testCreateTableLike() throws TableNotEx

Re: [PR] support create table like in flink catalog [iceberg]

stevenzwu commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1977919523 ## flink/v1.20/flink/src/test/java/org/apache/iceberg/flink/TestFlinkCatalogTable.java: ## @@ -188,6 +188,37 @@ public void testCreateTableLike() throws TableNotEx

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

szehon-ho commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1972285492 ## api/src/main/java/org/apache/iceberg/expressions/BoundLiteralPredicate.java: ## @@ -88,11 +91,64 @@ public boolean test(T value) { return String.valueOf

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

szehon-ho commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1972280624 ## api/src/main/java/org/apache/iceberg/Geography.java: ## @@ -0,0 +1,127 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contribu

Re: [PR] [1.7.x] Fix Kafka-connect `LICENSE` and `NOTICE` [iceberg]

Fokko commented on PR #12400: URL: https://github.com/apache/iceberg/pull/12400#issuecomment-2695032873 Cool, thanks for validating 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [1.7.x] Fix Kafka-connect `LICENSE` and `NOTICE` [iceberg]

Fokko merged PR #12400: URL: https://github.com/apache/iceberg/pull/12400 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [I] How to get rid of the warning [iceberg-python]

lk-1984 commented on issue #1336: URL: https://github.com/apache/iceberg-python/issues/1336#issuecomment-2695027565 @kevinjqliu So this did not get into 0.8.1? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

  1   2   >