Re: [I] how to create iceberg hive catalog-type catalog in flink with kerberos [iceberg]

zhouchitc commented on issue #8213: URL: https://github.com/apache/iceberg/issues/8213#issuecomment-2774609956 @ztqchn I think you should add when using kerberos, like: CREATE CATALOG hive_catalog WITH ( 'type'='iceberg', 'catalog-type'='hive', 'uri'='thrift://xx:9083',

Re: [PR] Core, Hive: Double check commit status in case of commit conflict for NoLock [iceberg]

lirui-apache commented on PR #12637: URL: https://github.com/apache/iceberg/pull/12637#issuecomment-2774555793 Thanks to @pvary and @deniskuzZ for reviewing and merging the PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [PR] Core: Add support for Avro's timestamp-millis LogicalType in DataReader [iceberg]

github-actions[bot] commented on PR #12397: URL: https://github.com/apache/iceberg/pull/12397#issuecomment-2774026650 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

[I] iceberg MOR - Rewrite Fails [iceberg]

nikhilindikuzha opened a new issue, #12710: URL: https://github.com/apache/iceberg/issues/12710 ### Query engine Spark ### Question I have created a iceberg table in version 2 with MOR statergy with one column as partition. Using merge command i have inserted 3 record

Re: [PR] Spark 3.5: Support case sensitive in replace where statement [iceberg]

ebyhr commented on code in PR #12706: URL: https://github.com/apache/iceberg/pull/12706#discussion_r2025756776 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkWrite.java: ## @@ -101,6 +101,7 @@ abstract class SparkWrite implements Write, RequiresDistribut

Re: [PR] Build: Retry flaky test [iceberg]

manuzhang commented on PR #12707: URL: https://github.com/apache/iceberg/pull/12707#issuecomment-2774457965 The flaky test is usually when we need to wait for a condition to come true. In this case, ``` AtomicInteger barrier = new AtomicInteger(0); Tasks.range(threadsCou

Re: [PR] feat: sort order [iceberg-cpp]

zhjwpku commented on code in PR #59: URL: https://github.com/apache/iceberg-cpp/pull/59#discussion_r2026151150 ## test/sort_field_test.cc: ## @@ -0,0 +1,82 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the

Re: [PR] Updated vendors documentation to add SingleStore [iceberg]

akoller commented on code in PR #12708: URL: https://github.com/apache/iceberg/pull/12708#discussion_r2026146490 ## site/docs/vendors.md: ## @@ -90,6 +90,12 @@ PuppyGraph is a cloud-native graph analytics engine that enables users to query [RisingWave](https://risingwave.com

Re: [I] Arrow uint32/64 support [iceberg-rust]

zuston commented on issue #1164: URL: https://github.com/apache/iceberg-rust/issues/1164#issuecomment-2774377307 Nice catch. But I think the iceberg spec's primitive type haven't supported this unsigned type. cc @liurenjie1024 If having any solution, I can take this ticket to be as m

Re: [PR] Spark 3.5: Support case sensitive in replace where statement [iceberg]

dolcino-li commented on PR #12706: URL: https://github.com/apache/iceberg/pull/12706#issuecomment-2774375518 Please review @nastra @ebyhr -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

[PR] feat(puffin): Make Puffin APIs public [iceberg-rust]

fqaiser94 opened a new pull request, #1165: URL: https://github.com/apache/iceberg-rust/pull/1165 ## Which issue does this PR close? Part of #744 ## What changes are included in this PR? - Makes Puffin APIs public - Turns dead-code warning back on (disabled earlier to

Re: [PR] feat:add init expression interface. [iceberg-cpp]

wgtmac commented on code in PR #58: URL: https://github.com/apache/iceberg-cpp/pull/58#discussion_r2025893097 ## src/iceberg/expression.h: ## @@ -0,0 +1,101 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the

Re: [PR] feat:add init expression interface. [iceberg-cpp]

wgtmac commented on code in PR #58: URL: https://github.com/apache/iceberg-cpp/pull/58#discussion_r2025884055 ## src/iceberg/expression.h: ## @@ -0,0 +1,101 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the

Re: [PR] feat:add init expression interface. [iceberg-cpp]

wgtmac commented on code in PR #58: URL: https://github.com/apache/iceberg-cpp/pull/58#discussion_r2025870041 ## test/CMakeLists.txt: ## @@ -39,6 +39,11 @@ target_sources(expected_test PRIVATE expected_test.cc) target_link_libraries(expected_test PRIVATE iceberg_static GTest::g

Re: [PR] feat: Add summary functionality to `SnapshotProduceAction` [iceberg-rust]

liurenjie1024 merged PR #1139: URL: https://github.com/apache/iceberg-rust/pull/1139 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] Spec: Allow the use of `source-id` in V3 [iceberg]

szehon-ho commented on PR #12644: URL: https://github.com/apache/iceberg/pull/12644#issuecomment-2773084895 I wonder should we remove multi-bucket into separate pr, to allow the source-id part to get in? -- This is an automated message from the Apache Git Service. To respond to the messag

Re: [PR] Core: Enhance remove snapshots efficiency by executing them in bulk [iceberg]

ricardopereira33 commented on code in PR #12670: URL: https://github.com/apache/iceberg/pull/12670#discussion_r2024439959 ## core/src/main/java/org/apache/iceberg/MetadataUpdateParser.java: ## @@ -150,7 +148,7 @@ private MetadataUpdateParser() {} .put(MetadataUpdate.S

Re: [PR] Core: Support incremental compute for partition stats [iceberg]

ajantha-bhat commented on PR #12629: URL: https://github.com/apache/iceberg/pull/12629#issuecomment-2774081547 I am traveling for Iceberg summit and may not be able to work on it this week. I will be moving this to 1.10.0 milestone. This is a good optimization. But need some testing for

Re: [PR] feat(io): add OSS storage implementation and update dependencies [iceberg-rust]

divinerapier commented on code in PR #1153: URL: https://github.com/apache/iceberg-rust/pull/1153#discussion_r2025831546 ## crates/iceberg/src/io/storage_oss.rs: ## @@ -0,0 +1,74 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license ag

Re: [PR] feat(io): add OSS storage implementation and update dependencies [iceberg-rust]

divinerapier commented on code in PR #1153: URL: https://github.com/apache/iceberg-rust/pull/1153#discussion_r2025822429 ## crates/iceberg/src/io/storage.rs: ## @@ -45,6 +47,17 @@ pub(crate) enum Storage { client: reqwest::Client, config: Arc, }, +#[cf

[PR] Spark3.4: Enable Native execution if ParquetReaderType is Comet [iceberg]

huaxingao opened a new pull request, #12709: URL: https://github.com/apache/iceberg/pull/12709 This PR has the following changes: - Make `SparkScan` implement `org.apache.comet.parquet.SupportsComet` , so on Comet side, it can check `SupportsComet.isCometEnabled()` and turn on native

Re: [I] Iceberg supports binlog logs [iceberg]

xiaoxuandev commented on issue #10452: URL: https://github.com/apache/iceberg/issues/10452#issuecomment-2774029104 Hi @smileyboy2019, could you clarify your specific goal with binlogs and what you are looking for in Iceberg? -- This is an automated message from the Apache Git Service. To

Re: [I] Docs: Fix MkDocs ASF nav links [iceberg]

github-actions[bot] commented on issue #8965: URL: https://github.com/apache/iceberg/issues/8965#issuecomment-2774026290 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] API: Define RepairManifests action interface [iceberg]

github-actions[bot] closed pull request #10784: API: Define RepairManifests action interface URL: https://github.com/apache/iceberg/pull/10784 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] feat:add init expression interface. [iceberg-cpp]

lidavidm commented on code in PR #58: URL: https://github.com/apache/iceberg-cpp/pull/58#discussion_r2025772416 ## src/iceberg/expression.h: ## @@ -0,0 +1,101 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See t

Re: [PR] Spark 3.5: Support case sensitive in replace where statement [iceberg]

ebyhr commented on code in PR #12706: URL: https://github.com/apache/iceberg/pull/12706#discussion_r2025770641 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestReplaceWhere.java: ## @@ -0,0 +1,60 @@ +/* + * Licensed to the Apache Software Foun

Re: [PR] Updated vendors documentation to add SingleStore [iceberg]

ebyhr commented on code in PR #12708: URL: https://github.com/apache/iceberg/pull/12708#discussion_r2025749421 ## site/docs/vendors.md: ## @@ -90,6 +90,12 @@ PuppyGraph is a cloud-native graph analytics engine that enables users to query [RisingWave](https://risingwave.com/)

Re: [I] Generation of Snapshot Summaries [iceberg-rust]

liurenjie1024 closed issue #724: Generation of Snapshot Summaries URL: https://github.com/apache/iceberg-rust/issues/724 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

Re: [PR] AWS: Add AWS integ tests to check task and enable tests based on required environment variables [iceberg]

lliangyu-lin commented on PR #12671: URL: https://github.com/apache/iceberg/pull/12671#issuecomment-2773959398 Updated based on the comments. Please take another look when you have time. cc: @nastra @xiaoxuandev -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] Core: HadoopFileIO to support bulk delete through the Hadoop Filesystem APIs [iceberg]

danielcweeks commented on code in PR #10233: URL: https://github.com/apache/iceberg/pull/10233#discussion_r2025719727 ## core/src/main/java/org/apache/iceberg/hadoop/HadoopFileIO.java: ## @@ -173,10 +207,49 @@ public void deletePrefix(String prefix) { } } + /** + *

[PR] feat: convert arrow schema to iceberg schema [iceberg-cpp]

wgtmac opened a new pull request, #57: URL: https://github.com/apache/iceberg-cpp/pull/57 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mai

Re: [PR] Pyarrow data type, default to small type and fix large type override [iceberg-python]

Fokko commented on code in PR #1859: URL: https://github.com/apache/iceberg-python/pull/1859#discussion_r2021694263 ## mkdocs/docs/configuration.md: ## @@ -199,7 +199,7 @@ PyIceberg uses [S3FileSystem](https://arrow.apache.org/docs/python/generated/pya | Key

Re: [PR] AWS: Support StaticCredentialsProvider in DefaultAwsClientFactory [iceberg]

xiaoxuandev commented on PR #12695: URL: https://github.com/apache/iceberg/pull/12695#issuecomment-2773788746 @danielcweeks, I think it's a valid use case, not just for user, even for CI pipeline, currently AWS integration tests are not running in pipeline, but we are spend effort to have i

Re: [PR] feat: Infer partition values from statistics [iceberg-rust]

jonathanc-n commented on PR #1079: URL: https://github.com/apache/iceberg-rust/pull/1079#issuecomment-2773806669 I think the function name is misleading I will change that. We are passing in the lower and upper bounds computed from the original parquet file read during the `parquet_to_data_

Re: [PR] feat: Add summary functionality to `SnapshotProduceAction` [iceberg-rust]

jonathanc-n commented on PR #1139: URL: https://github.com/apache/iceberg-rust/pull/1139#issuecomment-2773793133 @liurenjie1024 Thanks for the reivew, the changes have been implemented. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] Spark: when doing rewrite_data_files, check for partitioning schema compatibility [iceberg]

RussellSpitzer commented on code in PR #12651: URL: https://github.com/apache/iceberg/pull/12651#discussion_r2025635244 ## api/src/main/java/org/apache/iceberg/PartitionSpec.java: ## @@ -265,6 +265,22 @@ public boolean equals(Object other) { return Arrays.equals(fields, tha

Re: [PR] Spark: when doing rewrite_data_files, check for partitioning schema compatibility [iceberg]

RussellSpitzer commented on code in PR #12651: URL: https://github.com/apache/iceberg/pull/12651#discussion_r2025632539 ## api/src/main/java/org/apache/iceberg/PartitionSpec.java: ## @@ -265,6 +265,22 @@ public boolean equals(Object other) { return Arrays.equals(fields, tha

Re: [PR] feat: Add basic conflict detection [iceberg-rust]

jonathanc-n commented on PR #1154: URL: https://github.com/apache/iceberg-rust/pull/1154#issuecomment-2773774727 @Fokko @ZENOTME Yeah will change title, I just wanted to open this to discuss what our plan was for the conflict detection. -- This is an automated message from the Apache Git

Re: [PR] Spark: when doing rewrite_data_files, check for partitioning schema compatibility [iceberg]

danielcweeks commented on code in PR #12651: URL: https://github.com/apache/iceberg/pull/12651#discussion_r2025570136 ## api/src/main/java/org/apache/iceberg/PartitionSpec.java: ## @@ -265,6 +265,22 @@ public boolean equals(Object other) { return Arrays.equals(fields, that.

Re: [PR] Spark: when doing rewrite_data_files, check for partitioning schema compatibility [iceberg]

danielcweeks commented on code in PR #12651: URL: https://github.com/apache/iceberg/pull/12651#discussion_r2025566400 ## api/src/main/java/org/apache/iceberg/PartitionSpec.java: ## @@ -265,6 +265,22 @@ public boolean equals(Object other) { return Arrays.equals(fields, that.

Re: [PR] AWS: Support StaticCredentialsProvider in DefaultAwsClientFactory [iceberg]

danielcweeks commented on PR #12695: URL: https://github.com/apache/iceberg/pull/12695#issuecomment-2773642909 > Thanks @danielcweeks, while environment variables work in Kubernetes, they aren’t always the best solution for all deployment scenarios . . . @xiaoxuandev it feels like we'

Re: [PR] Updated vendors documentation to add SingleStore [iceberg]

danielcweeks commented on code in PR #12708: URL: https://github.com/apache/iceberg/pull/12708#discussion_r2025520638 ## site/docs/vendors.md: ## @@ -90,6 +90,12 @@ PuppyGraph is a cloud-native graph analytics engine that enables users to query [RisingWave](https://risingwav

Re: [I] Compaction results in Out Of Memory for >=million records [iceberg]

RussellSpitzer commented on issue #12704: URL: https://github.com/apache/iceberg/issues/12704#issuecomment-2773280197 1 million is actually not very many records. It would probably be helpful if you could share more details about the failure. A 134 could be thrown for a lot of reasons and i

Re: [PR] Core: ability to inject an AuthManager in RESTCatalog [iceberg]

danielcweeks commented on code in PR #12655: URL: https://github.com/apache/iceberg/pull/12655#discussion_r2025506322 ## core/src/main/java/org/apache/iceberg/rest/RESTCatalog.java: ## @@ -65,7 +68,14 @@ public RESTCatalog(Function, RESTClient> clientBuilder) { public RESTCa

Re: [PR] Scan Delete Support Part 5: Positional Delete Parsing [iceberg-rust]

sdd commented on code in PR #1011: URL: https://github.com/apache/iceberg-rust/pull/1011#discussion_r2025456457 ## crates/iceberg/src/arrow/delete_file_manager.rs: ## @@ -0,0 +1,638 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license

Re: [PR] Scan Delete Support Part 4: Delete File Loading; Skeleton for Processing [iceberg-rust]

sdd commented on code in PR #982: URL: https://github.com/apache/iceberg-rust/pull/982#discussion_r2025405342 ## crates/iceberg/src/arrow/delete_file_manager.rs: ## @@ -0,0 +1,564 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license a

[PR] Updated vendors documentation to add SingleStore [iceberg]

akoller opened a new pull request, #12708: URL: https://github.com/apache/iceberg/pull/12708 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-

Re: [PR] Core: ability to inject an AuthManager in RESTCatalog [iceberg]

gh-yzou commented on code in PR #12655: URL: https://github.com/apache/iceberg/pull/12655#discussion_r2025298157 ## core/src/main/java/org/apache/iceberg/rest/RESTCatalog.java: ## @@ -65,7 +68,14 @@ public RESTCatalog(Function, RESTClient> clientBuilder) { public RESTCatalog

Re: [PR] feat: add support for azure blob with connection string/sas token/account key [iceberg-go]

zeroshade merged PR #313: URL: https://github.com/apache/iceberg-go/pull/313 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

[I] Arrow uint32/64 support [iceberg-rust]

hugokitano opened a new issue, #1164: URL: https://github.com/apache/iceberg-rust/issues/1164 ### Is your feature request related to a problem or challenge? `iceberg::spec::Schema::try_from` tries to construct an Iceberg schema from an arrow schema, but does not support converting uin

Re: [I] Implement Sql Catalog. [iceberg-rust]

kevinjqliu commented on issue #248: URL: https://github.com/apache/iceberg-rust/issues/248#issuecomment-2773161659 @hendrikmakait i dont think so, i see a lot of `todos` in that PR https://github.com/apache/iceberg-rust/pull/524/files#diff-0095ccc22a02ff26cc0c2f2016a0573cecfeded94dfeb7fdf249

Re: [PR] Spec: Allow the use of `source-id` in V3 [iceberg]

RussellSpitzer commented on code in PR #12644: URL: https://github.com/apache/iceberg/pull/12644#discussion_r2025182220 ## format/spec.md: ## @@ -1414,12 +1414,16 @@ Each partition field in `fields` is stored as a JSON object with the following p | V1 | V2 | V3

Re: [PR] Core: ability to inject an AuthManager in RESTCatalog [iceberg]

danielcweeks commented on code in PR #12655: URL: https://github.com/apache/iceberg/pull/12655#discussion_r2025176480 ## core/src/main/java/org/apache/iceberg/rest/RESTCatalog.java: ## @@ -65,7 +68,14 @@ public RESTCatalog(Function, RESTClient> clientBuilder) { public RESTCa

Re: [PR] Doc: Remove Warning for resolved issue #11147. [iceberg]

nastra merged PR #12694: URL: https://github.com/apache/iceberg/pull/12694 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] feat(table): write an Arrow Table or stream of Arrow records [iceberg-go]

Fokko commented on PR #369: URL: https://github.com/apache/iceberg-go/pull/369#issuecomment-2772347270 This looks great, thanks @zeroshade 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Core: Pass storage credentials from LoadTableResponse to FileIO [iceberg]

nastra commented on PR #12591: URL: https://github.com/apache/iceberg/pull/12591#issuecomment-2771433944 thanks for the reviews @amogh-jahagirdar and @danielcweeks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] ci: fixing the golangci action [iceberg-go]

zeroshade merged PR #373: URL: https://github.com/apache/iceberg-go/pull/373 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] fix(rest): rename page-size & page-token query params [iceberg-go]

zeroshade merged PR #372: URL: https://github.com/apache/iceberg-go/pull/372 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [I] Wrong format of pagination query parameters [iceberg-go]

zeroshade closed issue #371: Wrong format of pagination query parameters URL: https://github.com/apache/iceberg-go/issues/371 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] fix: Preserve list name in schema conversion [iceberg-rust]

jonathanc-n closed pull request #1149: fix: Preserve list name in schema conversion URL: https://github.com/apache/iceberg-rust/pull/1149 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] feat:add init expression interface. [iceberg-cpp]

zhjwpku commented on code in PR #58: URL: https://github.com/apache/iceberg-cpp/pull/58#discussion_r2025025607 ## src/iceberg/expression.cc: ## @@ -0,0 +1,129 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See t

Re: [PR] feat: sort order [iceberg-cpp]

gty404 commented on code in PR #59: URL: https://github.com/apache/iceberg-cpp/pull/59#discussion_r2025032497 ## src/iceberg/sort_field.cc: ## @@ -0,0 +1,85 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the

Re: [PR] feat:add init expression interface. [iceberg-cpp]

gty404 commented on code in PR #58: URL: https://github.com/apache/iceberg-cpp/pull/58#discussion_r2024995368 ## src/iceberg/error.h: ## @@ -38,6 +38,8 @@ enum class ErrorKind { kNotImplemented, kUnknownError, kNotSupported, + kInvalidExpression, + kInvalidOperatorTyp

[PR] feat: sort order [iceberg-cpp]

zhjwpku opened a new pull request, #59: URL: https://github.com/apache/iceberg-cpp/pull/59 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-ma

Re: [PR] Spark 3.5: Support case sensitive in replace where statement [iceberg]

nastra commented on code in PR #12706: URL: https://github.com/apache/iceberg/pull/12706#discussion_r2024932153 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestReplaceWhere.java: ## @@ -0,0 +1,60 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] Spark 3.5: Support case sensitive in replace where statement [iceberg]

nastra commented on code in PR #12706: URL: https://github.com/apache/iceberg/pull/12706#discussion_r2024929650 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestReplaceWhere.java: ## @@ -0,0 +1,60 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] SPARK: Remove dependency on hadoop's filesystem class from remove orphan files [iceberg]

pvary commented on code in PR #12254: URL: https://github.com/apache/iceberg/pull/12254#discussion_r2024912004 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/DeleteOrphanFilesSparkAction.java: ## @@ -335,7 +344,39 @@ private Dataset listedFileDS() { retu

Re: [PR] Spark 3.4: Fix NotSerializableException when migrating Spark tables [iceberg]

nastra merged PR #12705: URL: https://github.com/apache/iceberg/pull/12705 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] SPARK: Remove dependency on hadoop's filesystem class from remove orphan files [iceberg]

pvary commented on code in PR #12254: URL: https://github.com/apache/iceberg/pull/12254#discussion_r2024900858 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/DeleteOrphanFilesSparkAction.java: ## @@ -335,7 +344,39 @@ private Dataset listedFileDS() { retu

Re: [PR] feat: Infer partition values from statistics [iceberg-rust]

ZENOTME commented on PR #1079: URL: https://github.com/apache/iceberg-rust/pull/1079#issuecomment-2772641331 > Hi, @jonathanc-n I'm quite confused about this pr, how can you infer partition value from statistics? First of all, statistics are optional, and they are maybe inaccurate. For exam

Re: [PR] feat: Add summary functionality to `SnapshotProduceAction` [iceberg-rust]

liurenjie1024 commented on code in PR #1139: URL: https://github.com/apache/iceberg-rust/pull/1139#discussion_r2024870362 ## crates/iceberg/src/spec/snapshot_summary.rs: ## @@ -507,7 +517,21 @@ fn update_totals( .insert(total_property.to_string(), new_total.to_string())

Re: [PR] feat: Add basic conflict detection [iceberg-rust]

ZENOTME commented on code in PR #1154: URL: https://github.com/apache/iceberg-rust/pull/1154#discussion_r2024859346 ## crates/iceberg/src/transaction/mod.rs: ## @@ -127,12 +127,14 @@ impl<'a> Transaction<'a> { } /// Creates a fast append action. -pub fn fast_appe

[I] Support memory catalog in cli tool [iceberg-rust]

liurenjie1024 opened a new issue, #1160: URL: https://github.com/apache/iceberg-rust/issues/1160 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

[I] Support glue catalog in cli tool [iceberg-rust]

liurenjie1024 opened a new issue, #1156: URL: https://github.com/apache/iceberg-rust/issues/1156 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

[I] Support s3 table in cli tool [iceberg-rust]

liurenjie1024 opened a new issue, #1161: URL: https://github.com/apache/iceberg-rust/issues/1161 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

[I] Support sql catalog in cli tool [iceberg-rust]

liurenjie1024 opened a new issue, #1159: URL: https://github.com/apache/iceberg-rust/issues/1159 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

Re: [PR] feat: Introduce C FFI for iceberg rust [iceberg-rust]

liurenjie1024 closed pull request #966: feat: Introduce C FFI for iceberg rust URL: https://github.com/apache/iceberg-rust/pull/966 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] feat: Introduce C FFI for iceberg rust [iceberg-rust]

liurenjie1024 commented on PR #966: URL: https://github.com/apache/iceberg-rust/pull/966#issuecomment-2772576767 I'll close this pr per discussion. Feel free to reopen it if necessary. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] Spark: Update RewriteDataFilesSparkAction and RewritePositionDeleteFilesSparkAction to use the new APIs [iceberg]

manuzhang commented on code in PR #12692: URL: https://github.com/apache/iceberg/pull/12692#discussion_r2024844226 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/actions/TestRewriteDataFilesAction.java: ## @@ -1720,143 +1717,15 @@ public void testInvalidAPIUsage() {

Re: [PR] feat: Introduce C FFI for iceberg rust [iceberg-rust]

liurenjie1024 commented on PR #966: URL: https://github.com/apache/iceberg-rust/pull/966#issuecomment-2772575109 > I had a deeper discussion with friends from iceberg-cpp and reached a consensus as proposed here: https://lists.apache.org/thread/2sd7f06b1z2qtv26d6nmsy0hh2xvn99s. Your comment

Re: [I] Feature: S3 Remote Signing [iceberg-rust]

liurenjie1024 commented on issue #506: URL: https://github.com/apache/iceberg-rust/issues/506#issuecomment-2772566367 > hii, can I help with this somehow? > > If I understand the state currently: reqsign is going through a refactor which makes it possible to configure a signer (based

Re: [PR] feat(io): add OSS storage implementation and update dependencies [iceberg-rust]

liurenjie1024 commented on code in PR #1153: URL: https://github.com/apache/iceberg-rust/pull/1153#discussion_r2024802340 ## crates/iceberg/src/io/storage.rs: ## @@ -45,6 +47,17 @@ pub(crate) enum Storage { client: reqwest::Client, config: Arc, }, +#[c

Re: [PR] fix: Preserve list name in schema conversion [iceberg-rust]

liurenjie1024 commented on PR #1149: URL: https://github.com/apache/iceberg-rust/pull/1149#issuecomment-2772517504 cc @jonathanc-n We should close this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] [DISCUSS] Exceptions vs status codes [iceberg-cpp]

zhjwpku commented on issue #14: URL: https://github.com/apache/iceberg-cpp/issues/14#issuecomment-2772516785 I think we can close this issue for now, we have exception and expected in our code base, lets see how it works. -- This is an automated message from the Apache Git Service. To res

Re: [PR] fix: Preserve list name in schema conversion [iceberg-rust]

liurenjie1024 commented on PR #1149: URL: https://github.com/apache/iceberg-rust/pull/1149#issuecomment-2772512366 > Would like to request some help here. I believe the catalog is running into an error while deserializing the response to query the catalog. > > Here is the message:

Re: [PR] feat: Add basic conflict detection [iceberg-rust]

Fokko commented on PR #1154: URL: https://github.com/apache/iceberg-rust/pull/1154#issuecomment-2772487148 @jonathanc-n This is the very first step, and we indeed want this. I don't think this is the actual conflict detection. When between the `fast_append()` and `commit()` something

Re: [PR] SPARK: Remove dependency on hadoop's filesystem class from remove orphan files [iceberg]

pvary commented on code in PR #12254: URL: https://github.com/apache/iceberg/pull/12254#discussion_r2024776922 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/DeleteOrphanFilesSparkAction.java: ## @@ -302,21 +303,29 @@ private Dataset actualFileIdentDS() {

Re: [PR] feat: re-export name mapping [iceberg-rust]

Fokko commented on code in PR #1116: URL: https://github.com/apache/iceberg-rust/pull/1116#discussion_r2024766851 ## crates/iceberg/src/spec/mapped_fields.rs: ## @@ -0,0 +1,123 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agre

Re: [PR] AWS: Update the aws-bundle with latest dependencies [iceberg]

Fokko merged PR #12553: URL: https://github.com/apache/iceberg/pull/12553 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Core, Hive: Double check commit status in case of commit conflict for NoLock [iceberg]

pvary commented on PR #12637: URL: https://github.com/apache/iceberg/pull/12637#issuecomment-2772423961 Thanks for the PR @lirui-apache! Thanks for the review @deniskuzZ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Core, Hive: Double check commit status in case of commit conflict for NoLock [iceberg]

pvary merged PR #12637: URL: https://github.com/apache/iceberg/pull/12637 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [I] Table corruption using lock-free Hive commits [iceberg]

pvary closed issue #11814: Table corruption using lock-free Hive commits URL: https://github.com/apache/iceberg/issues/11814 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Core, Hive: Double check commit status in case of commit conflict for NoLock [iceberg]

lirui-apache commented on code in PR #12637: URL: https://github.com/apache/iceberg/pull/12637#discussion_r2024510734 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveTableOperations.java: ## @@ -287,15 +277,33 @@ protected void doCommit(TableMetadata base, TableMetad

Re: [PR] feat(table): write an Arrow Table or stream of Arrow records [iceberg-go]

Fokko commented on code in PR #369: URL: https://github.com/apache/iceberg-go/pull/369#discussion_r2024683872 ## table/table_test.go: ## @@ -835,6 +840,297 @@ func (t *TableWritingTestSuite) TestReplaceDataFiles() { }, staged.CurrentSnapshot().Summary) } +func (t *Tab

Re: [PR] feat(table): write an Arrow Table or stream of Arrow records [iceberg-go]

Fokko merged PR #369: URL: https://github.com/apache/iceberg-go/pull/369 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Build: Bump guava from 33.4.0-jre to 33.4.6-jre [iceberg]

Fokko merged PR #12686: URL: https://github.com/apache/iceberg/pull/12686 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

[PR] Spark 3.4: Fix NotSerializableException when migrating Spark tables [iceberg]

manuzhang opened a new pull request, #12705: URL: https://github.com/apache/iceberg/pull/12705 Back-port of #11157 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

Re: [I] when drop a non-Iceberg table , the directory associated with the table was not deleted [iceberg]

wzx140 commented on issue #11820: URL: https://github.com/apache/iceberg/issues/11820#issuecomment-2772069922 @MonkeyCanCode I think I know why Spark 3.3.3 can clean up files even without PURGE. In older Spark versions, DROP TABLE is parsed as a V1Command, which delegates to V1Catalog. Howe

Re: [PR] AWS: Delegate part of AWS integration tests to using mock aws services and enable tests in check task [iceberg]

nastra commented on PR #12671: URL: https://github.com/apache/iceberg/pull/12671#issuecomment-2772086011 Since the PR is now using JUnit tags to distinguish between tests I was wondering whether it would make more sense to actually move the docker-specific tests to `integrationTest` instead

Re: [PR] Core: Extended header support for RESTClient implementations [iceberg]

gaborkaszab commented on code in PR #12194: URL: https://github.com/apache/iceberg/pull/12194#discussion_r2024437585 ## core/src/main/java/org/apache/iceberg/rest/BaseHTTPClient.java: ## @@ -77,6 +77,18 @@ public T get( return execute(request, responseType, errorHandler, h

  1   2   >