Re: [PR] Core,REST: extend httpClient builder to support tls factory [iceberg]

2025-02-26 Thread via GitHub
munendrasn commented on code in PR #11979: URL: https://github.com/apache/iceberg/pull/11979#discussion_r1971573647 ## core/src/main/java/org/apache/iceberg/rest/HTTPClient.java: ## @@ -480,6 +486,11 @@ public Builder withObjectMapper(ObjectMapper objectMapper) { return

Re: [I] [Feature] - Allow Schema Overwrite [iceberg-python]

2025-02-26 Thread via GitHub
smaheshwar-pltr commented on issue #1726: URL: https://github.com/apache/iceberg-python/issues/1726#issuecomment-2684966894 +1 - I'd also like for schema replacement to be supported like this, unless I'm mistaken and it is already. Without https://github.com/apache/iceberg-python/pul

Re: [PR] Build: Bump junit from 5.11.4 to 5.12.0 [iceberg]

2025-02-26 Thread via GitHub
dependabot[bot] commented on PR #12378: URL: https://github.com/apache/iceberg/pull/12378#issuecomment-2684996449 OK, I won't notify you again about this release, but will get in touch when a new version is available. You can also ignore all major, minor, or patch releases for a dependency

Re: [PR] Build: Bump junit from 5.11.4 to 5.12.0 [iceberg]

2025-02-26 Thread via GitHub
nastra closed pull request #12378: Build: Bump junit from 5.11.4 to 5.12.0 URL: https://github.com/apache/iceberg/pull/12378 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] ci(catalog/rest): initial framework for rest catalog integration tests [iceberg-go]

2025-02-26 Thread via GitHub
kevinjqliu commented on code in PR #310: URL: https://github.com/apache/iceberg-go/pull/310#discussion_r1971884342 ## .github/workflows/go-integration.yml: ## @@ -66,6 +66,7 @@ jobs: run: | go test -tags integration -v -run="^TestScanner" ./table

Re: [PR] 1191:Added docstrings to the pyiceberg/table/inspect.py file [iceberg-python]

2025-02-26 Thread via GitHub
gayatrikate04 commented on PR #1533: URL: https://github.com/apache/iceberg-python/pull/1533#issuecomment-2685638054 resolved conflicts -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] fix(table/scanner): Fix nested field scan [iceberg-go]

2025-02-26 Thread via GitHub
zeroshade commented on code in PR #311: URL: https://github.com/apache/iceberg-go/pull/311#discussion_r1971976048 ## table/arrow_utils.go: ## @@ -497,7 +497,10 @@ func (c convertToArrow) Field(field iceberg.NestedField, result arrow.Field) arr func (c convertToArrow) List(li

Re: [PR] ci(catalog/rest): initial framework for rest catalog integration tests [iceberg-go]

2025-02-26 Thread via GitHub
zeroshade merged PR #310: URL: https://github.com/apache/iceberg-go/pull/310 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Spark: Detect dangling DVs properly [iceberg]

2025-02-26 Thread via GitHub
singhpk234 commented on code in PR #12270: URL: https://github.com/apache/iceberg/pull/12270#discussion_r1972041116 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RemoveDanglingDeletesSparkAction.java: ## @@ -124,10 +125,15 @@ private List findDanglingDeletes

Re: [PR] Spark: Rewrite V2 deletes to V3 DVs [iceberg]

2025-02-26 Thread via GitHub
nastra closed pull request #12250: Spark: Rewrite V2 deletes to V3 DVs URL: https://github.com/apache/iceberg/pull/12250 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
dramaticlly commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972171501 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCommits.java: ## @@ -129,6 +130,52 @@ public void testThriftExceptionUnknownStateIfNotInHistoryFa

Re: [PR] List data and metadata directories instead of table root [iceberg]

2025-02-26 Thread via GitHub
karuppayya commented on code in PR #12278: URL: https://github.com/apache/iceberg/pull/12278#discussion_r1972303374 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/DeleteOrphanFilesSparkAction.java: ## @@ -301,24 +303,34 @@ private Dataset actualFileIdentDS()

Re: [I] Proxy Settings for catalog REST API client [iceberg]

2025-02-26 Thread via GitHub
akhilputhiry commented on issue #12059: URL: https://github.com/apache/iceberg/issues/12059#issuecomment-2684992598 I am working on this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[PR] View Spec: Fix typo [iceberg]

2025-02-26 Thread via GitHub
ajantha-bhat opened a new pull request, #12405: URL: https://github.com/apache/iceberg/pull/12405 fix copy paste error -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [PR] View Spec: Fix typo [iceberg]

2025-02-26 Thread via GitHub
nastra merged PR #12405: URL: https://github.com/apache/iceberg/pull/12405 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

[I] Support accessors for ManifestMetadata fields [iceberg-rust]

2025-02-26 Thread via GitHub
mnpw opened a new issue, #1012: URL: https://github.com/apache/iceberg-rust/issues/1012 Fields of [`ManifestMetadata`](https://github.com/apache/iceberg-rust/blob/eb4c66835d26028436375bf73e0fae28b8d390ce/crates/iceberg/src/spec/manifest.rs#L961) are private and there is no way to access the

Re: [PR] Core: Apply correct metric configs in GenericAppenderFactory [iceberg]

2025-02-26 Thread via GitHub
pvary commented on code in PR #12366: URL: https://github.com/apache/iceberg/pull/12366#discussion_r1971907946 ## data/src/main/java/org/apache/iceberg/data/GenericAppenderFactory.java: ## @@ -66,19 +69,66 @@ public GenericAppenderFactory( int[] equalityFieldIds, S

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
RussellSpitzer commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1971906584 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCommits.java: ## @@ -129,6 +130,52 @@ public void testThriftExceptionUnknownStateIfNotInHistor

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
RussellSpitzer commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1971907807 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCommits.java: ## @@ -129,6 +130,52 @@ public void testThriftExceptionUnknownStateIfNotInHistor

Re: [PR] Core: Apply correct metric configs in GenericAppenderFactory [iceberg]

2025-02-26 Thread via GitHub
pvary commented on code in PR #12366: URL: https://github.com/apache/iceberg/pull/12366#discussion_r1971909256 ## data/src/main/java/org/apache/iceberg/data/GenericAppenderFactory.java: ## @@ -66,19 +69,66 @@ public GenericAppenderFactory( int[] equalityFieldIds, S

Re: [PR] Core: Apply correct metric configs in GenericAppenderFactory [iceberg]

2025-02-26 Thread via GitHub
pvary commented on code in PR #12366: URL: https://github.com/apache/iceberg/pull/12366#discussion_r1971909256 ## data/src/main/java/org/apache/iceberg/data/GenericAppenderFactory.java: ## @@ -66,19 +69,66 @@ public GenericAppenderFactory( int[] equalityFieldIds, S

Re: [PR] Core: Apply correct metric configs in GenericAppenderFactory [iceberg]

2025-02-26 Thread via GitHub
pvary commented on code in PR #12366: URL: https://github.com/apache/iceberg/pull/12366#discussion_r1971911634 ## data/src/main/java/org/apache/iceberg/data/GenericAppenderFactory.java: ## @@ -252,4 +307,24 @@ public PositionDeleteWriter newPosDeleteWriter( throw new Unch

Re: [PR] ci(catalog/rest): initial framework for rest catalog integration tests [iceberg-go]

2025-02-26 Thread via GitHub
zeroshade commented on code in PR #310: URL: https://github.com/apache/iceberg-go/pull/310#discussion_r1971914021 ## catalog/rest/rest_integration_test.go: ## @@ -0,0 +1,178 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreeme

Re: [PR] Fix grammar issues in descriptions about Hive environment in hive-quickstart.md [iceberg]

2025-02-26 Thread via GitHub
pvary merged PR #12402: URL: https://github.com/apache/iceberg/pull/12402 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [I] Flink Table Maintenance [iceberg]

2025-02-26 Thread via GitHub
pvary commented on issue #10264: URL: https://github.com/apache/iceberg/issues/10264#issuecomment-2685550150 @akshat0395: Currently waiting for review on #12306 Getting API changes approved in the core is a slower process, so please be patient After that I will proceed the rema

Re: [PR] Fix grammar issues in descriptions about Hive environment in hive-quickstart.md [iceberg]

2025-02-26 Thread via GitHub
pvary commented on PR #12402: URL: https://github.com/apache/iceberg/pull/12402#issuecomment-2685552568 Thanks for the PR @fredcrawley! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] ci(catalog/rest): initial framework for rest catalog integration tests [iceberg-go]

2025-02-26 Thread via GitHub
zeroshade commented on PR #310: URL: https://github.com/apache/iceberg-go/pull/310#issuecomment-2685551787 @kevinjqliu I'll definitely look into the RCK! thanks for suggesting it! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] [hive]:Fix Hive table creation syntax errors [iceberg]

2025-02-26 Thread via GitHub
pvary merged PR #12394: URL: https://github.com/apache/iceberg/pull/12394 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] [hive]:Fix Hive table creation syntax errors [iceberg]

2025-02-26 Thread via GitHub
pvary commented on PR #12394: URL: https://github.com/apache/iceberg/pull/12394#issuecomment-2685559613 I have missed this PR and haven't read it until now. Thanks for the PR @qining-mj and @ebyhr for the review! -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] support create table like in flink catalog [iceberg]

2025-02-26 Thread via GitHub
stevenzwu commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1966252062 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/FlinkCreateTableOptions.java: ## @@ -0,0 +1,114 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
dramaticlly commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972583055 ## core/src/test/java/org/apache/iceberg/catalog/CatalogTests.java: ## @@ -2871,6 +2872,33 @@ public void testRegisterExistingTable() { assertThat(catalog.dr

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
dramaticlly commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972584384 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCommits.java: ## @@ -129,6 +130,52 @@ public void testThriftExceptionUnknownStateIfNotInHistoryFa

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
dramaticlly commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972585512 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCommits.java: ## @@ -129,6 +130,52 @@ public void testThriftExceptionUnknownStateIfNotInHistoryFa

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
dramaticlly commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972586972 ## core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java: ## @@ -71,23 +71,23 @@ public Table loadTable(TableIdentifier identifier) { } @Overrid

Re: [I] Limit the delete file/records [iceberg]

2025-02-26 Thread via GitHub
bharos commented on issue #12343: URL: https://github.com/apache/iceberg/issues/12343#issuecomment-2686474412 - Can we limit number based on number of data file removal ? In this case, I'm wondering what will be the outcome of the API call? Won't it can cause partial deletes if we do some

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
stevenzwu commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972597618 ## core/src/test/java/org/apache/iceberg/catalog/CatalogTests.java: ## @@ -2871,6 +2872,33 @@ public void testRegisterExistingTable() { assertThat(catalog.drop

Re: [PR] Spark : Derive Stats From Manifest on the Fly [iceberg]

2025-02-26 Thread via GitHub
github-actions[bot] commented on PR #11615: URL: https://github.com/apache/iceberg/pull/11615#issuecomment-2686493099 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] Having 'object name contains unsupported characters' when inserting using partitionedWriter [iceberg]

2025-02-26 Thread via GitHub
github-actions[bot] commented on issue #11051: URL: https://github.com/apache/iceberg/issues/11051#issuecomment-2686493072 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] support create table like in flink catalog [iceberg]

2025-02-26 Thread via GitHub
stevenzwu commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1966284312 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/FlinkCreateTableOptions.java: ## @@ -0,0 +1,114 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] Core: Add Variant logical type for Avro [iceberg]

2025-02-26 Thread via GitHub
rdblue commented on code in PR #12238: URL: https://github.com/apache/iceberg/pull/12238#discussion_r1972617367 ## core/src/main/java/org/apache/iceberg/avro/AvroCustomOrderSchemaVisitor.java: ## @@ -90,6 +105,10 @@ public T map(Schema map, Supplier value) { return null;

Re: [I] DOCS: Clearly document unsupported Iceberg Features [iceberg-python]

2025-02-26 Thread via GitHub
github-actions[bot] commented on issue #1109: URL: https://github.com/apache/iceberg-python/issues/1109#issuecomment-2686496465 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity

Re: [PR] Core: Add Variant logical type for Avro [iceberg]

2025-02-26 Thread via GitHub
rdblue commented on code in PR #12238: URL: https://github.com/apache/iceberg/pull/12238#discussion_r1972624954 ## core/src/main/java/org/apache/iceberg/avro/AvroCustomOrderSchemaVisitor.java: ## @@ -27,13 +27,20 @@ import org.apache.iceberg.relocated.com.google.common.collect.

Re: [PR] Core: Add Variant logical type for Avro [iceberg]

2025-02-26 Thread via GitHub
rdblue commented on code in PR #12238: URL: https://github.com/apache/iceberg/pull/12238#discussion_r1972625719 ## core/src/main/java/org/apache/iceberg/avro/AvroSchemaVisitor.java: ## @@ -25,13 +25,20 @@ import org.apache.iceberg.relocated.com.google.common.collect.Lists; p

Re: [PR] Core: Add Variant logical type for Avro [iceberg]

2025-02-26 Thread via GitHub
rdblue commented on code in PR #12238: URL: https://github.com/apache/iceberg/pull/12238#discussion_r1972625427 ## core/src/main/java/org/apache/iceberg/avro/AvroSchemaUtil.java: ## @@ -219,6 +219,15 @@ public static boolean isKeyValueSchema(Schema schema) { return schema.g

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
dramaticlly commented on PR #12228: URL: https://github.com/apache/iceberg/pull/12228#issuecomment-2686542999 Additional notes of switching from `TableOperations.commit(ops.current, newTableMetadata)` to drop and then re-register: 1. This ensure the same input metadata.json is used to com

Re: [PR] Spark: Rewrite V2 deletes to V3 DVs [iceberg]

2025-02-26 Thread via GitHub
danielcweeks commented on code in PR #12250: URL: https://github.com/apache/iceberg/pull/12250#discussion_r1972648555 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkPositionDeletesRewrite.java: ## @@ -200,7 +207,8 @@ static class PositionDeletesWriterFact

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
dramaticlly commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972649098 ## core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java: ## @@ -71,23 +70,35 @@ public Table loadTable(TableIdentifier identifier) { } @Overrid

Re: [PR] parallelize `add_files` [iceberg-python]

2025-02-26 Thread via GitHub
vtk9 commented on code in PR #1717: URL: https://github.com/apache/iceberg-python/pull/1717#discussion_r1972653853 ## pyiceberg/io/pyarrow.py: ## @@ -2464,38 +2464,37 @@ def _check_pyarrow_schema_compatible( _check_schema_compatible(requested_schema, provided_schema) -d

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
stevenzwu commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972654788 ## core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java: ## @@ -71,23 +70,35 @@ public Table loadTable(TableIdentifier identifier) { } @Override

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
stevenzwu commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972654788 ## core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java: ## @@ -71,23 +70,35 @@ public Table loadTable(TableIdentifier identifier) { } @Override

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
stevenzwu commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972654788 ## core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java: ## @@ -71,23 +70,35 @@ public Table loadTable(TableIdentifier identifier) { } @Override

Re: [PR] parallelize `add_files` [iceberg-python]

2025-02-26 Thread via GitHub
vtk9 commented on code in PR #1717: URL: https://github.com/apache/iceberg-python/pull/1717#discussion_r1972655425 ## pyiceberg/io/pyarrow.py: ## @@ -2464,38 +2464,37 @@ def _check_pyarrow_schema_compatible( _check_schema_compatible(requested_schema, provided_schema) -d

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
dramaticlly commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972649098 ## core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java: ## @@ -71,23 +70,35 @@ public Table loadTable(TableIdentifier identifier) { } @Overrid

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
stevenzwu commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972656806 ## core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java: ## @@ -71,23 +70,35 @@ public Table loadTable(TableIdentifier identifier) { } @Override

Re: [PR] Core: Interface changes for separating rewrite planner and runner [iceberg]

2025-02-26 Thread via GitHub
stevenzwu merged PR #12306: URL: https://github.com/apache/iceberg/pull/12306 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] parallelize `add_files` [iceberg-python]

2025-02-26 Thread via GitHub
vtk9 commented on code in PR #1717: URL: https://github.com/apache/iceberg-python/pull/1717#discussion_r1972652148 ## tests/integration/test_add_files.py: ## @@ -229,6 +229,35 @@ def test_add_files_to_unpartitioned_table_raises_has_field_ids( tbl.add_files(file_paths=f

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
dramaticlly commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972649098 ## core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java: ## @@ -71,23 +70,35 @@ public Table loadTable(TableIdentifier identifier) { } @Overrid

Re: [PR] Core: Interface changes for separating rewrite planner and runner [iceberg]

2025-02-26 Thread via GitHub
stevenzwu commented on PR #12306: URL: https://github.com/apache/iceberg/pull/12306#issuecomment-2686570993 yes, current state looks good to me. let me merge it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
stevenzwu commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972656806 ## core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java: ## @@ -71,23 +70,35 @@ public Table loadTable(TableIdentifier identifier) { } @Override

Re: [PR] fix: refine doc for write support [iceberg-rust]

2025-02-26 Thread via GitHub
liurenjie1024 commented on code in PR #999: URL: https://github.com/apache/iceberg-rust/pull/999#discussion_r1972682553 ## crates/iceberg/src/lib.rs: ## @@ -50,6 +50,87 @@ //! Ok(()) //! } //! ``` +//! +//! ## Fast append data to table +//! +//! ```rust, no_run Review Co

Re: [PR] feat: support position delete writer [iceberg-rust]

2025-02-26 Thread via GitHub
liurenjie1024 commented on code in PR #704: URL: https://github.com/apache/iceberg-rust/pull/704#discussion_r1972684368 ## crates/iceberg/src/writer/base_writer/position_delete_file_writer.rs: ## @@ -0,0 +1,277 @@ +// Licensed to the Apache Software Foundation (ASF) under one +/

Re: [PR] OpenAPI: Use more clear language in recommending error responses [iceberg]

2025-02-26 Thread via GitHub
sungwy commented on code in PR #12376: URL: https://github.com/apache/iceberg/pull/12376#discussion_r1972692336 ## open-api/rest-catalog-open-api.yaml: ## @@ -4566,7 +4568,9 @@ components: AuthenticationTimeoutResponse: description: -Credentials have timed

[I] Ensure absolute path when referencing any file paths [iceberg-python]

2025-02-26 Thread via GitHub
kevinjqliu opened a new issue, #1730: URL: https://github.com/apache/iceberg-python/issues/1730 ### Apache Iceberg version None ### Please describe the bug 🐞 I was able to generate a table which referencing relative paths. I dont think we check this anywhere. I think thi

Re: [PR] Incremental Append Scan [iceberg-python]

2025-02-26 Thread via GitHub
mrendi29 commented on PR #533: URL: https://github.com/apache/iceberg-python/pull/533#issuecomment-2686704958 Is anyone still working on this PR? If not i'd like to give it a try and pick it up from where it was left. :D -- This is an automated message from the Apache Git Service. To res

[PR] pass proxy configuration from environment vars to http client [iceberg]

2025-02-26 Thread via GitHub
akhilputhiry opened a new pull request, #12406: URL: https://github.com/apache/iceberg/pull/12406 This PR Fixes https://github.com/apache/iceberg/issues/12059 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] pass proxy configuration from environment vars to http client [iceberg]

2025-02-26 Thread via GitHub
akhilputhiry commented on PR #12406: URL: https://github.com/apache/iceberg/pull/12406#issuecomment-2686727721 @flyrain Could you please take a look at this PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [PR] [hive]:Fix Hive table creation syntax errors [iceberg]

2025-02-26 Thread via GitHub
qining-mj commented on PR #12394: URL: https://github.com/apache/iceberg/pull/12394#issuecomment-2685538927 May I ask why the PR I submitted has not been merged into the main branch yet? Is there anything else I need to do? -- This is an automated message from the Apache Git Service. To r

Re: [PR] fix(table/scanner): Fix nested field scan [iceberg-go]

2025-02-26 Thread via GitHub
zeroshade commented on code in PR #311: URL: https://github.com/apache/iceberg-go/pull/311#discussion_r1971950577 ## table/scanner_test.go: ## @@ -458,6 +458,23 @@ func (s *ScannerSuite) TestPartitionedTables() { } } +func (s *ScannerSuite) TestNestedColumns() { +

Re: [PR] ci(catalog/rest): initial framework for rest catalog integration tests [iceberg-go]

2025-02-26 Thread via GitHub
zeroshade commented on code in PR #310: URL: https://github.com/apache/iceberg-go/pull/310#discussion_r1971912875 ## .github/workflows/go-integration.yml: ## @@ -66,6 +66,7 @@ jobs: run: | go test -tags integration -v -run="^TestScanner" ./table

Re: [PR] Core: Interface changes for separating rewrite planner and runner [iceberg]

2025-02-26 Thread via GitHub
RussellSpitzer commented on PR #12306: URL: https://github.com/apache/iceberg/pull/12306#issuecomment-2685588950 Ok I'm on board, I think the interfaces are good and Java Doc is solid for now, if @stevenzwu is good with the changes I say we merge and start the rest of the refactor. -- Th

Re: [PR] Spark: Detect dangling DVs properly [iceberg]

2025-02-26 Thread via GitHub
singhpk234 commented on code in PR #12270: URL: https://github.com/apache/iceberg/pull/12270#discussion_r1972008956 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RemoveDanglingDeletesSparkAction.java: ## @@ -156,7 +162,12 @@ private List findDanglingDeletes(

Re: [PR] Spark: Detect dangling DVs properly [iceberg]

2025-02-26 Thread via GitHub
singhpk234 commented on code in PR #12270: URL: https://github.com/apache/iceberg/pull/12270#discussion_r1972041116 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RemoveDanglingDeletesSparkAction.java: ## @@ -124,10 +125,15 @@ private List findDanglingDeletes

Re: [I] Add nanosecond timestamp type [iceberg-python]

2025-02-26 Thread via GitHub
sungwy closed issue #1226: Add nanosecond timestamp type URL: https://github.com/apache/iceberg-python/issues/1226 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

Re: [I] Add nanosecond timestamp type [iceberg-python]

2025-02-26 Thread via GitHub
sungwy commented on issue #1226: URL: https://github.com/apache/iceberg-python/issues/1226#issuecomment-2685794783 Thank you @NoldAndreas for raising this issue. I'll close this as it is a duplicate of https://github.com/apache/iceberg-python/issues/1552 -- This is an automated me

Re: [PR] List data and metadata directories instead of table root [iceberg]

2025-02-26 Thread via GitHub
anuragmantri commented on PR #12278: URL: https://github.com/apache/iceberg/pull/12278#issuecomment-2686319047 Thanks for attempting to fix this @karuppayya. What happens if the `data` and `metadata` locations evolve (if that's possible)? At some point, users may decide to change the met

Re: [PR] OpenAPI: Use more clear language in recommending error responses [iceberg]

2025-02-26 Thread via GitHub
flyrain commented on code in PR #12376: URL: https://github.com/apache/iceberg/pull/12376#discussion_r1972482202 ## open-api/rest-catalog-open-api.yaml: ## @@ -4566,7 +4568,9 @@ components: AuthenticationTimeoutResponse: description: -Credentials have timed

[PR] Build: Bump mkdocstrings-python from 1.16.1 to 1.16.2 [iceberg-python]

2025-02-26 Thread via GitHub
dependabot[bot] opened a new pull request, #1729: URL: https://github.com/apache/iceberg-python/pull/1729 Bumps [mkdocstrings-python](https://github.com/mkdocstrings/python) from 1.16.1 to 1.16.2. Release notes Sourced from https://github.com/mkdocstrings/python/releases";>mkdocstr

Re: [I] How can we override auth.session-timeout-ms of catalog or avoid cache at all? [iceberg]

2025-02-26 Thread via GitHub
ebyhr commented on issue #12350: URL: https://github.com/apache/iceberg/issues/12350#issuecomment-2684226176 @nastra It's not configurable in Trino for now. @varpa89 Could you file an issue at https://github.com/trinodb/trino/issues/new? -- This is an automated message from the A

[PR] Fix grammar issues in descriptions about Hive environment in hive-quickstart.md [iceberg]

2025-02-26 Thread via GitHub
fredcrawley opened a new pull request, #12402: URL: https://github.com/apache/iceberg/pull/12402 Fix grammar issues in descriptions about Hive environment in hive-quickstart.md -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [I] How can we override auth.session-timeout-ms of catalog or avoid cache at all? [iceberg]

2025-02-26 Thread via GitHub
nastra commented on issue #12350: URL: https://github.com/apache/iceberg/issues/12350#issuecomment-2684216846 While it's possible to override `auth.session-timeout-ms` in Iceberg, I actually don't know if it's possible to set this in Trino. @ebyhr can you help out here please? -- This is

Re: [PR] fix: refine doc for write support [iceberg-rust]

2025-02-26 Thread via GitHub
ZENOTME commented on code in PR #999: URL: https://github.com/apache/iceberg-rust/pull/999#discussion_r1971357633 ## crates/iceberg/src/lib.rs: ## @@ -50,6 +50,87 @@ //! Ok(()) //! } //! ``` +//! +//! ## Fast append data to table +//! +//! ```rust, no_run Review Comment:

Re: [PR] ci(catalog/rest): initial framework for rest catalog integration tests [iceberg-go]

2025-02-26 Thread via GitHub
Fokko commented on code in PR #310: URL: https://github.com/apache/iceberg-go/pull/310#discussion_r1971366860 ## catalog/rest/rest_integration_test.go: ## @@ -0,0 +1,178 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

Re: [I] I do not understand the partition error: ValueError: Could not find in old schema: 2: {field}: identity(2) [iceberg-python]

2025-02-26 Thread via GitHub
christophediprima commented on issue #1100: URL: https://github.com/apache/iceberg-python/issues/1100#issuecomment-2684615395 Hi, guys! This thread is very interesting! Has the documentation been updated yet? -- This is an automated message from the Apache Git Service. To respond to th

Re: [PR] ci(catalog/rest): initial framework for rest catalog integration tests [iceberg-go]

2025-02-26 Thread via GitHub
Fokko commented on code in PR #310: URL: https://github.com/apache/iceberg-go/pull/310#discussion_r1971368617 ## .github/workflows/go-integration.yml: ## @@ -66,6 +66,7 @@ jobs: run: | go test -tags integration -v -run="^TestScanner" ./table

Re: [PR] fix(table/scanner): Fix nested field scan [iceberg-go]

2025-02-26 Thread via GitHub
Fokko commented on code in PR #311: URL: https://github.com/apache/iceberg-go/pull/311#discussion_r1971373579 ## table/arrow_utils.go: ## @@ -497,7 +497,10 @@ func (c convertToArrow) Field(field iceberg.NestedField, result arrow.Field) arr func (c convertToArrow) List(list i

Re: [PR] backport c++23 std::expected [iceberg-cpp]

2025-02-26 Thread via GitHub
Fokko merged PR #40: URL: https://github.com/apache/iceberg-cpp/pull/40 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [PR] backport c++23 std::expected [iceberg-cpp]

2025-02-26 Thread via GitHub
Fokko commented on PR #40: URL: https://github.com/apache/iceberg-cpp/pull/40#issuecomment-2684638023 Thanks @zhjwpku for adding this, and the `LICENSE` looks good to me. I don't think we need to update the `NOTICE`, but we can revisit that at the release when we have to do a full pass anyw

Re: [I] How can we override auth.session-timeout-ms of catalog or avoid cache at all? [iceberg]

2025-02-26 Thread via GitHub
varpa89 commented on issue #12350: URL: https://github.com/apache/iceberg/issues/12350#issuecomment-2684394474 @ebyhr thank you for your response. yes, looks like it's not configurable for now, but I found what should we add to Trino, so I'm preparing a pull request and we can discuss wheth

Re: [PR] Spark: Detect dangling DVs properly [iceberg]

2025-02-26 Thread via GitHub
nastra commented on code in PR #12270: URL: https://github.com/apache/iceberg/pull/12270#discussion_r1971224641 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RemoveDanglingDeletesSparkAction.java: ## @@ -124,10 +125,15 @@ private List findDanglingDeletes() {

Re: [PR] Spark: Detect dangling DVs properly [iceberg]

2025-02-26 Thread via GitHub
nastra commented on code in PR #12270: URL: https://github.com/apache/iceberg/pull/12270#discussion_r1971224641 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RemoveDanglingDeletesSparkAction.java: ## @@ -124,10 +125,15 @@ private List findDanglingDeletes() {

[PR] Scan Delete Support Part 5: Positional Delete Parsing [iceberg-rust]

2025-02-26 Thread via GitHub
sdd opened a new pull request, #1011: URL: https://github.com/apache/iceberg-rust/pull/1011 Continues the series of scan delete file support PRs. * Adds parsing of positional delete files into * Updates the integration tests in `crates/integration_tests/tests/shared_tests/read_positio

Re: [PR] feat(core): expose remove_snapshots at Transaction API [iceberg-rust]

2025-02-26 Thread via GitHub
ZENOTME commented on PR #884: URL: https://github.com/apache/iceberg-rust/pull/884#issuecomment-2684472066 > I am not sure if this is a server issue or a client issue, more research is needed. Looks like it depends on the catalog. For list of snapshots remove, it will apply to metada

Re: [PR] Build: Bump mkdocstrings from 0.28.1 to 0.28.2 [iceberg-python]

2025-02-26 Thread via GitHub
Fokko merged PR #1723: URL: https://github.com/apache/iceberg-python/pull/1723 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

[PR] fix: fix http custom headers [iceberg-rust]

2025-02-26 Thread via GitHub
chenzl25 opened a new pull request, #1010: URL: https://github.com/apache/iceberg-rust/pull/1010 - Add `extra_headers` to the request. Previously, we didn't use `extra_headers`. - Support multiple values for customer headers. Because we use a HashMap as RestCatalog properties, we can't s

Re: [I] How can we override auth.session-timeout-ms of catalog or avoid cache at all? [iceberg]

2025-02-26 Thread via GitHub
nastra commented on issue #12350: URL: https://github.com/apache/iceberg/issues/12350#issuecomment-2684431724 I'll go ahead and close this, since this is specific to Trino -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

Re: [I] How can we override auth.session-timeout-ms of catalog or avoid cache at all? [iceberg]

2025-02-26 Thread via GitHub
nastra closed issue #12350: How can we override auth.session-timeout-ms of catalog or avoid cache at all? URL: https://github.com/apache/iceberg/issues/12350 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [PR] fix: refine doc for write support [iceberg-rust]

2025-02-26 Thread via GitHub
liurenjie1024 commented on code in PR #999: URL: https://github.com/apache/iceberg-rust/pull/999#discussion_r1971332628 ## crates/iceberg/src/lib.rs: ## @@ -50,6 +50,87 @@ //! Ok(()) //! } //! ``` +//! +//! ## Fast append data to table +//! +//! ```rust, no_run Review Co

Re: [PR] fix: refine doc for write support [iceberg-rust]

2025-02-26 Thread via GitHub
liurenjie1024 commented on code in PR #999: URL: https://github.com/apache/iceberg-rust/pull/999#discussion_r1971332628 ## crates/iceberg/src/lib.rs: ## @@ -50,6 +50,87 @@ //! Ok(()) //! } //! ``` +//! +//! ## Fast append data to table +//! +//! ```rust, no_run Review Co

Re: [PR] Build: remove Hadoop 2 dependency [iceberg]

2025-02-26 Thread via GitHub
Fokko merged PR #12348: URL: https://github.com/apache/iceberg/pull/12348 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Build: remove Hadoop 2 dependency [iceberg]

2025-02-26 Thread via GitHub
Fokko commented on PR #12348: URL: https://github.com/apache/iceberg/pull/12348#issuecomment-2684745027 Looks good, thanks @Kontinuation for working on this, and thanks @manuzhang and @nastra for the review! -- This is an automated message from the Apache Git Service. To respond to the me

  1   2   >