Re: [PR] Core,REST: extend httpClient builder to support tls factory [iceberg]

2025-02-26 Thread via GitHub
munendrasn commented on code in PR #11979: URL: https://github.com/apache/iceberg/pull/11979#discussion_r1973048206 ## core/src/main/java/org/apache/iceberg/rest/HTTPClient.java: ## @@ -480,6 +486,13 @@ public Builder withObjectMapper(ObjectMapper objectMapper) { return

Re: [PR] Core,REST: extend httpClient builder to support tls factory [iceberg]

2025-02-26 Thread via GitHub
munendrasn commented on code in PR #11979: URL: https://github.com/apache/iceberg/pull/11979#discussion_r1973048206 ## core/src/main/java/org/apache/iceberg/rest/HTTPClient.java: ## @@ -480,6 +486,13 @@ public Builder withObjectMapper(ObjectMapper objectMapper) { return

Re: [PR] Data: Add partition stats writer and reader [iceberg]

2025-02-26 Thread via GitHub
ajantha-bhat commented on PR #11216: URL: https://github.com/apache/iceberg/pull/11216#issuecomment-2687054193 @RussellSpitzer, @aokolnychyi: Anymore comments? others have approved the changes. -- This is an automated message from the Apache Git Service. To respond to the message, plea

Re: [PR] Auth Manager API part 6: API enablement [iceberg]

2025-02-26 Thread via GitHub
ajantha-bhat commented on PR #12197: URL: https://github.com/apache/iceberg/pull/12197#issuecomment-2687048478 @danielcweeks and @nastra: Can we please start the review on this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Parquet: Implement Variant writers [iceberg]

2025-02-26 Thread via GitHub
aihuaxu commented on code in PR #12323: URL: https://github.com/apache/iceberg/pull/12323#discussion_r1972563111 ## core/src/main/java/org/apache/iceberg/variants/VariantVisitor.java: ## @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

Re: [PR] Spark: Detect dangling DVs properly [iceberg]

2025-02-26 Thread via GitHub
singhpk234 commented on code in PR #12270: URL: https://github.com/apache/iceberg/pull/12270#discussion_r1972008956 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RemoveDanglingDeletesSparkAction.java: ## @@ -156,7 +162,12 @@ private List findDanglingDeletes(

Re: [PR] support create table like in flink catalog [iceberg]

2025-02-26 Thread via GitHub
swapna267 commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1972887391 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalog.java: ## @@ -332,7 +335,22 @@ public List listTables(String databaseName) public Catalog

Re: [PR] support create table like in flink catalog [iceberg]

2025-02-26 Thread via GitHub
swapna267 commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1972885322 ## flink/v1.20/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceSql.java: ## @@ -162,4 +162,21 @@ public void testWatermarkOptionsDescending() t

Re: [PR] support create table like in flink catalog [iceberg]

2025-02-26 Thread via GitHub
swapna267 commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1972880531 ## flink/v1.20/flink/src/test/java/org/apache/iceberg/flink/TestFlinkCatalogTable.java: ## @@ -188,6 +188,37 @@ public void testCreateTableLike() throws TableNotEx

Re: [PR] support create table like in flink catalog [iceberg]

2025-02-26 Thread via GitHub
swapna267 commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1972877314 ## flink/v1.20/flink/src/test/java/org/apache/iceberg/flink/TestFlinkCatalogTable.java: ## @@ -188,6 +188,37 @@ public void testCreateTableLike() throws TableNotEx

Re: [PR] support create table like in flink catalog [iceberg]

2025-02-26 Thread via GitHub
swapna267 commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1972866279 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/FlinkCreateTableOptions.java: ## @@ -0,0 +1,114 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] chore(spec): add accessor methods for ManifestMetadata [iceberg-rust]

2025-02-26 Thread via GitHub
liurenjie1024 merged PR #1013: URL: https://github.com/apache/iceberg-rust/pull/1013 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [I] Support accessors for ManifestMetadata fields [iceberg-rust]

2025-02-26 Thread via GitHub
liurenjie1024 closed issue #1012: Support accessors for ManifestMetadata fields URL: https://github.com/apache/iceberg-rust/issues/1012 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [I] Support accessors for ManifestMetadata fields [iceberg-rust]

2025-02-26 Thread via GitHub
liurenjie1024 closed issue #1012: Support accessors for ManifestMetadata fields URL: https://github.com/apache/iceberg-rust/issues/1012 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] feat: support delete if empty for parquet writer [iceberg-rust]

2025-02-26 Thread via GitHub
ZENOTME commented on PR #838: URL: https://github.com/apache/iceberg-rust/pull/838#issuecomment-2686827200 I think this PR is ready to review. cc @Xuanwo @liurenjie1024 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [PR] feat: support position delete writer [iceberg-rust]

2025-02-26 Thread via GitHub
ZENOTME commented on code in PR #704: URL: https://github.com/apache/iceberg-rust/pull/704#discussion_r1972829889 ## crates/iceberg/src/writer/base_writer/position_delete_file_writer.rs: ## @@ -0,0 +1,277 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or m

Re: [PR] feat: support `arrow_struct_to_iceberg_struct` [iceberg-rust]

2025-02-26 Thread via GitHub
liurenjie1024 merged PR #731: URL: https://github.com/apache/iceberg-rust/pull/731 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] pass proxy configuration from environment vars to http client [iceberg]

2025-02-26 Thread via GitHub
akhilputhiry commented on PR #12406: URL: https://github.com/apache/iceberg/pull/12406#issuecomment-2686727721 @flyrain Could you please take a look at this PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[PR] pass proxy configuration from environment vars to http client [iceberg]

2025-02-26 Thread via GitHub
akhilputhiry opened a new pull request, #12406: URL: https://github.com/apache/iceberg/pull/12406 This PR Fixes https://github.com/apache/iceberg/issues/12059 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Incremental Append Scan [iceberg-python]

2025-02-26 Thread via GitHub
mrendi29 commented on PR #533: URL: https://github.com/apache/iceberg-python/pull/533#issuecomment-2686704958 Is anyone still working on this PR? If not i'd like to give it a try and pick it up from where it was left. :D -- This is an automated message from the Apache Git Service. To res

[I] Ensure absolute path when referencing any file paths [iceberg-python]

2025-02-26 Thread via GitHub
kevinjqliu opened a new issue, #1730: URL: https://github.com/apache/iceberg-python/issues/1730 ### Apache Iceberg version None ### Please describe the bug ๐Ÿž I was able to generate a table which referencing relative paths. I dont think we check this anywhere. I think thi

Re: [PR] OpenAPI: Use more clear language in recommending error responses [iceberg]

2025-02-26 Thread via GitHub
sungwy commented on code in PR #12376: URL: https://github.com/apache/iceberg/pull/12376#discussion_r1972692336 ## open-api/rest-catalog-open-api.yaml: ## @@ -4566,7 +4568,9 @@ components: AuthenticationTimeoutResponse: description: -Credentials have timed

Re: [PR] feat: support position delete writer [iceberg-rust]

2025-02-26 Thread via GitHub
liurenjie1024 commented on code in PR #704: URL: https://github.com/apache/iceberg-rust/pull/704#discussion_r1972684368 ## crates/iceberg/src/writer/base_writer/position_delete_file_writer.rs: ## @@ -0,0 +1,277 @@ +// Licensed to the Apache Software Foundation (ASF) under one +/

Re: [PR] fix: refine doc for write support [iceberg-rust]

2025-02-26 Thread via GitHub
liurenjie1024 commented on code in PR #999: URL: https://github.com/apache/iceberg-rust/pull/999#discussion_r1972682553 ## crates/iceberg/src/lib.rs: ## @@ -50,6 +50,87 @@ //! Ok(()) //! } //! ``` +//! +//! ## Fast append data to table +//! +//! ```rust, no_run Review Co

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
stevenzwu commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972656806 ## core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java: ## @@ -71,23 +70,35 @@ public Table loadTable(TableIdentifier identifier) { } @Override

Re: [PR] Core: Interface changes for separating rewrite planner and runner [iceberg]

2025-02-26 Thread via GitHub
stevenzwu commented on PR #12306: URL: https://github.com/apache/iceberg/pull/12306#issuecomment-2686570993 yes, current state looks good to me. let me merge it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
dramaticlly commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972649098 ## core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java: ## @@ -71,23 +70,35 @@ public Table loadTable(TableIdentifier identifier) { } @Overrid

Re: [PR] parallelize `add_files` [iceberg-python]

2025-02-26 Thread via GitHub
vtk9 commented on code in PR #1717: URL: https://github.com/apache/iceberg-python/pull/1717#discussion_r1972652148 ## tests/integration/test_add_files.py: ## @@ -229,6 +229,35 @@ def test_add_files_to_unpartitioned_table_raises_has_field_ids( tbl.add_files(file_paths=f

Re: [PR] Core: Interface changes for separating rewrite planner and runner [iceberg]

2025-02-26 Thread via GitHub
stevenzwu merged PR #12306: URL: https://github.com/apache/iceberg/pull/12306 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
stevenzwu commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972656806 ## core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java: ## @@ -71,23 +70,35 @@ public Table loadTable(TableIdentifier identifier) { } @Override

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
dramaticlly commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972649098 ## core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java: ## @@ -71,23 +70,35 @@ public Table loadTable(TableIdentifier identifier) { } @Overrid

Re: [PR] parallelize `add_files` [iceberg-python]

2025-02-26 Thread via GitHub
vtk9 commented on code in PR #1717: URL: https://github.com/apache/iceberg-python/pull/1717#discussion_r1972655425 ## pyiceberg/io/pyarrow.py: ## @@ -2464,38 +2464,37 @@ def _check_pyarrow_schema_compatible( _check_schema_compatible(requested_schema, provided_schema) -d

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
stevenzwu commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972654788 ## core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java: ## @@ -71,23 +70,35 @@ public Table loadTable(TableIdentifier identifier) { } @Override

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
stevenzwu commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972654788 ## core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java: ## @@ -71,23 +70,35 @@ public Table loadTable(TableIdentifier identifier) { } @Override

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
stevenzwu commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972654788 ## core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java: ## @@ -71,23 +70,35 @@ public Table loadTable(TableIdentifier identifier) { } @Override

Re: [PR] parallelize `add_files` [iceberg-python]

2025-02-26 Thread via GitHub
vtk9 commented on code in PR #1717: URL: https://github.com/apache/iceberg-python/pull/1717#discussion_r1972653853 ## pyiceberg/io/pyarrow.py: ## @@ -2464,38 +2464,37 @@ def _check_pyarrow_schema_compatible( _check_schema_compatible(requested_schema, provided_schema) -d

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
dramaticlly commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972649098 ## core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java: ## @@ -71,23 +70,35 @@ public Table loadTable(TableIdentifier identifier) { } @Overrid

Re: [PR] Spark: Rewrite V2 deletes to V3 DVs [iceberg]

2025-02-26 Thread via GitHub
danielcweeks commented on code in PR #12250: URL: https://github.com/apache/iceberg/pull/12250#discussion_r1972648555 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkPositionDeletesRewrite.java: ## @@ -200,7 +207,8 @@ static class PositionDeletesWriterFact

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
dramaticlly commented on PR #12228: URL: https://github.com/apache/iceberg/pull/12228#issuecomment-2686542999 Additional notes of switching from `TableOperations.commit(ops.current, newTableMetadata)` to drop and then re-register: 1. This ensure the same input metadata.json is used to com

Re: [PR] Core: Add Variant logical type for Avro [iceberg]

2025-02-26 Thread via GitHub
rdblue commented on code in PR #12238: URL: https://github.com/apache/iceberg/pull/12238#discussion_r1972625427 ## core/src/main/java/org/apache/iceberg/avro/AvroSchemaUtil.java: ## @@ -219,6 +219,15 @@ public static boolean isKeyValueSchema(Schema schema) { return schema.g

Re: [PR] Core: Add Variant logical type for Avro [iceberg]

2025-02-26 Thread via GitHub
rdblue commented on code in PR #12238: URL: https://github.com/apache/iceberg/pull/12238#discussion_r1972625719 ## core/src/main/java/org/apache/iceberg/avro/AvroSchemaVisitor.java: ## @@ -25,13 +25,20 @@ import org.apache.iceberg.relocated.com.google.common.collect.Lists; p

Re: [PR] Core: Add Variant logical type for Avro [iceberg]

2025-02-26 Thread via GitHub
rdblue commented on code in PR #12238: URL: https://github.com/apache/iceberg/pull/12238#discussion_r1972624954 ## core/src/main/java/org/apache/iceberg/avro/AvroCustomOrderSchemaVisitor.java: ## @@ -27,13 +27,20 @@ import org.apache.iceberg.relocated.com.google.common.collect.

Re: [I] DOCS: Clearly document unsupported Iceberg Features [iceberg-python]

2025-02-26 Thread via GitHub
github-actions[bot] commented on issue #1109: URL: https://github.com/apache/iceberg-python/issues/1109#issuecomment-2686496465 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity

Re: [PR] Core: Add Variant logical type for Avro [iceberg]

2025-02-26 Thread via GitHub
rdblue commented on code in PR #12238: URL: https://github.com/apache/iceberg/pull/12238#discussion_r1972617367 ## core/src/main/java/org/apache/iceberg/avro/AvroCustomOrderSchemaVisitor.java: ## @@ -90,6 +105,10 @@ public T map(Schema map, Supplier value) { return null;

Re: [PR] support create table like in flink catalog [iceberg]

2025-02-26 Thread via GitHub
stevenzwu commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1966284312 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/FlinkCreateTableOptions.java: ## @@ -0,0 +1,114 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [I] Having 'object name contains unsupported characters' when inserting using partitionedWriter [iceberg]

2025-02-26 Thread via GitHub
github-actions[bot] commented on issue #11051: URL: https://github.com/apache/iceberg/issues/11051#issuecomment-2686493072 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Spark : Derive Stats From Manifest on the Fly [iceberg]

2025-02-26 Thread via GitHub
github-actions[bot] commented on PR #11615: URL: https://github.com/apache/iceberg/pull/11615#issuecomment-2686493099 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think thatโ€™s incorrect or this pul

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
stevenzwu commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972597618 ## core/src/test/java/org/apache/iceberg/catalog/CatalogTests.java: ## @@ -2871,6 +2872,33 @@ public void testRegisterExistingTable() { assertThat(catalog.drop

Re: [I] Limit the delete file/records [iceberg]

2025-02-26 Thread via GitHub
bharos commented on issue #12343: URL: https://github.com/apache/iceberg/issues/12343#issuecomment-2686474412 - Can we limit number based on number of data file removal ? In this case, I'm wondering what will be the outcome of the API call? Won't it can cause partial deletes if we do some

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
dramaticlly commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972586972 ## core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java: ## @@ -71,23 +71,23 @@ public Table loadTable(TableIdentifier identifier) { } @Overrid

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
dramaticlly commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972585512 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCommits.java: ## @@ -129,6 +130,52 @@ public void testThriftExceptionUnknownStateIfNotInHistoryFa

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
dramaticlly commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972584384 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCommits.java: ## @@ -129,6 +130,52 @@ public void testThriftExceptionUnknownStateIfNotInHistoryFa

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
dramaticlly commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972583055 ## core/src/test/java/org/apache/iceberg/catalog/CatalogTests.java: ## @@ -2871,6 +2872,33 @@ public void testRegisterExistingTable() { assertThat(catalog.dr

Re: [PR] support create table like in flink catalog [iceberg]

2025-02-26 Thread via GitHub
stevenzwu commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1966252062 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/FlinkCreateTableOptions.java: ## @@ -0,0 +1,114 @@ +/* + * Licensed to the Apache Software Foundation (A

[PR] Build: Bump mkdocstrings-python from 1.16.1 to 1.16.2 [iceberg-python]

2025-02-26 Thread via GitHub
dependabot[bot] opened a new pull request, #1729: URL: https://github.com/apache/iceberg-python/pull/1729 Bumps [mkdocstrings-python](https://github.com/mkdocstrings/python) from 1.16.1 to 1.16.2. Release notes Sourced from https://github.com/mkdocstrings/python/releases";>mkdocstr

Re: [PR] List data and metadata directories instead of table root [iceberg]

2025-02-26 Thread via GitHub
anuragmantri commented on PR #12278: URL: https://github.com/apache/iceberg/pull/12278#issuecomment-2686319047 Thanks for attempting to fix this @karuppayya. What happens if the `data` and `metadata` locations evolve (if that's possible)? At some point, users may decide to change the met

Re: [PR] OpenAPI: Use more clear language in recommending error responses [iceberg]

2025-02-26 Thread via GitHub
flyrain commented on code in PR #12376: URL: https://github.com/apache/iceberg/pull/12376#discussion_r1972482202 ## open-api/rest-catalog-open-api.yaml: ## @@ -4566,7 +4568,9 @@ components: AuthenticationTimeoutResponse: description: -Credentials have timed

Re: [PR] List data and metadata directories instead of table root [iceberg]

2025-02-26 Thread via GitHub
karuppayya commented on code in PR #12278: URL: https://github.com/apache/iceberg/pull/12278#discussion_r1972303374 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/DeleteOrphanFilesSparkAction.java: ## @@ -301,24 +303,34 @@ private Dataset actualFileIdentDS()

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
dramaticlly commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1972171501 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCommits.java: ## @@ -129,6 +130,52 @@ public void testThriftExceptionUnknownStateIfNotInHistoryFa

Re: [I] Add nanosecond timestamp type [iceberg-python]

2025-02-26 Thread via GitHub
sungwy commented on issue #1226: URL: https://github.com/apache/iceberg-python/issues/1226#issuecomment-2685794783 Thank you @NoldAndreas for raising this issue. I'll close this as it is a duplicate of https://github.com/apache/iceberg-python/issues/1552 -- This is an automated me

Re: [I] Add nanosecond timestamp type [iceberg-python]

2025-02-26 Thread via GitHub
sungwy closed issue #1226: Add nanosecond timestamp type URL: https://github.com/apache/iceberg-python/issues/1226 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

Re: [PR] ci(catalog/rest): initial framework for rest catalog integration tests [iceberg-go]

2025-02-26 Thread via GitHub
zeroshade commented on code in PR #310: URL: https://github.com/apache/iceberg-go/pull/310#discussion_r1971912875 ## .github/workflows/go-integration.yml: ## @@ -66,6 +66,7 @@ jobs: run: | go test -tags integration -v -run="^TestScanner" ./table

Re: [PR] Spark: Rewrite V2 deletes to V3 DVs [iceberg]

2025-02-26 Thread via GitHub
nastra closed pull request #12250: Spark: Rewrite V2 deletes to V3 DVs URL: https://github.com/apache/iceberg/pull/12250 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

Re: [PR] Spark: Detect dangling DVs properly [iceberg]

2025-02-26 Thread via GitHub
singhpk234 commented on code in PR #12270: URL: https://github.com/apache/iceberg/pull/12270#discussion_r1972041116 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RemoveDanglingDeletesSparkAction.java: ## @@ -124,10 +125,15 @@ private List findDanglingDeletes

Re: [PR] Spark: Detect dangling DVs properly [iceberg]

2025-02-26 Thread via GitHub
singhpk234 commented on code in PR #12270: URL: https://github.com/apache/iceberg/pull/12270#discussion_r1972041116 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RemoveDanglingDeletesSparkAction.java: ## @@ -124,10 +125,15 @@ private List findDanglingDeletes

Re: [PR] Spark: Detect dangling DVs properly [iceberg]

2025-02-26 Thread via GitHub
singhpk234 commented on code in PR #12270: URL: https://github.com/apache/iceberg/pull/12270#discussion_r1972008956 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RemoveDanglingDeletesSparkAction.java: ## @@ -156,7 +162,12 @@ private List findDanglingDeletes(

Re: [PR] ci(catalog/rest): initial framework for rest catalog integration tests [iceberg-go]

2025-02-26 Thread via GitHub
zeroshade merged PR #310: URL: https://github.com/apache/iceberg-go/pull/310 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] fix(table/scanner): Fix nested field scan [iceberg-go]

2025-02-26 Thread via GitHub
zeroshade commented on code in PR #311: URL: https://github.com/apache/iceberg-go/pull/311#discussion_r1971976048 ## table/arrow_utils.go: ## @@ -497,7 +497,10 @@ func (c convertToArrow) Field(field iceberg.NestedField, result arrow.Field) arr func (c convertToArrow) List(li

Re: [PR] 1191:Added docstrings to the pyiceberg/table/inspect.py file [iceberg-python]

2025-02-26 Thread via GitHub
gayatrikate04 commented on PR #1533: URL: https://github.com/apache/iceberg-python/pull/1533#issuecomment-2685638054 resolved conflicts -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] fix(table/scanner): Fix nested field scan [iceberg-go]

2025-02-26 Thread via GitHub
zeroshade commented on code in PR #311: URL: https://github.com/apache/iceberg-go/pull/311#discussion_r1971950577 ## table/scanner_test.go: ## @@ -458,6 +458,23 @@ func (s *ScannerSuite) TestPartitionedTables() { } } +func (s *ScannerSuite) TestNestedColumns() { +

Re: [PR] Core: Interface changes for separating rewrite planner and runner [iceberg]

2025-02-26 Thread via GitHub
RussellSpitzer commented on PR #12306: URL: https://github.com/apache/iceberg/pull/12306#issuecomment-2685588950 Ok I'm on board, I think the interfaces are good and Java Doc is solid for now, if @stevenzwu is good with the changes I say we merge and start the rest of the refactor. -- Th

Re: [PR] [hive]:Fix Hive table creation syntax errors [iceberg]

2025-02-26 Thread via GitHub
qining-mj commented on PR #12394: URL: https://github.com/apache/iceberg/pull/12394#issuecomment-2685538927 May I ask why the PR I submitted has not been merged into the main branch yet? Is there anything else I need to do? -- This is an automated message from the Apache Git Service. To r

Re: [PR] [hive]:Fix Hive table creation syntax errors [iceberg]

2025-02-26 Thread via GitHub
pvary commented on PR #12394: URL: https://github.com/apache/iceberg/pull/12394#issuecomment-2685559613 I have missed this PR and haven't read it until now. Thanks for the PR @qining-mj and @ebyhr for the review! -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] [hive]:Fix Hive table creation syntax errors [iceberg]

2025-02-26 Thread via GitHub
pvary merged PR #12394: URL: https://github.com/apache/iceberg/pull/12394 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] ci(catalog/rest): initial framework for rest catalog integration tests [iceberg-go]

2025-02-26 Thread via GitHub
zeroshade commented on PR #310: URL: https://github.com/apache/iceberg-go/pull/310#issuecomment-2685551787 @kevinjqliu I'll definitely look into the RCK! thanks for suggesting it! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [I] Flink Table Maintenance [iceberg]

2025-02-26 Thread via GitHub
pvary commented on issue #10264: URL: https://github.com/apache/iceberg/issues/10264#issuecomment-2685550150 @akshat0395: Currently waiting for review on #12306 Getting API changes approved in the core is a slower process, so please be patient After that I will proceed the rema

Re: [PR] Fix grammar issues in descriptions about Hive environment in hive-quickstart.md [iceberg]

2025-02-26 Thread via GitHub
pvary merged PR #12402: URL: https://github.com/apache/iceberg/pull/12402 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Fix grammar issues in descriptions about Hive environment in hive-quickstart.md [iceberg]

2025-02-26 Thread via GitHub
pvary commented on PR #12402: URL: https://github.com/apache/iceberg/pull/12402#issuecomment-2685552568 Thanks for the PR @fredcrawley! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] ci(catalog/rest): initial framework for rest catalog integration tests [iceberg-go]

2025-02-26 Thread via GitHub
zeroshade commented on code in PR #310: URL: https://github.com/apache/iceberg-go/pull/310#discussion_r1971914021 ## catalog/rest/rest_integration_test.go: ## @@ -0,0 +1,178 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreeme

Re: [PR] Core: Apply correct metric configs in GenericAppenderFactory [iceberg]

2025-02-26 Thread via GitHub
pvary commented on code in PR #12366: URL: https://github.com/apache/iceberg/pull/12366#discussion_r1971911634 ## data/src/main/java/org/apache/iceberg/data/GenericAppenderFactory.java: ## @@ -252,4 +307,24 @@ public PositionDeleteWriter newPosDeleteWriter( throw new Unch

Re: [PR] Core: Apply correct metric configs in GenericAppenderFactory [iceberg]

2025-02-26 Thread via GitHub
pvary commented on code in PR #12366: URL: https://github.com/apache/iceberg/pull/12366#discussion_r1971909256 ## data/src/main/java/org/apache/iceberg/data/GenericAppenderFactory.java: ## @@ -66,19 +69,66 @@ public GenericAppenderFactory( int[] equalityFieldIds, S

Re: [PR] Core: Apply correct metric configs in GenericAppenderFactory [iceberg]

2025-02-26 Thread via GitHub
pvary commented on code in PR #12366: URL: https://github.com/apache/iceberg/pull/12366#discussion_r1971907946 ## data/src/main/java/org/apache/iceberg/data/GenericAppenderFactory.java: ## @@ -66,19 +69,66 @@ public GenericAppenderFactory( int[] equalityFieldIds, S

Re: [PR] Core: Apply correct metric configs in GenericAppenderFactory [iceberg]

2025-02-26 Thread via GitHub
pvary commented on code in PR #12366: URL: https://github.com/apache/iceberg/pull/12366#discussion_r1971909256 ## data/src/main/java/org/apache/iceberg/data/GenericAppenderFactory.java: ## @@ -66,19 +69,66 @@ public GenericAppenderFactory( int[] equalityFieldIds, S

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
RussellSpitzer commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1971906584 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCommits.java: ## @@ -129,6 +130,52 @@ public void testThriftExceptionUnknownStateIfNotInHistor

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-02-26 Thread via GitHub
RussellSpitzer commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r1971907807 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCommits.java: ## @@ -129,6 +130,52 @@ public void testThriftExceptionUnknownStateIfNotInHistor

Re: [PR] ci(catalog/rest): initial framework for rest catalog integration tests [iceberg-go]

2025-02-26 Thread via GitHub
kevinjqliu commented on code in PR #310: URL: https://github.com/apache/iceberg-go/pull/310#discussion_r1971884342 ## .github/workflows/go-integration.yml: ## @@ -66,6 +66,7 @@ jobs: run: | go test -tags integration -v -run="^TestScanner" ./table

Re: [PR] View Spec: Fix typo [iceberg]

2025-02-26 Thread via GitHub
nastra merged PR #12405: URL: https://github.com/apache/iceberg/pull/12405 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

[I] Support accessors for ManifestMetadata fields [iceberg-rust]

2025-02-26 Thread via GitHub
mnpw opened a new issue, #1012: URL: https://github.com/apache/iceberg-rust/issues/1012 Fields of [`ManifestMetadata`](https://github.com/apache/iceberg-rust/blob/eb4c66835d26028436375bf73e0fae28b8d390ce/crates/iceberg/src/spec/manifest.rs#L961) are private and there is no way to access the

[PR] View Spec: Fix typo [iceberg]

2025-02-26 Thread via GitHub
ajantha-bhat opened a new pull request, #12405: URL: https://github.com/apache/iceberg/pull/12405 fix copy paste error -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [PR] Build: Bump junit from 5.11.4 to 5.12.0 [iceberg]

2025-02-26 Thread via GitHub
dependabot[bot] commented on PR #12378: URL: https://github.com/apache/iceberg/pull/12378#issuecomment-2684996449 OK, I won't notify you again about this release, but will get in touch when a new version is available. You can also ignore all major, minor, or patch releases for a dependency

Re: [PR] Build: Bump junit from 5.11.4 to 5.12.0 [iceberg]

2025-02-26 Thread via GitHub
nastra closed pull request #12378: Build: Bump junit from 5.11.4 to 5.12.0 URL: https://github.com/apache/iceberg/pull/12378 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [I] Proxy Settings for catalog REST API client [iceberg]

2025-02-26 Thread via GitHub
akhilputhiry commented on issue #12059: URL: https://github.com/apache/iceberg/issues/12059#issuecomment-2684992598 I am working on this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [I] [Feature] - Allow Schema Overwrite [iceberg-python]

2025-02-26 Thread via GitHub
smaheshwar-pltr commented on issue #1726: URL: https://github.com/apache/iceberg-python/issues/1726#issuecomment-2684966894 +1 - I'd also like for schema replacement to be supported like this, unless I'm mistaken and it is already. Without https://github.com/apache/iceberg-python/pul

Re: [PR] Core,REST: extend httpClient builder to support tls factory [iceberg]

2025-02-26 Thread via GitHub
munendrasn commented on code in PR #11979: URL: https://github.com/apache/iceberg/pull/11979#discussion_r1971573647 ## core/src/main/java/org/apache/iceberg/rest/HTTPClient.java: ## @@ -480,6 +486,11 @@ public Builder withObjectMapper(ObjectMapper objectMapper) { return

[PR] Spark 3.5: Adjust repeated INFO logs to DEBUG in SparkWrite and SparkPositionDeltaWrite [iceberg]

2025-02-26 Thread via GitHub
manuzhang opened a new pull request, #12404: URL: https://github.com/apache/iceberg/pull/12404 I see repeated logs as follows from `SparkWrite` in Spark driver. ``` 25/02/26 01:48:56 INFO SparkWrite: Requesting 0 bytes advisory partition size for table *** 25/02/26 01:48:56 INFO

[I] Python defined schema does not match created schema [iceberg-python]

2025-02-26 Thread via GitHub
grbinho opened a new issue, #1728: URL: https://github.com/apache/iceberg-python/issues/1728 ### Apache Iceberg version 0.8.1 (latest release) ### Please describe the bug ๐Ÿž Hi, I'm using pyiceberg with Glue and S3 and while creating a table that contains list type

Re: [PR] Build: remove Hadoop 2 dependency [iceberg]

2025-02-26 Thread via GitHub
Fokko merged PR #12348: URL: https://github.com/apache/iceberg/pull/12348 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Build: remove Hadoop 2 dependency [iceberg]

2025-02-26 Thread via GitHub
Fokko commented on PR #12348: URL: https://github.com/apache/iceberg/pull/12348#issuecomment-2684745027 Looks good, thanks @Kontinuation for working on this, and thanks @manuzhang and @nastra for the review! -- This is an automated message from the Apache Git Service. To respond to the me

Re: [PR] backport c++23 std::expected [iceberg-cpp]

2025-02-26 Thread via GitHub
Fokko commented on PR #40: URL: https://github.com/apache/iceberg-cpp/pull/40#issuecomment-2684638023 Thanks @zhjwpku for adding this, and the `LICENSE` looks good to me. I don't think we need to update the `NOTICE`, but we can revisit that at the release when we have to do a full pass anyw

Re: [PR] backport c++23 std::expected [iceberg-cpp]

2025-02-26 Thread via GitHub
Fokko merged PR #40: URL: https://github.com/apache/iceberg-cpp/pull/40 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

  1   2   >