Re: [PR] Docs: add apache amoro(incubating) with iceberg (#11965) [iceberg]

nastra merged PR #11966: URL: https://github.com/apache/iceberg/pull/11966 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [I] Failed to snapshot iceberg table [iceberg]

DattaHembade commented on issue #2555: URL: https://github.com/apache/iceberg/issues/2555#issuecomment-2667833650 04:21:05Runtime Error in snapshot iceberg_table_snapshot (snapshots\iceberg_table_snapshot.sql) Database Error org.apache.hive.service.cli.HiveSQLException: Error

Re: [PR] Spark: Add option to provide partition spec in ADD_FILES procedure [iceberg]

bharos commented on code in PR #12327: URL: https://github.com/apache/iceberg/pull/12327#discussion_r1961148678 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestAddFilesProcedure.java: ## @@ -607,6 +607,40 @@ public void addFilteredPartitions

Re: [PR] Parquet: Fix errorprone warning [iceberg]

nastra merged PR #12324: URL: https://github.com/apache/iceberg/pull/12324 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Spark 3.5: Fix Incorrect Spec Used With AddFiles Procedure [iceberg]

szehon-ho commented on code in PR #12319: URL: https://github.com/apache/iceberg/pull/12319#discussion_r1961116327 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java: ## @@ -1085,4 +1088,43 @@ private ExecutorService getService() { return servi

Re: [I] Add option to provide partition spec in spark ADD_FILES procedure [iceberg]

bharos commented on issue #12325: URL: https://github.com/apache/iceberg/issues/12325#issuecomment-2667783024 @RussellSpitzer thanks, I see your change addresses the spec finding in `SparkTableUtil.importSparkTable` method I just created the PR for adding the spec as argument to add_f

Re: [PR] Docs: Add rewrite-table-path in spark procedure [iceberg]

szehon-ho merged PR #12115: URL: https://github.com/apache/iceberg/pull/12115 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Docs: Add rewrite-table-path in spark procedure [iceberg]

szehon-ho commented on PR #12115: URL: https://github.com/apache/iceberg/pull/12115#issuecomment-2667739875 Merged, thanks again ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [I] Add option to provide partition spec in spark ADD_FILES procedure [iceberg]

RussellSpitzer commented on issue #12325: URL: https://github.com/apache/iceberg/issues/12325#issuecomment-2667693931 Please see https://github.com/apache/iceberg/pull/12319 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Docs: add apache amoro(incubating) with iceberg (#11965) [iceberg]

czy006 commented on code in PR #11966: URL: https://github.com/apache/iceberg/pull/11966#discussion_r1961075017 ## docs/mkdocs.yml: ## @@ -69,6 +70,7 @@ nav: - Druid: https://druid.apache.org/docs/latest/development/extensions-contrib/iceberg/ - Kafka Connect: kafka-conne

Re: [PR] Spark: Detect dangling DVs properly [iceberg]

singhpk234 commented on code in PR #12270: URL: https://github.com/apache/iceberg/pull/12270#discussion_r1961054056 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RemoveDanglingDeletesSparkAction.java: ## @@ -156,7 +162,12 @@ private List findDanglingDeletes(

Re: [PR] Core: BugFix: PartitionStatsUtil#computeStats returns incomplete stats in case of partition evolution [iceberg]

ajantha-bhat commented on PR #12137: URL: https://github.com/apache/iceberg/pull/12137#issuecomment-2667681453 PR title can be `Core: Handle partition evolution case in partition stats compute` -- This is an automated message from the Apache Git Service. To respond to the message, ple

Re: [PR] Data: Add partition stats writer and reader [iceberg]

ajantha-bhat commented on code in PR #11216: URL: https://github.com/apache/iceberg/pull/11216#discussion_r1961038669 ## data/src/main/java/org/apache/iceberg/data/PartitionStatsHandler.java: ## @@ -0,0 +1,273 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] Wrap variant in PrimitiveHoder so serialization can result same instance [iceberg]

XBaith commented on code in PR #12317: URL: https://github.com/apache/iceberg/pull/12317#discussion_r1961023646 ## api/src/main/java/org/apache/iceberg/types/Types.java: ## @@ -451,6 +452,10 @@ public VariantType asVariantType() { return this; } +Object writeRe

Re: [I] Delete Files in Table Scans [iceberg-rust]

sdd commented on issue #630: URL: https://github.com/apache/iceberg-rust/issues/630#issuecomment-2667616082 @ZENOTME I think that we'll want to do that at some point but it feels more of a day 2 task. We're not touching the disk anywhere in the library so far, as far as I know, and so it wo

Re: [PR] support create table like in flink catalog [iceberg]

swapna267 commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1960980812 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalog.java: ## @@ -332,7 +335,15 @@ public List listTables(String databaseName) public Catalog

Re: [PR] support create table like in flink catalog [iceberg]

swapna267 commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1960984291 ## flink/v1.20/flink/src/test/java/org/apache/iceberg/flink/TestFlinkCatalogTable.java: ## @@ -188,6 +188,23 @@ public void testCreateTableLike() throws TableNotEx

Re: [PR] support create table like in flink catalog [iceberg]

swapna267 commented on code in PR #12199: URL: https://github.com/apache/iceberg/pull/12199#discussion_r1960981455 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalog.java: ## @@ -384,13 +395,6 @@ public void renameTable(ObjectPath tablePath, String newTabl

Re: [I] Enhance iceberg-go to Support Nessie API for All Catalog Operations [iceberg-go]

shubham-tomar commented on issue #291: URL: https://github.com/apache/iceberg-go/issues/291#issuecomment-2667534407 Hi @zeroshade, i think i found the issue, it is rather very small fix PR:[link]( https://github.com/apache/iceberg-go/pull/308) -- This is an automated message from the Ap

[PR] Fix: Nessie CreateTable [iceberg-go]

shubham-tomar opened a new pull request, #308: URL: https://github.com/apache/iceberg-go/pull/308 I am using Nessie as a catalog, while creating table i was getting below error ``` Failed to create table: failed to create table: WebApplicationException: HTTP 400 Bad Request ```

Re: [PR] API: Move variant to API and add extract expression [iceberg]

aihuaxu commented on code in PR #12304: URL: https://github.com/apache/iceberg/pull/12304#discussion_r1960945852 ## api/src/main/java/org/apache/iceberg/variants/Variant.java: ## @@ -25,4 +25,24 @@ public interface Variant { /** Returns the variant value. */ VariantValue

Re: [I] Support for Shallow Clone / Zero Copy Cloning in Apache Iceberg [iceberg]

RussellSpitzer commented on issue #12263: URL: https://github.com/apache/iceberg/issues/12263#issuecomment-2667530132 Only 1 is a difference here and it's at the catalog level. A branch cannot affect files in another branch. If you delete a snapshot from 1 branch but it still exists in anot

Re: [I] Support for Shallow Clone / Zero Copy Cloning in Apache Iceberg [iceberg]

RussellSpitzer commented on issue #12263: URL: https://github.com/apache/iceberg/issues/12263#issuecomment-2667481681 Again, how is this different than a branch? Are you just saying you want a branch with a different catalog identifier? -- This is an automated message from the Apache Git

Re: [PR] [infra] nightly pypi build for `pyiceberg_core` [iceberg-rust]

liurenjie1024 commented on PR #948: URL: https://github.com/apache/iceberg-rust/pull/948#issuecomment-2667430126 Let's wait for a moment for @Fokko to take a look at this as he's more familiar with python. -- This is an automated message from the Apache Git Service. To respond to the mess

Re: [PR] fix: Do not extract expression from cast to date [iceberg-rust]

liurenjie1024 merged PR #977: URL: https://github.com/apache/iceberg-rust/pull/977 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Core: Fix cleanup of orphaned statistics files in dropTableData [iceberg]

lliangyu-lin commented on PR #12132: URL: https://github.com/apache/iceberg/pull/12132#issuecomment-2667399890 @ajantha-bhat has confirmed through the [dev email list](https://lists.apache.org/list.html?d...@iceberg.apache.org) that this is an expected behavior for dropTableData and we shou

Re: [PR] Core: Fix cleanup of orphaned statistics files in dropTableData [iceberg]

lliangyu-lin closed pull request #12132: Core: Fix cleanup of orphaned statistics files in dropTableData URL: https://github.com/apache/iceberg/pull/12132 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] backport c++23 std::expected [iceberg-cpp]

lidavidm commented on PR #40: URL: https://github.com/apache/iceberg-cpp/pull/40#issuecomment-2667391713 An Apache member corrected this in arrow-java by making sure vendored files had their original license header: https://github.com/apache/arrow-java/pull/550 -- This is an automated mes

Re: [I] Add properties support for HadoopTables.load() [iceberg]

qqchang2nd commented on issue #12251: URL: https://github.com/apache/iceberg/issues/12251#issuecomment-2667390528 I understand your concern about the potential for increased parameterization and the implications it may have. However, I believe the changes I've made are quite generalizable a

Re: [PR] Data: Handle case where partition location is missing for `TableMigrationUtil` [iceberg]

jshmchenxi commented on code in PR #12212: URL: https://github.com/apache/iceberg/pull/12212#discussion_r1960871186 ## data/src/main/java/org/apache/iceberg/data/TableMigrationUtil.java: ## @@ -164,7 +164,10 @@ public static List listPartition( Path partitionDir = new Pat

Re: [I] Support for Shallow Clone / Zero Copy Cloning in Apache Iceberg [iceberg]

databius commented on issue #12263: URL: https://github.com/apache/iceberg/issues/12263#issuecomment-2667369068 > What are we looking for here that doesn't exist within branching? My case copying production data to a separate environment. Example: ``` CREATE TABLE myproject.m

Re: [PR] Docs: add apache amoro(incubating) with iceberg (#11965) [iceberg]

manuzhang commented on code in PR #11966: URL: https://github.com/apache/iceberg/pull/11966#discussion_r1960866695 ## docs/docs/amoro.md: ## @@ -0,0 +1,89 @@ +--- +title: "Apache Amoro" +--- + + +# Apache Amoro With Iceberg + +**[Apache Amoro(incubating)](https://amoro.apache.or

Re: [PR] Docs: add apache amoro(incubating) with iceberg (#11965) [iceberg]

manuzhang commented on code in PR #11966: URL: https://github.com/apache/iceberg/pull/11966#discussion_r1960870944 ## docs/mkdocs.yml: ## @@ -69,6 +70,7 @@ nav: - Druid: https://druid.apache.org/docs/latest/development/extensions-contrib/iceberg/ - Kafka Connect: kafka-co

Re: [PR] backport c++23 std::expected [iceberg-cpp]

zhjwpku commented on PR #40: URL: https://github.com/apache/iceberg-cpp/pull/40#issuecomment-2667321536 > I'm surprised by the license header. Shouldn't we use the Apache license there and record the provenance elsewhere? Yes, that was the initial version. I adopted the ASF License an

Re: [PR] Docs: Add rewrite-table-path in spark procedure [iceberg]

szehon-ho commented on code in PR #12115: URL: https://github.com/apache/iceberg/pull/12115#discussion_r1960797057 ## docs/docs/spark-procedures.md: ## @@ -972,4 +972,94 @@ CALL catalog_name.system.compute_table_stats(table => 'my_table', snapshot_id => Collect statistics of t

Re: [PR] Docs: Add rewrite-table-path in spark procedure [iceberg]

szehon-ho commented on code in PR #12115: URL: https://github.com/apache/iceberg/pull/12115#discussion_r1960777677 ## docs/docs/spark-procedures.md: ## @@ -972,4 +972,97 @@ CALL catalog_name.system.compute_table_stats(table => 'my_table', snapshot_id => Collect statistics of t

Re: [PR] Spec: Support geo type [iceberg]

desruisseaux commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1960782392 ## format/spec.md: ## @@ -228,6 +230,30 @@ Notes: For details on how to serialize a schema to JSON, see Appendix C. +[1001]:

Re: [PR] Docs: Add rewrite-table-path in spark procedure [iceberg]

szehon-ho commented on code in PR #12115: URL: https://github.com/apache/iceberg/pull/12115#discussion_r1960779365 ## docs/docs/spark-procedures.md: ## @@ -972,4 +972,97 @@ CALL catalog_name.system.compute_table_stats(table => 'my_table', snapshot_id => Collect statistics of t

Re: [PR] Docs: Add rewrite-table-path in spark procedure [iceberg]

szehon-ho commented on code in PR #12115: URL: https://github.com/apache/iceberg/pull/12115#discussion_r1960776871 ## docs/docs/spark-procedures.md: ## @@ -972,4 +972,97 @@ CALL catalog_name.system.compute_table_stats(table => 'my_table', snapshot_id => Collect statistics of t

Re: [I] Some schema updates do not support dots inside a field name [iceberg]

github-actions[bot] commented on issue #10875: URL: https://github.com/apache/iceberg/issues/10875#issuecomment-2667209001 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] JdbcCatalog fails to initialize with MS SQL Server [iceberg]

github-actions[bot] commented on issue #10068: URL: https://github.com/apache/iceberg/issues/10068#issuecomment-2667208963 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Fix #305: ManifestEntry partition field schema should be dynamically … [iceberg-go]

arnaudbriche commented on code in PR #307: URL: https://github.com/apache/iceberg-go/pull/307#discussion_r1960769877 ## manifest.go: ## @@ -292,10 +307,15 @@ func (b *ManifestV2Builder) KeyMetadata(km []byte) *ManifestV2Builder { // a pointer to the constructed manifest file.

Re: [PR] Fix #305: ManifestEntry partition field schema should be dynamically … [iceberg-go]

arnaudbriche commented on code in PR #307: URL: https://github.com/apache/iceberg-go/pull/307#discussion_r1960768123 ## utils.go: ## @@ -238,3 +239,68 @@ func avroEncode[T any](key string, version int, vals []T, out io.Writer) error { return enc.Close() } + +func str

Re: [PR] Fix #305: ManifestEntry partition field schema should be dynamically … [iceberg-go]

arnaudbriche commented on code in PR #307: URL: https://github.com/apache/iceberg-go/pull/307#discussion_r1960767524 ## utils.go: ## @@ -238,3 +239,68 @@ func avroEncode[T any](key string, version int, vals []T, out io.Writer) error { return enc.Close() } + +func str

Re: [I] Some schema updates do not support dots inside a field name [iceberg]

github-actions[bot] closed issue #10875: Some schema updates do not support dots inside a field name URL: https://github.com/apache/iceberg/issues/10875 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [I] deadlock when spark call delete row postition [iceberg]

github-actions[bot] commented on issue #10987: URL: https://github.com/apache/iceberg/issues/10987#issuecomment-2667209074 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] Rest catalog: write.metadata.delete-after-commit set true not deleting expired metadata files [iceberg]

github-actions[bot] commented on issue #10894: URL: https://github.com/apache/iceberg/issues/10894#issuecomment-2667209021 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] Rest catalog: write.metadata.delete-after-commit set true not deleting expired metadata files [iceberg]

github-actions[bot] closed issue #10894: Rest catalog: write.metadata.delete-after-commit set true not deleting expired metadata files URL: https://github.com/apache/iceberg/issues/10894 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

Re: [PR] AWS: Integrate S3 analytics accelerator library [iceberg]

geruh commented on code in PR #12299: URL: https://github.com/apache/iceberg/pull/12299#discussion_r1960726499 ## aws/src/main/java/org/apache/iceberg/aws/s3/analyticsaccelerator/S3SeekableInputStreamFactorySupplier.java: ## @@ -0,0 +1,56 @@ +/* + * Licensed to the Apache Softwa

Re: [PR] Arrow: Infer the types when reading [iceberg-python]

sungwy commented on code in PR #1669: URL: https://github.com/apache/iceberg-python/pull/1669#discussion_r1960579776 ## pyiceberg/io/pyarrow.py: ## @@ -1655,19 +1646,16 @@ class ArrowProjectionVisitor(SchemaWithPartnerVisitor[pa.Array, Optional[pa.Arra _file_schema: Schema

Re: [PR] Arrow: Infer the types when reading [iceberg-python]

sungwy commented on code in PR #1669: URL: https://github.com/apache/iceberg-python/pull/1669#discussion_r1960579776 ## pyiceberg/io/pyarrow.py: ## @@ -1655,19 +1646,16 @@ class ArrowProjectionVisitor(SchemaWithPartnerVisitor[pa.Array, Optional[pa.Arra _file_schema: Schema

Re: [PR] Core: Interface changes for separating rewrite planner and executor [iceberg]

stevenzwu commented on code in PR #12306: URL: https://github.com/apache/iceberg/pull/12306#discussion_r1960273582 ## core/src/main/java/org/apache/iceberg/actions/FileRewriteExecutor.java: ## @@ -0,0 +1,77 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[PR] Parquet: Fix errorprone warning [iceberg]

ebyhr opened a new pull request, #12324: URL: https://github.com/apache/iceberg/pull/12324 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-ma

Re: [PR] Spec: Support geo type [iceberg]

jiayuasu commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1960747123 ## format/spec.md: ## @@ -228,6 +230,30 @@ Notes: For details on how to serialize a schema to JSON, see Appendix C. +[1001]:

[PR] Use delimited column names in `CreateChangelogViewProcedure` [iceberg]

andyglow opened a new pull request, #12322: URL: https://github.com/apache/iceberg/pull/12322 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

Re: [PR] Spec: Support geo type [iceberg]

stefankandic commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1960739224 ## format/spec.md: ## @@ -228,6 +230,30 @@ Notes: For details on how to serialize a schema to JSON, see Appendix C. +[1001]:

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue commented on PR #12139: URL: https://github.com/apache/iceberg/pull/12139#issuecomment-2667157868 Thanks for reviewing, @aihuaxu, @danielcweeks, @RussellSpitzer, and @amogh-jahagirdar! -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue merged PR #12139: URL: https://github.com/apache/iceberg/pull/12139 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] build(deps): bump the gomod_updates group with 4 updates [iceberg-go]

zeroshade merged PR #304: URL: https://github.com/apache/iceberg-go/pull/304 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] AWS: Integrate S3 analytics accelerator library [iceberg]

geruh commented on PR #12299: URL: https://github.com/apache/iceberg/pull/12299#issuecomment-2667150596 Hey @SanjayMarreddi, I think this is a cool addition!! I was wondering if your team has any benchmark numbers or performance metrics for this? Meanwhile I'll ping some others to ge

Re: [PR] docs(cmd/iceberg): Added description of CLI usage in README [iceberg-go]

zeroshade merged PR #301: URL: https://github.com/apache/iceberg-go/pull/301 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] AWS: Integrate S3 analytics accelerator library [iceberg]

geruh commented on code in PR #12299: URL: https://github.com/apache/iceberg/pull/12299#discussion_r1960726499 ## aws/src/main/java/org/apache/iceberg/aws/s3/analyticsaccelerator/S3SeekableInputStreamFactorySupplier.java: ## @@ -0,0 +1,56 @@ +/* + * Licensed to the Apache Softwa

Re: [PR] AWS: Integrate S3 analytics accelerator library [iceberg]

geruh commented on code in PR #12299: URL: https://github.com/apache/iceberg/pull/12299#discussion_r1960724511 ## aws/src/main/java/org/apache/iceberg/aws/s3/analyticsaccelerator/S3SeekableInputStreamFactorySupplier.java: ## @@ -0,0 +1,56 @@ +/* + * Licensed to the Apache Softwa

Re: [PR] feat(catalog): Add pagination for list table operation across different catalog types [iceberg-go]

dttung2905 commented on code in PR #306: URL: https://github.com/apache/iceberg-go/pull/306#discussion_r1960716083 ## catalog/glue/glue_test.go: ## @@ -20,6 +20,7 @@ package glue import ( "context" "errors" + "github.com/apache/iceberg-go/table" Review Co

Re: [PR] Spark 3.5: Fix Incorrect Spec Used With AddFiles Procedure [iceberg]

singhpk234 commented on PR #12319: URL: https://github.com/apache/iceberg/pull/12319#issuecomment-2667136903 > i'm also afraid of breaking folks who are doing weird things. I do think that's worth checking out in a follow up, or maybe better yet, just not re-writing the spec every time into

Re: [PR] Materialized View Spec [iceberg]

stevenzwu commented on code in PR #11041: URL: https://github.com/apache/iceberg/pull/11041#discussion_r1960719638 ## format/view-spec.md: ## @@ -160,6 +179,57 @@ Each entry in `version-log` is a struct with the following fields: | _required_ | `timestamp-ms` | Timestamp when

[PR] Parquet: Remove deprecated VectorizedReader.setRowGroupInfo and ParquetValueReader.setPageSource [iceberg]

ebyhr opened a new pull request, #12321: URL: https://github.com/apache/iceberg/pull/12321 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-ma

Re: [PR] Spark 3.5: Fix Incorrect Spec Used With AddFiles Procedure [iceberg]

RussellSpitzer commented on PR #12319: URL: https://github.com/apache/iceberg/pull/12319#issuecomment-2667115159 @singhpk234 re: Forbidding Manifest writers. I think that may be a good idea but i'm also afraid of breaking folks who are doing weird things. I do think that's worth chec

Re: [PR] feat(catalog): Add pagination for list table operation across different catalog types [iceberg-go]

dttung2905 commented on code in PR #306: URL: https://github.com/apache/iceberg-go/pull/306#discussion_r1960713147 ## catalog/glue/glue_test.go: ## @@ -20,6 +20,7 @@ package glue import ( "context" "errors" + "github.com/apache/iceberg-go/table" Review Co

[PR] Core: Remove deprecated Util.blockLocations method and StructCopy class [iceberg]

ebyhr opened a new pull request, #12320: URL: https://github.com/apache/iceberg/pull/12320 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-ma

Re: [PR] feat(catalog): Add pagination for list table operation across different catalog types [iceberg-go]

dttung2905 commented on code in PR #306: URL: https://github.com/apache/iceberg-go/pull/306#discussion_r1960712467 ## catalog/sql/sql.go: ## @@ -524,9 +525,14 @@ func (c *Catalog) DropNamespace(ctx context.Context, namespace table.Identifier) return fmt.Errorf("

Re: [PR] Added description of CLI usage in README [iceberg-go]

zeroshade commented on code in PR #301: URL: https://github.com/apache/iceberg-go/pull/301#discussion_r1960704984 ## README.md: ## @@ -84,6 +84,9 @@ $ cd iceberg-go/cmd/iceberg && go build . ### CLI Usage +Run `go build` command to build the CLI command. +The CLI executabl

Re: [PR] AWS: Integrate S3 analytics accelerator library [iceberg]

geruh commented on code in PR #12299: URL: https://github.com/apache/iceberg/pull/12299#discussion_r1960709819 ## build.gradle: ## @@ -470,6 +470,8 @@ project(':iceberg-aws') { compileOnly("software.amazon.awssdk:dynamodb") compileOnly("software.amazon.awssdk:lakeforma

Re: [PR] Spark 3.5: Fix Incorrect Spec Used With AddFiles Procedure [iceberg]

RussellSpitzer commented on code in PR #12319: URL: https://github.com/apache/iceberg/pull/12319#discussion_r1960708406 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java: ## @@ -1085,4 +1088,43 @@ private ExecutorService getService() { return

Re: [PR] Spark 3.5: Fix Incorrect Spec Used With AddFiles Procedure [iceberg]

RussellSpitzer commented on code in PR #12319: URL: https://github.com/apache/iceberg/pull/12319#discussion_r1960707658 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java: ## @@ -1085,4 +1088,43 @@ private ExecutorService getService() { return

Re: [PR] Spark 3.5: Fix Incorrect Spec Used With AddFiles Procedure [iceberg]

singhpk234 commented on code in PR #12319: URL: https://github.com/apache/iceberg/pull/12319#discussion_r1960686757 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java: ## @@ -1085,4 +1088,43 @@ private ExecutorService getService() { return serv

Re: [PR] Spark 3.5: Fix Incorrect Spec Used With AddFiles Procedure [iceberg]

singhpk234 commented on code in PR #12319: URL: https://github.com/apache/iceberg/pull/12319#discussion_r1960699988 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestAddFilesProcedure.java: ## @@ -476,6 +485,53 @@ public void addPartitionToPar

Re: [PR] Added description of CLI usage in README [iceberg-go]

subkanthi commented on code in PR #301: URL: https://github.com/apache/iceberg-go/pull/301#discussion_r1960699071 ## README.md: ## @@ -82,6 +82,42 @@ $ cd iceberg-go/cmd/iceberg && go build . * Plan to add [Apache Arrow](https://pkg.go.dev/github.com/apache/arrow-go/) support

Re: [PR] Add all filles metadata tables [iceberg-python]

geruh commented on code in PR #1626: URL: https://github.com/apache/iceberg-python/pull/1626#discussion_r1960691377 ## pyiceberg/table/inspect.py: ## @@ -657,3 +669,35 @@ def all_manifests(self) -> "pa.Table": lambda args: self._generate_manifests_table(*args), [(sn

Re: [PR] Warn on `{write.folder-storage.path,write.object-storage.path}` properties [iceberg]

stevenzwu commented on code in PR #12315: URL: https://github.com/apache/iceberg/pull/12315#discussion_r1960689270 ## docs/docs/aws.md: ## @@ -385,6 +385,7 @@ Note, the path resolution logic for `ObjectStoreLocationProvider` is `write.data However, for the older versions up to

Re: [PR] Spark 3.5: Fix Incorrect Spec Used With AddFiles Procedure [iceberg]

RussellSpitzer commented on code in PR #12319: URL: https://github.com/apache/iceberg/pull/12319#discussion_r1960686497 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java: ## @@ -540,7 +543,7 @@ public static void importSparkTable( try {

Re: [PR] Spark 3.5: Fix Incorrect Spec Used With AddFiles Procedure [iceberg]

RussellSpitzer commented on code in PR #12319: URL: https://github.com/apache/iceberg/pull/12319#discussion_r1960684121 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java: ## @@ -1085,4 +1088,43 @@ private ExecutorService getService() { return

Re: [PR] Spark 3.5: Fix Incorrect Spec Used With AddFiles Procedure [iceberg]

RussellSpitzer commented on code in PR #12319: URL: https://github.com/apache/iceberg/pull/12319#discussion_r1960682985 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestAddFilesProcedure.java: ## @@ -968,6 +1024,28 @@ public void testAddFilesP

Re: [PR] Spark 3.5: Fix Incorrect Spec Used With AddFiles Procedure [iceberg]

RussellSpitzer commented on code in PR #12319: URL: https://github.com/apache/iceberg/pull/12319#discussion_r1960681428 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestAddFilesProcedure.java: ## @@ -476,6 +485,53 @@ public void addPartitionT

Re: [PR] Spark 3.5: Fix Incorrect Spec Used With AddFiles Procedure [iceberg]

RussellSpitzer commented on code in PR #12319: URL: https://github.com/apache/iceberg/pull/12319#discussion_r1960681428 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestAddFilesProcedure.java: ## @@ -476,6 +485,53 @@ public void addPartitionT

Re: [PR] Spark 3.5: Fix Incorrect Spec Used With AddFiles Procedure [iceberg]

RussellSpitzer commented on code in PR #12319: URL: https://github.com/apache/iceberg/pull/12319#discussion_r1960679959 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestAddFilesProcedure.java: ## @@ -476,6 +485,53 @@ public void addPartitionT

Re: [PR] Arrow: Infer the types when reading [iceberg-python]

sungwy commented on code in PR #1669: URL: https://github.com/apache/iceberg-python/pull/1669#discussion_r1960582838 ## pyiceberg/io/pyarrow.py: ## @@ -1573,11 +1561,16 @@ def _table_from_scan_task(task: FileScanTask) -> pa.Table: tables = [f.result() for f in comple

[PR] Spark 3.5: Fix Incorrect Spec Used With AddFiles Procedure [iceberg]

RussellSpitzer opened a new pull request, #12319: URL: https://github.com/apache/iceberg/pull/12319 Fixes #12273 I wrote a full overview of the behavior on the above ticket but the fix here is relatively simple. Instead of attempting to intuit what the Partition Spec should be and b

Re: [PR] Spark 3.5: Fix Incorrect Spec Used With AddFiles Procedure [iceberg]

RussellSpitzer commented on PR #12319: URL: https://github.com/apache/iceberg/pull/12319#issuecomment-2667058674 cc @singhpk234 + @sfc-gh-yijil + @aokolnychyi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] Docs: Refactor site navigation bar [iceberg]

stevenzwu merged PR #12289: URL: https://github.com/apache/iceberg/pull/12289 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Docs: Refactor site navigation bar [iceberg]

stevenzwu commented on PR #12289: URL: https://github.com/apache/iceberg/pull/12289#issuecomment-2667052630 seems like we have enough approvals. I am going to merge this. any other changes can be followed up separately. -- This is an automated message from the Apache Git Service. To

Re: [PR] Parquet: Implement Variant readers [iceberg]

danielcweeks commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1960633603 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetVariantReaders.java: ## @@ -0,0 +1,424 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] Spark: Structured Streaming read limit support follow-up [iceberg]

wypoon commented on PR #12260: URL: https://github.com/apache/iceberg/pull/12260#issuecomment-2666966545 @jackye1995 @RussellSpitzer can you please review? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Fix IndexOutOfBounds exception in FileFormat#fromFileName [iceberg]

rshkv commented on PR #12301: URL: https://github.com/apache/iceberg/pull/12301#issuecomment-2666980115 > I understand that this PR is just a fix for an existing method, but I have concerns about the original intention of the method. We are relying on the filename to deduce the actual file

Re: [PR] Core: Print un-pretty metadata files without whitespace [iceberg]

Fokko commented on PR #12318: URL: https://github.com/apache/iceberg/pull/12318#issuecomment-2666979430 For consistency, we could also consider applying the same for views: https://github.com/apache/iceberg/blob/e79295b53e7b1d4143401f863f5d918c3bf4/core/src/main/java/org/apache/ic

Re: [PR] Spark: Support singular form of years, months, days, and hours functions [iceberg]

wypoon commented on PR #12117: URL: https://github.com/apache/iceberg/pull/12117#issuecomment-2666968168 @Fokko ping again. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[PR] Core: Print un-pretty metadata files without whitespace [iceberg]

istreeter opened a new pull request, #12318: URL: https://github.com/apache/iceberg/pull/12318 Closes #12281 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1960605176 ## core/src/test/java/org/apache/iceberg/variants/VariantTestUtil.java: ## @@ -27,10 +29,55 @@ import java.util.stream.Collectors; import java.util.stream.Stream; i

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1960603787 ## core/src/test/java/org/apache/iceberg/variants/VariantTestUtil.java: ## @@ -27,10 +29,55 @@ import java.util.stream.Collectors; import java.util.stream.Stream; i

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1960602946 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetVariantVisitor.java: ## @@ -0,0 +1,285 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

  1   2   3   >