Re: [PR] fix snapshot schema issue: wrong usage of walrus operator [iceberg-python]

Fokko commented on code in PR #211: URL: https://github.com/apache/iceberg-python/pull/211#discussion_r1424978509 ## pyiceberg/table/__init__.py: ## @@ -944,8 +944,8 @@ def snapshot(self) -> Optional[Snapshot]: def projection(self) -> Schema: snapshot_schema = self

Re: [PR] Hive: Make HiveMetastoreExtension configurable [iceberg]

Fokko commented on code in PR #9288: URL: https://github.com/apache/iceberg/pull/9288#discussion_r1424982095 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveMetastoreExtension.java: ## @@ -80,4 +82,27 @@ public HiveConf hiveConf() { public TestHiveMetastore metast

Re: [PR] Hive: Make HiveMetastoreExtension configurable [iceberg]

ajantha-bhat commented on code in PR #9288: URL: https://github.com/apache/iceberg/pull/9288#discussion_r1424994351 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveMetastoreExtension.java: ## @@ -80,4 +82,27 @@ public HiveConf hiveConf() { public TestHiveMetastore

Re: [PR] Hive: Make HiveMetastoreExtension configurable [iceberg]

ajantha-bhat commented on code in PR #9288: URL: https://github.com/apache/iceberg/pull/9288#discussion_r1424994351 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveMetastoreExtension.java: ## @@ -80,4 +82,27 @@ public HiveConf hiveConf() { public TestHiveMetastore

Re: [PR] Support parameterized tests at class-level with JUnit5 [iceberg]

GianlucaPrincipini commented on code in PR #9161: URL: https://github.com/apache/iceberg/pull/9161#discussion_r1425003497 ## parquet/src/test/java/org/apache/iceberg/parquet/TestDictionaryRowGroupFilter.java: ## @@ -223,52 +219,33 @@ public void createInputFile() throws IOExcept

Re: [PR] Hive: Make HiveMetastoreExtension configurable [iceberg]

Fokko merged PR #9288: URL: https://github.com/apache/iceberg/pull/9288 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [PR] Hive: Make HiveMetastoreExtension configurable [iceberg]

Fokko commented on PR #9288: URL: https://github.com/apache/iceberg/pull/9288#issuecomment-1853486502 Thanks for fixing this @nastra and @ajantha-bhat for the review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] fix snapshot schema issue: wrong usage of walrus operator [iceberg-python]

Fokko merged PR #211: URL: https://github.com/apache/iceberg-python/pull/211 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Add doc for rewriting manifest with spec id [iceberg]

Fokko merged PR #9253: URL: https://github.com/apache/iceberg/pull/9253 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [PR] feat: suport read/write Manifest [iceberg-rust]

Fokko commented on code in PR #79: URL: https://github.com/apache/iceberg-rust/pull/79#discussion_r1425045199 ## crates/iceberg/src/spec/manifest.rs: ## @@ -0,0 +1,1864 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

Re: [PR] feat: suport read/write Manifest [iceberg-rust]

ZENOTME commented on code in PR #79: URL: https://github.com/apache/iceberg-rust/pull/79#discussion_r1425067275 ## crates/iceberg/src/spec/manifest.rs: ## @@ -0,0 +1,1864 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements

Re: [PR] Hive: Make HiveMetastoreExtension configurable [iceberg]

nastra commented on code in PR #9288: URL: https://github.com/apache/iceberg/pull/9288#discussion_r1425070360 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveMetastoreExtension.java: ## @@ -80,4 +82,27 @@ public HiveConf hiveConf() { public TestHiveMetastore metas

Re: [PR] Support parameterized tests at class-level with JUnit5 [iceberg]

nastra commented on code in PR #9161: URL: https://github.com/apache/iceberg/pull/9161#discussion_r1425072357 ## parquet/src/test/java/org/apache/iceberg/parquet/TestDictionaryRowGroupFilter.java: ## @@ -223,52 +219,33 @@ public void createInputFile() throws IOException { d

Re: [PR] test: Remove binary manifest list avro file [iceberg-rust]

Xuanwo commented on code in PR #118: URL: https://github.com/apache/iceberg-rust/pull/118#discussion_r1425080204 ## crates/iceberg/src/spec/manifest_list.rs: ## @@ -931,55 +931,104 @@ mod test { use super::_serde::ManifestListV2; -#[test] -fn test_parse_manifest

Re: [PR] test: Remove binary manifest list avro file [iceberg-rust]

liurenjie1024 commented on code in PR #118: URL: https://github.com/apache/iceberg-rust/pull/118#discussion_r1425089356 ## crates/iceberg/src/spec/manifest_list.rs: ## @@ -931,55 +931,104 @@ mod test { use super::_serde::ManifestListV2; -#[test] -fn test_parse_m

Re: [PR] Hive: Refactor TestHiveCatalog tests to use the core CatalogTests [iceberg]

nk1506 commented on code in PR #8918: URL: https://github.com/apache/iceberg/pull/8918#discussion_r1425058869 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCatalog.java: ## @@ -219,81 +208,6 @@ public void testInitializeCatalogWithProperties() { .isEqu

Re: [PR] Hive: Refactor TestHiveCatalog tests to use the core CatalogTests [iceberg]

nk1506 commented on PR #8918: URL: https://github.com/apache/iceberg/pull/8918#issuecomment-1853577807 Thanks @ajantha-bhat and @nastra for recommending configurable approach for `HiveMetastoreExtension` and fixing this. Although I had to make a small [fix](https://github.com/apache/iceber

Re: [PR] Hive: Refactor TestHiveCatalog tests to use the core CatalogTests [iceberg]

nastra commented on code in PR #8918: URL: https://github.com/apache/iceberg/pull/8918#discussion_r1425096603 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCatalog.java: ## @@ -219,81 +208,6 @@ public void testInitializeCatalogWithProperties() { .isEqu

Re: [PR] feat: suport read/write Manifest [iceberg-rust]

Fokko commented on code in PR #79: URL: https://github.com/apache/iceberg-rust/pull/79#discussion_r1425109506 ## crates/iceberg/src/spec/manifest_list.rs: ## @@ -1120,15 +1236,17 @@ mod test { #[tokio::test] async fn test_manifest_list_writer_v2() { -let expe

Re: [PR] feat: suport read/write Manifest [iceberg-rust]

Fokko commented on code in PR #79: URL: https://github.com/apache/iceberg-rust/pull/79#discussion_r1425117160 ## crates/iceberg/src/spec/manifest.rs: ## @@ -0,0 +1,1821 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

Re: [PR] feat: suport read/write Manifest [iceberg-rust]

Fokko commented on code in PR #79: URL: https://github.com/apache/iceberg-rust/pull/79#discussion_r1425129671 ## crates/iceberg/src/spec/manifest.rs: ## @@ -0,0 +1,1821 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

Re: [PR] feat: suport read/write Manifest [iceberg-rust]

Fokko commented on code in PR #79: URL: https://github.com/apache/iceberg-rust/pull/79#discussion_r1425137422 ## crates/iceberg/src/spec/manifest.rs: ## @@ -0,0 +1,1864 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

Re: [PR] feat: suport read/write Manifest [iceberg-rust]

ZENOTME commented on code in PR #79: URL: https://github.com/apache/iceberg-rust/pull/79#discussion_r1425139874 ## crates/iceberg/src/spec/manifest.rs: ## @@ -0,0 +1,1864 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements

[I] Support `initial-default` when reading Avro [iceberg-rust]

Fokko opened a new issue, #119: URL: https://github.com/apache/iceberg-rust/issues/119 I would suggest adding support for `initial-default` to read V1 Avro files into V2 objects in memory. For example, V1 does not have sequence numbers, and then need to be set to `0`. This is importa

Re: [I] test: Replace binary avro file by generating it on the fly. [iceberg-rust]

Fokko commented on issue #70: URL: https://github.com/apache/iceberg-rust/issues/70#issuecomment-1853637160 I would love to see this before the release, there is also some interesting data here: https://github.com/apache/iceberg-rust/pull/79#discussion_r1425129671 -- This is an automated

Re: [PR] feat: suport read/write Manifest [iceberg-rust]

Fokko merged PR #79: URL: https://github.com/apache/iceberg-rust/pull/79 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Core, API: Move SQLViewRepresentation to API [iceberg]

nastra commented on code in PR #9278: URL: https://github.com/apache/iceberg/pull/9278#discussion_r1425146761 ## .palantir/revapi.yml: ## @@ -877,6 +877,10 @@ acceptedBreaks: - code: "java.field.serialVersionUIDChanged" new: "field org.apache.iceberg.util.Serializabl

Re: [I] Apache Flink not committing new snapshots to Iceberg Table [iceberg]

FranMorilloAWS commented on issue #9089: URL: https://github.com/apache/iceberg/issues/9089#issuecomment-1853771914 When you say table commits, does that mean the entire table?. Also could there be a scenario that because of state mis allignment, that is not able to get the highest checkpo

Re: [I] Apache Flink not committing new snapshots to Iceberg Table [iceberg]

FranMorilloAWS commented on issue #9089: URL: https://github.com/apache/iceberg/issues/9089#issuecomment-1853776664 Do we need to add an operator uid, to make sure state compatibility? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [I] Apache Flink not committing new snapshots to Iceberg Table [iceberg]

FranMorilloAWS commented on issue #9089: URL: https://github.com/apache/iceberg/issues/9089#issuecomment-1853806769 Im facing issue that when flink restarts it is not able to commit to all tables.. Any reason for this? -- This is an automated message from the Apache Git Service. To respo

Re: [I] Apache Flink not committing new snapshots to Iceberg Table [iceberg]

pvary commented on issue #9089: URL: https://github.com/apache/iceberg/issues/9089#issuecomment-1853832988 > When you say table commits, does that mean the entire table? Not the whole table data, but the table metadata. Starting from the end of the history (most recent), it goes throu

Re: [PR] Hive: Refactor TestHiveCatalog tests to use the core CatalogTests [iceberg]

nk1506 commented on code in PR #8918: URL: https://github.com/apache/iceberg/pull/8918#discussion_r1425315871 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -261,6 +261,12 @@ public void renameTable(TableIdentifier from, TableIdentifier original

Re: [PR] Core: Use properties while initializing default HadoopFileIO for Hadoop catalog. [iceberg]

yingsu00 commented on PR #9283: URL: https://github.com/apache/iceberg/pull/9283#issuecomment-1853905363 PR message "to Initialise" -> to initialize -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] Core: Lookup targeted position deletes by path [iceberg]

aokolnychyi commented on code in PR #9251: URL: https://github.com/apache/iceberg/pull/9251#discussion_r1425368136 ## core/src/main/java/org/apache/iceberg/DeleteFileIndex.java: ## @@ -474,68 +421,52 @@ private Collection loadDeleteFiles() { DeleteFileIndex build() {

Re: [PR] Core: Lookup targeted position deletes by path [iceberg]

aokolnychyi commented on code in PR #9251: URL: https://github.com/apache/iceberg/pull/9251#discussion_r1425370277 ## core/src/main/java/org/apache/iceberg/util/ArrayUtil.java: ## @@ -320,4 +320,29 @@ public static boolean isStrictlyAscending(long[] array) { return true;

Re: [PR] Core: Lookup targeted position deletes by path [iceberg]

aokolnychyi commented on code in PR #9251: URL: https://github.com/apache/iceberg/pull/9251#discussion_r1425370701 ## core/src/main/java/org/apache/iceberg/util/ArrayUtil.java: ## @@ -320,4 +320,29 @@ public static boolean isStrictlyAscending(long[] array) { return true;

Re: [PR] Core: Lookup targeted position deletes by path [iceberg]

aokolnychyi commented on code in PR #9251: URL: https://github.com/apache/iceberg/pull/9251#discussion_r1425372419 ## core/src/main/java/org/apache/iceberg/DeleteFileIndex.java: ## @@ -582,93 +513,187 @@ private Iterable>> deleteManifestRea } } - // a group of indexe

Re: [PR] Core: Lookup targeted position deletes by path [iceberg]

aokolnychyi commented on code in PR #9251: URL: https://github.com/apache/iceberg/pull/9251#discussion_r1425375381 ## core/src/main/java/org/apache/iceberg/DeleteFileIndex.java: ## @@ -582,93 +513,187 @@ private Iterable>> deleteManifestRea } } - // a group of indexe

Re: [PR] Core: Lookup targeted position deletes by path [iceberg]

aokolnychyi commented on code in PR #9251: URL: https://github.com/apache/iceberg/pull/9251#discussion_r1425372419 ## core/src/main/java/org/apache/iceberg/DeleteFileIndex.java: ## @@ -582,93 +513,187 @@ private Iterable>> deleteManifestRea } } - // a group of indexe

Re: [PR] Core: Lookup targeted position deletes by path [iceberg]

aokolnychyi commented on code in PR #9251: URL: https://github.com/apache/iceberg/pull/9251#discussion_r1425376250 ## core/src/main/java/org/apache/iceberg/DeleteFileIndex.java: ## @@ -474,68 +421,52 @@ private Collection loadDeleteFiles() { DeleteFileIndex build() {

Re: [PR] Core: Lookup targeted position deletes by path [iceberg]

aokolnychyi commented on code in PR #9251: URL: https://github.com/apache/iceberg/pull/9251#discussion_r1425377180 ## core/src/main/java/org/apache/iceberg/DeleteFileIndex.java: ## @@ -582,93 +513,187 @@ private Iterable>> deleteManifestRea } } - // a group of indexe

Re: [PR] Core: Lookup targeted position deletes by path [iceberg]

aokolnychyi commented on code in PR #9251: URL: https://github.com/apache/iceberg/pull/9251#discussion_r1425379313 ## core/src/main/java/org/apache/iceberg/DeleteFileIndex.java: ## @@ -582,97 +524,214 @@ private Iterable>> deleteManifestRea } } - // a group of indexe

Re: [PR] Replace deprecated RuntimeIOException with UncheckedIOException [iceberg]

jbonofre commented on code in PR #4776: URL: https://github.com/apache/iceberg/pull/4776#discussion_r1425380109 ## api/src/main/java/org/apache/iceberg/Files.java: ## @@ -24,8 +24,8 @@ import java.io.RandomAccessFile; import java.nio.file.Paths; import org.apache.iceberg.exce

Re: [PR] Core: Iceberg streaming streaming-skip-overwrite-snapshots SparkMicroBatchStream only skips over one file per trigger [iceberg]

cccs-jc commented on PR #8980: URL: https://github.com/apache/iceberg/pull/8980#issuecomment-1853964957 so I did more digging. On our production tables I search for all manifests which have a `existing_data_files_count > 0` and `added_data_files_count > 0` and I find none. This leads me to

Re: [I] AWS access denied from metadata file on S3 [iceberg-python]

itaise commented on issue #201: URL: https://github.com/apache/iceberg-python/issues/201#issuecomment-1853965470 Hi @Fokko, The first suggestion did not work, but passing the credentials explicitly like in your second suggestion worked. Thank you very much! -- This is an automated

Re: [PR] Hive: Refactor TestHiveCatalog tests to use the core CatalogTests [iceberg]

nastra commented on code in PR #8918: URL: https://github.com/apache/iceberg/pull/8918#discussion_r1425406151 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -261,6 +261,12 @@ public void renameTable(TableIdentifier from, TableIdentifier original

Re: [PR] Hive: Refactor TestHiveCatalog tests to use the core CatalogTests [iceberg]

nk1506 commented on code in PR #8918: URL: https://github.com/apache/iceberg/pull/8918#discussion_r1425443702 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -261,6 +261,12 @@ public void renameTable(TableIdentifier from, TableIdentifier original

Re: [PR] Hive: Refactor TestHiveCatalog tests to use the core CatalogTests [iceberg]

nastra commented on code in PR #8918: URL: https://github.com/apache/iceberg/pull/8918#discussion_r1425445565 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCatalog.java: ## @@ -1211,33 +1183,48 @@ public void testDatabaseLocationWithSlashInWarehouseDir() {

Re: [PR] Hive: Refactor TestHiveCatalog tests to use the core CatalogTests [iceberg]

nastra commented on code in PR #8918: URL: https://github.com/apache/iceberg/pull/8918#discussion_r1425513915 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCatalog.java: ## @@ -1210,34 +1182,53 @@ public void testDatabaseLocationWithSlashInWarehouseDir() {

Re: [PR] API: Replace special case of deprecated RuntimeIOException [iceberg]

Fokko commented on code in PR #6887: URL: https://github.com/apache/iceberg/pull/6887#discussion_r1425537371 ## api/src/main/java/org/apache/iceberg/exceptions/FileHandlingException.java: ## @@ -0,0 +1,29 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

Re: [PR] API: Replace special case of deprecated RuntimeIOException [iceberg]

ajantha-bhat commented on code in PR #6887: URL: https://github.com/apache/iceberg/pull/6887#discussion_r1425552569 ## api/src/main/java/org/apache/iceberg/exceptions/FileHandlingException.java: ## @@ -0,0 +1,29 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

Re: [PR] API: Replace special case of deprecated RuntimeIOException [iceberg]

ajantha-bhat commented on code in PR #6887: URL: https://github.com/apache/iceberg/pull/6887#discussion_r142508 ## api/src/main/java/org/apache/iceberg/exceptions/FileHandlingException.java: ## @@ -0,0 +1,29 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

Re: [PR] Core: Add Catalog Transactions API [iceberg]

nastra commented on code in PR #6948: URL: https://github.com/apache/iceberg/pull/6948#discussion_r1425557919 ## core/src/main/java/org/apache/iceberg/catalog/CatalogTransaction.java: ## @@ -0,0 +1,84 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

Re: [PR] Replace deprecated RuntimeIOException with UncheckedIOException [iceberg]

ajantha-bhat commented on code in PR #4776: URL: https://github.com/apache/iceberg/pull/4776#discussion_r1425558869 ## api/src/main/java/org/apache/iceberg/Files.java: ## @@ -24,8 +24,8 @@ import java.io.RandomAccessFile; import java.nio.file.Paths; import org.apache.iceberg.

Re: [PR] Replace deprecated RuntimeIOException with UncheckedIOException [iceberg]

jbonofre commented on code in PR #4776: URL: https://github.com/apache/iceberg/pull/4776#discussion_r1425565613 ## api/src/main/java/org/apache/iceberg/Files.java: ## @@ -24,8 +24,8 @@ import java.io.RandomAccessFile; import java.nio.file.Paths; import org.apache.iceberg.exce

Re: [I] Unable to merge CDC data into snapshot data. java.lang.ClassCastException: org.apache.spark.unsafe.types.UTF8String cannot be cast to java.lang.Long [iceberg]

harshith-bolar-rapido commented on issue #8333: URL: https://github.com/apache/iceberg/issues/8333#issuecomment-1854204500 Noticed that this happens when the input folders are partitioned by date. Ex: `mmdd=20230304` Here's a minimal reproducible example. @RussellSpitzer @nastra

Re: [I] AWS access denied from metadata file on S3 [iceberg-python]

Fokko commented on issue #201: URL: https://github.com/apache/iceberg-python/issues/201#issuecomment-1854326828 Awesome, thanks for letting me know! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [I] AWS access denied from metadata file on S3 [iceberg-python]

Fokko closed issue #201: AWS access denied from metadata file on S3 URL: https://github.com/apache/iceberg-python/issues/201 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [I] Support 'schema.name-mapping.default' Column Projection property [iceberg-python]

Fokko commented on issue #202: URL: https://github.com/apache/iceberg-python/issues/202#issuecomment-1854334634 @syun64 Thanks for raising this, is this something that you're interested in contributing? -- This is an automated message from the Apache Git Service. To respond to the message

Re: [I] Support 'schema.name-mapping.default' Column Projection property [iceberg-python]

syun64 commented on issue #202: URL: https://github.com/apache/iceberg-python/issues/202#issuecomment-1854378926 @Fokko Absolutely 👍 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Implementation of Glue View Catalog [iceberg]

amogh-jahagirdar closed pull request #5085: Implementation of Glue View Catalog URL: https://github.com/apache/iceberg/pull/5085 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Core: Use properties while initializing default HadoopFileIO for Hadoop catalog. [iceberg]

amogh-jahagirdar commented on code in PR #9283: URL: https://github.com/apache/iceberg/pull/9283#discussion_r1425696907 ## core/src/main/java/org/apache/iceberg/hadoop/HadoopCatalog.java: ## @@ -112,11 +112,11 @@ public void initialize(String name, Map properties) { this.w

[I] Remove Spark-3.2 support for Iceberg 1.5.0 and onwards. [iceberg]

ajantha-bhat opened a new issue, #9290: URL: https://github.com/apache/iceberg/issues/9290 Based on the mailing list discussion https://lists.apache.org/thread/ttonfgrfb3ssrhl38kc7hljqmf5phlfw -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [I] Remove Spark-3.2 support for Iceberg 1.5.0 and onwards. [iceberg]

ajantha-bhat commented on issue #9290: URL: https://github.com/apache/iceberg/issues/9290#issuecomment-1854477223 I will have a PR by tomorrow (based on the community discussion conclusion). -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [I] Iceberg table querying and joining issues [iceberg]

amogh-jahagirdar commented on issue #9287: URL: https://github.com/apache/iceberg/issues/9287#issuecomment-1854489654 This seems like an AWS Athena issue, could you report the issue to them? -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [I] Iceberg table querying and joining issues [iceberg]

amogh-jahagirdar closed issue #9287: Iceberg table querying and joining issues URL: https://github.com/apache/iceberg/issues/9287 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] JDBC catalog fix namespaceExists check [iceberg]

amogh-jahagirdar commented on PR #8340: URL: https://github.com/apache/iceberg/pull/8340#issuecomment-1854514510 Merging, thanks again for this fix @ismailsimsek and your follow-through. Thanks for reviews @dramaticlly @nastra @rdblue -- This is an automated message from the Apache Git S

Re: [PR] JDBC catalog fix namespaceExists check [iceberg]

amogh-jahagirdar merged PR #8340: URL: https://github.com/apache/iceberg/pull/8340 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [I] [JDBC Catalog on PostgreSQL] Undesired query result from namespace with underscore in the name [iceberg]

amogh-jahagirdar closed issue #8321: [JDBC Catalog on PostgreSQL] Undesired query result from namespace with underscore in the name URL: https://github.com/apache/iceberg/issues/8321 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [I] [JdbcCatalog] Issue with Namespace Exists [iceberg]

amogh-jahagirdar closed issue #8832: [JdbcCatalog] Issue with Namespace Exists URL: https://github.com/apache/iceberg/issues/8832 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Core: Lookup targeted position deletes by path [iceberg]

aokolnychyi commented on code in PR #9251: URL: https://github.com/apache/iceberg/pull/9251#discussion_r1425822122 ## core/src/main/java/org/apache/iceberg/DeleteFileIndex.java: ## @@ -474,68 +421,52 @@ private Collection loadDeleteFiles() { DeleteFileIndex build() {

Re: [PR] JDBC catalog fix namespaceExists check [iceberg]

ismailsimsek commented on PR #8340: URL: https://github.com/apache/iceberg/pull/8340#issuecomment-1854653706 Thank you all for reviewing it, @amogh-jahagirdar should we also merge it to 1.4.x branch, for next release? created PR for it if makes sense https://github.com/apache/iceberg/pul

Re: [PR] WIP: first pass at `UnboundTransform` [iceberg-python]

jayceslesar commented on PR #209: URL: https://github.com/apache/iceberg-python/pull/209#issuecomment-1854687084 @Fokko this isnt completely done, but would appreciate a review + some guidance on next steps once you run linting and see whats up haha -- This is an automated message from th

[PR] Add name-mapping [iceberg-python]

Fokko opened a new pull request, #212: URL: https://github.com/apache/iceberg-python/pull/212 All the things to (de)serialize the name-mapping, and all the neccessary visitors and such -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[PR] Rest append files [iceberg]

geruh opened a new pull request, #9292: URL: https://github.com/apache/iceberg/pull/9292 Hi All, I have created this smaller PR focused more on the AppendFiles operations. These changes are stemming from this PR: [#9237](https://github.com/apache/iceberg/pull/9237). @jackye19

[PR] Build: Bump mypy-boto3-glue from 1.33.5 to 1.34.0 [iceberg-python]

dependabot[bot] opened a new pull request, #213: URL: https://github.com/apache/iceberg-python/pull/213 Bumps [mypy-boto3-glue](https://github.com/youtype/mypy_boto3_builder) from 1.33.5 to 1.34.0. Commits See full diff in https://github.com/youtype/mypy_boto3_builder/commits";

Re: [PR] Rest Catalog: Add RESTful AppendFiles data operation [iceberg]

jackye1995 commented on code in PR #9292: URL: https://github.com/apache/iceberg/pull/9292#discussion_r1425987620 ## open-api/rest-catalog-open-api.yaml: ## @@ -2367,6 +2368,21 @@ components: items: type: string +AppendFilesUpdate: + allOf: +

Re: [PR] Rest Catalog: Add RESTful AppendFiles data operation [iceberg]

jackye1995 commented on code in PR #9292: URL: https://github.com/apache/iceberg/pull/9292#discussion_r1425987620 ## open-api/rest-catalog-open-api.yaml: ## @@ -2367,6 +2368,21 @@ components: items: type: string +AppendFilesUpdate: + allOf: +

Re: [I] Sort order id not being respected [iceberg]

github-actions[bot] commented on issue #7600: URL: https://github.com/apache/iceberg/issues/7600#issuecomment-1854885694 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Sort order id not being respected [iceberg]

github-actions[bot] closed issue #7600: Sort order id not being respected URL: https://github.com/apache/iceberg/issues/7600 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [I] Support Delete & Update iceberg table with Flink [iceberg]

github-actions[bot] commented on issue #7311: URL: https://github.com/apache/iceberg/issues/7311#issuecomment-1854885763 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Support Delete & Update iceberg table with Flink [iceberg]

github-actions[bot] closed issue #7311: Support Delete & Update iceberg table with Flink URL: https://github.com/apache/iceberg/issues/7311 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] Add name-mapping [iceberg-python]

rdblue commented on code in PR #212: URL: https://github.com/apache/iceberg-python/pull/212#discussion_r1426017219 ## pyiceberg/table/name_mapping.py: ## @@ -0,0 +1,204 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. S

[I] Spark Rewrite Write Performance [iceberg]

W-I-D-EE opened a new issue, #9293: URL: https://github.com/apache/iceberg/issues/9293 ### Query engine Spark 3.2.3 ### Question When compacting small parquet files i noticed it seems slow. Writing a single ~256mb parquet file to HDFS is taking 4-5 mins (total time inclu

[PR] Adding Snowflake's public documentation [iceberg-docs]

scottteal opened a new pull request, #297: URL: https://github.com/apache/iceberg-docs/pull/297 Updating vendor description links with latest public documentation from Snowflake. Adding Snowflake's primary docs link for Iceberg to the sidebar. -- This is an automated message from the Apac

Re: [I] Support bucket transform on multiple data columns [iceberg]

pdames commented on issue #5626: URL: https://github.com/apache/iceberg/issues/5626#issuecomment-1854969584 +1 to adding support for this feature. On my side, the principal use-case where this is beneficial is when hashing a composite primary key column where the cardinality of each individ

Re: [PR] feat: suport read/write Manifest [iceberg-rust]

ZENOTME commented on code in PR #79: URL: https://github.com/apache/iceberg-rust/pull/79#discussion_r1426129491 ## crates/iceberg/src/spec/manifest_list.rs: ## @@ -1120,15 +1236,17 @@ mod test { #[tokio::test] async fn test_manifest_list_writer_v2() { -let ex

Re: [PR] test: Remove binary manifest list avro file [iceberg-rust]

ZENOTME commented on code in PR #118: URL: https://github.com/apache/iceberg-rust/pull/118#discussion_r1426135812 ## crates/iceberg/src/spec/manifest_list.rs: ## @@ -931,55 +931,104 @@ mod test { use super::_serde::ManifestListV2; -#[test] -fn test_parse_manifes

Re: [PR] test: Remove binary manifest list avro file [iceberg-rust]

liurenjie1024 commented on code in PR #118: URL: https://github.com/apache/iceberg-rust/pull/118#discussion_r1426141277 ## crates/iceberg/src/spec/manifest_list.rs: ## @@ -931,55 +931,104 @@ mod test { use super::_serde::ManifestListV2; -#[test] -fn test_parse_m

Re: [I] Design of Serialization/Deserialization [iceberg-rust]

liurenjie1024 closed issue #2: Design of Serialization/Deserialization URL: https://github.com/apache/iceberg-rust/issues/2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

Re: [I] Design of Serialization/Deserialization [iceberg-rust]

liurenjie1024 commented on issue #2: URL: https://github.com/apache/iceberg-rust/issues/2#issuecomment-1855090734 Close this as we have finished discussion. Feel free to open if needed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [I] Discussion: The design of in memory model of iceberg spec. [iceberg-rust]

liurenjie1024 closed issue #3: Discussion: The design of in memory model of iceberg spec. URL: https://github.com/apache/iceberg-rust/issues/3 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [I] Discussion: The design of in memory model of iceberg spec. [iceberg-rust]

liurenjie1024 commented on issue #3: URL: https://github.com/apache/iceberg-rust/issues/3#issuecomment-1855091407 Close for now. Feel free to reopen when necessary. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [I] Read ManifestList, Manifest [iceberg-rust]

liurenjie1024 closed issue #36: Read ManifestList, Manifest URL: https://github.com/apache/iceberg-rust/issues/36 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

Re: [I] Read ManifestList, Manifest [iceberg-rust]

liurenjie1024 commented on issue #36: URL: https://github.com/apache/iceberg-rust/issues/36#issuecomment-1855092587 Close by #79 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] Spark 3.5: Parallelize file listing in add_files procedure [iceberg]

amogh-jahagirdar commented on code in PR #9274: URL: https://github.com/apache/iceberg/pull/9274#discussion_r1426150782 ## core/src/main/java/org/apache/iceberg/SnapshotProducer.java: ## @@ -351,6 +351,8 @@ private Map summary(TableMetadata previous) { SnapshotSummary.

Re: [I] Flink related document improvement request [iceberg]

911432 commented on issue #7395: URL: https://github.com/apache/iceberg/issues/7395#issuecomment-1855109087 not-stale -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] Core: Use properties while initializing default HadoopFileIO for Hadoop catalog. [iceberg]

agrawalreetika commented on code in PR #9283: URL: https://github.com/apache/iceberg/pull/9283#discussion_r1426167674 ## core/src/main/java/org/apache/iceberg/hadoop/HadoopCatalog.java: ## @@ -112,11 +112,11 @@ public void initialize(String name, Map properties) { this.war

Re: [I] Support `initial-default` when reading Avro [iceberg-rust]

ZENOTME commented on issue #119: URL: https://github.com/apache/iceberg-rust/issues/119#issuecomment-1855134729 If we read a V1 Avro file as a V2 object, when we write back, we also need to set [format-version in manifest](https://iceberg.apache.org/spec/#manifests:~:text=required-,format%2

Re: [PR] test: Remove binary manifest list avro file [iceberg-rust]

ZENOTME commented on code in PR #118: URL: https://github.com/apache/iceberg-rust/pull/118#discussion_r1426184104 ## crates/iceberg/src/spec/manifest_list.rs: ## @@ -931,55 +931,104 @@ mod test { use super::_serde::ManifestListV2; -#[test] -fn test_parse_manifes

  1   2   >