Re: [PR] Build: Bump org.apache.httpcomponents.client5:httpclient5 from 5.2.3 to 5.3 [iceberg]

Fokko merged PR #9260: URL: https://github.com/apache/iceberg/pull/9260 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [PR] Modernize the readme's Status paragraph [iceberg]

Fokko commented on code in PR #9272: URL: https://github.com/apache/iceberg/pull/9272#discussion_r1424975883 ## README.md: ## @@ -31,15 +31,17 @@ Background and documentation is available at Iceberg is under active development at the Apache Softw

Re: [PR] Hive: Make HiveMetastoreExtension configurable [iceberg]

nastra commented on code in PR #9288: URL: https://github.com/apache/iceberg/pull/9288#discussion_r1424972913 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveMetastoreExtension.java: ## @@ -80,4 +82,27 @@ public HiveConf hiveConf() { public TestHiveMetastore metas

Re: [PR] Support parameterized tests at class-level with JUnit5 [iceberg]

nastra commented on PR #9161: URL: https://github.com/apache/iceberg/pull/9161#issuecomment-1853420221 @GianlucaPrincipini the PR still contains more changes than necessary. The scope of this PR is to add the parameterized test extensions and the conversion of `TestDictRowGroupFilter`. Can

Re: [PR] Core, Spark: Avoid manifest copies when importing data to V2 tables [iceberg]

aokolnychyi merged PR #8962: URL: https://github.com/apache/iceberg/pull/8962 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Core, Spark: Avoid manifest copies when importing data to V2 tables [iceberg]

aokolnychyi commented on PR #8962: URL: https://github.com/apache/iceberg/pull/8962#issuecomment-1853419504 Thanks, @szehon-ho! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] Hive: Make HiveMetastoreExtension configurable [iceberg]

ajantha-bhat commented on code in PR #9288: URL: https://github.com/apache/iceberg/pull/9288#discussion_r1424967124 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveMetastoreExtension.java: ## @@ -80,4 +82,27 @@ public HiveConf hiveConf() { public TestHiveMetastore

Re: [PR] Core, Spark: Avoid manifest copies when importing data to V2 tables [iceberg]

aokolnychyi commented on code in PR #8962: URL: https://github.com/apache/iceberg/pull/8962#discussion_r1424966640 ## api/src/main/java/org/apache/iceberg/AppendFiles.java: ## @@ -42,16 +42,17 @@ public interface AppendFiles extends SnapshotUpdate { * The manifest must cont

Re: [PR] Spark 3.5: Add Spark application id to summary of RewriteDataFilesSparkAction [iceberg]

nastra commented on code in PR #9273: URL: https://github.com/apache/iceberg/pull/9273#discussion_r1424962135 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RewritePositionDeleteFilesSparkAction.java: ## @@ -215,7 +215,9 @@ private ExecutorService rewriteServ

Re: [PR] Hive: Refactor TestHiveCatalog tests to use the core CatalogTests [iceberg]

nastra commented on code in PR #8918: URL: https://github.com/apache/iceberg/pull/8918#discussion_r1424960918 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCatalog.java: ## @@ -219,81 +208,6 @@ public void testInitializeCatalogWithProperties() { .isEqu

Re: [PR] Hive: Refactor TestHiveCatalog tests to use the core CatalogTests [iceberg]

nastra commented on PR #8918: URL: https://github.com/apache/iceberg/pull/8918#issuecomment-1853409896 @nk1506 @ajantha-bhat I've opened https://github.com/apache/iceberg/pull/9288 to make the extension configurable. -- This is an automated message from the Apache Git Service. To respond

[PR] Hive: Make HiveMetastoreExtension configurable [iceberg]

nastra opened a new pull request, #9288: URL: https://github.com/apache/iceberg/pull/9288 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mai

Re: [PR] Hive: Refactor TestHiveCatalog tests to use the core CatalogTests [iceberg]

nastra commented on PR #8918: URL: https://github.com/apache/iceberg/pull/8918#issuecomment-1853396886 I agree that creating the database inside the extension should be made configurable -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[PR] test: Remove binary manifest list avro file [iceberg-rust]

liurenjie1024 opened a new pull request, #118: URL: https://github.com/apache/iceberg-rust/pull/118 Close #70 We generate manifest list avro file on the fly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] feat: suport read/write Manifest [iceberg-rust]

ZENOTME commented on code in PR #79: URL: https://github.com/apache/iceberg-rust/pull/79#discussion_r1424936927 ## crates/iceberg/src/spec/partition.rs: ## @@ -69,6 +71,30 @@ impl PartitionSpec { .iter() .all(|f| matches!(f.transform, Transform:

[PR] fix snapshot schema issue: wrong usage of walrus operator [iceberg-python]

tonyps1223 opened a new pull request, #211: URL: https://github.com/apache/iceberg-python/pull/211 When given the first snapshot-id of the table, the schema got should have schema_id = 0. But under origin code's conditional statements, the use of walrus operator will cause the condition

Re: [PR] feat: suport read/write Manifest [iceberg-rust]

liurenjie1024 commented on code in PR #79: URL: https://github.com/apache/iceberg-rust/pull/79#discussion_r1424930591 ## crates/iceberg/src/spec/manifest_list.rs: ## @@ -156,12 +171,36 @@ impl ManifestListWriter { match self.format_version { FormatVersion::

Re: [PR] fix snapshot schema issue: wrong usage of walrus operator [iceberg-python]

Tonylin1998 closed pull request #210: fix snapshot schema issue: wrong usage of walrus operator URL: https://github.com/apache/iceberg-python/pull/210 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] feat: suport read/write Manifest [iceberg-rust]

liurenjie1024 commented on code in PR #79: URL: https://github.com/apache/iceberg-rust/pull/79#discussion_r1424928716 ## crates/iceberg/src/spec/manifest_list.rs: ## @@ -156,12 +171,36 @@ impl ManifestListWriter { match self.format_version { FormatVersion::

Re: [I] Move iceberg-orc files to iceberg-core module [iceberg]

ajantha-bhat commented on issue #8454: URL: https://github.com/apache/iceberg/issues/8454#issuecomment-1853372121 Not needed as going with the approach of registering stats files in core module and files are written from engine integration module with iceberg-data dependency (core will not

Re: [I] Move iceberg-parquet files to iceberg-core module [iceberg]

ajantha-bhat closed issue #8453: Move iceberg-parquet files to iceberg-core module URL: https://github.com/apache/iceberg/issues/8453 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [I] Move iceberg-orc files to iceberg-core module [iceberg]

ajantha-bhat closed issue #8454: Move iceberg-orc files to iceberg-core module URL: https://github.com/apache/iceberg/issues/8454 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] Move iceberg-parquet files to iceberg-core module [iceberg]

ajantha-bhat commented on issue #8453: URL: https://github.com/apache/iceberg/issues/8453#issuecomment-1853372009 Not needed as going with the approach of registering stats files in core module and files are written from engine integration module with iceberg-data dependency (core will not

Re: [PR] Hive: Refactor TestHiveCatalog tests to use the core CatalogTests [iceberg]

nk1506 commented on PR #8918: URL: https://github.com/apache/iceberg/pull/8918#issuecomment-1853370309 > We can have a **less disruptive change** by having a new constructor for `HiveMetastoreExtension` that doesn't take `DB_NAME` and skips DB creation if it is not set. > > Only use

Re: [PR] feat: suport read/write Manifest [iceberg-rust]

liurenjie1024 commented on code in PR #79: URL: https://github.com/apache/iceberg-rust/pull/79#discussion_r1424926195 ## crates/iceberg/src/spec/manifest_list.rs: ## @@ -156,12 +171,36 @@ impl ManifestListWriter { match self.format_version { FormatVersion::

Re: [PR] feat: suport read/write Manifest [iceberg-rust]

liurenjie1024 commented on code in PR #79: URL: https://github.com/apache/iceberg-rust/pull/79#discussion_r1424922090 ## crates/iceberg/src/spec/manifest_list.rs: ## @@ -156,12 +171,36 @@ impl ManifestListWriter { match self.format_version { FormatVersion::

Re: [PR] feat: suport read/write Manifest [iceberg-rust]

ZENOTME commented on code in PR #79: URL: https://github.com/apache/iceberg-rust/pull/79#discussion_r1424919618 ## crates/iceberg/src/spec/manifest_list.rs: ## @@ -156,12 +171,36 @@ impl ManifestListWriter { match self.format_version { FormatVersion::V1 =>

Re: [PR] feat: suport read/write Manifest [iceberg-rust]

ZENOTME commented on code in PR #79: URL: https://github.com/apache/iceberg-rust/pull/79#discussion_r1424917585 ## crates/iceberg/src/spec/manifest_list.rs: ## @@ -156,12 +171,36 @@ impl ManifestListWriter { match self.format_version { FormatVersion::V1 =>

Re: [PR] Spark 3.5: Parallelize file listing in add_files procedure [iceberg]

manuzhang commented on PR #9274: URL: https://github.com/apache/iceberg/pull/9274#issuecomment-1853355242 @amogh-jahagirdar please check again. I've restored public util APIs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] feat: suport read/write Manifest [iceberg-rust]

liurenjie1024 commented on code in PR #79: URL: https://github.com/apache/iceberg-rust/pull/79#discussion_r1424912963 ## .typos.toml: ## @@ -0,0 +1,2 @@ +[files] +extend-exclude = ["crates/iceberg/testdata"] Review Comment: Is it possible to sth like `**/testdata`? I think a

Re: [PR] feat: suport read/write Manifest [iceberg-rust]

liurenjie1024 commented on code in PR #79: URL: https://github.com/apache/iceberg-rust/pull/79#discussion_r1424901931 ## crates/iceberg/src/spec/partition.rs: ## @@ -69,6 +71,30 @@ impl PartitionSpec { .iter() .all(|f| matches!(f.transform, Tran

Re: [PR] Flink: Adds support for 1.18 version [iceberg]

pvary commented on PR #9211: URL: https://github.com/apache/iceberg/pull/9211#issuecomment-1853330619 @rodmeneses: Could you please update the docs too? Here are the supported Flink versions: https://iceberg.apache.org/multi-engine-support/#apache-flink And please check other place

Re: [PR] Flink: switch to use SortKey for data statistics [iceberg]

pvary commented on code in PR #9212: URL: https://github.com/apache/iceberg/pull/9212#discussion_r1424890097 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/SortKeySerializer.java: ## @@ -0,0 +1,353 @@ +/* + * Licensed to the Apache Software Foundation (

[PR] fix snapshot schema issue: wrong usage of walrus operator [iceberg-python]

Tonylin1998 opened a new pull request, #210: URL: https://github.com/apache/iceberg-python/pull/210 When given the first snapshot-id of the table, the schema got should have schema_id = 0. But under origin code's conditional statements, the use of walrus operator will cause the condition

Re: [PR] Add doc for rewriting manifest with spec id [iceberg]

ajantha-bhat commented on code in PR #9253: URL: https://github.com/apache/iceberg/pull/9253#discussion_r1424879348 ## docs/spark-procedures.md: ## @@ -442,10 +442,11 @@ This procedure invalidates all cached Spark plans that reference the affected ta Usage -| Argument

Re: [PR] feat: suport read/write Manifest [iceberg-rust]

ZENOTME commented on code in PR #79: URL: https://github.com/apache/iceberg-rust/pull/79#discussion_r1424859048 ## crates/iceberg/src/spec/manifest_list.rs: ## @@ -30,6 +30,9 @@ use self::{ use super::{FormatVersion, StructType}; +/// The seq number when no added files are

Re: [PR] feat: suport read/write Manifest [iceberg-rust]

ZENOTME commented on PR #79: URL: https://github.com/apache/iceberg-rust/pull/79#issuecomment-1853284225 > > I think we need to add configurations in `typos.toml` to skip the check: > > https://github.com/crate-ci/typos/blob/master/docs/reference.md > > Yep, we should skip check on

Re: [PR] feat: suport read/write Manifest [iceberg-rust]

Xuanwo commented on PR #79: URL: https://github.com/apache/iceberg-rust/pull/79#issuecomment-1853251575 > I think we need to add configurations in `typos.toml` to skip the check: > > https://github.com/crate-ci/typos/blob/master/docs/reference.md Yep, we should skip check on `te

Re: [PR] Modernize the readme's Status paragraph [iceberg]

szehon-ho commented on code in PR #9272: URL: https://github.com/apache/iceberg/pull/9272#discussion_r1424816240 ## README.md: ## @@ -31,15 +31,17 @@ Background and documentation is available at Iceberg is under active development at the Apache S

Re: [PR] Core: Lookup targeted position deletes by path [iceberg]

szehon-ho commented on code in PR #9251: URL: https://github.com/apache/iceberg/pull/9251#discussion_r1424802087 ## core/src/main/java/org/apache/iceberg/DeleteFileIndex.java: ## @@ -474,68 +421,52 @@ private Collection loadDeleteFiles() { DeleteFileIndex build() { I

Re: [PR] Core: Lookup targeted position deletes by path [iceberg]

szehon-ho commented on code in PR #9251: URL: https://github.com/apache/iceberg/pull/9251#discussion_r1424766968 ## core/src/main/java/org/apache/iceberg/DeleteFileIndex.java: ## @@ -582,93 +513,187 @@ private Iterable>> deleteManifestRea } } - // a group of indexed

[I] Iceberg table querying and joining issues [iceberg]

MummadiT opened a new issue, #9287: URL: https://github.com/apache/iceberg/issues/9287 we have integrated AWS Glue with Iceberg. I was able to create an iceberg table and insert the data into the table. but i have couple of issues listed below. 1. whenever I try to pull the data

Re: [PR] feat: suport read/write Manifest [iceberg-rust]

ZENOTME commented on code in PR #79: URL: https://github.com/apache/iceberg-rust/pull/79#discussion_r1424780979 ## crates/iceberg/src/spec/manifest_list.rs: ## @@ -30,6 +30,9 @@ use self::{ use super::{FormatVersion, StructType}; +/// The seq number when no added files are

Re: [PR] Spark: Use Awaitility instead of Thread.sleep [iceberg]

yyy1000 commented on PR #9224: URL: https://github.com/apache/iceberg/pull/9224#issuecomment-1853153204 Hi, I want to make an update of this PR: The CI failed indicated that : ``` Caused by: org.apache.iceberg.exceptions.CommitFailedException: Base metadata location

Re: [I] Flink: Add support for Flink 1.18 [iceberg]

YesOrNo828 closed issue #8930: Flink: Add support for Flink 1.18 URL: https://github.com/apache/iceberg/issues/8930 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

Re: [PR] Modernize the readme's Status paragraph [iceberg]

mt-ronkorving commented on PR #9272: URL: https://github.com/apache/iceberg/pull/9272#issuecomment-1853127445 @bitsondatadev Thanks for the suggestions, I've merged them in -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [PR] Core, API: Move SQLViewRepresentation to API [iceberg]

amogh-jahagirdar commented on code in PR #9278: URL: https://github.com/apache/iceberg/pull/9278#discussion_r1424740521 ## .palantir/revapi.yml: ## @@ -877,6 +877,10 @@ acceptedBreaks: - code: "java.field.serialVersionUIDChanged" new: "field org.apache.iceberg.util.S

Re: [PR] Core, API: Move SQLViewRepresentation to API [iceberg]

amogh-jahagirdar commented on code in PR #9278: URL: https://github.com/apache/iceberg/pull/9278#discussion_r1424739300 ## core/src/main/java/org/apache/iceberg/view/ImmutableSQLViewRepresentation.java: ## @@ -0,0 +1,241 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[PR] Spark: Add tests for select using tag and branch identifier [iceberg]

wypoon opened a new pull request, #9286: URL: https://github.com/apache/iceberg/pull/9286 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mai

Re: [I] Flink related document improvement request [iceberg]

github-actions[bot] commented on issue #7395: URL: https://github.com/apache/iceberg/issues/7395#issuecomment-1853061182 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Spark: migrate hive table to iceberg table(using hive metastore) [iceberg]

github-actions[bot] commented on issue #7328: URL: https://github.com/apache/iceberg/issues/7328#issuecomment-1853061203 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Flink: switch to use SortKey for data statistics [iceberg]

yegangy0718 commented on code in PR #9212: URL: https://github.com/apache/iceberg/pull/9212#discussion_r1424686796 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/SortKeySerializer.java: ## @@ -0,0 +1,353 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] Arrow: Allow missing field-ids from Schema [iceberg-python]

rdblue commented on code in PR #183: URL: https://github.com/apache/iceberg-python/pull/183#discussion_r1424585279 ## pyiceberg/io/pyarrow.py: ## @@ -713,28 +714,50 @@ def primitive(self, primitive: pa.DataType) -> Optional[T]: """Visit a primitive type.""" -def _g

Re: [PR] Arrow: Allow missing field-ids from Schema [iceberg-python]

rdblue commented on PR #183: URL: https://github.com/apache/iceberg-python/pull/183#issuecomment-1852828744 Per [my comment](https://github.com/apache/iceberg-python/pull/183#discussion_r1424585279), I'm -1 if the intent of this PR is to read data files without IDs. That must use a name ma

Re: [I] Add `CAST` to `parser.py` [iceberg-python]

jayceslesar commented on issue #198: URL: https://github.com/apache/iceberg-python/issues/198#issuecomment-1852806543 @Fokko I think im close -- will have a draft MR shortly -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] Add `CAST` to `parser.py` [iceberg-python]

Fokko commented on issue #198: URL: https://github.com/apache/iceberg-python/issues/198#issuecomment-1852805761 @jayceslesar No we need to add the `UnboundTransform` in Python 👍 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [I] Add `CAST` to `parser.py` [iceberg-python]

jayceslesar commented on issue #198: URL: https://github.com/apache/iceberg-python/issues/198#issuecomment-1852792721 hey @Fokko, ive been playing around with it and am wondering if the `BoundTransform` needs to be updated to take in a `Reference` like it does in the [java](https://github.

Re: [I] contributing.md file is missing... [iceberg-python]

Fokko closed issue #101: contributing.md file is missing... URL: https://github.com/apache/iceberg-python/issues/101 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

Re: [PR] added contributing.md file [iceberg-python]

Fokko closed pull request #102: added contributing.md file URL: https://github.com/apache/iceberg-python/pull/102 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

Re: [PR] Pull main into latest [iceberg-docs]

Fokko merged PR #296: URL: https://github.com/apache/iceberg-docs/pull/296 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Hive: Introduce HiveMetastoreExtension for Hive tests [iceberg]

nk1506 commented on code in PR #9282: URL: https://github.com/apache/iceberg/pull/9282#discussion_r1424384336 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveMetastoreExtension.java: ## @@ -0,0 +1,84 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

Re: [I] DatasourceV2 does not prune columns after V2ScanRelationPushDown [iceberg]

rdblue commented on issue #9268: URL: https://github.com/apache/iceberg/issues/9268#issuecomment-1852568441 @aokolnychyi are you aware of this issue? It looks like some additional pruning may be done after pushdown happens? -- This is an automated message from the Apache Git Service. To r

Re: [PR] Standard key manager [iceberg]

rdblue commented on PR #6884: URL: https://github.com/apache/iceberg/pull/6884#issuecomment-1852558659 Merged in #9277! Thanks, @ggershinsky! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Standard key manager [iceberg]

rdblue closed pull request #6884: Standard key manager URL: https://github.com/apache/iceberg/pull/6884 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Core: Add StandardEncryptionManager [iceberg]

rdblue commented on PR #9277: URL: https://github.com/apache/iceberg/pull/9277#issuecomment-1852557672 Thanks for the final review, @RussellSpitzer! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] Core: Add StandardEncryptionManager [iceberg]

rdblue merged PR #9277: URL: https://github.com/apache/iceberg/pull/9277 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Core: Add StandardEncryptionManager [iceberg]

rdblue commented on code in PR #9277: URL: https://github.com/apache/iceberg/pull/9277#discussion_r1424398342 ## core/src/main/java/org/apache/iceberg/encryption/EncryptionUtil.java: ## @@ -0,0 +1,107 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

Re: [PR] Hive: Introduce HiveMetastoreExtension for Hive tests [iceberg]

nk1506 commented on code in PR #9282: URL: https://github.com/apache/iceberg/pull/9282#discussion_r1424384336 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveMetastoreExtension.java: ## @@ -0,0 +1,84 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

Re: [PR] Core: Add StandardEncryptionManager [iceberg]

RussellSpitzer commented on code in PR #9277: URL: https://github.com/apache/iceberg/pull/9277#discussion_r1424319696 ## core/src/main/java/org/apache/iceberg/encryption/EncryptionUtil.java: ## @@ -0,0 +1,107 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [I] AWS access denied from metadata file on S3 [iceberg-python]

Fokko commented on issue #201: URL: https://github.com/apache/iceberg-python/issues/201#issuecomment-1852419470 @itaise Can you try: ``` PYICEBERG_CATALOG__PROD__S3__ACCESS_KEY_ID PYICEBERG_CATALOG__PROD__S3__SECRET_ACCESS_KEY ``` Since you're loading the prod catalog:

[PR] Openlineage support [iceberg]

pawel-big-lebowski opened a new pull request, #9285: URL: https://github.com/apache/iceberg/pull/9285 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

Re: [I] Flink: Add support for Flink 1.18 [iceberg]

pvary commented on issue #8930: URL: https://github.com/apache/iceberg/issues/8930#issuecomment-1852135783 As #9211 is merged, we can close this. WDYT: @YesOrNo828 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] Hive: Introduce HiveMetastoreExtension for Hive tests [iceberg]

ajantha-bhat commented on code in PR #9282: URL: https://github.com/apache/iceberg/pull/9282#discussion_r1424056591 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveMetastoreExtension.java: ## @@ -0,0 +1,84 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

Re: [PR] Spec: `snapshot_id` is optional for V1 [iceberg]

JFinis commented on PR #8704: URL: https://github.com/apache/iceberg/pull/8704#issuecomment-1852118074 @rdblue you closed this issue, so the code continues treating the snapshot-id as optional. However, the spec still says it's required in v1. Do I see it correctly, that if it is considered

Re: [PR] Hive: Introduce HiveMetastoreExtension for Hive tests [iceberg]

nastra merged PR #9282: URL: https://github.com/apache/iceberg/pull/9282 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Hive: Introduce HiveMetastoreExtension for Hive tests [iceberg]

nk1506 commented on code in PR #9282: URL: https://github.com/apache/iceberg/pull/9282#discussion_r1424040166 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveMetastoreExtension.java: ## @@ -0,0 +1,84 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

Re: [I] Create Branches / TAGS between 2 snapshots [iceberg]

fanaticjo commented on issue #9281: URL: https://github.com/apache/iceberg/issues/9281#issuecomment-1852055449 we just wanted using a tag/ branch to pull out the data written into that period only . i saw there is an incremental read available in in dataframe df = spark.read \ .format(

Re: [I] Create Branches / TAGS between 2 snapshots [iceberg]

nastra commented on issue #9281: URL: https://github.com/apache/iceberg/issues/9281#issuecomment-1852001153 Are you saying you want to create a branch/tag and refer to a snapshot without its history? -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [I] Documentation [iceberg-rust]

Xuanwo commented on issue #114: URL: https://github.com/apache/iceberg-rust/issues/114#issuecomment-1851993567 I'm fine with not keeping a main branch for the API documentation. We can begin with the simplest approach and expand it later. -- This is an automated message from the Apache Gi

Re: [I] Create Branches / TAGS between 2 snapshots [iceberg]

fanaticjo closed issue #9281: Create Branches / TAGS between 2 snapshots URL: https://github.com/apache/iceberg/issues/9281 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [I] Create Branches / TAGS between 2 snapshots [iceberg]

fanaticjo commented on issue #9281: URL: https://github.com/apache/iceberg/issues/9281#issuecomment-1851990066 i want to create a branch / tag only for the latest data load while AS of version considers the latest data and the previous data also . For example insert 1 , 2

Re: [PR] Hive: Introduce HiveMetastoreExtension for Hive tests [iceberg]

nastra commented on code in PR #9282: URL: https://github.com/apache/iceberg/pull/9282#discussion_r1423924054 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveMetastoreExtension.java: ## @@ -0,0 +1,84 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

Re: [PR] Core: Lookup targeted position deletes by path [iceberg]

aokolnychyi commented on PR #9251: URL: https://github.com/apache/iceberg/pull/9251#issuecomment-1851931126 @szehon-ho @RussellSpitzer @flyrain @amogh-jahagirdar @nastra @Fokko, could you check this one? -- This is an automated message from the Apache Git Service. To respond to the messag

Re: [PR] Modernize the readme's Status paragraph [iceberg]

bitsondatadev commented on code in PR #9272: URL: https://github.com/apache/iceberg/pull/9272#discussion_r1423895171 ## README.md: ## @@ -31,15 +31,17 @@ Background and documentation is available at Iceberg is under active development at the Apac

Re: [PR] Spark-3.5: Ensure that partition stats files are considered for GC procedures [iceberg]

ajantha-bhat commented on code in PR #9284: URL: https://github.com/apache/iceberg/pull/9284#discussion_r1423868004 ## core/src/main/java/org/apache/iceberg/ReachableFileUtil.java: ## @@ -148,12 +148,49 @@ public static List statisticsFilesLocations(Table table) { * @param

Re: [PR] Hive: Introduce HiveMetastoreExtension for Hive tests [iceberg]

ajantha-bhat commented on code in PR #9282: URL: https://github.com/apache/iceberg/pull/9282#discussion_r1423884213 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveMetastoreExtension.java: ## @@ -0,0 +1,84 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

Re: [PR] Hive: Introduce HiveMetastoreExtension for Hive tests [iceberg]

ajantha-bhat commented on code in PR #9282: URL: https://github.com/apache/iceberg/pull/9282#discussion_r1423882541 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveCreateReplaceTableTest.java: ## @@ -60,8 +66,23 @@ public class HiveCreateReplaceTableTest extends Hiv

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

ajantha-bhat commented on PR #8502: URL: https://github.com/apache/iceberg/pull/8502#issuecomment-1851876373 @aokolnychyi: Fixed all the comments and also opened a new Spark module PR (which is dependent on this) to ensure partition stats are considered for GC (expire snapshots and remove o

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

ajantha-bhat commented on code in PR #8502: URL: https://github.com/apache/iceberg/pull/8502#discussion_r142387 ## core/src/main/java/org/apache/iceberg/SetPartitionStatistics.java: ## @@ -0,0 +1,74 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

ajantha-bhat commented on code in PR #8502: URL: https://github.com/apache/iceberg/pull/8502#discussion_r1423873054 ## core/src/main/java/org/apache/iceberg/ReachableFileUtil.java: ## @@ -137,7 +137,9 @@ public static List manifestListLocations(Table table, Set snapshot *

Re: [PR] Hive: Introduce HiveMetastoreExtension for Hive tests [iceberg]

nastra commented on code in PR #9282: URL: https://github.com/apache/iceberg/pull/9282#discussion_r1423864361 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveCreateReplaceTableTest.java: ## @@ -60,8 +66,23 @@ public class HiveCreateReplaceTableTest extends HiveMetas

[PR] Spark-3.5: Ensure that partition stats files are considered for GC procedures [iceberg]

ajantha-bhat opened a new pull request, #9284: URL: https://github.com/apache/iceberg/pull/9284 Currently depends on https://github.com/apache/iceberg/pull/8502/ Please refer only the top commit for this PR related changes. -- This is an automated message from the Apache Gi

Re: [PR] Spark 3.5: Add Spark application id to summary of RewriteDataFilesSparkAction [iceberg]

manuzhang commented on code in PR #9273: URL: https://github.com/apache/iceberg/pull/9273#discussion_r1423851348 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RewritePositionDeleteFilesSparkAction.java: ## @@ -215,7 +215,9 @@ private ExecutorService rewriteS

Re: [PR] Hive: Introduce HiveMetastoreExtension for Hive tests [iceberg]

nk1506 commented on code in PR #9282: URL: https://github.com/apache/iceberg/pull/9282#discussion_r1423846032 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestCachedClientPool.java: ## @@ -28,30 +28,46 @@ import java.util.Map; import java.util.concurrent.TimeUnit;

Re: [PR] Hive: Introduce HiveMetastoreExtension for Hive tests [iceberg]

ajantha-bhat commented on code in PR #9282: URL: https://github.com/apache/iceberg/pull/9282#discussion_r1423842984 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveCreateReplaceTableTest.java: ## @@ -60,8 +66,23 @@ public class HiveCreateReplaceTableTest extends Hiv

Re: [PR] Hive: Introduce HiveMetastoreExtension for Hive tests [iceberg]

nastra commented on code in PR #9282: URL: https://github.com/apache/iceberg/pull/9282#discussion_r1423831381 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCommitLocks.java: ## @@ -88,13 +95,33 @@ public class TestHiveCommitLocks extends HiveTableBaseTest {

Re: [PR] Hive: Introduce HiveMetastoreExtension for Hive tests [iceberg]

nastra commented on code in PR #9282: URL: https://github.com/apache/iceberg/pull/9282#discussion_r1423827359 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveCreateReplaceTableTest.java: ## @@ -60,8 +66,23 @@ public class HiveCreateReplaceTableTest extends HiveMetas

Re: [PR] Hive: Introduce HiveMetastoreExtension for Hive tests [iceberg]

ajantha-bhat commented on code in PR #9282: URL: https://github.com/apache/iceberg/pull/9282#discussion_r1423817274 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestCachedClientPool.java: ## @@ -28,30 +28,46 @@ import java.util.Map; import java.util.concurrent.TimeU

Re: [PR] Hive: Introduce HiveMetastoreExtension for Hive tests [iceberg]

nk1506 commented on code in PR #9282: URL: https://github.com/apache/iceberg/pull/9282#discussion_r1423816230 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveCreateReplaceTableTest.java: ## @@ -60,8 +66,23 @@ public class HiveCreateReplaceTableTest extends HiveMetas

Re: [PR] Hive: Introduce HiveMetastoreExtension for Hive tests [iceberg]

nk1506 commented on code in PR #9282: URL: https://github.com/apache/iceberg/pull/9282#discussion_r1423814496 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveMetastoreExtension.java: ## @@ -0,0 +1,84 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

  1   2   >