Re: [I] [bug] dependabot is using `Poetry 1.8.5` [iceberg-python]

2025-02-21 Thread via GitHub
omkenge commented on issue #1702: URL: https://github.com/apache/iceberg-python/issues/1702#issuecomment-2676070920 Can we add ignore rule? ignore: - dependency-name: "poetry" versions: - "1.8.5" -- This is an automated message from the Apache Git Servi

Re: [PR] [1.8.x] Core: Remove namespace/table/view HEAD endpoints from defaults (#12351) [iceberg]

2025-02-21 Thread via GitHub
nastra merged PR #12368: URL: https://github.com/apache/iceberg/pull/12368 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

[PR] Patch: Downgrade Log Level for S3 Region Mismatch [iceberg-python]

2025-02-21 Thread via GitHub
omkenge opened a new pull request, #1706: URL: https://github.com/apache/iceberg-python/pull/1706 This patch updates the logging behavior when there is a mismatch between the provided S3 bucket region and the actual bucket region. Instead of logging a warning—which may expose internal confi

Re: [I] don't show warning about bucket [iceberg-python]

2025-02-21 Thread via GitHub
omkenge commented on issue #1705: URL: https://github.com/apache/iceberg-python/issues/1705#issuecomment-2676066291 ref this #1706 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Ensure Partition Specs can only contain primitive types [iceberg-rust]

2025-02-21 Thread via GitHub
jonathanc-n commented on PR #780: URL: https://github.com/apache/iceberg-rust/pull/780#issuecomment-2676061908 @Lordworms Hello, are you still working on this, if not I would be interested in taking this up or making the corrections? -- This is an automated message from the Apache Git Ser

Re: [PR] Add Variant custom logical type for Avro [iceberg]

2025-02-21 Thread via GitHub
aihuaxu commented on code in PR #12238: URL: https://github.com/apache/iceberg/pull/12238#discussion_r1966427898 ## core/src/main/java/org/apache/iceberg/avro/TypeToSchema.java: ## @@ -187,6 +187,21 @@ public Schema map(Types.MapType map, Schema keySchema, Schema valueSchema) {

Re: [PR] Add Variant custom logical type for Avro [iceberg]

2025-02-21 Thread via GitHub
aihuaxu commented on code in PR #12238: URL: https://github.com/apache/iceberg/pull/12238#discussion_r1966427898 ## core/src/main/java/org/apache/iceberg/avro/TypeToSchema.java: ## @@ -187,6 +187,21 @@ public Schema map(Types.MapType map, Schema keySchema, Schema valueSchema) {

[I] don't show warning about bucket [iceberg-python]

2025-02-21 Thread via GitHub
djouallah opened a new issue, #1705: URL: https://github.com/apache/iceberg-python/issues/1705 ### Apache Iceberg version main (development) ### Please describe the bug 🐞 using 0.9 rc1 when connection to polaris, I am getting this warning `WARNING:pyiceberg.

Re: [I] Explore potential issue with `scan` returning the incorrect results [iceberg-python]

2025-02-21 Thread via GitHub
vtk9 commented on issue #1506: URL: https://github.com/apache/iceberg-python/issues/1506#issuecomment-2676001441 @kevinjqliu @Fokko Do you know if this bug is also present in 0.7.1? or was it introduced in 0.8.0/0.8.1? -- This is an automated message from the Apache Git Service.

Re: [PR] API: Define RepairManifests action interface [iceberg]

2025-02-21 Thread via GitHub
danielcweeks commented on PR #10784: URL: https://github.com/apache/iceberg/pull/10784#issuecomment-2675997406 > Do we still pursue this? Yes, @amogh-jahagirdar is still looking at this and we continue to see use cases where this is necessary. -- This is an automated message from t

Re: [PR] [1.7.x] Fix `{LICENSE,NOTICE}` for Flink Runtime [iceberg]

2025-02-21 Thread via GitHub
Fokko commented on PR #12358: URL: https://github.com/apache/iceberg/pull/12358#issuecomment-2675322760 Thanks for the review @jbonofre and @jbonofre 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] API: Define RepairManifests action interface [iceberg]

2025-02-21 Thread via GitHub
flyrain commented on PR #10784: URL: https://github.com/apache/iceberg/pull/10784#issuecomment-2675966634 Do we still pursue this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] Spec: Add implementation note on `current-snapshot-id` [iceberg]

2025-02-21 Thread via GitHub
danielcweeks commented on code in PR #12334: URL: https://github.com/apache/iceberg/pull/12334#discussion_r1966349930 ## format/spec.md: ## @@ -1754,6 +1754,14 @@ Snapshot summary can include metrics fields to track numeric stats of the snapsh | **`engine-name`**| "spa

[I] RUSTSEC-2025-0007: *ring* is unmaintained [iceberg-rust]

2025-02-21 Thread via GitHub
github-actions[bot] opened a new issue, #985: URL: https://github.com/apache/iceberg-rust/issues/985 > *ring* is unmaintained | Details || | --- | -- | | St

Re: [PR] Spec: Add implementation note on `current-snapshot-id` [iceberg]

2025-02-21 Thread via GitHub
danielcweeks commented on code in PR #12334: URL: https://github.com/apache/iceberg/pull/12334#discussion_r1966349930 ## format/spec.md: ## @@ -1754,6 +1754,14 @@ Snapshot summary can include metrics fields to track numeric stats of the snapsh | **`engine-name`**| "spa

Re: [PR] Spec: Add implementation note on `current-snapshot-id` [iceberg]

2025-02-21 Thread via GitHub
danielcweeks commented on code in PR #12334: URL: https://github.com/apache/iceberg/pull/12334#discussion_r1966349930 ## format/spec.md: ## @@ -1754,6 +1754,14 @@ Snapshot summary can include metrics fields to track numeric stats of the snapsh | **`engine-name`**| "spa

Re: [PR] AWS: Add support for enabling access to S3 Requester Pays bucket [iceberg]

2025-02-21 Thread via GitHub
github-actions[bot] commented on PR #11915: URL: https://github.com/apache/iceberg/pull/11915#issuecomment-2675860079 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Fix #305: ManifestEntry partition field schema should be dynamically … [iceberg-go]

2025-02-21 Thread via GitHub
zeroshade commented on PR #307: URL: https://github.com/apache/iceberg-go/pull/307#issuecomment-2675865197 The only way to get the file length from the file itself is via Seeking to the end unfortunately. ReadAt could be sufficient, but would require having to externally provide the

Re: [I] Support Rest Catalog Metrics Endpoint [iceberg-python]

2025-02-21 Thread via GitHub
github-actions[bot] commented on issue #474: URL: https://github.com/apache/iceberg-python/issues/474#issuecomment-2675861959 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity oc

Re: [PR] Spark3.5: Standardizing Error Handling in Iceberg Spark Module - TestViews [iceberg]

2025-02-21 Thread via GitHub
github-actions[bot] commented on PR #11993: URL: https://github.com/apache/iceberg/pull/11993#issuecomment-2675860122 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Core, Test: Parsing and Writing Tests for V3 Metadata [iceberg]

2025-02-21 Thread via GitHub
github-actions[bot] commented on PR #12025: URL: https://github.com/apache/iceberg/pull/12025#issuecomment-2675860150 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] WIP: Interface based DataFile reader and writer API [iceberg]

2025-02-21 Thread via GitHub
rdblue commented on PR #12298: URL: https://github.com/apache/iceberg/pull/12298#issuecomment-2675856645 While I think the goal here is a good one, the implementation looks too complex to be workable in its current form. The primary issue that we currently have is adapting object mode

Re: [PR] Fix #305: ManifestEntry partition field schema should be dynamically … [iceberg-go]

2025-02-21 Thread via GitHub
arnaudbriche commented on PR #307: URL: https://github.com/apache/iceberg-go/pull/307#issuecomment-2675851774 Ok. I though ReadAt would be sufficient. Thx for answering! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Fix #305: ManifestEntry partition field schema should be dynamically … [iceberg-go]

2025-02-21 Thread via GitHub
zeroshade commented on PR #307: URL: https://github.com/apache/iceberg-go/pull/307#issuecomment-2675842086 The seek is necessary for efficient Parquet processing, and required by the interface APIs for reading Parquet -- This is an automated message from the Apache Git Service. To respon

Re: [PR] Fix #305: ManifestEntry partition field schema should be dynamically … [iceberg-go]

2025-02-21 Thread via GitHub
arnaudbriche commented on PR #307: URL: https://github.com/apache/iceberg-go/pull/307#issuecomment-2675839884 Nothing to do with the PR but just a quick question regarding io API. Why is there a requirement for `io.ReadSeekCloser` on `File` ? It seems like `Seek` is not used anywhere

Re: [PR] Fix #305: ManifestEntry partition field schema should be dynamically … [iceberg-go]

2025-02-21 Thread via GitHub
arnaudbriche commented on PR #307: URL: https://github.com/apache/iceberg-go/pull/307#issuecomment-2675831964 @zeroshade it's working fine ! API looks better. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] Upgrading to iceberg 1.8.0 and spark 3.5.4 causes NoClassDefFoundError: org/apache/spark/sql/catalyst/expressions/AnsiCast [iceberg]

2025-02-21 Thread via GitHub
nastra commented on issue #12371: URL: https://github.com/apache/iceberg/issues/12371#issuecomment-2675010777 @Knorreman make sure you don't have any old Iceberg + Spark dependencies on the classpath. Which Iceberg + Spark version were you running prior to upgrading? -- This is an automa

Re: [PR] [test] Run partition transform tests for all transforms [iceberg-python]

2025-02-21 Thread via GitHub
kevinjqliu commented on PR #1592: URL: https://github.com/apache/iceberg-python/pull/1592#issuecomment-2675439057 [`TruncateTransform(10),`](https://github.com/apache/iceberg-python/blob/a58f099aa501f6fd4345a331295d81fe0133554f/tests/table/test_partitioning.py#L203) for [`(BinaryType(), b"

Re: [PR] WIP: Interface based DataFile reader and writer API [iceberg]

2025-02-21 Thread via GitHub
rdblue commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r1966319213 ## core/src/main/resources/META-INF/services/org.apache.iceberg.io.datafile.DataFileServiceRegistry$WriterService: ## @@ -0,0 +1,20 @@ +# +# Licensed to the Apache Sof

Re: [PR] [test] Run partition transform tests for all transforms [iceberg-python]

2025-02-21 Thread via GitHub
kevinjqliu commented on PR #1592: URL: https://github.com/apache/iceberg-python/pull/1592#issuecomment-2675814915 all good here, thanks for the quick fix @Fokko -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] WIP: Interface based DataFile reader and writer API [iceberg]

2025-02-21 Thread via GitHub
rdblue commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r1966319213 ## core/src/main/resources/META-INF/services/org.apache.iceberg.io.datafile.DataFileServiceRegistry$WriterService: ## @@ -0,0 +1,20 @@ +# +# Licensed to the Apache Sof

Re: [PR] Add Variant custom logical type for Avro [iceberg]

2025-02-21 Thread via GitHub
aihuaxu commented on code in PR #12238: URL: https://github.com/apache/iceberg/pull/12238#discussion_r1966315547 ## core/src/main/java/org/apache/iceberg/avro/PruneColumns.java: ## @@ -92,7 +93,8 @@ public Schema record(Schema record, List names, List fields) { hasCh

Re: [PR] Add Variant custom logical type for Avro [iceberg]

2025-02-21 Thread via GitHub
aihuaxu commented on code in PR #12238: URL: https://github.com/apache/iceberg/pull/12238#discussion_r1966315547 ## core/src/main/java/org/apache/iceberg/avro/PruneColumns.java: ## @@ -92,7 +93,8 @@ public Schema record(Schema record, List names, List fields) { hasCh

Re: [PR] API: Move Variant interfaces and serialized implementations to API [iceberg]

2025-02-21 Thread via GitHub
rdblue merged PR #12374: URL: https://github.com/apache/iceberg/pull/12374 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Parquet: Implement Variant writers [iceberg]

2025-02-21 Thread via GitHub
rdblue commented on PR #12323: URL: https://github.com/apache/iceberg/pull/12323#issuecomment-2675805303 Rebased after moving variants to API in #12374. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] API: Move Variant interfaces and serialized implementations to API [iceberg]

2025-02-21 Thread via GitHub
rdblue commented on PR #12374: URL: https://github.com/apache/iceberg/pull/12374#issuecomment-2675800286 Thanks for the reviews, @danielcweeks and @aihuaxu! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] API: Move Variant interfaces and serialized implementations to API [iceberg]

2025-02-21 Thread via GitHub
rdblue commented on PR #12374: URL: https://github.com/apache/iceberg/pull/12374#issuecomment-2675799837 > I think the one preference I would have is that if we're moving some of the type info (basic/logical/physical), I feel it would be better to move it inner enum to Variant interface so

Re: [PR] [1.7.x] Fix `{LICENSE,NOTICE}` for Kafka Connect Runtime [iceberg]

2025-02-21 Thread via GitHub
Fokko commented on PR #12353: URL: https://github.com/apache/iceberg/pull/12353#issuecomment-2675322638 Thanks for the review @jbonofre and @jbonofre 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Fix #305: ManifestEntry partition field schema should be dynamically … [iceberg-go]

2025-02-21 Thread via GitHub
zeroshade commented on PR #307: URL: https://github.com/apache/iceberg-go/pull/307#issuecomment-2675784737 Ah i had to accept the invite -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] fix(catalog/rest): Nessie CreateTable [iceberg-go]

2025-02-21 Thread via GitHub
zeroshade merged PR #308: URL: https://github.com/apache/iceberg-go/pull/308 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [I] Support for list types? [iceberg-go]

2025-02-21 Thread via GitHub
zeroshade commented on issue #309: URL: https://github.com/apache/iceberg-go/issues/309#issuecomment-2675788295 @GabrielM98 please take a look at the linked PR and confirm for me that it solves your problem? -- This is an automated message from the Apache Git Service. To respond to the me

Re: [PR] Fix #305: ManifestEntry partition field schema should be dynamically … [iceberg-go]

2025-02-21 Thread via GitHub
zeroshade commented on PR #307: URL: https://github.com/apache/iceberg-go/pull/307#issuecomment-2675786497 @arnaudbriche can you take a look at my updated version here and make sure that it works with what you were trying for clickhouse? Feel free to comment on the actual API also if you li

Re: [PR] Fix #305: ManifestEntry partition field schema should be dynamically … [iceberg-go]

2025-02-21 Thread via GitHub
zeroshade commented on PR #307: URL: https://github.com/apache/iceberg-go/pull/307#issuecomment-2675783109 Looks like i have read access but not write access currently :frowning: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] Remove top-level import of `pyarrow` [iceberg-python]

2025-02-21 Thread via GitHub
kevinjqliu commented on PR #1703: URL: https://github.com/apache/iceberg-python/pull/1703#issuecomment-2675602138 🥳 onwards to 0.9.0 RC, thanks for the review @Fokko -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] [1.8.x] Fix netty-buffer version in aws-bundle LICENSE [iceberg]

2025-02-21 Thread via GitHub
jbonofre commented on code in PR #12365: URL: https://github.com/apache/iceberg/pull/12365#discussion_r1965663251 ## aws-bundle/LICENSE: ## @@ -219,7 +219,7 @@ License: The Apache Software License, Version 2.0 - http://www.apache.org/licens -

Re: [PR] Check binary array length when applying truncate [iceberg-rust]

2025-02-21 Thread via GitHub
kevinjqliu merged PR #984: URL: https://github.com/apache/iceberg-rust/pull/984 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@icebe

Re: [PR] Fix #305: ManifestEntry partition field schema should be dynamically … [iceberg-go]

2025-02-21 Thread via GitHub
arnaudbriche commented on PR #307: URL: https://github.com/apache/iceberg-go/pull/307#issuecomment-2675759950 Seems like the feature does not exists when the fork repository is owner by an Organization. I just gave you write access to the fork repository. Is that ok ? -- This is an

Re: [PR] Fix #305: ManifestEntry partition field schema should be dynamically … [iceberg-go]

2025-02-21 Thread via GitHub
arnaudbriche commented on PR #307: URL: https://github.com/apache/iceberg-go/pull/307#issuecomment-2675751583 @zeroshade I cannot find the "Allow edits from maintainers" button. Can you point-out where it is supposed to be ? -- This is an automated message from the Apache Git Serv

Re: [PR] feat(catalog/glue): add checkNamespaceExist [iceberg-go]

2025-02-21 Thread via GitHub
zeroshade merged PR #312: URL: https://github.com/apache/iceberg-go/pull/312 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Fix #305: ManifestEntry partition field schema should be dynamically … [iceberg-go]

2025-02-21 Thread via GitHub
zeroshade commented on PR #307: URL: https://github.com/apache/iceberg-go/pull/307#issuecomment-2675721589 @arnaudbriche would it be easier for you to enable maintainers to push to this branch and update or should I just open a new PR with the changes? -- This is an automated message from

Re: [PR] Spec: Add implementation note on `current-snapshot-id` [iceberg]

2025-02-21 Thread via GitHub
emkornfield commented on code in PR #12334: URL: https://github.com/apache/iceberg/pull/12334#discussion_r1966257174 ## format/spec.md: ## @@ -1754,6 +1754,14 @@ Snapshot summary can include metrics fields to track numeric stats of the snapsh | **`engine-name`**| "spar

Re: [PR] Add Variant custom logical type for Avro [iceberg]

2025-02-21 Thread via GitHub
rdblue commented on code in PR #12238: URL: https://github.com/apache/iceberg/pull/12238#discussion_r1966252517 ## core/src/main/java/org/apache/iceberg/avro/PruneColumns.java: ## @@ -92,7 +93,8 @@ public Schema record(Schema record, List names, List fields) { hasCha

Re: [PR] Variant: Move interfaces and serialized implementations to API [iceberg]

2025-02-21 Thread via GitHub
rdblue commented on code in PR #12374: URL: https://github.com/apache/iceberg/pull/12374#discussion_r1966203879 ## api/src/test/java/org/apache/iceberg/variants/TestSerializedMetadata.java: ## @@ -44,7 +44,7 @@ public void testEmptyVariantMetadata() { @Test public void t

Re: [PR] Remove top-level import of `pyarrow` [iceberg-python]

2025-02-21 Thread via GitHub
kevinjqliu merged PR #1703: URL: https://github.com/apache/iceberg-python/pull/1703 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

Re: [PR] API, Core: Update inclusive metrics evaluator for extract and transforms [iceberg]

2025-02-21 Thread via GitHub
rdblue commented on PR #12311: URL: https://github.com/apache/iceberg/pull/12311#issuecomment-2675600178 This now relies on #12374 to move the serialized variant classes into API so that `InclusiveMetricsEvaluator` can use them for deserializing bounds. -- This is an automated message fro

Re: [PR] Spec: Allow Equality Deletes with Row Lineage and Define Behavior [iceberg]

2025-02-21 Thread via GitHub
aokolnychyi commented on code in PR #12230: URL: https://github.com/apache/iceberg/pull/12230#discussion_r1966211402 ## format/spec.md: ## @@ -392,8 +392,7 @@ In v3 and later, an Iceberg table can track row lineage fields for all newly cre These fields are assigned and updat

Re: [PR] Use a join for upsert deduplication [iceberg-python]

2025-02-21 Thread via GitHub
Fokko commented on code in PR #1685: URL: https://github.com/apache/iceberg-python/pull/1685#discussion_r1965766188 ## pyiceberg/table/upsert_util.py: ## @@ -53,42 +53,18 @@ def get_rows_to_update(source_table: pa.Table, target_table: pa.Table, join_cols """ all_colum

Re: [PR] Remove top-level import of `pyarrow` [iceberg-python]

2025-02-21 Thread via GitHub
Fokko commented on PR #1703: URL: https://github.com/apache/iceberg-python/pull/1703#issuecomment-2675560289 @kevinjqliu Not surprised, `pa` is only available for the type checks: ```python if TYPE_CHECKING: ... import pyarrow as pa ``` -- This is an automated messag

Re: [PR] Remove top-level import of `pyarrow` [iceberg-python]

2025-02-21 Thread via GitHub
kevinjqliu commented on PR #1703: URL: https://github.com/apache/iceberg-python/pull/1703#issuecomment-2675557580 There are a couple places we need to add quotes around https://grep.app/search?f.path=pyiceberg%2Ftable%2F&f.repo=apache%2Ficeberg-python&f.repo.pattern=iceberg-python®exp=tr

Re: [PR] WIP: Interface based DataFile reader and writer API [iceberg]

2025-02-21 Thread via GitHub
danielcweeks commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r1966087640 ## core/src/main/java/org/apache/iceberg/avro/Avro.java: ## @@ -786,4 +831,51 @@ public AvroIterable build() { public static long rowCount(InputFile file) {

Re: [PR] [1.7.x] Fix `{LICENSE,NOTICE}` for Spark Runtime [iceberg]

2025-02-21 Thread via GitHub
Fokko merged PR #12355: URL: https://github.com/apache/iceberg/pull/12355 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Remove top-level import of `pyarrow` [iceberg-python]

2025-02-21 Thread via GitHub
Fokko commented on code in PR #1703: URL: https://github.com/apache/iceberg-python/pull/1703#discussion_r1966170554 ## pyiceberg/table/__init__.py: ## @@ -1150,6 +1149,7 @@ def upsert( Returns: An UpsertResult class (contains details of rows updated and in

Re: [PR] [test] Run partition transform tests for all transforms [iceberg-python]

2025-02-21 Thread via GitHub
kevinjqliu commented on PR #1592: URL: https://github.com/apache/iceberg-python/pull/1592#issuecomment-2675539874 nightly build doing its job :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] Remove top-level import of `pyarrow` [iceberg-python]

2025-02-21 Thread via GitHub
kevinjqliu commented on PR #1703: URL: https://github.com/apache/iceberg-python/pull/1703#issuecomment-2675539012 I also removed the previous RC tag ``` git push --delete apache pyiceberg-0.9.0rc1 git tag -d pyiceberg-0.9.0rc1 ``` -- This is an automated message from the Apache

Re: [PR] [1.7.x] Fix `{LICENSE,NOTICE}` for Spark Runtime [iceberg]

2025-02-21 Thread via GitHub
Fokko commented on PR #12355: URL: https://github.com/apache/iceberg/pull/12355#issuecomment-2675536769 Thanks @jbonofre, @nastra and @manuzhang 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Remove top-level import of `pyarrow` [iceberg-python]

2025-02-21 Thread via GitHub
kevinjqliu commented on PR #1703: URL: https://github.com/apache/iceberg-python/pull/1703#issuecomment-2675536253 I opened #1704 to figure out a long term solution -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[I] [bug] figure out how to prevent accidentally importing extra dependency [iceberg-python]

2025-02-21 Thread via GitHub
kevinjqliu opened a new issue, #1704: URL: https://github.com/apache/iceberg-python/issues/1704 ### Apache Iceberg version None ### Please describe the bug 🐞 See #1703 Sometimes we accidentally add `pyarrow` to the top-level import which cause tests to fail when `pya

[PR] Remove top-level import of `pyarrow` [iceberg-python]

2025-02-21 Thread via GitHub
kevinjqliu opened a new pull request, #1703: URL: https://github.com/apache/iceberg-python/pull/1703 The [release candidate artifact build environment](https://github.com/apache/iceberg-python/blob/a58f099aa501f6fd4345a331295d81fe0133554f/.github/workflows/pypi-build-artifacts.yml#L72-L74)

Re: [PR] [test] Run partition transform tests for all transforms [iceberg-python]

2025-02-21 Thread via GitHub
Fokko commented on PR #1592: URL: https://github.com/apache/iceberg-python/pull/1592#issuecomment-2675526454 Yes, looks like it. Let me submit a patch, thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [PR] Variant: Move interfaces and serialized implementations to API [iceberg]

2025-02-21 Thread via GitHub
aihuaxu commented on code in PR #12374: URL: https://github.com/apache/iceberg/pull/12374#discussion_r1966134331 ## api/src/test/java/org/apache/iceberg/variants/TestSerializedMetadata.java: ## @@ -44,7 +44,7 @@ public void testEmptyVariantMetadata() { @Test public void

Re: [PR] [1.7.x] Fix `{LICENSE,NOTICE}` for GCP Bundle [iceberg]

2025-02-21 Thread via GitHub
Fokko merged PR #12359: URL: https://github.com/apache/iceberg/pull/12359 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] WIP: Interface based DataFile reader and writer API [iceberg]

2025-02-21 Thread via GitHub
danielcweeks commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r1966087640 ## core/src/main/java/org/apache/iceberg/avro/Avro.java: ## @@ -786,4 +831,51 @@ public AvroIterable build() { public static long rowCount(InputFile file) {

Re: [PR] [1.7.x] Fix `{LICENSE,NOTICE}` for Kafka Connect Runtime [iceberg]

2025-02-21 Thread via GitHub
Fokko merged PR #12353: URL: https://github.com/apache/iceberg/pull/12353 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Parquet: Implement Variant writers [iceberg]

2025-02-21 Thread via GitHub
rdblue closed pull request #12323: Parquet: Implement Variant writers URL: https://github.com/apache/iceberg/pull/12323 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

Re: [PR] Build: remove Hadoop 2 dependency [iceberg]

2025-02-21 Thread via GitHub
Kontinuation commented on code in PR #12348: URL: https://github.com/apache/iceberg/pull/12348#discussion_r1966100776 ## spark/v3.4/spark/src/test/java/org/apache/iceberg/spark/source/TestStructuredStreaming.java: ## @@ -118,6 +120,7 @@ public void testStreamingWriteAppendMode()

Re: [PR] Variant: Move interfaces and serialized implementations to API [iceberg]

2025-02-21 Thread via GitHub
rdblue commented on code in PR #12374: URL: https://github.com/apache/iceberg/pull/12374#discussion_r1966110815 ## api/src/main/java/org/apache/iceberg/variants/SerializedMetadata.java: ## @@ -23,15 +23,16 @@ import org.apache.iceberg.relocated.com.google.common.annotations.Vi

Re: [PR] Build: remove Hadoop 2 dependency [iceberg]

2025-02-21 Thread via GitHub
Kontinuation commented on code in PR #12348: URL: https://github.com/apache/iceberg/pull/12348#discussion_r1966100776 ## spark/v3.4/spark/src/test/java/org/apache/iceberg/spark/source/TestStructuredStreaming.java: ## @@ -118,6 +120,7 @@ public void testStreamingWriteAppendMode()

Re: [PR] Variant: Move interfaces and serialized implementations to API [iceberg]

2025-02-21 Thread via GitHub
rdblue commented on code in PR #12374: URL: https://github.com/apache/iceberg/pull/12374#discussion_r1966111525 ## api/src/main/java/org/apache/iceberg/variants/VariantValue.java: ## @@ -59,4 +59,21 @@ default VariantObject asObject() { default VariantArray asArray() { t

Re: [PR] Variant: Move interfaces and serialized implementations to API [iceberg]

2025-02-21 Thread via GitHub
rdblue commented on code in PR #12374: URL: https://github.com/apache/iceberg/pull/12374#discussion_r1966109918 ## api/src/main/java/org/apache/iceberg/variants/SerializedObject.java: ## @@ -133,8 +132,8 @@ public boolean hasNext() { } @Override -

Re: [PR] Variant: Move interfaces and serialized implementations to API [iceberg]

2025-02-21 Thread via GitHub
rdblue commented on code in PR #12374: URL: https://github.com/apache/iceberg/pull/12374#discussion_r1966109369 ## api/src/main/java/org/apache/iceberg/variants/Variant.java: ## @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contr

Re: [PR] Variant: Move interfaces and serialized implementations to API [iceberg]

2025-02-21 Thread via GitHub
rdblue commented on code in PR #12374: URL: https://github.com/apache/iceberg/pull/12374#discussion_r1966107634 ## api/src/main/java/org/apache/iceberg/variants/Serialized.java: ## @@ -18,11 +18,8 @@ */ package org.apache.iceberg.variants; -/** A variant metadata and value

[PR] Variant: Move interfaces and serialized implementations to API [iceberg]

2025-02-21 Thread via GitHub
rdblue opened a new pull request, #12374: URL: https://github.com/apache/iceberg/pull/12374 This has been part of other PRs, but because the `Serialized*` classes are moving it is getting big enough to be a separate PR. This moves the Variant interfaces from core to API and also moves

Re: [PR] Build: remove Hadoop 2 dependency [iceberg]

2025-02-21 Thread via GitHub
Kontinuation commented on code in PR #12348: URL: https://github.com/apache/iceberg/pull/12348#discussion_r1966100776 ## spark/v3.4/spark/src/test/java/org/apache/iceberg/spark/source/TestStructuredStreaming.java: ## @@ -118,6 +120,7 @@ public void testStreamingWriteAppendMode()

Re: [PR] [test] Run partition transform tests for all transforms [iceberg-python]

2025-02-21 Thread via GitHub
kevinjqliu commented on PR #1592: URL: https://github.com/apache/iceberg-python/pull/1592#issuecomment-2675436701 CI failed https://github.com/apache/iceberg-python/actions/runs/13464196511/job/37626276454?pr=1592 ``` === FAILURES ==

Re: [PR] Build: remove Hadoop 2 dependency [iceberg]

2025-02-21 Thread via GitHub
Kontinuation commented on PR #12348: URL: https://github.com/apache/iceberg/pull/12348#issuecomment-2675434683 > The title sounds a bit strange to me. How about `Build: remove Hadoop 2 dependency`? Renamed the title as requested. -- This is an automated message from the Apache Git

Re: [PR] [test] Run partition transform tests for all transforms [iceberg-python]

2025-02-21 Thread via GitHub
kevinjqliu commented on PR #1592: URL: https://github.com/apache/iceberg-python/pull/1592#issuecomment-2675433509 CI failed https://github.com/apache/iceberg-python/actions/runs/13464196511/job/37626276454?pr=1592 ``` === FAILURES ==

Re: [PR] WIP: Interface based DataFile reader and writer API [iceberg]

2025-02-21 Thread via GitHub
danielcweeks commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r1966102494 ## core/src/main/java/org/apache/iceberg/io/datafile/DataFileServiceRegistry.java: ## @@ -0,0 +1,420 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] Build: remove Hadoop 2 dependency [iceberg]

2025-02-21 Thread via GitHub
Kontinuation commented on code in PR #12348: URL: https://github.com/apache/iceberg/pull/12348#discussion_r1966100776 ## spark/v3.4/spark/src/test/java/org/apache/iceberg/spark/source/TestStructuredStreaming.java: ## @@ -118,6 +120,7 @@ public void testStreamingWriteAppendMode()

Re: [PR] [docs] update new iceberg catalog link [iceberg-python]

2025-02-21 Thread via GitHub
Fokko merged PR #1701: URL: https://github.com/apache/iceberg-python/pull/1701 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] [1.7.x] Fix `{LICENSE,NOTICE}` for AWS Bundle [iceberg]

2025-02-21 Thread via GitHub
Fokko commented on PR #12360: URL: https://github.com/apache/iceberg/pull/12360#issuecomment-2675328111 Thanks for the review @jbonofre and @jbonofre 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] WIP: Interface based DataFile reader and writer API [iceberg]

2025-02-21 Thread via GitHub
danielcweeks commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r1966074053 ## core/src/main/java/org/apache/iceberg/io/datafile/AppenderBuilder.java: ## @@ -0,0 +1,86 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] WIP: Interface based DataFile reader and writer API [iceberg]

2025-02-21 Thread via GitHub
danielcweeks commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r1966063531 ## core/src/main/java/org/apache/iceberg/io/datafile/DataWriterBuilder.java: ## @@ -0,0 +1,82 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

Re: [PR] [docs] Update "how to release" doc [iceberg-python]

2025-02-21 Thread via GitHub
kevinjqliu commented on PR #1700: URL: https://github.com/apache/iceberg-python/pull/1700#issuecomment-2675346138 CI errored caused by #1701 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [PR] [fix] nightly pypi build for `pyiceberg_core` [iceberg-rust]

2025-02-21 Thread via GitHub
kevinjqliu commented on PR #983: URL: https://github.com/apache/iceberg-rust/pull/983#issuecomment-2675341559 Thank you @Fokko! I triggered a [run](https://github.com/apache/iceberg-rust/actions/runs/13463335249) and verified the [new version on testpypi](https://test.pypi.org/project/

Re: [PR] Build: remove Hadoop 2 dependency [iceberg]

2025-02-21 Thread via GitHub
Kontinuation commented on code in PR #12348: URL: https://github.com/apache/iceberg/pull/12348#discussion_r1966038780 ## aliyun/src/test/java/org/apache/iceberg/aliyun/oss/mock/AliyunOSSMockLocalStore.java: ## @@ -87,7 +86,7 @@ static String md5sum(InputStream is) throws IOExcep

[PR] [docs] Update "how to release" doc [iceberg-python]

2025-02-21 Thread via GitHub
kevinjqliu opened a new pull request, #1700: URL: https://github.com/apache/iceberg-python/pull/1700 Updating the docs as we learn from the 0.9.0 release. Doc improvements: * Updating `KEYS` file requires PMC access. -- This is an automated message from the Apache Git Servic

Re: [PR] [1.7.x] Fix `{LICENSE,NOTICE}` for Azure Bundle [iceberg]

2025-02-21 Thread via GitHub
Fokko commented on PR #12361: URL: https://github.com/apache/iceberg/pull/12361#issuecomment-2675328696 Thanks for the review @jbonofre and @nastra 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] [1.7.x] Fix `{LICENSE,NOTICE}` for GCP Bundle [iceberg]

2025-02-21 Thread via GitHub
Fokko commented on PR #12359: URL: https://github.com/apache/iceberg/pull/12359#issuecomment-2675327539 Thanks for the review @jbonofre and @jbonofre 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] [1.7.x] Fix `{LICENSE,NOTICE}` for AWS Bundle [iceberg]

2025-02-21 Thread via GitHub
Fokko merged PR #12360: URL: https://github.com/apache/iceberg/pull/12360 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] [1.7.x] Fix `{LICENSE,NOTICE}` for Azure Bundle [iceberg]

2025-02-21 Thread via GitHub
Fokko merged PR #12361: URL: https://github.com/apache/iceberg/pull/12361 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

  1   2   3   >