Re: [PR] feat(timestamp_ns): Implement timestamps with nanosecond precision [iceberg-rust]

sdd commented on PR #542: URL: https://github.com/apache/iceberg-rust/pull/542#issuecomment-2294695447 Looks good to me - thanks for your contribution! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [I] Review new ImmutablesReferenceEquality error-prone check [iceberg]

danielhumanmod commented on issue #10855: URL: https://github.com/apache/iceberg/issues/10855#issuecomment-2294654962 Additionally, I do notice that there are several `DangerousJavaDeserialization` warning in our code, do we have a plan to do some investigation on that? -- This is an aut

Re: [I] Review new ImmutablesReferenceEquality error-prone check [iceberg]

danielhumanmod commented on issue #10855: URL: https://github.com/apache/iceberg/issues/10855#issuecomment-2294653351 Hi @findepi , based on my investigation, the current status regarding the usage of `ImmutablesReferenceEquality` is as follows: Currently, `ImmutablesReferenceEquality

Re: [I] FlinkSchemaUtil.toSchema should return Schema or ResolvedSchema instead of deprecated TableSchema [iceberg]

pvary commented on issue #10950: URL: https://github.com/apache/iceberg/issues/10950#issuecomment-2294630782 Hi @alexmorley, I think I did try to solve this myself some time ago, and found some non-trivial blockers. Sadly, it was so long time ago, that I don't remember what was the issue

Re: [I] Support relative paths in Table Metadata [iceberg]

akizminet commented on issue #1617: URL: https://github.com/apache/iceberg/issues/1617#issuecomment-2294629384 We also face the same issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] feat: SQL Catalog - namespaces [iceberg-rust]

liurenjie1024 commented on code in PR #534: URL: https://github.com/apache/iceberg-rust/pull/534#discussion_r1720598113 ## crates/catalog/sql/src/catalog.rs: ## @@ -167,43 +177,335 @@ impl SqlCatalog { .await .map_err(from_sqlx_error) } + +///

Re: [PR] arrow/schema:new func `convert_schema` for `ArrowSchemaConverter` [iceberg-rust]

liurenjie1024 commented on PR #539: URL: https://github.com/apache/iceberg-rust/pull/539#issuecomment-2294593295 Hi, @AndreMouche Thanks for your contribution, I have some concerns for this pr: > for performance: remove unnecessary match operations that could have a negative impact o

Re: [PR] Implement Kerberos authentication support for Hive Catalog [iceberg-python]

yothinix commented on PR #766: URL: https://github.com/apache/iceberg-python/pull/766#issuecomment-2294533715 Hi @kevinjqliu Thank you for look into this PR, I just rebased the PR branch to latest main branch as requested. -- This is an automated message from the Apache Git Service. To re

Re: [PR] DOC: Strawman proposal for PR merging [iceberg]

wmoustafa commented on code in PR #10780: URL: https://github.com/apache/iceberg/pull/10780#discussion_r1720492906 ## site/docs/contribute.md: ## @@ -45,6 +45,16 @@ The Iceberg community prefers to receive contributions as [Github pull requests] * If a PR is related to an issu

Re: [PR] Flink: Refactoring StreamingReaderOperator to read data nonblocking [iceberg]

github-actions[bot] commented on PR #5493: URL: https://github.com/apache/iceberg/pull/5493#issuecomment-2294473785 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] Parquet: Set parquet bloom filter config with compatible column name [iceberg]

github-actions[bot] commented on PR #5435: URL: https://github.com/apache/iceberg/pull/5435#issuecomment-2294473766 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] API, Spark: Generate symlink manifest action [iceberg]

github-actions[bot] commented on PR #5398: URL: https://github.com/apache/iceberg/pull/5398#issuecomment-2294473743 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] Build: Do not let Iceberg build fail with `-DscalaVersion=2.13` [iceberg]

github-actions[bot] commented on PR #5361: URL: https://github.com/apache/iceberg/pull/5361#issuecomment-2294473656 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] Substitute the method of PropertyUtil#propertyAsLong for IcebergSource#propertyAsLong [iceberg]

github-actions[bot] commented on PR #5346: URL: https://github.com/apache/iceberg/pull/5346#issuecomment-2294473636 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] Support force option on RegisterTable procedure [iceberg]

github-actions[bot] commented on PR #5327: URL: https://github.com/apache/iceberg/pull/5327#issuecomment-2294473598 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] Core : Catalog Tables Migration API [iceberg]

github-actions[bot] commented on PR #5297: URL: https://github.com/apache/iceberg/pull/5297#issuecomment-2294473564 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] Spark: Spark SQL read from Snapshot ref [iceberg]

github-actions[bot] commented on PR #5294: URL: https://github.com/apache/iceberg/pull/5294#issuecomment-2294473546 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] [wip]: Decouple Hadoop Configuration from FlinkCategoryFactory [iceberg]

github-actions[bot] commented on PR #4768: URL: https://github.com/apache/iceberg/pull/4768#issuecomment-2294473422 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] API: Add Generate Symlink Manifest API [iceberg]

github-actions[bot] commented on PR #5516: URL: https://github.com/apache/iceberg/pull/5516#issuecomment-2294473827 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] API - Do not validate input length to String Truncate Transform on every call [iceberg]

github-actions[bot] commented on PR #5496: URL: https://github.com/apache/iceberg/pull/5496#issuecomment-2294473811 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] Incremental pull ignore overwrite commit if table property is set [iceberg]

github-actions[bot] commented on PR #5533: URL: https://github.com/apache/iceberg/pull/5533#issuecomment-2294473858 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [I] read iceberg table by flink timeout [iceberg]

github-actions[bot] commented on issue #5388: URL: https://github.com/apache/iceberg/issues/5388#issuecomment-2294473724 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Chores: using bulk delete if it's possible [iceberg]

github-actions[bot] commented on PR #5375: URL: https://github.com/apache/iceberg/pull/5375#issuecomment-2294473682 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] Build: Do not specify scala-library version explicitly in Spark 3.2 and 3.3 build scripts [iceberg]

github-actions[bot] commented on PR #5319: URL: https://github.com/apache/iceberg/pull/5319#issuecomment-2294473585 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] Docs: Add doc of the upsert option [iceberg]

github-actions[bot] commented on PR #5334: URL: https://github.com/apache/iceberg/pull/5334#issuecomment-2294473613 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] [wip]: Decouple Hadoop Configuration from FlinkCategoryFactory [iceberg]

github-actions[bot] closed pull request #4768: [wip]: Decouple Hadoop Configuration from FlinkCategoryFactory URL: https://github.com/apache/iceberg/pull/4768 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Make sort strategy an idempotent action [iceberg]

github-actions[bot] closed pull request #4759: Make sort strategy an idempotent action URL: https://github.com/apache/iceberg/pull/4759 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Make sort strategy an idempotent action [iceberg]

github-actions[bot] commented on PR #4759: URL: https://github.com/apache/iceberg/pull/4759#issuecomment-2294473406 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Docs: To delete a table and its underlying data, use the `PURGE` keyword [iceberg]

github-actions[bot] commented on PR #4746: URL: https://github.com/apache/iceberg/pull/4746#issuecomment-2294473381 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] API: Add default value API [iceberg]

github-actions[bot] closed pull request #4732: API: Add default value API URL: https://github.com/apache/iceberg/pull/4732 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [PR] Docs: add types supported by truncate [iceberg]

github-actions[bot] commented on PR #4725: URL: https://github.com/apache/iceberg/pull/4725#issuecomment-2294473329 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] API: Add default value API [iceberg]

github-actions[bot] commented on PR #4732: URL: https://github.com/apache/iceberg/pull/4732#issuecomment-2294473352 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] API: Introduce endWiths Predicate. [iceberg]

github-actions[bot] commented on PR #4721: URL: https://github.com/apache/iceberg/pull/4721#issuecomment-2294473313 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Docs: add types supported by truncate [iceberg]

github-actions[bot] closed pull request #4725: Docs: add types supported by truncate URL: https://github.com/apache/iceberg/pull/4725 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] [AWS]Add AwsKmsClient for table encryption KMS client implementation [iceberg]

github-actions[bot] commented on PR #4714: URL: https://github.com/apache/iceberg/pull/4714#issuecomment-2294473294 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] WIP: Remove redundant sorts from copy on write deletes [iceberg]

github-actions[bot] commented on PR #4692: URL: https://github.com/apache/iceberg/pull/4692#issuecomment-2294473255 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] WIP: Remove redundant sorts from copy on write deletes [iceberg]

github-actions[bot] closed pull request #4692: WIP: Remove redundant sorts from copy on write deletes URL: https://github.com/apache/iceberg/pull/4692 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] WIP: Improve performance of expire snapshot by not double-scanning non-expired manifests [iceberg]

github-actions[bot] closed pull request #4736: WIP: Improve performance of expire snapshot by not double-scanning non-expired manifests URL: https://github.com/apache/iceberg/pull/4736 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [PR] Docs: To delete a table and its underlying data, use the `PURGE` keyword [iceberg]

github-actions[bot] closed pull request #4746: Docs: To delete a table and its underlying data, use the `PURGE` keyword URL: https://github.com/apache/iceberg/pull/4746 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] WIP: Improve performance of expire snapshot by not double-scanning non-expired manifests [iceberg]

github-actions[bot] commented on PR #4736: URL: https://github.com/apache/iceberg/pull/4736#issuecomment-2294473372 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Docs:Modify the flink version from 0.11.1 to 0.14.4 and fix several errors. [iceberg]

github-actions[bot] commented on PR #4665: URL: https://github.com/apache/iceberg/pull/4665#issuecomment-2294473238 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] API: Introduce endWiths Predicate. [iceberg]

github-actions[bot] closed pull request #4721: API: Introduce endWiths Predicate. URL: https://github.com/apache/iceberg/pull/4721 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] [AWS]Add AwsKmsClient for table encryption KMS client implementation [iceberg]

github-actions[bot] closed pull request #4714: [AWS]Add AwsKmsClient for table encryption KMS client implementation URL: https://github.com/apache/iceberg/pull/4714 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Core: Make snapshot summary return default values [iceberg]

github-actions[bot] commented on PR #4660: URL: https://github.com/apache/iceberg/pull/4660#issuecomment-2294473225 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] AWS: Add Cache for LakeFormation Credentials [iceberg]

github-actions[bot] commented on PR #4695: URL: https://github.com/apache/iceberg/pull/4695#issuecomment-2294473271 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] AWS: Add Cache for LakeFormation Credentials [iceberg]

github-actions[bot] closed pull request #4695: AWS: Add Cache for LakeFormation Credentials URL: https://github.com/apache/iceberg/pull/4695 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] Docs:Modify the flink version from 0.11.1 to 0.14.4 and fix several errors. [iceberg]

github-actions[bot] closed pull request #4665: Docs:Modify the flink version from 0.11.1 to 0.14.4 and fix several errors. URL: https://github.com/apache/iceberg/pull/4665 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [PR] Core: Make snapshot summary return default values [iceberg]

github-actions[bot] closed pull request #4660: Core: Make snapshot summary return default values URL: https://github.com/apache/iceberg/pull/4660 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Flink: Support drop non-empty namespace with CASCADE [iceberg]

github-actions[bot] closed pull request #4658: Flink: Support drop non-empty namespace with CASCADE URL: https://github.com/apache/iceberg/pull/4658 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] Flink: Support drop non-empty namespace with CASCADE [iceberg]

github-actions[bot] commented on PR #4658: URL: https://github.com/apache/iceberg/pull/4658#issuecomment-2294473214 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Spec: Minor modifications for v3 [iceberg]

emkornfield commented on code in PR #10948: URL: https://github.com/apache/iceberg/pull/10948#discussion_r1720475417 ## format/spec.md: ## @@ -44,6 +44,15 @@ The primary change in version 2 adds delete files to encode rows that are delete In addition to row-level deletes, ve

Re: [PR] Spec: Minor modifications for v3 [iceberg]

emkornfield commented on code in PR #10948: URL: https://github.com/apache/iceberg/pull/10948#discussion_r1720474856 ## format/spec.md: ## @@ -113,9 +122,9 @@ Tables do not require random-access writes. Once written, data and metadata file Tables do not require rename, except

Re: [PR] Support changelog scan for table with delete files [iceberg]

wypoon commented on PR #10935: URL: https://github.com/apache/iceberg/pull/10935#issuecomment-2294451831 @aokolnychyi @flyrain @stevenzwu @szehon-ho can you please review this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

Re: [PR] OpenAPI: Clarify in REST spec that server implementations of commit endpoints must fail with 400 for unknown requirements/updates [iceberg]

amogh-jahagirdar merged PR #10848: URL: https://github.com/apache/iceberg/pull/10848 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] OpenAPI: Clarify in REST spec that server implementations of commit endpoints must fail with 400 for unknown requirements/updates [iceberg]

amogh-jahagirdar commented on PR #10848: URL: https://github.com/apache/iceberg/pull/10848#issuecomment-2294441794 Thanks everyone for reviewing, the vote has passed https://lists.apache.org/thread/99lo7stnprchjzosjcq9k3mns1mq8fwc and I will go ahead and merge. -- This is an automated

Re: [PR] Spec: Minor modifications for v3 [iceberg]

rdblue commented on code in PR #10948: URL: https://github.com/apache/iceberg/pull/10948#discussion_r1720451537 ## format/spec.md: ## @@ -1308,7 +1321,7 @@ Default values are added to struct fields in v3. Types `timestamp_ns` and `timestamptz_ns` are added in v3. -All reade

Re: [PR] Support changelog scan for table with delete files [iceberg]

wypoon commented on PR #10935: URL: https://github.com/apache/iceberg/pull/10935#issuecomment-2294429604 @manuzhang thank you for pointing me to your PR, https://github.com/apache/iceberg/pull/9888. I have tried testing it and left comments on your PR. -- This is an automated message fro

Re: [PR] Spec: Minor modifications for v3 [iceberg]

rdblue commented on code in PR #10948: URL: https://github.com/apache/iceberg/pull/10948#discussion_r1720345226 ## format/spec.md: ## @@ -193,16 +204,14 @@ Supported primitive types are defined in the table below. Primitive types added Notes: -1. Decimal scale is fixed and

Re: [PR] Core: Support IncrementalChangelogScan with deletes [iceberg]

wypoon commented on code in PR #9888: URL: https://github.com/apache/iceberg/pull/9888#discussion_r1720445018 ## core/src/main/java/org/apache/iceberg/BaseIncrementalChangelogScan.java: ## @@ -134,50 +138,81 @@ private static Map computeSnapshotOrdinals(Deque snapsh } p

Re: [PR] Core: V3 Metadata Upgrade Validation and Testing [iceberg]

RussellSpitzer commented on PR #10861: URL: https://github.com/apache/iceberg/pull/10861#issuecomment-2294428381 Thanks @leangjonathan ! Also thanks to @amogh-jahagirdar and @nastra for reviewing! We are one step closer to V3 :) -- This is an automated message from the Apache Git Service.

Re: [PR] Core: V3 Metadata Upgrade Validation and Testing [iceberg]

RussellSpitzer merged PR #10861: URL: https://github.com/apache/iceberg/pull/10861 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Spec: Minor modifications for v3 [iceberg]

rdblue commented on code in PR #10948: URL: https://github.com/apache/iceberg/pull/10948#discussion_r1720443701 ## format/spec.md: ## @@ -44,6 +44,15 @@ The primary change in version 2 adds delete files to encode rows that are delete In addition to row-level deletes, version

Re: [PR] Core: Support IncrementalChangelogScan with deletes [iceberg]

wypoon commented on PR #9888: URL: https://github.com/apache/iceberg/pull/9888#issuecomment-2294424912 You have made changes in `BaseIncrementalChangelogScan` that produces `BaseDeletedRowsScanTask` in certain cases. However, you have not made changes in the Spark `ChangelogRowReader` to ha

[PR] Bump mypy-boto3-glue from 1.34.160 to 1.35.0 [iceberg-python]

dependabot[bot] opened a new pull request, #1070: URL: https://github.com/apache/iceberg-python/pull/1070 Bumps [mypy-boto3-glue](https://github.com/youtype/mypy_boto3_builder) from 1.34.160 to 1.35.0. Commits See full diff in https://github.com/youtype/mypy_boto3_builder/commi

[PR] Bump griffe from 0.49.0 to 1.0.0 [iceberg-python]

dependabot[bot] opened a new pull request, #1069: URL: https://github.com/apache/iceberg-python/pull/1069 Bumps [griffe](https://github.com/mkdocstrings/griffe) from 0.49.0 to 1.0.0. Release notes Sourced from https://github.com/mkdocstrings/griffe/releases";>griffe's releases.

Re: [PR] Spec: Minor modifications for v3 [iceberg]

rdblue commented on code in PR #10948: URL: https://github.com/apache/iceberg/pull/10948#discussion_r1720346551 ## format/spec.md: ## @@ -1308,7 +1321,7 @@ Default values are added to struct fields in v3. Types `timestamp_ns` and `timestamptz_ns` are added in v3. -All reade

Re: [PR] Spec: Minor modifications for v3 [iceberg]

rdblue commented on code in PR #10948: URL: https://github.com/apache/iceberg/pull/10948#discussion_r1720345226 ## format/spec.md: ## @@ -193,16 +204,14 @@ Supported primitive types are defined in the table below. Primitive types added Notes: -1. Decimal scale is fixed and

Re: [PR] Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction [iceberg]

pvary commented on code in PR #10179: URL: https://github.com/apache/iceberg/pull/10179#discussion_r1720335116 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergCommitter.java: ## @@ -0,0 +1,309 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

[PR] chore: update dep (https://rustsec.org/advisories/RUSTSEC-2024-0363) [iceberg-rust]

sdd opened a new pull request, #559: URL: https://github.com/apache/iceberg-rust/pull/559 There is a security advisory out for `sqlx` (https://rustsec.org/advisories/RUSTSEC-2024-0363) that causes our CI to fail. Unfortunately the suggested mitigation of updating to 0.8.1 is not as ye

[PR] Table Scan: Add Row Group Skipping and Row Selection Filtering [iceberg-rust]

sdd opened a new pull request, #558: URL: https://github.com/apache/iceberg-rust/pull/558 This PR introduces two more advanced features that can improve performance when executing table reads on parquet-backed tables when using a filter predicate: "row group filtering" and "row selection sk

Re: [PR] Expose Bucket Transform to Python Binding [iceberg-rust]

sungwy commented on code in PR #556: URL: https://github.com/apache/iceberg-rust/pull/556#discussion_r1720311292 ## bindings/python/pyproject.toml: ## @@ -31,8 +31,30 @@ classifiers = [ "Programming Language :: Python :: 3.12", ] -[project.optional-dependencies] -test = ["

Re: [PR] prevent adding duplicate files [iceberg-python]

amitgilad3 commented on PR #1036: URL: https://github.com/apache/iceberg-python/pull/1036#issuecomment-2294096857 Hey @sungwy + @kevinjqliu , again thanks for all the help and guidance , i went over all the comments and fixed them -- This is an automated message from the Apache Git Serv

Re: [PR] Spark partial limit push down [iceberg]

singhpk234 commented on code in PR #10943: URL: https://github.com/apache/iceberg/pull/10943#discussion_r1720266361 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkScanBuilder.java: ## @@ -407,14 +422,35 @@ public Scan build() { private Scan buildBatc

Re: [PR] prevent adding duplicate files [iceberg-python]

amitgilad3 commented on code in PR #1036: URL: https://github.com/apache/iceberg-python/pull/1036#discussion_r1720268289 ## tests/integration/test_add_files.py: ## @@ -732,3 +732,76 @@ def test_add_files_subset_of_schema(spark: SparkSession, session_catalog: Catalo for col

Re: [PR] try 3.12 [iceberg-python]

Fokko commented on code in PR #1068: URL: https://github.com/apache/iceberg-python/pull/1068#discussion_r1720255476 ## .markdownlint.yaml: ## @@ -0,0 +1,22 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE

[PR] feat(manifest): fix partition data map [iceberg-go]

zeroshade opened a new pull request, #124: URL: https://github.com/apache/iceberg-go/pull/124 Split out from #118 Fixup the partition data map that we get from the manifest entry's datafile information and preserve the field name -> field-id mapping in partition data for a datafile.

Re: [PR] feat(manifest): fix partition data map [iceberg-go]

zeroshade commented on PR #124: URL: https://github.com/apache/iceberg-go/pull/124#issuecomment-2293997238 CC @Fokko @nastra -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Table Scan Performance Tests [iceberg-rust]

sdd commented on code in PR #497: URL: https://github.com/apache/iceberg-rust/pull/497#discussion_r1720200715 ## justfile: ## @@ -0,0 +1,53 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distrib

Re: [PR] Spec: Minor modifications for v3 [iceberg]

rdblue commented on code in PR #10948: URL: https://github.com/apache/iceberg/pull/10948#discussion_r1720183537 ## format/spec.md: ## @@ -113,9 +122,9 @@ Tables do not require random-access writes. Once written, data and metadata file Tables do not require rename, except for t

Re: [PR] Spec: Minor modifications for v3 [iceberg]

rdblue commented on code in PR #10948: URL: https://github.com/apache/iceberg/pull/10948#discussion_r1720181535 ## format/spec.md: ## @@ -44,6 +44,15 @@ The primary change in version 2 adds delete files to encode rows that are delete In addition to row-level deletes, version

Re: [PR] Spec: Minor modifications for v3 [iceberg]

rdblue commented on code in PR #10948: URL: https://github.com/apache/iceberg/pull/10948#discussion_r1720179715 ## format/spec.md: ## @@ -44,6 +44,15 @@ The primary change in version 2 adds delete files to encode rows that are delete In addition to row-level deletes, version

Re: [PR] Spec: Minor modifications for v3 [iceberg]

rdblue commented on code in PR #10948: URL: https://github.com/apache/iceberg/pull/10948#discussion_r1720178070 ## format/spec.md: ## @@ -44,6 +44,15 @@ The primary change in version 2 adds delete files to encode rows that are delete In addition to row-level deletes, version

Re: [PR] Spec: Minor modifications for v3 [iceberg]

rdblue commented on code in PR #10948: URL: https://github.com/apache/iceberg/pull/10948#discussion_r1720176739 ## format/spec.md: ## @@ -44,6 +44,15 @@ The primary change in version 2 adds delete files to encode rows that are delete In addition to row-level deletes, version

Re: [PR] Spark partial limit push down [iceberg]

huaxingao commented on code in PR #10943: URL: https://github.com/apache/iceberg/pull/10943#discussion_r1720160710 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkScanBuilder.java: ## @@ -407,14 +422,35 @@ public Scan build() { private Scan buildBatch

Re: [PR] Spark partial limit push down [iceberg]

huaxingao commented on code in PR #10943: URL: https://github.com/apache/iceberg/pull/10943#discussion_r1720157880 ## arrow/src/main/java/org/apache/iceberg/arrow/vectorized/parquet/VectorizedColumnIterator.java: ## @@ -69,12 +69,20 @@ public boolean producesDictionaryEncodedVec

Re: [PR] Spark partial limit push down [iceberg]

singhpk234 commented on code in PR #10943: URL: https://github.com/apache/iceberg/pull/10943#discussion_r1720144903 ## arrow/src/main/java/org/apache/iceberg/arrow/vectorized/parquet/VectorizedColumnIterator.java: ## @@ -69,12 +69,20 @@ public boolean producesDictionaryEncodedVe

Re: [PR] Expose Bucket Transform to Python Binding [iceberg-rust]

sungwy commented on code in PR #556: URL: https://github.com/apache/iceberg-rust/pull/556#discussion_r1720136976 ## bindings/python/pyproject.toml: ## @@ -31,8 +31,30 @@ classifiers = [ "Programming Language :: Python :: 3.12", ] -[project.optional-dependencies] -test = ["

Re: [PR] Expose Bucket Transform to Python Binding [iceberg-rust]

sungwy commented on code in PR #556: URL: https://github.com/apache/iceberg-rust/pull/556#discussion_r1720136976 ## bindings/python/pyproject.toml: ## @@ -31,8 +31,30 @@ classifiers = [ "Programming Language :: Python :: 3.12", ] -[project.optional-dependencies] -test = ["

Re: [PR] Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction [iceberg]

rodmeneses commented on code in PR #10179: URL: https://github.com/apache/iceberg/pull/10179#discussion_r1720132474 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -0,0 +1,752 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

Re: [PR] Spark partial limit push down [iceberg]

singhpk234 commented on code in PR #10943: URL: https://github.com/apache/iceberg/pull/10943#discussion_r1720122775 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkScanBuilder.java: ## @@ -407,14 +422,35 @@ public Scan build() { private Scan buildBatc

Re: [PR] Core: V3 Metadata Upgrade Validation and Testing [iceberg]

leangjonathan commented on code in PR #10861: URL: https://github.com/apache/iceberg/pull/10861#discussion_r1720108098 ## core/src/test/java/org/apache/iceberg/TestFormatVersions.java: ## @@ -23,56 +23,95 @@ import java.util.Arrays; import java.util.List; +import java.util.s

Re: [PR] Spark partial limit push down [iceberg]

huaxingao commented on code in PR #10943: URL: https://github.com/apache/iceberg/pull/10943#discussion_r1720105551 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkScanBuilder.java: ## @@ -407,14 +422,35 @@ public Scan build() { private Scan buildBatch

Re: [PR] Core: fix NPE with HadoopFileIO with Hadoop conf is not set [iceberg]

stevenzwu commented on code in PR #10926: URL: https://github.com/apache/iceberg/pull/10926#discussion_r1720058500 ## core/src/main/java/org/apache/iceberg/hadoop/HadoopFileIO.java: ## @@ -63,7 +63,11 @@ public class HadoopFileIO implements HadoopConfigurable, DelegateFileIO {

Re: [PR] Spark partial limit push down [iceberg]

huaxingao commented on code in PR #10943: URL: https://github.com/apache/iceberg/pull/10943#discussion_r1720102633 ## arrow/src/main/java/org/apache/iceberg/arrow/vectorized/parquet/VectorizedColumnIterator.java: ## @@ -69,12 +69,20 @@ public boolean producesDictionaryEncodedVec

[PR] Spark 3.5: Verify Iceberg catalog in TestCreateActions [iceberg]

manuzhang opened a new pull request, #10952: URL: https://github.com/apache/iceberg/pull/10952 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] Spark partial limit push down [iceberg]

huaxingao commented on code in PR #10943: URL: https://github.com/apache/iceberg/pull/10943#discussion_r1720099765 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkSQLProperties.java: ## @@ -45,6 +45,10 @@ private SparkSQLProperties() {} "spark.sql.iceberg.

Re: [PR] Spark partial limit push down [iceberg]

huaxingao commented on code in PR #10943: URL: https://github.com/apache/iceberg/pull/10943#discussion_r1720099440 ## parquet/src/main/java/org/apache/iceberg/parquet/VectorizedParquetReader.java: ## @@ -141,8 +148,15 @@ public T next() { advance(); } + lo

Re: [PR] Spark partial limit push down [iceberg]

huaxingao commented on code in PR #10943: URL: https://github.com/apache/iceberg/pull/10943#discussion_r1720099159 ## parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java: ## @@ -1151,6 +1152,11 @@ public ReadBuilder withAADPrefix(ByteBuffer aadPrefix) { return t

Re: [PR] test: refactor datafusion test with memory catalog [iceberg-rust]

Xuanwo merged PR #557: URL: https://github.com/apache/iceberg-rust/pull/557 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] Flink: put everything together for range distribution in Flink sink [iceberg]

stevenzwu commented on code in PR #10859: URL: https://github.com/apache/iceberg/pull/10859#discussion_r1720073124 ## docs/docs/flink-configuration.md: ## @@ -146,14 +146,54 @@ INSERT INTO tableName /*+ OPTIONS('upsert-enabled'='true') */ ... ``` -| Flink option |

  1   2   >