Re: [I] Flink: Fix flaky TestIcebergSourceFailover > testBoundedWithSavepoint [iceberg]

pvary commented on issue #10671: URL: https://github.com/apache/iceberg/issues/10671#issuecomment-2255071498 @stevenzwu: Created a pull request for disabling the tests: #10802 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[PR] Flink: Disabling flaky test TestIcebergSourceFailover.testBoundedWithSavepoint [iceberg]

pvary opened a new pull request, #10802: URL: https://github.com/apache/iceberg/pull/10802 Disabling flaky tests. #10671 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] Scan does not work as expected [iceberg-rust]

Xuanwo commented on issue #495: URL: https://github.com/apache/iceberg-rust/issues/495#issuecomment-2254952953 I'm guessing https://github.com/apache/iceberg-rust/pull/498 should close this issue. Would you like to verify it? -- This is an automated message from the Apache Git Service. To

Re: [I] Scan does not work as expected [iceberg-rust]

Xuanwo commented on issue #495: URL: https://github.com/apache/iceberg-rust/issues/495#issuecomment-2254928209 Hi, does `remote signing` means `presign` in s3? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [PR] fix: Return error on reader task [iceberg-rust]

Xuanwo merged PR #498: URL: https://github.com/apache/iceberg-rust/pull/498 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [I] flink iceberg may occur duplication when succeed to write datafile and commit but checkpoint fail [iceberg]

maekchi commented on issue #10765: URL: https://github.com/apache/iceberg/issues/10765#issuecomment-2254833226 @pvary > Which version of Flink do you using btw? I use flink version 1.15.4 > What Catalog are you using? Is there any cache, or something which might return wron

[PR] Add Github Actions Screenshots to Release Notes [iceberg-python]

sungwy opened a new pull request, #975: URL: https://github.com/apache/iceberg-python/pull/975 Adding screenshots -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1694390238 ## open-api/rest-catalog-open-api.yaml: ## @@ -3647,6 +3826,151 @@ components: type: integer description: "List of equality field IDs" +De

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1694390238 ## open-api/rest-catalog-open-api.yaml: ## @@ -3647,6 +3826,151 @@ components: type: integer description: "List of equality field IDs" +De

Re: [I] Support commit retries [iceberg-python]

github-actions[bot] closed issue #269: Support commit retries URL: https://github.com/apache/iceberg-python/issues/269 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

Re: [I] Support commit retries [iceberg-python]

github-actions[bot] commented on issue #269: URL: https://github.com/apache/iceberg-python/issues/269#issuecomment-2254722571 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apac

Re: [I] In CDC to Iceberg， why data files is still growing rapidly when finished Expire Snapshots & Compact data files [iceberg]

github-actions[bot] commented on issue #3447: URL: https://github.com/apache/iceberg/issues/3447#issuecomment-2254721543 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] commit snap of merge small files and normal java job has complict [iceberg]

github-actions[bot] commented on issue #3417: URL: https://github.com/apache/iceberg/issues/3417#issuecomment-2254721493 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] about table properties [iceberg]

github-actions[bot] commented on issue #3410: URL: https://github.com/apache/iceberg/issues/3410#issuecomment-2254721464 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Data: Use proper temp file in TestMetricsRowGroupFilterTypes [iceberg]

github-actions[bot] commented on PR #3416: URL: https://github.com/apache/iceberg/pull/3416#issuecomment-2254721473 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] Doc: add a page to explain row-level deletes [iceberg]

github-actions[bot] commented on PR #3432: URL: https://github.com/apache/iceberg/pull/3432#issuecomment-2254721513 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] API: Add function for removing Specs from Metadata.json which are no … [iceberg]

github-actions[bot] commented on PR #3462: URL: https://github.com/apache/iceberg/pull/3462#issuecomment-2254721554 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [I] After merging small files, the number of files is still large [iceberg]

github-actions[bot] commented on issue #3409: URL: https://github.com/apache/iceberg/issues/3409#issuecomment-2254721451 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Add schema validation during parquet file Import [iceberg]

github-actions[bot] commented on PR #3435: URL: https://github.com/apache/iceberg/pull/3435#issuecomment-2254721528 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [I] Outdated object-store-file-layout documentation [iceberg]

github-actions[bot] commented on issue #3401: URL: https://github.com/apache/iceberg/issues/3401#issuecomment-2254721437 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] While I tring to select table1 join table2，if fields of table2 was choosed, error like "java.lang.ArrayIndexOutOfBoundsException: 6" occurred [iceberg]

github-actions[bot] commented on issue #3395: URL: https://github.com/apache/iceberg/issues/3395#issuecomment-2254721416 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] How to get Data file from a query [iceberg]

github-actions[bot] commented on issue #3396: URL: https://github.com/apache/iceberg/issues/3396#issuecomment-2254721426 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] After deleting some parquet in one partition, then read iceberg will throw out error [iceberg]

github-actions[bot] commented on issue #3394: URL: https://github.com/apache/iceberg/issues/3394#issuecomment-2254721402 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Hive: Bug when runing SQL with multiple table join [iceberg]

github-actions[bot] commented on issue #3393: URL: https://github.com/apache/iceberg/issues/3393#issuecomment-2254721385 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Core: enable setting table property as action config option default [iceberg]

github-actions[bot] commented on PR #3387: URL: https://github.com/apache/iceberg/pull/3387#issuecomment-2254721374 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [I] The incremental consumption should keep the order of data [iceberg]

github-actions[bot] commented on issue #3371: URL: https://github.com/apache/iceberg/issues/3371#issuecomment-2254721354 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] The tools that hadoopCatalog and hiveCatalog convert to each other [iceberg]

github-actions[bot] commented on issue #3362: URL: https://github.com/apache/iceberg/issues/3362#issuecomment-2254721345 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Flink: support insert options set equality fields columns [iceberg]

github-actions[bot] closed pull request #3101: Flink: support insert options set equality fields columns URL: https://github.com/apache/iceberg/pull/3101 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Flink: support insert options set equality fields columns [iceberg]

github-actions[bot] commented on PR #3101: URL: https://github.com/apache/iceberg/pull/3101#issuecomment-2254721144 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [I] Scan does not work as expected [iceberg-rust]

ndrluis commented on issue #495: URL: https://github.com/apache/iceberg-rust/issues/495#issuecomment-2254686134 To scan with remote-signing we need to implement this https://github.com/apache/iceberg/blob/main/aws/src/main/java/org/apache/iceberg/aws/s3/signer/S3V4RestSignerClient.jav

[PR] fix: Return error on reader task [iceberg-rust]

ndrluis opened a new pull request, #498: URL: https://github.com/apache/iceberg-rust/pull/498 For some reason that I don't understand, this change makes the scan work correctly in the scenario described in #495. This change was made to solve the 'Access Denied' problem that was being suppre

Re: [I] Scan does not work as expected [iceberg-rust]

ndrluis commented on issue #495: URL: https://github.com/apache/iceberg-rust/issues/495#issuecomment-2254673245 I encountered an 'access denied' problem. The FileIO does not receive the S3 credentials. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [I] Scan does not work as expected [iceberg-rust]

ndrluis commented on issue #495: URL: https://github.com/apache/iceberg-rust/issues/495#issuecomment-2254669702 I found the problem. I don’t know how to solve it, but I will try. The `while let Some(Ok(task)) = tasks.next().await statement` is hiding some errors. In my previous attemp

Re: [I] Geospatial Support [iceberg]

jiayuasu commented on issue #10260: URL: https://github.com/apache/iceberg/issues/10260#issuecomment-2254662914 @dmeaux Great. Can you also explain that what are the OSS libraries available to parse these CRS formats? Ideally, we are looking for options in both C, Java, and Python. I

Re: [I] Scan does not work as expected [iceberg-rust]

ndrluis commented on issue #495: URL: https://github.com/apache/iceberg-rust/issues/495#issuecomment-2254661198 I performed another test using the Tabular catalog, attempting to scan the sandbox warehouse in the examples namespace, specifically targeting the nyc_taxi_yellow table, but it re

Re: [PR] feat: performance testing harness and perf tests for scan file plan [iceberg-rust]

sdd commented on PR #497: URL: https://github.com/apache/iceberg-rust/pull/497#issuecomment-2254661106 Additionally the performance tests in here depend on https://github.com/apache/iceberg-rust/pull/496 being merged in order to run. -- This is an automated message from the Apache Git Ser

[PR] feat: performance testing harness and perf tests for scan file plan [iceberg-rust]

sdd opened a new pull request, #497: URL: https://github.com/apache/iceberg-rust/pull/497 This PR adds some performance testing capabilities. It includes the following features: * docker-compose environment that includes containers for Minio, Spark, HAProxy and the Iceberg REST Catalog

[I] Scan does not work as expected [iceberg-rust]

ndrluis opened a new issue, #495: URL: https://github.com/apache/iceberg-rust/issues/495 I'm testing using the [iceberg rest image](https://github.com/tabular-io/iceberg-rest-image) from Tabular as a catalog. Here's the docker-compose.yml file: ```yaml version: '3.8'

Re: [PR] Treat warning as error in CI/Dev [iceberg-python]

ndrluis commented on PR #973: URL: https://github.com/apache/iceberg-python/pull/973#issuecomment-2254637205 Thank you for the review, @sungwy. I made the change in commit [1c6c409](https://github.com/apache/iceberg-python/pull/973/commits/1c6c40903edc448ca67daa2460a5e84a2abe1639). -- Thi

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

amogh-jahagirdar commented on PR #9008: URL: https://github.com/apache/iceberg/pull/9008#issuecomment-2254623684 Thanks @jacobmarble @epgif it looks good from my analysis. I'll wait before merging tomorrow in case @nastra has anything else. We can merge this into main, but I'd say #10775 sh

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

amogh-jahagirdar commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1694311600 ## api/src/main/java/org/apache/iceberg/expressions/ExpressionUtil.java: ## @@ -515,6 +523,8 @@ private static String sanitize(Type type, Object value, long n

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

amogh-jahagirdar commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1694308507 ## api/src/main/java/org/apache/iceberg/expressions/ExpressionUtil.java: ## @@ -515,6 +523,8 @@ private static String sanitize(Type type, Object value, long n

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

amogh-jahagirdar commented on code in PR #9008: URL: https://github.com/apache/iceberg/pull/9008#discussion_r1694308507 ## api/src/main/java/org/apache/iceberg/expressions/ExpressionUtil.java: ## @@ -515,6 +523,8 @@ private static String sanitize(Type type, Object value, long n

Re: [I] Configure root path in Catalog or FileIO? [iceberg-rust]

Xuanwo commented on issue #488: URL: https://github.com/apache/iceberg-rust/issues/488#issuecomment-2254577316 My reason for not including a `default_table_root_location` (or `default_warehouse_location`) at the catalog implementation level is that it might confuse our users about the actua

Re: [I] Geospatial Support [iceberg]

desruisseaux commented on issue #10260: URL: https://github.com/apache/iceberg/issues/10260#issuecomment-2254563957 A CRS as a string field is fine. I suggest to limit the allowed formats to the following: * WKT 2 as defined by ISO 19162 * SRID in the following forms: * HTTP U

Re: [I] Configure root path in Catalog or FileIO? [iceberg-rust]

fqaiser94 commented on issue #488: URL: https://github.com/apache/iceberg-rust/issues/488#issuecomment-2254552826 Created a PR to switch to option 1 here: https://github.com/apache/iceberg-rust/pull/494 Might be worth waiting for @Xuanwo to chime in first though since they proposed optio

[PR] Configure default table root location in catalog [iceberg-rust]

fqaiser94 opened a new pull request, #494: URL: https://github.com/apache/iceberg-rust/pull/494 Fixes: https://github.com/apache/iceberg-rust/issues/488 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] Configure root path in Catalog or FileIO? [iceberg-rust]

liurenjie1024 commented on issue #488: URL: https://github.com/apache/iceberg-rust/issues/488#issuecomment-2254534856 Thanks @fqaiser94 for raising this. In short, I'm also in favor of option 1 to align with java/python implementations. The root path is a property of underlyin

Re: [PR] feat: add check compatible func for primitive type [iceberg-rust]

ZENOTME commented on PR #492: URL: https://github.com/apache/iceberg-rust/pull/492#issuecomment-2254531291 > > Or we can just use `PrimitiveLiteral::Long`.🤔 > > Do we always need a specific type to represent logical types? If we convert `Timestamp` to `Long`, we still need to assign a

Re: [PR] View Spec implementation [iceberg-rust]

liurenjie1024 merged PR #331: URL: https://github.com/apache/iceberg-rust/pull/331 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] View Spec implementation [iceberg-rust]

liurenjie1024 commented on PR #331: URL: https://github.com/apache/iceberg-rust/pull/331#issuecomment-2254530124 I think this pr is good enough, so I will merge it since it has been long. Thanks @c-thiel 's effort on this, and thanks @Xuanwo @nastra @ZENOTME 's review! -- This is an auto

Re: [PR] feat: add check compatible func for primitive type [iceberg-rust]

Xuanwo commented on PR #492: URL: https://github.com/apache/iceberg-rust/pull/492#issuecomment-2254516478 > Or we can just use `PrimitiveLiteral::Long`.🤔 Do we always need a specific type to represent logical types? If we convert `Timestamp` to `Long`, we still need to assign a type t

Re: [PR] chore(deps): Bump crate-ci/typos from 1.23.2 to 1.23.5 [iceberg-rust]

Xuanwo merged PR #493: URL: https://github.com/apache/iceberg-rust/pull/493 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] feat: add check compatible func for primitive type [iceberg-rust]

ZENOTME commented on PR #492: URL: https://github.com/apache/iceberg-rust/pull/492#issuecomment-2254509475 cc @liurenjie1024 @Xuanwo @Fokko -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [PR] feat: add check compatible func for primitive type [iceberg-rust]

ZENOTME commented on PR #492: URL: https://github.com/apache/iceberg-rust/pull/492#issuecomment-2254508575 It gives me a reflection that do we need the literal type like `PrimitiveLiteral::Timestamp`. Or we can just use `PrimitiveLiteral::Long`.🤔 -- This is an automated message from t

[PR] chore(deps): Bump crate-ci/typos from 1.23.2 to 1.23.5 [iceberg-rust]

dependabot[bot] opened a new pull request, #493: URL: https://github.com/apache/iceberg-rust/pull/493 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.23.2 to 1.23.5. Release notes Sourced from https://github.com/crate-ci/typos/releases";>crate-ci/typos's releases.

[PR] feat: add check compatible func for primitive type [iceberg-rust]

ZENOTME opened a new pull request, #492: URL: https://github.com/apache/iceberg-rust/pull/492 This PR is separated from https://github.com/apache/iceberg-rust/pull/349, we need to check whether the partition value(Struct) is compatible(same) as the partition spec. https://github.com/apach