Re: [I] Create table format version constants [iceberg-python]

2024-09-15 Thread via GitHub
tanmayrauth commented on issue #851: URL: https://github.com/apache/iceberg-python/issues/851#issuecomment-2352048137 Thanks @kevinjqliu. What I understood I shall create a enum declaration for (v1, v2, v3.. ) and get it replaced everywhere in code where we use 1, 2, 3... for table version.

[I] to_pandas(), to_arrow() fail because case_sensitive doesn't if column in row_filter doesn't match the case even if case_sensitive is set to False in scan [iceberg-python]

2024-09-15 Thread via GitHub
leonidmakarovsky opened a new issue, #1177: URL: https://github.com/apache/iceberg-python/issues/1177 ### Apache Iceberg version 0.7.1 (latest release) ### Please describe the bug 🐞 I have a table with column business_time_hour (all lower case). If I capitalize one chara

Re: [PR] Flink: Increase the number of checkpoints from 4 to 6 to fix flakiness. [iceberg]

2024-09-15 Thread via GitHub
stevenzwu commented on PR #11121: URL: https://github.com/apache/iceberg/pull/11121#issuecomment-2351975559 I have triggered the CI run 5 times and they all succeeded. https://github.com/user-attachments/assets/72612a39-d1da-4625-8886-bc51b86080a1";> -- This is an automated mes

Re: [PR] REST: Handle Requests with Page Sizes Exceeding Available Number of Namespaces /Tables/Views [iceberg]

2024-09-15 Thread via GitHub
rcjverhoef commented on code in PR #11143: URL: https://github.com/apache/iceberg/pull/11143#discussion_r1760461893 ## core/src/main/java/org/apache/iceberg/rest/CatalogHandlers.java: ## @@ -124,20 +133,27 @@ public static ListNamespacesResponse listNamespaces( SupportsNa

Re: [I] Create table format version constants [iceberg-python]

2024-09-15 Thread via GitHub
kevinjqliu commented on issue #851: URL: https://github.com/apache/iceberg-python/issues/851#issuecomment-2351854564 @tanmayrauth this seem stale, please feel free to work on it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [I] Clarity on serialization format of schema.name-mapping.default in Iceberg metadata [iceberg]

2024-09-15 Thread via GitHub
github-actions[bot] commented on issue #8437: URL: https://github.com/apache/iceberg/issues/8437#issuecomment-2351849908 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] When iceberg stream reads table data, the data of update and delete operations will not be read out [iceberg]

2024-09-15 Thread via GitHub
github-actions[bot] commented on issue #7835: URL: https://github.com/apache/iceberg/issues/7835#issuecomment-2351849678 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] CDC data inconsistencies with schema changes [iceberg]

2024-09-15 Thread via GitHub
github-actions[bot] closed issue #7822: CDC data inconsistencies with schema changes URL: https://github.com/apache/iceberg/issues/7822 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [I] Failed to check if LessThan(status,2) can be pushed down: Cannot find field 'status' in struct: struct<> [iceberg]

2024-09-15 Thread via GitHub
github-actions[bot] closed issue #7774: Failed to check if LessThan(status,2) can be pushed down: Cannot find field 'status' in struct: struct<> URL: https://github.com/apache/iceberg/issues/7774 -- This is an automated message from the Apache Git Service. To respond to the message, please l

Re: [I] Build incremental update for a stats file based on incremental scan. [iceberg]

2024-09-15 Thread via GitHub
github-actions[bot] commented on issue #8460: URL: https://github.com/apache/iceberg/issues/8460#issuecomment-2351849977 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Flaky test/env TestFileRewriteCoordinator [iceberg]

2024-09-15 Thread via GitHub
github-actions[bot] commented on issue #8441: URL: https://github.com/apache/iceberg/issues/8441#issuecomment-2351849938 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Logic about Hive version checking in MetastoreLock [iceberg]

2024-09-15 Thread via GitHub
github-actions[bot] commented on issue #8440: URL: https://github.com/apache/iceberg/issues/8440#issuecomment-2351849925 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] When iceberg stream reads table data, the data of update and delete operations will not be read out [iceberg]

2024-09-15 Thread via GitHub
github-actions[bot] closed issue #7835: When iceberg stream reads table data, the data of update and delete operations will not be read out URL: https://github.com/apache/iceberg/issues/7835 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [I] [bug] Spark SQL phase optimization failed on concurrent write attempt [iceberg]

2024-09-15 Thread via GitHub
github-actions[bot] closed issue #7800: [bug] Spark SQL phase optimization failed on concurrent write attempt URL: https://github.com/apache/iceberg/issues/7800 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [I] CDC data inconsistencies with schema changes [iceberg]

2024-09-15 Thread via GitHub
github-actions[bot] commented on issue #7822: URL: https://github.com/apache/iceberg/issues/7822#issuecomment-2351849652 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] The Orc file (via iceberg)because large than Orc file(only via spark) ? [iceberg]

2024-09-15 Thread via GitHub
github-actions[bot] commented on issue #7775: URL: https://github.com/apache/iceberg/issues/7775#issuecomment-2351849614 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] [bug] Spark SQL phase optimization failed on concurrent write attempt [iceberg]

2024-09-15 Thread via GitHub
github-actions[bot] commented on issue #7800: URL: https://github.com/apache/iceberg/issues/7800#issuecomment-2351849632 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] The Orc file (via iceberg)because large than Orc file(only via spark) ? [iceberg]

2024-09-15 Thread via GitHub
github-actions[bot] closed issue #7775: The Orc file (via iceberg)because large than Orc file(only via spark) ? URL: https://github.com/apache/iceberg/issues/7775 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [I] Create table format version constants [iceberg-python]

2024-09-15 Thread via GitHub
tanmayrauth commented on issue #851: URL: https://github.com/apache/iceberg-python/issues/851#issuecomment-2351818325 @harsha-mangena are you still working on this ? If not, can I work on this ? -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [I] Modify SQLCatalog initialization so that classes are not always created and update how these classes are created to be more open to tother DB's [iceberg-python]

2024-09-15 Thread via GitHub
sungwy commented on issue #1148: URL: https://github.com/apache/iceberg-python/issues/1148#issuecomment-2351786822 Hi @isc-patrick - thank you for putting up the PR and, sharing the related Java link. I thought about this a lot more in light of @Fokko ’s review comment on https://g

Re: [PR] Test: Add coverage for Output (WIP) [iceberg-go]

2024-09-15 Thread via GitHub
alex-kar commented on PR #149: URL: https://github.com/apache/iceberg-go/pull/149#issuecomment-2351750960 Also I tried to write something like "parameterized" test to provide different test cases, but not sure if arguments are readable. Here is the idea: ``` var testArgs = []struct

Re: [PR] Test: Add coverage for Output (WIP) [iceberg-go]

2024-09-15 Thread via GitHub
zeroshade commented on PR #149: URL: https://github.com/apache/iceberg-go/pull/149#issuecomment-2351740407 @alex-kar oh perfect! I missed that. Nice! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Test: Add coverage for Output (WIP) [iceberg-go]

2024-09-15 Thread via GitHub
alex-kar commented on PR #149: URL: https://github.com/apache/iceberg-go/pull/149#issuecomment-2351738067 @zeroshade ``` Can we only disable the colors for the tests? I personally don't want to lose the colors for regular output and use of the CLI, perhaps we make an option for disab

Re: [PR] Test: Add coverage for Output (WIP) [iceberg-go]

2024-09-15 Thread via GitHub
zeroshade commented on PR #149: URL: https://github.com/apache/iceberg-go/pull/149#issuecomment-2351717136 This is awesome for a find, thanks for this! As for your questions: I personally prefer this one: ```go pterm.SetDefaultOutput(os.Stdout) text{}.DescribeTable(ta

Re: [PR] Impl rest catalog + table updates & requirements [iceberg-go]

2024-09-15 Thread via GitHub
zeroshade commented on code in PR #146: URL: https://github.com/apache/iceberg-go/pull/146#discussion_r1760197376 ## catalog/rest.go: ## @@ -546,12 +640,8 @@ func (r *RestCatalog) ListTables(ctx context.Context, namespace table.Identifier path := []string{"namespaces",

Re: [PR] Test: Add coverage for Output (WIP) [iceberg-go]

2024-09-15 Thread via GitHub
alex-kar commented on PR #149: URL: https://github.com/apache/iceberg-go/pull/149#issuecomment-2351671261 @zeroshade 1. I've figured out how to run tests with pterm, but there are a few questions I need your input on. ``` var buf bytes.Buffer pterm.SetDefaultOutput(&buf)

Re: [PR] REST: Handle Requests with Page Sizes Exceeding Available Number of Namespaces /Tables/Views [iceberg]

2024-09-15 Thread via GitHub
singhpk234 commented on code in PR #11143: URL: https://github.com/apache/iceberg/pull/11143#discussion_r1760113483 ## core/src/main/java/org/apache/iceberg/rest/CatalogHandlers.java: ## @@ -124,20 +133,27 @@ public static ListNamespacesResponse listNamespaces( SupportsNa

Re: [I] Modify SQLCatalog initialization so that classes are not always created and update how these classes are created to be more open to tother DB's [iceberg-python]

2024-09-15 Thread via GitHub
isc-patrick commented on issue #1148: URL: https://github.com/apache/iceberg-python/issues/1148#issuecomment-2351621680 I'm not sure how much the Python API is trying to conform to the Java API, but this same feature for the same reason was requested and implemented for the JDBC catalog, h

[PR] REST: Handle Requests with Page Sizes Exceeding Available Number of Namespaces /Tables/Views [iceberg]

2024-09-15 Thread via GitHub
rcjverhoef opened a new pull request, #11143: URL: https://github.com/apache/iceberg/pull/11143 When requesting a pagesize that exceeds the number of namespaces/views/tables from the REST Catalog, it throws an `IndexOutOfBoundsException`. This is due to using `List.subList` with the end-in

[I] REST Catalog pagination can throw IndexOutOfBoundsException [iceberg]

2024-09-15 Thread via GitHub
rcjverhoef opened a new issue, #11142: URL: https://github.com/apache/iceberg/issues/11142 ### Apache Iceberg version 1.6.1 (latest release) ### Query engine None ### Please describe the bug 🐞 The calls to do pagination of namespaces, tables and views throws

Re: [I] Table maintenace procedure(expire_snapshots) not work as expceted [iceberg]

2024-09-15 Thread via GitHub
SanjayKhoros commented on issue #10907: URL: https://github.com/apache/iceberg/issues/10907#issuecomment-2351566866 I'm facing the exact issue myself !! The **metadata/** folder is piling up and the contents from **data/** & **metadata/** are not getting deleted. Sharing my **table pr

[PR] chore(deps): Update arrow-string requirement from 52 to 53 [iceberg-rust]

2024-09-15 Thread via GitHub
dependabot[bot] opened a new pull request, #633: URL: https://github.com/apache/iceberg-rust/pull/633 Updates the requirements on [arrow-string](https://github.com/apache/arrow-rs) to permit the latest version. Changelog Sourced from https://github.com/apache/arrow-rs/blob/master/C

[PR] chore(deps): Update arrow-select requirement from 52 to 53 [iceberg-rust]

2024-09-15 Thread via GitHub
dependabot[bot] opened a new pull request, #632: URL: https://github.com/apache/iceberg-rust/pull/632 Updates the requirements on [arrow-select](https://github.com/apache/arrow-rs) to permit the latest version. Changelog Sourced from https://github.com/apache/arrow-rs/blob/master/C

Re: [PR] build(deps): bump github.com/hamba/avro/v2 from 2.23.0 to 2.25.1 [iceberg-go]

2024-09-15 Thread via GitHub
dependabot[bot] closed pull request #142: build(deps): bump github.com/hamba/avro/v2 from 2.23.0 to 2.25.1 URL: https://github.com/apache/iceberg-go/pull/142 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[PR] build(deps): bump github.com/hamba/avro/v2 from 2.23.0 to 2.25.2 [iceberg-go]

2024-09-15 Thread via GitHub
dependabot[bot] opened a new pull request, #148: URL: https://github.com/apache/iceberg-go/pull/148 Bumps [github.com/hamba/avro/v2](https://github.com/hamba/avro) from 2.23.0 to 2.25.2. Release notes Sourced from https://github.com/hamba/avro/releases";>github.com/hamba/avro/v2's

Re: [PR] build(deps): bump github.com/hamba/avro/v2 from 2.23.0 to 2.25.1 [iceberg-go]

2024-09-15 Thread via GitHub
dependabot[bot] commented on PR #142: URL: https://github.com/apache/iceberg-go/pull/142#issuecomment-2351472286 Superseded by #148. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/glue from 1.95.0 to 1.96.0 [iceberg-go]

2024-09-15 Thread via GitHub
dependabot[bot] opened a new pull request, #147: URL: https://github.com/apache/iceberg-go/pull/147 Bumps [github.com/aws/aws-sdk-go-v2/service/glue](https://github.com/aws/aws-sdk-go-v2) from 1.95.0 to 1.96.0. Changelog Sourced from https://github.com/aws/aws-sdk-go-v2/blob/servi