Re: [PR] Python: Add support for Python 3.12 [iceberg-python]

2023-10-13 Thread via GitHub
steinsgateted commented on PR #35: URL: https://github.com/apache/iceberg-python/pull/35#issuecomment-1762594663 @Fokko Thank you. Encountered this problem: > Note: This error originates from the build backend, and is likely not a problem with poetry but with aiohttp (3.8.5) not su

Re: [PR] Flink: Custom partitioner for bucket partitions [iceberg]

2023-10-13 Thread via GitHub
stevenzwu commented on PR #7161: URL: https://github.com/apache/iceberg/pull/7161#issuecomment-1762592732 @chenwyi2 Is your point that we shouldn't only consider bucketing column (like did in this PR). you just want a plain keyBy in this case? that would be a fair point. Do you get balanced

Re: [PR] Flink: Read parquet BINARY column as String for expected [iceberg]

2023-10-13 Thread via GitHub
fengjiajie commented on PR #8808: URL: https://github.com/apache/iceberg/pull/8808#issuecomment-1762465495 @nastra Thank you for taking the time to review my code -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] push down min/max/count to iceberg [iceberg]

2023-10-13 Thread via GitHub
atifiu commented on PR #6252: URL: https://github.com/apache/iceberg/pull/6252#issuecomment-1762436892 @huaxingao So, unless we get this message "Evaluating completely on Iceberg" which means full filter pushdown is down, otherwise filter pushdown is partial or or not at all. In my case I

Re: [I] Multiple entries with same key [iceberg]

2023-10-13 Thread via GitHub
github-actions[bot] commented on issue #7221: URL: https://github.com/apache/iceberg/issues/7221#issuecomment-1762413736 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Multiple entries with same key [iceberg]

2023-10-13 Thread via GitHub
github-actions[bot] closed issue #7221: Multiple entries with same key URL: https://github.com/apache/iceberg/issues/7221 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] Update to the latest version [iceberg-python]

2023-10-13 Thread via GitHub
rdblue commented on PR #67: URL: https://github.com/apache/iceberg-python/pull/67#issuecomment-1762317780 Fine with me, but it looks like tests are failing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[I] [JdbcCatalog] Issue with Namespace Exists [iceberg]

2023-10-13 Thread via GitHub
haizhou-zhao opened a new issue, #8832: URL: https://github.com/apache/iceberg/issues/8832 ### Apache Iceberg version main (development) ### Query engine Other ### Please describe the bug 🐞 Currently, JdbcCatalog relies on this sql statement to check whether

Re: [PR] Add spec_id back to data file and also make data file inherit from manifest partition spec id [iceberg-python]

2023-10-13 Thread via GitHub
Fokko commented on PR #63: URL: https://github.com/apache/iceberg-python/pull/63#issuecomment-1762298723 Thanks! It looks like some changes on the request side broke the CI: https://github.com/apache/iceberg-python/pull/69 -- This is an automated message from the Apache Git Service

Re: [PR] Add spec_id back to data file and also make data file inherit from manifest partition spec id [iceberg-python]

2023-10-13 Thread via GitHub
puchengy commented on PR #63: URL: https://github.com/apache/iceberg-python/pull/63#issuecomment-1762294785 @Fokko Thanks, adopted your suggestion and tried a local integration test and it passed. -- This is an automated message from the Apache Git Service. To respond to the message, ple

Re: [PR] feat: manifest list writer [iceberg-rust]

2023-10-13 Thread via GitHub
barronw commented on code in PR #76: URL: https://github.com/apache/iceberg-rust/pull/76#discussion_r1358889289 ## crates/iceberg/src/spec/manifest_list.rs: ## @@ -90,24 +113,72 @@ impl ManifestList { ]; Schema::builder().with_fields(fields).build().unwrap()

Re: [PR] feat: manifest list writer [iceberg-rust]

2023-10-13 Thread via GitHub
barronw commented on code in PR #76: URL: https://github.com/apache/iceberg-rust/pull/76#discussion_r1358866633 ## crates/iceberg/src/spec/manifest_list.rs: ## @@ -69,6 +73,25 @@ impl ManifestList { &self.entries } +/// Get the v1 schema of the manifest list

Re: [PR] feat: manifest list writer [iceberg-rust]

2023-10-13 Thread via GitHub
barronw commented on code in PR #76: URL: https://github.com/apache/iceberg-rust/pull/76#discussion_r1358866184 ## crates/iceberg/src/spec/manifest_list.rs: ## @@ -940,4 +1017,104 @@ mod test { r#"[{"manifest_path":"s3a://icebergdata/demo/s1/t1/metadata/05ffe08b-81

[PR] Fix fixed type [iceberg-python]

2023-10-13 Thread via GitHub
Fokko opened a new pull request, #70: URL: https://github.com/apache/iceberg-python/pull/70 Should be `FIXED_LEN_BYTE_ARRAY` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Make `next_sequence_number` private [iceberg-python]

2023-10-13 Thread via GitHub
Fokko merged PR #62: URL: https://github.com/apache/iceberg-python/pull/62 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

[PR] Check for empty responses [iceberg-python]

2023-10-13 Thread via GitHub
Fokko opened a new pull request, #69: URL: https://github.com/apache/iceberg-python/pull/69 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

[PR] Bump version to 0.5.1 [iceberg-python]

2023-10-13 Thread via GitHub
Fokko opened a new pull request, #68: URL: https://github.com/apache/iceberg-python/pull/68 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

Re: [PR] allow override env-variables in load_catalog [iceberg-python]

2023-10-13 Thread via GitHub
Fokko merged PR #45: URL: https://github.com/apache/iceberg-python/pull/45 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

[PR] Update to the latest version [iceberg-python]

2023-10-13 Thread via GitHub
Fokko opened a new pull request, #67: URL: https://github.com/apache/iceberg-python/pull/67 Check if the dependencies are okay -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] push down min/max/count to iceberg [iceberg]

2023-10-13 Thread via GitHub
huaxingao commented on PR #6252: URL: https://github.com/apache/iceberg/pull/6252#issuecomment-1762149585 @atifiu I suspect somehow your partition filter isn't completely pushed down. In this [PR](https://github.com/apache/iceberg/pull/6524), we will discard filters that can be completely e

Re: [PR] Add spec_id back to data file [iceberg-python]

2023-10-13 Thread via GitHub
Fokko commented on PR #63: URL: https://github.com/apache/iceberg-python/pull/63#issuecomment-1762074280 @puchengy I think this fixes it: https://github.com/puchengy/iceberg-python/pull/1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] Add spec_id back to data file [iceberg-python]

2023-10-13 Thread via GitHub
Fokko commented on PR #63: URL: https://github.com/apache/iceberg-python/pull/63#issuecomment-1762064930 @puchengy let me give it a try -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Add spec_id back to data file [iceberg-python]

2023-10-13 Thread via GitHub
puchengy commented on PR #63: URL: https://github.com/apache/iceberg-python/pull/63#issuecomment-1762044854 @Fokko thanks I will address that today or tomorrow. If this becomes a blocker of the release, feel free to take it over. -- This is an automated message from the Apache Git Service

Re: [PR] Add spec_id back to data file [iceberg-python]

2023-10-13 Thread via GitHub
Fokko commented on PR #63: URL: https://github.com/apache/iceberg-python/pull/63#issuecomment-1762031697 @puchengy `make install && make lint` should fix it. It also looks like the integration test is failing because FastAvro is missing the `spec_id` (and this is correct because it

Re: [PR] Add spec_id back to data file [iceberg-python]

2023-10-13 Thread via GitHub
puchengy commented on PR #63: URL: https://github.com/apache/iceberg-python/pull/63#issuecomment-1762024206 @Fokko Will do that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [I] AvroWriter Issue: Incorrect iceberg_to_avro Schema Conversion for Decimal, Fixed, and UUID [iceberg-python]

2023-10-13 Thread via GitHub
Fokko closed issue #14: AvroWriter Issue: Incorrect iceberg_to_avro Schema Conversion for Decimal, Fixed, and UUID URL: https://github.com/apache/iceberg-python/issues/14 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] Fix Iceberg to Avro Schema Conversion: Fixed, Decimal, UUID [iceberg-python]

2023-10-13 Thread via GitHub
Fokko merged PR #53: URL: https://github.com/apache/iceberg-python/pull/53 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Add spec_id back to data file [iceberg-python]

2023-10-13 Thread via GitHub
Fokko commented on PR #63: URL: https://github.com/apache/iceberg-python/pull/63#issuecomment-1762014542 @puchengy can you fix the CI? We need to make this part of 0.5.1 since the `spec_id `was there before (as you pointed out on Slack :) -- This is an automated message from the Apache Gi

Re: [PR] Expression: Fix for when multiple and/or expressions are specified via string [iceberg-python]

2023-10-13 Thread via GitHub
Fokko merged PR #65: URL: https://github.com/apache/iceberg-python/pull/65 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [I] [BUG] string row filter ignore 2nd (and onwards) And [iceberg-python]

2023-10-13 Thread via GitHub
Fokko closed issue #64: [BUG] string row filter ignore 2nd (and onwards) And URL: https://github.com/apache/iceberg-python/issues/64 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] Expression: Fix for when multiple and/or expressions are specified via string [iceberg-python]

2023-10-13 Thread via GitHub
Fokko commented on PR #65: URL: https://github.com/apache/iceberg-python/pull/65#issuecomment-1761997037 > @Fokko, should we release 0.5.1 with this patch? Yes, I think that's a good idea -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Expression: Fix for when multiple and/or expressions are specified via string [iceberg-python]

2023-10-13 Thread via GitHub
rdblue commented on PR #65: URL: https://github.com/apache/iceberg-python/pull/65#issuecomment-1761946877 Thanks for jumping on the fix, @amogh-jahagirdar! @Fokko, should we release 0.5.1 with this patch? -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Expression: Fix for when multiple and/or expressions are specified via string [iceberg-python]

2023-10-13 Thread via GitHub
rdblue commented on code in PR #65: URL: https://github.com/apache/iceberg-python/pull/65#discussion_r1358624936 ## pyiceberg/expressions/parser.py: ## @@ -233,11 +233,11 @@ def handle_not(result: ParseResults) -> Not: def handle_and(result: ParseResults) -> And: -retur

Re: [PR] Expression: Fix for when multiple and/or expressions are specified via string [iceberg-python]

2023-10-13 Thread via GitHub
rdblue commented on code in PR #65: URL: https://github.com/apache/iceberg-python/pull/65#discussion_r1358624936 ## pyiceberg/expressions/parser.py: ## @@ -233,11 +233,11 @@ def handle_not(result: ParseResults) -> Not: def handle_and(result: ParseResults) -> And: -retur

Re: [PR] Core: Add View support for REST catalog [iceberg]

2023-10-13 Thread via GitHub
nastra commented on code in PR #7913: URL: https://github.com/apache/iceberg/pull/7913#discussion_r1358620356 ## core/src/main/java/org/apache/iceberg/rest/CatalogHandlers.java: ## @@ -374,4 +385,107 @@ static TableMetadata commit(TableOperations ops, UpdateTableRequest request

Re: [PR] Core: Replace `.size() > 0` with `!.isEmpty()` [iceberg]

2023-10-13 Thread via GitHub
amogh-jahagirdar merged PR #8813: URL: https://github.com/apache/iceberg/pull/8813 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] push down min/max/count to iceberg [iceberg]

2023-10-13 Thread via GitHub
atifiu commented on PR #6252: URL: https://github.com/apache/iceberg/pull/6252#issuecomment-1761916077 It's not working. Either with between, > or <. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] push down min/max/count to iceberg [iceberg]

2023-10-13 Thread via GitHub
huaxingao commented on PR #6252: URL: https://github.com/apache/iceberg/pull/6252#issuecomment-1761909655 If filters are on partitioned columns, aggregate pushdown should work. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [I] Upgrade to spring-web 5.3.30 [iceberg]

2023-10-13 Thread via GitHub
nastra closed issue #8827: Upgrade to spring-web 5.3.30 URL: https://github.com/apache/iceberg/issues/8827 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mai

Re: [PR] Upgrade to Jetty 9.4.53.v20231009 [iceberg]

2023-10-13 Thread via GitHub
nastra merged PR #8830: URL: https://github.com/apache/iceberg/pull/8830 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [I] Upgrade to Jetty 9.4.53.v20231009 [iceberg]

2023-10-13 Thread via GitHub
nastra closed issue #8829: Upgrade to Jetty 9.4.53.v20231009 URL: https://github.com/apache/iceberg/issues/8829 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] Upgrade to spring-web 5.3.30 [iceberg]

2023-10-13 Thread via GitHub
nastra merged PR #8828: URL: https://github.com/apache/iceberg/pull/8828 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Expression: Fix for when multiple and/or expressions are specified via string [iceberg-python]

2023-10-13 Thread via GitHub
puchengy commented on PR #65: URL: https://github.com/apache/iceberg-python/pull/65#issuecomment-1761695595 @amogh-jahagirdar The test result LGTM. However, I am not familiar with the implementation, please feel free to go ahead and merge. -- This is an automated message from the Apache G

Re: [PR] push down min/max/count to iceberg [iceberg]

2023-10-13 Thread via GitHub
atifiu commented on PR #6252: URL: https://github.com/apache/iceberg/pull/6252#issuecomment-1761688659 @huaxingao so you meant to say that with filters whether on partitioned or non partitioned column(s), aggregate pushdown will not work ? -- This is an automated message from the Apache G

Re: [PR] push down min/max/count to iceberg [iceberg]

2023-10-13 Thread via GitHub
huaxingao commented on PR #6252: URL: https://github.com/apache/iceberg/pull/6252#issuecomment-1761680652 @atifiu File statistics are not accurate and can't be used any more if you use filters. For example, you have table (col int), the max of col is 100, and the min is 0, so the sta

Re: [PR] Build: Bump org.springframework:spring-web from 5.3.9 to 6.0.13 [iceberg]

2023-10-13 Thread via GitHub
dependabot[bot] commented on PR #8811: URL: https://github.com/apache/iceberg/pull/8811#issuecomment-1761669389 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let

Re: [PR] Build: Bump org.springframework:spring-web from 5.3.9 to 6.0.13 [iceberg]

2023-10-13 Thread via GitHub
ajantha-bhat closed pull request #8811: Build: Bump org.springframework:spring-web from 5.3.9 to 6.0.13 URL: https://github.com/apache/iceberg/pull/8811 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Build: Bump org.springframework:spring-web from 5.3.9 to 6.0.13 [iceberg]

2023-10-13 Thread via GitHub
ajantha-bhat commented on PR #8811: URL: https://github.com/apache/iceberg/pull/8811#issuecomment-1761669307 Closing in the favour of https://github.com/apache/iceberg/pull/8828 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Build: Bump jetty from 9.4.43.v20210629 to 11.0.17 [iceberg]

2023-10-13 Thread via GitHub
ajantha-bhat closed pull request #8788: Build: Bump jetty from 9.4.43.v20210629 to 11.0.17 URL: https://github.com/apache/iceberg/pull/8788 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] Build: Bump jetty from 9.4.43.v20210629 to 11.0.17 [iceberg]

2023-10-13 Thread via GitHub
ajantha-bhat commented on PR #8788: URL: https://github.com/apache/iceberg/pull/8788#issuecomment-1761667236 I propose to close this PR in favor of https://github.com/apache/iceberg/pull/8830 Agree. Thanks for raising the PR to support the last JDK8 version. -- This is an automated

Re: [PR] Build: Bump jetty from 9.4.43.v20210629 to 11.0.17 [iceberg]

2023-10-13 Thread via GitHub
dependabot[bot] commented on PR #8788: URL: https://github.com/apache/iceberg/pull/8788#issuecomment-1761667373 OK, I won't notify you again about this release, but will get in touch when a new version is available. You can also ignore all major, minor, or patch releases for a dependency by

Re: [PR] Build: Bump jetty from 9.4.43.v20210629 to 11.0.15 [iceberg]

2023-10-13 Thread via GitHub
dependabot[bot] commented on PR #8238: URL: https://github.com/apache/iceberg/pull/8238#issuecomment-1761663265 OK, I won't notify you again about this release, but will get in touch when a new version is available. You can also ignore all major, minor, or patch releases for a dependency by

Re: [PR] Build: Bump jetty from 9.4.43.v20210629 to 11.0.15 [iceberg]

2023-10-13 Thread via GitHub
ajantha-bhat closed pull request #8238: Build: Bump jetty from 9.4.43.v20210629 to 11.0.15 URL: https://github.com/apache/iceberg/pull/8238 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] Build: Bump jetty from 9.4.43.v20210629 to 11.0.15 [iceberg]

2023-10-13 Thread via GitHub
ajantha-bhat commented on PR #8238: URL: https://github.com/apache/iceberg/pull/8238#issuecomment-1761663164 > I propose to close this PR in favor of https://github.com/apache/iceberg/pull/8830 Agree. Thanks for raising the PR to support the last JDK8 version. -- This is an automa

Re: [PR] Build: Bump jetty from 9.4.43.v20210629 to 11.0.17 [iceberg]

2023-10-13 Thread via GitHub
jbonofre commented on PR #8788: URL: https://github.com/apache/iceberg/pull/8788#issuecomment-1761560686 I propose to close this PR in favor of #8830 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Flink: Read parquet BINARY column as String for expected [iceberg]

2023-10-13 Thread via GitHub
nastra commented on code in PR #8808: URL: https://github.com/apache/iceberg/pull/8808#discussion_r1358331014 ## flink/v1.15/flink/src/test/java/org/apache/iceberg/flink/data/TestFlinkParquetReader.java: ## @@ -90,17 +92,72 @@ public void testTwoLevelList() throws IOException {

Re: [PR] Flink: Read parquet BINARY column as String for expected [iceberg]

2023-10-13 Thread via GitHub
nastra commented on code in PR #8808: URL: https://github.com/apache/iceberg/pull/8808#discussion_r1358327260 ## flink/v1.15/flink/src/test/java/org/apache/iceberg/flink/data/TestFlinkParquetReader.java: ## @@ -90,17 +92,72 @@ public void testTwoLevelList() throws IOException {

Re: [PR] Flink: Read parquet BINARY column as String for expected [iceberg]

2023-10-13 Thread via GitHub
nastra commented on code in PR #8808: URL: https://github.com/apache/iceberg/pull/8808#discussion_r1358326874 ## flink/v1.15/flink/src/test/java/org/apache/iceberg/flink/data/TestFlinkParquetReader.java: ## @@ -90,17 +92,72 @@ public void testTwoLevelList() throws IOException {

[I] Build: enforce reproducible build [iceberg]

2023-10-13 Thread via GitHub
jbonofre opened a new issue, #8825: URL: https://github.com/apache/iceberg/issues/8825 ### Feature Request / Improvement Reproducible builds is a development practice that create an independently-verifiable path from source to binary code. A build is reproducible if given the same

[PR] Upgrade to spring-web 5.3.30 [iceberg]

2023-10-13 Thread via GitHub
jbonofre opened a new pull request, #8828: URL: https://github.com/apache/iceberg/pull/8828 Close #8827 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

Re: [PR] Build: Bump jetty from 9.4.43.v20210629 to 11.0.15 [iceberg]

2023-10-13 Thread via GitHub
jbonofre commented on PR #8238: URL: https://github.com/apache/iceberg/pull/8238#issuecomment-1761561894 I propose to close this PR in favor of #8830 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[PR] Build: add gradle configuration to enforce reproducible build [iceberg]

2023-10-13 Thread via GitHub
jbonofre opened a new pull request, #8826: URL: https://github.com/apache/iceberg/pull/8826 Close #8825 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

Re: [PR] Core: Add View support for REST catalog [iceberg]

2023-10-13 Thread via GitHub
nastra commented on code in PR #7913: URL: https://github.com/apache/iceberg/pull/7913#discussion_r1358250961 ## core/src/main/java/org/apache/iceberg/rest/CatalogHandlers.java: ## @@ -374,4 +385,107 @@ static TableMetadata commit(TableOperations ops, UpdateTableRequest request

Re: [PR] Spark: spark/*: replaced .size() > 0 with isEmpty() [iceberg]

2023-10-13 Thread via GitHub
amogh-jahagirdar merged PR #8814: URL: https://github.com/apache/iceberg/pull/8814 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [I] Make `location` in `TableCreation` optional [iceberg-rust]

2023-10-13 Thread via GitHub
Xuanwo commented on issue #67: URL: https://github.com/apache/iceberg-rust/issues/67#issuecomment-1761422712 I'm going to make this change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] Expression: Fix for when multiple and/or expressions are specified via string [iceberg-python]

2023-10-13 Thread via GitHub
amogh-jahagirdar commented on PR #65: URL: https://github.com/apache/iceberg-python/pull/65#issuecomment-1761366348 Yes thanks @puchengy for reporting this, please also take a look a this fix when you get a chance! -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] Expression: Fix for when multiple and/or expressions are specified via string [iceberg-python]

2023-10-13 Thread via GitHub
amogh-jahagirdar commented on code in PR #65: URL: https://github.com/apache/iceberg-python/pull/65#discussion_r1358138359 ## pyiceberg/expressions/parser.py: ## @@ -233,10 +233,14 @@ def handle_not(result: ParseResults) -> Not: def handle_and(result: ParseResults) -> And:

Re: [PR] Expression: Fix for when multiple and/or expressions are specified via string [iceberg-python]

2023-10-13 Thread via GitHub
amogh-jahagirdar commented on code in PR #65: URL: https://github.com/apache/iceberg-python/pull/65#discussion_r1358137792 ## pyiceberg/expressions/parser.py: ## @@ -233,10 +233,14 @@ def handle_not(result: ParseResults) -> Not: def handle_and(result: ParseResults) -> And:

Re: [PR] feat: First version of rest catalog. [iceberg-rust]

2023-10-13 Thread via GitHub
liurenjie1024 commented on code in PR #78: URL: https://github.com/apache/iceberg-rust/pull/78#discussion_r1357791341 ## crates/iceberg/Cargo.toml: ## @@ -41,20 +41,24 @@ either = "1" futures = "0.3" itertools = "0.11" lazy_static = "1" +log = "^0.4" murmur3 = "0.5.2" once_

Re: [PR] feat: suport read/write Manifest [iceberg-rust]

2023-10-13 Thread via GitHub
liurenjie1024 commented on code in PR #79: URL: https://github.com/apache/iceberg-rust/pull/79#discussion_r1358133729 ## crates/iceberg/src/spec/values.rs: ## @@ -966,6 +978,547 @@ mod timestamptz { } } +mod serde { Review Comment: I'm ok with this approach, cc @JanK

Re: [PR] Expression: Fix for when multiple and/or expressions are specified via string [iceberg-python]

2023-10-13 Thread via GitHub
Fokko commented on code in PR #65: URL: https://github.com/apache/iceberg-python/pull/65#discussion_r1358122959 ## pyiceberg/expressions/parser.py: ## @@ -233,10 +233,14 @@ def handle_not(result: ParseResults) -> Not: def handle_and(result: ParseResults) -> And: +if len

Re: [PR] Expression: Fix for when multiple and/or expressions are specified via string [iceberg-python]

2023-10-13 Thread via GitHub
Fokko commented on PR #65: URL: https://github.com/apache/iceberg-python/pull/65#issuecomment-1761329387 This is a serious one, thanks for reporting @puchengy -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] Expression: Fix for when multiple and/or expressions are specified via string [iceberg-python]

2023-10-13 Thread via GitHub
amogh-jahagirdar commented on code in PR #65: URL: https://github.com/apache/iceberg-python/pull/65#discussion_r1358113965 ## pyiceberg/expressions/parser.py: ## @@ -233,10 +233,14 @@ def handle_not(result: ParseResults) -> Not: def handle_and(result: ParseResults) -> And:

[PR] Run dependabot daily [iceberg-python]

2023-10-13 Thread via GitHub
Fokko opened a new pull request, #66: URL: https://github.com/apache/iceberg-python/pull/66 I would love to run dependabot daily instead of weekly. Now we're in our own repository, we'll introduce less noise. This would have helped us to identify the issue with Pydantic earlier. -- This

[PR] Expression: Fix for when multiple and/or expressions are specified via string [iceberg-python]

2023-10-13 Thread via GitHub
amogh-jahagirdar opened a new pull request, #65: URL: https://github.com/apache/iceberg-python/pull/65 Fixes #64 . If there are multiple and/or conditions currently, our expression parser will ignore anything after the second predicate. This change fixes the issue by forwarding the remainin

Re: [PR] Flink, Aliyun, MR, Delta-lake, Hive3, Parquet, Data: replaced .size() > 0 with isEmpty() [iceberg]

2023-10-13 Thread via GitHub
Fokko commented on PR #8819: URL: https://github.com/apache/iceberg/pull/8819#issuecomment-1761324612 Thanks @PickBas for picking this up 🙌 and @nastra and @ajantha-bhat for the review 👍 -- This is an automated message from the Apache Git Service. To respond to the message, please log o

Re: [PR] Flink, Aliyun, MR, Delta-lake, Hive3, Parquet, Data: replaced .size() > 0 with isEmpty() [iceberg]

2023-10-13 Thread via GitHub
Fokko merged PR #8819: URL: https://github.com/apache/iceberg/pull/8819 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [PR] Python: Add support for Python 3.12 [iceberg-python]

2023-10-13 Thread via GitHub
Fokko commented on PR #35: URL: https://github.com/apache/iceberg-python/pull/35#issuecomment-1761321623 @steinsgateted can you pull in the main branch? It looks like that 3.12 is available: https://github.com/actions/python-versions/releases -- This is an automated message from the Apach

Re: [PR] Build: Fix compiler warnings [iceberg]

2023-10-13 Thread via GitHub
nastra merged PR #8763: URL: https://github.com/apache/iceberg/pull/8763 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Flink: Read parquet BINARY column as String for expected [iceberg]

2023-10-13 Thread via GitHub
nastra commented on code in PR #8808: URL: https://github.com/apache/iceberg/pull/8808#discussion_r1358102450 ## flink/v1.15/flink/src/test/java/org/apache/iceberg/flink/data/TestFlinkParquetReader.java: ## @@ -98,6 +100,62 @@ public void testTwoLevelList() throws IOException {

Re: [I] [BUG] string row filter ignore 2nd (and onwards) And [iceberg-python]

2023-10-13 Thread via GitHub
amogh-jahagirdar commented on issue #64: URL: https://github.com/apache/iceberg-python/issues/64#issuecomment-1761250090 https://github.com/apache/iceberg-python/blob/main/pyiceberg/expressions/parser.py#L236 this is where it'll skip anything after. I think this needs to combine all result

Re: [I] [BUG] string row filter ignore 2nd (and onwards) And [iceberg-python]

2023-10-13 Thread via GitHub
amogh-jahagirdar commented on issue #64: URL: https://github.com/apache/iceberg-python/issues/64#issuecomment-1761230308 Looking into this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [I] [BUG] string row filter ignore 2nd (and onwards) And [iceberg-python]

2023-10-13 Thread via GitHub
amogh-jahagirdar commented on issue #64: URL: https://github.com/apache/iceberg-python/issues/64#issuecomment-1761229740 Hm yeah I can repro this with a simple test in `table/test_init.py` ``` scan = table.scan(row_filter="x=1 AND y=1 AND z=1") assert scan.row_filter == And(E

Re: [PR] Add spec_id back to data file [iceberg-python]

2023-10-13 Thread via GitHub
Fokko commented on code in PR #63: URL: https://github.com/apache/iceberg-python/pull/63#discussion_r1358033938 ## pyiceberg/manifest.py: ## @@ -182,6 +182,7 @@ def __repr__(self) -> str: doc="Splittable offsets", ), NestedField(field_id=140, name=

Re: [PR] feat(tables): add basic table implementation [iceberg-go]

2023-10-13 Thread via GitHub
nastra merged PR #11: URL: https://github.com/apache/iceberg-go/pull/11 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [PR] feat(tables): add basic table implementation [iceberg-go]

2023-10-13 Thread via GitHub
nastra commented on code in PR #11: URL: https://github.com/apache/iceberg-go/pull/11#discussion_r1358032220 ## table/table.go: ## @@ -0,0 +1,97 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +//

Re: [PR] feat(tables): add basic table implementation [iceberg-go]

2023-10-13 Thread via GitHub
nastra commented on code in PR #11: URL: https://github.com/apache/iceberg-go/pull/11#discussion_r1358030652 ## table/table.go: ## @@ -0,0 +1,97 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +//

Re: [PR] feat(tables): add basic table implementation [iceberg-go]

2023-10-13 Thread via GitHub
nastra commented on code in PR #11: URL: https://github.com/apache/iceberg-go/pull/11#discussion_r1358030118 ## table/metadata.go: ## @@ -0,0 +1,401 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file

Re: [PR] Flink: Read parquet BINARY column as String for expected [iceberg]

2023-10-13 Thread via GitHub
nastra commented on PR #8808: URL: https://github.com/apache/iceberg/pull/8808#issuecomment-1761210176 @fengjiajie thanks for working on this. Could you please add a test that reproduces the issue? -- This is an automated message from the Apache Git Service. To respond to the message, ple

Re: [PR] Flink, Aliyun, MR, Delta-lake, Hive3, Parquet, Data: replaced .size() > 0 with isEmpty() [iceberg]

2023-10-13 Thread via GitHub
nastra commented on code in PR #8819: URL: https://github.com/apache/iceberg/pull/8819#discussion_r1358006481 ## aws/src/integration/java/org/apache/iceberg/aws/glue/TestGlueCatalogNamespace.java: ## @@ -105,7 +105,7 @@ public void testNamespaceExists() { public void testList

Re: [PR] Flink, Aliyun, MR, Delta-lake, Hive3, Parquet, Data: replaced .size() > 0 with isEmpty() [iceberg]

2023-10-13 Thread via GitHub
Fokko commented on code in PR #8819: URL: https://github.com/apache/iceberg/pull/8819#discussion_r1357986438 ## aws/src/integration/java/org/apache/iceberg/aws/glue/TestGlueCatalogNamespace.java: ## @@ -105,7 +105,7 @@ public void testNamespaceExists() { public void testListN

Re: [PR] Flink, Aliyun, MR, Delta-lake, Hive3, Parquet, Data: replaced .size() > 0 with isEmpty() [iceberg]

2023-10-13 Thread via GitHub
Fokko commented on PR #8819: URL: https://github.com/apache/iceberg/pull/8819#issuecomment-1761169780 > If that works for both you and @Fokko, I will surely do that. I'm open to everything. -- This is an automated message from the Apache Git Service. To respond to the message, plea

Re: [PR] Flink: Custom partitioner for bucket partitions [iceberg]

2023-10-13 Thread via GitHub
chenwyi2 commented on PR #7161: URL: https://github.com/apache/iceberg/pull/7161#issuecomment-1761169778 Hi @stevenzwu @kengtin this PR can be create too many small files when parition with dt,hout,minute and bucekt(id), suppose paralisim is 120 and bucke number is 8, then 15 writes can wri

Re: [PR] Flink, Aliyun, MR, Delta-lake, Hive3, Parquet, Data: replaced .size() > 0 with isEmpty() [iceberg]

2023-10-13 Thread via GitHub
PickBas commented on PR #8819: URL: https://github.com/apache/iceberg/pull/8819#issuecomment-1761167713 @amogh-jahagirdar @nastra @Fokko Done. Now the rest of the modules are included in this pull request. -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [PR] Api: api/*: replaced .size() > 0 with isEmpty() [iceberg]

2023-10-13 Thread via GitHub
PickBas closed pull request #8822: Api: api/*: replaced .size() > 0 with isEmpty() URL: https://github.com/apache/iceberg/pull/8822 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] Data: data/*: replaced .size() > 0 with isEmpty() [iceberg]

2023-10-13 Thread via GitHub
PickBas closed pull request #8815: Data: data/*: replaced .size() > 0 with isEmpty() URL: https://github.com/apache/iceberg/pull/8815 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] Parquet: parquet/*: replaced .size() > 0 with isEmpty() [iceberg]

2023-10-13 Thread via GitHub
PickBas closed pull request #8816: Parquet: parquet/*: replaced .size() > 0 with isEmpty() URL: https://github.com/apache/iceberg/pull/8816 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] Hive3: hive3/*: replaced .size() > 0 with isEmpty() [iceberg]

2023-10-13 Thread via GitHub
PickBas closed pull request #8817: Hive3: hive3/*: replaced .size() > 0 with isEmpty() URL: https://github.com/apache/iceberg/pull/8817 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Delta-lake: delta-lake/*: replaced .size() > 0 with isEmpty() [iceberg]

2023-10-13 Thread via GitHub
PickBas closed pull request #8818: Delta-lake: delta-lake/*: replaced .size() > 0 with isEmpty() URL: https://github.com/apache/iceberg/pull/8818 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Mr: mr/*: replaced .size() > 0 with isEmpty() [iceberg]

2023-10-13 Thread via GitHub
PickBas closed pull request #8820: Mr: mr/*: replaced .size() > 0 with isEmpty() URL: https://github.com/apache/iceberg/pull/8820 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

  1   2   >