[GitHub] [iceberg] Fokko merged pull request #5940: Build: Bump actions/stale from 6.0.0 to 6.0.1

2022-10-10 Thread GitBox
Fokko merged PR #5940: URL: https://github.com/apache/iceberg/pull/5940 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

[GitHub] [iceberg] Fokko merged pull request #5941: Build: Bump pytest-checkdocs from 2.8.1 to 2.9.0 in /python

2022-10-10 Thread GitBox
Fokko merged PR #5941: URL: https://github.com/apache/iceberg/pull/5941 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

[GitHub] [iceberg] nastra commented on a diff in pull request #5944: API: Update expression sanitization for relative dates and times

2022-10-10 Thread GitBox
nastra commented on code in PR #5944: URL: https://github.com/apache/iceberg/pull/5944#discussion_r990983232 ## api/src/main/java/org/apache/iceberg/expressions/ExpressionUtil.java: ## @@ -111,7 +122,15 @@ public static boolean selectsPartitions( private static class Expres

[GitHub] [iceberg] hililiwei commented on pull request #5938: Flink: update deprecated code in flink

2022-10-10 Thread GitBox
hililiwei commented on PR #5938: URL: https://github.com/apache/iceberg/pull/5938#issuecomment-1272919051 https://github.com/apache/iceberg/pull/4246 Just to list the one I had before. I remember someone else mentioning one or two similar ones. -- This is an automated message from the

[GitHub] [iceberg] 95liu opened a new issue, #5945: Read Iceberg Table Bug(cannot find field start_date from [org.apache.iceberg.mr.hive.serde.objectinspector.IcebergRecordObj)

2022-10-10 Thread GitBox
95liu opened a new issue, #5945: URL: https://github.com/apache/iceberg/issues/5945 ### Apache Iceberg version 0.14.0 ### Query engine Hive ### Please describe the bug 🐞 When I join two iceberg tables, an error occurs that a field cannot be found。(0.14.0 ve

[GitHub] [iceberg] 95liu commented on issue #5945: Read Iceberg Table Bug(cannot find field start_date from [org.apache.iceberg.mr.hive.serde.objectinspector.IcebergRecordObj)

2022-10-10 Thread GitBox
95liu commented on issue #5945: URL: https://github.com/apache/iceberg/issues/5945#issuecomment-1272938243 hive 2.1.0 iceberg 0.14.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [iceberg] hunter-cloud09 commented on pull request #5938: Flink: update deprecated code in flink

2022-10-10 Thread GitBox
hunter-cloud09 commented on PR #5938: URL: https://github.com/apache/iceberg/pull/5938#issuecomment-1272941443 > #4246 Just to list the one I had before. I remember someone else mentioning one or two similar ones. get it, but why is it not merged in? The current TableSchema is @Deprec

[GitHub] [iceberg] nastra commented on a diff in pull request #5887: Core: Clear queue and future task when close ParallelIterable

2022-10-10 Thread GitBox
nastra commented on code in PR #5887: URL: https://github.com/apache/iceberg/pull/5887#discussion_r991019523 ## core/src/test/java/org/apache/iceberg/TestManifestGroupPlanFiles.java: ## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

[GitHub] [iceberg] nastra commented on a diff in pull request #5887: Core: Clear queue and future task when close ParallelIterable

2022-10-10 Thread GitBox
nastra commented on code in PR #5887: URL: https://github.com/apache/iceberg/pull/5887#discussion_r991035716 ## core/src/main/java/org/apache/iceberg/util/ParallelIterable.java: ## @@ -81,13 +81,18 @@ private ParallelIterator( @Override public void close() { + /

[GitHub] [iceberg] 0xffmeta opened a new issue, #5946: Not able to run spark procedure rewrite_data_files

2022-10-10 Thread GitBox
0xffmeta opened a new issue, #5946: URL: https://github.com/apache/iceberg/issues/5946 ### Apache Iceberg version 0.13.1 ### Query engine Spark ### Please describe the bug 🐞 When I tried to run spark procedure ``` CALL spark_catalog.system.rewrite_data

[GitHub] [iceberg] Heltman commented on a diff in pull request #5887: Core: Clear queue and future task when close ParallelIterable

2022-10-10 Thread GitBox
Heltman commented on code in PR #5887: URL: https://github.com/apache/iceberg/pull/5887#discussion_r991058325 ## core/src/test/java/org/apache/iceberg/TestManifestGroupPlanFiles.java: ## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

[GitHub] [iceberg] hunter-cloud09 commented on a diff in pull request #4246: Flink: Add support for ResolvedSchema

2022-10-10 Thread GitBox
hunter-cloud09 commented on code in PR #4246: URL: https://github.com/apache/iceberg/pull/4246#discussion_r991080676 ## flink/v1.15/flink/src/main/java/org/apache/iceberg/flink/sink/FlinkSink.java: ## @@ -495,19 +502,27 @@ private DataStream distributeDataStream(DataStream inpu

[GitHub] [iceberg] hunter-cloud09 commented on a diff in pull request #4246: Flink: Add support for ResolvedSchema

2022-10-10 Thread GitBox
hunter-cloud09 commented on code in PR #4246: URL: https://github.com/apache/iceberg/pull/4246#discussion_r991080676 ## flink/v1.15/flink/src/main/java/org/apache/iceberg/flink/sink/FlinkSink.java: ## @@ -495,19 +502,27 @@ private DataStream distributeDataStream(DataStream inpu

[GitHub] [iceberg] nastra commented on a diff in pull request #5887: Core: Clear queue and future task when close ParallelIterable

2022-10-10 Thread GitBox
nastra commented on code in PR #5887: URL: https://github.com/apache/iceberg/pull/5887#discussion_r991082129 ## core/src/test/java/org/apache/iceberg/TestManifestGroupPlanFiles.java: ## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

[GitHub] [iceberg] hunter-cloud09 closed pull request #5938: Flink: update deprecated code in flink

2022-10-10 Thread GitBox
hunter-cloud09 closed pull request #5938: Flink: update deprecated code in flink URL: https://github.com/apache/iceberg/pull/5938 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [iceberg] Heltman commented on a diff in pull request #5887: Core: Clear queue and future task when close ParallelIterable

2022-10-10 Thread GitBox
Heltman commented on code in PR #5887: URL: https://github.com/apache/iceberg/pull/5887#discussion_r991088413 ## core/src/test/java/org/apache/iceberg/TestManifestGroupPlanFiles.java: ## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

[GitHub] [iceberg] Heltman commented on a diff in pull request #5887: Core: Clear queue and future task when close ParallelIterable

2022-10-10 Thread GitBox
Heltman commented on code in PR #5887: URL: https://github.com/apache/iceberg/pull/5887#discussion_r991088413 ## core/src/test/java/org/apache/iceberg/TestManifestGroupPlanFiles.java: ## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

[GitHub] [iceberg] Fokko merged pull request #5897: Python: Catch ValidationError on invalid REST response

2022-10-10 Thread GitBox
Fokko merged PR #5897: URL: https://github.com/apache/iceberg/pull/5897 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

[GitHub] [iceberg] nastra commented on a diff in pull request #5887: Core: Clear queue and future task when close ParallelIterable

2022-10-10 Thread GitBox
nastra commented on code in PR #5887: URL: https://github.com/apache/iceberg/pull/5887#discussion_r991095816 ## core/src/test/java/org/apache/iceberg/io/TestParallelIterable.java: ## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

[GitHub] [iceberg] Heltman commented on a diff in pull request #5887: Core: Clear queue and future task when close ParallelIterable

2022-10-10 Thread GitBox
Heltman commented on code in PR #5887: URL: https://github.com/apache/iceberg/pull/5887#discussion_r991100201 ## core/src/test/java/org/apache/iceberg/io/TestParallelIterable.java: ## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

[GitHub] [iceberg] nastra commented on a diff in pull request #5887: Core: Clear queue and future task when close ParallelIterable

2022-10-10 Thread GitBox
nastra commented on code in PR #5887: URL: https://github.com/apache/iceberg/pull/5887#discussion_r991100685 ## core/src/test/java/org/apache/iceberg/util/TestParallelIterable.java: ## @@ -0,0 +1,72 @@ +/* + * + * * Licensed to the Apache Software Foundation (ASF) under one Re

[GitHub] [iceberg] Heltman commented on a diff in pull request #5887: Core: Clear queue and future task when close ParallelIterable

2022-10-10 Thread GitBox
Heltman commented on code in PR #5887: URL: https://github.com/apache/iceberg/pull/5887#discussion_r991103327 ## core/src/test/java/org/apache/iceberg/util/TestParallelIterable.java: ## @@ -0,0 +1,72 @@ +/* + * + * * Licensed to the Apache Software Foundation (ASF) under one R

[GitHub] [iceberg] sontt710 opened a new issue, #5947: [Hadoop] [Iceberg] How to change hadoop location from iceberg metadata

2022-10-10 Thread GitBox
sontt710 opened a new issue, #5947: URL: https://github.com/apache/iceberg/issues/5947 ### Query engine _No response_ ### Question Hello, I'm using Iceberg + Hadoop with Hive Metastore. After create the table, the location of table will be "location" : "hdf

[GitHub] [iceberg] hililiwei closed pull request #5029: Flink: Use Tag or Branch to scan data.

2022-10-10 Thread GitBox
hililiwei closed pull request #5029: Flink: Use Tag or Branch to scan data. URL: https://github.com/apache/iceberg/pull/5029 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [iceberg] hililiwei opened a new pull request, #5948: Flink: Use Tag or Branch to read data.

2022-10-10 Thread GitBox
hililiwei opened a new pull request, #5948: URL: https://github.com/apache/iceberg/pull/5948 ## What is the purpose of the change Scan data using a specified tag or branch. ## Brief change log Add the following syntax SQL: ```sql * SELECT * FROM sample /*+ OPTION

[GitHub] [iceberg] hililiwei closed pull request #5948: Flink: Use Tag or Branch to read data.

2022-10-10 Thread GitBox
hililiwei closed pull request #5948: Flink: Use Tag or Branch to read data. URL: https://github.com/apache/iceberg/pull/5948 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [iceberg] hililiwei opened a new pull request, #5029: Flink: Use Tag or Branch to scan data.

2022-10-10 Thread GitBox
hililiwei opened a new pull request, #5029: URL: https://github.com/apache/iceberg/pull/5029 This PR is blocked on #4428 ## What is the purpose of the change Scan data using a specified tag or branch. ## Brief change log Add the following syntax SQL: ```sql

[GitHub] [iceberg] Fokko opened a new pull request, #5949: Python: Visitor to convert Iceberg to PyArrow schema

2022-10-10 Thread GitBox
Fokko opened a new pull request, #5949: URL: https://github.com/apache/iceberg/pull/5949 This is required for manually specifying datasets: https://arrow.apache.org/docs/python/dataset.html#manual-specification-of-the-dataset From PyArrow: The `dataset()` function allows easy c

[GitHub] [iceberg] Fokko opened a new pull request, #5950: Python: Fix annotation StringType ➜ BinaryType

2022-10-10 Thread GitBox
Fokko opened a new pull request, #5950: URL: https://github.com/apache/iceberg/pull/5950 This is only a cosmetic bug, because it would take the Type from the decorator. Also, removed the types from the decorator to reduce visual noise. -- This is an automated message from the Apache Git S

[GitHub] [iceberg] hililiwei opened a new pull request, #5951: Docs: Using FlinkSQL for time travel in batch mode

2022-10-10 Thread GitBox
hililiwei opened a new pull request, #5951: URL: https://github.com/apache/iceberg/pull/5951 Using FlinkSQL for time travel in batch mode -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[GitHub] [iceberg] stevenzwu commented on pull request #4246: Flink: Add support for ResolvedSchema

2022-10-10 Thread GitBox
stevenzwu commented on PR #4246: URL: https://github.com/apache/iceberg/pull/4246#issuecomment-1273477159 @hililiwei I saw @hunter-cloud09 closed PR #5938 and moved the discussion to this PR. Can you rebase the branch so that we can move forward on this one? -- This is an automated messa

[GitHub] [iceberg] huaxingao commented on pull request #5872: push down min/max/count to iceberg

2022-10-10 Thread GitBox
huaxingao commented on PR #5872: URL: https://github.com/apache/iceberg/pull/5872#issuecomment-1273482239 @rdblue Thank you very much for your detailed review! I will address the comments. I will split this PR and get the expression changes ready first. -- This is an automated message fro

[GitHub] [iceberg] sumeetgajjar opened a new pull request, #5952: Spark: Perform nullability check on metastore and spark instance before invoking stop

2022-10-10 Thread GitBox
sumeetgajjar opened a new pull request, #5952: URL: https://github.com/apache/iceberg/pull/5952 In this PR, we fix `SparkTestBase#stopMetastoreAndSpark`. When `SparkTestBase#startMetastoreAndSpark` throws an exception while starting embedded HMS or while creating `SparkSession`, the

[GitHub] [iceberg] sumeetgajjar commented on pull request #5952: Spark: Perform nullability check on metastore and spark instance before invoking stop

2022-10-10 Thread GitBox
sumeetgajjar commented on PR #5952: URL: https://github.com/apache/iceberg/pull/5952#issuecomment-1273528765 Hi @Fokko @nastra @pvary, Can you please review this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [iceberg] matt-slalom commented on issue #3044: Unable to use GlueCatalog in flink environments without hadoop

2022-10-10 Thread GitBox
matt-slalom commented on issue #3044: URL: https://github.com/apache/iceberg/issues/3044#issuecomment-1273637838 > Last one-two days I wrote a streaming job and packaged as a fat jar. I had to add the minimal hadoop dependency and in the code I had to initialize the hadoop conf with the con

[GitHub] [iceberg-docs] danielcweeks merged pull request #166: https://github.com/apache/iceberg/issues/5914

2022-10-10 Thread GitBox
danielcweeks merged PR #166: URL: https://github.com/apache/iceberg-docs/pull/166 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ice

[GitHub] [iceberg-docs] samredai commented on pull request #152: Add Cloudera blog about using Iceberg in CDP Open Lakehouse

2022-10-10 Thread GitBox
samredai commented on PR #152: URL: https://github.com/apache/iceberg-docs/pull/152#issuecomment-1273682168 Hey @gaborkaszab, sorry for the delay here. Can you rebase this and I'll get someone to merge it in? Thanks! -- This is an automated message from the Apache Git Service. To respond

[GitHub] [iceberg] danielcweeks merged pull request #5862: Core: Deflake TestManifestCaching.testWeakFileIOReferenceCleanUp

2022-10-10 Thread GitBox
danielcweeks merged PR #5862: URL: https://github.com/apache/iceberg/pull/5862 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

[GitHub] [iceberg] danielcweeks closed issue #5861: Test flakiness: TestManifestCaching > testWeakFileIOReferenceCleanUp

2022-10-10 Thread GitBox
danielcweeks closed issue #5861: Test flakiness: TestManifestCaching > testWeakFileIOReferenceCleanUp URL: https://github.com/apache/iceberg/issues/5861 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [iceberg] brucearctor commented on issue #5914: Add "Good First Issue" suggestion/link in Contributing docs

2022-10-10 Thread GitBox
brucearctor commented on issue #5914: URL: https://github.com/apache/iceberg/issues/5914#issuecomment-1273683687 merged -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

[GitHub] [iceberg] brucearctor closed issue #5914: Add "Good First Issue" suggestion/link in Contributing docs

2022-10-10 Thread GitBox
brucearctor closed issue #5914: Add "Good First Issue" suggestion/link in Contributing docs URL: https://github.com/apache/iceberg/issues/5914 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [iceberg] JonasJ-ap commented on a diff in pull request #5939: AWS: Fix NotSerializableException when using AssumeRoleAwsClientFactory in Spark

2022-10-10 Thread GitBox
JonasJ-ap commented on code in PR #5939: URL: https://github.com/apache/iceberg/pull/5939#discussion_r991570121 ## aws/src/test/java/org/apache/iceberg/aws/TestAwsClientFactories.java: ## @@ -71,6 +73,53 @@ public void testS3FileIoCredentialsVerification() { () -> AwsCl

[GitHub] [iceberg] JonasJ-ap commented on a diff in pull request #5939: AWS: Fix NotSerializableException when using AssumeRoleAwsClientFactory in Spark

2022-10-10 Thread GitBox
JonasJ-ap commented on code in PR #5939: URL: https://github.com/apache/iceberg/pull/5939#discussion_r991570121 ## aws/src/test/java/org/apache/iceberg/aws/TestAwsClientFactories.java: ## @@ -71,6 +73,53 @@ public void testS3FileIoCredentialsVerification() { () -> AwsCl

[GitHub] [iceberg] Fokko opened a new pull request, #5953: Python: Simplify the logic of caching

2022-10-10 Thread GitBox
Fokko opened a new pull request, #5953: URL: https://github.com/apache/iceberg/pull/5953 We don't need to have the `_scheme_to_fs` dict and moving the cache outside of the class makes it a bit simpler. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [iceberg] the-other-tim-brown commented on pull request #5932: Parquet: support nested fields when assigning fallback ids

2022-10-10 Thread GitBox
the-other-tim-brown commented on PR #5932: URL: https://github.com/apache/iceberg/pull/5932#issuecomment-1273885025 I see that I just need to set the `TableProperties.DEFAULT_NAME_MAPPING` to `NameMappingParser.toJson(MappingUtil.create(transaction.table().schema())` in the properties when

[GitHub] [iceberg] matt-slalom commented on issue #3044: Unable to use GlueCatalog in flink environments without hadoop

2022-10-10 Thread GitBox
matt-slalom commented on issue #3044: URL: https://github.com/apache/iceberg/issues/3044#issuecomment-1273900117 I found this thread while trying to make Flink, Iceberg, and Glue work together without the Hadoop dependencies, so I'm hoping this is useful for someone else in the future. To a

[GitHub] [iceberg] stevenzwu commented on a diff in pull request #4925: API: Add view interfaces

2022-10-10 Thread GitBox
stevenzwu commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r991712969 ## api/src/main/java/org/apache/iceberg/view/ViewVersion.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contr

[GitHub] [iceberg] stevenzwu commented on a diff in pull request #4925: API: Add view interfaces

2022-10-10 Thread GitBox
stevenzwu commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r991712969 ## api/src/main/java/org/apache/iceberg/view/ViewVersion.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contr

[GitHub] [iceberg] github-actions[bot] commented on issue #4549: HIVE_METASTORE_ERROR: Table storage descriptor is missing SerDe info - when query a view using an Iceberg table on Athena

2022-10-10 Thread GitBox
github-actions[bot] commented on issue #4549: URL: https://github.com/apache/iceberg/issues/4549#issuecomment-1273935222 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

[GitHub] [iceberg] github-actions[bot] commented on issue #4542: Schema Evolution exception: too many data columns

2022-10-10 Thread GitBox
github-actions[bot] commented on issue #4542: URL: https://github.com/apache/iceberg/issues/4542#issuecomment-1273935234 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #5877: Refactor commit lock mechanism from HiveTableOperations

2022-10-10 Thread GitBox
szehon-ho commented on code in PR #5877: URL: https://github.com/apache/iceberg/pull/5877#discussion_r991716721 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCommitLock.java: ## @@ -0,0 +1,306 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #5877: Refactor commit lock mechanism from HiveTableOperations

2022-10-10 Thread GitBox
szehon-ho commented on code in PR #5877: URL: https://github.com/apache/iceberg/pull/5877#discussion_r991726735 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCommitLock.java: ## @@ -0,0 +1,306 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #5877: Refactor commit lock mechanism from HiveTableOperations

2022-10-10 Thread GitBox
szehon-ho commented on code in PR #5877: URL: https://github.com/apache/iceberg/pull/5877#discussion_r991731360 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCommitLock.java: ## @@ -0,0 +1,306 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [iceberg] aokolnychyi commented on pull request #2276: Core: Add option to combine tasks by partition

2022-10-10 Thread GitBox
aokolnychyi commented on PR #2276: URL: https://github.com/apache/iceberg/pull/2276#issuecomment-1273967116 I think this is an essential PR to support storage-partitioned joins in Spark 3.3. It would be great to rebase it. Here is what I noted based on our discussion earlier: - We

[GitHub] [iceberg] zinking commented on pull request #5951: Docs: Using FlinkSQL for time travel in batch mode

2022-10-10 Thread GitBox
zinking commented on PR #5951: URL: https://github.com/apache/iceberg/pull/5951#issuecomment-1273984347 is this supported in flink already? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [iceberg] chenjunjiedada commented on pull request #5760: [wip] Core: Add mininum sequence number for delete file

2022-10-10 Thread GitBox
chenjunjiedada commented on PR #5760: URL: https://github.com/apache/iceberg/pull/5760#issuecomment-1273992674 @flyrain Just reread your idea, one question is how do you handle the #seq if the writer is an upsert writer? The #seq is unknown when committing. -- This is an automated messa

[GitHub] [iceberg] chenjunjiedada commented on a diff in pull request #5913: Core: Add dataSequenceNumber to ManifestEntry

2022-10-10 Thread GitBox
chenjunjiedada commented on code in PR #5913: URL: https://github.com/apache/iceberg/pull/5913#discussion_r991754698 ## core/src/main/java/org/apache/iceberg/GenericManifestEntry.java: ## @@ -44,14 +45,16 @@ private GenericManifestEntry(GenericManifestEntry toCopy, boolean full

[GitHub] [iceberg] hunter-cloud09 opened a new pull request, #5954: refactor(flink): update deprecated code in flink

2022-10-10 Thread GitBox
hunter-cloud09 opened a new pull request, #5954: URL: https://github.com/apache/iceberg/pull/5954 update deprecated code in FLIP-164 re #5937 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [iceberg] hunter-cloud09 commented on pull request #4246: Flink: Add support for ResolvedSchema

2022-10-10 Thread GitBox
hunter-cloud09 commented on PR #4246: URL: https://github.com/apache/iceberg/pull/4246#issuecomment-1274032181 I accidentally deleted my branch yesterday, which caused the pr to be closed. I reopened a [pr](https://github.com/apache/iceberg/pull/5954), and we can discuss it together. @steve

[GitHub] [iceberg] jackye1995 commented on a diff in pull request #5870: Python: Add GlueCatalog Table Creation and Loading Support

2022-10-10 Thread GitBox
jackye1995 commented on code in PR #5870: URL: https://github.com/apache/iceberg/pull/5870#discussion_r991781949 ## python/pyiceberg/catalog/glue.py: ## @@ -0,0 +1,247 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

[GitHub] [iceberg] jackye1995 commented on a diff in pull request #5870: Python: Add GlueCatalog Table Creation and Loading Support

2022-10-10 Thread GitBox
jackye1995 commented on code in PR #5870: URL: https://github.com/apache/iceberg/pull/5870#discussion_r991782678 ## python/pyproject.toml: ## @@ -63,6 +63,8 @@ python-snappy = { version = "^0.6.1", optional = true } thrift = { version = "^0.16.0", optional = true } s3fs = {

[GitHub] [iceberg] jackye1995 commented on a diff in pull request #5870: Python: Add GlueCatalog Table Creation and Loading Support

2022-10-10 Thread GitBox
jackye1995 commented on code in PR #5870: URL: https://github.com/apache/iceberg/pull/5870#discussion_r991785153 ## python/pyiceberg/catalog/glue.py: ## @@ -0,0 +1,247 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

[GitHub] [iceberg] jackye1995 commented on a diff in pull request #5870: Python: Add GlueCatalog Table Creation and Loading Support

2022-10-10 Thread GitBox
jackye1995 commented on code in PR #5870: URL: https://github.com/apache/iceberg/pull/5870#discussion_r991785324 ## python/pyiceberg/catalog/glue.py: ## @@ -0,0 +1,247 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

[GitHub] [iceberg] jzhuge commented on a diff in pull request #4925: API: Add view interfaces

2022-10-10 Thread GitBox
jzhuge commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r991822977 ## api/src/main/java/org/apache/iceberg/catalog/ViewCatalog.java: ## @@ -0,0 +1,148 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more cont

[GitHub] [iceberg] jzhuge commented on a diff in pull request #4925: API: Add view interfaces

2022-10-10 Thread GitBox
jzhuge commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r991823367 ## api/src/main/java/org/apache/iceberg/catalog/ViewCatalog.java: ## @@ -0,0 +1,148 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more cont

[GitHub] [iceberg] jzhuge commented on a diff in pull request #4925: API: Add view interfaces

2022-10-10 Thread GitBox
jzhuge commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r991826408 ## api/src/main/java/org/apache/iceberg/catalog/ViewCatalog.java: ## @@ -0,0 +1,153 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more cont

[GitHub] [iceberg] jzhuge commented on a diff in pull request #4925: API: Add view interfaces

2022-10-10 Thread GitBox
jzhuge commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r991828199 ## api/src/main/java/org/apache/iceberg/view/ViewVersion.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contribu

[GitHub] [iceberg] jzhuge commented on a diff in pull request #4925: API: Add view interfaces

2022-10-10 Thread GitBox
jzhuge commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r991840645 ## api/src/main/java/org/apache/iceberg/view/ViewVersion.java: ## @@ -0,0 +1,64 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contribu

[GitHub] [iceberg] singhpk234 commented on a diff in pull request #5939: AWS: Fix NotSerializableException when using AssumeRoleAwsClientFactory in Spark

2022-10-10 Thread GitBox
singhpk234 commented on code in PR #5939: URL: https://github.com/apache/iceberg/pull/5939#discussion_r991828441 ## aws/src/test/java/org/apache/iceberg/aws/TestAwsClientFactories.java: ## @@ -71,6 +73,53 @@ public void testS3FileIoCredentialsVerification() { () -> AwsC

[GitHub] [iceberg] jzhuge commented on a diff in pull request #4925: API: Add view interfaces

2022-10-10 Thread GitBox
jzhuge commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r991848646 ## api/src/main/java/org/apache/iceberg/catalog/ViewCatalog.java: ## @@ -0,0 +1,148 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more cont

[GitHub] [iceberg] JonasJ-ap commented on a diff in pull request #5939: AWS: Fix NotSerializableException when using AssumeRoleAwsClientFactory in Spark

2022-10-10 Thread GitBox
JonasJ-ap commented on code in PR #5939: URL: https://github.com/apache/iceberg/pull/5939#discussion_r991864782 ## aws/src/test/java/org/apache/iceberg/aws/TestAwsClientFactories.java: ## @@ -71,6 +73,53 @@ public void testS3FileIoCredentialsVerification() { () -> AwsCl

[GitHub] [iceberg] 95liu commented on issue #5945: Read Iceberg Table Bug(cannot find field start_date from [org.apache.iceberg.mr.hive.serde.objectinspector.IcebergRecordObj)

2022-10-10 Thread GitBox
95liu commented on issue #5945: URL: https://github.com/apache/iceberg/issues/5945#issuecomment-1274169053 select soqte1.org_no from (select eid , credit_no , org_no , name , start_date , reg_no