[GitHub] [iceberg] dependabot[bot] opened a new pull request, #6082: Build: Bump zstandard from 0.18.0 to 0.19.0 in /python

2022-10-29 Thread GitBox
dependabot[bot] opened a new pull request, #6082: URL: https://github.com/apache/iceberg/pull/6082 Bumps [zstandard](https://github.com/indygreg/python-zstandard) from 0.18.0 to 0.19.0. Release notes Sourced from https://github.com/indygreg/python-zstandard/releases";>zstandard's

[GitHub] [iceberg] dependabot[bot] opened a new pull request, #6081: Build: Bump pyarrow from 9.0.0 to 10.0.0 in /python

2022-10-29 Thread GitBox
dependabot[bot] opened a new pull request, #6081: URL: https://github.com/apache/iceberg/pull/6081 Bumps [pyarrow](https://github.com/apache/arrow) from 9.0.0 to 10.0.0. Commits https://github.com/apache/arrow/commit/89f9a0948961f6e94f1ef5e4f310b707d22a3c11";>89f9a09 MINOR: [Re

[GitHub] [iceberg] dependabot[bot] opened a new pull request, #6080: Build: Bump pytest from 7.1.3 to 7.2.0 in /python

2022-10-29 Thread GitBox
dependabot[bot] opened a new pull request, #6080: URL: https://github.com/apache/iceberg/pull/6080 Bumps [pytest](https://github.com/pytest-dev/pytest) from 7.1.3 to 7.2.0. Release notes Sourced from https://github.com/pytest-dev/pytest/releases";>pytest's releases. 7.2.0

[GitHub] [iceberg] dependabot[bot] opened a new pull request, #6079: Build: Bump fastavro from 1.6.1 to 1.7.0 in /python

2022-10-29 Thread GitBox
dependabot[bot] opened a new pull request, #6079: URL: https://github.com/apache/iceberg/pull/6079 Bumps [fastavro](https://github.com/fastavro/fastavro) from 1.6.1 to 1.7.0. Changelog Sourced from https://github.com/fastavro/fastavro/blob/master/ChangeLog";>fastavro's changelog.

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #5150: Spark Integration to read from Snapshot ref

2022-10-29 Thread GitBox
amogh-jahagirdar commented on code in PR #5150: URL: https://github.com/apache/iceberg/pull/5150#discussion_r1008787248 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkBatchQueryScan.java: ## @@ -270,6 +282,8 @@ && readSchema().equals(that.readSchema())

[GitHub] [iceberg] stevenzwu commented on a diff in pull request #5984: Core, API: Support incremental scanning with branch

2022-10-29 Thread GitBox
stevenzwu commented on code in PR #5984: URL: https://github.com/apache/iceberg/pull/5984#discussion_r1008784212 ## api/src/main/java/org/apache/iceberg/IncrementalScan.java: ## @@ -21,6 +21,23 @@ /** API for configuring an incremental scan. */ public interface IncrementalScan

[GitHub] [iceberg] stevenzwu commented on a diff in pull request #5984: Core, API: Support incremental scanning with branch

2022-10-29 Thread GitBox
stevenzwu commented on code in PR #5984: URL: https://github.com/apache/iceberg/pull/5984#discussion_r1008784212 ## api/src/main/java/org/apache/iceberg/IncrementalScan.java: ## @@ -21,6 +21,23 @@ /** API for configuring an incremental scan. */ public interface IncrementalScan

[GitHub] [iceberg] ajantha-bhat commented on pull request #6048: Docs: Fix broken link for puffin in Spec

2022-10-29 Thread GitBox
ajantha-bhat commented on PR #6048: URL: https://github.com/apache/iceberg/pull/6048#issuecomment-1296078419 > I'll look into if there's a hugo feature for converting links (We could convert the GitHub-friendly links to site links at build time). Thanks. -- This is an automated mes

[GitHub] [iceberg] ajantha-bhat closed pull request #6048: Docs: Fix broken link for puffin in Spec

2022-10-29 Thread GitBox
ajantha-bhat closed pull request #6048: Docs: Fix broken link for puffin in Spec URL: https://github.com/apache/iceberg/pull/6048 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [iceberg] namrathamyske commented on a diff in pull request #5150: Spark Integration to read from Snapshot ref

2022-10-29 Thread GitBox
namrathamyske commented on code in PR #5150: URL: https://github.com/apache/iceberg/pull/5150#discussion_r1008783463 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkScanBuilder.java: ## @@ -226,6 +222,12 @@ public Scan build() { scan = scan.asOfTime

[GitHub] [iceberg] namrathamyske commented on a diff in pull request #5150: Spark Integration to read from Snapshot ref

2022-10-29 Thread GitBox
namrathamyske commented on code in PR #5150: URL: https://github.com/apache/iceberg/pull/5150#discussion_r1008782974 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkBatchQueryScan.java: ## @@ -270,6 +282,8 @@ && readSchema().equals(that.readSchema())

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #5150: Spark Integration to read from Snapshot ref

2022-10-29 Thread GitBox
amogh-jahagirdar commented on code in PR #5150: URL: https://github.com/apache/iceberg/pull/5150#discussion_r1008768894 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkScanBuilder.java: ## @@ -226,6 +222,12 @@ public Scan build() { scan = scan.asOfT

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #5150: Spark Integration to read from Snapshot ref

2022-10-29 Thread GitBox
amogh-jahagirdar commented on code in PR #5150: URL: https://github.com/apache/iceberg/pull/5150#discussion_r1008768894 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkScanBuilder.java: ## @@ -226,6 +222,12 @@ public Scan build() { scan = scan.asOfT

[GitHub] [iceberg] Fokko opened a new pull request, #6078: Python: Pin versions explicitly

2022-10-29 Thread GitBox
Fokko opened a new pull request, #6078: URL: https://github.com/apache/iceberg/pull/6078 PR https://github.com/apache/iceberg/pull/6076 fails because the poetry update command also bumped the version of FastAvro. I don't think this is wanted behavior, therefore I suggest pinning the

[GitHub] [iceberg] aokolnychyi commented on a diff in pull request #6077: Spark 3.3: Use separate scan during file filtering in copy-on-write operations

2022-10-29 Thread GitBox
aokolnychyi commented on code in PR #6077: URL: https://github.com/apache/iceberg/pull/6077#discussion_r1008756137 ## spark/v3.3/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/SparkRowLevelOperationsTestBase.java: ## @@ -57,6 +60,8 @@ public abstract class Sp

[GitHub] [iceberg] aokolnychyi commented on a diff in pull request #6077: Spark 3.3: Use separate scan during file filtering in copy-on-write operations

2022-10-29 Thread GitBox
aokolnychyi commented on code in PR #6077: URL: https://github.com/apache/iceberg/pull/6077#discussion_r1008755739 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkCopyOnWriteScan.java: ## @@ -102,6 +103,14 @@ public NamedReference[] filterAttributes() {

[GitHub] [iceberg] aokolnychyi commented on a diff in pull request #6077: Spark 3.3: Use separate scan during file filtering in copy-on-write operations

2022-10-29 Thread GitBox
aokolnychyi commented on code in PR #6077: URL: https://github.com/apache/iceberg/pull/6077#discussion_r1008755707 ## spark/v3.3/spark-extensions/src/main/scala/org/apache/spark/sql/execution/dynamicpruning/RowLevelCommandDynamicPruning.scala: ## @@ -69,27 +73,34 @@ case class R

[GitHub] [iceberg] aokolnychyi opened a new pull request, #6077: Spark 3.3: Use separate scan during file filtering in copy-on-write operations

2022-10-29 Thread GitBox
aokolnychyi opened a new pull request, #6077: URL: https://github.com/apache/iceberg/pull/6077 This PR enables using a separate scan during file filtering in row-level copy-on-write operations. This means the runtime filter subquery will now be able to push down filters into row groups as w

[GitHub] [iceberg] samredai commented on pull request #6048: Docs: Fix broken link for puffin in Spec

2022-10-29 Thread GitBox
samredai commented on PR #6048: URL: https://github.com/apache/iceberg/pull/6048#issuecomment-1295884052 > I want to have links that work in both website and offline markdown file. @ajantha-bhat this might be difficult to do without changing up at least the way we deploy docs. I'll lo

[GitHub] [iceberg] Fokko opened a new pull request, #6076: Python: Replace mmh3 with mmhash3

2022-10-29 Thread GitBox
Fokko opened a new pull request, #6076: URL: https://github.com/apache/iceberg/pull/6076 The mmh3 project is abandoned, and not being maintained. This results in that there are no wheels (compiled binaries) available for newer versions of Python (3.10 and 3.11). I've forked the repo,

[GitHub] [iceberg] ConeyLiu commented on a diff in pull request #4577: Fixes read metadata table failed due to illegal character

2022-10-29 Thread GitBox
ConeyLiu commented on code in PR #4577: URL: https://github.com/apache/iceberg/pull/4577#discussion_r1008683069 ## core/src/test/java/org/apache/iceberg/TestMetadataTableScans.java: ## @@ -978,6 +1091,32 @@ private Set expectedManifestListPaths(Iterable snapshots, Long

[GitHub] [iceberg] ConeyLiu commented on a diff in pull request #4577: Fixes read metadata table failed due to illegal character

2022-10-29 Thread GitBox
ConeyLiu commented on code in PR #4577: URL: https://github.com/apache/iceberg/pull/4577#discussion_r1008682913 ## core/src/main/java/org/apache/iceberg/avro/BuildAvroProjection.java: ## @@ -107,13 +107,15 @@ public Schema record(Schema record, List names, Iterable s

[GitHub] [iceberg] hililiwei opened a new pull request, #6075: Flink 1.15: Support change log scan task

2022-10-29 Thread GitBox
hililiwei opened a new pull request, #6075: URL: https://github.com/apache/iceberg/pull/6075 ## What is the purpose of the change Support for Changlog scanning in the new Flink Source(FLIP-27) It can be turned on in the following ways: ``` SELECT * FROM tableName /*+ O

[GitHub] [iceberg] ConeyLiu commented on a diff in pull request #5632: Core: Avoid reading ManifestFile when create ManifestReader

2022-10-29 Thread GitBox
ConeyLiu commented on code in PR #5632: URL: https://github.com/apache/iceberg/pull/5632#discussion_r1008680666 ## core/src/main/java/org/apache/iceberg/ManifestReader.java: ## @@ -101,20 +100,32 @@ private String fileClass() { protected ManifestReader( InputFile fil

[GitHub] [iceberg] singhpk234 commented on a diff in pull request #6034: Python: GlueCatalog Full Implementation

2022-10-29 Thread GitBox
singhpk234 commented on code in PR #6034: URL: https://github.com/apache/iceberg/pull/6034#discussion_r1008612703 ## python/pyiceberg/catalog/glue.py: ## @@ -0,0 +1,453 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

[GitHub] [iceberg] singhpk234 commented on a diff in pull request #6034: Python: GlueCatalog Full Implementation

2022-10-29 Thread GitBox
singhpk234 commented on code in PR #6034: URL: https://github.com/apache/iceberg/pull/6034#discussion_r1008659414 ## python/pyiceberg/catalog/glue.py: ## @@ -0,0 +1,453 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

[GitHub] [iceberg] luoyuxia commented on issue #3395: While I tring to select table1 join table2,if fields of table2 was choosed, error like "java.lang.ArrayIndexOutOfBoundsException: 6" occurred

2022-10-29 Thread GitBox
luoyuxia commented on issue #3395: URL: https://github.com/apache/iceberg/issues/3395#issuecomment-1295758977 It should be fixed by latest Flink. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t