This is an automated email from the ASF dual-hosted git repository.
srowen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 7e8aafd2c0f [SPARK-45334][SQL] Remove misleading comment in
parquetSchemaConverter
7e8aafd2c0f is described below
commit 7e8aafd2c0f1f6fcd03a69afe2b85fd3fda95d20
Author: lanmengran1 <[email protected]>
AuthorDate: Tue Sep 26 21:01:02 2023 -0500
[SPARK-45334][SQL] Remove misleading comment in parquetSchemaConverter
### What changes were proposed in this pull request?
Remove one line of comment, the detail info is described in JIRA
https://issues.apache.org/jira/browse/SPARK-45334
### Why are the changes needed?
The comment is outdated and misleading.
- the parquet-hive module has been removed from the parquet-mr project
https://issues.apache.org/jira/browse/PARQUET-1676
- Hive always uses "array_element" as the name
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
No need
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #43119 from
amoylan2/remove_misleading_comment_in_parquetSchemaConverter.
Authored-by: lanmengran1 <[email protected]>
Signed-off-by: Sean Owen <[email protected]>
---
.../spark/sql/execution/datasources/parquet/ParquetSchemaConverter.scala | 1 -
1 file changed, 1 deletion(-)
diff --git
a/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaConverter.scala
b/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaConverter.scala
index 9c9e7ce729c..eedd165278a 100644
---
a/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaConverter.scala
+++
b/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaConverter.scala
@@ -646,7 +646,6 @@ class SparkToParquetSchemaConverter(
.buildGroup(repetition).as(LogicalTypeAnnotation.listType())
.addField(Types
.buildGroup(REPEATED)
- // "array" is the name chosen by parquet-hive (1.7.0 and prior
version)
.addField(convertField(StructField("array", elementType,
nullable)))
.named("bag"))
.named(field.name)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
