This is an automated email from the ASF dual-hosted git repository.
wenchen pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
from 65755bde3cb8 [SPARK-54036][INFRA] Add `build_python_3.11_macos26.yml`
GitHub Action job
add ec944cc52c8b [SPARK-53659][SQL] Infer Variant shredding schema when
writing to Parquet
No new revisions were added by this update.
Summary of changes:
.../org/apache/spark/types/variant/Variant.java | 5 +
.../org/apache/spark/sql/internal/SQLConf.scala | 25 +
.../parquet/InferVariantShreddingSchema.scala | 385 +++++++++++++
.../datasources/parquet/ParquetOptions.scala | 8 +
.../ParquetOutputWriterWithVariantShredding.scala | 120 ++++
.../datasources/parquet/ParquetUtils.scala | 28 +-
.../parquet/VariantInferShreddingSuite.scala | 634 +++++++++++++++++++++
7 files changed, 1204 insertions(+), 1 deletion(-)
create mode 100644
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/InferVariantShreddingSchema.scala
create mode 100644
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetOutputWriterWithVariantShredding.scala
create mode 100644
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/VariantInferShreddingSuite.scala
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
