Repository: spark Updated Branches: refs/heads/master 8d6207206 -> 358692932
[SPARK-14278][SQL] Initialize columnar batch with proper memory mode ## What changes were proposed in this pull request? Fixes a minor bug in the record reader constructor that was possibly introduced during refactoring. ## How was this patch tested? N/A Author: Sameer Agarwal <[email protected]> Closes #12070 from sameeragarwal/vectorized-rr. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/35869293 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/35869293 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/35869293 Branch: refs/heads/master Commit: 3586929320bba8b8d09c2a451189f76821fdfba4 Parents: 8d62072 Author: Sameer Agarwal <[email protected]> Authored: Thu Mar 31 11:56:28 2016 -0700 Committer: Yin Huai <[email protected]> Committed: Thu Mar 31 11:56:28 2016 -0700 ---------------------------------------------------------------------- .../datasources/parquet/VectorizedParquetRecordReader.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/35869293/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedParquetRecordReader.java ---------------------------------------------------------------------- diff --git a/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedParquetRecordReader.java b/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedParquetRecordReader.java index 0bdf4aa..a0b6276 100644 --- a/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedParquetRecordReader.java +++ b/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedParquetRecordReader.java @@ -191,7 +191,7 @@ public class VectorizedParquetRecordReader extends SpecificParquetRecordReaderBa } } - columnarBatch = ColumnarBatch.allocate(batchSchema); + columnarBatch = ColumnarBatch.allocate(batchSchema, memMode); if (partitionColumns != null) { int partitionIdx = sparkSchema.fields().length; for (int i = 0; i < partitionColumns.fields().length; i++) { --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
