[1/4] spark git commit: [SPARK-15543][SQL] Rename DefaultSources to make them more self-describing

2016-05-25 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 bcad1d13f -> b3ee53b84 http://git-wip-us.apache.org/repos/asf/spark/blob/b3ee53b8/sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala -- diff --git

[3/4] spark git commit: [SPARK-15543][SQL] Rename DefaultSources to make them more self-describing

2016-05-25 Thread rxin
http://git-wip-us.apache.org/repos/asf/spark/blob/361ebc28/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala -- diff --git a/sql/core/src/main/scala/org/apache/spark/sql/execution/d

[1/4] spark git commit: [SPARK-15543][SQL] Rename DefaultSources to make them more self-describing

2016-05-25 Thread rxin
Repository: spark Updated Branches: refs/heads/master dfc9fc02c -> 361ebc282 http://git-wip-us.apache.org/repos/asf/spark/blob/361ebc28/sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala -- diff --git a/sq

[2/4] spark git commit: [SPARK-15543][SQL] Rename DefaultSources to make them more self-describing

2016-05-25 Thread rxin
http://git-wip-us.apache.org/repos/asf/spark/blob/b3ee53b8/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/text/DefaultSource.scala -- diff --git a/sql/core/src/main/scala/org/apache/spark/sql/execution/datasour

[4/4] spark git commit: [SPARK-15543][SQL] Rename DefaultSources to make them more self-describing

2016-05-25 Thread rxin
[SPARK-15543][SQL] Rename DefaultSources to make them more self-describing ## What changes were proposed in this pull request? This patch renames various DefaultSources to make their names more self-describing. The choice of "DefaultSource" was from the days when we did not have a good way to sp

[3/4] spark git commit: [SPARK-15543][SQL] Rename DefaultSources to make them more self-describing

2016-05-25 Thread rxin
http://git-wip-us.apache.org/repos/asf/spark/blob/b3ee53b8/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala -- diff --git a/sql/core/src/main/scala/org/apache/spark/sql/execution/d

[4/4] spark git commit: [SPARK-15543][SQL] Rename DefaultSources to make them more self-describing

2016-05-25 Thread rxin
[SPARK-15543][SQL] Rename DefaultSources to make them more self-describing ## What changes were proposed in this pull request? This patch renames various DefaultSources to make their names more self-describing. The choice of "DefaultSource" was from the days when we did not have a good way to sp

[2/4] spark git commit: [SPARK-15543][SQL] Rename DefaultSources to make them more self-describing

2016-05-25 Thread rxin
http://git-wip-us.apache.org/repos/asf/spark/blob/361ebc28/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/text/DefaultSource.scala -- diff --git a/sql/core/src/main/scala/org/apache/spark/sql/execution/datasour

spark git commit: [SPARK-10372] [CORE] basic test framework for entire spark scheduler

2016-05-25 Thread irashid
Repository: spark Updated Branches: refs/heads/master 06bae8af1 -> dfc9fc02c [SPARK-10372] [CORE] basic test framework for entire spark scheduler This is a basic framework for testing the entire scheduler. The tests this adds aren't very interesting -- the point of this PR is just to setup t

spark git commit: [SPARK-15439][SPARKR] Failed to run unit test in SparkR

2016-05-25 Thread shivaram
Repository: spark Updated Branches: refs/heads/branch-2.0 15a2dba66 -> bcad1d13f [SPARK-15439][SPARKR] Failed to run unit test in SparkR ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) There are some failures when running SparkR unit tests. In

spark git commit: [SPARK-15439][SPARKR] Failed to run unit test in SparkR

2016-05-25 Thread shivaram
Repository: spark Updated Branches: refs/heads/master 06ed1fa3e -> 06bae8af1 [SPARK-15439][SPARKR] Failed to run unit test in SparkR ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) There are some failures when running SparkR unit tests. In thi

spark git commit: [SPARK-15533][SQL] Deprecate Dataset.explode

2016-05-25 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 733cb44e3 -> 15a2dba66 [SPARK-15533][SQL] Deprecate Dataset.explode ## What changes were proposed in this pull request? This patch deprecates `Dataset.explode` and documents appropriate workarounds to use `flatMap()` or `functions.exp

spark git commit: [SPARK-15533][SQL] Deprecate Dataset.explode

2016-05-25 Thread rxin
Repository: spark Updated Branches: refs/heads/master 527499b62 -> 06ed1fa3e [SPARK-15533][SQL] Deprecate Dataset.explode ## What changes were proposed in this pull request? This patch deprecates `Dataset.explode` and documents appropriate workarounds to use `flatMap()` or `functions.explode

spark git commit: [SPARK-15525][SQL][BUILD] Upgrade ANTLR4 SBT plugin

2016-05-25 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 347acc4ea -> 733cb44e3 [SPARK-15525][SQL][BUILD] Upgrade ANTLR4 SBT plugin ## What changes were proposed in this pull request? The ANTLR4 SBT plugin has been moved from its own repo to one on bintray. The version was also changed from

spark git commit: [SPARK-15525][SQL][BUILD] Upgrade ANTLR4 SBT plugin

2016-05-25 Thread rxin
Repository: spark Updated Branches: refs/heads/master ee682fe29 -> 527499b62 [SPARK-15525][SQL][BUILD] Upgrade ANTLR4 SBT plugin ## What changes were proposed in this pull request? The ANTLR4 SBT plugin has been moved from its own repo to one on bintray. The version was also changed from `0.7

spark git commit: [SPARK-15534][SPARK-15535][SQL] Truncate table fixes

2016-05-25 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 589cce93c -> ee682fe29 [SPARK-15534][SPARK-15535][SQL] Truncate table fixes ## What changes were proposed in this pull request? Two changes: - When things fail, `TRUNCATE TABLE` just returns nothing. Instead, we should throw exceptions. -

spark git commit: [SPARK-15534][SPARK-15535][SQL] Truncate table fixes

2016-05-25 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-2.0 0064a4dcb -> 347acc4ea [SPARK-15534][SPARK-15535][SQL] Truncate table fixes ## What changes were proposed in this pull request? Two changes: - When things fail, `TRUNCATE TABLE` just returns nothing. Instead, we should throw exception

spark git commit: Log warnings for numIterations * miniBatchFraction < 1.0

2016-05-25 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 f7158c482 -> 0064a4dcb Log warnings for numIterations * miniBatchFraction < 1.0 ## What changes were proposed in this pull request? Add a warning log for the case that `numIterations * miniBatchFraction <1.0` during gradient descent.

spark git commit: Log warnings for numIterations * miniBatchFraction < 1.0

2016-05-25 Thread srowen
Repository: spark Updated Branches: refs/heads/master 9c297df3d -> 589cce93c Log warnings for numIterations * miniBatchFraction < 1.0 ## What changes were proposed in this pull request? Add a warning log for the case that `numIterations * miniBatchFraction <1.0` during gradient descent. If t

spark git commit: [MINOR] [PYSPARK] [EXAMPLES] Changed examples to use SparkSession.sparkContext instead of _sc

2016-05-25 Thread davies
Repository: spark Updated Branches: refs/heads/branch-2.0 8629537cc -> f7158c482 [MINOR] [PYSPARK] [EXAMPLES] Changed examples to use SparkSession.sparkContext instead of _sc ## What changes were proposed in this pull request? Some PySpark examples need a SparkContext and get it by accessing

spark git commit: [MINOR] [PYSPARK] [EXAMPLES] Changed examples to use SparkSession.sparkContext instead of _sc

2016-05-25 Thread davies
Repository: spark Updated Branches: refs/heads/master 698ef762f -> 9c297df3d [MINOR] [PYSPARK] [EXAMPLES] Changed examples to use SparkSession.sparkContext instead of _sc ## What changes were proposed in this pull request? Some PySpark examples need a SparkContext and get it by accessing _sc

spark git commit: [SPARK-14269][SCHEDULER] Eliminate unnecessary submitStage() call.

2016-05-25 Thread kayousterhout
Repository: spark Updated Branches: refs/heads/master c875d81a3 -> 698ef762f [SPARK-14269][SCHEDULER] Eliminate unnecessary submitStage() call. ## What changes were proposed in this pull request? Currently a method `submitStage()` for waiting stages is called on every iteration of the event

spark git commit: [SPARK-15493][SQL] default QuoteEscapingEnabled flag to true when writing CSV

2016-05-25 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 20cc2eb1b -> 8629537cc [SPARK-15493][SQL] default QuoteEscapingEnabled flag to true when writing CSV ## What changes were proposed in this pull request? Default QuoteEscapingEnabled flag to true when writing CSV and add an escapeQuote

spark git commit: [SPARK-15493][SQL] default QuoteEscapingEnabled flag to true when writing CSV

2016-05-25 Thread rxin
Repository: spark Updated Branches: refs/heads/master 4b8806741 -> c875d81a3 [SPARK-15493][SQL] default QuoteEscapingEnabled flag to true when writing CSV ## What changes were proposed in this pull request? Default QuoteEscapingEnabled flag to true when writing CSV and add an escapeQuotes op

spark git commit: [SPARK-15483][SQL] IncrementalExecution should use extra strategies.

2016-05-25 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-2.0 409eb28f7 -> 20cc2eb1b [SPARK-15483][SQL] IncrementalExecution should use extra strategies. ## What changes were proposed in this pull request? Extra strategies does not work for streams because `IncrementalExecution` uses modified pl

spark git commit: [SPARK-15483][SQL] IncrementalExecution should use extra strategies.

2016-05-25 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 1cb347fbc -> 4b8806741 [SPARK-15483][SQL] IncrementalExecution should use extra strategies. ## What changes were proposed in this pull request? Extra strategies does not work for streams because `IncrementalExecution` uses modified planne

spark git commit: [SPARK-15500][DOC][ML][PYSPARK] Remove default value in Param doc field in ALS

2016-05-25 Thread mlnick
Repository: spark Updated Branches: refs/heads/branch-2.0 6fc367e50 -> 409eb28f7 [SPARK-15500][DOC][ML][PYSPARK] Remove default value in Param doc field in ALS Remove "Default: MEMORY_AND_DISK" from `Param` doc field in ALS storage level params. This fixes up the output of `explainParam(s)` s

spark git commit: [SPARK-15500][DOC][ML][PYSPARK] Remove default value in Param doc field in ALS

2016-05-25 Thread mlnick
Repository: spark Updated Branches: refs/heads/master 02c8072ee -> 1cb347fbc [SPARK-15500][DOC][ML][PYSPARK] Remove default value in Param doc field in ALS Remove "Default: MEMORY_AND_DISK" from `Param` doc field in ALS storage level params. This fixes up the output of `explainParam(s)` so th

spark git commit: [MINOR][MLLIB][STREAMING][SQL] Fix typos

2016-05-25 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master d6d3e5071 -> 02c8072ee [MINOR][MLLIB][STREAMING][SQL] Fix typos fixed typos for source code for components [mllib] [streaming] and [SQL] None and obvious. Author: lfzCarlosC Closes #13298 from lfzCarlosC/master. Project: http://git-wi

spark git commit: [MINOR][MLLIB][STREAMING][SQL] Fix typos

2016-05-25 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-2.0 4009ddafd -> 6fc367e50 [MINOR][MLLIB][STREAMING][SQL] Fix typos fixed typos for source code for components [mllib] [streaming] and [SQL] None and obvious. Author: lfzCarlosC Closes #13298 from lfzCarlosC/master. (cherry picked from

spark git commit: [MINOR][CORE] Fix a HadoopRDD log message and remove unused imports in rdd files.

2016-05-25 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-2.0 c75ec5eaa -> 4009ddafd [MINOR][CORE] Fix a HadoopRDD log message and remove unused imports in rdd files. ## What changes were proposed in this pull request? This PR fixes the following typos in log message and comments of `HadoopRDD.

spark git commit: [MINOR][CORE] Fix a HadoopRDD log message and remove unused imports in rdd files.

2016-05-25 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 8239fdcb9 -> d6d3e5071 [MINOR][CORE] Fix a HadoopRDD log message and remove unused imports in rdd files. ## What changes were proposed in this pull request? This PR fixes the following typos in log message and comments of `HadoopRDD.scal

spark git commit: [SPARK-15520][SQL] SparkSession builder in python should also allow overriding confs of existing sessions

2016-05-25 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 01e7b9c85 -> 8239fdcb9 [SPARK-15520][SQL] SparkSession builder in python should also allow overriding confs of existing sessions ## What changes were proposed in this pull request? This fixes the python SparkSession builder to allow setti

spark git commit: [SPARK-15520][SQL] SparkSession builder in python should also allow overriding confs of existing sessions

2016-05-25 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-2.0 27f26a39d -> c75ec5eaa [SPARK-15520][SQL] SparkSession builder in python should also allow overriding confs of existing sessions ## What changes were proposed in this pull request? This fixes the python SparkSession builder to allow s

spark git commit: [SPARK-15345][SQL][PYSPARK] SparkSession's conf doesn't take effect when this already an existing SparkContext

2016-05-25 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-2.0 69327667d -> 27f26a39d [SPARK-15345][SQL][PYSPARK] SparkSession's conf doesn't take effect when this already an existing SparkContext ## What changes were proposed in this pull request? Override the existing SparkContext is the provid

spark git commit: [SPARK-15345][SQL][PYSPARK] SparkSession's conf doesn't take effect when this already an existing SparkContext

2016-05-25 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master b120fba6a -> 01e7b9c85 [SPARK-15345][SQL][PYSPARK] SparkSession's conf doesn't take effect when this already an existing SparkContext ## What changes were proposed in this pull request? Override the existing SparkContext is the provided S

spark git commit: [SPARK-9044] Fix "Storage" tab in UI so that it reflects RDD name change.

2016-05-25 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.0 f63ba2210 -> 69327667d [SPARK-9044] Fix "Storage" tab in UI so that it reflects RDD name change. ## What changes were proposed in this pull request? 1. Making 'name' field of RDDInfo mutable. 2. In StorageListener: catching the fact th

spark git commit: [SPARK-9044] Fix "Storage" tab in UI so that it reflects RDD name change.

2016-05-25 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master 4f27b8dd5 -> b120fba6a [SPARK-9044] Fix "Storage" tab in UI so that it reflects RDD name change. ## What changes were proposed in this pull request? 1. Making 'name' field of RDDInfo mutable. 2. In StorageListener: catching the fact that R

spark git commit: [SPARK-15436][SQL] Remove DescribeFunction and ShowFunctions

2016-05-25 Thread hvanhovell
Repository: spark Updated Branches: refs/heads/master 9082b7968 -> 4f27b8dd5 [SPARK-15436][SQL] Remove DescribeFunction and ShowFunctions ## What changes were proposed in this pull request? This patch removes the last two commands defined in the catalyst module: DescribeFunction and ShowFunct

spark git commit: [SPARK-15436][SQL] Remove DescribeFunction and ShowFunctions

2016-05-25 Thread hvanhovell
Repository: spark Updated Branches: refs/heads/branch-2.0 1dad1a891 -> f63ba2210 [SPARK-15436][SQL] Remove DescribeFunction and ShowFunctions ## What changes were proposed in this pull request? This patch removes the last two commands defined in the catalyst module: DescribeFunction and ShowF