spark git commit: [SPARK-24972][SQL] PivotFirst could not handle pivot columns of complex types

2018-07-30 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 8141d5592 -> b4fd75fb9 [SPARK-24972][SQL] PivotFirst could not handle pivot columns of complex types ## What changes were proposed in this pull request? When the pivot column is of a complex type, the eval() result will be an UnsafeRow, w

svn commit: r28442 - in /dev/spark/2.4.0-SNAPSHOT-2018_07_30_20_02-8141d55-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-07-30 Thread pwendell
Author: pwendell Date: Tue Jul 31 03:16:07 2018 New Revision: 28442 Log: Apache Spark 2.4.0-SNAPSHOT-2018_07_30_20_02-8141d55 docs [This commit notification would consist of 1470 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-23633][SQL] Update Pandas UDFs section in sql-programming-guide

2018-07-30 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master f1550aaf1 -> 8141d5592 [SPARK-23633][SQL] Update Pandas UDFs section in sql-programming-guide ## What changes were proposed in this pull request? Update Pandas UDFs section in sql-programming-guide. Add section for grouped aggregate panda

spark git commit: [SPARK-24956][BUILD][FOLLOWUP] Upgrade Maven version to 3.5.4 for AppVeyor as well

2018-07-30 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master d20c10fdf -> f1550aaf1 [SPARK-24956][BUILD][FOLLOWUP] Upgrade Maven version to 3.5.4 for AppVeyor as well ## What changes were proposed in this pull request? Maven version was upgraded and AppVeyor should also use upgraded maven version.

spark git commit: [SPARK-24952][SQL] Support LZMA2 compression by Avro datasource

2018-07-30 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master 2fbe294cf -> d20c10fdf [SPARK-24952][SQL] Support LZMA2 compression by Avro datasource ## What changes were proposed in this pull request? In the PR, I propose to support `LZMA2` (`XZ`) and `BZIP2` compressions by `AVRO` datasource in wr

svn commit: r28439 - in /dev/spark/2.4.0-SNAPSHOT-2018_07_30_16_02-2fbe294-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-07-30 Thread pwendell
Author: pwendell Date: Mon Jul 30 23:16:19 2018 New Revision: 28439 Log: Apache Spark 2.4.0-SNAPSHOT-2018_07_30_16_02-2fbe294 docs [This commit notification would consist of 1470 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-24963][K8S][TESTS] Add user-specified service account name for client mode test driver pod

2018-07-30 Thread mcheah
Repository: spark Updated Branches: refs/heads/master abbb4ab4d -> 2fbe294cf [SPARK-24963][K8S][TESTS] Add user-specified service account name for client mode test driver pod ## What changes were proposed in this pull request? Adds the user-set service account name for the driver pod in the

spark git commit: [SPARK-24865][SQL] Remove AnalysisBarrier addendum

2018-07-30 Thread lixiao
Repository: spark Updated Branches: refs/heads/master d6b7545b5 -> abbb4ab4d [SPARK-24865][SQL] Remove AnalysisBarrier addendum ## What changes were proposed in this pull request? I didn't want to pollute the diff in the previous PR and left some TODOs. This is a follow-up to address those TO

svn commit: r28437 - in /dev/spark/2.4.0-SNAPSHOT-2018_07_30_12_02-d6b7545-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-07-30 Thread pwendell
Author: pwendell Date: Mon Jul 30 19:16:26 2018 New Revision: 28437 Log: Apache Spark 2.4.0-SNAPSHOT-2018_07_30_12_02-d6b7545 docs [This commit notification would consist of 1470 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-24963][K8S][TESTS] Don't set service account name for client mode test

2018-07-30 Thread mcheah
Repository: spark Updated Branches: refs/heads/master 47d84e4d0 -> d6b7545b5 [SPARK-24963][K8S][TESTS] Don't set service account name for client mode test ## What changes were proposed in this pull request? Don't set service account name for the pod created in client mode ## How was this pat

svn commit: r28431 - in /dev/spark/2.4.0-SNAPSHOT-2018_07_30_08_03-47d84e4-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-07-30 Thread pwendell
Author: pwendell Date: Mon Jul 30 15:17:18 2018 New Revision: 28431 Log: Apache Spark 2.4.0-SNAPSHOT-2018_07_30_08_03-47d84e4 docs [This commit notification would consist of 1470 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-22814][SQL] Support Date/Timestamp in a JDBC partition column

2018-07-30 Thread lixiao
Repository: spark Updated Branches: refs/heads/master b90bfe3c4 -> 47d84e4d0 [SPARK-22814][SQL] Support Date/Timestamp in a JDBC partition column ## What changes were proposed in this pull request? This pr supported Date/Timestamp in a JDBC partition column (a numeric column is only supported

spark git commit: [SPARK-24771][BUILD] Upgrade Apache AVRO to 1.8.2

2018-07-30 Thread lixiao
Repository: spark Updated Branches: refs/heads/master fca0b8528 -> b90bfe3c4 [SPARK-24771][BUILD] Upgrade Apache AVRO to 1.8.2 ## What changes were proposed in this pull request? Upgrade Apache Avro from 1.7.7 to 1.8.2. The major new features: 1. More logical types. From the spec of 1.8.2 h

svn commit: r28430 - in /dev/spark/2.3.3-SNAPSHOT-2018_07_30_06_34-25ea27b-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-07-30 Thread pwendell
Author: pwendell Date: Mon Jul 30 13:47:54 2018 New Revision: 28430 Log: Apache Spark 2.3.3-SNAPSHOT-2018_07_30_06_34-25ea27b docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-24967][SQL] Avro: Use internal.Logging instead for logging

2018-07-30 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 85505fc8a -> fca0b8528 [SPARK-24967][SQL] Avro: Use internal.Logging instead for logging ## What changes were proposed in this pull request? Looks Avro uses direct `getLogger` to create a SLF4J logger. Should better use `internal.Logging`

spark git commit: [SPARK-24957][SQL] Average with decimal followed by aggregation returns wrong result

2018-07-30 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.3 aa51c070f -> 25ea27b09 [SPARK-24957][SQL] Average with decimal followed by aggregation returns wrong result ## What changes were proposed in this pull request? When we do an average, the result is computed dividing the sum of the valu

spark git commit: [SPARK-24957][SQL] Average with decimal followed by aggregation returns wrong result

2018-07-30 Thread wenchen
Repository: spark Updated Branches: refs/heads/master bfe60fcdb -> 85505fc8a [SPARK-24957][SQL] Average with decimal followed by aggregation returns wrong result ## What changes were proposed in this pull request? When we do an average, the result is computed dividing the sum of the values b

svn commit: r28427 - in /dev/spark/2.3.3-SNAPSHOT-2018_07_30_02_02-aa51c07-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-07-30 Thread pwendell
Author: pwendell Date: Mon Jul 30 09:16:14 2018 New Revision: 28427 Log: Apache Spark 2.3.3-SNAPSHOT-2018_07_30_02_02-aa51c07 docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

svn commit: r28426 - in /dev/spark/2.4.0-SNAPSHOT-2018_07_30_00_02-bfe60fc-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-07-30 Thread pwendell
Author: pwendell Date: Mon Jul 30 07:16:38 2018 New Revision: 28426 Log: Apache Spark 2.4.0-SNAPSHOT-2018_07_30_00_02-bfe60fc docs [This commit notification would consist of 1470 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---