Repository: spark Updated Branches: refs/heads/branch-1.1 54ccd93e6 -> 233c283e3
[HOTFIX][SQL] Remove cleaning of UDFs It is not safe to run the closure cleaner on slaves. #2153 introduced this which broke all UDF execution on slaves. Will re-add cleaning of UDF closures in a follow-up PR. Author: Michael Armbrust <[email protected]> Closes #2174 from marmbrus/fixUdfs and squashes the following commits: 55406de [Michael Armbrust] [HOTFIX] Remove cleaning of UDFs (cherry picked from commit 024178c57419f915d26414e1b91ea0019c3650db) Signed-off-by: Patrick Wendell <[email protected]> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/233c283e Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/233c283e Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/233c283e Branch: refs/heads/branch-1.1 Commit: 233c283e3d946bdcbf418375122c5763559c0119 Parents: 54ccd93 Author: Michael Armbrust <[email protected]> Authored: Wed Aug 27 23:05:34 2014 -0700 Committer: Patrick Wendell <[email protected]> Committed: Wed Aug 27 23:06:14 2014 -0700 ---------------------------------------------------------------------- .../org/apache/spark/sql/catalyst/expressions/ScalaUdf.scala | 3 --- 1 file changed, 3 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/233c283e/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUdf.scala ---------------------------------------------------------------------- diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUdf.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUdf.scala index 0b3c1df..589816c 100644 --- a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUdf.scala +++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUdf.scala @@ -23,9 +23,6 @@ import org.apache.spark.util.ClosureCleaner case class ScalaUdf(function: AnyRef, dataType: DataType, children: Seq[Expression]) extends Expression { - // Clean function when not called with default no-arg constructor. - if (function != null) { ClosureCleaner.clean(function) } - type EvaluatedType = Any def nullable = true --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
