Repository: spark Updated Branches: refs/heads/branch-1.3 ea69cf28e -> 1b8ab5752
[SPARK-6121][SQL][MLLIB] simpleString for UDT `df.dtypes` shows `null` for UDTs. This PR uses `udt` by default and `VectorUDT` overwrites it with `vector`. jkbradley davies Author: Xiangrui Meng <[email protected]> Closes #4858 from mengxr/SPARK-6121 and squashes the following commits: 34f0a77 [Xiangrui Meng] simpleString for UDT (cherry picked from commit 2db6a853a53b4c25e35983bc489510abb8a73e1d) Signed-off-by: Xiangrui Meng <[email protected]> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/1b8ab575 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/1b8ab575 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/1b8ab575 Branch: refs/heads/branch-1.3 Commit: 1b8ab5752fccbc08c3f76c50bc384b89231d0a78 Parents: ea69cf2 Author: Xiangrui Meng <[email protected]> Authored: Mon Mar 2 17:14:34 2015 -0800 Committer: Xiangrui Meng <[email protected]> Committed: Mon Mar 2 17:14:43 2015 -0800 ---------------------------------------------------------------------- python/pyspark/mllib/linalg.py | 3 +++ python/pyspark/sql/types.py | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/1b8ab575/python/pyspark/mllib/linalg.py ---------------------------------------------------------------------- diff --git a/python/pyspark/mllib/linalg.py b/python/pyspark/mllib/linalg.py index 597012b..f5aad28 100644 --- a/python/pyspark/mllib/linalg.py +++ b/python/pyspark/mllib/linalg.py @@ -152,6 +152,9 @@ class VectorUDT(UserDefinedType): else: raise ValueError("do not recognize type %r" % tpe) + def simpleString(self): + return "vector" + class Vector(object): http://git-wip-us.apache.org/repos/asf/spark/blob/1b8ab575/python/pyspark/sql/types.py ---------------------------------------------------------------------- diff --git a/python/pyspark/sql/types.py b/python/pyspark/sql/types.py index 31a861e..0169028 100644 --- a/python/pyspark/sql/types.py +++ b/python/pyspark/sql/types.py @@ -468,7 +468,7 @@ class UserDefinedType(DataType): raise NotImplementedError("UDT must implement deserialize().") def simpleString(self): - return 'null' + return 'udt' def json(self): return json.dumps(self.jsonValue(), separators=(',', ':'), sort_keys=True) --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
