This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new 10208908 [SPARK-31802][SQL] Format Java date-time types in
`Row.jsonValue` directly
10208908 is described below
commit 102089081e246bab306def2b63385bca35b87fe9
Author: Max Gekk <[email protected]>
AuthorDate: Mon May 25 12:50:38 2020 +0900
[SPARK-31802][SQL] Format Java date-time types in `Row.jsonValue` directly
### What changes were proposed in this pull request?
Use `format()` methods for Java date-time types in `Row.jsonValue`. The PR
https://github.com/apache/spark/pull/28582 added the methods to avoid
conversions to days and microseconds.
### Why are the changes needed?
To avoid unnecessary overhead of converting Java date-time types to
micros/days before formatting. Also formatters have to convert input
micros/days back to Java types to pass instances to standard library API.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
By existing tests in `RowJsonSuite`.
Closes #28620 from MaxGekk/toJson-format-Java-datetime-types.
Authored-by: Max Gekk <[email protected]>
Signed-off-by: HyukjinKwon <[email protected]>
(cherry picked from commit 7f36310500a7ab6e80cc1b1e6c764ae7b9657758)
Signed-off-by: HyukjinKwon <[email protected]>
---
sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala | 12 ++++--------
1 file changed, 4 insertions(+), 8 deletions(-)
diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala
index 4487a2d..5b17f1d 100644
--- a/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala
+++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala
@@ -571,14 +571,10 @@ trait Row extends Serializable {
case (s: String, _) => JString(s)
case (b: Array[Byte], BinaryType) =>
JString(Base64.getEncoder.encodeToString(b))
- case (d: LocalDate, _) =>
- JString(dateFormatter.format(DateTimeUtils.localDateToDays(d)))
- case (d: Date, _) =>
- JString(dateFormatter.format(DateTimeUtils.fromJavaDate(d)))
- case (i: Instant, _) =>
- JString(timestampFormatter.format(DateTimeUtils.instantToMicros(i)))
- case (t: Timestamp, _) =>
- JString(timestampFormatter.format(DateTimeUtils.fromJavaTimestamp(t)))
+ case (d: LocalDate, _) => JString(dateFormatter.format(d))
+ case (d: Date, _) => JString(dateFormatter.format(d))
+ case (i: Instant, _) => JString(timestampFormatter.format(i))
+ case (t: Timestamp, _) => JString(timestampFormatter.format(t))
case (i: CalendarInterval, _) => JString(i.toString)
case (a: Array[_], ArrayType(elementType, _)) =>
iteratorToJsonArray(a.iterator, elementType)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
