Repository: spark
Updated Branches:
refs/heads/branch-2.2 83bdb0487 -> 99ce551a1
[SPARK-21445] Make IntWrapper and LongWrapper in UTF8String Serializable
## What changes were proposed in this pull request?
Making those two classes will avoid Serialization issues like below:
```
Caused by: java.io.NotSerializableException:
org.apache.spark.unsafe.types.UTF8String$IntWrapper
Serialization stack:
- object not serializable (class:
org.apache.spark.unsafe.types.UTF8String$IntWrapper, value:
org.apache.spark.unsafe.types.UTF8String$IntWrapper326450e)
- field (class:
org.apache.spark.sql.catalyst.expressions.Cast$$anonfun$castToInt$1, name:
result$2, type: class org.apache.spark.unsafe.types.UTF8String$IntWrapper)
- object (class
org.apache.spark.sql.catalyst.expressions.Cast$$anonfun$castToInt$1,
<function1>)
```
## How was this patch tested?
- [x] Manual testing
- [ ] Unit test
Author: Burak Yavuz <[email protected]>
Closes #18660 from brkyvz/serializableutf8.
(cherry picked from commit 26cd2ca0402d7d49780116d45a5622a45c79f661)
Signed-off-by: Wenchen Fan <[email protected]>
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/99ce551a
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/99ce551a
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/99ce551a
Branch: refs/heads/branch-2.2
Commit: 99ce551a13f0918b440ddc094c3a32167d7ab3dd
Parents: 83bdb04
Author: Burak Yavuz <[email protected]>
Authored: Tue Jul 18 12:09:07 2017 +0800
Committer: Wenchen Fan <[email protected]>
Committed: Tue Jul 18 12:09:27 2017 +0800
----------------------------------------------------------------------
.../main/java/org/apache/spark/unsafe/types/UTF8String.java | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/99ce551a/common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java
----------------------------------------------------------------------
diff --git
a/common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java
b/common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java
index 5437e99..23636ca 100644
--- a/common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java
+++ b/common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java
@@ -854,8 +854,8 @@ public final class UTF8String implements
Comparable<UTF8String>, Externalizable,
* Wrapper over `long` to allow result of parsing long from string to be
accessed via reference.
* This is done solely for better performance and is not expected to be used
by end users.
*/
- public static class LongWrapper {
- public long value = 0;
+ public static class LongWrapper implements Serializable {
+ public transient long value = 0;
}
/**
@@ -865,8 +865,8 @@ public final class UTF8String implements
Comparable<UTF8String>, Externalizable,
* {@link LongWrapper} could have been used here but using `int` directly
save the extra cost of
* conversion from `long` to `int`
*/
- public static class IntWrapper {
- public int value = 0;
+ public static class IntWrapper implements Serializable {
+ public transient int value = 0;
}
/**
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
