[
https://issues.apache.org/jira/browse/HADOOP-9295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13611545#comment-13611545
]
Karthik Kambatla commented on HADOOP-9295:
------------------------------------------
Actually, I don't think the test MapWritableBugTest#reproduceMapWritableBug()
tests adding MapWritable objects either. Also, if X and Y are the two different
classes named MapWritable, isn't X.readFields(Y.write(out)) expected to fail?
Shouldn't it be as below (there are two objects map1 and map2 is not because we
are calling readFields twice, but primarily because they are two different
MapWritables):
{code}
org.apache.hadoop.io.MapWritable map1 = new org.apache.hadoop.io.MapWritable();
map1.readFields(in);
Assert.assertTrue( map1.get(new Text("testKey1")) instanceof CustomWritableOne
);
MapWritable map1 = new MapWritable();
map2.readFields(in);
Assert.assertTrue( map2.get(new Text("testKey2")) instanceof CustomWritableTwo
);
{code}
> AbstractMapWritable throws exception when calling readFields() multiple times
> when the maps contain different class types
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-9295
> URL: https://issues.apache.org/jira/browse/HADOOP-9295
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 1.0.3
> Reporter: David Parks
> Assignee: Karthik Kambatla
> Priority: Critical
> Attachments: MapWritableBugTest.java, test-hadoop-9295.patch
>
>
> Verified the trunk looks the same as 1.0.3 for this issue.
> When mappers output MapWritables with different class types, then they are
> read in on the Reducer via an iterator (multiple calls to readFields without
> instantiating a new object) you'll get this:
> java.lang.IllegalArgumentException: Id 1 exists but maps to
> org.me.ClassTypeOne and not org.me.ClassTypeTwo
> at
> org.apache.hadoop.io.AbstractMapWritable.addToMap(AbstractMapWritable.java:73)
> at
> org.apache.hadoop.io.AbstractMapWritable.readFields(AbstractMapWritable.java:201)
> It happens because AbstractMapWritable accumulates class type entries in its
> ClassType to ID (and vice versa) hashmaps.
> Those accumulating classtype-to-id hashmaps need to be cleared to support
> multiple calls to readFields().
> I've attached a JUnit test that both demonstrates the problem and contains an
> embedded, fixed version of MapWritable and ArrayMapWritable (note the //TODO
> comments in the code where it was fixed in 2 places).
> If there's a better way to submit this recommended bug fix, someone please
> feel free to let me know.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
