[
https://issues.apache.org/jira/browse/HADOOP-10846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14292519#comment-14292519
]
Steve Loughran commented on HADOOP-10846:
-----------------------------------------
-1
Looking at [http://code.metager.de/source/xref/gnu/glibc/bits/byteswap.h] to
answer my own question.
with the transform defined in the hadoop code, {{htole32()}} is doing something
on all platforms.
{code}
#define htole32(x) (bswap_32(htonl(x)))
{code}
Yet I don't see it being needed on x86: after all, it's not there until then.
So the patch is adding a fair amount of needless byteswapping.
The transform should go away entirely on a little endian part, with something
like:
{code}
#if defined __LITTLE_ENDIAN
# define htole32(x) (x)
#else
# define htole32(x) (bswap_32(htonl(x)))
#endif
{code}
> DataChecksum#calculateChunkedSums not working for PPC when buffers not backed
> by array
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-10846
> URL: https://issues.apache.org/jira/browse/HADOOP-10846
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
> Affects Versions: 2.4.1, 2.5.2
> Environment: PowerPC platform
> Reporter: Jinghui Wang
> Assignee: Jinghui Wang
> Attachments: HADOOP-10846-v1.patch, HADOOP-10846-v2.patch,
> HADOOP-10846-v3.patch, HADOOP-10846.patch
>
>
> Got the following exception when running Hadoop on Power PC. The
> implementation for computing checksum when the data buffer and checksum
> buffer are not backed by arrays.
> 13/09/16 04:06:57 ERROR security.UserGroupInformation:
> PriviledgedActionException as:biadmin (auth:SIMPLE)
> cause:org.apache.hadoop.ipc.RemoteException(java.io.IOException):
> org.apache.hadoop.fs.ChecksumException: Checksum error
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
