[
https://issues.apache.org/jira/browse/HADOOP-10358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Maykov updated HADOOP-10358:
---------------------------------
Description:
The native component of hadoop-common (libhadoop.so on linux, libhadoop.dylib
on mac) fails to compile on Mac OS X. The problem is in
hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c
at lines 76-78:
[exec]
/Users/ilyam/src/github/apache/hadoop-common/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c:77:26:
error: invalid operands to binary expression ('void' and 'int')
[exec] if(setnetgrent(cgroup) == 1) {
[exec] ~~~~~~~~~~~~~~~~~~~ ^ ~
There are two problems in the code:
1) The #ifndef guard only checks for __FreeBSD__ but should check for either
one of __FreeBSD__ or __APPLE__. This is because Mac OS X inherits its syscalls
from FreeBSD rather than Linux, and thus the setnetgrent() syscall returns void.
2) setnetgrentCalledFlag = 1 is set outside the #ifndef guard, but the syscall
is only invoked inside the guard. This means that on FreeBSD, endnetgrent() can
be called in the cleanup code without a corresponding setnetgrent() invocation.
I have a patch that fixes both issues (will attach in a bit). With this patch,
I'm able to compile libhadoop.dylib on Mac OS X, which in turn lets me install
native snappy, lzo, etc compressor libraries on my client. That lets me run
commands like 'hadoop fs -text somefile.lzo' from the macbook rather than
having to ssh to a linux box, etc.
Note that this patch only fixes the native build of hadoop-common-project. Some
other components of hadoop still fail to build their native components, but
libhadoop.dylib is enough for the client.
was:
The native component of hadoop-common (libhadoop.so on linux, libhadoop.dylib
on mac) fails to compile on Mac OS X. The problem is in
hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c
at lines 76-78:
[exec]
/Users/ilyam/src/github/apache/hadoop-common/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c:77:26:
error: invalid operands to binary expression ('void' and 'int')
[exec] if(setnetgrent(cgroup) == 1) {
[exec] ~~~~~~~~~~~~~~~~~~~ ^ ~
There are two problems in the code:
1) The #ifndef guard only checks for __FreeBSD__ but should check for either
one of __FreeBSD__ or __APPLE__. This is because Mac OS X inherits its syscalls
from FreeBSD rather than Linux, and thus the setnetgrent() syscall returns void.
2) setnetgrentCalledFlag = 1 is called outside the #ifndef guard, but the
syscall is only called inside the guard. This means that on FreeBSD,
endnetgrent() can be called in the cleanup code without a corresponding
setnetgrent() invocation.
I have a patch that fixes both issues (will attach in a bit). With this patch,
I'm able to compile libhadoop.dylib on Mac OS X, which in turn lets me install
native snappy, lzo, etc compressor libraries on my client. That lets me run
commands like 'hadoop fs -text somefile.lzo' from the macbook rather than
having to ssh to a linux box, etc.
Note that this patch only fixes the native build of hadoop-common-project. Some
other components of hadoop still fail to build their native components, but
libhadoop.dylib is enough for the client.
> libhadoop doesn't compile on Mac OS X
> -------------------------------------
>
> Key: HADOOP-10358
> URL: https://issues.apache.org/jira/browse/HADOOP-10358
> Project: Hadoop Common
> Issue Type: Improvement
> Components: native
> Environment: Mac OS X 10.8.5
> Oracle JDK 1.7.0_51
> Reporter: Ilya Maykov
> Priority: Minor
> Attachments: HADOOP-10358-fix-hadoop-common-native-on-os-x.patch
>
>
> The native component of hadoop-common (libhadoop.so on linux, libhadoop.dylib
> on mac) fails to compile on Mac OS X. The problem is in
> hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c
> at lines 76-78:
> [exec]
> /Users/ilyam/src/github/apache/hadoop-common/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c:77:26:
> error: invalid operands to binary expression ('void' and 'int')
> [exec] if(setnetgrent(cgroup) == 1) {
> [exec] ~~~~~~~~~~~~~~~~~~~ ^ ~
> There are two problems in the code:
> 1) The #ifndef guard only checks for __FreeBSD__ but should check for either
> one of __FreeBSD__ or __APPLE__. This is because Mac OS X inherits its
> syscalls from FreeBSD rather than Linux, and thus the setnetgrent() syscall
> returns void.
> 2) setnetgrentCalledFlag = 1 is set outside the #ifndef guard, but the
> syscall is only invoked inside the guard. This means that on FreeBSD,
> endnetgrent() can be called in the cleanup code without a corresponding
> setnetgrent() invocation.
> I have a patch that fixes both issues (will attach in a bit). With this
> patch, I'm able to compile libhadoop.dylib on Mac OS X, which in turn lets me
> install native snappy, lzo, etc compressor libraries on my client. That lets
> me run commands like 'hadoop fs -text somefile.lzo' from the macbook rather
> than having to ssh to a linux box, etc.
> Note that this patch only fixes the native build of hadoop-common-project.
> Some other components of hadoop still fail to build their native components,
> but libhadoop.dylib is enough for the client.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
