[
https://issues.apache.org/jira/browse/HADOOP-13717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572596#comment-15572596
]
Allen Wittenauer commented on HADOOP-13717:
-------------------------------------------
Ah, OK.
bq. Is it reasonable to also skip these checks in this situation?
In the general daemon case we definitely need to leave the hadoop_verify_logdir
in place. This will get followed up with verifying the pid dir as well. This
is needed because many folks run the daemon commands "interactively" from a
shell code perspective but do their own daemonization from init. There's also
the docker case where individual daemons are in their own containers. etc,
etc. We're almost always better checking for the existence of this dirs rather
than letting Java go haywire.
But balancer (and mover and probably a handful of others) are, in many ways,
special.
We effectively have a problem of history. sbin/start-balancer.sh was added way
back when as a convenience to background the balancer. When the rewrite
happened, balancer was added to the commands that could be daemonized as a
result.
There might also be a privilege problem here too. I don't think balancer can
be run as by a non-privileged user, usually making the log and pid dir
verification (usually) safe and somewhat required if we want all of our daemons
to function in the same way (consistency!). The pid check at the UNIX level
definitely helps prevent multiple instances running on the same host way faster
(and somewhat more reliably!) than the Java checks.
To me, the questions become:
a) should balancer stay as a daemon? what about the other
not-really-daemons-but-can-run-in-daemon-mode subcommands?
b) should instead there be logic to say that failing logdir and piddir are
optionally fatal?
c) should a local override be used instead? (e.g., define a shell profile that
replaces the hdfs balancer command)
> Shell scripts call hadoop_verify_logdir even when command is not started as
> daemon
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-13717
> URL: https://issues.apache.org/jira/browse/HADOOP-13717
> Project: Hadoop Common
> Issue Type: Bug
> Components: scripts
> Affects Versions: 3.0.0-alpha1
> Reporter: Andrew Wang
>
> Issue found when working with the HDFS balancer.
> In {{hadoop_daemon_handler}}, it calls {{hadoop_verify_logdir}} even for the
> "default" case which calls {{hadoop_start_daemon}}. {{daemon_outfile}} which
> specifies the log location isn't even used here, since the command is being
> started in the foreground.
> I think we can push the {{hadoop_verify_logdir}} call down into
> {{hadoop_start_daemon_wrapper}} instead, which does use the outfile.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
