ashutoshcipher commented on PR #4536: URL: https://github.com/apache/hadoop/pull/4536#issuecomment-1199039873
> Hmm, hadoop_add_param does not work if the value contains `"`. (`HADOOP_OPTS='-XX:OnOutOfMemoryError="kill -9 %p"'` instead of previous `HADOOP_OPTS="-XX:OnOutOfMemoryError='kill -9 %p'"`) @ashutoshcipher > > ``` > $ export HADOOP_OPTS='-XX:OnOutOfMemoryError="kill -9 %p"' > $ echo $HADOOP_OPTS > -XX:OnOutOfMemoryError="kill -9 %p" > > $ bash -x -c 'export SHELLOPTS && bin/hadoop version' 2>&1 | less > ... > + hadoop_add_param HADOOP_OPTS yarn.log.dir -Dyarn.log.dir=/home/rocky/dist/hadoop-3.4.0-SNAPSHOT/logs > + [[ ! -XX:OnOutOfMemoryError="kill -9 %p" =~ yarn.log.dir ]] > + eval 'HADOOP_OPTS="-XX:OnOutOfMemoryError="kill -9 %p" -Dyarn.log.dir=/home/rocky/dist/hadoop-3.4.0-SNAPSHOT/logs"' > ++ HADOOP_OPTS=-XX:OnOutOfMemoryError=kill > ++ -9 '%p -Dyarn.log.dir=/home/rocky/dist/hadoop-3.4.0-SNAPSHOT/logs' > /home/rocky/dist/hadoop-3.4.0-SNAPSHOT/bin/../libexec/hadoop-functions.sh: line 1143: -9: command not found > ... > ``` @iwasakims Prefix exec by eval in Hadoop bin scripts Prior to this change, if HADOOP_OPTS contains any arguments that include a space, the command is not parsed correctly. For example, if HADOOP_OPTS="... -XX:OnOutOfMemoryError="kill -9 %p" ...", the bin/hadoop script will fail with the error "Unrecognized option: -9". No amount of clever escaping of the quotes or spaces in the "kill -9 %p" command will fix this. The only alternative appears to be to use 'eval'. Switching to use 'eval' instead of 'exec' also works, but it results in an intermediate bash process being left alive throughout the entire lifetime of the Java proces being started. Using 'exec' prefixed by 'eval' as has been done in this commit gets the best of both worlds, in that options with spaces are parsed correctly, and you don't end up with an intermediate bash process as the parent of the Java process. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
