memleak in execute_function?

[fireshen]

Hey all, 

I suspect there is a memleak in bash.

I use the bash 4.4.23 (fedora Linux) with ASAN. Then I run some testcases
and find a memleak.

 

I think we need to free this "gs" in execute_cmd.c

*gs* = sh_getopt_save_istate ();
  if (subshell == 0)
{
  begin_unwind_frame ("function_calling");
  push_context (var->name, subshell, temporary_env);
  /* This has to be before the pop_context(), because the unwinding of
 local variables may cause the restore of a local declaration of
 OPTIND to force a getopts state reset. */
  add_unwind_protect (maybe_restore_getopt_state, *gs*);
  add_unwind_protect (pop_context, (char *)NULL);
  unwind_protect_int (line_number);
  unwind_protect_int (line_number_for_err_trap);
  unwind_protect_int (function_line_number);
  unwind_protect_int (return_catch_flag);
  unwind_protect_jmp_buf (return_catch);
  add_unwind_protect (dispose_command, (char *)tc);
  unwind_protect_pointer (this_shell_function);
  unwind_protect_int (funcnest);
  unwind_protect_int (loop_level);
}
  else
push_context (var->name, subshell, temporary_env);  /* don't
unwind-protect for subshells */

Is this a bug? Just analyze the code.





--
Sent from: http://gnu-bash.2382.n7.nabble.com/

syntax errors produced by interaction between process substitution and a SIGCHLD trap

[Travis Everett]

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: darwin17.7.0
Compiler: clang
Compilation CFLAGS: -g -O2 -Wno-parentheses -Wno-format-security
uname output: Darwin 6841b968 18.6.0 Darwin Kernel Version 18.6.0: Thu Apr
25 23:16:27 PDT 2019; root:xnu-4903.261.4~2/RELEASE_X86_64 x86_64
Machine Type: x86_64-apple-darwin17.7.0

Bash Version: 5.0
Patch Level: 7
Release Status: release

Description:
   I stumbled into an odd interaction between process substitution and
a SIGCHLD trap that causes syntax errors in otherwise valid commands.

   I managed to pare this down to a very minimal reproduction. I also
included a more realistic command at the end just in case I'm mistaken
about the minimal reproduction showing exactly the same issue.

   I see the same behavior in bash 4.4.23(1)-release and do *not* see
it in the macos system bash 3.2.57(1)-release.

Repeat-By:
$ bash5 --norc --noprofile
bash-5.0$ trap '$()' SIGCHLD
bash-5.0$ $(:) $(:)
bash: command substitution: line 3: syntax error near unexpected
token `)'
bash: command substitution: line 3: `:)'

# Note that backticks don't cause the same trouble
bash-5.0$ `:` `:`

# in case there are multiple issues here
# I found it while composing a command like this (to expand a set
of globs, one per line, saved in file "exclude")
bash-5.0$ /usr/bin/env cat <(printf "%s\n" $(< exclude ));
bash: command substitution: line 6: syntax error near unexpected
token `)'
bash: command substitution: line 6: `< exclude ));'

Re: Optimize bash string handling?

[Chet Ramey]

On 7/26/19 5:55 AM, Alkis Georgopoulos wrote:
> While handling some big strings, I noticed that bash is a lot slower
> than other shells like dash, posh and busybox ash.
> I came up with the following little benchmark and results.
> While the specific benchmark isn't important, maybe some developer
> would like to use it to pinpoint and optimize some internal bash
> function that is a lot slower than in other shells?

Thanks for the report. There are places in bash where it copies and
re-processes strings too many times, and you uncovered a couple.

> 
> # Avoid UTF-8 complications
> export LANG=C
> 
> # Run the following COMMANDs with `time bash -c`
> # or `time busybox ash -c`
> # The time columns are in seconds, on an i5-4440 CPU
> 
> ASH BASH  COMMAND
> 0.1  0.1  printf "%1s" "." >/dev/null
> 0.7  1.1  x=$(printf "%1s" ".")

The first assignment is dominated by the command substitution and
reading the data through a pipe.

> 0.8  2.4  x=$(printf "%1s" "."); echo ${#x}
> 0.9  3.7  x=$(printf "%1s" "."); echo ${#x}; echo ${#x}

The length function was too general, and didn't optimize for the common
case. Bash would expand the parameter name following the `#' as if the
`#' were not present, then take the length of the results. Most uses don't
need that generality, or the common error handling if `set -u' is enabled.
Factoring out the common case provides substantial improvement:

$ time ./bash ./x1a

real0m1.215s
user0m0.959s
sys 0m0.248s
$ time ./bash ./x1b
1

real0m1.242s
user0m0.982s
sys 0m0.256s
$ time ./bash ./x1c
1
1

real0m1.290s
user0m1.020s
sys 0m0.265s

where the three scripts are the three cases above.

There's always more work to do, though.

Chet


-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: memleak in execute_function?

[Chet Ramey]

On 7/31/19 11:26 AM, fireshen wrote:
> Hey all, 
> 
> I suspect there is a memleak in bash.
> 
> I use the bash 4.4.23 (fedora Linux) with ASAN. Then I run some testcases
> and find a memleak.

It's not a memory leak. It's a false positive from asan or valgrind.

The `gs' object is freed by maybe_restore_getopt_state or a function it
calls (sh_getopt_dispose_istate via sh_getopt_restore_istate).

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: memleak in execute_function?

[fireshen]

Yeah, I agree with your opinion, in the branch "if (subshell == 0) ", we have
the function "maybe_restore_getopt_state" to free gs; however, in the branch
"else", it seems like forget to free "gs", or we don't have to free it？




--
Sent from: http://gnu-bash.2382.n7.nabble.com/

Re: memleak in execute_function?

[Chet Ramey]

On 7/31/19 8:57 PM, fireshen wrote:
> Yeah, I agree with your opinion, in the branch "if (subshell == 0) ", we have
> the function "maybe_restore_getopt_state" to free gs; however, in the branch
> "else", it seems like forget to free "gs", or we don't have to free it？

We don't have to free it. If subshell != 0 we are called from a function
that executes a function in a subshell environment, and all it does is
call exit().


-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/