2011/5/9 Mårten Wikström <marten.wikst...@keystream.se>:
> Configuration Information [Automatically generated, do not change]:
> Machine: x86_64
> OS: linux-gnu
> Compiler: gcc
> Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64'
> -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu'
> -DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash'
> -DSHELL -DHAVE_CONFIG_H -I. -I../bash -I../bash/include
> -I../bash/lib -g -O2 -Wall
> uname output: Linux heinrich 2.6.38-8-generic #42-Ubuntu SMP Mon Apr
> 11 03:31:24 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
> Machine Type: x86_64-pc-linux-gnu
>
> Bash Version: 4.2
> Patch Level: 8
> Release Status: release
>
> Description:
> When parsing double double-quotes (i.e. """") it will be replaced by
> the
> value 0x7f, if there are characters before or after it. In bash 4.1
> empty
> double-quotes were simply removed.
>
> Repeat-By:
> $ echo """"a >t
> $ hexdump -C t
> 00000000 7f 61 0a |.a.|
> 00000003
>
Fix:
After some debugging it turns out that the problem lies in
expand_word_internal()
in subst.c. In 4.1.0 the "" will be removed in expand_word_internal()
when we hit
line 8040:
/* We do not want to add quoted nulls to strings that are only
partially quoted; we can throw them away. */
if (temp == 0 && quoted_state == PARTIALLY_QUOTED)
continue;
However, in 4.2.10 the "" will be converted to CTLNUL (0x7f). Because the above
code has changed into
/* We do not want to add quoted nulls to strings that are only
partially quoted; we can throw them away. */
if (temp == 0 && quoted_state == PARTIALLY_QUOTED && (word->flags &
(W_NOSPLIT|W_NOSPLIT2)))
continue;
which won't match our case (to only flag set in word->flags is
W_QUOTED). So instead we fall down to add_quoted_string: and it will
add the CTLNUL character. So we end up with two 0x7f bytes in the
resulting string when we get back to shell_expand_word_list(). Later
only the first 0x7f will be removed by
word_list_remove_quoted_nulls().
There are two problems/solutions here. The comment in the code above
seems to indicate that the quotes should actually be thrown away as is
done in 4.1. But on the other hand, word_list_remove_quoted_nulls()
seems to indicate it should remove all nulls, not just the first.
If I fix word_list_remove_quoted_nulls() to actually remove all
consecutive nulls, the problem is
solved. (At least my simple test-case works). If I revert the line
above to the 4.1 version it also
solves my problem.
Alas, my understanding of the bash code is fairly limited so my fixes
will likely break something. Perhaps someone with a little more
insight could tell the right(tm) solution.
Anyway, here are the patches.
Solution 1, fixing remove_quoted_nulls():
*** subst.c 2011-05-10 01:48:54.816322136 +0200
--- ../bash-4.2-patched/subst.c 2011-05-10 01:53:31.350806960 +0200
*************** remove_quoted_nulls (string)
*** 3706,3712 ****
break;
}
else if (string[i] == CTLNUL)
! i++;
prev_i = i;
ADVANCE_CHAR (string, slen, i);
--- 3706,3713 ----
break;
}
else if (string[i] == CTLNUL)
! while (string[i] == CTLNUL)
! i++;
prev_i = i;
ADVANCE_CHAR (string, slen, i);
Solution 2, reverting to 4.1 behaviour:
*** subst.c 2011-05-10 01:48:54.816322136 +0200
--- ../bash-4.2-patched/subst.c 2011-05-10 01:58:15.309470318 +0200
*************** add_twochars:
*** 8356,8362 ****
/* We do not want to add quoted nulls to strings that are only
partially quoted; we can throw them away. */
! if (temp == 0 && quoted_state == PARTIALLY_QUOTED && (word->flags
& (W_NOSPLIT|W_NOSPLIT2)))
continue;
add_quoted_string:
--- 8356,8362 ----
/* We do not want to add quoted nulls to strings that are only
partially quoted; we can throw them away. */
! if (temp == 0 && quoted_state == PARTIALLY_QUOTED)
continue;
add_quoted_string:
Regards,
Mårten Wikström
