a problem of mapfile
Configuration Information : Machine: i686 OS: linux-gnu Compiler: gcc Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='i686' -DCONF_OSTYPE='linux-gnu' -D$ uname output: Linux ubuntu 3.2.0-35-generic #55-Ubuntu SMP Wed Dec 5 17:45:18 UTC 2012 i6$ Machine Type: i686-pc-linux-gnu Bash Version: 4.2 Patch Level: 24 Release Status: release Description: I'm trying to use command mapfile.But there is something confused me. I used commands as follows: $seq 5 | mapfile -C echo -c1 0 1 1 2 2 3 3 4 4 5 I would have expectd 1,2,3,4,5 above from the"help mapfile". Is it a bug of mapfile?
Re: a problem of mapfile
On Fri, Jan 11, 2013 at 10:21:55PM +0800, Ma Shimiao wrote: > I used commands as follows: > $seq 5 | mapfile -C echo -c1 > 0 1 > > 1 2 > > 2 3 > > 3 4 > > 4 5 > > I would have expectd 1,2,3,4,5 above from the"help mapfile". > Is it a bug of mapfile? The help says: When CALLBACK is evaluated, it is supplied the index of the next array element to be assigned and the line to be assigned to that element as additional arguments. So your full callback command, executed on each item read, is: echo 'index number' 'line of input including newline' echo adds an additional newline of its own. The first array index number is 0, which corresponds to the line containing $'1\n'.
Re: a problem of mapfile
Thank Greg for your answer very much. This problem has confused me for 2 days. That seems I didn't read the manpage carefully. On 01/11/2013 10:54 PM, Greg Wooledge wrote: On Fri, Jan 11, 2013 at 10:21:55PM +0800, Ma Shimiao wrote: I used commands as follows: $seq 5 | mapfile -C echo -c1 0 1 1 2 2 3 3 4 4 5 I would have expectd 1,2,3,4,5 above from the"help mapfile". Is it a bug of mapfile? The help says: When CALLBACK is evaluated, it is supplied the index of the next array element to be assigned and the line to be assigned to that element as additional arguments. So your full callback command, executed on each item read, is: echo 'index number' 'line of input including newline' echo adds an additional newline of its own. The first array index number is 0, which corresponds to the line containing $'1\n'.
Assignment errors with no additional words expanded in non-POSIX mode fails to abort
Whether or not this type of error aborts depends upon there being an actual newline. $ bash -c 'echo pre; foo=$((8#9)); echo post' 2>&1 pre bash: 8#9: value too great for base (error token is "8#9") $ bash -c $'echo pre\nfoo=$((8#9))\necho post' 2>&1 pre bash: line 1: 8#9: value too great for base (error token is "8#9") post Only applies to non-POSIX mode. -- Dan Douglas signature.asc Description: This is a digitally signed message part.
typeset -p on an empty integer variable is an error. (plus -v test w/ array elements)
Bash treats the variable as essentially undefined until given at least an
empty value.
$ bash -c 'typeset -i x; [[ -v x ]]; echo "$?, ${x+foo}"; typeset -p x'
1,
bash: line 0: typeset: x: not found
$ ksh -c 'typeset -i x; [[ -v x ]]; echo "$?, ${x+foo}"; typeset -p x'
0,
typeset -i x
Zsh implicitly gives integers a zero value if none are specified and the
variable was previously undefined. Either the ksh or zsh ways are fine IMO.
Also I'll throw this in:
$ arr[1]=test; [[ -v arr[1] ]]; echo $?
1
This now works in ksh to test if an individual element is set, though it
hasn't always. Maybe Bash should do the same? -v is tricky because it adds
some extra nuances to what it means for something to be defined...
--
Dan Douglas
signature.asc
Description: This is a digitally signed message part.
printf %q represents null argument as empty string.
$ set --; printf %q\\n "$@"
''
printf should perhaps only output '' when there is actually a corresponding
empty argument, else eval "$(printf %q ...)" and similar may give different
results than expected. Other shells don't output '', even mksh's ${var@Q}
expansion. Zsh's ${(q)var} does.
--
Dan Douglas
signature.asc
Description: This is a digitally signed message part.
Re: printf %q represents null argument as empty string.
Am 11.01.2013 19:38, schrieb Dan Douglas:
> $ set --; printf %q\\n "$@"
> ''
>
> printf should perhaps only output '' when there is actually a corresponding
> empty argument, else eval "$(printf %q ...)" and similar may give different
> results than expected. Other shells don't output '', even mksh's ${var@Q}
> expansion. Zsh's ${(q)var} does.
that is not a bug in printf %q
it what you expect to happen with "${@}"
should that be 0 arguments if $# is 0.
I however find the behavior irritating, but correct from the description.
to do what you are suggesting you would need a special case handler for this
"${@}" as oposed to "${@}j" or any other variation.
what I tend to do as a workaround is
printf() {
if [ $# -eq 2 -a -z "${2}" ];then
builtin printf "${1}"
else
builtin printf "${@}"
fi
}
or not as good but ok in most cases something like
printf "%q" ${1:+"${@}"}
Re: typeset -p on an empty integer variable is an error. (plus -v test w/ array elements)
Am 11.01.2013 19:27, schrieb Dan Douglas:
> Bash treats the variable as essentially undefined until given at least an
> empty value.
>
> $ bash -c 'typeset -i x; [[ -v x ]]; echo "$?, ${x+foo}"; typeset -p x'
> 1,
> bash: line 0: typeset: x: not found
> $ ksh -c 'typeset -i x; [[ -v x ]]; echo "$?, ${x+foo}"; typeset -p x'
> 0,
> typeset -i x
>
> Zsh implicitly gives integers a zero value if none are specified and the
> variable was previously undefined. Either the ksh or zsh ways are fine IMO.
>
> Also I'll throw this in:
>
> $ arr[1]=test; [[ -v arr[1] ]]; echo $?
> 1
>
> This now works in ksh to test if an individual element is set, though it
> hasn't always. Maybe Bash should do the same? -v is tricky because it adds
> some extra nuances to what it means for something to be defined...
>
Personally I like the current behavior, disclaimer I use nounset.
I see no problem with getting people to initialize variables.
it is a more robust programming approach.
Re: printf %q represents null argument as empty string.
On Friday, January 11, 2013 09:39:00 PM John Kearney wrote:
> Am 11.01.2013 19:38, schrieb Dan Douglas:
> > $ set --; printf %q\\n "$@"
> > ''
> >
> > printf should perhaps only output '' when there is actually a
corresponding
> > empty argument, else eval "$(printf %q ...)" and similar may give
different
> > results than expected. Other shells don't output '', even mksh's ${var@Q}
> > expansion. Zsh's ${(q)var} does.
>
> that is not a bug in printf %q
>
> it what you expect to happen with "${@}"
> should that be 0 arguments if $# is 0.
>
> I however find the behavior irritating, but correct from the description.
>
> to do what you are suggesting you would need a special case handler for this
> "${@}" as oposed to "${@}j" or any other variation.
>
>
> what I tend to do as a workaround is
>
> printf() {
> if [ $# -eq 2 -a -z "${2}" ];then
> builtin printf "${1}"
> else
> builtin printf "${@}"
> fi
> }
>
>
> or not as good but ok in most cases something like
>
> printf "%q" ${1:+"${@}"}
>
>
I don't understand what you mean. The issue I'm speaking of is that printf %q
produces a quoted empty string both when given no args and when given one
empty arg. A quoted "$@" with no positional parameters present expands to zero
words (and correspondingly for "${arr[@]}"). Why do you think "x${@}x" is
special? (Note that expansion didn't even work correctly a few patchsets ago.)
Also as pointed out, every other shell with a printf %q feature disagrees with
Bash. Are you saying that something in the manual says that it should do
otherwise? I'm aware you could write a wrapper, I just don't see any utility
in the default behavior.
--
Dan Douglas
checkwinsize only works in interactive shells
shopt -s checkwinsize only works in interactive shells. However, this is not mentioned in the manual. Tested in Bash 4.2.37. #!/bin/bash trap 'echo "COLUMNS is now <$COLUMNS>"' WINCH test -z "$COLUMNS" && COLUMNS=$(tput cols) shopt -s checkwinsize while sleep 1; do :; done Run in a terminal, resize the terminal repeatedly. COLUMNS is never updated after its initial setting. Set the same trap in an interactive shell, and resize the terminal repeatedly. COLUMNS will be updated each time.
Re: typeset -p on an empty integer variable is an error. (plus -v test w/ array elements)
On Friday, January 11, 2013 09:48:32 PM John Kearney wrote:
> Am 11.01.2013 19:27, schrieb Dan Douglas:
> > Bash treats the variable as essentially undefined until given at least an
> > empty value.
> >
> > $ bash -c 'typeset -i x; [[ -v x ]]; echo "$?, ${x+foo}"; typeset -p x'
> > 1,
> > bash: line 0: typeset: x: not found
> > $ ksh -c 'typeset -i x; [[ -v x ]]; echo "$?, ${x+foo}"; typeset -p x'
> > 0,
> > typeset -i x
> >
> > Zsh implicitly gives integers a zero value if none are specified and the
> > variable was previously undefined. Either the ksh or zsh ways are fine IMO.
> >
> > Also I'll throw this in:
> >
> > $ arr[1]=test; [[ -v arr[1] ]]; echo $?
> > 1
> >
> > This now works in ksh to test if an individual element is set, though it
> > hasn't always. Maybe Bash should do the same? -v is tricky because it adds
> > some extra nuances to what it means for something to be defined...
> >
>
> Personally I like the current behavior, disclaimer I use nounset.
> I see no problem with getting people to initialize variables.
How is this relevant? It's an inconsistency in the way set/unset variables
are normally handled. You don't use variadic functions? Unset variables /
parameters are a normal part of most scripts.
> it is a more robust programming approach.
I strongly disagree. (Same goes for errexit.)
--
Dan Douglas
Re: printf %q represents null argument as empty string.
On 1/11/13 4:05 PM, Dan Douglas wrote:
>
> I don't understand what you mean. The issue I'm speaking of is that printf %q
> produces a quoted empty string both when given no args and when given one
> empty arg. A quoted "$@" with no positional parameters present expands to
> zero
> words (and correspondingly for "${arr[@]}"). Why do you think "x${@}x" is
> special? (Note that expansion didn't even work correctly a few patchsets ago.)
>
> Also as pointed out, every other shell with a printf %q feature disagrees
> with
> Bash. Are you saying that something in the manual says that it should do
> otherwise? I'm aware you could write a wrapper, I just don't see any utility
> in the default behavior.
This is how bash behaves:
The format is reused as necessary to consume all of the argu-
ments. If the format requires more arguments than are supplied,
the extra format specifications behave as if a zero value or
null string, as appropriate, had been supplied.
This is how Posix specifies printf to work. I know it doesn't have %q,
but bash doesn't really differentiate between %q and %s.
Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: checkwinsize only works in interactive shells
On 1/11/13 4:13 PM, Greg Wooledge wrote: > shopt -s checkwinsize only works in interactive shells. However, this > is not mentioned in the manual. Yes, that's true. This was reported in June, and the development versions since then have been changed to make it work in non-interactive shells as well. It will be that way in bash-4.3. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: checkwinsize only works in interactive shells
On Fri, Jan 11, 2013 at 4:43 PM, Chet Ramey wrote: > On 1/11/13 4:13 PM, Greg Wooledge wrote: > > shopt -s checkwinsize only works in interactive shells. However, this > > is not mentioned in the manual. > > Yes, that's true. This was reported in June, and the development versions > since then have been changed to make it work in non-interactive shells > as well. It will be that way in bash-4.3. > > Chet > > -- > ``The lyf so short, the craft so long to lerne.'' - Chaucer > ``Ars longa, vita brevis'' - Hippocrates > Chet Ramey, ITS, CWRUc...@case.edu > http://cnswww.cns.cwru.edu/~chet/ > > Will they still be updated on WINCH without checkwinsize, as well? And will they be set by default, or continue to be unset in scripts unless set manually? Thanks, DJ
Re: printf %q represents null argument as empty string.
On Friday, January 11, 2013 04:37:56 PM Chet Ramey wrote:
> On 1/11/13 4:05 PM, Dan Douglas wrote:
>
> >
> > I don't understand what you mean. The issue I'm speaking of is that printf
> > %q
> > produces a quoted empty string both when given no args and when given one
> > empty arg. A quoted "$@" with no positional parameters present expands to
> > zero
> > words (and correspondingly for "${arr[@]}"). Why do you think "x${@}x" is
> > special? (Note that expansion didn't even work correctly a few patchsets
> > ago.)
> >
> > Also as pointed out, every other shell with a printf %q feature disagrees
> > with
> > Bash. Are you saying that something in the manual says that it should do
> > otherwise? I'm aware you could write a wrapper, I just don't see any
> > utility
> > in the default behavior.
>
> This is how bash behaves:
>
> The format is reused as necessary to consume all of the argu-
> ments. If the format requires more arguments than are supplied,
> the extra format specifications behave as if a zero value or
> null string, as appropriate, had been supplied.
>
> This is how Posix specifies printf to work. I know it doesn't have %q,
> but bash doesn't really differentiate between %q and %s.
>
> Chet
Ah so I'm confusing the very same thing as the "no argument along with %()T" you
pointed out to me on earlier... so this would have to be yet another special
case.
Funny that never crossed my mind.
--
Dan Douglas
Re: checkwinsize only works in interactive shells
On 1/11/13 4:46 PM, DJ Mills wrote: > Will they still be updated on WINCH without checkwinsize, as well? And will > they be set by default, or continue to be unset in scripts unless set > manually? The only thing that's changed is that checkwinsize works in non-interactive shells. SIGWINCH still updates LINES and COLUMNS, and bash won't set LINES and COLUMNS itself. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: checkwinsize only works in interactive shells
On Fri, Jan 11, 2013 at 4:51 PM, Chet Ramey wrote: > On 1/11/13 4:46 PM, DJ Mills wrote: > > > Will they still be updated on WINCH without checkwinsize, as well? And > will > > they be set by default, or continue to be unset in scripts unless set > manually? > > The only thing that's changed is that checkwinsize works in non-interactive > shells. SIGWINCH still updates LINES and COLUMNS, and bash won't set LINES > and COLUMNS itself. > > Chet > -- > ``The lyf so short, the craft so long to lerne.'' - Chaucer > ``Ars longa, vita brevis'' - Hippocrates > Chet Ramey, ITS, CWRUc...@case.edu > http://cnswww.cns.cwru.edu/~chet/ > > As of right now, WINCH will not update COLUMNS or LINES in a non-interactive shell. I've had to use: trap 'COLUMNS=$(tput cols); LINES=$(tput lines)' WINCH to get that behavior.
Re: typeset -p on an empty integer variable is an error. (plus -v test w/ array elements)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 1/11/13 1:27 PM, Dan Douglas wrote: > Bash treats the variable as essentially undefined until given at least an > empty value. This is what Posix says: A variable is a parameter denoted by a name. A parameter is set if it has an assigned value (null is a valid value). Once a variable is set, it can only be unset by using the unset special built-in command. A variable is unset until it has been assigned a value. Setting attributes for a variable can change how it behaves when a value is assigned, but does not assign a value itself, and the variable remains unset -- a simple placeholder -- until it gets one. I know there are some inconsistencies in how bash treats these `invisible' variables, but the above is the way things are supposed to be. Chet - -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (Darwin) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlDwi7gACgkQu1hp8GTqdKsD3QCfQ4JahzpSlhLiSdyTe1cLAU3R LbEAnjzZFXI96APxrxfDlLDk9a9+lPuE =n1+I -END PGP SIGNATURE-
Re: checkwinsize only works in interactive shells
On 1/11/13 4:58 PM, DJ Mills wrote: > As of right now, WINCH will not update COLUMNS or LINES in a > non-interactive shell. > > I've had to use: > trap 'COLUMNS=$(tput cols); LINES=$(tput lines)' WINCH > to get that behavior. And that hasn't changed. Maybe I should look at changing it. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: printf %q represents null argument as empty string.
Am 11.01.2013 22:05, schrieb Dan Douglas:
> On Friday, January 11, 2013 09:39:00 PM John Kearney wrote:
>> Am 11.01.2013 19:38, schrieb Dan Douglas:
>>> $ set --; printf %q\\n "$@"
>>> ''
>>>
>>> printf should perhaps only output '' when there is actually a
> corresponding
>>> empty argument, else eval "$(printf %q ...)" and similar may give
> different
>>> results than expected. Other shells don't output '', even mksh's ${var@Q}
>>> expansion. Zsh's ${(q)var} does.
>> that is not a bug in printf %q
>>
>> it what you expect to happen with "${@}"
>> should that be 0 arguments if $# is 0.
>>
>> I however find the behavior irritating, but correct from the description.
>>
>> to do what you are suggesting you would need a special case handler for this
>> "${@}" as oposed to "${@}j" or any other variation.
>>
>>
>> what I tend to do as a workaround is
>>
>> printf() {
>> if [ $# -eq 2 -a -z "${2}" ];then
>> builtin printf "${1}"
>> else
>> builtin printf "${@}"
>> fi
>> }
>>
>>
>> or not as good but ok in most cases something like
>>
>> printf "%q" ${1:+"${@}"}
>>
>>
> I don't understand what you mean. The issue I'm speaking of is that printf %q
> produces a quoted empty string both when given no args and when given one
> empty arg. A quoted "$@" with no positional parameters present expands to
> zero
> words (and correspondingly for "${arr[@]}"). Why do you think "x${@}x" is
> special? (Note that expansion didn't even work correctly a few patchsets ago.)
>
> Also as pointed out, every other shell with a printf %q feature disagrees
> with
> Bash. Are you saying that something in the manual says that it should do
> otherwise? I'm aware you could write a wrapper, I just don't see any utility
> in the default behavior.
um maybe an example will calrify my attempted point
set -- arg1 arg2 arg3
set -- "--(${@})--"
printf "<%q> " "${@}"
<--\(arg1>
set --
set -- "--(${@})--"
printf "<%q> " "${@}"
<--\(\)-->
so there is always at least one word or one arg, just because its "${@}"
should not affect this behavior.
is that clearer as such bash is doing the right thing as far as I'm
concerned, truthfully its not normally what I want but that is beside
the point consistency is more important, especially when its so easy to
work around.
the relevant part of the man page is
When there are no array members, ${name[@]} expands to nothing.
<<>>>
If the double-quoted expansion occurs within a word, the expansion of
the first parameter is joined with the beginning part of the
original word, and the expansion of the last parameter is joined
with the last part of the original word. This is analogous to the
expansion
of the special parameters * and @ (see Special Parameters
above). ${#name[subscript]} expands to the length of
${name[subscript]}. If sub-
script is * or @, the expansion is the number of elements in the
array. Referencing an array variable without a subscript is
equivalent to
referencing the array with a subscript of 0.
so
set --
printf "%q" "${@}"
becomes
printf "%q" ""
which is correct as ''
Re: typeset -p on an empty integer variable is an error. (plus -v test w/ array elements)
Am 11.01.2013 22:34, schrieb Dan Douglas:
> On Friday, January 11, 2013 09:48:32 PM John Kearney wrote:
>> Am 11.01.2013 19:27, schrieb Dan Douglas:
>>> Bash treats the variable as essentially undefined until given at least an
>>> empty value.
>>>
>>> $ bash -c 'typeset -i x; [[ -v x ]]; echo "$?, ${x+foo}"; typeset -p x'
>>> 1,
>>> bash: line 0: typeset: x: not found
>>> $ ksh -c 'typeset -i x; [[ -v x ]]; echo "$?, ${x+foo}"; typeset -p x'
>>> 0,
>>> typeset -i x
>>>
>>> Zsh implicitly gives integers a zero value if none are specified and the
>>> variable was previously undefined. Either the ksh or zsh ways are fine IMO.
>>>
>>> Also I'll throw this in:
>>>
>>> $ arr[1]=test; [[ -v arr[1] ]]; echo $?
>>> 1
>>>
>>> This now works in ksh to test if an individual element is set, though it
>>> hasn't always. Maybe Bash should do the same? -v is tricky because it adds
>>> some extra nuances to what it means for something to be defined...
>>>
>> Personally I like the current behavior, disclaimer I use nounset.
>> I see no problem with getting people to initialize variables.
> How is this relevant? It's an inconsistency in the way set/unset variables
> are normally handled. You don't use variadic functions? Unset variables /
> parameters are a normal part of most scripts.
>
>> it is a more robust programming approach.
> I strongly disagree. (Same goes for errexit.)
>
:)
we agree on errexit however SIGERROR is another matter quite like that.
Note the only reason I don't like errexit is because it doesn't tell you
why it exited, nounset deos.
no unset is very valuable. during the entire testing and validation
phase. Admittedly bash is more of a hobby for me. but I still have unit
testing for the function and more complex harness testing for the higher
level stuff.
Before I ship ship code I may turn it off but normally if its really
critical I won't use bash for it anyway, I mainly use bash for analysis.
as such if bash stops because it finds a unset variable it is always a
bug that bash has helped me track down.
I guess it also depends on how big your scripts are I guess up to a
couple thousand lines is ok but once you get into the 10s of thousands
to keep your sanity and keep a high reliability you become more and more
strict with what you allow, strict naming conventions and coding styles.
setting nounset is in the same category of setting warnings to all and
and treat warnings as errors.
but then again I do mission critical designs so I guess I have a
different mindset.
Re: printf %q represents null argument as empty string.
On Saturday, January 12, 2013 02:35:34 AM John Kearney wrote:
> so there is always at least one word or one arg, just because its "${@}"
> should not affect this behavior.
...
> printf "%q" "${@}"
> becomes
> printf "%q" ""
>
> which is correct as ''
No, "${@}" doesn't always become at least one word. "$@" can expand to zero or
more words. It can become nothing, one word, or more, possibly concatenated
with adjacent words. That's completely beside the point.
BTW, your wrappers won't work. A wrapper would need to implement format string
parsing in order to determine which argument(s) correspond with %q so that
they could be removed from the output if not given an argument. It also would
have to work around handling -v, which can take up either 1 or 2 args, plus
possibly --. It isn't feasible to wrap printf itself this way.
I just used "$@" as an example of something that can expand to zero words and
preserves exactly the positional parameters present (possibly none). This is
important because printf %q is often used to safely create a level of escaping
that guarantees getting out exactly what you put in.
$ ksh -c 'f() { cmd=$(printf "%q " "$@"); }; f; eval x=moo "$cmd"; echo "$x"'
moo
Bash yields a "command not found" error for obvious reasons, but it goes
against the spirit of what %q is supposed to do IMHO. It's a minor detail.
This isn't even a particularly good example.
See Chet's previous message for the actual explanation. The reason is to be
consistent with the defaults for other format specifiers which act like they
were given a null or zero argument if more formats than arguments are given.
We already had pretty much this same discussion here:
http://lists.gnu.org/archive/html/bug-bash/2012-12/msg00083.html
It somehow slipped my mind.
--
Dan Douglas
