Re: Curious case of arithmetic expansion

[Florian Mayer]

Ok, I accept your points, but please read on and decide after that.

> to do anything but assign a value to `var'.  Very few people, when asked,
> would say that it were more intuitive to cause a variable named `bar' to
> spring into existence with the value 7. If you want nameref behavior, you
> have to explicitly declare it.

But why is nameref behavior in the evaluation part activated by default
and not for the assignments? I think that one should
either make nameref behavior the default case for both, 
assignment and evaluation, or the non-default case for both.

In the latter case, where namerefs are off for both things, the lines (1) and 
(2)
1) var=bar; bar=3; echo ((var))
2) var=bar; ((var=3))   or var=bar; bar=0; ((var++))
should lead to an error message!
However the current status quo is, that bash happily does (1) as you’d 
expect but gives out no error in (2) and overwrites (1).

> That is variable indirection. It has superficially similar effects to what
> we are discussing, which is arithmetic expansion; however, it has nothing
> to do with the behavior of (( )) or $(( )).

Again, I never wrote, that, from a technical point of view, it has something to 
do
with (()) or $(()). Merely the visible behavior is similar in the cases I 
showed.

> Am 24.04.2017 um 01:43 schrieb Chet Ramey :
> 
> On 4/23/17 4:25 PM, Florian Mayer wrote:
>>> That's not a reasonable expectation.
>> Why not? Why is it not reasonable to expect an intuitive
>> result from (())? The most intuitive thing, in my opinion,
>> would be to use nameref for side effects by default, because in order
>> to get a value from an id, (()) already follows namerefs.
> 
> The thing that makes that result intuitive for you is your opinion about
> how things should work.  That's fine. I don't happen to share your opinion
> of what is "intuitive" in this case.
> 
> However, it is unreasonable to expect
> 
>   var=bar
>   (( var=7 ))
> 
> to do anything but assign a value to `var'.  Very few people, when asked,
> would say that it were more intuitive to cause a variable named `bar' to
> spring into existence with the value 7. If you want nameref behavior, you
> have to explicitly declare it.
> 
> 
>>> It's not indirection, and I am not sure why you show the completely
>> I was mentioning that, not because I use „a mental model“ that falsely 
>> unifies
>> both things, but because I wanted to point out that there exists something
>> with a similar behavior.
>> 
>> And if ${!} does not portray some kind of indirection, what do you
>> call it then?
> 
> That is variable indirection. It has superficially similar effects to what
> we are discussing, which is arithmetic expansion; however, it has nothing
> to do with the behavior of (( )) or $(( )).

Syntax error near unexpected token `newline' within loops

[werner]

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc -I/home/abuild/rpmbuild/BUILD/bash-4.4 
-L/home/abuild/rpmbuild/BUILD/bash-4.4/../readline-7.0
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' 
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-suse-linux-gnu' 
-DCONF_VENDOR='suse' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL 
-DHAVE_CONFIG_H   -I.  -I. -I./include -I./lib   -fmessage-length=0 
-grecord-gcc-switches -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector-strong 
-funwind-tables -fasynchronous-unwind-tables -g  -D_GNU_SOURCE -DRECYCLES_PIDS 
-Wall -g -Wuninitialized -Wextra -Wno-switch-enum -Wno-unused-variable 
-Wno-unused-parameter -Wno-parentheses -ftree-loop-linear -pipe -DBNC382214=0 
-DIMPORT_FUNCTIONS_DEF=0 -fprofile-use -fprofile-correction
uname output: Linux noether 4.4.57-18.3-default #1 SMP Thu Mar 30 06:39:47 UTC 
2017 (39c8557) x86_64 x86_64 x86_64 GNU/Linux
Machine Type: x86_64-suse-linux-gnu

Bash Version: 4.4
Patch Level: 12
Release Status: release

Description:
Since 4.3 including 4.4 bash shows a bug in loops using nested ((...)), 
$((..)), and $(...)

Repeat-By:
The bash code

V_NAME=Friday
for (( INDEX=0; INDEX<$((10-$(expr length $V_NAME))); INDEX++ ))
do
echo $INDEX
done

leads to

bash: syntax error near unexpected token `newline'

whereas using backticks simply works

V_NAME=Friday
for (( INDEX=0; INDEX<$((10-`expr length $V_NAME`)); INDEX++ ))
do
echo $INDEX
done

and yes

V_NAME=Friday
for (( INDEX=0; INDEX<$((10-${#V_NAME})) ; INDEX++ ))
do
echo $INDEX
done

does also work, nevertheless using $(...) in the very first example is allowed

Re: Syntax error near unexpected token `newline' within loops

[Pierre Gaston]

On Mon, Apr 24, 2017 at 1:59 PM,  wrote:

> (...)
>
> and yes
>
> V_NAME=Friday
> for (( INDEX=0; INDEX<$((10-${#V_NAME})) ; INDEX++ ))
> do
> echo $INDEX
> done
>
> does also work, nevertheless using $(...) in the very first example is
> allowed
>
>
or even  for (( INDEX=0; INDEX<(10-${#V_NAME}) ; INDEX++ ))
but otherwise yes, syntax looks ok

Re: Syntax error near unexpected token `newline' within loops

[Dr. Werner Fink]

On Mon, Apr 24, 2017 at 02:25:41PM +0300, Pierre Gaston wrote:
> On Mon, Apr 24, 2017 at 1:59 PM,  wrote:
> 
> > (...)
> >
> > and yes
> >
> > V_NAME=Friday
> > for (( INDEX=0; INDEX<$((10-${#V_NAME})) ; INDEX++ ))
> > do
> > echo $INDEX
> > done
> >
> > does also work, nevertheless using $(...) in the very first example is
> > allowed
> >
> >
> or even  for (( INDEX=0; INDEX<(10-${#V_NAME}) ; INDEX++ ))
> but otherwise yes, syntax looks ok

Yep ... the code could be more efficiently like

 V_NAME=Friday
 typeset -i len=$((10 - ${#V_NAME}))
 for (( INDEX=0; INDEX < len; INDEX++ )); do   echo $INDEX; done

... nevertheless even the intricate code should not cause an error I guess :)

-- 
  "Having a smoking section in a restaurant is like having
  a peeing section in a swimming pool." -- Edward Burr


signature.asc
Description: PGP signature

Re: Curious case of arithmetic expansion

[Steve Amerige]

  
  
It is essential to not break backward
  compatibility. Imagine the huge number of scripts that would be
  impacted by the semantic shift you're suggesting. If a proposal
  were to be made that would cause backward incompatibility, then
  any such proposal should be rejected unless there are truly
  compelling reasons for the proposal. Now, if you're proposing some
  new shopt (Bash-specific) or set -o (a broader set of sh shells)
  that would deliver some alternative behavior, then at least you
  wouldn't be breaking backward compatibility. However, any such
  addition would complicate the test matrix for shell behavior and
  should again be subject to very strong scrutiny and strong
  community desire for such a feature. I'm not sensing that this is
  the case for what you're describing.
  
  Best regards,
  Steve Amerige
  Eggsh: A Bash-Scripting Platform
  https://eggsh.com
  
  On 4/24/2017 3:59 AM, Florian Mayer wrote:


  
  Ok, I accept your points, but please read on and
decide after that.
  
to do anything but assign a
  value to `var'.  Very few people, when asked,
  would say that it were more intuitive to cause a variable
  named `bar' to
  spring into existence with the value 7. If you want nameref
  behavior, you
  have to explicitly declare it.
  
  But why is nameref behavior in the
evaluation part activated by default
  and not for the assignments? I think that one should
  either make nameref behavior the default case for
both, 
  assignment and evaluation, or the non-default case
for both.
  
  
  In the latter case, where namerefs are off for both
things, the lines (1) and (2)
  1) var=bar; bar=3; echo ((var))
  2) var=bar; ((var=3))   or var=bar; bar=0; ((var++))
  should lead to an error message!
  However the current status quo is, that bash happily
does (1) as you’d 
  expect but gives out no error in (2) and overwrites
(1).
  
  
  
That is variable indirection.
  It has superficially similar effects to what
  we are discussing, which is arithmetic expansion; however, it
  has nothing
  to do with the behavior of (( )) or $(( )).
  
  Again, I never wrote, that, from a technical point
of view, it has something to do
  with (()) or $(()). Merely the visible behavior is
similar in the cases I showed.
  
  
  

  Am 24.04.2017 um 01:43 schrieb Chet Ramey :
  
  
On 4/23/17 4:25 PM, Florian Mayer wrote:
  
That's not a reasonable
  expectation.

Why not? Why is it not reasonable to expect an intuitive
result from (())? The most intuitive thing, in my
opinion,
would be to use nameref for side effects by default,
because in order
to get a value from an id, (()) already follows
namerefs.
  
  
  The thing that makes that result intuitive for you is your
  opinion about
  how things should work.  That's fine. I don't happen to
  share your opinion
  of what is "intuitive" in this case.
  
  However, it is unreasonable to expect
  
  	var=bar
  	((
  var=7 ))
  
  to do anything but assign a value to `var'.  Very few
  people, when asked,
  would say that it were more intuitive to cause a variable
  named `bar' to
  spring into existence with the value 7. If you want
  nameref behavior, you
  have to explicitly declare it.
  
  
  
It's not indirection,
  and I am not sure why you show the completely

I was mentioning that, not because I use „a mental
model“ that falsely unifies
both things, but because I wanted to point out that
there exists something
with a similar behavior.

And if ${!} does not portray some kind of
indirection, what do you
call it then?
  
  
  That is variable indirection. It has superficially similar
  effects to what
  we are discussing, which is arithmetic expansion; however,
  it has nothing
  to do with the behavior of (( )) or $(( )).
  

  



  

Re: Syntax error near unexpected token `newline' within loops

[Greg Wooledge]

On Mon, Apr 24, 2017 at 12:59:01PM +0200, wer...@suse.de wrote:
>   V_NAME=Friday
>   for (( INDEX=0; INDEX<$((10-$(expr length $V_NAME))); INDEX++ ))

The outer (( )) in the C-style for loop already create an arithmetic
expression context.  You don't need to use $(( )) inside them.  You can
simply write:

for (( INDEX=0; INDEX<10-${#V_NAME};; INDEX++ ))

By the way, the [ ] index syntax in an indexed (not associative) array
expansion works the same way.  You can simply write:

"${a[x+1]}"

instead of

"${a[$((x+1))]}"

Re: Syntax error near unexpected token `newline' within loops

[Eduardo Bustamante]

On Mon, Apr 24, 2017 at 7:44 AM, Greg Wooledge  wrote:
[...]
> The outer (( )) in the C-style for loop already create an arithmetic
> expression context.  You don't need to use $(( )) inside them.  You can
> simply write:
>
> for (( INDEX=0; INDEX<10-${#V_NAME};; INDEX++ ))

I think this is just to show the bug. i.e. these two should do the same:

dualbus@debian:~$ bash -c 'for (( ; $(($(:))); )); do :; done'
bash: -c: line 0: syntax error near unexpected token `newline'
bash: -c: line 0: `for (( ; $(($(:))); )); do :; done'

dualbus@debian:~$ bash -c 'for (( ; $((`:`)); )); do :; done'

Re: Syntax error near unexpected token `newline' within loops

[Dr. Werner Fink]

On Mon, Apr 24, 2017 at 08:44:11AM -0400, Greg Wooledge wrote:
> On Mon, Apr 24, 2017 at 12:59:01PM +0200, wer...@suse.de wrote:
> > V_NAME=Friday
> > for (( INDEX=0; INDEX<$((10-$(expr length $V_NAME))); INDEX++ ))
> 
> The outer (( )) in the C-style for loop already create an arithmetic
> expression context.  You don't need to use $(( )) inside them.  You can
> simply write:
> 
> for (( INDEX=0; INDEX<10-${#V_NAME};; INDEX++ ))
> 
> By the way, the [ ] index syntax in an indexed (not associative) array
> expansion works the same way.  You can simply write:
> 
> "${a[x+1]}"
> 
> instead of
> 
> "${a[$((x+1))]}"

Yes I know ... nevertheless, the initial code should not trigger an error
... to make it sure, the initial example was given by a user asking me
why this triggers an error, which IMHO it should not.

-- 
  "Having a smoking section in a restaurant is like having
  a peeing section in a swimming pool." -- Edward Burr


signature.asc
Description: PGP signature

Re: Syntax error near unexpected token `newline' within loops

[Greg Wooledge]

On Mon, Apr 24, 2017 at 07:49:36AM -0500, Eduardo Bustamante wrote:
> On Mon, Apr 24, 2017 at 7:44 AM, Greg Wooledge  wrote:
> [...]
> > The outer (( )) in the C-style for loop already create an arithmetic
> > expression context.  You don't need to use $(( )) inside them.  You can
> > simply write:
> >
> > for (( INDEX=0; INDEX<10-${#V_NAME};; INDEX++ ))
> 
> I think this is just to show the bug. i.e. these two should do the same:
> 
> dualbus@debian:~$ bash -c 'for (( ; $(($(:))); )); do :; done'
> bash: -c: line 0: syntax error near unexpected token `newline'
> bash: -c: line 0: `for (( ; $(($(:))); )); do :; done'
> 
> dualbus@debian:~$ bash -c 'for (( ; $((`:`)); )); do :; done'

Yeah, I'm not disputing whether there's actually a bug here, just
pointing out that the code can be simplified to avoid it.  The fact that
no sane person should write code this way is probably why the bug went
undiscovered for so long.

Seriously, "expr length"?!  In a script that is already using bashisms?

Re: Syntax error near unexpected token `newline' within loops

[Reuti]

> Am 24.04.2017 um 14:58 schrieb Greg Wooledge :
> 
> On Mon, Apr 24, 2017 at 07:49:36AM -0500, Eduardo Bustamante wrote:
>> On Mon, Apr 24, 2017 at 7:44 AM, Greg Wooledge  wrote:
>> [...]
>>> The outer (( )) in the C-style for loop already create an arithmetic
>>> expression context.  You don't need to use $(( )) inside them.  You can
>>> simply write:
>>> 
>>> for (( INDEX=0; INDEX<10-${#V_NAME};; INDEX++ ))
>> 
>> I think this is just to show the bug. i.e. these two should do the same:
>> 
>> dualbus@debian:~$ bash -c 'for (( ; $(($(:))); )); do :; done'
>> bash: -c: line 0: syntax error near unexpected token `newline'
>> bash: -c: line 0: `for (( ; $(($(:))); )); do :; done'
>> 
>> dualbus@debian:~$ bash -c 'for (( ; $((`:`)); )); do :; done'
> 
> Yeah, I'm not disputing whether there's actually a bug here, just
> pointing out that the code can be simplified to avoid it.  The fact that
> no sane person should write code this way is probably why the bug went
> undiscovered for so long.

AFAICS in version 4.2.45(1) it works as expected.

-- Reuti


> Seriously, "expr length"?!  In a script that is already using bashisms?



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Using Clang's static analyzer on bash

[Chet Ramey]

On 4/23/17 9:02 PM, Eduardo Bustamante wrote:
> I built bash using scan-build
> (https://clang-analyzer.llvm.org/scan-build.html) and I noticed that
> it was able to detect the null pointer dereference reported earlier by
> Jaren (https://lists.gnu.org/archive/html/bug-bash/2017-04/msg00100.html).
> 
> dualbus@debian:~/src/gnu/bash$ scan-build-3.9 make
> scan-build: Using '/usr/lib/llvm-3.9/bin/clang' for static analysis

I'd be interested in seeing the results.  In my experience with similar
tools, the false positive ratio is very high.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Curious case of arithmetic expansion

[Chet Ramey]

On 4/24/17 3:59 AM, Florian Mayer wrote:
> Ok, I accept your points, but please read on and decide after that.
> 
>> to do anything but assign a value to `var'.  Very few people, when asked,
>> would say that it were more intuitive to cause a variable named `bar' to
>> spring into existence with the value 7. If you want nameref behavior, you
>> have to explicitly declare it.
> But why/ /is nameref behavior in the evaluation part activated by default
> and not for the assignments? I think that one should
> either make nameref behavior the default case for both, 
> assignment and evaluation, or the non-default case for both.

OK, you can certainly have whatever opinion you want.  However, 27 years
ago when I put this into bash, there was considerable benefit from
supporting running variable values through the expression evaluator instead
of simply coercing their value to an integer.  That allowed, in the most
basic use case, a simple arithmetic macro facility.  You could also
do more complicated things like compose complex expressions.

There was no similar benefit from recursively expanding variable names.
There still isn't.  And there is no compelling reason for such a radical
break with existing behavior.  Nobody besides you has ever requested it.

> In the latter case, where namerefs are off for both things, the lines (1)
> and (2)
> 1) var=bar; bar=3; echo ((var))
> 2) var=bar; ((var=3))   or var=bar; bar=0; ((var++))
> should lead to an error message!
> However the current status quo is, that bash happily does (1) as you’d 
> expect but gives out no error in (2) and overwrites (1).

You're saying that something like ((var=3)) should produce an error
instead of assign `3' to `var'? Really? Or that var++ should not behave
identically to `ovar=$var, var=$(( ovar + 1 )), ovar'?

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Syntax error near unexpected token `newline' within loops

[Chet Ramey]

On 4/24/17 6:59 AM, wer...@suse.de wrote:

> Bash Version: 4.4
> Patch Level: 12
> Release Status: release
> 
> Description:
>   Since 4.3 including 4.4 bash shows a bug in loops using nested ((...)), 
> $((..)), and $(...)
> 
> Repeat-By:
> The bash code
> 
>   V_NAME=Friday
>   for (( INDEX=0; INDEX<$((10-$(expr length $V_NAME))); INDEX++ ))
>   do
>   echo $INDEX
>   done
> 
> leads to
> 
> bash: syntax error near unexpected token `newline'

Thanks for the report.  It's a consequence of recursively invoking the
parser for the command substitution, which bash-4.3 and later do.  There's
more and more yacc/bison internal state I find out about as these semi-
obscure reports come in.  I'll fix it.

Chet

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Syntax error near unexpected token `newline' within loops

[Chet Ramey]

On 4/24/17 8:58 AM, Greg Wooledge wrote:

> Seriously, "expr length"?!  In a script that is already using bashisms?

That works on Linux, but not on the BSDs, since Posix says it doesn't
have to.


-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/