Re: Any chance of multi-dimensional arrays?

[Aharon Robbins]

In article ,
Eduardo Bustamante   wrote:
>There are a lot of general purpose languages (not shell languages), that
>support multi-dimensional arrays. And these languages can call external
>tools just fine. Python, Perl, Ruby, ... pick one. Even Awk has faked
>support for multi-dimensional arrays.

Off topic, but current GNU Awk has true multidimensional arrays.

Arnold
-- 
Aharon (Arnold) Robbins arnold AT skeeve DOT com
P.O. Box 354Home Phone: +972  8 979-0381
Nof Ayalon  Cell Phone: +972 50 729-7545
D.N. Shimshon 99785 ISRAEL

Re: fd leak with {fd}>

[Chet Ramey]

On 11/22/12 5:18 PM, Sam Liddicott wrote:

> > As an interesting aside it seems not to be possible to close the FD
> within
> > the block either:
> >
> > { echo $fd ; eval exec "$fd>&-" ; } {fd}> /dev/null
> 
> But this is not.  There should be a way to ensure the fd's survival while
> allowing it to be closed within the block.  I will fix this for the next
> version.
> 
> 
> It may well be closed within the block; I meant to state that externally it
> was visible despite the internal close.

Part of the rationale for the {x} redirection syntax is to give the user
a handle to the file descriptor and allow him to maniuplate it, so it
seems like the shell should honor the user's close request.  I think the
shell should not close it when the block exits, so the user can close it
when he chooses.

Chet

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: problem with extended regular expression in bash 4.1

[Greg Wooledge]

On Sat, Nov 24, 2012 at 10:29:30PM +, Lawrence Steeger wrote:
> Alex Chupin (achupin  cisco.com> writes:
> > $ bash --version; s=12345;if [[ "$s" =~ '^[0-9]+$' ]]; then echo it is a
> number; else echo it is NOT a number; fi

> The single quotes are being used in the match.  If you remove them,
> it will work.

The second sentence is correct, but the first is not.  The single quotes
(or any other kind of quoting) override the =~ and cause the right hand
side to be matched as a string, rather than as a regular expression (or
even a glob).

However, the quotes themselves are not being treated as literal
characters.  They are removed before the string comparison occurs.

imadev:~$ x=abc; if [[ $x =~ 'abc' ]]; then echo match; fi
match

As others have already said, the best way to use =~ is to place the
regular expression in a variable, and then use that variable without
quotes on the right hand side of =~.

re='^[st](uff)*$'; if [[ $string =~ $re ]]; then ...

Re: fd leak with {fd}>

[Chet Ramey]

On 11/23/12 2:04 AM, Pierre Gaston wrote:

> It seems rather counter intuitive that the fd is not closed after leaving
> the block.
> With the normal redirection the fd is only available inside the block
> 
> $ { : ;} 3>&1;echo bar >&3
> -bash: 3: Bad file descriptor 
> 
> if 3 is closed why should I expect {fd} to be still open?

Because that's part of the reason to have {x}: so the user can handle the
disposition of the file descriptor himself.

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: fd leak with {fd}>

[Pierre Gaston]

On Mon, Nov 26, 2012 at 3:37 PM, Chet Ramey  wrote:

> On 11/23/12 2:04 AM, Pierre Gaston wrote:
>
> > It seems rather counter intuitive that the fd is not closed after leaving
> > the block.
> > With the normal redirection the fd is only available inside the block
> >
> > $ { : ;} 3>&1;echo bar >&3
> > -bash: 3: Bad file descriptor
> >
> > if 3 is closed why should I expect {fd} to be still open?
>
> Because that's part of the reason to have {x}: so the user can handle the
> disposition of the file descriptor himself.

.
I don't see any difference between 3> and {x}> except that the later free
me from the hassle of avoid conflicting fd




>

Re: problem with extended regular expression in bash 4.1

[chupin007]

> It changed between bash 3.1 and 3.2, documented in the NEWS file. There is
> 
> a "compat31" option that can be turned on to restore the 3.1 behavior.
> 

As you see it works for me in 3.25

Re: Any chance of multi-dimensional arrays?

[Greg Wooledge]

On Sun, Nov 25, 2012 at 06:33:19AM +0100, Rene Herman wrote:
> I'm currently writing a larger bash script to manage my (ogg vorbis) 
> music collection, including maintaining tags. Vorbis files can and 
> (mine) often will contain repeated tags such as, say, "artist=David 
> Crosby" and "artist=Graham Nash" and so on and as such, my "artist" 
> variable is an array.

Sounds like what you actually want is an associative array of lists,
which is not something bash provides.  You can mimic it using associative
arrays with specially mangled keys (bash 4.0 and up), but it is not a
native feature of the language.

Re: problem with extended regular expression in bash 4.1

[chupin007]

Thank you guys for comprehensive explanation.

Regards,
Alexander Chupin

Re: fd leak with {fd}>

[Pierre Gaston]

On Mon, Nov 26, 2012 at 3:41 PM, Pierre Gaston wrote:

>
>
> On Mon, Nov 26, 2012 at 3:37 PM, Chet Ramey  wrote:
>
>> On 11/23/12 2:04 AM, Pierre Gaston wrote:
>>
>> > It seems rather counter intuitive that the fd is not closed after
>> leaving
>> > the block.
>> > With the normal redirection the fd is only available inside the block
>> >
>> > $ { : ;} 3>&1;echo bar >&3
>> > -bash: 3: Bad file descriptor
>> >
>> > if 3 is closed why should I expect {fd} to be still open?
>>
>> Because that's part of the reason to have {x}: so the user can handle the
>> disposition of the file descriptor himself.
>
> .
> I don't see any difference between 3> and {x}> except that the later free
> me from the hassle of avoid conflicting fd
>

It seems that ksh93 behaves just like bash in this regard
Well, as I don't use it I don't really care, but I vote for this as a bug
as I fail to see the benefit of this behavior as i find it useless and not
consistent with the normal redirection.

Re: problem with extended regular expression in bash 4.1

[Davide Brini]

On Mon, 26 Nov 2012 15:03:35 +0100, Davide Brini  wrote:

> On Mon, 26 Nov 2012 05:40:09 -0800 (PST), chupin...@gmail.com wrote:
> 
> > > It changed between bash 3.1 and 3.2, documented in the NEWS file.
> > > There is
> > > 
> > > a "compat31" option that can be turned on to restore the 3.1 behavior.
> > > 
> > 
> > As you see it works for me in 3.25
> 
> Then maybe you have the compat31 option set (or your distro sets it for
> you), or you may have a patched, non-vanilla version of bash (redhat is
> well known for doing this).
> 

And indeed:

https://groups.google.com/forum/?fromgroups=#!topic/gnu.bash.bug/hq7mAJQB31s


-- 
D.

Re: problem with extended regular expression in bash 4.1

[Davide Brini]

On Mon, 26 Nov 2012 05:40:09 -0800 (PST), chupin...@gmail.com wrote:

> > It changed between bash 3.1 and 3.2, documented in the NEWS file. There
> > is
> > 
> > a "compat31" option that can be turned on to restore the 3.1 behavior.
> > 
> 
> As you see it works for me in 3.25

Then maybe you have the compat31 option set (or your distro sets it for
you), or you may have a patched, non-vanilla version of bash (redhat is
well known for doing this).

-- 
D.

Re: fd leak with {fd}>

[Sam Liddicott]

On Mon, Nov 26, 2012 at 1:49 PM, Pierre Gaston wrote:

>
>
> On Mon, Nov 26, 2012 at 3:41 PM, Pierre Gaston wrote:
>
>>
>>
>> On Mon, Nov 26, 2012 at 3:37 PM, Chet Ramey  wrote:
>>
>>> On 11/23/12 2:04 AM, Pierre Gaston wrote:
>>>
>>> > It seems rather counter intuitive that the fd is not closed after
>>> leaving
>>> > the block.
>>> > With the normal redirection the fd is only available inside the block
>>> >
>>> > $ { : ;} 3>&1;echo bar >&3
>>> > -bash: 3: Bad file descriptor
>>> >
>>> > if 3 is closed why should I expect {fd} to be still open?
>>>
>>> Because that's part of the reason to have {x}: so the user can handle the
>>> disposition of the file descriptor himself.
>>
>> .
>> I don't see any difference between 3> and {x}> except that the later free
>> me from the hassle of avoid conflicting fd
>>
>
> It seems that ksh93 behaves just like bash in this regard
> Well, as I don't use it I don't really care, but I vote for this as a bug
> as I fail to see the benefit of this behavior as i find it useless and not
> consistent with the normal redirection.
>
>

And also potentially renders the feature useless.

I had hours of debugging why my {fd} >( ... ) co-processes were not
terminating. A less tenacious user would give up on either {fd} or >( ... )
or both

For users to actually USE these features, the principle of least surprise
should be followed.

Ironically I was changing from my own method of allocating fd's (
http://blog.sam.liddicott.com/2012/03/local-variables-are-great.html) to
the simple "bash do it for me" and things broke because they didn't work
like anything similar in bash.

I would expect: exec {fd}>( ... }
to keep the descriptor open. if the user wants that he can do:
{ exec {fd}>... ; ... ; }
if he wants it to be closed when the scope ends he will do:
{ ... ; } {fd}>...

So I think this surprising behaviour adds nothing that wasn't there
already, and nothing that the user expects and also adds things that the
user will not expect

Sam

Re: fd leak with {fd}>

[Chet Ramey]

On 11/26/12 8:41 AM, Pierre Gaston wrote:
> 
> 
> On Mon, Nov 26, 2012 at 3:37 PM, Chet Ramey  > wrote:
> 
> On 11/23/12 2:04 AM, Pierre Gaston wrote:
> 
> > It seems rather counter intuitive that the fd is not closed after 
> leaving
> > the block.
> > With the normal redirection the fd is only available inside the block
> >
> > $ { : ;} 3>&1;echo bar >&3
> > -bash: 3: Bad file descriptor
> >
> > if 3 is closed why should I expect {fd} to be still open?
> 
> Because that's part of the reason to have {x}: so the user can handle the
> disposition of the file descriptor himself.
> 
> . 
> I don't see any difference between 3> and {x}> except that the later free
> me from the hassle of avoid conflicting fd

That's not really an issue.  Bash goes to great effort to allow users to
specify fds it is already using internally.

The user/shell programmer getting a handle to the fd is one benefit.  The
ability to use those named fds in operators that don't allow words (e.g.
variable expansions) to replace the explicit file descriptor number is
another.

David Korn beat all of us in implementing this feature (we first began
discussing it in 2005).  I should ask him if he has additional insight.

Chet

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: fd leak with {fd}>

[Chet Ramey]

On 11/26/12 9:26 AM, Sam Liddicott wrote:

> It seems that ksh93 behaves just like bash in this regard
> Well, as I don't use it I don't really care, but I vote for this as a
> bug as I fail to see the benefit of this behavior as i find it useless
> and not consistent with the normal redirection.
>  
> 
> 
> And also potentially renders the feature useless.

OK, so how does a misunderstanding in how a feature works render it
useless?  Maybe better documentation is enough to make the features
clear.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: problem with extended regular expression in bash 4.1

[Chet Ramey]

On 11/26/12 9:07 AM, Davide Brini wrote:

>> Then maybe you have the compat31 option set (or your distro sets it for
>> you), or you may have a patched, non-vanilla version of bash (redhat is
>> well known for doing this).
>>
> 
> And indeed:
> 
> https://groups.google.com/forum/?fromgroups=#!topic/gnu.bash.bug/hq7mAJQB31s

Redhat, Debian, and SUSE have done this kind of thing in the past.

It wouldn't be so bad, but they tend not to

1.  Change the installed manual page to document what they've done;
2.  Tell anyone they've done it

Chet


-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: fd leak with {fd}>

[Sam Liddicott]

On Mon, Nov 26, 2012 at 5:02 PM, Chet Ramey  wrote:

> On 11/26/12 9:26 AM, Sam Liddicott wrote:
>
> > It seems that ksh93 behaves just like bash in this regard
> > Well, as I don't use it I don't really care, but I vote for this as a
> > bug as I fail to see the benefit of this behavior as i find it
> useless
> > and not consistent with the normal redirection.
> >
> >
> >
> > And also potentially renders the feature useless.
>
> OK, so how does a misunderstanding in how a feature works render it
> useless?  Maybe better documentation is enough to make the features
> clear.
>

I explained how in the lines of my response that you deleted.

It is potentially useless because:

1. it is non-obvious, most users will not expect this behaviour (unless
already initiated into the secret) and so will not try to get that benefit.
2. it is an unexpected side effect and will do something which is not
expected, and be widely and "wrongly" labelled as buggy or unreliable by
misunderstanding users and therefore damage the bash reputation.
3. there already exists simple and explicit way to get the supposed benefit
using the existing mechanism "exec"

Certainly better documentation is required as a defense against the wrong
understanding being given, but users will still not expect this subtle
different and will not naturally fall to this interpretation or expect
therefore that there is any need to read the documentation.

I fear that the better documentation merely will serve as an excuse for
unexpected behaviour of a confusing and not-needed side-feature of an
important feature.

Sam

Re: savestring() macro in general.h runs function arguments twice.

[Chet Ramey]

On 11/24/12 2:32 AM, John E. Malmberg wrote:
> The execute_cmd.c module uses make_command_string() as a parameter to the
> savestring() macro.
> 
> This causes the savestring() macro to call that function twice().

Thanks, good catch.  I made the same temporary variable fix you did.

> I have not done any timing tests to see how this is impacting performance,
> but it definitely shows up when stepping through commands doing debugging.
> 
> This issue was first noticed in bash 1.14.7 and is also seen in the current
> bash 4.2.39.
> 
> I have locally patched execute_cmd.c to use an intermediate variable. The
> proper fix should probably be in general.h.

That's tough to do and still keep it in an expression context.

Chet

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: fd leak with {fd}>

[Chet Ramey]

On 11/26/12 12:11 PM, Sam Liddicott wrote:

> I explained how in the lines of my response that you deleted.  
> 
> It is potentially useless because:
> 
> 1. it is non-obvious, most users will not expect this behaviour (unless
> already initiated into the secret) and so will not try to get that benefit.

OK, so there's a need for documentation on how it differs from anonymous
or, in certain circumstances, explicitly numbered redirections.

> 2. it is an unexpected side effect and will do something which is not
> expected, and be widely and "wrongly" labelled as buggy or unreliable by
> misunderstanding users and therefore damage the bash reputation.

Hmmm...again, it's a new feature which shares some things with the old but
has its own behavior.  Think of it as something closer to open/fcntl than
the historical redirections.  The differences can be made more apparent
with better documentation.  Certainly nobody has used it by accident
before.

> 3. there already exists simple and explicit way to get the supposed benefit
> using the existing mechanism "exec"

Not quite.  You still have to pick the file descriptor you want to use with
`exec'.  But you are not being forced to use it -- by all means, if you
think it's not what you need or want, feel free to avoid it and encourage
your friends to do the same.  There have been unsuccessful new features --
the case-modifying expansions are one example of a swing and miss.

Chet

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Any chance of multi-dimensional arrays?

[Chet Ramey]

On 11/25/12 12:33 AM, Rene Herman wrote:
> Good day.
> 
> I know that bash arrays are 1 dimensional -- but are there any plans for
> providing multi-dimensional arrays?

I don't have any current plans to do so.  I would take a look at any
contributed code to add them, though.

Chet


-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: RFE: read -d STRING

[Chet Ramey]

On 11/23/12 7:40 PM, Chris F.A. Johnson wrote:
> 
>I would find it very useful to allow a string of delimiters to be
>used with 'read -d', with any member of the string terminating the
>input and the character used being stored in a variable, e.g.
>READ_DELIM.

Yes, this is still on the list, as is multibyte character support for
the delimiter.


-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: fd leak with {fd}>

[Chris F.A. Johnson]

On Mon, 26 Nov 2012, Chet Ramey wrote:
,,,

There have been unsuccessful new features -- the case-modifying
expansions are one example of a swing and miss.


   A miss? I use them a lot.

--
   Chris F.A. Johnson, 
   Author:
   Pro Bash Programming: Scripting the GNU/Linux Shell (2009, Apress)
   Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)

Re: fd leak with {fd}>

[Dennis Williamson]

On Nov 26, 2012 2:48 PM, "Chet Ramey"  wrote:
>
> On 11/26/12 12:11 PM, Sam Liddicott wrote:
>
> > I explained how in the lines of my response that you deleted.
> >
> > It is potentially useless because:
> >
> > 1. it is non-obvious, most users will not expect this behaviour (unless
> > already initiated into the secret) and so will not try to get that
benefit.
>
> OK, so there's a need for documentation on how it differs from anonymous
> or, in certain circumstances, explicitly numbered redirections.
>
> > 2. it is an unexpected side effect and will do something which is not
> > expected, and be widely and "wrongly" labelled as buggy or unreliable by
> > misunderstanding users and therefore damage the bash reputation.
>
> Hmmm...again, it's a new feature which shares some things with the old but
> has its own behavior.  Think of it as something closer to open/fcntl than
> the historical redirections.  The differences can be made more apparent
> with better documentation.  Certainly nobody has used it by accident
> before.
>
> > 3. there already exists simple and explicit way to get the supposed
benefit
> > using the existing mechanism "exec"
>
> Not quite.  You still have to pick the file descriptor you want to use
with
> `exec'.  But you are not being forced to use it -- by all means, if you
> think it's not what you need or want, feel free to avoid it and encourage
> your friends to do the same.  There have been unsuccessful new features --
> the case-modifying expansions are one example of a swing and miss.
>
> Chet
>

Case modification in Bash is a highly viewed and highly upvoted question on
Stack Overflow.

http://stackoverflow.com/a/2265268/26428

> --
> ``The lyf so short, the craft so long to lerne.'' - Chaucer
>  ``Ars longa, vita brevis'' - Hippocrates
> Chet Ramey, ITS, CWRUc...@case.edu
http://cnswww.cns.cwru.edu/~chet/
>

How to initialize a read-only, global, associative array in Bash?

[Tim Friske]

Hi folks,

I execute the following code in Bash version "GNU bash, Version
4.2.39(1)-release (x86_64-redhat-linux-gnu)":

function foobar {
  declare -rgA FOOBAR=([foo]=bar)
}
foobar
declare -p FOOBAR
# Output: declare -Ar FOOBAR='()'

Why doesn't Bash initialize FOOBAR with ([foo]=bar) according to
declare -p? The same declaration works outside of a function, e.g.

declare -rgA FOOBAR=([foo]=bar)
declare -p FOOBAR
# Output: declare -Ar FOOBAR='([foo]="bar" )'

Similarly the following code but without FOOBAR being read-only works:

function foobar {
  declare -gA FOOBAR
  FOOBAR=([foo]=bar)
}
foobar
declare -p FOOBAR
# Output: declare -A FOOBAR='([foo]="bar" )'

Is this a bug or feature?

Cheers,
Tim

BTW: I couldn't find a "bashbug" RPM package in the Fedora 17
repositories; that's why I wrote this formless mail. Sorry for that.
--
`°<
C92A E44E CC19 58E2 FA35 4048 2217 3C6E 0338 83FC

Re: fd leak with {fd}>

[Chet Ramey]

On 11/26/12 6:30 PM, Dennis Williamson wrote:

> Case modification in Bash is a highly viewed and highly upvoted question on
> Stack Overflow.

Wow.  I stand corrected.  I had the impression it did not get very much
use based on the discussion -- or lack thereof -- here and on help-bash.

Chet

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Any chance of multi-dimensional arrays?

[Rene Herman]

On 11/26/2012 11:27 PM, Chet Ramey wrote:


I know that bash arrays are 1 dimensional -- but are there any
plans for providing multi-dimensional arrays?


I don't have any current plans to do so.  I would take a look at any
contributed code to add them, though.


Thanks for the reply. It's fairly unlikely that I will myself be able to 
try (on a sensible timescale) but it's good to know they're not vetoed.


Not being hugely experienced in bash, I was sort of impressed with there 
seemingly not being any good point in NOT using it for my collection 
management stuff -- except maybe those multi-dimensional arrays. Thought 
I'd go see what their status was...


Rene

Re: fd leak with {fd}>

[Pierre Gaston]

On Mon, Nov 26, 2012 at 10:48 PM, Chet Ramey  wrote:

> On 11/26/12 12:11 PM, Sam Liddicott wrote:
> > 3. there already exists simple and explicit way to get the supposed
> benefit
> > using the existing mechanism "exec"
>
> Not quite.  You still have to pick the file descriptor you want to use with
> `exec'.  But you are not being forced to use it -- by all means, if you
> think it's not what you need or want, feel free to avoid it and encourage
> your friends to do the same.  There have been unsuccessful new features --
> the case-modifying expansions are one example of a swing and miss.


You seem to say that there are 2 aspects of this new feature, giving
 control on the fd
and letting the system choose the number.

Let's see the first aspect, you are saying that leaving the fd open doing
"{ : } {fd}>file"
is a feature to let the user control the file descriptor.

But why would one use this when you can do:

exec {fd}>file

That is there is already exec to answer the problem of letting the user
manage the fd,
Why would you use another new, non intuitive non consistent feature?

For me having bashing assigning the fd number solves another problem.