5.1 regression?
Hi folks,
a colleague pointed me to a changed behavior of bash 5.1.4 in Debian 11.
Would you mind to take a look at this code?
#! /bin/bash
# set -x
insert()
{
local data="$1"
local lineNumber="$2"
head -n "$lineNumber"
echo ' '
cat <<< "$data"
echo ' '
cat
}
text="this is line one
this is line two
this is line three
this is line four
this is line five
this is line six
this is line seven"
output="$( insert "Hello" 3 <<< "${text}" )"
echo "$output"
On bash 5.0.3 (Debian 10) it shows
this is line one
this is line two
this is line three
Hello
this is line four
this is line five
this is line six
this is line seven
On bash 5.1.4 (Debian 11) and 5.1.6 (openBSD 7.1) I get
this is line one
this is line two
this is line three
Hello
Lines 4 to 7 are lost. How comes? Is this a problem of
bash at all?
Every insightful comment is highly appreciated
Harri
Re: 5.1 regression?
Harald Dunkel wrote on Mon, Aug 01, 2022 at 09:08:40AM +0200:
> a colleague pointed me to a changed behavior of bash 5.1.4 in Debian 11.
> Would you mind to take a look at this code?
>
>
> #! /bin/bash
> # set -x
>
> insert()
> {
>local data="$1"
>local lineNumber="$2"
>
>head -n "$lineNumber"
The problem is that stdin is eaten up by this head command.
What changed is that bash used to store stdin input in a temporary file,
so head could seek back to the actual position in the stream after the
first three lines but that no longer works:
1007746 read(0, "this is line one\nthis is line two\nthis is line three\nthis
is line four\nthis is line five\nthis is line six\nthis is line seven\n", 8192)
= 125
1007746 lseek(0, -72, SEEK_CUR) = -1 ESPIPE (Illegal seek)
In bash 5.0 0 here was a temporary file in /tmp, and that lseek worked,
but you cannot rely on that in general -- splitting stdin is difficult
business...
You'll get the same behaviour back if you go through a temporary file
again e.g. (don't actually do that!!)
echo "$text" > /tmp/temp
output="$( insert "Hello" 3 < /tmp/temp )"
rm /tmp/temp
Now as for how to actually do this, tough question. If your example is
representative of your real text you can just use awk or any other
single command:
insert()
{
local data="$1"
local lineNumber="$2"
awk -v lineno=$lineNumber -v data="$data" \
'{ print }
NR == lineno {
print "blabla", data
}'
}
--
Dominique
Re: 5.1 regression?
Date:Mon, 1 Aug 2022 09:08:40 +0200 From:Harald Dunkel Message-ID: <0b0342b5-be15-ebc4-989f-a0cc9ec6c...@aixigo.com> | a colleague pointed me to a changed behavior of bash 5.1.4 in Debian 11. The change is probably related to a change in the way bash implements here documents (temp files vs pipes), but | Would you mind to take a look at this code? It is nonsense. | Lines 4 to 7 are lost. How comes? Is this a problem of bash at all? No, of the function. The lines aren't "lost", they will have been read by head - head only outputs the first N lines of the file (or stdin in this case) to the output, but it will read in buffer sized chunks. When input is seekable, it might (but is not required to) seek back to the position in the input file where it stopped writing, but it cannot do that when reading from a pipe - which is always possible in this example, even if here docs weren't implemented as pipes, consider: cat file | insert data N That wouldn't have worked on any version I suspect (and bash has nothing to do with that at all). If you want to insert data into the middle of a file, use sed, that is what (well one of the things) it is designed to do. kre
Re: 5.1 regression?
On Mon, Aug 01, 2022 at 04:37:18PM +0900, Dominique Martinet wrote:
> Harald Dunkel wrote on Mon, Aug 01, 2022 at 09:08:40AM +0200:
> > a colleague pointed me to a changed behavior of bash 5.1.4 in Debian 11.
> >
> > #! /bin/bash
> > # set -x
> >
> > insert()
> > {
> >local data="$1"
> >local lineNumber="$2"
> >
> >head -n "$lineNumber"
>
> The problem is that stdin is eaten up by this head command.
>
> What changed is that bash used to store stdin input in a temporary file,
> so head could seek back to the actual position in the stream after the
> first three lines but that no longer works:
Specifically, this part of the bash 5.1 changelog:
c. Here documents and here strings now use pipes for the expanded document if
it's smaller than the pipe buffer size, reverting to temporary files if it's
larger.
Re: Missing word in rluser.texi
On 7/31/22 4:49 PM, pan...@gmail.com wrote: Description: There's a missing word in rluser.texi Repeat-By: N/A Fix: 800c800 < bindings in a format that can put directly into an initialization file. --- > bindings in a format that can be put directly into an initialization file. Thanks for the report. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: functions can fully unset local vars in other scopes
On 7/30/22 8:05 AM, Kerin Millar wrote: What consideration was given to creating “local --unset *VAR*” or “unset --local *VAR*” or some other equivalent that doesn't overload any existing command form? Obviously, only Chet can meaningfully respond to this. I mentioned the existence of the option because it appeared likely that Emanuele was unaware of it, not to attempt to obviate any of his suggestions, nor endorse the status quo. Sometimes it's useful to read the discussion that prompted the change. https://lists.gnu.org/archive/html/bug-bash/2018-02/msg00065.html In this case, the primary requirement was to enable scripts that wanted a wholesale behavior change, while not breaking backwards compatibility by making that option the default. Enabling these semantics on a per-variable basis wasn't part of the discussion (kre was, however, very vocal about making localvar_unset the default). -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: 5.1 regression?
On 8/1/22 4:17 AM, Robert Elz wrote: | a colleague pointed me to a changed behavior of bash 5.1.4 in Debian 11. The change is probably related to a change in the way bash implements here documents (temp files vs pipes), but Here's the first message in the thread that prompted that change: https://lists.gnu.org/archive/html/bug-bash/2019-03/msg00073.html It continued into the next month: https://lists.gnu.org/archive/html/bug-bash/2019-04/msg7.html Suffice it to say that there are other considerations: security, using here documents without any write-mounted file systems, whether or not using pipes for here-documents can lead to deadlock when using builtins, and in general whether or not you should write programs that depend on a particular implementation. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: functions can fully unset local vars in other scopes
I just finished reading that 2018 discussion and the current discussion. Its hard for me to understand why this is an issue. kre's use case was unset X y=$X He called it 'braindead' for it to do anything expect to assign NULL/empty to Y but I think that is far from true. If he's thinking of 'unset X' as assigning a NULL value to X (distinct from ""), 'local X' (with no initialize) seems to be a much clearer way to do that. What are people trying to accomplish by unsetting a variable in a function that did not first declare it? (besides upVar which specifically exploits the long standing behavior). --BobG On 8/1/22 09:58, Chet Ramey wrote: On 7/30/22 8:05 AM, Kerin Millar wrote: What consideration was given to creating “local --unset *VAR*” or “unset --local *VAR*” or some other equivalent that doesn't overload any existing command form? Obviously, only Chet can meaningfully respond to this. I mentioned the existence of the option because it appeared likely that Emanuele was unaware of it, not to attempt to obviate any of his suggestions, nor endorse the status quo. Sometimes it's useful to read the discussion that prompted the change. https://lists.gnu.org/archive/html/bug-bash/2018-02/msg00065.html In this case, the primary requirement was to enable scripts that wanted a wholesale behavior change, while not breaking backwards compatibility by making that option the default. Enabling these semantics on a per-variable basis wasn't part of the discussion (kre was, however, very vocal about making localvar_unset the default).
Re: functions can fully unset local vars in other scopes
On 8/1/22 3:38 PM, Robert E. Griffith wrote: I just finished reading that 2018 discussion and the current discussion. Its hard for me to understand why this is an issue. kre's use case was unset X y=$X He called it 'braindead' for it to do anything expect to assign NULL/empty to Y but I think that is far from true. Well, that's the crux of the discussion, isn't it? -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
