Re: "here document" causing failures. Implementation is inconsistent and not documented.

2023-10-11 Thread Chet Ramey
On 10/10/23 8:56 PM, Jim McD wrote: Bug: Trailing white space after the delimiting tag cause the here document to fail with an error like /./: line : warning: here-document at line delimited by end-of-file (wanted `msg_end')Trailing white space/ This is not a bug. Trailing white space afte

Re: "here document" causing failures. Implementation is inconsistent and not documented.

2023-10-10 Thread Andreas Kähäri
On Wed, Oct 11, 2023 at 11:56:42AM +1100, Jim McD wrote: > Bug: > Trailing white space after the delimiting tag cause the here document to > fail with an error like > /./: line : warning: here-document at line > delimited by end-of-file (wanted `msg_end')Trailing white > space/ > > Trailing white

Re: "here document" causing failures. Implementation is inconsistent and not documented.

2023-10-10 Thread Greg Wooledge
On Wed, Oct 11, 2023 at 11:56:42AM +1100, Jim McD wrote: > Bug: > Trailing white space after the delimiting tag cause the here document to > fail with an error like > /./: line : warning: here-document at line > delimited by end-of-file (wanted `msg_end')Trailing white > space/ > > Trailing white

Re: here strings fold newlines on MacOS

2021-01-31 Thread Rich Lafferty
Ah, yep — sorry for the false positive. I *have* 5.1.0 installed (and bashbug picked that up!) but the context in which I encountered the bug was indeed Apple’s ancient version. I could’ve sworn I tested it in both, but clearly messed that up - verified it’s fixed in 5.1.0.   -Rich On Jan 31,

Re: here strings fold newlines on MacOS

2021-01-31 Thread Chet Ramey
On 1/30/21 5:44 PM, Rich Lafferty wrote: Bash Version: 5.1 Patch Level: 0 Release Status: release Description: Here strings ('<<<') fold newlines into spaces on MacOS, but not on Linux, leading to incompatibilites in bash code expected to work the same on both platforms. This was fixed quite

Re: here strings fold newlines on MacOS

2021-01-30 Thread Lawrence Velázquez
> On Jan 30, 2021, at 9:28 PM, "" > wrote: > > Are you certain that you're not testing /bin/bash (version 3.2.57) in the > case of macOS? I ask because the bug you describe is said to have been > addressed by the release of 4.4-beta [1]. > > z. Bash no longer splits the expansion of here-st

Re: here strings fold newlines on MacOS

2021-01-30 Thread
On 30/01/2021 22:44, Rich Lafferty wrote: Configuration Information [Automatically generated, do not change]: Machine: x86_64 OS: darwin18.7.0 Compiler: clang Compilation CFLAGS: -DSSH_SOURCE_BASHRC uname output: Darwin flounder.home.mati.ca 18.7.0 Darwin Kernel Version 18.7.0: Tue Nov 10 00:07:

Re: "here strings" and tmpfiles

2019-04-12 Thread konsolebox
On Fri, Apr 12, 2019 at 10:32 PM Chet Ramey wrote: > > On 4/11/19 3:08 PM, konsolebox wrote: > > >>> It has slightly inconvenient semantics, in that you can't open it more > >>> than once, and if you can't do that, you can't convert it from read-write > >>> to readonly. > >> > >> > >> Perhaps it c

Re: "here strings" and tmpfiles

2019-04-12 Thread Chet Ramey
On 4/11/19 3:08 PM, konsolebox wrote: >>> It has slightly inconvenient semantics, in that you can't open it more >>> than once, and if you can't do that, you can't convert it from read-write >>> to readonly. >> >> >> Perhaps it can be reopened via /dev/fd. > > Also file sealing maybe. The way it

Re: "here strings" and tmpfiles

2019-04-11 Thread Chet Ramey
On 4/11/19 3:15 AM, Robert Elz wrote: > (Substitute cat if you're that kind of weirdo!). We're really going to throw down right here? -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@c

Re: "here strings" and tmpfiles

2019-04-11 Thread Chet Ramey
On 4/11/19 12:12 AM, Jason A. Donenfeld wrote: > I keep forgetting things. The other thing I wanted to bring up is that > I suspect bash's actual implementation of temporary files is > problematic and might have some of the classic /tmp and TOCTOU style > attacks. It's a peripheral issue, since t

Re: "here strings" and tmpfiles

2019-04-11 Thread Chet Ramey
On 4/11/19 12:02 AM, Jason A. Donenfeld wrote: > Hi Chet, Hi. > I hope that can shed light on the motivation a bit. Pass got hit by > this a bit ago: > https://git.zx2c4.com/password-store/commit/?id=367efa5846492e1b0898aad8a2c26ce94163ba24 I note that the pipe-for-small-enough-heredocs works fo

Re: "here strings" and tmpfiles

2019-04-11 Thread konsolebox
On Thu, Apr 11, 2019 at 5:31 AM konsolebox wrote: > > On Thu, Apr 11, 2019, 4:45 AM Chet Ramey wrote: >> >> On 4/10/19 4:33 PM, konsolebox wrote: >> > On Wed, Apr 10, 2019 at 11:15 PM Chet Ramey wrote: >> >> If we're going to go off into hypotheticals and speculation, it would be >> >> nice if m

Re: "here strings" and tmpfiles

2019-04-11 Thread Chet Ramey
On 4/10/19 7:18 PM, Daniel Kahn Gillmor wrote: > On Wed 2019-04-10 16:16:44 -0400, Chet Ramey wrote: >> Is it just that people have not realized all along that most shells, >> certainly all historical shells, that implement here documents use temp >> files to do it? It's really only the ash-based s

Re: "here strings" and tmpfiles

2019-04-11 Thread Daniel Kahn Gillmor
On Thu 2019-04-11 10:04:02 +0200, Andreas Schwab wrote: > On Apr 10 2019, Daniel Kahn Gillmor wrote: > >> data written to the local filesystem can be discovered by someone >> analyzing the disk controller data path, or by someone with access to >> the underlying storage medium. > > Do you have swa

Re: "here strings" and tmpfiles

2019-04-11 Thread konsolebox
On Thu, Apr 11, 2019, 10:42 PM Andreas Kusalananda Kähäri < andreas.kah...@abc.se> wrote: > On Thu, Apr 11, 2019 at 09:01:50PM +0800, konsolebox wrote: > > On Thu, Apr 11, 2019, 4:04 PM Andreas Schwab wrote: > > > > > On Apr 10 2019, Daniel Kahn Gillmor wrote: > > > > > > > data written to the l

Re: "here strings" and tmpfiles

2019-04-11 Thread Andreas Kusalananda Kähäri
On Thu, Apr 11, 2019 at 09:01:50PM +0800, konsolebox wrote: > On Thu, Apr 11, 2019, 4:04 PM Andreas Schwab wrote: > > > On Apr 10 2019, Daniel Kahn Gillmor wrote: > > > > > data written to the local filesystem can be discovered by someone > > > analyzing the disk controller data path, or by some

Re: "here strings" and tmpfiles

2019-04-11 Thread konsolebox
On Thu, Apr 11, 2019 at 9:06 PM Greg Wooledge wrote: > So... yes. Because everyone in 2019 has a laptop and therefore has swap > enabled because it's used for hibernation. Sure captain. It was a joke. -- konsolebox

Re: "here strings" and tmpfiles

2019-04-11 Thread Greg Wooledge
On Thu, Apr 11, 2019 at 09:01:50PM +0800, konsolebox wrote: > On Thu, Apr 11, 2019, 4:04 PM Andreas Schwab wrote: > > Do you have swap enabled? > > It's 2019. So... yes. Because everyone in 2019 has a laptop and therefore has swap enabled because it's used for hibernation. That was what you me

Re: "here strings" and tmpfiles

2019-04-11 Thread konsolebox
On Thu, Apr 11, 2019, 4:04 PM Andreas Schwab wrote: > On Apr 10 2019, Daniel Kahn Gillmor wrote: > > > data written to the local filesystem can be discovered by someone > > analyzing the disk controller data path, or by someone with access to > > the underlying storage medium. > > Do you have sw

Re: "here strings" and tmpfiles

2019-04-11 Thread Greg Wooledge
On Thu, Apr 11, 2019 at 06:02:41AM +0200, Jason A. Donenfeld wrote: > what about internally treating "x < x"? Are these somehow not quite equivalent because x is in a subshell > in one but not the other, or something like that? cmd <<< string opens a temporary file for writing, dumps the string

Re: "here strings" and tmpfiles

2019-04-11 Thread Andreas Schwab
On Apr 10 2019, Daniel Kahn Gillmor wrote: > data written to the local filesystem can be discovered by someone > analyzing the disk controller data path, or by someone with access to > the underlying storage medium. Do you have swap enabled? Andreas. -- Andreas Schwab, SUSE Labs, sch...@suse.

Re: "here strings" and tmpfiles

2019-04-11 Thread Robert Elz
Date:Thu, 11 Apr 2019 06:02:41 +0200 From:"Jason A. Donenfeld" Message-ID: | Now, it might be the case that bash really isn't the | right tool for that kind of thing, and I shouldn't use bash for tasks | with security requirements as such. But I sort of love

Re: "here strings" and tmpfiles

2019-04-10 Thread Jason A. Donenfeld
I keep forgetting things. The other thing I wanted to bring up is that I suspect bash's actual implementation of temporary files is problematic and might have some of the classic /tmp and TOCTOU style attacks. The current implementation is three-fold via ifdefs: char * sh_mktmpname (nameroot, flag

Re: "here strings" and tmpfiles

2019-04-10 Thread Jason A. Donenfeld
On Thu, Apr 11, 2019 at 6:02 AM Jason A. Donenfeld wrote: > curious about is - what about internally treating "x <

Re: "here strings" and tmpfiles

2019-04-10 Thread Jason A. Donenfeld
Hi Chet, On Wed, Apr 10, 2019 at 3:07 PM Chet Ramey wrote: > This is unnecessary hyperbole. The existing file-based mechanism works > just fine. We're talking about what's essentially an optimization. > [...] > This doesn't make any sense. > [...] > There isn't an "insecure path." I'm a bit late

Re: "here strings" and tmpfiles

2019-04-10 Thread Daniel Kahn Gillmor
On Wed 2019-04-10 16:16:44 -0400, Chet Ramey wrote: > Is it just that people have not realized all along that most shells, > certainly all historical shells, that implement here documents use temp > files to do it? It's really only the ash-based shells (not an insignificant > portion of the shells

Re: "here strings" and tmpfiles

2019-04-10 Thread konsolebox
On Thu, Apr 11, 2019, 4:45 AM Chet Ramey wrote: > On 4/10/19 4:33 PM, konsolebox wrote: > > On Wed, Apr 10, 2019 at 11:15 PM Chet Ramey wrote: > >> If we're going to go off into hypotheticals and speculation, it would be > >> nice if memfd_create were available universally. > > > > I see many pa

Re: "here strings" and tmpfiles

2019-04-10 Thread Chet Ramey
On 4/10/19 4:33 PM, konsolebox wrote: > On Wed, Apr 10, 2019 at 11:15 PM Chet Ramey wrote: >> If we're going to go off into hypotheticals and speculation, it would be >> nice if memfd_create were available universally. > > I see many parts in lib/* that adapts to available system features > like

Re: "here strings" and tmpfiles

2019-04-10 Thread Peter & Kelly Passchier
On 10/4/2019 09:04, Greg Wooledge wrote: > On Wed, Apr 10, 2019 at 11:59:19AM -0400, Daniel Kahn Gillmor wrote: >> If we look at the problem from the perspective of the risk of >> herestring/heredoc content leaking to non-ephemeral storage, > > The content is already in the damned SHELL SCRIPT. >

Re: "here strings" and tmpfiles

2019-04-10 Thread konsolebox
On Wed, Apr 10, 2019 at 11:15 PM Chet Ramey wrote: > If we're going to go off into hypotheticals and speculation, it would be > nice if memfd_create were available universally. I see many parts in lib/* that adapts to available system features like mmap and MAP_ANONYMOUS. I don't see why memfd_c

Re: "here strings" and tmpfiles

2019-04-10 Thread konsolebox
On Tue, Apr 9, 2019 at 10:01 PM Jason A. Donenfeld wrote: > A real solution for this issue involves getting rid of the temporary file > all together. Since we're talking about a bash string, it's already in > memory. Why not just fork() if the write() will block? A simple way would be > to always

Re: "here strings" and tmpfiles

2019-04-10 Thread Chet Ramey
On 4/10/19 11:59 AM, Daniel Kahn Gillmor wrote: > On Wed 2019-04-10 09:07:27 -0400, Chet Ramey wrote: > I think we all agree that avoiding the filesystem where possible is > likely to be an optimization and improvement (it means the heredoc will > work in some circumstances where it didn't work be

Re: "here strings" and tmpfiles

2019-04-10 Thread Greg Wooledge
On Wed, Apr 10, 2019 at 11:59:19AM -0400, Daniel Kahn Gillmor wrote: > If we look at the problem from the perspective of the risk of > herestring/heredoc content leaking to non-ephemeral storage, The content is already in the damned SHELL SCRIPT. How much more "non-ephemeral" can it get?

Re: "here strings" and tmpfiles

2019-04-10 Thread Daniel Kahn Gillmor
On Wed 2019-04-10 09:07:27 -0400, Chet Ramey wrote: > On 4/9/19 2:56 AM, Jason A. Donenfeld wrote: >> Since originally raising this issue with dkg (leading to this email >> thread), I've only followed along from a bit of a distance. But it does >> look like there's been some good progress: there's

Re: "here strings" and tmpfiles

2019-04-10 Thread Chet Ramey
On 4/9/19 12:19 AM, Robert Elz wrote: > Date:Mon, 08 Apr 2019 17:04:41 -0700 > From:L A Walsh > Message-ID: <5cabe199.9030...@tlinx.org> > > | On 4/8/2019 7:10 AM, Chet Ramey wrote: > > | > Pipes are objectively not the same as files. They > | > > | > 1. Do n

Re: "here strings" and tmpfiles

2019-04-10 Thread Chet Ramey
On 4/9/19 7:48 PM, L A Walsh wrote: > > I am aware of that, however, if a pipe implementation > *stops* on reaching a full condition from some 'tmp-storage-space' > and awaits for space to become available, a similar dynamic would > apply. That's all. Is there a pipe implementation ou

Re: "here strings" and tmpfiles

2019-04-10 Thread konsolebox
On Wed, Apr 10, 2019, 1:09 AM Eli Schwartz wrote: > That being said, it seems like a rather odd place to configure and use a > heavyweight shell merely to allow third parties to include > downstream-specific bashisms. I think there is a great deal of wisdom in > the fact that the referenced issue

Re: "here strings" and tmpfiles

2019-04-10 Thread Chet Ramey
On 4/9/19 10:07 AM, konsolebox wrote: > Perhaps bash can also look at /dev/shm. It's a common tmpfs, but I > haven't checked if it's standard and what utility mounts it. I don't > really use it. Another non-portable feature. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer

Re: "here strings" and tmpfiles

2019-04-10 Thread Chet Ramey
On 4/9/19 2:56 AM, Jason A. Donenfeld wrote: > Since originally raising this issue with dkg (leading to this email > thread), I've only followed along from a bit of a distance. But it does > look like there's been some good progress: there's now a commit that > fills the pipe up to the OS's maximum

Re: "here strings" and tmpfiles

2019-04-10 Thread Chet Ramey
On 4/9/19 1:03 AM, pepa65 wrote: > I think Linda's main drive is to seek improvement in how bash works. Now > that lack of memory is in no way a constraint for the vast majority of > situations where bash is commonly used, it would be great if that memory > could be used instead of writing to a fi

Re: "here strings" and tmpfiles

2019-04-10 Thread Greg Wooledge
On Tue, Apr 09, 2019 at 04:48:30PM -0700, L A Walsh wrote: > But the implementation of process substitution in bash > isn't implemented that way in the currently released version. It > uses a tmp file on a disk of fixed size to store *all* of the output > of the 'writer' before the reader is c

Re: "here strings" and tmpfiles

2019-04-10 Thread Greg Wooledge
On Tue, Apr 09, 2019 at 04:48:30PM -0700, L A Walsh wrote: > > cp <(process) /tmp/foo > --- > *red face* I'd never tried to copy something that > looked like input redirection. My apologies on my misconception. One of the archetypal examples that we give when explaining process substitut

Re: "here strings" and tmpfiles

2019-04-09 Thread L A Walsh
On 4/8/2019 9:19 PM, Robert Elz wrote: > > | Optionally, I would accept that > | an implementation would support forward seeking as some equivalent > | to having read the bytes. > > I suppose one could make pipes do that, but no implementation I have > ever seen does, so I don't think yo

Re: "here strings" and tmpfiles

2019-04-09 Thread Eli Schwartz
On 4/9/19 10:25 AM, konsolebox wrote: > On Mon, Apr 8, 2019 at 10:39 PM Greg Wooledge wrote: >> That's incorrect in this context. We're talking about boot scripts here, >> not interactive user shells. In boot scripts, on every operating system >> I've ever used, the shell being used is either PO

Re: "here strings" and tmpfiles

2019-04-09 Thread konsolebox
On Tue, Apr 9, 2019 at 11:28 PM Chet Ramey wrote: > > On 4/9/19 11:25 AM, konsolebox wrote: > > On Tue, Apr 9, 2019 at 10:28 PM Chet Ramey wrote: > >> > >> On 4/9/19 10:10 AM, konsolebox wrote: > >>> On Wed, Mar 20, 2019 at 8:19 PM Greg Wooledge wrote: > > Just like that one time L. Wa

Re: "here strings" and tmpfiles

2019-04-09 Thread Chet Ramey
On 4/9/19 11:25 AM, konsolebox wrote: > On Tue, Apr 9, 2019 at 10:28 PM Chet Ramey wrote: >> >> On 4/9/19 10:10 AM, konsolebox wrote: >>> On Wed, Mar 20, 2019 at 8:19 PM Greg Wooledge wrote: Just like that one time L. Walsh tried to write a bash boot script that used <() to populat

Re: "here strings" and tmpfiles

2019-04-09 Thread konsolebox
On Tue, Apr 9, 2019 at 10:28 PM Chet Ramey wrote: > > On 4/9/19 10:10 AM, konsolebox wrote: > > On Wed, Mar 20, 2019 at 8:19 PM Greg Wooledge wrote: > >> > >> Just like that one time L. Walsh tried to write a bash boot script that > >> used <() to populate an array, and it failed because she was

Re: "here strings" and tmpfiles

2019-04-09 Thread Chet Ramey
On 4/9/19 10:10 AM, konsolebox wrote: > On Wed, Mar 20, 2019 at 8:19 PM Greg Wooledge wrote: >> >> Just like that one time L. Walsh tried to write a bash boot script that >> used <() to populate an array, and it failed because she was running >> it too early in the boot sequence, and /dev/fd/ wasn

Re: "here strings" and tmpfiles

2019-04-09 Thread konsolebox
On Mon, Apr 8, 2019 at 10:39 PM Greg Wooledge wrote: > That's incorrect in this context. We're talking about boot scripts here, > not interactive user shells. In boot scripts, on every operating system > I've ever used, the shell being used is either POSIX sh or Bourne sh. > > Everyone who write

Re: "here strings" and tmpfiles

2019-04-09 Thread Greg Wooledge
On Tue, Apr 09, 2019 at 10:10:44PM +0800, konsolebox wrote: > @Chet, Isn't bash supposed to use named pipes alternatively, and > dynamically? Or does it just decide what to use based on the current > system? The second thing. On platform X, bash uses named pipes. On platform Y, bash uses /dev/f

Re: "here strings" and tmpfiles

2019-04-09 Thread konsolebox
On Wed, Mar 20, 2019 at 8:19 PM Greg Wooledge wrote: > > Just like that one time L. Walsh tried to write a bash boot script that > used <() to populate an array, and it failed because she was running > it too early in the boot sequence, and /dev/fd/ wasn't available yet. @Chet, Isn't bash suppose

Re: "here strings" and tmpfiles

2019-04-09 Thread konsolebox
On Wed, Mar 20, 2019 at 9:05 AM Robert Elz wrote: > Note: I am not suggesting bash should change - using files for here docs > is the way they were originally implemented (in the Bourne sh) (though it > had bugs, which could leave the files lying around in some cases). > > However, using files for

Re: "here strings" and tmpfiles

2019-04-09 Thread Jason A. Donenfeld
Since originally raising this issue with dkg (leading to this email thread), I've only followed along from a bit of a distance. But it does look like there's been some good progress: there's now a commit that fills the pipe up to the OS's maximum pipe size, and then falls back to the old (buggy, vu

Re: "here strings" and tmpfiles

2019-04-09 Thread Chet Ramey
On 4/9/19 8:36 AM, Greg Wooledge wrote: > Bash always forks for $() as far as I'm aware, which is why bash 3.1 > introduced printf -v var. It's not, but it was a nice side effect. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hipp

Re: "here strings" and tmpfiles

2019-04-09 Thread Greg Wooledge
On Tue, Apr 09, 2019 at 02:32:38PM +0700, Robert Elz wrote: > The idea is basically just to do > > var=$( cmd ) > > right? But without a fork. That's something that can be done today, > no new syntax needed (bash might even do it sometimes, I don't know, the > FreeBSD shell does.) wool

Re: "here strings" and tmpfiles

2019-04-09 Thread Robert Elz
Date:Mon, 8 Apr 2019 23:36:39 -0700 From:pepa65 Message-ID: | When in the past I proposed this syntax: | cmd >>>var | the idea was to commit the output of a command into memory (in the form | of a variable), without requiring a pipe or file. In general

Re: "here strings" and tmpfiles

2019-04-08 Thread pepa65
On 8/4/2019 22:53, Robert Elz wrote: > [Aside: when the destination is a builtin, another strategy is to > simply write the here doc into mem, and have the builtin read directly > from the mem buffer - no actual I/O of any kind involved at all.] When in the past I proposed this syntax: cmd >>>va

Re: "here strings" and tmpfiles

2019-04-08 Thread Robert Elz
Date:Mon, 8 Apr 2019 22:03:25 -0700 From:pepa65 Message-ID: | What is nice about here docs/strings is that there are no subshells | involved. When they use files, that's correct, but when a pipe is used, unless the data size is both known in advance, and limited

Re: "here strings" and tmpfiles

2019-04-08 Thread pepa65
On 8/4/2019 21:19, Robert Elz wrote: > In general here docs (and here strings) are overused - it is always > possible to simply write a pipe instead What is nice about here docs/strings is that there are no subshells involved. I think Linda's main drive is to seek improvement in how bash works. N

Re: "here strings" and tmpfiles

2019-04-08 Thread Robert Elz
Date:Mon, 08 Apr 2019 17:04:41 -0700 From:L A Walsh Message-ID: <5cabe199.9030...@tlinx.org> | On 4/8/2019 7:10 AM, Chet Ramey wrote: | > Pipes are objectively not the same as files. They | > | > 1. Do not have file semantics. For instance, they are not seeka

Re: "here strings" and tmpfiles

2019-04-08 Thread L A Walsh
On 4/8/2019 7:10 AM, Chet Ramey wrote: > On 4/7/19 4:21 PM, L A Walsh wrote: > >> On 3/22/2019 6:49 AM, Chet Ramey wrote: >> >>> Yes, that's how bash chooses to implement it. There are a few portable >>> ways >>> to turn a string into a file descriptor, and a temp file is one of them (a

Re: "here strings" and tmpfiles

2019-04-08 Thread Greg Wooledge
On Mon, Apr 08, 2019 at 10:53:46AM -0400, Chet Ramey wrote: > On 4/8/19 10:36 AM, Greg Wooledge wrote: > > > That's incorrect in this context. We're talking about boot scripts here, > > not interactive user shells. In boot scripts, on every operating system > > I've ever used, the shell being us

Re: "here strings" and tmpfiles

2019-04-08 Thread Chet Ramey
On 4/8/19 10:36 AM, Greg Wooledge wrote: > That's incorrect in this context. We're talking about boot scripts here, > not interactive user shells. In boot scripts, on every operating system > I've ever used, the shell being used is either POSIX sh or Bourne sh. This is clearly wrong in general,

Re: "here strings" and tmpfiles

2019-04-08 Thread Greg Wooledge
On Sun, Apr 07, 2019 at 01:06:21PM -0700, L A Walsh wrote: > On 3/20/2019 5:19 AM, Greg Wooledge wrote: > > Just like that one time L. Walsh tried to write a bash boot script that > > used <() to populate an array, and it failed because she was running > > it too early in the boot sequence, and /de

Re: "here strings" and tmpfiles

2019-04-08 Thread Chet Ramey
On 4/7/19 4:21 PM, L A Walsh wrote: > On 3/22/2019 6:49 AM, Chet Ramey wrote: >> Yes, that's how bash chooses to implement it. There are a few portable >> ways >> to turn a string into a file descriptor, and a temp file is one of them (a >> child process using a pipe is another, but pipes have othe

Re: "here strings" and tmpfiles

2019-04-08 Thread Chet Ramey
On 4/7/19 4:06 PM, L A Walsh wrote: >> Just like that one time L. Walsh tried to write a bash boot script that >> used <() to populate an array, and it failed because she was running >> it too early in the boot sequence, and /dev/fd/ wasn't available yet >> > > --- > /dev/fd was available,

Re: "here strings" and tmpfiles

2019-04-07 Thread L A Walsh
On 3/22/2019 6:49 AM, Chet Ramey wrote: > Yes, that's how bash chooses to implement it. There are a few portable > ways > to turn a string into a file descriptor, and a temp file is one of them (a > child process using a pipe is another, but pipes have other issues). > Such as? That are more co

Re: "here strings" and tmpfiles

2019-04-07 Thread L A Walsh
On 3/20/2019 5:19 AM, Greg Wooledge wrote: > On Wed, Mar 20, 2019 at 07:49:34AM +0700, Robert Elz wrote: > >> However, using files for here docs makes here docs unusable in a shell >> running in single user mode with no writable filesystems (whatever is >> mounted is read only, until after file

Re: "here strings" and tmpfiles

2019-03-24 Thread konsolebox
On Tue, Mar 19, 2019, 9:36 PM Greg Wooledge wrote: > On Tue, Mar 19, 2019 at 09:20:33AM -0400, Daniel Kahn Gillmor wrote: > > On Tue 2019-03-19 08:25:50 -0400, Greg Wooledge wrote: > > > On Mon, Mar 18, 2019 at 05:18:10PM -0400, Daniel Kahn Gillmor wrote: > > >> strace -o tmp/bash.herestring.

Re: "here strings" and tmpfiles

2019-03-22 Thread Greg Wooledge
On Fri, Mar 22, 2019 at 10:28:52AM -0400, Chet Ramey wrote: > On 3/20/19 8:19 AM, Greg Wooledge wrote: > > > Even if Chet changed how here docs work in bash 5.1, nobody would > > be safe to use those features in their "I'm feeding a password with > > a here string" scripts for at least 20 years, b

Re: "here strings" and tmpfiles

2019-03-22 Thread Chet Ramey
On 3/20/19 8:19 AM, Greg Wooledge wrote: > Even if Chet changed how here docs work in bash 5.1, nobody would > be safe to use those features in their "I'm feeding a password with > a here string" scripts for at least 20 years, because there will > still be people running older versions of bash for

Re: "here strings" and tmpfiles

2019-03-22 Thread Chet Ramey
On 3/19/19 9:07 AM, Daniel Kahn Gillmor wrote: > Thanks for the feedback, Eduardo-- > > On Mon 2019-03-18 17:40:17 -0700, Eduardo A. Bustamante López wrote: >> I don't think the implementation details of herestrings are documented >> anywhere, >> and I'm not too sure if they should (i.e. IMO if y

Re: "here strings" and tmpfiles

2019-03-22 Thread Chet Ramey
On 3/20/19 7:36 AM, Daniel Kahn Gillmor wrote: > On Tue 2019-03-19 09:31:55 -0400, Greg Wooledge wrote: >> There are scripts that *rely* on the seekability of the temporary files >> created by here-documents and here-strings. "Improving" the "situation" >> would break backward compatibility. > >

Re: "here strings" and tmpfiles

2019-03-22 Thread Chet Ramey
On 3/19/19 8:49 PM, Robert Elz wrote: > Date:Tue, 19 Mar 2019 08:25:50 -0400 > From:Greg Wooledge > Message-ID: <20190319122550.khv5jp66iobjo...@eeg.ccf.org> > > | Yes, just like here documents do. And have always done, in all shells. > > That's not correct. The

Re: "here strings" and tmpfiles

2019-03-22 Thread Chet Ramey
On 3/18/19 8:40 PM, Eduardo A. Bustamante López wrote: > Having said that, have you tried process substitution as an option? > > You should be able to do something like: > > > mycommand < <(printf %s 'super secret') > > > That will: > > - not write the 'super secret' string to the file-sys

Re: "here strings" and tmpfiles

2019-03-22 Thread Chet Ramey
On 3/19/19 8:25 AM, Greg Wooledge wrote: > On Mon, Mar 18, 2019 at 05:18:10PM -0400, Daniel Kahn Gillmor wrote: >> strace -o tmp/bash.herestring.strace -f bash -c 'cat <<<"hello there"' >> It turns out that this creates a temporary file, actually touching the >> underlying filesystem: > > Yes,

Re: "here strings" and tmpfiles

2019-03-22 Thread Chet Ramey
On 3/18/19 5:18 PM, Daniel Kahn Gillmor wrote: > hi bash developers-- > > I ran the following command to get a sense of how bash deals with here > strings under the hood: > > strace -o tmp/bash.herestring.strace -f bash -c 'cat <<<"hello there"' > > (i'm testing with bash 5.0-2 on debian tes

Re: "here strings" and tmpfiles

2019-03-20 Thread Greg Wooledge
On Wed, Mar 20, 2019 at 07:49:34AM +0700, Robert Elz wrote: > However, using files for here docs makes here docs unusable in a shell > running in single user mode with no writable filesystems (whatever is > mounted is read only, until after file system checks are finished). Meanwhile, proposals ba

Re: "here strings" and tmpfiles

2019-03-20 Thread Greg Wooledge
On Wed, Mar 20, 2019 at 07:36:41AM -0400, Daniel Kahn Gillmor wrote: > How do you replicate 3<<<"$secret" with a pipeline? Not strictly a pipeline, but: 3< <(printf %s "$secret") This is actually preferred in many cases, because it doesn't add a newline. <<< always adds a newline to the result,

Re: "here strings" and tmpfiles

2019-03-20 Thread Daniel Kahn Gillmor
On Tue 2019-03-19 09:31:55 -0400, Greg Wooledge wrote: > There are scripts that *rely* on the seekability of the temporary files > created by here-documents and here-strings. "Improving" the "situation" > would break backward compatibility. i hope you noticed that of my suggested improvements, on

Re: "here strings" and tmpfiles

2019-03-19 Thread Robert Elz
Date:Tue, 19 Mar 2019 08:25:50 -0400 From:Greg Wooledge Message-ID: <20190319122550.khv5jp66iobjo...@eeg.ccf.org> | Yes, just like here documents do. And have always done, in all shells. That's not correct. There are shells that don't use files for here docs.

Re: "here strings" and tmpfiles

2019-03-19 Thread Daniel Kahn Gillmor
On Tue 2019-03-19 08:25:50 -0400, Greg Wooledge wrote: > On Mon, Mar 18, 2019 at 05:18:10PM -0400, Daniel Kahn Gillmor wrote: >> strace -o tmp/bash.herestring.strace -f bash -c 'cat <<<"hello there"' >> It turns out that this creates a temporary file, actually touching the >> underlying filesys

Re: "here strings" and tmpfiles

2019-03-19 Thread Greg Wooledge
On Tue, Mar 19, 2019 at 09:20:33AM -0400, Daniel Kahn Gillmor wrote: > On Tue 2019-03-19 08:25:50 -0400, Greg Wooledge wrote: > > On Mon, Mar 18, 2019 at 05:18:10PM -0400, Daniel Kahn Gillmor wrote: > >> strace -o tmp/bash.herestring.strace -f bash -c 'cat <<<"hello there"' > >> It turns out th

Re: "here strings" and tmpfiles

2019-03-19 Thread Daniel Kahn Gillmor
Thanks for the feedback, Eduardo-- On Mon 2019-03-18 17:40:17 -0700, Eduardo A. Bustamante López wrote: > I don't think the implementation details of herestrings are documented > anywhere, > and I'm not too sure if they should (i.e. IMO if you need that degree of > control > over the implementat

Re: "here strings" and tmpfiles

2019-03-19 Thread Daniel Kahn Gillmor
On Mon 2019-03-18 17:18:10 -0400, Daniel Kahn Gillmor wrote: > A few possible options for trying to improve the situation: > > a) use socketpair(2) or pipe(2) instead of making a tmpfile. this has > the potential downside that the semantics of access to the remaining > file descriptor wou

Re: "here strings" and tmpfiles

2019-03-19 Thread Greg Wooledge
On Mon, Mar 18, 2019 at 05:18:10PM -0400, Daniel Kahn Gillmor wrote: > strace -o tmp/bash.herestring.strace -f bash -c 'cat <<<"hello there"' > It turns out that this creates a temporary file, actually touching the > underlying filesystem: Yes, just like here documents do. And have always don

Re: "here strings" and tmpfiles

2019-03-18 Thread Eduardo A . Bustamante López
On Mon, Mar 18, 2019 at 05:18:10PM -0400, Daniel Kahn Gillmor wrote: > hi bash developers-- (...) > a) use socketpair(2) or pipe(2) instead of making a tmpfile. this has > the potential downside that the semantics of access to the remaining > file descriptor would be subtly different from

Re: [here string] uncompatible change on here string function

2017-11-27 Thread Vladimir Marek
> > It's not the point. the problem is bash-4.4 auto add quote around $var or > > $(cmd) after '<<<' > > > > On bash-4.2 and before following command always works fine > > read ignore fstype <<<$(df --output=fstype $mountpoint) > > > > But after Update bash-4.4, our customer's script get fa

Re: Re: [here string] uncompatible change on here string function

2017-11-27 Thread Greg Wooledge
On Fri, Nov 24, 2017 at 11:22:56AM +0800, 尹剑虹 wrote: > It's not the point. the problem is bash-4.4 auto add quote around $var or > $(cmd) after '<<<' > > On bash-4.2 and before following command always works fine > read ignore fstype <<<$(df --output=fstype $mountpoint) > > But after Update

Re: [here string] uncompatible change on here string function

2017-11-22 Thread Chet Ramey
On 11/22/17 8:17 PM, PePa wrote: > On 11/23/2017 02:23 AM, Chet Ramey wrote: >> The bash-4.2 man page defers the description of here strings to the >> description of here documents. The lines in a here document do not undergo >> word splitting.  It was a bug in bash-4.2 that the WORD in a here stri

Re: [here string] uncompatible change on here string function

2017-11-22 Thread PePa
On 11/23/2017 02:23 AM, Chet Ramey wrote: The bash-4.2 man page defers the description of here strings to the description of here documents. The lines in a here document do not undergo word splitting. It was a bug in bash-4.2 that the WORD in a here string was split. This finally got fixed in b

Re: [here string] uncompatible change on here string function

2017-11-22 Thread Chet Ramey
On 11/22/17 9:19 AM, DJ Mills wrote: > On Wed, Nov 22, 2017 at 9:00 AM, Peter & Kelly Passchier < > peterke...@passchier.net> wrote: > >> On 11/22/2017 08:04 PM, Greg Wooledge wrote: >> >>> wooledg:~$ bash-4.2 >>> wooledg:~$ var=$'foo\t\tbar' >>> wooledg:~$ cat <<< "$var" >>> foo bar >

Re: [here string] uncompatible change on here string function

2017-11-22 Thread DJ Mills
On Wed, Nov 22, 2017 at 9:00 AM, Peter & Kelly Passchier < peterke...@passchier.net> wrote: > On 11/22/2017 08:04 PM, Greg Wooledge wrote: > >> wooledg:~$ bash-4.2 >> wooledg:~$ var=$'foo\t\tbar' >> wooledg:~$ cat <<< "$var" >> foo bar >> > > I think you are missing the point. He is cl

Re: [here string] uncompatible change on here string function

2017-11-22 Thread Greg Wooledge
On Wed, Nov 22, 2017 at 09:00:09PM +0700, Peter & Kelly Passchier wrote: > I think you are missing the point. He is claiming/reporting: Why write code that relies on the idiosyncratic behavior of a shell when it is given incorrect code? Whether bash handles incorrect code in one way or another se

Re: [here string] uncompatible change on here string function

2017-11-22 Thread Peter & Kelly Passchier
On 11/22/2017 08:04 PM, Greg Wooledge wrote: wooledg:~$ bash-4.2 wooledg:~$ var=$'foo\t\tbar' wooledg:~$ cat <<< "$var" foo bar I think you are missing the point. He is claiming/reporting: $ bash-4.4-12 $ var=$'foo\t\tbar' $ cat <<<$var foo bar # Without the quotes, the

Re: [here string] uncompatible change on here string function

2017-11-22 Thread Greg Wooledge
On Wed, Nov 22, 2017 at 11:22:47AM +0800, 尹剑虹 wrote: > The command line to the right of "<<<" is no longer cut by IFS > > [yjh@bash4.4 ~]$ var=$(echo -e "foo\t\tbar") > [yjh@bash4.4 ~]$ read a <<<$var; echo "$a" > foo bar > > [yjh@bash4.2 ~]$ var=$(echo

Re: Here strings and brace expansion not working as per man page

2017-02-15 Thread Conor McCarthy
On 9 February 2017 at 20:55, Chet Ramey wrote: > On 2/8/17 3:04 PM, Conor McCarthy wrote: > >> Bash Version: 4.4 >> Patch Level: 12 >> Release Status: release >> >> Description: >> Here strings are documented as: >> >>[n]<<> >> The word undergoes brace expansion, tild

Re: Here strings and brace expansion not working as per man page

2017-02-09 Thread Chet Ramey
On 2/8/17 3:04 PM, Conor McCarthy wrote: > Bash Version: 4.4 > Patch Level: 12 > Release Status: release > > Description: > Here strings are documented as: > >[n]<< > The word undergoes brace expansion, tilde expansion, parameter > and > variable expan

Re: here-documents in $(command) substitution

2016-11-28 Thread Chet Ramey
On 11/27/16 9:51 AM, Eduardo Bustamante wrote: > Delimiter (in your case the three character string "EOF"), has to be > on its own line, with no leading or trailing blanks (or any other > characters). If bash 3.x used to behave different, it's because it was > buggy. Not exactly. Bash has always

  1   2   >