Re: Directing into a variable doesn't work
Op 24-06-18 om 05:08 schreef Peter Passchier: With memory being abundant and filesystem access expensive, I want to put stdout and stderr of a command into variables (without needing to write to a file): output=$($command 2>>>errors) This would not work even if the feature is implemented. The $(command substitution) forks a subshell process to execute the command in, because the main shell process needs a process it can set up a pipe with. So the 'errors' variable would only exist in that subshell process and would be lost as soon as the command substitution completes. Or: $command >>>output 2>>>errors This form seems conceivable to me. However, note that here-documents and here-strings internally use temporary files, so they do involve file system access. I'm not Chet, but I don't think that would be different for your proposed feature. So while this might be some nice syntactic sugar, I'm afraid you would be disappointed about the performance. I still kind of like the idea, though. As far as I know, there's currently no way to capture more than one output stream into separate variables without involving rather laborious handling of temporary files. Your proposal would probably still involve that, but the shell would do the hard work for you which seems like an improvement to me. BTW, 'reverse here-document' doesn't sound quite right. You're not specifying any document or string containing input, you're specifying a variable in which to store output. So, here-variable? - M.
Re: Directing into a variable doesn't work
Thank you for the feedback, very insightful. Yes, scratch that first 'example'. Yay for the here-variable redirection! I am surprised by the general internal usage of temporary files for here-documents & here-strings, because (generally speaking) memory is quite abundant, and here-strings and even here-documents are usually not huge. I can see for memory-depleted systems this might be an issue, and there are no definite guarantees about the eventual size that is required, but (again, generally) this could all be done in memory. (And how about storage-depleted systems??) Peter On 06/24/2018 05:38 PM, Martijn Dekker wrote: > Op 24-06-18 om 05:08 schreef Peter Passchier: >> With memory being abundant and filesystem access expensive, I want to >> put stdout and stderr of a command into variables (without needing to >> write to a file): >> >> output=$($command 2>>>errors) > > This would not work even if the feature is implemented. The $(command > substitution) forks a subshell process to execute the command in, > because the main shell process needs a process it can set up a pipe > with. So the 'errors' variable would only exist in that subshell process > and would be lost as soon as the command substitution completes. > >> Or: >> >> $command >>>output 2>>>errors > > This form seems conceivable to me. > > However, note that here-documents and here-strings internally use > temporary files, so they do involve file system access. I'm not Chet, > but I don't think that would be different for your proposed feature. So > while this might be some nice syntactic sugar, I'm afraid you would be > disappointed about the performance. > > I still kind of like the idea, though. As far as I know, there's > currently no way to capture more than one output stream into separate > variables without involving rather laborious handling of temporary > files. Your proposal would probably still involve that, but the shell > would do the hard work for you which seems like an improvement to me. > > BTW, 'reverse here-document' doesn't sound quite right. You're not > specifying any document or string containing input, you're specifying a > variable in which to store output. So, here-variable? > > - M. >
Re: Directing into a variable doesn't work
Date:Sun, 24 Jun 2018 22:26:52 +0700 From:Peter Passchier Message-ID: | I am surprised by the general internal usage of temporary files for | here-documents & here-strings, because (generally speaking) memory is | quite abundant, That's not the real issue - rather it is that a here doc is presented to the command beng run as a file descrptior - that command will want to do a read() to obtain the data. Simply sitting in memory doesn't help, that memory isn't available to the command that is being run - it needs to be in a file, or sent through a pipe. The file method is more general, as it also allows mmap(), and is much easier to write the code to avoid issues when there is more than one here doc for the same command - so the data needs to be available to both in whatever order the command wants it. Much the same arguments would apply to output. kre
Re: Directing into a variable doesn't work
On 06/25/2018 12:27 AM, Robert Elz wrote: > That's not the real issue - rather it is that a here doc is presented to the > command beng run as a file descrptior OK, thanks, that makes sense. In the case of a here-variable, that would definitely be the case then. Peter
Re: Directing into a variable doesn't work
On Sun, Jun 24, 2018 at 8:35 PM, Peter Passchier wrote: > On 06/25/2018 12:27 AM, Robert Elz wrote: > > That's not the real issue - rather it is that a here doc is presented to > the > > command beng run as a file descrptior > > OK, thanks, that makes sense. In the case of a here-variable, that would > definitely be the case then. > > Peter > > > Also, in practice, there is also a good chance that the tempfiles are written on a tmpfs filesystem and thus memory, and even if it is not the case with all the caching done by the OS there is a chance your data will never actually be written on the disk.
