[Rd] pipe(): input to, and output from, a single process
hi. i'd like to instantiate sed(1), send it some input, and retrieve its output, all via pipes (rather than an intermediate file). my sense from pipe and looking at the sources (sys-unix.c) is that is not possible. is that true? are there any thoughts of providing such a facility? cheers, Greg __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] pipe(): input to, and output from, a single process
I am not sure if `pipe()` works for this, but if it turns out that it
does not, then you can use the processx package, e.g.:
> p <- processx::process$new("sed", c("-l", "s/a/x/g"), stdin = "|", stdout =
> "|")
> p$write_input("foobar\n")
> p$read_output()
[1] "foobxr\n"
The `-l` sed flag is to make sed line-buffered, otherwise it is will
not produce output until there is enough.
`$write_input()` and `$read_output()` are not easy to program, in particular:
* `$write_input()` returns the chunk of data that it hasn't managed to
write into the pipe. You need to call `$write_input() again, with this
data next, usually.
* `$read_output()` returns an empty string if there is no data to
read, so typically you want to call `p$poll()` first, to make sure
that there is something to read.
* `$read_output()` might not read whole lines, so maybe
`$read_output_lines()` is better for you.
* Close the stdin of the process if you want to quit cleanly:
`close(p$get_input_connection())`.
* There is currently no way to poll the input side of the pipe. :(
HTH, Gabor
On Mon, Mar 16, 2020 at 11:31 AM Greg Minshall wrote:
>
> hi. i'd like to instantiate sed(1), send it some input, and retrieve
> its output, all via pipes (rather than an intermediate file).
>
> my sense from pipe and looking at the sources (sys-unix.c) is that is
> not possible. is that true? are there any thoughts of providing such a
> facility?
>
> cheers, Greg
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] pipe(): input to, and output from, a single process
On Fri, 13 Mar 2020 20:26:43 +0300 Greg Minshall wrote: > my sense from pipe and looking at the sources (sys-unix.c) is that is > not possible. is that true? are there any thoughts of providing > such a facility? Pipes (including those created by popen(3), which R pipe() uses internally) are uni-directional data channels. While it could be possible to open two pipes for both stdin and stdout of the child process, doing so correctly is complicated because of differences in buffering provided by the runtime: when stdin/stdout is not a terminal, buffering mode may be set to block-oriented instead of line-oriented, resulting in both parent and child being dead-locked, waiting to fill the buffer instead of returning from the blocking call after the first newline. (Hence the -l flag to sed mentioned by Gábor Csárdi, which avoids this problem for sed). Programs designed to first read stdin until end-of-file, then process the input and print results on the stdout are usually safe to use in this way, but others may be not. Software specifically designed to control other software (e.g. Expect [*]) gets around this limitation by running the child processes inside pseudo-terminals and/or running in event-driven manner, being ready to service the child process whether it wants to read its stdin or write to stdout. Since sed has its -l flag, it should be possible to safely drive it line-by-line with the help of processx, but not via pipe(). -- Best regards, Ivan [*] https://core.tcl-lang.org/expect/index __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] pipe(): input to, and output from, a single process
On 13 March 2020 at 20:26, Greg Minshall wrote: | hi. i'd like to instantiate sed(1), send it some input, and retrieve | its output, all via pipes (rather than an intermediate file). | | my sense from pipe and looking at the sources (sys-unix.c) is that is | not possible. is that true? are there any thoughts of providing such a | facility? Octave had this already in the 1990s, see documentation for 'popen2' here: https://octave.org/doc/v4.2.1/Controlling-Subprocesses.html As it says 'Start a subprocess with two-way communication'. Dirk -- http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] pipe(): input to, and output from, a single process
Well, if you want blocking, you can poll with an infinite timeout. This returns if 1) there is output, 2) the process terminates, or 3) you interrupt with CTRL+C / ESC /etc. and then right after the polling, you can read the output. This still works if the process has finished already. Gabor On Mon, Mar 16, 2020 at 7:06 PM Greg Minshall wrote: > > Gabor, thanks. yes, managing the two-way communication is always a bit > error-prone, as it depends on the input/output characteristics of the > two ends -- they either match, or deadlock. it's too bad if polling is > always *required* -- i'd think sometimes a programmer would be happy > blocking, though other times one wants better control over when to > block. cheers, Greg __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] pipe(): input to, and output from, a single process
FWIW if you're on unix, you can use named pipes (fifos) for that:
> system("mkfifo my.output")
> p = pipe("sed -l s:hello:oops: > my.output", "w")
> i = file("my.output", "r", blocking=FALSE, raw=TRUE)
> writeLines("hello!\n", p)
> flush(p)
> readLines(i, 1)
[1] "oops!"
Cheers,
Simon
> On 14/03/2020, at 6:26 AM, Greg Minshall wrote:
>
> hi. i'd like to instantiate sed(1), send it some input, and retrieve
> its output, all via pipes (rather than an intermediate file).
>
> my sense from pipe and looking at the sources (sys-unix.c) is that is
> not possible. is that true? are there any thoughts of providing such a
> facility?
>
> cheers, Greg
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] new bquote feature splice does not address a common LISP @ use case?
Dear R-devel, There is a new feature in R-devel, which explicitly refers to LISP @ operator for splicing. > The backquote function bquote() has a new argument splice to enable splicing > a computed list of values into an expression, like ,@ in LISP's backquote. Although the most upvoted SO question asking for exactly LISP's @ functionality in R doesn't seems to be addressed by this new feature. Is it possible to use new splice feature to create `6 - 5 + 4` expression rather than `6 - (5 + 4)`? b = quote(5+4) b #5 + 4 c = bquote(6-.(b)) c #6 - (5 + 4) d = bquote(6-..(b), splice=TRUE) d #6 - (5 + 4) There is corresponding LISP code provided CL-USER> (setf b `(5 + 4)) (5 + 4) CL-USER> (setf c `(6 - ,@b)) (6 - 5 + 4) CL-USER> (setf c-non-spliced `(6 - ,b)) (6 - (5 + 4)) CL-USER> Thanks, Jan Gorecki __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] strange bahaviour of predict.lm
Hello, Below is my code: > A <- matrix(rnorm(10*3),ncol=3) > b <- runif(10) > reg <- lm(b ~ A) > A1 <- matrix(rnorm(5*3),ncol=3) > A1 <- as.data.frame(A1) > b1 <- predict(reg,A1) Warning message: 'newdata' had 5 rows but variables found have 10 rows And instead of being an array of length 5, b1 is of length 10 and is identical to reg$fitted.values I think that it should not be like this. Let me note that for lm I do not care about this as much since I can use reg$coefficients, but unfortunately this behaviour is "inherited" by other methods. When I am trying to fit a regression tree, predicting from the object without using 'predict' method is less trivial. Thank you,Moshe. P.S. just in case:> sessionInfo() R version 3.6.2 (2019-12-12) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Linux Mint 19.1 Matrix products: default BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1 LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1 locale: [1] C attached base packages: [1] stats graphics grDevices utils datasets methods base loaded via a namespace (and not attached): [1] compiler_3.6.2 tools_3.6.2 [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] ":::" operator doesn't work with data object Ecdat:::Crime
The ":::" operator doesn't work for me with "Ecdat:::Crime" on either macOS 10.15.3 or Windows 10. A different but related issue is that "plm::Crime" says "Error: 'Crime' is not an exported object from 'namespace:plm'", even though "library(plm); data(Crime); Crime" works. I would naively think a user should be able to compare "Crime" objects documented in different packages using the "::" and ":::" operators, even if a package maintainer chooses not to "export" data objects. What do you think? Thanks, Spencer Graves *** The following is from my Mac; I could give you the comparable results from Windows 10 if you want it. > dim(Ecdat::Crime) [1] 630 24 > Ecdat:::Crime Error in get(name, envir = asNamespace(pkg), inherits = FALSE) : object 'Crime' not found > sessionInfo() R version 3.6.3 (2020-02-29) Platform: x86_64-apple-darwin15.6.0 (64-bit) Running under: macOS Catalina 10.15.3 Matrix products: default BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib locale: [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8 attached base packages: [1] stats graphics grDevices utils datasets [6] methods base loaded via a namespace (and not attached): [1] Rcpp_1.0.3 lattice_0.20-40 [3] mvtnorm_1.1-0 BMA_3.18.12 [5] Ecdat_0.3-7 rrcov_1.5-2 [7] MASS_7.3-51.5 leaps_3.1 [9] grid_3.6.3 pcaPP_1.9-73 [11] stats4_3.6.3 TeachingDemos_2.10 [13] Ecfun_0.2-4 robustbase_0.93-5 [15] xml2_1.2.5 Matrix_1.2-18 [17] splines_3.6.3 tools_3.6.3 [19] DEoptimR_1.0-8 jpeg_0.1-8.1 [21] survival_3.1-11 compiler_3.6.3 [23] inline_0.3.15 fda_2.4.8.1 [1] The six "Crime" objects I found were in the following packages: Ecdat, BSDA, plm, mosaicModel, statisticalModeling, and gpk. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] strange bahaviour of predict.lm
Hello, The problem seems to be that A is a matrix. The following solves the error. # create A and b as in your code then run A <- as.data.frame(A) df1 <- cbind(A, b) reg <- lm(b ~ ., df1) # etc Hope this helps, Rui Barradas Às 04:36 de 17/03/20, Moshe Olshansky via R-devel escreveu: Hello, Below is my code: A <- matrix(rnorm(10*3),ncol=3) b <- runif(10) reg <- lm(b ~ A) A1 <- matrix(rnorm(5*3),ncol=3) A1 <- as.data.frame(A1) b1 <- predict(reg,A1) Warning message: 'newdata' had 5 rows but variables found have 10 rows And instead of being an array of length 5, b1 is of length 10 and is identical to reg$fitted.values I think that it should not be like this. Let me note that for lm I do not care about this as much since I can use reg$coefficients, but unfortunately this behaviour is "inherited" by other methods. When I am trying to fit a regression tree, predicting from the object without using 'predict' method is less trivial. Thank you,Moshe. P.S. just in case:> sessionInfo() R version 3.6.2 (2019-12-12) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Linux Mint 19.1 Matrix products: default BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1 LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1 locale: [1] C attached base packages: [1] stats graphics grDevices utils datasets methods base loaded via a namespace (and not attached): [1] compiler_3.6.2 tools_3.6.2 [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
