Re: [Rd] error in parallel:::sendMaster

[Andreas Kersting]

Hi all,

With the help of Tomas, I was able to track the issue down: Prior to R v3.6.0 
the parallel package passes an uninitialized variable as the file descriptor 
argument to the close system call. 

In my particular R session this uninitialized variable (reproducibly) was 
holding the value 7, which corresponded to the file descriptor of the write end 
of the pipe the second child would use to send its results to the master. 
Hence, the child unintentionally closed this pipe directly after fork in 
close_fds_child_ci() resulting in sendMaster() later failing with EBADF.

It was fixed with this commit: 
https://github.com/wch/r-source/commit/e08cffac1c5b9015a1625938d568b648eb1d8aee

Regards,
Andreas

2019-11-28 13:54 GMT+01:00 Andreas Kersting:
> Hi Tomas,
> 
> I rebuild R (v3.5.2 for now, R-devel to follow) from the Debian package with 
> MC_DEBUG defined and hopefully also with "-Wall -O0 -gdwarf-2 -g3", though I 
> still have to verify this.
> 
> Below is the output. I think it is a total of two mclapply invocations in 
> this R session, the failing one starting around the lines "[1] 15381" and 
> "[1] 15382". The "Error in partDF::write_partDF ..." is because the 
> script/package checks the return value of mclapply and detects that it is not 
> a list of length 85 with only the elements "TRUE".
> 
> Regarding sending you the complete example: I first have to figure out if 
> this is possible at all, because it would involve data of a client.
> 
> Regards,
> Andreas
> 
> parent[15366] created pipes: comm (6->5), sir (8->7)
> parent registers new child 15379
> child process 15379 started
> parent[15366] created pipes: comm (7->6), sir (10->9)
> parent registers new child 15380
> child process 15380 started
> select_children: added child 15380 (6)
> select_children: added child 15379 (5)
> select_children: maxfd=6, wlen=2, wcount=2, timeout=-1.00
> child 15380: send_master (550 bytes)
>   sr = 1
>  - read select 1 children: 15380 
> child 15380: 'mcexit' called
> child 15380 is waiting for permission to exit
> read_child_ci(15380) - read length returned 8
> read_child_ci(15380) - read 550 at 0 returned 550
> select_children: added child 15380 (6)
> select_children: added child 15379 (5)
> select_children: maxfd=6, wlen=2, wcount=2, timeout=-1.00
>   sr = 1
>  - read select 1 children: 15380 
> read_child_ci(15380) - read length returned 8
> detached child 15380 (signal 10)
> child process 15380 got SIGUSR1; child_exit_status=-1
> child 15380: exiting
> select_children: added child 15379 (5)
> select_children: maxfd=5, wlen=1, wcount=1, timeout=-1.00
> child 15380 terminated with exit status 0
> child 15379: send_master (550 bytes)
>   sr = 1
>  - read select 1 children: 15379 
> read_child_ci(15379) - read length returned 8
> read_child_ci(15379) - read 550 at 0 returned 550
> select_children: added child 15379 (5)
> select_children: maxfd=5, wlen=1, wcount=1, timeout=-1.00
> child 15379: 'mcexit' called
> child 15379 is waiting for permission to exit
>   sr = 1
>  - read select 1 children: 15379 
> read_child_ci(15379) - read length returned 8
> detached child 15379 (signal 10)
> child process 15379 got SIGUSR1; child_exit_status=-1
> child 15379: exiting
> removing waited-for child 15380 from the list
> killed detached child 15379 (signal 15)
> removing waited-for child 2147483647 from the list
> child 15379 terminated with exit status 0
> removing waited-for child 15379 from the list
> parent[15366] created pipes: comm (6->5), sir (8->7)
> parent registers new child 15381
> child process 15381 started
> parent[15366] created pipes: comm (7->6), sir (10->9)
> [1] 15381
> parent registers new child 15382
> child process 15382 started
> select_children: added child 15382 (6)
> select_children: added child 15381 (5)
> select_children: maxfd=6, wlen=2, wcount=2, timeout=-1.00
>   sr = 1
>  - read select 1 children: 15382 
> read_child_ci(15382) - read length returned 0
> detached child 15382 (signal 10)
> child process 15382 got SIGUSR1; child_exit_status=-1
> select_children: added child 15381 (5)
> select_children: maxfd=5, wlen=1, wcount=1, timeout=-1.00
> [1] 15382
> child 15382: send_master (526 bytes)
> Error in sendMaster(try(lapply(X = S, FUN = FUN, ...), silent = TRUE)) : 
>   write error, closing pipe to the master
> Calls: lapply ...  ->  -> lapply -> FUN -> sendMaster
> child 15382: 'mcexit' called
> child 15382: exiting
> child 15382 terminated with exit status 1
> child 15381: send_master (538 bytes)
>   sr = 1
>  - read select 1 children: 15381 
> read_child_ci(15381) - read length returned 8
> read_child_ci(15381) - read 538 at 0 returned 538
> child 15381: 'mcexit' called
> child 15381 is waiting for permission to exit
> select_children: added child 15381 (5)
> select_children: maxfd=5, wlen=1, wcount=1, timeout=-1.00
>   sr = 1
>  - read select 1 children: 15381 
> read_child_ci(15381) - read length returned 8
> detached child 15381 (signal 10)
> child proce

Re: [Rd] Maybe bug? Using non-integer frequencies in stats::ts

[Johann R. Kleinbub]

It's been three months without an answer, is it ok to thread bump?
Would someone provide a pointer?

Thank you for your consideration,
Johann


On Mon, 16 Sep 2019 at 15:53, Johann R. Kleinbub 
wrote:
>
> I am developing a package to analyse physiological time-series and I
thought that the most reliable and robust solution was to base it on the
native stats::ts class. In my domain it is common to express series
frequencies as samples-per-second. So ts(..., frequency=10) would mean a
signal sampled 10 times every second, and ts(..., frequency = 1) a signal
sampled every second. Following this logic, a few slower signals are
sampled every 5 seconds (or more), resulting in a frequency of e.g. 0.2
> Nowhere in the documentation is stated that the frequency must be an
integer value, but using fractional values gives inconsistent results.
> For instance, in this example, foo and bar are identical, just with
start-end values shifted by 1. Yet when extracting an arbitrary window, the
'bar' series gives error.
>
> x = 1:22
> foo = ts(x, start = 1.5, end = 106.5, frequency = 0.2)
> bar = ts(x, start = 2.5, end = 107.5, frequency = 0.2)
>
> window(foo, start = 20, end = 30, extend=TRUE)
>
> # Time Series:
> # Start = 20
> # End = 25
> # Frequency = 0.2
> # [1] 5 6
>
> window(bar, start = 20, end = 30, extend=TRUE)
>
> # Error in attr(y, "tsp") <- c(ystart, yend, xfreq) :
> #   invalid time series parameters specified
>
> The reason is in the rounding procedures for ystart and yend at the end
of the stats::window function. For the 'foo' series the ystart and yend
values are calculated as: c(20, 25), whereas for the 'bar' series, they
become c(20, 30) although the window should be of the very same size in
both cases. (A further discussion on the example is at:
https://stackoverflow.com/questions/57928054 )
> Should I report a bug or am I misunderstanding something?
>
> --
> Johann R. Kleinbub, PhD
> University of Padova
> FISPPA Dep. - Section of Applied Psychology
> Cell: +39 3495986373



--
Johann R. Kleinbub, PhD
University of Padova
FISPPA Dep. - Section of Applied Psychology
Cell: +39 3495986373

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Maybe bug? Using non-integer frequencies in stats::ts

[Duncan Murdoch]

On 05/12/2019 11:00 a.m., Johann R. Kleinbub wrote:

It's been three months without an answer, is it ok to thread bump?
Would someone provide a pointer?


I agree it's a bug, and agree with your analysis.  You should report it 
on bugs.r-project.org.  (If you don't have an account there, let us 
know, and either someone will give you one, or someone will report it 
for you.)


As a workaround, I don't see it happening with extend=FALSE, but of 
course that might not suit your needs in general.


Duncan Murdoch





Thank you for your consideration,
Johann


On Mon, 16 Sep 2019 at 15:53, Johann R. Kleinbub 
wrote:


I am developing a package to analyse physiological time-series and I

thought that the most reliable and robust solution was to base it on the
native stats::ts class. In my domain it is common to express series
frequencies as samples-per-second. So ts(..., frequency=10) would mean a
signal sampled 10 times every second, and ts(..., frequency = 1) a signal
sampled every second. Following this logic, a few slower signals are
sampled every 5 seconds (or more), resulting in a frequency of e.g. 0.2

Nowhere in the documentation is stated that the frequency must be an

integer value, but using fractional values gives inconsistent results.

For instance, in this example, foo and bar are identical, just with

start-end values shifted by 1. Yet when extracting an arbitrary window, the
'bar' series gives error.


x = 1:22
foo = ts(x, start = 1.5, end = 106.5, frequency = 0.2)
bar = ts(x, start = 2.5, end = 107.5, frequency = 0.2)

window(foo, start = 20, end = 30, extend=TRUE)

# Time Series:
# Start = 20
# End = 25
# Frequency = 0.2
# [1] 5 6

window(bar, start = 20, end = 30, extend=TRUE)

# Error in attr(y, "tsp") <- c(ystart, yend, xfreq) :
#   invalid time series parameters specified

The reason is in the rounding procedures for ystart and yend at the end

of the stats::window function. For the 'foo' series the ystart and yend
values are calculated as: c(20, 25), whereas for the 'bar' series, they
become c(20, 30) although the window should be of the very same size in
both cases. (A further discussion on the example is at:
https://stackoverflow.com/questions/57928054 )

Should I report a bug or am I misunderstanding something?

--
Johann R. Kleinbub, PhD
University of Padova
FISPPA Dep. - Section of Applied Psychology
Cell: +39 3495986373




--
Johann R. Kleinbub, PhD
University of Padova
FISPPA Dep. - Section of Applied Psychology
Cell: +39 3495986373

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel



__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Error in close.connection(p) : ignoring SIGPIPE signal

[Benjamin Tyner]

Not sure if this is a bug, so posting here first. If I run:

   cnt <- 0L
   while (TRUE) {
       cnt <- cnt + 1L
       p <- pipe("echo /dev/stdin > /dev/null", open = "w")
       writeLines("foobar", p)
       tryCatch(close(p), error = function(e) { print(cnt); stop(e)})
   }

then once cnt gets to around 650, it fails with:

   [1] 654
   Error in close.connection(p) : ignoring SIGPIPE signal

Should I not be using pipe() in this way? Here is my sessionInfo()

   R version 3.6.0 (2019-04-26)
   Platform: x86_64-pc-linux-gnu (64-bit)
   Running under: Ubuntu 18.04.3 LTS

   Matrix products: default
   BLAS:   /home/btyner/R360/lib64/R/lib/libRblas.so
   LAPACK: /home/btyner/R360/lib64/R/lib/libRlapack.so

   locale:
    [1] LC_CTYPE=en_US.UTF-8   LC_NUMERIC=C
    [3] LC_TIME=en_US.UTF-8    LC_COLLATE=en_US.UTF-8
    [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
    [7] LC_PAPER=en_US.UTF-8   LC_NAME=C
    [9] LC_ADDRESS=C   LC_TELEPHONE=C
   [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

   attached base packages:
   [1] stats graphics  grDevices utils datasets  methods base

   loaded via a namespace (and not attached):
   [1] compiler_3.6.0

Regards,
Ben

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Error in close.connection(p) : ignoring SIGPIPE signal

[Andreas Kersting]

Hi Benjamin,

you cannot pipe to echo, since it does not read from stdin. 

echo just echos is first arg, i.e. echo /dev/stdin > /dev/null will echo the 
string "/dev/stdin"to /dev/stdout, which is redirected to /dev/null.

Try 

p <- pipe("cat > /dev/null", open = "w")

instead.

Regards,
Andreas

2019-12-06 02:46 GMT+01:00 Benjamin Tyner:
> Not sure if this is a bug, so posting here first. If I run:
>    cnt <- 0L
>    while (TRUE) {
>        cnt <- cnt + 1L
>        p <- pipe("echo /dev/stdin > /dev/null", open = "w")
>        writeLines("foobar", p)
>        tryCatch(close(p), error = function(e) { print(cnt); stop(e)})
>    }
> 
> then once cnt gets to around 650, it fails with:
> 
>    [1] 654
>    Error in close.connection(p) : ignoring SIGPIPE signal
> 
> Should I not be using pipe() in this way? Here is my sessionInfo()
> 
>    R version 3.6.0 (2019-04-26)
>    Platform: x86_64-pc-linux-gnu (64-bit)
>    Running under: Ubuntu 18.04.3 LTS
> 
>    Matrix products: default
>    BLAS:   /home/btyner/R360/lib64/R/lib/libRblas.so
>    LAPACK: /home/btyner/R360/lib64/R/lib/libRlapack.so
> 
>    locale:
>     [1] LC_CTYPE=en_US.UTF-8   LC_NUMERIC=C
>     [3] LC_TIME=en_US.UTF-8    LC_COLLATE=en_US.UTF-8
>     [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
>     [7] LC_PAPER=en_US.UTF-8   LC_NAME=C
>     [9] LC_ADDRESS=C   LC_TELEPHONE=C
>    [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
> 
>    attached base packages:
>    [1] stats graphics  grDevices utils datasets  methods base
> 
>    loaded via a namespace (and not attached):
>    [1] compiler_3.6.0
> 
> Regards,
> Ben
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
> 
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel