Re: [Rd] package check fail on Windows-release only?
> Paul Johnson > on Mon, 20 Nov 2017 14:59:26 -0600 writes: > I mistakenly left a write in "/tmp" in the rockchalk package (version > 1.8.109) that I uploaded last Friday. Kurt H wrote and asked me to fix > today. > While uploading a new one, I became aware of a problem I had not seen. > The version I uploaded last Friday, 1.8.109, has OK status on all > platforms except r-release-windows-ix86+x86_64. I get OK on > oldrel-windows and also on devel-windows. > https://cran.r-project.org/web/checks/check_results_rockchalk.html > Can you please advise me on what to do? I don't understand, well, > anything about this :( Dear Paul, something like this really should have been posted to the R-package-devel list, not to R-devel; as it is here now, I'm crossposting to there for one msg Please remove R-devel@.. from CC if/when replying and keep it there. > The error says: > * installing *source* package 'rockchalk' ... > ** package 'rockchalk' successfully unpacked and MD5 sums checked > ** R > ** data > ** inst > ** preparing package for lazy loading > Warning: S3 methods 'print.sparseSummary', 'print.diagSummary', > 'c.abIndex', 'c.sparseVector', 'as.array.Matrix', > 'as.array.sparseVector', 'as.matrix.Matrix', 'as.matrix.sparseVector', > 'as.vector.Matrix', 'as.vector.sparseVector' were declared in > NAMESPACE but not found > Error in namespaceExport(ns, exports) : > undefined exports: %&%, Cholesky, .SuiteSparse_version, Diagonal, > .symDiagonal, .sparseDiagonal, .trDiagonal, Hilbert, KhatriRao, > Matrix, MatrixClass, spMatrix, sparseMatrix, rsparsematrix, Schur, > abIseq, abIseq1, rep2abI, band, bandSparse, bdiag, .bdiag, > c.sparseVector, condest, onenormest, .asmatrix, .dsy2mat, .dsy2dsp, > .dxC2mat, .T2Cmat, ..2dge, .dense2sy, .C2nC, .nC2d, .nC2l, .diag.dsC, > .solve.dgC.chol, .solve.dgC.qr, .solve.dgC.lu, diagN2U, diagU2N, > .diagU2N, .diag2tT, .diag2sT, .diag2mat, drop0, expand, expm, facmul, > fac2sparse, fac2Sparse, forceSymmetric, T2graph, graph2T, > anyDuplicatedT, uniqTsparse, isTriangular, isDiagonal, isLDL, > is.null.DN, invPerm, lu, nearPD, nnzero, formatSpMatrix, > formatSparseM, .formatSparseSimple, printSpMatrix, printSpMatrix2, > qrR, rankMatrix, readHB, readMM, sparse.model.matrix, sparseVector, > symmpart, skewpart, tril, triu, updown, pack, unpack, > .updateCHMfactor, .validateCsparse, writeMM, cBind, rBind > ERROR: lazy loading failed for package 'rockchalk' > * removing 'd:/Rcompile/CRANpkg/lib/3.4/rockchalk' > * restoring previous 'd:/Rcompile/CRANpkg/lib/3.4/rockchalk' all of these these are names of Matrix-package objects.. and I know that yesterday Matrix package version changes (1.1-11 --> 1.1-12) happened on CRAN and hence in servers which update.. So it *could* be this is all just a temporary problem aka "race situation" , where the check of your package very unhappily happened exactly at a moment where the new Matrix package was already there, but not quite seems quite improbable, but then your problem seems "rare".. Martin __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Small performance bug in [.Date
> Hadley Wickham
> on Mon, 20 Nov 2017 12:50:24 -0600 writes:
> Hi all,
> I think there's an unnecessary line in [.Date which has a considerable
> impact on performance when subsetting large dates:
> x <- Sys.Date() + 1:1e6
> microbenchmark::microbenchmark(x[1])
> #> Unit: microseconds
> #> expr min lq mean median uq max neval
> #> x[1] 920.651 1039.346 3624.833 2294.404 3786.881 41176.38 100
> `[.Date` <- function(x, ..., drop = TRUE) {
> cl <- oldClass(x)
> # class(x) <- NULL
> val <- NextMethod("[")
> class(val) <- cl
> val
> }
> microbenchmark::microbenchmark(x[1])
> #> Unit: microseconds
> #> expr min lq mean medianuq max neval
> #> x[1] 2.738 3.0225 28.40893 3.269 3.513 2470.068 100
> Setting the class of x to NULL is problematic because it forces a
> copy, and I'm pretty sure it's unnecessary as NextMethod() does not
> consult the class of x, but instead uses .Class.
Yes, at least so it looks in src/main/objects.c
Also, we had a very similar change a while ago :
r65926 | luke | 2014-06-12 15:54:38 +0200 (Thu, 12. Jun 2014) | 2 Zeilen
Geänderte Pfade:
M src/library/base/R/datetime.R
Commented out class(x) <- NULL in [.POSIXct and [[.POSICct.
and we never seemed to have followed up in a systematic manner
finding other places where this happens and could be
eliminated. I see about half a dozen examples in
base/R/dates.R alone and am trying to find more in other places.
[maybe this used to be necessary for very early different
versions of NextMethod() which were not yet optimized using .Class etc]
Thank you very much,
Hadley!
Martin
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Truncating vectors by reference in C-backend
Dear all,
I want to create a function shrinkVector(x) that takes x and truncates
it to the first element without copy.
With SETLENGTH and SET_TRUELENGTH, I can achieve that. You can find a
reproducible example below.
But the memory that was freed is not available for other objects
afterwards, except if x is a list (VECSXP). Any suggestions?
library(inline)
## define the shrinking function
shrinkVector <- cfunction(signature(x = "ANY"),
body = paste0("SETLENGTH(x, 1);",
"SET_TRUELENGTH(x,
1);",
"return(R_NilValue);"))
## create a large vector that only fits into memory once
x <- 1 : 2e9
## shrink it
shrinkVector(x)
## shrinking seems to have worked
print(length(x) == 1)
# [1] TRUE
print(object.size(x))
# 48 bytes
## but I can't reuse the memory for a large x2:
x2 <- 1 : 2e8
# Error: cannot allocate vector of size 762.9 Mb
## if I remove x, it works
rm(x)
gc()
x2 <- 1 : 2e8
Thank you very much for your help.
Kind regards,
Markus Bonsch
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Truncating vectors by reference in C-backend
DO NOT DO THIS!!! SETLENGTH and SET_TRUELENGTH are NOT part of the API
and are for internal use only.
Tho proper way to do this is to copy the data you want into a smaller
vector and allow the larger one to be garbage collected.
[VECSXPs are like all other vectors in that they are contigously
allocated by malloc, and there is no mprtable way to return part
a memory region allocated by malloc back to malloc].
There is experimental INTERNAL support for growable vectors that is
managed with INTERNALLY with TRUELENGTH and a bit setting, but this is
NOT, at leat not yet, intended for use by packages. The ALTREP
framework that should be available in the next release might allow
something like this to be done, but care is needed to make sure R's
copying semantics are adhered to.
Best,
luke
On Tue, 21 Nov 2017, mbon...@posteo.de wrote:
Dear all,
I want to create a function shrinkVector(x) that takes x and truncates it
to the first element without copy.
With SETLENGTH and SET_TRUELENGTH, I can achieve that. You can find a
reproducible example below.
But the memory that was freed is not available for other objects
afterwards, except if x is a list (VECSXP). Any suggestions?
library(inline)
## define the shrinking function
shrinkVector <- cfunction(signature(x = "ANY"),
body = paste0("SETLENGTH(x, 1);",
"SET_TRUELENGTH(x, 1);",
"return(R_NilValue);"))
## create a large vector that only fits into memory once
x <- 1 : 2e9
## shrink it
shrinkVector(x)
## shrinking seems to have worked
print(length(x) == 1)
# [1] TRUE
print(object.size(x))
# 48 bytes
## but I can't reuse the memory for a large x2:
x2 <- 1 : 2e8
# Error: cannot allocate vector of size 762.9 Mb
## if I remove x, it works
rm(x)
gc()
x2 <- 1 : 2e8
Thank you very much for your help.
Kind regards,
Markus Bonsch
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
--
Luke Tierney
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa Phone: 319-335-3386
Department of Statistics andFax: 319-335-3017
Actuarial Science
241 Schaeffer Hall email: luke-tier...@uiowa.edu
Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [R-pkg-devel] package check fail on Windows-release only?
On 21.11.2017 10:15, Martin Maechler wrote: Paul Johnson on Mon, 20 Nov 2017 14:59:26 -0600 writes: > I mistakenly left a write in "/tmp" in the rockchalk package (version > 1.8.109) that I uploaded last Friday. Kurt H wrote and asked me to fix > today. > While uploading a new one, I became aware of a problem I had not seen. > The version I uploaded last Friday, 1.8.109, has OK status on all > platforms except r-release-windows-ix86+x86_64. I get OK on > oldrel-windows and also on devel-windows. > https://cran.r-project.org/web/checks/check_results_rockchalk.html > Can you please advise me on what to do? I don't understand, well, > anything about this :( Dear Paul, something like this really should have been posted to the R-package-devel list, not to R-devel; as it is here now, I'm crossposting to there for one msg Please remove R-devel@.. from CC if/when replying and keep it there. > The error says: > * installing *source* package 'rockchalk' ... > ** package 'rockchalk' successfully unpacked and MD5 sums checked > ** R > ** data > ** inst > ** preparing package for lazy loading > Warning: S3 methods 'print.sparseSummary', 'print.diagSummary', > 'c.abIndex', 'c.sparseVector', 'as.array.Matrix', > 'as.array.sparseVector', 'as.matrix.Matrix', 'as.matrix.sparseVector', > 'as.vector.Matrix', 'as.vector.sparseVector' were declared in > NAMESPACE but not found > Error in namespaceExport(ns, exports) : > undefined exports: %&%, Cholesky, .SuiteSparse_version, Diagonal, > .symDiagonal, .sparseDiagonal, .trDiagonal, Hilbert, KhatriRao, > Matrix, MatrixClass, spMatrix, sparseMatrix, rsparsematrix, Schur, > abIseq, abIseq1, rep2abI, band, bandSparse, bdiag, .bdiag, > c.sparseVector, condest, onenormest, .asmatrix, .dsy2mat, .dsy2dsp, > .dxC2mat, .T2Cmat, ..2dge, .dense2sy, .C2nC, .nC2d, .nC2l, .diag.dsC, > .solve.dgC.chol, .solve.dgC.qr, .solve.dgC.lu, diagN2U, diagU2N, > .diagU2N, .diag2tT, .diag2sT, .diag2mat, drop0, expand, expm, facmul, > fac2sparse, fac2Sparse, forceSymmetric, T2graph, graph2T, > anyDuplicatedT, uniqTsparse, isTriangular, isDiagonal, isLDL, > is.null.DN, invPerm, lu, nearPD, nnzero, formatSpMatrix, > formatSparseM, .formatSparseSimple, printSpMatrix, printSpMatrix2, > qrR, rankMatrix, readHB, readMM, sparse.model.matrix, sparseVector, > symmpart, skewpart, tril, triu, updown, pack, unpack, > .updateCHMfactor, .validateCsparse, writeMM, cBind, rBind > ERROR: lazy loading failed for package 'rockchalk' > * removing 'd:/Rcompile/CRANpkg/lib/3.4/rockchalk' > * restoring previous 'd:/Rcompile/CRANpkg/lib/3.4/rockchalk' all of these these are names of Matrix-package objects.. and I know that yesterday Matrix package version changes (1.1-11 --> 1.1-12) happened on CRAN and hence in servers which update.. So it *could* be this is all just a temporary problem aka "race situation" , where the check of your package very unhappily happened exactly at a moment where the new Matrix package was already there, but not quite seems quite improbable, but then your problem seems "rare".. Perfect analysis: Race condition. Happens from time to time. Can you simply resubmit it, please. Best, Uwe Ligges Martin __ r-package-de...@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Small performance bug in [.Date
> Yes, at least so it looks in src/main/objects.c > > Also, we had a very similar change a while ago : > > r65926 | luke | 2014-06-12 15:54:38 +0200 (Thu, 12. Jun 2014) | 2 Zeilen > Geänderte Pfade: >M src/library/base/R/datetime.R > > Commented out class(x) <- NULL in [.POSIXct and [[.POSICct. > > > and we never seemed to have followed up in a systematic manner > finding other places where this happens and could be > eliminated. I see about half a dozen examples in > base/R/dates.R alone and am trying to find more in other places. > > [maybe this used to be necessary for very early different > versions of NextMethod() which were not yet optimized using .Class etc] Thanks for making the fix! Hadley -- http://hadley.nz __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] str() not displaying names
[This is a "re-post" -- for some reason it never appeared on R-devel]
> Etienne Sanchez
> on Tue, 14 Nov 2017 19:33:07 +0100 writes:
> In some cases, str() does not print the "names" attribute of the object:
>
> u <- structure(c(5, 6), names = c("a", "b"), color = "blue")
> str(u)
> # atomic [1:2] 5 6
> # - attr(*, "color")= chr "blue"
>
> Is it a bug or a design choice?
> Originally asked here:
> https://stackoverflow.com/q/47185756/6656269
It's not a bug in the sense that a long time ago -- when I wrote
the first version of str(), for S-plus, before R existed --
I had decided that when is.vector(.) was false, for whatever
reason, the atomic vectors should be shown as above.
I don't remember if S-plus used the same somewhat surprising
definition of is.vector(.) as R does (but I think it did):
it is only TRUE for a vector that has no other attributes than
possibly "names".
I did occasionally find that the historical choice probably was
not quite the best in hindsight, but never got convinced that it
should be changed...
Once I'll have finished the deparse/dput/dump changes in
R-devel (hopefully within a week), and as R-devel has quite a
few small changes to R <= 3.4.x anyway, I may consider to change
utils:::str.default here ... and have a few dozens of package
maintainers and R users live with the fact the str() outputs
will have changed in late spring next year ...
Notably if you or other give convincing reasons why it's worth
to change... but note that it's quite easy to give "Pro"
reasons, but there are "Cons" and for such cases a lot of "Cons"
are related to "there must 100's of 1000s of R code lines using
str(), and so there will be 100s of places where the output
changes, ( ... but then I'd guestimate that the change would be
to the better in most cases).
Martin
--
Martin Maechler, ETH Zurich
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Are Rprintf and REprintf thread-safe?
Is it safe to call Rprintf and REprintf from a background thread? I'm working on a package that makes calls to fprintf(stderr, ...) on a background thread when errors happen, but when I run R CMD check, it says: Compiled code should not call entry points which might terminate R nor write to stdout/stderr instead of to the console, nor the system RNG. Is it safe to replace these calls with REprintf()? -Winston __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Are Rprintf and REprintf thread-safe?
On Tue, 21 Nov 2017, Winston Chang wrote: Is it safe to call Rprintf and REprintf from a background thread? I'm working on a package that makes calls to fprintf(stderr, ...) on a background thread when errors happen, but when I run R CMD check, it says: Compiled code should not call entry points which might terminate R nor write to stdout/stderr instead of to the console, nor the system RNG. Is it safe to replace these calls with REprintf()? Only if you enjoy race conditions or segfaults. Rprintf and REprintf are not thread-safe. Best, luke -Winston __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Luke Tierney Ralph E. Wareham Professor of Mathematical Sciences University of Iowa Phone: 319-335-3386 Department of Statistics andFax: 319-335-3017 Actuarial Science 241 Schaeffer Hall email: luke-tier...@uiowa.edu Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Are Rprintf and REprintf thread-safe?
Thanks - I'll find another way to send messages to the main thread for printing. -Winston On Tue, Nov 21, 2017 at 12:42 PM, wrote: > On Tue, 21 Nov 2017, Winston Chang wrote: > >> Is it safe to call Rprintf and REprintf from a background thread? I'm >> working on a package that makes calls to fprintf(stderr, ...) on a >> background thread when errors happen, but when I run R CMD check, it >> says: >> >> Compiled code should not call entry points which might terminate R nor >> write to stdout/stderr instead of to the console, nor the system RNG. >> >> Is it safe to replace these calls with REprintf()? > > > Only if you enjoy race conditions or segfaults. > > Rprintf and REprintf are not thread-safe. > > Best, > > luke > > >> >> -Winston >> >> __ >> R-devel@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel >> > > -- > Luke Tierney > Ralph E. Wareham Professor of Mathematical Sciences > University of Iowa Phone: 319-335-3386 > Department of Statistics andFax: 319-335-3017 >Actuarial Science > 241 Schaeffer Hall email: luke-tier...@uiowa.edu > Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Are Rprintf and REprintf thread-safe?
On 11/21/2017 04:12 PM, Winston Chang wrote:
Thanks - I'll find another way to send messages to the main thread for printing.
The CRAN synchronicity and Bioconductor BiocParallel packages provide
inter-process locks that you could use to surround writes (instead of
sending message to the main thread), also easy enough to incorporate at
the C level using the BH package as source for relevant boost header.
Martin
-Winston
On Tue, Nov 21, 2017 at 12:42 PM, wrote:
On Tue, 21 Nov 2017, Winston Chang wrote:
Is it safe to call Rprintf and REprintf from a background thread? I'm
working on a package that makes calls to fprintf(stderr, ...) on a
background thread when errors happen, but when I run R CMD check, it
says:
Compiled code should not call entry points which might terminate R nor
write to stdout/stderr instead of to the console, nor the system RNG.
Is it safe to replace these calls with REprintf()?
Only if you enjoy race conditions or segfaults.
Rprintf and REprintf are not thread-safe.
Best,
luke
-Winston
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
--
Luke Tierney
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa Phone: 319-335-3386
Department of Statistics andFax: 319-335-3017
Actuarial Science
241 Schaeffer Hall email: luke-tier...@uiowa.edu
Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
This email message may contain legally privileged and/or...{{dropped:2}}
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] `[[<-.data.frame` leaves holes after existing columns and returns a corrupt data frame
Hi, `[<-.data.frame` is cautious about not leaving holes after existing columns: > `[<-`(data.frame(id=1:6), 3, value=data.frame(V3=11:16)) Error in `[<-.data.frame`(data.frame(id = 1:6), 3, value = data.frame(V3 = 11:16)) : new columns would leave holes after existing columns but `[[<-.data.frame` not so much: > `[[<-`(data.frame(id=1:6), 3, value=11:16) id V3 1 1 NULL 11 2 2 12 3 3 13 4 4 14 5 5 15 6 6 16 Warning message: In format.data.frame(x, digits = digits, na.encode = FALSE) : corrupt data frame: columns will be truncated or padded with NAs The latter should probably behave like the former in that case. Maybe by sharing more code with it? Thanks, H. -- Hervé Pagès Program in Computational Biology Division of Public Health Sciences Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N, M1-B514 P.O. Box 19024 Seattle, WA 98109-1024 E-mail: hpa...@fredhutch.org Phone: (206) 667-5791 Fax:(206) 667-1319 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] `[<-.data.frame` sets rownames incorrectly
Hi,
Here is another problem with data frame subsetting:
> df <- data.frame(aa=1:3)
> value <- data.frame(aa=11:12, row.names=c("A", "B"))
> `[<-`(df, 4:5, , value=value)
aa
1 1
2 2
3 3
A 11
B 12
> `[<-`(df, 5:4, , value=value)
aa
1 1
2 2
3 3
B 12
A 11
For this last result, the rownames of the 2 last rows should
be swapped.
H.
--
Hervé Pagès
Program in Computational Biology
Division of Public Health Sciences
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N, M1-B514
P.O. Box 19024
Seattle, WA 98109-1024
E-mail: hpa...@fredhutch.org
Phone: (206) 667-5791
Fax:(206) 667-1319
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] `[<-.data.frame` sets rownames incorrectly
On 11/21/2017 06:19 PM, Hervé Pagès wrote:
Hi,
Here is another problem with data frame subsetting:
> df <- data.frame(aa=1:3)
> value <- data.frame(aa=11:12, row.names=c("A", "B"))
> `[<-`(df, 4:5, , value=value)
aa
1 1
2 2
3 3
A 11
B 12
> `[<-`(df, 5:4, , value=value)
aa
1 1
2 2
3 3
B 12
A 11
This actually produces:
> `[<-`(df, 5:4, , value=value)
aa
1 1
2 2
3 3
A 12
B 11
but should instead produce:
aa
1 1
2 2
3 3
B 12
A 11
sorry for the confusion.
H.
For this last result, the rownames of the 2 last rows should
be swapped.
H.
--
Hervé Pagès
Program in Computational Biology
Division of Public Health Sciences
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N, M1-B514
P.O. Box 19024
Seattle, WA 98109-1024
E-mail: hpa...@fredhutch.org
Phone: (206) 667-5791
Fax:(206) 667-1319
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
