[Rd] MKL with latest Rs
Dear R-devel team, I've encountered problems with recent Rs (>= 3.5.3) and MKL. I've followed Dirk's (http://dirk.eddelbuettel.com/blog/2018/04/15/) and Intel's ( https://software.intel.com/en-us/articles/quick-linking-intel-mkl-blas-lapack-to-r) instructions and many versions of MKL. All works fine in my Ubuntu 18 setup for R 3.5.2 and older. Carrying out the install and linking instructions works fine for later Rs, but calculations involving linear algebra become corrupted. E.g., following Dirk's very simple svd example from that link above ... > n <- 1e3 > X <- matrix(rnorm(n*n),n,n) > Xsvd <- svd(X) > Xsvd$d[1:5] [1] 63.09366 62.47868 62.26147 61.94530 61.84727 ... you get Eigenvalues that are bigger than 1e26 in R-devel, say, but (correct) values near 60 with R 3.5.2. I just wanted to throw that out there in case anyone else had come across something similar. I don't mind using a slightly older R/MKL for my stuff. I might speculate that the Microsoft R Open folks are having similar issues because they're still using older Rs. I haven't tried compiling and linking from source ( https://cran.r-project.org/doc/manuals/R-admin.html#MKL) because I haven't had success with that in some time. -B [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] ALTREP wrappers and factors
Hello,
I’m experimenting with ALTREP and was wondering if there is a preferred way to
create an ALTREP wrapper vector without using .Internal(wrap_meta(…)), which R
CMD check doesn’t like since it uses an .Internal() function.
I was trying to create a factor that used an ALTREP integer, but attempting to
set the class and levels attributes always ended up duplicating and
materializing the integer vector. Using the wrapper avoided this issue.
Here is my initial ALTREP integer vector:
> fc0 <- factor(c("a", "a", "b"))
>
> y <- matter::as.matter(as.integer(fc0))
> y <- matter:::as.altrep(y)
>
> .Internal(inspect(y))
@7fb0ce78c0f0 13 INTSXP g0c0 [NAM(7)] matter vector (mode=3, len=3, mem=0)
Here is what I get without a wrapper:
> fc1 <- structure(y, class="factor", levels=levels(x))
> .Internal(inspect(fc1))
@7fb0cae66408 13 INTSXP g0c2 [OBJ,NAM(2),ATT] (len=3, tl=0) 1,1,2
ATTRIB:
@7fb0ce771868 02 LISTSXP g0c0 []
TAG: @7fb0c80043d0 01 SYMSXP g1c0 [MARK,LCK,gp=0x4000] "class" (has value)
@7fb0c9fcbe90 16 STRSXP g0c1 [NAM(7)] (len=1, tl=0)
@7fb0c80841a0 09 CHARSXP g1c1 [MARK,gp=0x61] [ASCII] [cached] "factor"
TAG: @7fb0c8004050 01 SYMSXP g1c0 [MARK,NAM(7),LCK,gp=0x4000] "levels" (has
value)
@7fb0d1dd58c8 16 STRSXP g0c2 [MARK,NAM(7)] (len=2, tl=0)
@7fb0c81bf4c0 09 CHARSXP g1c1 [MARK,gp=0x61] [ASCII] [cached] "a"
@7fb0c90ba728 09 CHARSXP g1c1 [MARK,gp=0x61] [ASCII] [cached] "b"
Here is what I get with a wrapper:
> fc2 <- structure(.Internal(wrap_meta(y, 0, 0)), class="factor",
> levels=levels(x))
> .Internal(inspect(fc2))
@7fb0ce764630 13 INTSXP g0c0 [OBJ,NAM(2),ATT] wrapper [srt=0,no_na=0]
@7fb0ce78c0f0 13 INTSXP g0c0 [NAM(7)] matter vector (mode=3, len=3, mem=0)
ATTRIB:
@7fb0ce764668 02 LISTSXP g0c0 []
TAG: @7fb0c80043d0 01 SYMSXP g1c0 [MARK,LCK,gp=0x4000] "class" (has value)
@7fb0c9fcb010 16 STRSXP g0c1 [NAM(7)] (len=1, tl=0)
@7fb0c80841a0 09 CHARSXP g1c1 [MARK,gp=0x61] [ASCII] [cached] "factor"
TAG: @7fb0c8004050 01 SYMSXP g1c0 [MARK,NAM(7),LCK,gp=0x4000] "levels" (has
value)
@7fb0d1dd58c8 16 STRSXP g0c2 [MARK,NAM(7)] (len=2, tl=0)
@7fb0c81bf4c0 09 CHARSXP g1c1 [MARK,gp=0x61] [ASCII] [cached] "a"
@7fb0c90ba728 09 CHARSXP g1c1 [MARK,gp=0x61] [ASCII] [cached] "b"
Is there a way to do this that doesn’t rely on .Internal() and won’t produce R
CMD check warnings?
~~~
Kylie Ariel Bemis
Khoury College of Computer Sciences
Northeastern University
kuwisdelu.github.io
[[alternative HTML version deleted]]
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Addition of a meta viewport tag to HTML manuals
Thanks for both the support & sage advice, Martin!
And, aye, tis straightforward to convert the perl one-liner to a
shell/sed idiom.
A kind soul from the list has also offered to walk me through the
"provide a patch" process and I'll do my best to get it right on Par 1
:-)
-Bob
On Mon, Jul 15, 2019 at 5:54 AM Martin Maechler
wrote:
>
> > Bob Rudis
> > on Tue, 9 Jul 2019 14:24:24 -0400 writes:
>
> > The addition of a single line:
> >
>
> > at in the of the R HTML generated manuals would make them much
> easier to read on mobile devices.
>
> > texi2any (which generates the HTML files) is based on long-working Perl
> code that includes many modern HTML elements but does not include this one.
>
> > A Perl one-liner in the install-html: Makefile directive in Makefile.in:
>
> > install-html: installdirs
> > @for f in $(OBJECTS_HTML); do \
> > if test -f $${f} ; then \
> > $(INSTALL_DATA) $${f} "$(DESTDIR)$(rdocdir)/manual"; \
> > perl -pi -e 's/\ content="width=device-width, initial-scale=1.0">\n > fi \
> > done
>
> > would insert this (I still need to read Makefile.win to see where it
> should go there) and I'd be glad to create a PR unless folks do not think
> better accessibility on mobile is a good idea.
>
> To the contrary.
> Thank you very much, Bob, for bringing this up, here!
>
> > $(PERL) does not seem to be defined but Perl itself is a requirement
> for texi2any so it is definitely something that would work in the current
> installation process.
>
> > -Bob
>
> Hmm,.. a very long time ago, perl was an absolute requirement
> for building R from the sources, but in the mean time, it's not
> been required anymore strictly *). AFAIK, there are alternative versions
> of versions/alternatives to texi2any (say on Windoze .. or
> bizarre Linux distros or non-linux unices), and I'm almost sure
> we do not want to require perl explicitly.
>
> We are using R itself in many places for installation things,
> but here, it should be possible to use smaller unix tools (such
> as 'sed' and 'grep' say) instead.
>
> If you (or someone else) provided a small patch for using those
> instead of perl, I don't see a reason not to be grateful and
> apply it to the sources.
>
> Thank you once more
> Martin
>
>
> --
> *) perl is mentioned twice in the "R Administration and
> Installation" manual:
> 1. maybe needed for 'install-info' *if* there's no
> 'install-info' command on the system [but on my Fedora and
>probably most "math-y" Linux dist there is a binary]
>
> 2. On Windoze, the texinfo 5.x package needs perl
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] ALTREP wrappers and factors
Hi Kylie,
For your question, I don't think a wrapper can completely solve your
problem. The duplication occurs since your variable y has more than 1
reference number( Please see highlighted), so even you have a wrapper, any
changes on the value of the wrapper still can trigger the duplication.
> .Internal(inspect(y))
> @7fb0ce78c0f0 13 INTSXP g0c0 *[NAM(7)]* matter vector (mode=3, len=3,
> mem=0)
My guess is that *matter:::as.altrep* function assigned the variable *y* to
a local variable so that it increases the reference number. For example:
*This would not cause a duplication*
> > a=c(1,2,3)
> > .Internal(inspect(a))
> @0x2384f530 14 REALSXP g0c3 [NAM(1)] (len=3, tl=0) 1,2,3
> > attr(a,"dim")=c(1,3)
> > .Internal(inspect(a))
> @0x2384f530 14 REALSXP g0c3 [NAM(1),ATT] (len=3, tl=0) 1,2,3
> ATTRIB:
> @0x23864b58 02 LISTSXP g0c0 []
> TAG: @0x044b1a90 01 SYMSXP g1c0 [MARK,LCK,gp=0x4000] "dim"
> (has value)
> @0x2384cb48 13 INTSXP g0c1 [NAM(7)] (len=2, tl=0) 1,3
>
*This would cause a duplication, even though the function test does
nothing.*
> > test<-function(x) x1=x
> > a=c(1,2,3)
> > .Internal(inspect(a))
> @0x2384f260 14 REALSXP g0c3 [NAM(1)] (len=3, tl=0) 1,2,3
> > test(a)
> > .Internal(inspect(a))
> @0x2384f260 14 REALSXP g0c3 [NAM(7)] (len=3, tl=0) 1,2,3
> > attr(a,"dim")=c(1,3)
> > .Internal(inspect(a))
> @0x2384f0d0 14 REALSXP g0c3 [NAM(1),ATT] (len=3, tl=0) 1,2,3
> ATTRIB:
> @0x238666c0 02 LISTSXP g0c0 []
> TAG: @0x044b1a90 01 SYMSXP g1c0 [MARK,LCK,gp=0x4000] "dim"
> (has value)
> @0x2384c6e8 13 INTSXP g0c1 [NAM(7)] (len=2, tl=0) 1,3
>
If that is the case and you are 100% sure the reference number should be 1
for your variable *y*, my solution is to call *SET_NAMED *in C++ to reset
the reference number. Note that you need to unbind your local variable
before you reset the number. To return an unbound SEXP, the C++ function
should be placed at the end of your *matter:::as.altrep *function. I don't
know if there is any simpler way to do that and I'll be happy to see any
opinion.
Also, I notice that you are using ALTREP to create a wrapper for your
*matter_vec *class. I'm an author of AltWrapper package and the package is
able to define an ALTREP in pure R level, it is capable to add an attribute
to ALTREP object when creating the object and has a correct reference
number. The simplest example would be
*CODE*
```
library(AltWrapper)
inspectFunc <- function(x) cat("Altrep object\n")
lengthFunc <- function(x) return(length(x))
getPtrFunc <- function(x, writeable) return(x)
setAltClass(className = "test", classType = "real")
setAltMethod(className = "test", inspect = inspectFunc)
setAltMethod(className = "test", getLength = lengthFunc)
setAltMethod(className = "test", getDataptr = getPtrFunc)
A = runif(6)
A_alt = makeAltrep(className = "test", x = A, *attributes = list(dim = c(2,
3))*)
```
*RESULT*
```
> .Internal(inspect(A_alt))
@0x2385ac00 14 REALSXP g0c0 [NAM(1),ATT] Altrep object
ATTRIB:
@0x2385a8b8 02 LISTSXP g0c0 []
TAG: @0x044b1a90 01 SYMSXP g1c0 [MARK,LCK,gp=0x4000] "dim" (has
value)
@0x2384d590 13 INTSXP g0c1 [NAM(7)] (len=2, tl=0) 2,3
> A_alt
[,1] [,2] [,3]
[1,] 0.9430458 0.548670 0.4148741
[2,] 0.9550899 0.251857 0.6077540
```
I will be happy to talk more about it if you are interested in the package,
it is available at
https://github.com/Jiefei-Wang/AltWrapper
Best,
Jiefei
On Thu, Jul 18, 2019 at 3:28 AM Bemis, Kylie
wrote:
> Hello,
>
> I’m experimenting with ALTREP and was wondering if there is a preferred
> way to create an ALTREP wrapper vector without using
> .Internal(wrap_meta(…)), which R CMD check doesn’t like since it uses an
> .Internal() function.
>
> I was trying to create a factor that used an ALTREP integer, but
> attempting to set the class and levels attributes always ended up
> duplicating and materializing the integer vector. Using the wrapper avoided
> this issue.
>
> Here is my initial ALTREP integer vector:
>
> > fc0 <- factor(c("a", "a", "b"))
> >
> > y <- matter::as.matter(as.integer(fc0))
> > y <- matter:::as.altrep(y)
> >
> > .Internal(inspect(y))
> @7fb0ce78c0f0 13 INTSXP g0c0 [NAM(7)] matter vector (mode=3, len=3, mem=0)
>
> Here is what I get without a wrapper:
>
> > fc1 <- structure(y, class="factor", levels=levels(x))
> > .Internal(inspect(fc1))
> @7fb0cae66408 13 INTSXP g0c2 [OBJ,NAM(2),ATT] (len=3, tl=0) 1,1,2
> ATTRIB:
> @7fb0ce771868 02 LISTSXP g0c0 []
> TAG: @7fb0c80043d0 01 SYMSXP g1c0 [MARK,LCK,gp=0x4000] "class" (has
> value)
> @7fb0c9fcbe90 16 STRSXP g0c1 [NAM(7)] (len=1, tl=0)
> @7fb0c80841a0 09 CHARSXP g1c1 [MARK,gp=0x61] [ASCII] [cached]
> "factor"
> TAG: @7fb0c8004050 01 SYMSXP g1c0 [MARK,NAM(7),LCK,gp=0x4000] "levels"
> (has value)
> @7fb0d1dd58c8 16 STRSXP g0c2 [MARK,NAM(7)] (len=2, tl=0)
> @7fb0c81bf4c0 09 CHARSXP g1c1 [MARK,gp=0x61]
Re: [Rd] ALTREP wrappers and factors
Hi Jiefei and Kylie,
Great to see people engaging with the ALTREP framework and identifying
places we may need more tooling. Comments inline.
On Thu, Jul 18, 2019 at 12:22 PM King Jiefei wrote:
>
> If that is the case and you are 100% sure the reference number should be 1
> for your variable *y*, my solution is to call *SET_NAMED *in C++ to reset
> the reference number. Note that you need to unbind your local variable
> before you reset the number. To return an unbound SEXP, the C++ function
> should be placed at the end of your *matter:::as.altrep *function. I don't
> know if there is any simpler way to do that and I'll be happy to see any
> opinion.
>
So as far as I know, manually setting the NAMED value on any SEXP the
garbage collector is aware of is a direct violation of C-API contract and
not something that package code should ever be doing.
Its not at all clear to me that you can *ever* be 100% sure that the
reference number should be 1 when it is not currently one for an R object
that exists at the R-level (as opposed to only in pure C code). Sure, maybe
the object is created within the body of your R function instead of being
passed in, but what if someone is debugging your function and assigns the
value to the global environment using <<- for later inspection; now you
have an invalidly low NAMED value, ie you have a segfault coming. I know of
no way for you to prevent this or even know it has happened.
> On Thu, Jul 18, 2019 at 3:28 AM Bemis, Kylie
> wrote:
>
> > Hello,
> >
> > I’m experimenting with ALTREP and was wondering if there is a preferred
> > way to create an ALTREP wrapper vector without using
> > .Internal(wrap_meta(…)), which R CMD check doesn’t like since it uses an
> > .Internal() function.
>
So there is the .doSortWrap (and its currently inexplicably identical
clone .doWrap) function in base, which is an R level function that calls
down to .Internal(wrap_meta(...)), which you can use, but it doesn't look
general enough for what I think you need (it was written for things that
have just been sorted, thus the name). Specifically, its not able to
indicate that things are of unknown sortedness as currently written. If
matter vectors are guaranteed to be sorted for some reason, though, you can
use this. I'll talk to Luke about whether we want to generalize this, it
would be easy to have this support the full space of metadata for wrappers
and be a general purpose wrapper-maker, but that isn't what it is right now.
At the C-level, it looks like we do make R_tryWrap available (it appears in
Rinternals.h, and not within a USE_RINTERNALS section),so you can call that
from your own C(++) code. This creates a wrapper that has no metadata on it
(or rather it has metadata but the metadata indicates that no special info
is known about the vector).
>
> > I was trying to create a factor that used an ALTREP integer, but
> > attempting to set the class and levels attributes always ended up
> > duplicating and materializing the integer vector. Using the wrapper
> avoided
> > this issue.
> >
> > Here is my initial ALTREP integer vector:
> >
> > > fc0 <- factor(c("a", "a", "b"))
> > >
> > > y <- matter::as.matter(as.integer(fc0))
> > > y <- matter:::as.altrep(y)
> > >
> > > .Internal(inspect(y))
> > @7fb0ce78c0f0 13 INTSXP g0c0 [NAM(7)] matter vector (mode=3, len=3,
> mem=0)
> >
> > Here is what I get without a wrapper:
> >
> > > fc1 <- structure(y, class="factor", levels=levels(x))
> > > .Internal(inspect(fc1))
> > @7fb0cae66408 13 INTSXP g0c2 [OBJ,NAM(2),ATT] (len=3, tl=0) 1,1,2
> > ATTRIB:
> > @7fb0ce771868 02 LISTSXP g0c0 []
> > TAG: @7fb0c80043d0 01 SYMSXP g1c0 [MARK,LCK,gp=0x4000] "class" (has
> > value)
> > @7fb0c9fcbe90 16 STRSXP g0c1 [NAM(7)] (len=1, tl=0)
> > @7fb0c80841a0 09 CHARSXP g1c1 [MARK,gp=0x61] [ASCII] [cached]
> > "factor"
> > TAG: @7fb0c8004050 01 SYMSXP g1c0 [MARK,NAM(7),LCK,gp=0x4000]
> "levels"
> > (has value)
> > @7fb0d1dd58c8 16 STRSXP g0c2 [MARK,NAM(7)] (len=2, tl=0)
> > @7fb0c81bf4c0 09 CHARSXP g1c1 [MARK,gp=0x61] [ASCII] [cached] "a"
> > @7fb0c90ba728 09 CHARSXP g1c1 [MARK,gp=0x61] [ASCII] [cached] "b"
> >
> > Here is what I get with a wrapper:
> >
> > > fc2 <- structure(.Internal(wrap_meta(y, 0, 0)), class="factor",
> > levels=levels(x))
> > > .Internal(inspect(fc2))
> > @7fb0ce764630 13 INTSXP g0c0 [OBJ,NAM(2),ATT] wrapper [srt=0,no_na=0]
> > @7fb0ce78c0f0 13 INTSXP g0c0 [NAM(7)] matter vector (mode=3, len=3,
> > mem=0)
> > ATTRIB:
> > @7fb0ce764668 02 LISTSXP g0c0 []
> > TAG: @7fb0c80043d0 01 SYMSXP g1c0 [MARK,LCK,gp=0x4000] "class" (has
> > value)
> > @7fb0c9fcb010 16 STRSXP g0c1 [NAM(7)] (len=1, tl=0)
> > @7fb0c80841a0 09 CHARSXP g1c1 [MARK,gp=0x61] [ASCII] [cached]
> > "factor"
> > TAG: @7fb0c8004050 01 SYMSXP g1c0 [MARK,NAM(7),LCK,gp=0x4000]
> "levels"
> > (has value)
> > @7fb0d1dd58c8 16 STRSXP g0c2 [MARK,NAM(7)] (len=2, tl=0)
> > @7fb0c81bf4c0 09 CHARSXP g1c1 [MARK,gp=0x61] [A
