[Rd] Linking to native routines in other packages
Hello,
I'm currently working on making Rcpp use the feature described here more:
http://cran.r-project.org/doc/manuals/R-exts.html#Linking-to-native-routines-in-other-packages
To give more context, Rcpp has for a long time built what we called "the
Rcpp user library", i.e. a library we could link against user the
linker. We were then producing appropriate linker flag with
Rcpp:::LdFlags(), ...
Now, I'm moving away from this and the intention is that a package using
Rcpp would only have to use
LinkingTo: Rcpp
This sets the -I flag as before to find headers from Rcpp, but this also
now takes advantage of what is described in writing R extensions:
http://cran.r-project.org/doc/manuals/R-exts.html#Linking-to-native-routines-in-other-packages
I'm doing this in a way that, when we are not compiling Rcpp, for
example the "type2name" function is defined in Rcpp's headers as an
inline function that grabs the registered function from Rcpp.
inline const char* type2name(SEXP x){
typedef const char* (*Fun)(SEXP) ;
static Fun fun = GET_(Fun) R_GetCCallable( "Rcpp", "type2name") ;
return fun(x) ;
}
This works great.
Now for the question. The documentation says:
"It must also specify ‘Imports’ or ‘Depends’ of those packages, for they
have to be loaded prior to this one (so the path to their compiled code
has been registered)."
Indeed if I don't have Depends or Imports, the R_init_Rcpp is not
called, and so the function is not registered.
But now if I do Depends: Rcpp or Imports: Rcpp for the sole purpose of
this LinkingTo mechanism, I'm getting
* checking dependencies in R code ... NOTE
Namespace in Imports field not imported from: ‘Rcpp’
All declared Imports should be used.
See the information on DESCRIPTION files in the chapter ‘Creating R
packages’ of the ‘Writing R Extensions’ manual.
It would be simple enough to require of our users that they have
Imports: Rcpp and import(Rcpp) or less in their NAMESPACE, but I was
wondering if we could make this more transparent, i.e. having LinkingTo:
Rcpp mean loading Rcpp.
I'm also curious about this sentence from the doc:
"In the future R may provide some automated tools to simplify exporting
larger numbers of routines."
Is there a draft of something ?
Romain
For details on how we will be using LinkingTo. Please see:
https://github.com/RcppCore/Rcpp/blob/master/inst/include/Rcpp/routines.h
where depending
- when we are compiling Rcpp, we just have a declaration of the
function, which is then defined in any of our .cpp files.
- when we are using Rcpp from another package, we are retrieving the
function
https://github.com/RcppCore/Rcpp/blob/master/src/Rcpp_init.cpp
where the functions are registered with the RCPP_REGISTER macro.
This way of using it moves all the logic to the package exposing its
functions. I find this nicer to use than other tactics I've seen, such
as the sub technique from Matrix ...
--
Romain Francois
Professional R Enthusiast
+33(0) 6 28 91 30 30
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Linking to native routines in other packages
Le 16/11/2013 11:02, Romain Francois a écrit :
Hello,
I'm currently working on making Rcpp use the feature described here more:
http://cran.r-project.org/doc/manuals/R-exts.html#Linking-to-native-routines-in-other-packages
To give more context, Rcpp has for a long time built what we called "the
Rcpp user library", i.e. a library we could link against user the
linker. We were then producing appropriate linker flag with
Rcpp:::LdFlags(), ...
Now, I'm moving away from this and the intention is that a package using
Rcpp would only have to use
LinkingTo: Rcpp
This sets the -I flag as before to find headers from Rcpp, but this also
now takes advantage of what is described in writing R extensions:
http://cran.r-project.org/doc/manuals/R-exts.html#Linking-to-native-routines-in-other-packages
I'm doing this in a way that, when we are not compiling Rcpp, for
example the "type2name" function is defined in Rcpp's headers as an
inline function that grabs the registered function from Rcpp.
inline const char* type2name(SEXP x){
typedef const char* (*Fun)(SEXP) ;
static Fun fun = GET_(Fun) R_GetCCallable( "Rcpp", "type2name") ;
return fun(x) ;
}
This works great.
Now for the question. The documentation says:
"It must also specify ‘Imports’ or ‘Depends’ of those packages, for they
have to be loaded prior to this one (so the path to their compiled code
has been registered)."
Indeed if I don't have Depends or Imports, the R_init_Rcpp is not
called, and so the function is not registered.
But now if I do Depends: Rcpp or Imports: Rcpp for the sole purpose of
this LinkingTo mechanism, I'm getting
* checking dependencies in R code ... NOTE
Namespace in Imports field not imported from: ‘Rcpp’
All declared Imports should be used.
See the information on DESCRIPTION files in the chapter ‘Creating R
packages’ of the ‘Writing R Extensions’ manual.
It would be simple enough to require of our users that they have
Imports: Rcpp and import(Rcpp) or less in their NAMESPACE, but I was
wondering if we could make this more transparent, i.e. having LinkingTo:
Rcpp mean loading Rcpp.
I'm also curious about this sentence from the doc:
"In the future R may provide some automated tools to simplify exporting
larger numbers of routines."
Is there a draft of something ?
Romain
For details on how we will be using LinkingTo. Please see:
https://github.com/RcppCore/Rcpp/blob/master/inst/include/Rcpp/routines.h
where depending
- when we are compiling Rcpp, we just have a declaration of the
function, which is then defined in any of our .cpp files.
- when we are using Rcpp from another package, we are retrieving the
function
https://github.com/RcppCore/Rcpp/blob/master/src/Rcpp_init.cpp
where the functions are registered with the RCPP_REGISTER macro.
This way of using it moves all the logic to the package exposing its
functions. I find this nicer to use than other tactics I've seen, such
as the sub technique from Matrix ...
Typo alert. Of course here I meant the "stub" technique.
Romain
--
Romain Francois
Professional R Enthusiast
+33(0) 6 28 91 30 30
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] serialization for external pointers
Hello,
Are there any recipe to handle serialization / deserialization of
external pointers.
I'm thinking about something similar in spirit to the way we handle
finalization of external pointers.
Currently, if we create an external pointer, save the session, quit R,
then load the session, we get a null pointer.
One way I'm thinking of is to have an environment in the "protected"
part of the xp, then have an active binding there, since apparently
active bindings:
- are "get" during serialization
- lose their active ness when reloaded:
$ R
[...]
> f <- local( {
+ x <- 1
+ function(v) {
+if (missing(v))
+cat("get\n")
+else {
+cat("set\n")
+x <<- v
+}
+x
+ }
+ })
> makeActiveBinding("fred", f, .GlobalEnv)
> bindingIsActive("fred", .GlobalEnv)
[1] TRUE
>
> q("yes")
get
get
romain@naxos /tmp $ R
[..]
> fred
[1] 1
> bindingIsActive("fred", .GlobalEnv)
[1] FALSE
Is this possible ? Is there any other hook to handle serialization,
unserialization of external pointers ?
Romain
--
Romain Francois
Professional R Enthusiast
+33(0) 6 28 91 30 30
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] serialization for external pointers
Le 16/11/2013 14:30, Romain Francois a écrit :
Hello,
Are there any recipe to handle serialization / deserialization of
external pointers.
I'm thinking about something similar in spirit to the way we handle
finalization of external pointers.
Currently, if we create an external pointer, save the session, quit R,
then load the session, we get a null pointer.
One way I'm thinking of is to have an environment in the "protected"
part of the xp, then have an active binding there, since apparently
active bindings:
- are "get" during serialization
- lose their active ness when reloaded:
This will not work. Apparently the active feature is kept on other
environments:
$ R
[...]
> f <- local( {
+ x <- 1
+ function(v) {
+if (missing(v))
+cat("get\n")
+else {
+cat("set\n")
+x <<- v
+}
+x
+ }
+ })
> makeActiveBinding("fred", f, .GlobalEnv)
> bindingIsActive("fred", .GlobalEnv)
[1] TRUE
>
> e <- new.env()
> makeActiveBinding("fred", f, e)
> bindingIsActive("fred", e)
[1] TRUE
>
> q()
Save workspace image? [y/n/c]: y
get
get
Then:
$ R
[...]
> e
> bindingIsActive("fred", .GlobalEnv)
[1] FALSE
> bindingIsActive("fred", e)
[1] TRUE
Is this the expected behavior ?
--
Romain Francois
Professional R Enthusiast
+33(0) 6 28 91 30 30
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] serialization for external pointers
On Nov 16, 2013, at 8:30 AM, Romain Francois wrote:
> Hello,
>
> Are there any recipe to handle serialization / deserialization of external
> pointers.
>
> I'm thinking about something similar in spirit to the way we handle
> finalization of external pointers.
>
See refhook in serialize/unserialize.
> Currently, if we create an external pointer, save the session, quit R, then
> load the session, we get a null pointer.
>
Saving the session is a different story than serialization, because it also
involves loading the right code in addition to the data and you cannot use
refhooks. I recall discussing this with Luke a few years ago, and the main
problem is that R has no way of knowing what is needed to call the particular
hooks, because the package is not even loaded, so it cannot install hooks.
AFAIR the net result was that the proper way to do it is to handle the NULL
pointer code when you first access the pointer to restore the object based on
additional info that you store in the object. E.g. , in rJava we store the Java
serialization of the object so it can be restored on first use of the new
session.
One piece that is still missing it the ability to set a hook to update the save
object on save() - since we don’t necessarily want to add the extra information
every time the object is created or the serialization may become stale over
time. That would still be very useful to add …
Cheers,
Simon
> One way I'm thinking of is to have an environment in the "protected" part of
> the xp, then have an active binding there, since apparently active bindings:
> - are "get" during serialization
> - lose their active ness when reloaded:
>
> $ R
> [...]
> > f <- local( {
> + x <- 1
> + function(v) {
> +if (missing(v))
> +cat("get\n")
> +else {
> +cat("set\n")
> +x <<- v
> +}
> +x
> + }
> + })
> > makeActiveBinding("fred", f, .GlobalEnv)
> > bindingIsActive("fred", .GlobalEnv)
> [1] TRUE
> >
> > q("yes")
> get
> get
>
>
> romain@naxos /tmp $ R
> [..]
> > fred
> [1] 1
> > bindingIsActive("fred", .GlobalEnv)
> [1] FALSE
>
> Is this possible ? Is there any other hook to handle serialization,
> unserialization of external pointers ?
>
> Romain
>
> --
> Romain Francois
> Professional R Enthusiast
> +33(0) 6 28 91 30 30
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
