[Rd] S3 lookup rules changed in R 3.6.1
tl;dr: S3 lookup no longer works in custom non-namespace environments as of
R 3.6.1. Is this a bug?
I am implementing S3 dispatch for generic methods in environments that are
not
packages. I am trying to emulate the R package namespace mechanism by
having a
“namespace” environment that defines generics and methods, but only exposes
the
generics themselves, not the methods.
To make S3 lookup work when using the generics, I am using
`registerS3method`.
While this method itself has no extensive documentation, the documentation
of
`UseMethod` contains this relevant passage:
> Namespaces can register methods for generic functions. To support this,
> ‘UseMethod’ and ‘NextMethod’ search for methods in two places: in the
> environment in which the generic function is called, and in the
registration
> data base for the environment in which the generic is defined (typically a
> namespace). So methods for a generic function need to be available in the
> environment of the call to the generic, or they must be registered. (It
does
> not matter whether they are visible in the environment in which the
generic is
> defined.) As from R 3.5.0, the registration data base is searched after
the
> top level environment (see ‘topenv’) of the calling environment (but
before
> the parents of the top level environment).
This used to work but it stopped working in R 3.6.1 and I cannot figure out
(a)
why, and (b) how to fix it. Unfortunately I am unable to find the relevant
information by reading the R source code, even when “diff”ing what seem to
be
the only even remotely relevant changes [1].
The R NEWS merely list the following change for R 3.6.0:
> * S3method() directives in ‘NAMESPACE’ can now also be used to perform
delayed
> S3 method registration.
> […]
> * Method dispatch uses more relevant environments when looking up class
> definitions.
Unfortunately it is not clear to me what exactly this means.
Here’s a minimal example code that works under R 3.5.3 but breaks under
R 3.6.1
(I don’t know about 3.6.0).
```
# Define “package namespace”:
ns = new.env(parent = .BaseNamespaceEnv)
local(envir = ns, {
test = function (x) UseMethod('test')
test.default = function (x) message('test.default')
test.foo = function (x) message('test.foo')
.__S3MethodsTable__. = new.env(parent = .BaseNamespaceEnv)
.__S3MethodsTable__.$test.default = test.default
.__S3MethodsTable__.$test.foo = test.foo
# Or, equivalently:
# registerS3method('test', 'default', test.default)
# registerS3method('test', 'foo', test.foo)
})
# Expose generic publicly:
test = ns$test
# Usage:
test(1)
test(structure(1, class = 'foo'))
```
Output in R up to 3.5.3:
```
test.default
test.foo
```
Output in R 3.6.1:
```
Error in UseMethod("test") :
no applicable method for 'test' applied to an object of class
"c('double', 'numeric')"
```
It’s worth noting that the output of `.S3methods` is the same for all R
versions, and from my understanding of its output, this *should* indicate
that
S3 lookup should behave identically, too. Furthermore, lookup via
`getS3method`
succeeds in all R versions, and (again, in my understanding) the logic of
this
function should be identical to the logic of R’s internal S3 dispatch:
```
getS3method('test', 'default')(1)
getS3method('test', 'foo')(1)
```
Conversely, specialising an existing generic from a loaded package works.
E.g.:
```
local(envir = ns, {
print.foo = function (x) message('print.foo')
registerS3method('print', 'foo', print.foo)
})
print(structure(1, class = 'foo'))
```
This prints “print.foo” in all R versions as expected.
So my question is: Why do the `test(…)` calls in R 3.6.1 no longer trigger
S3
method lookup in the generic function’s environment? Is this behaviour by
design
or is it a bug? If it’s by design, why does `getS3method` still use the old
behaviour? And, most importantly, how can I fix my definition of `ns` to
make
S3 dispatch for non-exposed methods work again?
… actually I just found a workaround:
```
ns$.packageName = 'not important'
```
This marks `ns` as a package namespace. To me, the documentation seems to
imply
that this shouldn’t be necessary (and it previously wasn’t). Furthermore,
the
code for `registerS3method` explicitly supports non-package namespace
environments. Unfortunately this workaround is not satisfactory because
pretending that the environment is a package namespace, when it really
isn’t,
might break other things.
[1] See r75273; there’s also r74625, which changes the actual lookup
mechanism
used by `UseMethod`, but that seems even less relevant, because it is
disabled unless a specific environment variable is set.
--
Konrad Rudolph
[[alternative HTML version deleted]]
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] S3 lookup rules changed in R 3.6.1
On 09/10/2019 3:22 p.m., Konrad Rudolph wrote:
tl;dr: S3 lookup no longer works in custom non-namespace environments as of
R 3.6.1. Is this a bug?
I don't know whether this was intentional or not, but a binary search
through the svn commits finds that the errors started in this one:
r75127 | hornik | 2018-08-13 09:58:47 -0400 (Mon, 13 Aug 2018) | 2 lines
Changed paths:
M /trunk/src/main/objects.c
M /trunk/tests/reg-tests-1a.R
Have S3 methods lookup by default look for the S3 registry in the topenv
of the generic.
Duncan Murdoch
I am implementing S3 dispatch for generic methods in environments that are
not
packages. I am trying to emulate the R package namespace mechanism by
having a
“namespace” environment that defines generics and methods, but only exposes
the
generics themselves, not the methods.
To make S3 lookup work when using the generics, I am using
`registerS3method`.
While this method itself has no extensive documentation, the documentation
of
`UseMethod` contains this relevant passage:
Namespaces can register methods for generic functions. To support this,
‘UseMethod’ and ‘NextMethod’ search for methods in two places: in the
environment in which the generic function is called, and in the
registration
data base for the environment in which the generic is defined (typically a
namespace). So methods for a generic function need to be available in the
environment of the call to the generic, or they must be registered. (It
does
not matter whether they are visible in the environment in which the
generic is
defined.) As from R 3.5.0, the registration data base is searched after
the
top level environment (see ‘topenv’) of the calling environment (but
before
the parents of the top level environment).
This used to work but it stopped working in R 3.6.1 and I cannot figure out
(a)
why, and (b) how to fix it. Unfortunately I am unable to find the relevant
information by reading the R source code, even when “diff”ing what seem to
be
the only even remotely relevant changes [1].
The R NEWS merely list the following change for R 3.6.0:
* S3method() directives in ‘NAMESPACE’ can now also be used to perform
delayed
S3 method registration.
[…]
* Method dispatch uses more relevant environments when looking up class
definitions.
Unfortunately it is not clear to me what exactly this means.
Here’s a minimal example code that works under R 3.5.3 but breaks under
R 3.6.1
(I don’t know about 3.6.0).
```
# Define “package namespace”:
ns = new.env(parent = .BaseNamespaceEnv)
local(envir = ns, {
test = function (x) UseMethod('test')
test.default = function (x) message('test.default')
test.foo = function (x) message('test.foo')
.__S3MethodsTable__. = new.env(parent = .BaseNamespaceEnv)
.__S3MethodsTable__.$test.default = test.default
.__S3MethodsTable__.$test.foo = test.foo
# Or, equivalently:
# registerS3method('test', 'default', test.default)
# registerS3method('test', 'foo', test.foo)
})
# Expose generic publicly:
test = ns$test
# Usage:
test(1)
test(structure(1, class = 'foo'))
```
Output in R up to 3.5.3:
```
test.default
test.foo
```
Output in R 3.6.1:
```
Error in UseMethod("test") :
no applicable method for 'test' applied to an object of class
"c('double', 'numeric')"
```
It’s worth noting that the output of `.S3methods` is the same for all R
versions, and from my understanding of its output, this *should* indicate
that
S3 lookup should behave identically, too. Furthermore, lookup via
`getS3method`
succeeds in all R versions, and (again, in my understanding) the logic of
this
function should be identical to the logic of R’s internal S3 dispatch:
```
getS3method('test', 'default')(1)
getS3method('test', 'foo')(1)
```
Conversely, specialising an existing generic from a loaded package works.
E.g.:
```
local(envir = ns, {
print.foo = function (x) message('print.foo')
registerS3method('print', 'foo', print.foo)
})
print(structure(1, class = 'foo'))
```
This prints “print.foo” in all R versions as expected.
So my question is: Why do the `test(…)` calls in R 3.6.1 no longer trigger
S3
method lookup in the generic function’s environment? Is this behaviour by
design
or is it a bug? If it’s by design, why does `getS3method` still use the old
behaviour? And, most importantly, how can I fix my definition of `ns` to
make
S3 dispatch for non-exposed methods work again?
… actually I just found a workaround:
```
ns$.packageName = 'not important'
```
This marks `ns` as a package namespace. To me, the documentation seems to
imply
that this shouldn’t be necessary (and it previously wasn’t). Furthermore,
the
code for `registerS3method` explicitly supports non-package namespace
environments. Unfortunately this workaround is not satisfactory because
pretending that the enviro
[Rd] R-specific environment variables: Naming convention?
In base R, there are lots of environment variables with either prefix 'R_' or '_R_', e.g. R_ENABLE_JIT and _R_RNG_VERSION_. I always considered R_* variables to be "public" and _R_*_ ones being "internal" but realized I don't have a reference for this. Is this true, or is there another reason? Is the difference between the two kinds documented anywhere? Thank you, Henrik __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
