[Rd] S3 dispatch does not work for generics defined inside an environment
Dear all,
I have a generic function and a bunch of methods defined in a separate
environment. Here is a reduced example:
env <- local({
# define the generic function and the method
myfun <- function(x) UseMethod("myfun")
myfun.myclass <- function(x) print("called myfun.myclass”)
# register the method
.S3method("myfun", "myclass", myfun.myclass)
environment()
})
Since the method has been registered, I hoped that invocation like this would
work:
env$myfun(structure(0, class = "myclass”))
However, this results in a “no applicable method" error.
It is my understanding that registerS3method (called by .S3method) will install
the method string in the .__S3MethodsTable__. table of the environment where
the generic function is defined, and this table is subsequently used by
usemethod() inside R, so I am puzzled that the dispatch does not work. I
checked and the .__S3MethodsTable__. of env is indeed setup correctly. I also
tried manually adding the method string to the global .__S3MethodsTable__.
inside .BaseNamespaceEnv to no effect.
In fact, the only way to make it work is to define either myfun or
myfun.myclas in the global environment, which is something I would like to
avoid.
Thank you in advance for any pointers!
Best,
Taras
P.S. If you are wondering what I am trying to achieve here — we have a very
large codebase and I am trying to use environments as a type of “poor man’s
namespaces” to organize code in a modular fashion. But of course it’s all
pointless if I can’t get the generics to work reliably.
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] S3 dispatch does not work for generics defined inside an environment
On 30/06/2021 5:22 a.m., Taras Zakharko wrote:
Dear all,
I have a generic function and a bunch of methods defined in a separate
environment. Here is a reduced example:
env <- local({
# define the generic function and the method
myfun <- function(x) UseMethod("myfun")
myfun.myclass <- function(x) print("called myfun.myclass”)
# register the method
.S3method("myfun", "myclass", myfun.myclass)
environment()
})
Since the method has been registered, I hoped that invocation like this would
work:
env$myfun(structure(0, class = "myclass”))
However, this results in a “no applicable method" error.
It is my understanding that registerS3method (called by .S3method) will install
the method string in the .__S3MethodsTable__. table of the environment where
the generic function is defined, and this table is subsequently used by
usemethod() inside R, so I am puzzled that the dispatch does not work. I
checked and the .__S3MethodsTable__. of env is indeed setup correctly. I also
tried manually adding the method string to the global .__S3MethodsTable__.
inside .BaseNamespaceEnv to no effect.
In fact, the only way to make it work is to define either myfun or
myfun.myclas in the global environment, which is something I would like to
avoid.
Thank you in advance for any pointers!
registerS3method has an additional parameter "envir" which I believe
would end up set to env in your code. So this works:
> eval(expression(myfun(structure(0, class = "myclass"))), envir = env)
[1] "called myfun.myclass"
You could probably also call registerS3method with envir specified
appropriately and get your original expression to work.
Duncan Murdoch
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] S3 dispatch does not work for generics defined inside an environment
On Wed, Jun 30, 2021 at 5:17 AM Duncan Murdoch wrote:
>
> On 30/06/2021 5:22 a.m., Taras Zakharko wrote:
> > Dear all,
> >
> > I have a generic function and a bunch of methods defined in a separate
> > environment. Here is a reduced example:
> >
> > env <- local({
> > # define the generic function and the method
> > myfun <- function(x) UseMethod("myfun")
> > myfun.myclass <- function(x) print("called myfun.myclass”)
> >
> > # register the method
> > .S3method("myfun", "myclass", myfun.myclass)
> >
> > environment()
> >})
> >
> > Since the method has been registered, I hoped that invocation like this
> > would work:
> >
> > env$myfun(structure(0, class = "myclass”))
> >
> > However, this results in a “no applicable method" error.
> >
> > It is my understanding that registerS3method (called by .S3method) will
> > install the method string in the .__S3MethodsTable__. table of the
> > environment where the generic function is defined, and this table is
> > subsequently used by usemethod() inside R, so I am puzzled that the
> > dispatch does not work. I checked and the .__S3MethodsTable__. of env is
> > indeed setup correctly. I also tried manually adding the method string to
> > the global .__S3MethodsTable__. inside .BaseNamespaceEnv to no effect.
> >
> > In fact, the only way to make it work is to define either myfun or
> > myfun.myclas in the global environment, which is something I would like to
> > avoid.
> >
> > Thank you in advance for any pointers!
> >
>
> registerS3method has an additional parameter "envir" which I believe
> would end up set to env in your code. So this works:
>
> > eval(expression(myfun(structure(0, class = "myclass"))), envir = env)
> [1] "called myfun.myclass"
>
> You could probably also call registerS3method with envir specified
> appropriately and get your original expression to work.
>
That doesn't seem to work on 4.1.0 for me. The code below worked for
me in Oct-2020, though I'm not sure what version of R I was using at
the time. I was slow to upgrade to 4.0, so it was probably the latest
3.x version.
env <- new.env()
local({
# define the generic function and the method
myfun <- function(x) { UseMethod("myfun", x) }
# register the method
registerS3method("myfun", "myclass",
function(x) { print("called myfun.myclass") },
envir = env)
}, envir = env)
attach(env)
myfun(structure(0, class = "myclass"))
> Duncan Murdoch
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] S3 dispatch does not work for generics defined inside an environment
@Duncan: .S3method() calls registerS3method() with appropriate environmental
argument under the good, so that’s not the problem.
Anyway, I’ve been doing some debugging and I think I have found the issue. The
following snippet in src/objects.c
(https://github.com/wch/r-source/blob/ecc633b37d77fdd1cb27dda74d7f6b3684f0c01c/src/main/objects.c#L525)
sets the global lookup_use_topenv_as_defenv variable:
if(lookup_use_topenv_as_defenv == -1) {
lookup = getenv("_R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_");
lookup_use_topenv_as_defenv =
((lookup != NULL) && StringFalse(lookup)) ? 0 : 1;
}
Isn’t that supposed to be
lookup_use_topenv_as_defenv = ((lookup != NULL) &&
StringFalse(lookup)) ? 1 : 0;
instead?
The way the code works right now, methods will be looked up in top environment
exactly if _R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_ is not set. This seems
incompatible with what registerS3method() does (setting the
.__S3MethodsTable__. on the defining environment instead of the topenv). When I
change 0 and 1 around, everything works as expected.
In the meantime, I can work around it by manually injecting __S3MethodsTable__
into .GlobalEnv (which is my topenv here).
I can open a bug report, but I would like to wait for some more comments.
Best,
Taras
> On 30 Jun 2021, at 12:39, Joshua Ulrich wrote:
>
> On Wed, Jun 30, 2021 at 5:17 AM Duncan Murdoch
> wrote:
>>
>> On 30/06/2021 5:22 a.m., Taras Zakharko wrote:
>>> Dear all,
>>>
>>> I have a generic function and a bunch of methods defined in a separate
>>> environment. Here is a reduced example:
>>>
>>>env <- local({
>>> # define the generic function and the method
>>> myfun <- function(x) UseMethod("myfun")
>>> myfun.myclass <- function(x) print("called myfun.myclass”)
>>>
>>> # register the method
>>> .S3method("myfun", "myclass", myfun.myclass)
>>>
>>> environment()
>>> })
>>>
>>> Since the method has been registered, I hoped that invocation like this
>>> would work:
>>>
>>>env$myfun(structure(0, class = "myclass”))
>>>
>>> However, this results in a “no applicable method" error.
>>>
>>> It is my understanding that registerS3method (called by .S3method) will
>>> install the method string in the .__S3MethodsTable__. table of the
>>> environment where the generic function is defined, and this table is
>>> subsequently used by usemethod() inside R, so I am puzzled that the
>>> dispatch does not work. I checked and the .__S3MethodsTable__. of env is
>>> indeed setup correctly. I also tried manually adding the method string to
>>> the global .__S3MethodsTable__. inside .BaseNamespaceEnv to no effect.
>>>
>>> In fact, the only way to make it work is to define either myfun or
>>> myfun.myclas in the global environment, which is something I would like to
>>> avoid.
>>>
>>> Thank you in advance for any pointers!
>>>
>>
>> registerS3method has an additional parameter "envir" which I believe
>> would end up set to env in your code. So this works:
>>
>>> eval(expression(myfun(structure(0, class = "myclass"))), envir = env)
>> [1] "called myfun.myclass"
>>
>> You could probably also call registerS3method with envir specified
>> appropriately and get your original expression to work.
>>
> That doesn't seem to work on 4.1.0 for me. The code below worked for
> me in Oct-2020, though I'm not sure what version of R I was using at
> the time. I was slow to upgrade to 4.0, so it was probably the latest
> 3.x version.
>
> env <- new.env()
> local({
> # define the generic function and the method
> myfun <- function(x) { UseMethod("myfun", x) }
>
> # register the method
> registerS3method("myfun", "myclass",
> function(x) { print("called myfun.myclass") },
> envir = env)
> }, envir = env)
> attach(env)
> myfun(structure(0, class = "myclass"))
>
>
>> Duncan Murdoch
>>
>> __
>> R-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>
>
>
> --
> Joshua Ulrich | about.me/joshuaulrich
> FOSS Trading | www.fosstrading.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] S3 dispatch does not work for generics defined inside an environment
I had another glance at the code and now I’m convinced that this is the bug in
registerS3method(). Default R behavior (in objects.c) appears to be to look for
method definitions in the top environment, not the defining environment, but
registerS3method() registers the method in the defining environment. I think
registerS3method() should be changed to something like:
- if (typeof(genfun) == "closure”)
-environment(genfun)
+ if (typeof(genfun) == "closure”)
+ if(isFALSE(Sys.getenv("_R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_”))
environment(genfun) else topenv(environment(genfun))
in order to match the behavior of do_usemethod()
Best,
Taras
> On 30 Jun 2021, at 12:51, Taras Zakharko wrote:
>
> @Duncan: .S3method() calls registerS3method() with appropriate environmental
> argument under the good, so that’s not the problem.
>
> Anyway, I’ve been doing some debugging and I think I have found the issue.
> The following snippet in src/objects.c
> (https://github.com/wch/r-source/blob/ecc633b37d77fdd1cb27dda74d7f6b3684f0c01c/src/main/objects.c#L525)
> sets the global lookup_use_topenv_as_defenv variable:
>
>
> if(lookup_use_topenv_as_defenv == -1) {
> lookup = getenv("_R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_");
> lookup_use_topenv_as_defenv =
> ((lookup != NULL) && StringFalse(lookup)) ? 0 : 1;
> }
>
> Isn’t that supposed to be
>
> lookup_use_topenv_as_defenv = ((lookup != NULL) &&
> StringFalse(lookup)) ? 1 : 0;
>
> instead?
>
> The way the code works right now, methods will be looked up in top
> environment exactly if _R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_ is not set.
> This seems incompatible with what registerS3method() does (setting the
> .__S3MethodsTable__. on the defining environment instead of the topenv). When
> I change 0 and 1 around, everything works as expected.
>
> In the meantime, I can work around it by manually injecting
> __S3MethodsTable__ into .GlobalEnv (which is my topenv here).
>
> I can open a bug report, but I would like to wait for some more comments.
>
> Best,
>
> Taras
>
>> On 30 Jun 2021, at 12:39, Joshua Ulrich wrote:
>>
>> On Wed, Jun 30, 2021 at 5:17 AM Duncan Murdoch
>> wrote:
>>>
>>> On 30/06/2021 5:22 a.m., Taras Zakharko wrote:
Dear all,
I have a generic function and a bunch of methods defined in a separate
environment. Here is a reduced example:
env <- local({
# define the generic function and the method
myfun <- function(x) UseMethod("myfun")
myfun.myclass <- function(x) print("called myfun.myclass”)
# register the method
.S3method("myfun", "myclass", myfun.myclass)
environment()
})
Since the method has been registered, I hoped that invocation like this
would work:
env$myfun(structure(0, class = "myclass”))
However, this results in a “no applicable method" error.
It is my understanding that registerS3method (called by .S3method) will
install the method string in the .__S3MethodsTable__. table of the
environment where the generic function is defined, and this table is
subsequently used by usemethod() inside R, so I am puzzled that the
dispatch does not work. I checked and the .__S3MethodsTable__. of env is
indeed setup correctly. I also tried manually adding the method string to
the global .__S3MethodsTable__. inside .BaseNamespaceEnv to no effect.
In fact, the only way to make it work is to define either myfun or
myfun.myclas in the global environment, which is something I would like to
avoid.
Thank you in advance for any pointers!
>>>
>>> registerS3method has an additional parameter "envir" which I believe
>>> would end up set to env in your code. So this works:
>>>
eval(expression(myfun(structure(0, class = "myclass"))), envir = env)
>>> [1] "called myfun.myclass"
>>>
>>> You could probably also call registerS3method with envir specified
>>> appropriately and get your original expression to work.
>>>
>> That doesn't seem to work on 4.1.0 for me. The code below worked for
>> me in Oct-2020, though I'm not sure what version of R I was using at
>> the time. I was slow to upgrade to 4.0, so it was probably the latest
>> 3.x version.
>>
>> env <- new.env()
>> local({
>> # define the generic function and the method
>> myfun <- function(x) { UseMethod("myfun", x) }
>>
>> # register the method
>> registerS3method("myfun", "myclass",
>> function(x) { print("called myfun.myclass") },
>> envir = env)
>> }, envir = env)
>> attach(env)
>> myfun(structure(0, class = "myclass"))
>>
>>
>>> Duncan Murdoch
>>>
>>> __
>>> R-devel@r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>>
>>
>> --
>> Joshua Ulrich | about.me/joshuaulrich
>> FOSS Trading
Re: [Rd] S3 dispatch does not work for generics defined inside an environment
On 30/06/2021 6:51 a.m., Taras Zakharko wrote:
@Duncan: .S3method() calls registerS3method() with appropriate environmental
argument under the good, so that’s not the problem.
Anyway, I’ve been doing some debugging and I think I have found the issue. The
following snippet in src/objects.c
(https://github.com/wch/r-source/blob/ecc633b37d77fdd1cb27dda74d7f6b3684f0c01c/src/main/objects.c#L525)
sets the global lookup_use_topenv_as_defenv variable:
if(lookup_use_topenv_as_defenv == -1) {
lookup = getenv("_R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_");
lookup_use_topenv_as_defenv =
((lookup != NULL) && StringFalse(lookup)) ? 0 : 1;
}
Isn’t that supposed to be
lookup_use_topenv_as_defenv = ((lookup != NULL) &&
StringFalse(lookup)) ? 1 : 0;
instead?
Surely not. That would set the result to 1 if the user had
_R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_=FALSE. The default for that
env variable in the R code in src/library/utils/objects.R is explicitly
TRUE, and it should surely be treated the same in C code.
The way the code works right now, methods will be looked up in top environment exactly if _R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_ is not set.
No, the StringFalse() call tries to determine if the string is set to
something that looks like FALSE. So it will be 1 if not set or set but
not "FALSE" (or equivalent), which is the same thing as saying the
default for that env var should be treated as TRUE.
Duncan Murdoch
This seems incompatible with what registerS3method() does (setting the
.__S3MethodsTable__. on the defining environment instead of the topenv).
When I change 0 and 1 around, everything works as expected.
In the meantime, I can work around it by manually injecting __S3MethodsTable__
into .GlobalEnv (which is my topenv here).
I can open a bug report, but I would like to wait for some more comments.
Best,
Taras
On 30 Jun 2021, at 12:39, Joshua Ulrich wrote:
On Wed, Jun 30, 2021 at 5:17 AM Duncan Murdoch wrote:
On 30/06/2021 5:22 a.m., Taras Zakharko wrote:
Dear all,
I have a generic function and a bunch of methods defined in a separate
environment. Here is a reduced example:
env <- local({
# define the generic function and the method
myfun <- function(x) UseMethod("myfun")
myfun.myclass <- function(x) print("called myfun.myclass”)
# register the method
.S3method("myfun", "myclass", myfun.myclass)
environment()
})
Since the method has been registered, I hoped that invocation like this would
work:
env$myfun(structure(0, class = "myclass”))
However, this results in a “no applicable method" error.
It is my understanding that registerS3method (called by .S3method) will install
the method string in the .__S3MethodsTable__. table of the environment where
the generic function is defined, and this table is subsequently used by
usemethod() inside R, so I am puzzled that the dispatch does not work. I
checked and the .__S3MethodsTable__. of env is indeed setup correctly. I also
tried manually adding the method string to the global .__S3MethodsTable__.
inside .BaseNamespaceEnv to no effect.
In fact, the only way to make it work is to define either myfun or
myfun.myclas in the global environment, which is something I would like to
avoid.
Thank you in advance for any pointers!
registerS3method has an additional parameter "envir" which I believe
would end up set to env in your code. So this works:
eval(expression(myfun(structure(0, class = "myclass"))), envir = env)
[1] "called myfun.myclass"
You could probably also call registerS3method with envir specified
appropriately and get your original expression to work.
That doesn't seem to work on 4.1.0 for me. The code below worked for
me in Oct-2020, though I'm not sure what version of R I was using at
the time. I was slow to upgrade to 4.0, so it was probably the latest
3.x version.
env <- new.env()
local({
# define the generic function and the method
myfun <- function(x) { UseMethod("myfun", x) }
# register the method
registerS3method("myfun", "myclass",
function(x) { print("called myfun.myclass") },
envir = env)
}, envir = env)
attach(env)
myfun(structure(0, class = "myclass"))
Duncan Murdoch
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] S3 dispatch does not work for generics defined inside an environment
On 30/06/2021 7:23 a.m., Taras Zakharko wrote:
I had another glance at the code and now I’m convinced that this is the bug in
registerS3method(). Default R behavior (in objects.c) appears to be to look for
method definitions in the top environment, not the defining environment, but
registerS3method() registers the method in the defining environment. I think
registerS3method() should be changed to something like:
It registers wherever you ask it to. The default is the defining
environment.
.S3method is the one that always uses the defining environment, since it
has no way to override the default.
Duncan Murdoch
- if (typeof(genfun) == "closure”)
-environment(genfun)
+ if (typeof(genfun) == "closure”)
+ if(isFALSE(Sys.getenv("_R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_”))
environment(genfun) else topenv(environment(genfun))
in order to match the behavior of do_usemethod()
Best,
Taras
On 30 Jun 2021, at 12:51, Taras Zakharko wrote:
@Duncan: .S3method() calls registerS3method() with appropriate environmental
argument under the good, so that’s not the problem.
Anyway, I’ve been doing some debugging and I think I have found the issue. The
following snippet in src/objects.c
(https://github.com/wch/r-source/blob/ecc633b37d77fdd1cb27dda74d7f6b3684f0c01c/src/main/objects.c#L525)
sets the global lookup_use_topenv_as_defenv variable:
if(lookup_use_topenv_as_defenv == -1) {
lookup = getenv("_R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_");
lookup_use_topenv_as_defenv =
((lookup != NULL) && StringFalse(lookup)) ? 0 : 1;
}
Isn’t that supposed to be
lookup_use_topenv_as_defenv = ((lookup != NULL) &&
StringFalse(lookup)) ? 1 : 0;
instead?
The way the code works right now, methods will be looked up in top environment
exactly if _R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_ is not set. This seems
incompatible with what registerS3method() does (setting the
.__S3MethodsTable__. on the defining environment instead of the topenv). When I
change 0 and 1 around, everything works as expected.
In the meantime, I can work around it by manually injecting __S3MethodsTable__
into .GlobalEnv (which is my topenv here).
I can open a bug report, but I would like to wait for some more comments.
Best,
Taras
On 30 Jun 2021, at 12:39, Joshua Ulrich wrote:
On Wed, Jun 30, 2021 at 5:17 AM Duncan Murdoch wrote:
On 30/06/2021 5:22 a.m., Taras Zakharko wrote:
Dear all,
I have a generic function and a bunch of methods defined in a separate
environment. Here is a reduced example:
env <- local({
# define the generic function and the method
myfun <- function(x) UseMethod("myfun")
myfun.myclass <- function(x) print("called myfun.myclass”)
# register the method
.S3method("myfun", "myclass", myfun.myclass)
environment()
})
Since the method has been registered, I hoped that invocation like this would
work:
env$myfun(structure(0, class = "myclass”))
However, this results in a “no applicable method" error.
It is my understanding that registerS3method (called by .S3method) will install
the method string in the .__S3MethodsTable__. table of the environment where
the generic function is defined, and this table is subsequently used by
usemethod() inside R, so I am puzzled that the dispatch does not work. I
checked and the .__S3MethodsTable__. of env is indeed setup correctly. I also
tried manually adding the method string to the global .__S3MethodsTable__.
inside .BaseNamespaceEnv to no effect.
In fact, the only way to make it work is to define either myfun or
myfun.myclas in the global environment, which is something I would like to
avoid.
Thank you in advance for any pointers!
registerS3method has an additional parameter "envir" which I believe
would end up set to env in your code. So this works:
eval(expression(myfun(structure(0, class = "myclass"))), envir = env)
[1] "called myfun.myclass"
You could probably also call registerS3method with envir specified
appropriately and get your original expression to work.
That doesn't seem to work on 4.1.0 for me. The code below worked for
me in Oct-2020, though I'm not sure what version of R I was using at
the time. I was slow to upgrade to 4.0, so it was probably the latest
3.x version.
env <- new.env()
local({
# define the generic function and the method
myfun <- function(x) { UseMethod("myfun", x) }
# register the method
registerS3method("myfun", "myclass",
function(x) { print("called myfun.myclass") },
envir = env)
}, envir = env)
attach(env)
myfun(structure(0, class = "myclass"))
Duncan Murdoch
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailma
Re: [Rd] S3 dispatch does not work for generics defined inside an environment
Thats not how I read the code? Consider this snippet from registerS3method:
genfun <- get(genname, envir = envir)
if (.isMethodsDispatchOn() && methods::is(genfun, "genericFunction"))
genfun <- methods::finalDefaultMethod(genfun@default)
if (typeof(genfun) == "closure")
environment(genfun)
else .BaseNamespaceEnv
This is the environment where the method cache (.__S3MethodsTable__.) will be
updated. The problem hoverer, the with the default setting of
_R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_ this is *not* the environment where R
will look for the method cache. Manually injecting the entry into the method
cache of the top environment instead works like a charm.
The envir argument is just for looking up the generic function.
Best,
Taras
> On 30 Jun 2021, at 13:29, Duncan Murdoch wrote:
>
> On 30/06/2021 7:23 a.m., Taras Zakharko wrote:
>> I had another glance at the code and now I’m convinced that this is the bug
>> in registerS3method(). Default R behavior (in objects.c) appears to be to
>> look for method definitions in the top environment, not the defining
>> environment, but registerS3method() registers the method in the defining
>> environment. I think registerS3method() should be changed to something like:
>
> It registers wherever you ask it to. The default is the defining environment.
>
> .S3method is the one that always uses the defining environment, since it has
> no way to override the default.
>
> Duncan Murdoch
>
>> - if (typeof(genfun) == "closure”)
>> -environment(genfun)
>>
>> + if (typeof(genfun) == "closure”)
>> + if(isFALSE(Sys.getenv("_R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_”))
>> environment(genfun) else topenv(environment(genfun))
>> in order to match the behavior of do_usemethod()
>> Best,
>> Taras
>>> On 30 Jun 2021, at 12:51, Taras Zakharko wrote:
>>>
>>> @Duncan: .S3method() calls registerS3method() with appropriate
>>> environmental argument under the good, so that’s not the problem.
>>>
>>> Anyway, I’ve been doing some debugging and I think I have found the issue.
>>> The following snippet in src/objects.c
>>> (https://github.com/wch/r-source/blob/ecc633b37d77fdd1cb27dda74d7f6b3684f0c01c/src/main/objects.c#L525)
>>> sets the global lookup_use_topenv_as_defenv variable:
>>>
>>>
>>> if(lookup_use_topenv_as_defenv == -1) {
>>> lookup = getenv("_R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_");
>>> lookup_use_topenv_as_defenv =
>>> ((lookup != NULL) && StringFalse(lookup)) ? 0 : 1;
>>> }
>>>
>>> Isn’t that supposed to be
>>>
>>> lookup_use_topenv_as_defenv = ((lookup != NULL) &&
>>> StringFalse(lookup)) ? 1 : 0;
>>>
>>> instead?
>>>
>>> The way the code works right now, methods will be looked up in top
>>> environment exactly if _R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_ is not
>>> set. This seems incompatible with what registerS3method() does (setting the
>>> .__S3MethodsTable__. on the defining environment instead of the topenv).
>>> When I change 0 and 1 around, everything works as expected.
>>>
>>> In the meantime, I can work around it by manually injecting
>>> __S3MethodsTable__ into .GlobalEnv (which is my topenv here).
>>>
>>> I can open a bug report, but I would like to wait for some more comments.
>>>
>>> Best,
>>>
>>> Taras
>>>
On 30 Jun 2021, at 12:39, Joshua Ulrich wrote:
On Wed, Jun 30, 2021 at 5:17 AM Duncan Murdoch
wrote:
>
> On 30/06/2021 5:22 a.m., Taras Zakharko wrote:
>> Dear all,
>>
>> I have a generic function and a bunch of methods defined in a separate
>> environment. Here is a reduced example:
>>
>> env <- local({
>> # define the generic function and the method
>> myfun <- function(x) UseMethod("myfun")
>> myfun.myclass <- function(x) print("called myfun.myclass”)
>>
>> # register the method
>> .S3method("myfun", "myclass", myfun.myclass)
>>
>> environment()
>> })
>>
>> Since the method has been registered, I hoped that invocation like this
>> would work:
>>
>> env$myfun(structure(0, class = "myclass”))
>>
>> However, this results in a “no applicable method" error.
>>
>> It is my understanding that registerS3method (called by .S3method) will
>> install the method string in the .__S3MethodsTable__. table of the
>> environment where the generic function is defined, and this table is
>> subsequently used by usemethod() inside R, so I am puzzled that the
>> dispatch does not work. I checked and the .__S3MethodsTable__. of env
>> is indeed setup correctly. I also tried manually adding the method
>> string to the global .__S3MethodsTable__. inside .BaseNamespaceEnv to no
>> effect.
>>
>> In fact, the only way to make it work is to define either myfun or
>> myfun.myclas in the global environment, which is somethi
Re: [Rd] S3 dispatch does not work for generics defined inside an environment
On 30/06/2021 7:37 a.m., Taras Zakharko wrote:
Thats not how I read the code? Consider this snippet from registerS3method:
genfun <- get(genname, envir = envir)
if (.isMethodsDispatchOn() && methods::is(genfun, "genericFunction"))
genfun <- methods::finalDefaultMethod(genfun@default)
if (typeof(genfun) == "closure")
environment(genfun)
else .BaseNamespaceEnv
This is the environment where the method cache (.__S3MethodsTable__.) will be
updated. The problem hoverer, the with the default setting of
_R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_ this is *not* the environment where R
will look for the method cache. Manually injecting the entry into the method
cache of the top environment instead works like a charm.
The envir argument is just for looking up the generic function.
Yes, I think you're right: the method is registered in the environment
of the generic. I think by default it is searched for in the topenv()
of that environment.
In most cases those are the same thing, but in your code they aren't the
same: the topenv is the global environment.
So I think I'd agree this is a bug. I'd say the bug is in
registerS3method, which should pay attention to the
_R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_ setting when it computes defenv.
Here's a workaround that copies methods to the appropriate place:
fixS3registrations <- function(genericname, envir = parent.frame()) {
generic <- get(genericname, envir=envir)
genenv <- environment(generic)
top <- topenv(genenv)
if (!identical(genenv, top)) {
toptable <- top[[".__S3MethodsTable__."]]
if (is.null(toptable)) {
toptable <- new.env(hash = TRUE, parent = baseenv())
top[[".__S3MethodsTable__."]] <- toptable
}
table <- genenv[[".__S3MethodsTable__."]]
if (!is.null(table)) {
methodnames <- ls(table, pattern = paste0(genericname, "[.][^.]*$"))
for (m in methodnames)
toptable[[m]] <- table[[m]]
}
}
}
env <- local({
# define the generic function and the method
myfun <- function(x) UseMethod("myfun")
myfun.myclass <- function(x) print("called myfun.myclass")
# register the method
.S3method("myfun", "myclass", myfun.myclass)
fixS3registrations("myfun")
environment()
})
env$myfun(structure(0, class = "myclass"))
#> [1] "called myfun.myclass"
Duncan Murdoch
Best,
Taras
On 30 Jun 2021, at 13:29, Duncan Murdoch wrote:
On 30/06/2021 7:23 a.m., Taras Zakharko wrote:
I had another glance at the code and now I’m convinced that this is the bug in
registerS3method(). Default R behavior (in objects.c) appears to be to look for
method definitions in the top environment, not the defining environment, but
registerS3method() registers the method in the defining environment. I think
registerS3method() should be changed to something like:
It registers wherever you ask it to. The default is the defining environment.
.S3method is the one that always uses the defining environment, since it has no
way to override the default.
Duncan Murdoch
- if (typeof(genfun) == "closure”)
-environment(genfun)
+ if (typeof(genfun) == "closure”)
+ if(isFALSE(Sys.getenv("_R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_”))
environment(genfun) else topenv(environment(genfun))
in order to match the behavior of do_usemethod()
Best,
Taras
On 30 Jun 2021, at 12:51, Taras Zakharko wrote:
@Duncan: .S3method() calls registerS3method() with appropriate environmental
argument under the good, so that’s not the problem.
Anyway, I’ve been doing some debugging and I think I have found the issue. The
following snippet in src/objects.c
(https://github.com/wch/r-source/blob/ecc633b37d77fdd1cb27dda74d7f6b3684f0c01c/src/main/objects.c#L525)
sets the global lookup_use_topenv_as_defenv variable:
if(lookup_use_topenv_as_defenv == -1) {
lookup = getenv("_R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_");
lookup_use_topenv_as_defenv =
((lookup != NULL) && StringFalse(lookup)) ? 0 : 1;
}
Isn’t that supposed to be
lookup_use_topenv_as_defenv = ((lookup != NULL) &&
StringFalse(lookup)) ? 1 : 0;
instead?
The way the code works right now, methods will be looked up in top environment
exactly if _R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_ is not set. This seems
incompatible with what registerS3method() does (setting the
.__S3MethodsTable__. on the defining environment instead of the topenv). When I
change 0 and 1 around, everything works as expected.
In the meantime, I can work around it by manually injecting __S3MethodsTable__
into .GlobalEnv (which is my topenv here).
I can open a bug report, but I would like to wait for some more comments.
Best,
Taras
On 30 Jun 2021, at 12:39, Joshua Ulrich wrote:
On Wed, Jun 30, 2021 at 5:17 AM Duncan Murdoch wrote:
On 30/06/2021 5:22 a.m., Taras Zakharko wrote:
Dear all,
I have a generic function and a bunch of methods defined in a s
Re: [Rd] S3 dispatch does not work for generics defined inside an environment
Thanks Duncan, I will submit a bug report to R core then.
Best,
Taras
> On 30 Jun 2021, at 14:16, Duncan Murdoch wrote:
>
> On 30/06/2021 7:37 a.m., Taras Zakharko wrote:
>> Thats not how I read the code? Consider this snippet from registerS3method:
>> genfun <- get(genname, envir = envir)
>> if (.isMethodsDispatchOn() && methods::is(genfun, "genericFunction"))
>> genfun <- methods::finalDefaultMethod(genfun@default)
>> if (typeof(genfun) == "closure")
>> environment(genfun)
>> else .BaseNamespaceEnv
>> This is the environment where the method cache (.__S3MethodsTable__.) will
>> be updated. The problem hoverer, the with the default setting of
>> _R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_ this is *not* the environment
>> where R will look for the method cache. Manually injecting the entry into
>> the method cache of the top environment instead works like a charm.
>> The envir argument is just for looking up the generic function.
>
> Yes, I think you're right: the method is registered in the environment of
> the generic. I think by default it is searched for in the topenv() of that
> environment.
>
> In most cases those are the same thing, but in your code they aren't the
> same: the topenv is the global environment.
>
> So I think I'd agree this is a bug. I'd say the bug is in registerS3method,
> which should pay attention to the _R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_
> setting when it computes defenv.
>
> Here's a workaround that copies methods to the appropriate place:
>
> fixS3registrations <- function(genericname, envir = parent.frame()) {
> generic <- get(genericname, envir=envir)
> genenv <- environment(generic)
> top <- topenv(genenv)
>
> if (!identical(genenv, top)) {
>toptable <- top[[".__S3MethodsTable__."]]
>if (is.null(toptable)) {
> toptable <- new.env(hash = TRUE, parent = baseenv())
> top[[".__S3MethodsTable__."]] <- toptable
>}
>table <- genenv[[".__S3MethodsTable__."]]
>if (!is.null(table)) {
> methodnames <- ls(table, pattern = paste0(genericname, "[.][^.]*$"))
> for (m in methodnames)
>toptable[[m]] <- table[[m]]
>}
> }
> }
>
> env <- local({
> # define the generic function and the method
> myfun <- function(x) UseMethod("myfun")
> myfun.myclass <- function(x) print("called myfun.myclass")
>
> # register the method
> .S3method("myfun", "myclass", myfun.myclass)
> fixS3registrations("myfun")
>
> environment()
> })
>
> env$myfun(structure(0, class = "myclass"))
> #> [1] "called myfun.myclass"
>
>
> Duncan Murdoch
>
>> Best,
>> Taras
>>> On 30 Jun 2021, at 13:29, Duncan Murdoch wrote:
>>>
>>> On 30/06/2021 7:23 a.m., Taras Zakharko wrote:
I had another glance at the code and now I’m convinced that this is the
bug in registerS3method(). Default R behavior (in objects.c) appears to be
to look for method definitions in the top environment, not the defining
environment, but registerS3method() registers the method in the defining
environment. I think registerS3method() should be changed to something
like:
>>>
>>> It registers wherever you ask it to. The default is the defining
>>> environment.
>>>
>>> .S3method is the one that always uses the defining environment, since it
>>> has no way to override the default.
>>>
>>> Duncan Murdoch
>>>
- if (typeof(genfun) == "closure”)
-environment(genfun)
+ if (typeof(genfun) == "closure”)
+ if(isFALSE(Sys.getenv("_R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_”))
environment(genfun) else topenv(environment(genfun))
in order to match the behavior of do_usemethod()
Best,
Taras
> On 30 Jun 2021, at 12:51, Taras Zakharko wrote:
>
> @Duncan: .S3method() calls registerS3method() with appropriate
> environmental argument under the good, so that’s not the problem.
>
> Anyway, I’ve been doing some debugging and I think I have found the
> issue. The following snippet in src/objects.c
> (https://github.com/wch/r-source/blob/ecc633b37d77fdd1cb27dda74d7f6b3684f0c01c/src/main/objects.c#L525)
> sets the global lookup_use_topenv_as_defenv variable:
>
>
> if(lookup_use_topenv_as_defenv == -1) {
> lookup = getenv("_R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_");
> lookup_use_topenv_as_defenv =
> ((lookup != NULL) && StringFalse(lookup)) ? 0 : 1;
> }
>
> Isn’t that supposed to be
>
> lookup_use_topenv_as_defenv = ((lookup != NULL) &&
> StringFalse(lookup)) ? 1 : 0;
>
> instead?
>
> The way the code works right now, methods will be looked up in top
> environment exactly if _R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_ is not
> set. This seems incompatible with what registerS3method() does (setting
> the .__S3MethodsTable__. on the defining environment instead of the
> topenv
Re: [Rd] S3 dispatch does not work for generics defined inside an environment
One error in my workaround: it also ignores
_R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_. If that evaluates to FALSE,
it shouldn't need to make any changes.
Duncan Murdoch
On 30/06/2021 9:20 a.m., Taras Zakharko wrote:
Thanks Duncan, I will submit a bug report to R core then.
Best,
Taras
On 30 Jun 2021, at 14:16, Duncan Murdoch wrote:
On 30/06/2021 7:37 a.m., Taras Zakharko wrote:
Thats not how I read the code? Consider this snippet from registerS3method:
genfun <- get(genname, envir = envir)
if (.isMethodsDispatchOn() && methods::is(genfun, "genericFunction"))
genfun <- methods::finalDefaultMethod(genfun@default)
if (typeof(genfun) == "closure")
environment(genfun)
else .BaseNamespaceEnv
This is the environment where the method cache (.__S3MethodsTable__.) will be
updated. The problem hoverer, the with the default setting of
_R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_ this is *not* the environment where R
will look for the method cache. Manually injecting the entry into the method
cache of the top environment instead works like a charm.
The envir argument is just for looking up the generic function.
Yes, I think you're right: the method is registered in the environment of the
generic. I think by default it is searched for in the topenv() of that
environment.
In most cases those are the same thing, but in your code they aren't the same:
the topenv is the global environment.
So I think I'd agree this is a bug. I'd say the bug is in registerS3method,
which should pay attention to the _R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_
setting when it computes defenv.
Here's a workaround that copies methods to the appropriate place:
fixS3registrations <- function(genericname, envir = parent.frame()) {
generic <- get(genericname, envir=envir)
genenv <- environment(generic)
top <- topenv(genenv)
if (!identical(genenv, top)) {
toptable <- top[[".__S3MethodsTable__."]]
if (is.null(toptable)) {
toptable <- new.env(hash = TRUE, parent = baseenv())
top[[".__S3MethodsTable__."]] <- toptable
}
table <- genenv[[".__S3MethodsTable__."]]
if (!is.null(table)) {
methodnames <- ls(table, pattern = paste0(genericname, "[.][^.]*$"))
for (m in methodnames)
toptable[[m]] <- table[[m]]
}
}
}
env <- local({
# define the generic function and the method
myfun <- function(x) UseMethod("myfun")
myfun.myclass <- function(x) print("called myfun.myclass")
# register the method
.S3method("myfun", "myclass", myfun.myclass)
fixS3registrations("myfun")
environment()
})
env$myfun(structure(0, class = "myclass"))
#> [1] "called myfun.myclass"
Duncan Murdoch
Best,
Taras
On 30 Jun 2021, at 13:29, Duncan Murdoch wrote:
On 30/06/2021 7:23 a.m., Taras Zakharko wrote:
I had another glance at the code and now I’m convinced that this is the bug in
registerS3method(). Default R behavior (in objects.c) appears to be to look for
method definitions in the top environment, not the defining environment, but
registerS3method() registers the method in the defining environment. I think
registerS3method() should be changed to something like:
It registers wherever you ask it to. The default is the defining environment.
.S3method is the one that always uses the defining environment, since it has no
way to override the default.
Duncan Murdoch
- if (typeof(genfun) == "closure”)
-environment(genfun)
+ if (typeof(genfun) == "closure”)
+ if(isFALSE(Sys.getenv("_R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_”))
environment(genfun) else topenv(environment(genfun))
in order to match the behavior of do_usemethod()
Best,
Taras
On 30 Jun 2021, at 12:51, Taras Zakharko wrote:
@Duncan: .S3method() calls registerS3method() with appropriate environmental
argument under the good, so that’s not the problem.
Anyway, I’ve been doing some debugging and I think I have found the issue. The
following snippet in src/objects.c
(https://github.com/wch/r-source/blob/ecc633b37d77fdd1cb27dda74d7f6b3684f0c01c/src/main/objects.c#L525)
sets the global lookup_use_topenv_as_defenv variable:
if(lookup_use_topenv_as_defenv == -1) {
lookup = getenv("_R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_");
lookup_use_topenv_as_defenv =
((lookup != NULL) && StringFalse(lookup)) ? 0 : 1;
}
Isn’t that supposed to be
lookup_use_topenv_as_defenv = ((lookup != NULL) &&
StringFalse(lookup)) ? 1 : 0;
instead?
The way the code works right now, methods will be looked up in top environment
exactly if _R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_ is not set. This seems
incompatible with what registerS3method() does (setting the
.__S3MethodsTable__. on the defining environment instead of the topenv). When I
change 0 and 1 around, everything works as expected.
In the meantime, I can work around it by manually injecting __S3MethodsTable__
into .GlobalEnv (which is
Re: [Rd] S3 dispatch does not work for generics defined inside an environment
I have opened a bug report here:
https://bugs.r-project.org/bugzilla/show_bug.cgi?id=18138
Regarding _R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_… maybe it’s a good time to
consider purging it alltogether? This behavior appears to be completely
undocumented and this variable is mentioned exactly two times in the R source
code. Interestingly enough, utils::.defenv_for_S3_registry does consider it.
Best,
Taras
> On 30 Jun 2021, at 17:38, Duncan Murdoch wrote:
>
> One error in my workaround: it also ignores
> _R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_. If that evaluates to FALSE, it
> shouldn't need to make any changes.
>
> Duncan Murdoch
>
> On 30/06/2021 9:20 a.m., Taras Zakharko wrote:
>> Thanks Duncan, I will submit a bug report to R core then.
>> Best,
>> Taras
>>> On 30 Jun 2021, at 14:16, Duncan Murdoch wrote:
>>>
>>> On 30/06/2021 7:37 a.m., Taras Zakharko wrote:
Thats not how I read the code? Consider this snippet from registerS3method:
genfun <- get(genname, envir = envir)
if (.isMethodsDispatchOn() && methods::is(genfun,
"genericFunction"))
genfun <- methods::finalDefaultMethod(genfun@default)
if (typeof(genfun) == "closure")
environment(genfun)
else .BaseNamespaceEnv
This is the environment where the method cache (.__S3MethodsTable__.) will
be updated. The problem hoverer, the with the default setting of
_R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_ this is *not* the environment
where R will look for the method cache. Manually injecting the entry into
the method cache of the top environment instead works like a charm.
The envir argument is just for looking up the generic function.
>>>
>>> Yes, I think you're right: the method is registered in the environment of
>>> the generic. I think by default it is searched for in the topenv() of that
>>> environment.
>>>
>>> In most cases those are the same thing, but in your code they aren't the
>>> same: the topenv is the global environment.
>>>
>>> So I think I'd agree this is a bug. I'd say the bug is in
>>> registerS3method, which should pay attention to the
>>> _R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_ setting when it computes defenv.
>>>
>>> Here's a workaround that copies methods to the appropriate place:
>>>
>>> fixS3registrations <- function(genericname, envir = parent.frame()) {
>>> generic <- get(genericname, envir=envir)
>>> genenv <- environment(generic)
>>> top <- topenv(genenv)
>>>
>>> if (!identical(genenv, top)) {
>>>toptable <- top[[".__S3MethodsTable__."]]
>>>if (is.null(toptable)) {
>>> toptable <- new.env(hash = TRUE, parent = baseenv())
>>> top[[".__S3MethodsTable__."]] <- toptable
>>>}
>>>table <- genenv[[".__S3MethodsTable__."]]
>>>if (!is.null(table)) {
>>> methodnames <- ls(table, pattern = paste0(genericname, "[.][^.]*$"))
>>> for (m in methodnames)
>>>toptable[[m]] <- table[[m]]
>>>}
>>> }
>>> }
>>>
>>> env <- local({
>>> # define the generic function and the method
>>> myfun <- function(x) UseMethod("myfun")
>>> myfun.myclass <- function(x) print("called myfun.myclass")
>>>
>>> # register the method
>>> .S3method("myfun", "myclass", myfun.myclass)
>>> fixS3registrations("myfun")
>>>
>>> environment()
>>> })
>>>
>>> env$myfun(structure(0, class = "myclass"))
>>> #> [1] "called myfun.myclass"
>>>
>>>
>>> Duncan Murdoch
>>>
Best,
Taras
> On 30 Jun 2021, at 13:29, Duncan Murdoch wrote:
>
> On 30/06/2021 7:23 a.m., Taras Zakharko wrote:
>> I had another glance at the code and now I’m convinced that this is the
>> bug in registerS3method(). Default R behavior (in objects.c) appears to
>> be to look for method definitions in the top environment, not the
>> defining environment, but registerS3method() registers the method in the
>> defining environment. I think registerS3method() should be changed to
>> something like:
>
> It registers wherever you ask it to. The default is the defining
> environment.
>
> .S3method is the one that always uses the defining environment, since it
> has no way to override the default.
>
> Duncan Murdoch
>
>> - if (typeof(genfun) == "closure”)
>> -environment(genfun)
>>
>> + if (typeof(genfun) == "closure”)
>> + if(isFALSE(Sys.getenv("_R_S3_METHOD_LOOKUP_USE_TOPENV_AS_DEFENV_”))
>> environment(genfun) else topenv(environment(genfun))
>> in order to match the behavior of do_usemethod()
>> Best,
>> Taras
>>> On 30 Jun 2021, at 12:51, Taras Zakharko wrote:
>>>
>>> @Duncan: .S3method() calls registerS3method() with appropriate
>>> environmental argument under the good, so that’s not the problem.
>>>
>>> Anyway, I’ve been doing some debugging and I think I have found the
>>> issue. The following
[Rd] On read.csv and write.csv
Apologies if this is a well-worn question; I haven’t found it so far but
there's a lot of r-dev and I may have missed it in the archives. In the mean
time:
I've managed to avoid writing csv files with R for a couple of decades but
we're swopping data with a collaborator and I've tripped over an inconsistency
between read.csv and write.csv that seems less than helpful.
The default line number behaviour for read.csv is to assume that, when the
number of items in the first row is one less than the number in the second,
that the first column contains row names. write.csv, however, includes an empty
string ("") as the first header entry over row names when writing. On
rereading, the original row names are then treated as data with unknown name,
replaced by "X".
That means that, unlike read.table and write.table, something written with
write.csv is not read back correctly by read.csv .
Is that intentional?
And whether it is intentional or not, is it wise?
Example:
( D1 <- data.frame(A=letters[1:5], N=1:5, Y=rnorm(5) ) )
write.csv(D1, "temp.csv")
( D1w <- read.csv("temp.csv") )
# Note the unnecessary new X column ...
#Tidy up
unlink("temp.csv")
This differs from the parent .table defaults; write.table doesn’t add the extra
"" column label, so the object read back with read.table does not contain an
unwanted extra column.
Wouldn’t it be more sensible if write.csv() and read.csv() were consistent in
the same sense as read.table and write.table?
Or at least if there were a switch (as.read.csv=TRUE ?) to tell write.csv to
omit the initial "", or vice versa?
Currently using R version 4.1.0 on Windows, but this reproduces at least as far
back as 3.6
Steve E
***
This email and any attachments are confidential. Any use, copying or
disclosure other than by the intended recipient is unauthorised. If
you have received this message in error, please notify the sender
immediately via +44(0)20 8943 7000 or notify postmas...@lgcgroup.com
and delete this message and any copies from your computer and network.
LGC Limited. Registered in England 2991879.
Registered office: Queens Road, Teddington, Middlesex, TW11 0LY, UK
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] On read.csv and write.csv
Hi Stephen,
Personally, I don't have super strong feelings about this, but
https://datatracker.ietf.org/doc/html/rfc4180#section-2 does say that the
optional header line should have the same number of fields as the data
records, so in as much as that is the "CSV specification", R's read.csv
behavior is supporting an extension, whereas its write.csv is
outputting "standard" compliant csv.
It is possible that one or a few of the mentioned multitude of independent
specs do specify header can have one less, I don't know, but if so,
according to the ietf, its not overly common.
I can't even speak to whether that is why the behavior is as it is, but I
figured it was worth mentioning.
~G
On Wed, Jun 30, 2021 at 2:15 PM Stephen Ellison
wrote:
> Apologies if this is a well-worn question; I haven’t found it so far but
> there's a lot of r-dev and I may have missed it in the archives. In the
> mean time:
>
> I've managed to avoid writing csv files with R for a couple of decades but
> we're swopping data with a collaborator and I've tripped over an
> inconsistency between read.csv and write.csv that seems less than helpful.
> The default line number behaviour for read.csv is to assume that, when the
> number of items in the first row is one less than the number in the second,
> that the first column contains row names. write.csv, however, includes an
> empty string ("") as the first header entry over row names when writing. On
> rereading, the original row names are then treated as data with unknown
> name, replaced by "X".
>
> That means that, unlike read.table and write.table, something written
> with write.csv is not read back correctly by read.csv .
>
> Is that intentional?
> And whether it is intentional or not, is it wise?
>
> Example:
>
> ( D1 <- data.frame(A=letters[1:5], N=1:5, Y=rnorm(5) ) )
> write.csv(D1, "temp.csv")
>
> ( D1w <- read.csv("temp.csv") )
>
> # Note the unnecessary new X column ...
> #Tidy up
> unlink("temp.csv")
>
> This differs from the parent .table defaults; write.table doesn’t add the
> extra "" column label, so the object read back with read.table does not
> contain an unwanted extra column.
>
> Wouldn’t it be more sensible if write.csv() and read.csv() were consistent
> in the same sense as read.table and write.table?
> Or at least if there were a switch (as.read.csv=TRUE ?) to tell write.csv
> to omit the initial "", or vice versa?
>
> Currently using R version 4.1.0 on Windows, but this reproduces at least
> as far back as 3.6
>
> Steve E
>
>
> ***
> This email and any attachments are confidential. Any u...{{dropped:15}}
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] undefined subclass warning
A colleague recently submitted a paper to JSS and was advised to
address the following warning which occurs when their package
(https://CRAN.R-project.org/package=pcoxtime) is loaded:
Warning message:
In .recacheSubclasses(def@className, def, env) :
undefined subclass "numericVector" of class "Mnumeric"; definition
not updated
After much digging I *think* I've concluded that this comes from the
following import chain:
pcoxtime -> riskRegression -> rms -> quantreg -> MatrixModels
that is, loading any of these packages throws the warning.
MatrixModels Imports: *only* {stats, methods, Matrix} and loading these
by themselves is warning-less.
I assume there is some mismatch/incompatibility between MatrixModels
(which was last updated 2021-03-01) and Matrix (2021-05-24), which has
this NEWS item in the most release 1.3-3
:
* removed the nowhere used (and unexported but still active) class union
"Mnumeric" which actually trickled into many base classes properties.
Notably would it break validity of factor with a proposed change in
validity checking, as factors were also "Mnumeric" but did not fulfill
its validity method. Similarly removed (disabled) unused class union
"numericVector".
It seems that REINSTALLING the package from source solves the
problem, which is nice, but I don't fully understand why; I guess
there are class structures that are evaluated at install time and stored
in the package environment ...
Any more explanations would be welcome.
cheers
Ben
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] On read.csv and write.csv
Stephen,
the "unhelpful" column are the row names. They are considered an important part
of a data frame and therefore the default (row.names = TRUE) is to not lose
them (as there is no way back once you do). If you don't want to preserve the
row names you can simply set row.names=FALSE.
Cheers,
Simon
PS: this is likely a question for R-help rather than R-devel
> On 1/07/2021, at 9:15 AM, Stephen Ellison wrote:
>
> Apologies if this is a well-worn question; I haven’t found it so far but
> there's a lot of r-dev and I may have missed it in the archives. In the mean
> time:
>
> I've managed to avoid writing csv files with R for a couple of decades but
> we're swopping data with a collaborator and I've tripped over an
> inconsistency between read.csv and write.csv that seems less than helpful.
> The default line number behaviour for read.csv is to assume that, when the
> number of items in the first row is one less than the number in the second,
> that the first column contains row names. write.csv, however, includes an
> empty string ("") as the first header entry over row names when writing. On
> rereading, the original row names are then treated as data with unknown name,
> replaced by "X".
>
> That means that, unlike read.table and write.table, something written with
> write.csv is not read back correctly by read.csv .
>
> Is that intentional?
> And whether it is intentional or not, is it wise?
>
> Example:
>
> ( D1 <- data.frame(A=letters[1:5], N=1:5, Y=rnorm(5) ) )
> write.csv(D1, "temp.csv")
>
> ( D1w <- read.csv("temp.csv") )
>
> # Note the unnecessary new X column ...
> #Tidy up
> unlink("temp.csv")
>
> This differs from the parent .table defaults; write.table doesn’t add the
> extra "" column label, so the object read back with read.table does not
> contain an unwanted extra column.
>
> Wouldn’t it be more sensible if write.csv() and read.csv() were consistent in
> the same sense as read.table and write.table?
> Or at least if there were a switch (as.read.csv=TRUE ?) to tell write.csv to
> omit the initial "", or vice versa?
>
> Currently using R version 4.1.0 on Windows, but this reproduces at least as
> far back as 3.6
>
> Steve E
>
>
> ***
> This email and any attachments are confidential. Any u...{{dropped:13}}
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] S3 dispatch does not work for generics defined inside an environment
Taras, > P.S. If you are wondering what I am trying to achieve here — we have a > very large codebase and I am trying to use environments as a type of > “poor man’s namespaces” to organize code in a modular fashion. But of > course it’s all pointless if I can’t get the generics to work > reliably. i'm not knowledgeable about S3. but, a different way to try to modularize large code bases is to split them into separate packages. just in case you hadn't already thought about, and rejected, that idea. cheers, Greg __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] undefined subclass warning
> Ben Bolker
> on Wed, 30 Jun 2021 20:23:27 -0400 writes:
>A colleague recently submitted a paper to JSS and was
> advised to address the following warning which occurs when
> their package
> (https://CRAN.R-project.org/package=pcoxtime) is loaded:
> Warning message: In .recacheSubclasses(def@className, def,
> env) : undefined subclass "numericVector" of class
> "Mnumeric"; definition not updated
> After much digging I *think* I've concluded that this
> comes from the following import chain:
> pcoxtime -> riskRegression -> rms -> quantreg ->
> MatrixModels
>that is, loading any of these packages throws the
> warning. MatrixModels Imports: *only* {stats, methods,
> Matrix} and loading these by themselves is warning-less.
> I assume there is some mismatch/incompatibility
> between MatrixModels (which was last updated 2021-03-01)
> and Matrix (2021-05-24), which has this NEWS item in the
> most release 1.3-3
> :
> * removed the nowhere used (and unexported but still
> active) class union "Mnumeric" which actually trickled
> into many base classes properties. Notably would it break
> validity of factor with a proposed change in validity
> checking, as factors were also "Mnumeric" but did not
> fulfill its validity method. Similarly removed (disabled)
> unused class union "numericVector".
> It seems that REINSTALLING the package from source
> solves the problem, which is nice, but I don't fully
> understand why; I guess there are class structures
> that are evaluated at install time and stored in the
> package environment ...
>Any more explanations would be welcome.
>cheers Ben
Yes, Ben,
you are right on spot and very close with your final guess.
Installation and even building of packages using S4 classes (their own *or* of
other packages where they import from) does store the class
definitions already in the binary "dump" of all the R code.
So yes, Matrix cleanup (dropping unused classes actually
helping/improving the class hiearchy by making it slightly
simpler) does need to re-installation of direct Matrix
dependencies in order to avoid the above warning --- which
otherwise has zero consequences.
So the referees of your colleague's paper / package should really learn
that they are wrong in their requirement of getting rid of that
warning.
and as you've suggested in another thread, I should alleviate
the problem by uploading a new version of 'MatrixModels' models
to CRAN {solving another small unrelated buglet} so the warnings
will go away for everyone who updates their installed packages.
Martin
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
