Re: [Rd] Running package tests and not stop on first fail
> Hervé Pagès > on Mon, 7 Nov 2016 14:37:15 -0800 writes: > On 11/05/2016 01:53 PM, Martin Maechler wrote: >>> Oliver Keyes >>> on Fri, 4 Nov 2016 12:42:54 -0400 writes: >> >> > On Friday, 4 November 2016, Martin Maechler >> > wrote: >> >> >> > Dirk Eddelbuettel > >> >> > on Fri, 4 Nov 2016 10:36:52 -0500 writes: >> >> >> >> > On 4 November 2016 at 16:24, Martin Maechler wrote: | >> >> My > proposed name '--no-stop-on-error' was a quick shot; >> >> if | > somebody has a more concise or better "English >> >> style" > wording | (which is somewhat compatible with all >> >> the other > options you see | from 'R CMD check --help'), >> >> | please > speak up. >> >> >> >> > Why not keep it simple? The similar feature this most >> >> > resembles is 'make -k' and its help page has >> >> >> >> > -k, --keep-going >> >> >> >> > Continue as much as possible after an > error. While >> >> the target that failed, and those that > depend on it, >> >> cannot be remade, the other dependencies of > these >> >> targets can be processed all the same. >> >> >> >> Yes, that would be quite a bit simpler and nice in my >> >> view. One may think it to be too vague, >> >> > Mmn, I would agree on vagueness (and it breaks the pattern >> > set by other flags of human-readability). Deep familiarity >> > with make is probably not something we should ask of >> > everyone who needs to test a package, too. >> >> > I quite like stop-on-error=true (exactly the same as the >> > previous suggestion but shaves off some characters by >> > inverting the Boolean) >> >> Thank you, Brian, Dirk and Oliver for these (and some offline) >> thoughts and suggestions! >> >> My current summary: >> >> 1) I really don't want a --=value >> but rather stay with logical/binary variables that "express >> themselves"... in the same way I strongly prefer >> >> if (A_is_special) >> to >> if (A_special == TRUE) >> >> for a logical variable A_* . Yes, this is mostly a matter >> of taste,.. but related to how R style itself "works" >> >> 2) Brian mentioned that this is only about ./tests/ tests which >> are continued, not about the Examples which are treated separately. >> That's why we had contemplated additionally using 'tests' (because that's >> the directory name used for unit/regression/.. tests) in the option >> name. >> >> Even though Brian is correct, ideally we *would* want to also influence the >> examples' running to *not* stop on a first error.. However that would >> need more work, reorganizing how the examples are run and that may not be >> worth the pain. However it should be considered a goal in the long run. > My name is Hervé, and I was not suggesting that what happens with the > examples should be changed. I was just preaching consistency (again > sorry) between what happens with the examples and what happens with > the tests. Thank you, Hervé and excuse me for not answering more focused on what you said. I think I do understand what you say (at least by now :-)) and agree that consistency is something important and to be strived for, also with these options. > Why not simply change the latter? > Do we really need an option to control this? Very good questions. If the change could be made much better, I'd agree we'd not need a new option because the change could be considerided uniformly better than the current (R 3.3.2, say) behavior. However the change as it is currently, is not good enough to be the only option (see below). > The behavior was changed for the examples a couple of > years ago and nobody felt the need to introduce an option > to control this at the time. Yes, that change was made very nicely (not by me) and I'd say the result *was* uniformly better than the previous behavior, so there did not seem much of a reason to still provide the old behavior. >> After all that, I tend to remain with the original proposed name. It is at >> least easy to pronounce and spell correctly... > Unless --no-stop-on-error controls both (i.e. examples and tests), and > both behave the same way by default, this is a misnomer. Well, if you choose the option, there *is* no stop on errors anymore ... because the examples nowadays never stop the (R CMD check Pkg) from running on an error as we've mentioned above. [[--- Digression on Examples' checking However / on the other hand: Because of the way the examples are run --- efficiently from one single R source file --- it is not so easy there, to let them run further: The first error from all the examples stops running the example-checks, i.e., the -Ex.R script, but at least *does* continue to run the
[Rd] confusing error from model.frame when var name=function name
This took me a few minutes of head-scratching:
Normally model.frame() gives an easily interpretable error if a variable
can't be found (in the data frame *or* elsewhere in the environment):
model.frame(~a,data=data.frame(x=1:5))
## Error in eval(predvars, data, env) : object 'a' not found
Now suppose you happen to have a variable name that matches an R
function name:
model.frame(~replicate,data=data.frame(x=1:5))
## Error in model.frame.default(~replicate, data = data.frame(x = 1:5)) :
## object is not a matrix
This happens somewhere inside a .External() call:
data <- .External2(C_modelframe, formula, rownames, variables,
varnames, extras, extranames, subset, na.action)
so I haven't had the heart to track it all the way to its source yet.
FWIW this happens whether the function is built-in or user-created.
I don't think the possibly forthcoming "well just don't name your
variables that way" advice is entirely reasonable here ('replicate' is a
perfectly respectable variable name, as are many names that happen to
coincide with R function names (like 'c' !)
I can easily implement my own checking function, at least for the
contents of the data frame (all(all.vars(formula) %in% names(data)):
checking for *non-function variables only* that exist anywhere in the
searchable environment is a bigger task), but this seems to me to be an
infelicity that would be lovely to have corrected ...
I will post to R-bugs if this is not shot down in flames here.
cheers
Ben Bolker
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Running package tests and not stop on first fail
Thanks Martin. These changes are great and improve the usefulness of 'R CMD check'. Especially in the context of the Bioconductor daily builds where we'll use --no-stop-on-test-error so developers will get a full picture of all the errors in their package at once. Cheers, H. To provide some context to my special interest for this, On 11/08/2016 01:34 AM, Martin Maechler wrote: Hervé Pagès on Mon, 7 Nov 2016 14:37:15 -0800 writes: > On 11/05/2016 01:53 PM, Martin Maechler wrote: >>> Oliver Keyes >>> on Fri, 4 Nov 2016 12:42:54 -0400 writes: >> >> > On Friday, 4 November 2016, Martin Maechler >> > wrote: >> >> >> > Dirk Eddelbuettel > >> >> > on Fri, 4 Nov 2016 10:36:52 -0500 writes: >> >> >> >> > On 4 November 2016 at 16:24, Martin Maechler wrote: | >> >> My > proposed name '--no-stop-on-error' was a quick shot; >> >> if | > somebody has a more concise or better "English >> >> style" > wording | (which is somewhat compatible with all >> >> the other > options you see | from 'R CMD check --help'), >> >> | please > speak up. >> >> >> >> > Why not keep it simple? The similar feature this most >> >> > resembles is 'make -k' and its help page has >> >> >> >> > -k, --keep-going >> >> >> >> > Continue as much as possible after an > error. While >> >> the target that failed, and those that > depend on it, >> >> cannot be remade, the other dependencies of > these >> >> targets can be processed all the same. >> >> >> >> Yes, that would be quite a bit simpler and nice in my >> >> view. One may think it to be too vague, >> >> > Mmn, I would agree on vagueness (and it breaks the pattern >> > set by other flags of human-readability). Deep familiarity >> > with make is probably not something we should ask of >> > everyone who needs to test a package, too. >> >> > I quite like stop-on-error=true (exactly the same as the >> > previous suggestion but shaves off some characters by >> > inverting the Boolean) >> >> Thank you, Brian, Dirk and Oliver for these (and some offline) >> thoughts and suggestions! >> >> My current summary: >> >> 1) I really don't want a --=value >> but rather stay with logical/binary variables that "express >> themselves"... in the same way I strongly prefer >> >> if (A_is_special) >> to >> if (A_special == TRUE) >> >> for a logical variable A_* . Yes, this is mostly a matter >> of taste,.. but related to how R style itself "works" >> >> 2) Brian mentioned that this is only about ./tests/ tests which >> are continued, not about the Examples which are treated separately. >> That's why we had contemplated additionally using 'tests' (because that's >> the directory name used for unit/regression/.. tests) in the option >> name. >> >> Even though Brian is correct, ideally we *would* want to also influence the >> examples' running to *not* stop on a first error.. However that would >> need more work, reorganizing how the examples are run and that may not be >> worth the pain. However it should be considered a goal in the long run. > My name is Hervé, and I was not suggesting that what happens with the > examples should be changed. I was just preaching consistency (again > sorry) between what happens with the examples and what happens with > the tests. Thank you, Hervé and excuse me for not answering more focused on what you said. I think I do understand what you say (at least by now :-)) and agree that consistency is something important and to be strived for, also with these options. > Why not simply change the latter? > Do we really need an option to control this? Very good questions. If the change could be made much better, I'd agree we'd not need a new option because the change could be considerided uniformly better than the current (R 3.3.2, say) behavior. However the change as it is currently, is not good enough to be the only option (see below). > The behavior was changed for the examples a couple of > years ago and nobody felt the need to introduce an option > to control this at the time. Yes, that change was made very nicely (not by me) and I'd say the result *was* uniformly better than the previous behavior, so there did not seem much of a reason to still provide the old behavior. >> After all that, I tend to remain with the original proposed name. It is at >> least easy to pronounce and spell correctly... > Unless --no-stop-on-error controls both (i.e. examples and tests), and > both behave the same way by default, this is a misnomer. Well, if you choose the option, there *is* no stop on errors anymore ... because the examples nowadays never stop the (R CMD check Pkg) from running on an error as we've mentioned above.
