I have unit tests using testthat but these are typically of these types:
1) Check for correct calculation for a single set of valid inputs
2) Check for correct calculation for a larger set of valid inputs
3) Check for errors when providing incorrect inputs
4) Check for known frailties / past issues
This is more for where changes are needed to functions that apply various bits
of business logic that can change over time, so there is no "one answer". A
unit test (at least as I understand it) can be worked through to make sure that
given inputs, the output is computationally correct. What I'd like to do is
overall the impact of a potential change by testing version 1 of a function in
a package for a sample, then test version 2 of a function in a package for a
sample and compare the results.
My difficulties encountered so far is I'm reluctantly to manually do this
change invasively by overwriting the relevant files in the R directory, and
then say using devtools to load it and test it with testthat as I risk
producing incorrect states of my package and potentially releasing the wrong
thing. My preference would be a non-invasive method. Currently, where I'm
trying to do this non-invasively I source a new version of the function stored
in a separate directory, but some of the functions dependent on it continue to
reference to the package version of the functions, this means that when I'm
doing test #2 I have to load lots more functions and hope I've caught them all
(or do some sort of dependency hunting programmatically).
I may be missing something about testthat, but what I'm doing now seems to be
nowhere near optimal and I'd love to have a better solution.
Cheers
Stephanie Locke
BI & Credit Risk Analyst
-Original Message-
From: ONKELINX, Thierry [mailto:thierry.onkel...@inbo.be]
Sent: 10 September 2014 09:30
To: Stephanie Locke; r-devel@r-project.org
Subject: RE: How to test impact of candidate changes to package?
Dear Stephanie,
Have a look at the testthat package and the related article in the R Journal.
Best regards,
ir. Thierry Onkelinx
Instituut voor natuur- en bosonderzoek / Research Institute for Nature and
Forest team Biometrie & Kwaliteitszorg / team Biometrics & Quality Assurance
Kliniekstraat 25
1070 Anderlecht
Belgium
+ 32 2 525 02 51
+ 32 54 43 61 85
thierry.onkel...@inbo.be
www.inbo.be
To call in the statistician after the experiment is done may be no more than
asking him to perform a post-mortem examination: he may be able to say what the
experiment died of.
~ Sir Ronald Aylmer Fisher
The plural of anecdote is not data.
~ Roger Brinner
The combination of some data and an aching desire for an answer does not ensure
that a reasonable answer can be extracted from a given body of data.
~ John Tukey
-Oorspronkelijk bericht-
Van: r-devel-boun...@r-project.org [mailto:r-devel-boun...@r-project.org]
Namens Stephanie Locke
Verzonden: woensdag 10 september 2014 9:55
Aan: r-devel@r-project.org
Onderwerp: [Rd] How to test impact of candidate changes to package?
I use a package to contain simple functions that can be handled by unit tests
for correctness and more complex functions that combine the simple functions
with business logic. Where there are proposals to change either the simple
functions or the business logic, a sample needs to be run before the change and
then after it to understand the impact of the change.
I do this currently by
1. Using Rmarkdown documents
2. Loading the package as-is
3. Getting my sample
4. Running my sample through the package as-is and outputting table of results
5. sourceing new copies of functions 6. Running my sample again and outputting
table of results 7. Reloading package and sourceing different copies of
functions as required
I really don't think this is a good way to do this as it risks missing
downstream dependencies of the functions I'm trying to load into the global
namespace to test.
Has anyone else had to do this sort of testing before on their packages? How
did you do it? Am I missing an obvious package / framework that can do this?
Cheers,
Steph
--
Stephanie Locke
BI & Credit Risk Analyst
The contents of this e-mail and of any attachments transmitted with it are
confidential and intended solely for the use of the individual(s) to whom they
are addressed. If you are not an intended recipient or the person responsible
for delivering this e-mail to the intended recipient(s), you have received this
e-mail in error, and access to it by you is not authorised. You may not use,
copy, distribute, disclose or rely on it or any attachment, or any part of it
or any attachment in any way. If you have received this e-mail in error please
notify Optimum Credit Ltd by email on webmas...@optimumcredit.co.uk or phone on
03330 143125 and then delete it.
All reasonable precautions have been taken to ensure that no viruses
