Peter,
I had an argument with someone at one of the big companies (google?)
several years ago
over the coding standard you mention, who was claiming that the survival
package had a bug
due to a wrong result using survival::strata() in a formula. I argued back.
I agree with you that names
In response to the tidyverse habit of adding another zillion functions to one's
search space, the use of things like survival::coxph is becoming more common.
But this practice breaks the use of the specials argument in terms.formula,
something that I make heavy use of in the survival package.
The latest version of the survival package has two important additions. In
prior code the call
coxph(Surv(time, status) ~ age + strata(inst), data=lung)
could fail if a version of either Surv() or strata() existed elsewhere on the
search path; the wrong function could be picked up
I got good feedback from the list about a scope issue, so I am coming back for
more.
Prior issue: users who type survival::coxph(survival::Surv(time, status) ~ x1
+ x2 + surv ival::strata(group), data=mydata)
This messes up the character string matching for strata, done via tt <-
terms(formu
4570.10780.8521
>
>
> Or, to be facetious:
>
>> lm(y~base::"+"(x,z))
> Call:
> lm(formula = y ~ base::"+"(x, z))
>
> Coefficients:
> (Intercept) base::"+"(x, z)
> 0.4516 0.4383
>
space everything. It just takes one
conflicting method to
taint it :)
On Mon, Aug 26, 2024 at 12:44 PM Therneau, Terry M., Ph.D. via R-devel
mailto:r-devel@r-project.org>> wrote:
Thanks to all for the responses. A couple notes It is nice to get the overall
feedback
that I'm not nuts
Thanks to all for the responses. A couple notes It is nice to get the overall
feedback
that I'm not nuts to be terribly annoyed by this, and don't need to fix it
tomorrow.
Berwin 's note brings to mind the old adage that "The reason it is so hard to
make things
foolproof is that fools are so
The survival package makes significant use of the "specials" argument of
terms(), before
calling model.frame; it is part of nearly every modeling function. The reason
is that
strata argments simply have to be handled differently than other things on the
right hand
side. Likewise for tt() and
I am trying to clear up all the "NOTE"s before a CRAN submission, but am a bit
confused
about this one. What is it complaining about -- that it doesn't like my name?
...
* checking for file ‘deming/DESCRIPTION’ ... OK
* this is package ‘deming’ version ‘1.4-1’
* checking CRAN incoming feasibil
Thank you. An update of lme4 fixed the issue. No warning, no crash.
On 6/15/24 2:06 AM, Ivan Krylov wrote:
В Sat, 15 Jun 2024 02:04:31 +
"Therneau, Terry M., Ph.D. via R-devel" пишет:
other attached packages:
[1] lme4_1.1-35.1 Matrix_1.7-0
I see you have a new Matrix (1.7-0
I ran across this by accident when working up an example.It uses a data set
from the survival package, but nothing else from there. Fails on the Intel
machine shown below, and on a virtual linux instance on a newer Mac.
Terry
> library(survival)
> library(lme4)
Loading required package: Ma
Let me give partial assent to Michael's suggestion: a) have an easy way to
turn this on
and b) add a strong suggestion to do so to the WRE manual. Kurt's example in
the email
shows how to do (a); but I just looked in the WRE manual and don't see any
reference to
it, nor any mention from R C
Is there a way to include the compiled version of a vignette in the doc
directory but mark
it to NOT be rerun by CRAN? I think I remember that this is possible, but
have forgotton
how. (It might even be a false memory.)
Terry T.
Background: Beth Atkinson and I are splitting out many of t
I want to thank Ivan and Bill for useful advice. I eventually found my memory
mistake,
which was of the 'obvious once you see it' variety. Ivan's note that it
appeared to be in
a small allocation was correct.
> I've hit a roadblock debugging a new update to the survival package. I do
> de
I've hit a roadblock debugging a new update to the survival package. I do
debugging in
a developement envinment, i.e. I don't create and load a package but rather
source all
the .R files and dyn.load an .so file, which makes things a bit easier.
Running with R -d "valgrind --tool=memch
Add my name to those who think this is a good change.
As someone who works daily with medical research data, the number of times I
have been
caugth by failure of statements like
as.Date(ifelse( is.na(prog.dt), lastfu.dt, prog.dt)
is a large annoyance. Replace ifelse by any number of selec
The connected blog has the statement "Most authors will not have to do
anything as the
number of CRAN packages that will need some attention is below 1%, but authors
of packages
using native (C, C++ or Fortran) code should read the following lines."
My packages do use a lot of C, but I never
This is a follow-up to my earlier note on [.terms. Based on a couple days'
work getting
the survival package to work around issues, this will hopefully be more
concise and
better expressed than the prior note.
1.
test1 <- terms( y ~ x1:x2 + x3)
check <- drop.terms(termobj =test1, dropx = 1)
The survival package uses [.terms a fair bit, in particular it makes use of the
index
returned in the 'specials' attribute, but the base R code has at least two
problems.
First, it does not account for offset terms, if present, and also fails for a
formula such
as y ~ age + (sex=='male'). Y
ers,
> Simon
>
>
>
>> On Mar 16, 2021, at 7:19 AM, Therneau, Terry M., Ph.D. via R-devel
>> wrote:
>>
>> This change in R-devel just bit me. Under the newest release, if I
>> attach() another
>> .RData directory, the methods are not detected
This change in R-devel just bit me. Under the newest release, if I attach()
another
.RData directory, the methods are not detected.
Was it intentional? Running in Linux. Here is a script of an example that
works fine
under 3.6.2. but fails in R-devel.
tmt% mkdir temp1
tmt% cd temp1
tmt%
Thank you Brian. I had not quite grasped how the process works, now the
descriptions and
usage make sense.
Terry
On 2/19/21 4:28 AM, Prof Brian Ripley wrote:
> On 18/02/2021 18:30, Therneau, Terry M., Ph.D. via R-devel wrote:
>> This is a CRAN question:
>>
>> I have t
There are some nice tools to automate reverse dependency checks, but for a
large package
the real issue is the envirionment. The description of the crandalf site on
github has a
nice summary. One package uses a cryto libraries (oops, install those),
another uses some
latex macros I've never
This is a CRAN question:
I have taken care to compress files in the data directory using "xz" (and
checked that it
is the best). Is there then any impact or use for the LazyDataCompression
option in the
DESCRIPTION file?
--
Terry M Therneau, PhD
Department of Health Science Research
Mayo C
I am testing out the next release of survival, which involves running R CMD
check on 868
CRAN packages that import, depend or suggest it.
The survival package has a lot of data sets, most of which are non-trivial real
examples
(something I'm proud of). To save space I've bundled many of them,
Roger,
Over the years I have converted many of the .C calls in the survival package
to .Call.
As others have said, the big advantage is memory footprint. I did it because
there are a
few users who call survfit or coxph with really large data sets, and not
copying the data
can be the diffe
dyverse but not
> with others). And of course none work in sapply or lapply. Providing a
> shorthand in base may help to improve this. You don't have to use it
> if you don't want to, and you can establish coding standards that
> disallow it if you like.
>
> Best,
>
“The shorthand form \(x) x + 1 is parsed as function(x) x + 1. It may be helpful in making
code containing simple function expressions more readable.”
Color me unimpressed.
Over the decades I've seen several "who can write the shortest code" threads: in Fortran,
in C, in Splus, ... The same o
a
single .rda file does save space, but it causes some issues with data(). The
overall
tarball goes from 7480 to 6100 in size (ls -s).
Terry
On 10/24/20 4:28 AM, Duncan Murdoch wrote:
> On 23/10/2020 9:25 p.m., Therneau, Terry M., Ph.D. via R-devel wrote:
>> I found an issue with
I found an issue with the data() command this evening when working on the
survival package.
1. I have a lot of data sets in the package, almost all used in at least one
vignette,
help file, or test. As a space saving measure, I have bundled many of them
together,
i.e., the file data/cancer.r
When I run R CMD check on the survival package I invariably get a note:
...
* checking for file ‘survival/DESCRIPTION’ ... OK
* this is package ‘survival’ version ‘3.2-6’
* checking CRAN incoming feasibility ... NOTE
Maintainer: ‘Terry M Therneau ’
...
This is sufficient for the auto-check process
I have a version of R-devel on my development box that has the address
sanitizer turned
on. This was instrumental in finding a pair of subtle memory issues. (I had
read, but
never written, one element past the end of an array, which caused issues on
some
architectures.)
1. I now get a end
I'm trying to chase down a possible issue with the survival package, and
so was trying to resurrect my ASAN version of r-devel. I added the
lines to config.site for CC, CFLAGS and MAIN_LDFLAGS per section 4.3.3
of the 'packages' documentation, and lines to ~/.R/Makevars.
I followed that with
I ended up finding the issue by a focused code review.
Once in the past, I had a version that would fail under one architecture but
not another,
in that case some help from Brian Ripley pointed me to the offending line of C
code.
That line read, but did not write, at an invalid memory loca
This is a small heads up for package maintainers. Under the more recent
R-devel, R CMD
check turned up some changes in the *.out files. The simple demonstration is
to type
"round(51/80, 3)", which gives .638 under the old and .637 under the new.
(One of my
coxph test cases has a concor
My latest submission of survival3.1-10 to CRAN fails a check, but only on
windows, which
I don't use.
How do I track this down?
The test in question works fine on my Linux box.
Terry
[[alternative HTML version deleted]]
__
R-devel@r-projec
alling
model.frame,
so that it does not try to call cluster?
Another approach I tried was not exporting cluster(), but that fails when
model.frame
tries to call it.
Terry
On 2/24/20 12:21 PM, Duncan Murdoch wrote:
> On 24/02/2020 8:55 a.m., Therneau, Terry M., Ph.D. via R-devel wrote:
I recently had a long argument wrt the survival package, namely that the
following code
didn't do what they expected, and so they reported it as a bug
survival::coxph( survival::Surv(time, status) ~ age + sex +
survival::strata(inst),
data=lung)
a. The Google R style guide recommends that
Thank you to all who replied with helpful suggestions. I had to run off to
meetings and
talks for a bit so am now processing it all.
1. It turns out that the issue was not with coxme, but with bsdmatrix, a
package that
coxme calls. It just happens to have a function ismat() with the same ge
rk Eddelbuettel wrote:
On 13 January 2020 at 10:02, Therneau, Terry M., Ph.D. via R-devel wrote:
| Where can I find out (and replicate) what options as-cran turns on?
See the file src/library/tools/R/check.R in the R sources, and grep for
as_cran which is the internal variable controlled by the -
I've been fighting a CMD check error for coxme this morning. I thought I had
it fixed,
but instead I had forgotton --as-cran on my last test run. So the version just
submitted
to CRAN has warning messages in the log.
I think it is an issue with CRAN. I've sent a message to R-devel asking
Where can I find out (and replicate) what options as-cran turns on?
The issue: the following lines generate an error in R CMD check --as-cran for
coxme. But
there is no error without as-cran nor is there one when I run the code in a
terminal window.
ismat <- function(x) inherits(x, "matrix"
As a footnote, the update.formula function shares one of the flaws I mentioned
in the
earlier post
> test <- y ~ x1 + (x2=='abc') + x3
> update(test, . ~ .-x3)
y ~ x1 + x2 == "abc"
The original formula is valid but the updated one is not.
Terry T.
[[alternative HTML version delete
Martin,
There are a couple of issues with [.terms that have bitten my survival code.
At the
useR conference I promised you a detailed (readable) explanation, and have been
lax in
getting it to you. The error was first pointed out in a bugzilla note from
2016, by the
way. The current sur
Many thanks. Something obvious is sometimes the hardest thing for me to see.
Terry
On 9/6/19 7:10 AM, Iñaki Ucar wrote:
> On Fri, 6 Sep 2019 at 14:08, Therneau, Terry M., Ph.D. via R-devel
> wrote:
>> Yes, that is exactly the problem. The code found in the "config" s
Yes, that is exactly the problem. The code found in the "config" script is
never run.
But why doesn't it get run?
On 9/6/19 5:44 AM, Georgi Boshnakov wrote:
> I cloned therneau/survival and the installation failed since there is no
> definition for exported function survfit().
> A file seems
I treat CRAN as the main repository for survival, but I have also had a github
(therneau/survival) version for a couple of years. It has a vignette2
directory, for
instance, that contains extra vignettes that either take too long to run or
depend on
other packages. It also gets updated more
I remember there was advice about a server that one could use for reverse
dependency
checks, but I forgot to write it down. (Or I did save the info and forgot
where I saved
it...) I have been doing the checks for survival myself, but the count is
getting out of
hand (663, not counting bioc
I'm running "R CMD check" for 600+ of the packages that depend on survival, and
at the end
look for
grep Status *.Rcheck/00check.log | grep ERROR
to find any that failed. But by accident I just looked at the log for the
Greg package,
which finishes with the lines found below. Is the f
For a new release of survival, I normally run R CMD check for every one of the
packages
that depend on survival (forewarned is forearmed).
I updated R-devel on the test machine this morning, and R CMD check --as-cran
survival_3.0-9.tar.gz works as desired.
When I run the check routine on any
Yes, it is almost certainly the same issue. At useR I promised Martin that I
would put
together a clear example and fix for him and I have not yet done so. I will
try to do
that this week.
The heart of the issue is that in a terms object the offset expression will
apear in
the 'variable
I have an ASAN enabled version of R-devel on my test machine, and can get it to
relably
crash. Here is the first part of the session:
tmt-local2434% R --vanilla
R Under development (unstable) (2019-06-28 r76752) -- "Unsuffered Consequences"
Platform: x86_64-pc-linux-gnu (64-bit)
R is free sof
I had a problem with the latest iteration of the survival package (one that I
hope to
post to Github next week) where it would die in strange ways, i.e., work on one
box and
not on another, a vignette would compile if I invoked Sweave myself but fail in
R CMD
build, etc. The type of thing
I had added a vignette to the coxme package and all worked well locally, but it
failed at
CRAN. The issue is that the vignette involves using coxme for pedigree
data, it
doesn't work without the kinship2 package, and I hadn't put in the necessary
"if
(require(" logic.
The question is
On 6/1/19 1:32 PM, Marc Schwartz wrote:
On Jun 1, 2019, at 12:59 PM, Peter Langfelder
wrote:
On Sat, Jun 1, 2019 at 3:22 AM Therneau, Terry M., Ph.D. via R-devel
wrote:
In the next version of the survival package I intend to make a non-upwardly
compatable
change to the survfit object
In the next version of the survival package I intend to make a non-upwardly
compatable
change to the survfit object. With over 600 dependent packages this is not
something to
take lightly, and I am currently undecided about the best way to go about it.
I'm looking
for advice.
The change: 2
The all.equal was a side issue for me; I don't have strong opinions one way or
the other.
You are welcome to leave me out of the loop on that. (Or leave me on the cc,
whatever is
easiest).
I will update the survival package once the [.terms issues are addressed.
One debatable issues is
was not a good tool for testing
certain code issues".
Thanks for the pointer,
Terry
On 4/5/19 9:00 AM, Duncan Murdoch wrote:
On 05/04/2019 9:03 a.m., Therneau, Terry M., Ph.D. via R-devel wrote:
This arose in testing [.terms and has me confused.
data(esoph) # use a standard data set
wrote:
> On 05/04/2019 9:03 a.m., Therneau, Terry M., Ph.D. via R-devel wrote:
>> This arose in testing [.terms and has me confused.
>>
>> data(esoph) # use a standard data set
>>
>> t0x <- terms(model.frame( ~ tobgp, data=esoph))
>> t1 <- t
As footnote, the error in survival:::untangle.specials is that it assumed that
if
attr(myterms, 'specials')[['strata']] was = to 4, then one could use
myterms[-4] to remove
the strata term. Not so. In the model y ~ x1 + x2 + strata(x3) the
attritube will
be 4 -- the response counts --
This arose in testing [.terms and has me confused.
data(esoph) # use a standard data set
t0x <- terms(model.frame( ~ tobgp, data=esoph))
t1 <- terms(model.frame(ncases ~ agegp + tobgp, data=esoph))
t1x <- (delete.response(t1))[-1]
> all.equal(t0x, t1x)
[1] TRUE
# the above is wrong, because
Someone sent me a bug report for survival2.44.1-1 that involves a model with
both cluster
and offset. It turns out to be a 3 part issue with [.terms and my own
untangle.specials
routine. I've spent an evening sorting out the details.
1. The delete.response() function doesn't remove the
export _R_RNG_VERSION_=3.5.0
>
> might remove/explain those errors.
>
> Just a thought
>
> Henrik
>
> On Wed, Mar 27, 2019 at 8:16 PM Therneau, Terry M., Ph.D. via R-devel
> wrote:
>> I'm getting ready to submit an update of survival, and is my habit I ru
The addition of significant stars was, in my opinion, one of the worst defaults
ever added
to R. I would be delighted to see it removed, or at least change the default.
It is one
of the few overrides that I have argued to add to our site-wide defaults file.
My bias comes from 30+ years in a
I'm getting ready to submit an update of survival, and is my habit I run the
checks on all
packages that depend/import/suggest survival. I am getting some very odd
behaviour wrt
non-reproducability. It came to a head when some things failed on one machine
and worked
on another. I found t
ter(factor) if you need it,
> whereas it is rather more painful to convert a character vector to a factor
> with level names determined by the dimension names of the appropriate extent
> of the original table.
>
> -pd
>
>> On 15 Mar 2019, at 13:13 , Therneau, Terry M., Ph.D. via
I have to disagree with both Peter and Martin on this.
The underneath issue is that the automatic conversion of characters to factors
by the
data.frame functions was the single most egregious design blunder in the
Statistical
Models in S book, and we are still living with it. The stringsAsFac
Is there a canonical place to add a hex sticker to a package? I've found use
of
man/figures and inst/.
A nice sticker has been made for survival and since it is a required package I
don't want
to mess it up.
Terry T.
[[alternative HTML version deleted]]
__
Background: I run tools::testInstalledPackage on all packages that dependend on
survival
(605 as of today) before sending a new release to CRAN. It has a few false
positives which
I then follow up on. (Mostly packages with as-yet-incomplete tests in their
inst directory).
Issue: testInstal
The survival package, like many others, has several helper functions that are
not declared
in the namespace, since their only use is to be called by other "main"
functions of the
package. This works well since the functions in the survival namespace can see
them ---
without ::: arguments ---
oved.
>
> Peter
> On Tue, Oct 2, 2018 at 9:43 AM Therneau, Terry M., Ph.D. via R-devel
> wrote:
>> I am now getting the occasional complaint about survival routines that are
>> not able to
>> handle big data. I looked in the manuals to try and update my
>> unders
I am now getting the occasional complaint about survival routines that are not
able to
handle big data. I looked in the manuals to try and update my understanding
of max
vector size, max matrix, max data set, etc; but it is either not there or I
missed it (the
latter more likely). Is it s
Thanks to multiple readers for comments and patience as I sorted this out. I
now have
working length and names methods for Surv objects, which do not seem to break
anything.
I just ran the test suites for 471 packages that depend on survival, but I
don't test
against bioconductor so cannot
I now understand the issue, which leads to a different and deeper issue which
is "how to
assign a proper length to Surv objects".
> Surv(c(1,2,3), c(1,0,1))
[1] 1 2+ 3
The above prints as 3 elements and is conceptually 3 elements. But if I give it
length
method to return a 3 then I need a n
Charles Berry pointed out an error in my reasoning. In the current survival I
forgot the
S3method line for length in the NAMESPACE file, so the behavior is really not
new.
Nonetheless it remains surprising and non-intuitive. Why does model.response
sometimes
attach spurious names, when th
I am getting some unexplained changes in the latest version of survival, and
finally
traced it down to this: model.response acts differently for Surv objects.
Here is a closed form example using a made up class Durv = diagnose survival.
I tracked
it down by removing methods one by one from Su
rence. But Surv
> gets a lot less for free since length() returns twice their logical
> length, an unfortunate inconsistency.
>
> Michael
>
>
>
> On Tue, Jun 26, 2018 at 11:24 AM, Therneau, Terry M., Ph.D. via
> R-devel wrote:
>> I recently got a request to add head
I recently got a request to add head() and tail() methods for Surv objects,
which is quite
reasonable, but not unlike other requests for logLik, vcov, extractAIC, ...
What they
all have in common is that are methods added since creation of the survival
package, and
that I didn't know they
I want to add that the priority for this is rather low, since we have a couple of work
arounds for the user/data set in question. I have some ideas about changing the way in
which ridge() works, which might make the problem moot. The important short-term result
was finding that it wasn't an er
is to use multiple ridge() calls to break it up; since
he/she was using a fixed tuning parameter the result is the same.
Terry T.
On 05/01/2018 11:43 AM, Berry, Charles wrote:
On May 1, 2018, at 6:11 AM, Therneau, Terry M., Ph.D. via R-devel
wrote:
A user sent me an example where coxph
A user sent me an example where coxph fails, and the root of the failure is a case where
names(mf) is not equal to the term.labels attribute of the formula -- the latter has an
extraneous newline. Here is an example that does not use the survival library.
# first create a data set with many lon
A user asked me about this and I can't figure it out.
tmt% R
R Under development (unstable) (2018-04-09 r74565) -- "Unsuffered Consequences"
Copyright (C) 2018 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
> library(survival)
> data(cgd0)
Warning message:
In d
I've run the latest version of survival through the test suites of 486 of the 565 packages
that depend on it (Depends, Imports, LinkingTo, Suggests), and have a couple small issues
that I'm taking to other authors about.
The exercise turned up a half dozen real errors in my package. I plan to su
Ben,
I looked at the source code you pointed out, and the line
fr <- fr[attr(terms(fr),"varnames.fixed")]
sure looks to me as though the terms() function has returned an object with a
varnames.fixed attribute. Either that or your code has inside knowledge that a reader
like me won't know.
the model frame (without
backticks) as an attribute for later retrieval:
https://github.com/lme4/lme4/commit/56416fc8b3b5153df7df5547082835c5d5725e89.
On Wed, Mar 7, 2018 at 8:22 AM, Therneau, Terry M., Ph.D. via R-devel
wrote:
Thanks to Bill Dunlap for the clarification. On follow-up it turns out th
d`)" "`x y z`"
> ll <- attr(Terms, "term.labels")
> gsub("^`|`$", "", ll)
[1] "log(`b$a$d`)" "x y z"
It is a bit of a mess.
Bill Dunlap
TIBCO Software
wdunlap tibco.com <http://tibco.com>
On Mon, Mar 5, 2018 at
A user reported a problem with the survdiff function and the use of variables that contain
a space. Here is a simple example. The same issue occurs in survfit for the same reason.
lung2 <- lung
names(lung2)[1] <- "in st" # old name is inst
survdiff(Surv(time, status) ~ `in st`, data=lung2)
E
87 matches
Mail list logo
