Re: [Rd] Wrongly converging glm()

2017-07-21 Thread Mark Leeds
* Ravi Varadhan > *Cc:* Therneau, Terry M., Ph.D. ; r-devel@r-project.org; > jorism...@gmail.com; westra.harm...@outlook.com > > *Subject:* Re: [Rd] Wrongly converging glm() > > > > Hi Ravi: Well said. In John's Rvmmin package, he has codes for explaining > the cause &

Re: [Rd] Wrongly converging glm()

2017-07-21 Thread Ravi Varadhan
: Friday, July 21, 2017 3:09 PM To: Ravi Varadhan Cc: Therneau, Terry M., Ph.D. ; r-devel@r-project.org; jorism...@gmail.com; westra.harm...@outlook.com Subject: Re: [Rd] Wrongly converging glm() Hi Ravi: Well said. In John's Rvmmin package, he has codes for explaining the cause of the termin

Re: [Rd] Wrongly converging glm()

2017-07-21 Thread Mark Leeds
el-boun...@r-project.org] On Behalf Of > Therneau, Terry M., Ph.D. > Sent: Friday, July 21, 2017 8:04 AM > To: r-devel@r-project.org; Mark Leeds ; > jorism...@gmail.com; westra.harm...@outlook.com > Subject: Re: [Rd] Wrongly converging glm() > > I'm chiming in late sin

Re: [Rd] Wrongly converging glm()

2017-07-21 Thread Ravi Varadhan
devel@r-project.org; Mark Leeds ; jorism...@gmail.com; westra.harm...@outlook.com Subject: Re: [Rd] Wrongly converging glm() I'm chiming in late since I read the news in digest form, and I won't copy the entire conversation to date. The issue raised comes up quite often in Cox mode

Re: [Rd] Wrongly converging glm()

2017-07-21 Thread Therneau, Terry M., Ph.D.
I'm chiming in late since I read the news in digest form, and I won't copy the entire conversation to date. The issue raised comes up quite often in Cox models, so often that the Therneau and Grambsch book has a section on the issue (3.5, p 58). After a few initial iterations the offending co

Re: [Rd] Wrongly converging glm()

2017-07-20 Thread Harm-Jan Westra
() function. With kind regards, Harm-Jan From: Joris Meys<mailto:jorism...@gmail.com> Sent: Thursday, July 20, 2017 16:06 To: Harm-Jan Westra<mailto:westra.harm...@outlook.com> Cc: r-devel@r-project.org<mailto:r-devel@r-project.org> Subject: Re: [Rd] Wrongly converging glm()

Re: [Rd] Wrongly converging glm()

2017-07-20 Thread Joris Meys
On Thu, Jul 20, 2017 at 8:32 PM, Harm-Jan Westra wrote: > My apologies if I seemed to ‘blame R’. This was in no way my intention. I > get the feeling that you’re missing my point as well. > I get that now. But you're on R-devel and you started with the claim that R "falsely reports...". That loo

Re: [Rd] Wrongly converging glm()

2017-07-20 Thread Harm-Jan Westra
a.harm...@outlook.com<mailto:westra.harm...@outlook.com>> Cc: r-devel@r-project.org<mailto:r-devel@r-project.org><mailto:r-devel@r-project.org<mailto:r-devel@r-project.org>> Subject: Re: [Rd] Wrongly converging glm() On Thu, Jul 20, 2017 at 6:21 PM, Harm-Ja

Re: [Rd] Wrongly converging glm()

2017-07-20 Thread Mark Leeds
e is > always paying attention (even if they know what they're doing). > > If R would do that, it wouldn't start the fitting procedure but just > return an error "Your analysis died due to a lack of useable data." . > Because that's the problem here. >

Re: [Rd] Wrongly converging glm()

2017-07-20 Thread Harm-Jan Westra
Meys<mailto:jorism...@gmail.com> Sent: Thursday, July 20, 2017 13:16 To: Harm-Jan Westra<mailto:westra.harm...@outlook.com> Cc: r-devel@r-project.org<mailto:r-devel@r-project.org> Subject: Re: [Rd] Wrongly converging glm() On Thu, Jul 20, 2017 at 6:21 PM, Harm-Jan Wes

Re: [Rd] Wrongly converging glm()

2017-07-20 Thread Harm-Jan Westra
gards, Harm-Jan From: Simon Bonner Sent: Thursday, July 20, 2017 12:32 PM To: Joris Meys; Harm-Jan Westra Cc: r-devel@r-project.org Subject: RE: [Rd] Wrongly converging glm() In defence of Harma-Jan's original post I would say that there is a difference be

Re: [Rd] Wrongly converging glm()

2017-07-20 Thread Joris Meys
even if they know what they're doing). > If R would do that, it wouldn't start the fitting procedure but just return an error "Your analysis died due to a lack of useable data." . Because that's the problem here. > > > With kind regards, > > >

Re: [Rd] Wrongly converging glm()

2017-07-20 Thread Simon Bonner
lto:r-devel-boun...@r-project.org] On Behalf Of Joris Meys > Sent: July 20, 2017 11:39 AM > To: Harm-Jan Westra > Cc: r-devel@r-project.org > Subject: Re: [Rd] Wrongly converging glm() > > Allow me to chime in. That's an interesting case you present, but as far as > I'm

Re: [Rd] Wrongly converging glm()

2017-07-20 Thread Harm-Jan Westra
ey know what they're doing). With kind regards, Harm-Jan From: Joris Meys Sent: Thursday, July 20, 2017 11:38 AM To: Harm-Jan Westra Cc: r-devel@r-project.org Subject: Re: [Rd] Wrongly converging glm() Allow me to chime in. That's an interesting case

Re: [Rd] Wrongly converging glm()

2017-07-20 Thread Joris Meys
Allow me to chime in. That's an interesting case you present, but as far as I'm concerned the algorithm did converge. The estimate of -9.25 has an estimated standard error of 72.4, meaning that frequentists would claim the true value would lie anywhere between appx. -151 and 132 (CI) and hence the

[Rd] Wrongly converging glm()

2017-07-20 Thread Harm-Jan Westra
Dear R-core, I have found an edge-case where the glm function falsely concludes that the model has converged. The issue is the following: my data contains a number of covariates, one of these covariates has a very small variance. For most of the rows of this covariate, the value is 0, except f