Hi Stefan,
Thanks so much. This will help me i am sure. These past 2 days i was away on a trip so please excuse my delayed answer. Monica > From: stefan.ev...@uos.de > To: pisican...@hotmail.com > Subject: Re: [R] statistical significance of accuracy increase in > classification > Date: Thu, 26 Feb 2009 15:18:32 +0100 > CC: r-help@r-project.org > > > On 26 Feb 2009, at 14:14, Max Kuhn wrote: > > >> Do you know about any good reference that discusses kappa for > >> classification and maybe CI for kappa??? > > You might also want to take a look at this survey article on kappa and > its alternatives: > > Artstein, Ron and Poesio, Massimo (2008). Survey article: Inter-coder > agreement for computational linguistics. Computational Linguistics, > 34(4), 555596. > > which you can download from > > http://www.aclweb.org/anthology-new/J/J08/ > > Alternatives to the standard Fleiss-Cohen asymptotic confidence > intervals in the elementary 2x2 case are discussed in > > Lee, J.J., Tu, Z. N.:"A Better Confidence for Kappa on Measuring > Agreement Between Two Raters with Binary Outcomes" Journal of > Computational and Graphical Statistics, 3:301-321, 1994. > > which is available from JSTOR: > > http://www.jstor.org/stable/1390914 > > An S implementation of their approximations can be downloaded here: > > http://lib.stat.cmu.edu/S/kappa > > I've started to evaluate the accuracy of these approximations with > simulation experiments some time ago, but haven't found the time to > follow up on it. > > Hope this helps, > Stefan > > _________________________________________________________________ 50F681DAD532637!7540.entry?ocid=TXT_TAGLM_WL_t2_ugc_post_022009 [[alternative HTML version deleted]]
______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
