Peter, Scot, Jim, and Jason,
Thanks for the various solutions to Jason's problem. In the next
release of psych I will a) document how to use non-numeric
categorical variables (e.g., red, blue, etc.) when using
cohen.kappa, b) suggest that cbind does the job (Peter's solution),
c) make it work with data.frames without the need to do the recode
that Scot did.
For the record, here is the output you get by using cbind:
ck <- cohen.kappa(cbind(x,y))
ck
Call: cohen.kappa1(x = x, w = w, n.obs = n.obs, alpha = alpha)
Cohen Kappa and Weighted Kappa correlation coefficients and
confidence boundaries
lower estimate upper
unweighted kappa 0.098 0.6 1.10
weighted kappa -0.693 0.0 0.69
Number of subjects = 4
ck$agree #show the table of matches
x2f
x1f blue red yellow
blue 0.25 0.00 0.00
red 0.00 0.50 0.00
yellow 0.25 0.00 0.00
Bill
At 7:57 AM -0600 5/25/10, Peter Ehlers wrote:
cohen.kappa(cbind(x,y)) works for me.
-Peter Ehlers
On 2010-05-25 7:42, Scot W. McNary wrote:
Hi,
It doesn't seem happy with non-numeric data in the data frame version.
Maybe a recode would work?
> x1 <- c(1, 2, 3, 1)
> y1 <- c(1, 3, 3, 1)
> cohen.kappa(data.frame(x = x1, y = y1))
Call: cohen.kappa1(x = x, w = w, n.obs = n.obs, alpha = alpha)
Cohen Kappa and Weighted Kappa correlation coefficients and confidence
boundaries
lower estimate upper
unweighted kappa 0.098 0.60 1.1
weighted kappa 0.601 0.86 1.1
The unweighted kappa is .60.
I also tried to create the p x p table version, thinking that it would
be unhappy with less than a p x p table. The example dataset produces a
3 x 2 table, so this pads it out to a 3 x 3:
> x <- factor(x)
> y <- factor(y, levels = c("blue", "red", "yellow"))
> table(x,y)
y
x blue red yellow
blue 1 0 0
red 0 2 0
yellow 1 0 0
> cohen.kappa(table(x,y))
Call: cohen.kappa1(x = x, w = w, n.obs = n.obs, alpha = alpha)
Cohen Kappa and Weighted Kappa correlation coefficients and confidence
boundaries
lower estimate upper
unweighted kappa 0.098 0.6 1.10
weighted kappa -0.693 0.0 0.69
Number of subjects = 4>
The unweighted kappa is the same as above and as found by Jim with other
packages.
Scot
On 5/25/2010 7:31 AM, Jim Lemon wrote:
On 05/25/2010 06:01 PM, Jason Priem wrote:
Hi,
I've got two vectors with ratings from two coders, like this:
x<-c("red", "yellow", "blue", "red") #coder number 1
y<-c("red", "blue", "blue", "red") #coder number 2
I want to find Cohen's Kappa using the wkappa function in the psych
package. The only example in the docs is using a matrix, which I'm
afraid I don't understand--I don't know how to get there from what I've
got. Could anyone point me in the right direction? Thanks!
Hi Jason,
You're not alone. I tried to work out how to run this and it took a
while. Both kappa2 in the "irr" package:
kappa2(cbind(x,y))
and classAgreement in the "e1071" package:
classAgreement(table(x,y))
produce a kappa of 0.6. I was unable to work out how to use the wkappa
function in the "psych" package:
wkappa(table(x,y))
The above led to a kappa of -0.8. Although wkappa is presented as a
test of the reliability of ratings of nominal level data, the example
uses numeric data.
Jim
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
