Hello, I did a cross-validation using cvlm from DAAG package but wasn't sure how to assess the result. Does this result means my model is a good model? I understand that the overall ms is the mean of sum of squares. But is 0.0987 a good number? The response (i.e. gailRel5yr) has min,1st Quantile, median, mean and 3rd Quantile, and max as follows: (0.462, 0.628, 0.806, 0.896, 1.000, 2.400) The plot generated by cvlm, the point does not look too tight. Thanks in advance
> CVlm(gailRel5yr~risk.sum,m=10) Analysis of Variance Table Response: gailRel5yr Df Sum Sq Mean Sq F value Pr(>F) risk.sum 1 4.19 4.19 44.8 2e-09 *** Residuals 88 8.24 0.09 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 fold 1 Observations in test set: 9 3 7 17 27 46 66 67 83 89 risk.sum 27.2345 66.447 29.20988 33.806 28.861 20.293 29.210 1.883 12.482 cvpred 0.9693 1.607 1.00148 1.076 0.996 0.856 1.001 0.557 0.729 gailRel5yr 1.0000 1.333 1.00000 0.778 0.667 1.000 0.750 0.727 1.000 CV residual 0.0307 -0.274 -0.00148 -0.298 -0.329 0.144 -0.251 0.170 0.271 Sum of squares = 0.46 Mean square = 0.05 n = 9 fold 2 Observations in test set: 9 5 41 42 49 51 64 69 81 84 risk.sum 28.529 24.779 28.529 16.194 47.222 8.383 5.813 1.8832 16.1937 cvpred 0.975 0.922 0.975 0.800 1.241 0.688 0.652 0.5958 0.7996 gailRel5yr 0.625 0.533 1.143 0.636 1.833 0.462 1.000 0.5385 0.7143 CV residual -0.350 -0.389 0.168 -0.163 0.592 -0.227 0.348 -0.0573 -0.0853 Sum of squares = 0.86 Mean square = 0.1 n = 9 fold 3 Observations in test set: 9 2 8 12 25 30 47 56 74 82 risk.sum 24.043 12.5825 10.969 16.803 29.017 49.341 15.455 28.256 21.906 cvpred 0.925 0.7651 0.743 0.824 0.995 1.279 0.805 0.984 0.896 gailRel5yr 0.545 0.6923 0.571 0.500 0.714 1.857 0.714 0.667 0.500 CV residual -0.380 -0.0728 -0.171 -0.324 -0.281 0.578 -0.091 -0.318 -0.396 Sum of squares = 0.96 Mean square = 0.11 n = 9 fold 4 Observations in test set: 9 16 22 26 44 50 61 71 72 79 risk.sum 32.960 44.11 17.1 32.628 16.194 5.9823 5.9823 21.955 21.168 cvpred 1.030 1.19 0.8 1.025 0.786 0.6379 0.6379 0.870 0.858 gailRel5yr 1.667 1.57 1.0 0.500 1.000 0.6000 0.6000 0.625 1.143 CV residual 0.637 0.38 0.2 -0.525 0.214 -0.0379 -0.0379 -0.245 0.284 Sum of squares = 1.06 Mean square = 0.12 n = 9 fold 5 Observations in test set: 9 13 15 37 40 48 59 62 76 78 risk.sum 5.8134 28.5287 28.5287 5.982 29.766 45.754 10.468 28.878 1.883 cvpred 0.6144 0.9569 0.9569 0.617 0.976 1.217 0.685 0.962 0.555 gailRel5yr 0.6667 1.0000 1.0000 1.000 0.875 1.833 0.933 1.214 0.909 CV residual 0.0523 0.0431 0.0431 0.383 -0.101 0.617 0.249 0.252 0.354 Sum of squares = 0.79 Mean square = 0.09 n = 9 fold 6 Observations in test set: 9 19 32 33 55 57 68 80 86 88 risk.sum 14.719 28.529 24.043 10.468 20.293 12.48 1.883 5.813 5.982 cvpred 0.764 0.980 0.910 0.698 0.852 0.73 0.564 0.625 0.628 gailRel5yr 1.000 0.667 0.667 0.538 0.667 1.00 0.778 1.000 0.500 CV residual 0.236 -0.314 -0.243 -0.160 -0.185 0.27 0.214 0.375 -0.128 Sum of squares = 0.55 Mean square = 0.06 n = 9 fold 7 Observations in test set: 9 20 24 36 45 52 63 65 87 90 risk.sum 35.3605 10.620 26.44 5.9823 29.766 31.074 16.194 20.293 1.883 cvpred 1.0896 0.702 0.95 0.6289 1.002 1.022 0.789 0.853 0.565 gailRel5yr 1.0000 1.000 0.50 0.6000 1.143 0.714 0.600 1.000 0.933 CV residual -0.0896 0.298 -0.45 -0.0289 0.141 -0.308 -0.189 0.147 0.369 Sum of squares = 0.61 Mean square = 0.07 n = 9 fold 8 Observations in test set: 9 18 21 23 28 38 70 73 75 77 risk.sum 25.656 26.239 49.353 16.682 9.7323 6.870 1.883 1.883 20.293 cvpred 0.943 0.953 1.337 0.794 0.6782 0.631 0.548 0.548 0.854 gailRel5yr 0.700 0.929 0.667 1.000 0.7500 0.944 0.667 0.778 0.462 CV residual -0.243 -0.024 -0.670 0.206 0.0718 0.314 0.119 0.230 -0.392 Sum of squares = 0.88 Mean square = 0.1 n = 9 fold 9 Observations in test set: 9 6 9 34 35 39 43 54 60 85 risk.sum 46.480 29.030 16.19369 40.364 14.7192 17.826 17.8264 26.588 16.194 cvpred 1.241 0.985 0.79725 1.151 0.7757 0.821 0.8212 0.950 0.797 gailRel5yr 1.667 0.846 0.80000 1.000 0.8125 1.083 0.8333 0.556 0.533 CV residual 0.426 -0.139 0.00275 -0.151 0.0368 0.262 0.0122 -0.394 -0.264 Sum of squares = 0.52 Mean square = 0.06 n = 9 fold 10 Observations in test set: 9 1 4 10 11 14 29 31 53 58 risk.sum 37.400 50.409 47.61 47.433 56.210 23.484 29.030 28.529 54.90 cvpred 1.065 1.224 1.19 1.188 1.296 0.894 0.962 0.956 1.28 gailRel5yr 0.909 1.667 0.90 1.650 1.444 0.600 0.545 0.571 2.40 CV residual -0.156 0.442 -0.29 0.462 0.149 -0.294 -0.416 -0.384 1.12 Sum of squares = 2.2 Mean square = 0.24 n = 9 Overall (Sum over all 9 folds) ms 0.0987 ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
