At 02:45 PM 11/18/01 -0700, Roy St Laurent wrote: >Comments interspersed below... > > >Sure, I wouldn't give a student full credit if their process was correct but >their >final result was wrong. But an answer that shows me they know the process >but have the wrong final result is worth MUCH, MUCH more than an answer >that has the final result but provides me with no clue as to whether they >understand >how to get it.
again ... this makes sense with one item ... it does not necessarily make sense with multiple items > > > > 2. that answers without any OTHER supplied information on the part of the > > examinee can't be taken as "knowledge" when, it (sometimes) can be > > > > what if you asked on an exam ... the following: > > > > 1. what is the mean of 10, 9, 8, 8 and 7? _____ > > > > 2. what is the mean of 27, 23, 19, 17 and 16? ____ > > > > 3. what is the mean of 332, 234, 198, 239, and 200? _____ > > > > 4. what is the mean of 23.4, 19.8, 23.1, 19.0, and 26.4? _____ > > > > and, for each of 1 to 4 ... they put down in the blanks, the correct > answers > > > > would you be willing to say that they know how to calculate the mean ... > > ie, they know the process that is needed (and can implement it)? > >Depending upon whether they have access to a calculator and other information, >I may or may not be able to conclude that they know how to calculate a mean. >More importantly, the example you give is the worst possible example of how >to write an examination. Why should I ask them to do four calculations when >one, with properly supported work, will suffice? AND will permit me to better >assess their understanding of the calculation. you miss the point entirely ... i did not suggest that the items above would be specific ones i would put on any exam ... it was to illustrate that one CAN determine "skill" without having work shown ... if someone did the above without the aid of a calculator ... i think you have to admit they understand the process ... now, if you are talking about efficient test item administration ... surely, one item that has to be worked out is much faster to give ... it is not necessarily faster to take necessaril and, surely, it is not faster to grade ... especially with more complex problems >BTW, I would be inclined to ask questions that minimize but do not eliminate >calculations >in favor of questions that ask students to tell me what the quantity >calculated >means in >the context of a data analysis (or other) problem. Though I would not >eliminate >evaluation >of students ability to calculate, I agree with Herman that we need to >focus more >on >students' understanding of concepts. sure, agreed ... but lots of statistical work involves as part of the concept ... calculations ... when doing significance tests for example ... THE calculation makes all the difference in whether you are able to retain or reject some null value ... > > > > i think you would EVEN though there is no other supporting process > > information given by the examinee > > > > so, the statement that no credit should be given when there is no > > supporting other evidence (ie, the process is shown) ... can't be > > considered necessarily valid > > > > the problem here is NOT that no supporting evidence is given, the problem > > is that with ONLY ONE instance of some given concept/skill that we are > > attempting to assess on the part of the examinee, you are not nearly as > > sure given only ONE CORRECT RESPONSE to one item ... whether it could have > > been an answer because of real knowledge or, just happens to be the right > > answer that was arrived at (luckily for the examinee) through some faulty > > process > >Even several instances of correct responses of the sort you described above >do not provide necessarily convincing evidence, and they require more >unnecessary >busy-work for the student. (I am not stricly opposed to "busy-work" in the >context >of student learning of a concept or calculation, but I am opposed to it in the >context of evaluation >of student learning.) the bottom line is that ... for any single item ... whether supporting work is given or not ... one cannot be sure about the level of knowledge of the examinee ... showing work when only a single item is given does give one a higher p value of knowing but .. never ... for sure of course, there is also in interaction with the complexity of the problem ... the more complex the item which requires many more steps in the process, the less likely someone can get to the final point correctly without knowing the correct intermediate steps ... for much simpler problems ... even shown work could have been lucked into ... i would also like to again make a push for correct answers rising to the approximate same level of importance as process ... we just cannot take lightly the fact that when someone gets the wrong answer, saying that this is not THAT important ... there are many many situations where making a mistake in the answer can be critical ... to some decision that is being made, to some NEXT action that is taken, etc. ... we need to stress both process and accuracy ... i sure hope that the bombardier is not just graded on process > > _______________________________________________________ > > dennis roberts, educational psychology, penn state university > > 208 cedar, AC 8148632401, mailto:[EMAIL PROTECTED] > > http://roberts.ed.psu.edu/users/droberts/drober~1.htm > > > > ================================================================= > > Instructions for joining and leaving this list and remarks about > > the problem of INAPPROPRIATE MESSAGES are available at > > http://jse.stat.ncsu.edu/ > > ================================================================= > >-- >Roy St. Laurent >Mathematics & Statistics >Northern Arizona University >http//odin.math.nau.edu/~rts ============================================================== dennis roberts, penn state university educational psychology, 8148632401 http://roberts.ed.psu.edu/users/droberts/drober~1.htm ================================================================= Instructions for joining and leaving this list and remarks about the problem of INAPPROPRIATE MESSAGES are available at http://jse.stat.ncsu.edu/ =================================================================
