On the one hand, multiple-choice questions do an insufficient job of measuring student understanding; on the other hand, they can be scored by computer and instant feedback can be provided for students, which is the only kind that actually helps them learn.
A happy medium is the idea of certainty-based scoring, confidence-based marking, or whatever it may be called. It involves asking students on a quiz or test a closed-ended question, usually in the form of multiple choice or some multiple-choice derivative, like matching, drop-down menus, drag-and-drop, and so forth. Then, right after the question is answered, they respond to a question about how certain they are in their answer. Choices range from level 1 (almost guessing) to level 2 (medium certainty) to level 3 (absolute certainty).
You could say certainty-based marking, as the Voxitatis Moodle learning management system calls it, fixes a flaw of standard testing with closed questions, which don’t reveal what students really know. A student may give a correct answer, but was it by guessing or was it on the basis of solid knowledge?
A student who is a very lucky guesser can pass a test while a student who omits uncertain answers can fail it. The marking method rewards students who know which answers they do know and who acknowledge when they are uncertain. It ‘punishes’ students, by deducting more points, who:
- are never confident
- claim confidence for incorrect answers
This ties a student’s understanding of the material, at least as it is self-reported, much more closely to the actual score.
It makes the results much more informative for students as well, promoting what we believe to be a deeper learning. The “How sure are you?” question prompts them to think more deeply about the answer, or, if the test is not time limited (none of our tests have time limits, although the system will log you off after two hours), to look up the correct answer.
After submitting the answers, he can see if his certainty estimation was accurate, because our tests and quizzes are set up to provide instant feedback. We encourage students on the system to answer honestly, especially since this information is valuable for them and, perhaps more importantly, the tests and quizzes on our system are set up not for their grades in school but just to help them master the material.
At the end of the tests they know which parts of the material they have to study more in depth and which parts are firmly planted in their brains. We hope they also pay attention to the feedback our system provides with an explanation of the correct answers. In combination, these test-development methodologies create a learning environment with what we hope will be a much higher yield.
