I am working with a set of new eLearning designers and we've had a discussion about assessment questions. "What is the right amount of questions for a course?" Should try to align the number of questions to the length of the course, or stick to what we need the learners to leave with?
I suspect you will get several different answers to this. My opinion (and that is just it, an opinion) is that most assessments don't come close to measuring competencies, regardless of the quantity. But, I digress...
Learning objectives are the outcomes you want for your course. The assessment is supposed to evaluate whether or not the learners have achieved those objectives. How many questions does that take? 1-3 per learning objective?
Interested in hearing what others have to say on this. I suspect it will be, "It depends...."
I agree with Richard that most assessments do not measure compentencies regardless of the number of questions. My view is that you need to define your objectives and then create the assessment questions up front before you create the course content. The number of questions is subjective but should be sufficient to test the objectives you have specified. Only then, create the course content. Assessment questions are far too often an afterthought.
That's a great question to discuss. My take is that it's better to focus your assessment questions on what you need learners to take away from your course. You could have a longer course with only a few key takeaways or a shorter course where learners legitimately need to take away a lot of details. Always connecting course length to the number of assessment questions you include wouldn't account for those situations.
Also, the quality of the assessment questions comes into play too. Ten simple memorization questions may not be as effective for measuring what someone will actually do with the information on the job as, for instance, three questions positioned around a real-world scenario.
Finally, it's always worth considering if there are additional components of the learning experience that will also be assessing learner knowledge. Your assessment may need to be more robust for a stand-alone e-learning course than a pre-work e-learning course that's then followed by in-person training.
Most of the time, I think the correct answer is "none." Quiz questions tend to focus on the bottom two levels of Bloom's taxonomy: Remembering and Understanding. But usually, for the training to be useful, it has to train people to do something.
If your course is teaching me how to correctly dispose of hazardous waste, then asking me a bunch of questions about various aspect of the process doesn't really give you a clear sense of whether or not I can actually do it correctly in the real world.
Instead, your course should put me in various situations where I have hazardous waste I need to dispose of. It should then allow me to make decisions at every step that allow me to demonstrate that I can complete the entire process correctly for the waste I need to dispose of.
Using this approach, most of the course is a "quiz" and no fact-based questions are necessary. You know I am able to correctly dispose of the waste by the end of the course, because in order to get to the end of the course, I had to demonstrate my ability to correctly dispose of the waste.
This is true for anything your course is teaching. Are you teaching interviewing skills? Then your course should ask me to interview several people. Are you teaching me how to complete a performance review? Then you should make me a manager of 3-4 employees and have me write up their performance reviews.
You get the idea. I should be practicing whatever you're teaching me during most of the course. If I'm doing that, you have most of the course to give me feedback about my performance. This will give you a much better assessment of my skills than any typical end-of-course set of quiz questions.
6 Replies
Courtney,
I suspect you will get several different answers to this. My opinion (and that is just it, an opinion) is that most assessments don't come close to measuring competencies, regardless of the quantity. But, I digress...
Learning objectives are the outcomes you want for your course. The assessment is supposed to evaluate whether or not the learners have achieved those objectives. How many questions does that take? 1-3 per learning objective?
Interested in hearing what others have to say on this. I suspect it will be, "It depends...."
Richard
I agree with Richard that most assessments do not measure compentencies regardless of the number of questions. My view is that you need to define your objectives and then create the assessment questions up front before you create the course content. The number of questions is subjective but should be sufficient to test the objectives you have specified. Only then, create the course content. Assessment questions are far too often an afterthought.
Yes! (Although no organization has ever allowed me to do this.) But yes!
That is strange. I wonder why?
Hi Courtney,
That's a great question to discuss. My take is that it's better to focus your assessment questions on what you need learners to take away from your course. You could have a longer course with only a few key takeaways or a shorter course where learners legitimately need to take away a lot of details. Always connecting course length to the number of assessment questions you include wouldn't account for those situations.
Also, the quality of the assessment questions comes into play too. Ten simple memorization questions may not be as effective for measuring what someone will actually do with the information on the job as, for instance, three questions positioned around a real-world scenario.
Finally, it's always worth considering if there are additional components of the learning experience that will also be assessing learner knowledge. Your assessment may need to be more robust for a stand-alone e-learning course than a pre-work e-learning course that's then followed by in-person training.
Most of the time, I think the correct answer is "none." Quiz questions tend to focus on the bottom two levels of Bloom's taxonomy: Remembering and Understanding. But usually, for the training to be useful, it has to train people to do something.
If your course is teaching me how to correctly dispose of hazardous waste, then asking me a bunch of questions about various aspect of the process doesn't really give you a clear sense of whether or not I can actually do it correctly in the real world.
Instead, your course should put me in various situations where I have hazardous waste I need to dispose of. It should then allow me to make decisions at every step that allow me to demonstrate that I can complete the entire process correctly for the waste I need to dispose of.
Using this approach, most of the course is a "quiz" and no fact-based questions are necessary. You know I am able to correctly dispose of the waste by the end of the course, because in order to get to the end of the course, I had to demonstrate my ability to correctly dispose of the waste.
This is true for anything your course is teaching. Are you teaching interviewing skills? Then your course should ask me to interview several people. Are you teaching me how to complete a performance review? Then you should make me a manager of 3-4 employees and have me write up their performance reviews.
You get the idea. I should be practicing whatever you're teaching me during most of the course. If I'm doing that, you have most of the course to give me feedback about my performance. This will give you a much better assessment of my skills than any typical end-of-course set of quiz questions.