I have a final assessment in a course. The user must pass with an 80% and can retake the assessment as many times as they like.
I chose not to provide question level feedback because the user could just write down the answers to what they got wrong, retake the quiz and without thinking, pass the assessment using their cheat sheet.
I think that assessments are often overlooked as part of the learning process. Most of the time I see them used just as a way to record that the course was taken. Once they reach 80%, SCORM or AICC reports to the LMS that the course has been completed. If all I really need is a record that someone viewed it or tested out of it, that's fine. They can be so much more than that, though.
For one, I think that if the goal is to actually measure a person's understanding of the material, unless you take the time to write a well-constructed assessment the only thing you're measuring is a person's ability to come to logical conclusions based on a process of elimination. (3 of the answers are ridiculous or obviously written as wrong answers so it must be choice C!).
Yes I want to track that a course has been completed and I can do that with the post test. However, I also see the final assessment as a continuation of the learning experience. I don't mean trick them or make it unnecessarily hard. I mean use thought-provoking questions with plausible options.
This is a great area to include frequently made mistakes. When I write questions, I will have a conversation with the SME specifically to determine realistic distractors (the wrong answer options). Then if I'm using feedback per question, instead of giving the correct answer I'll tell them WHY that answer was wrong and when applicable, what the consequence would be if they made that same choice on the job. And that's what I mean by continuing the learning experience into the assessment.
In our LMS, learners can go straight to the post test. Not because that's how we want it. That's just how it's set up. If the questions aren't well thought out, they can easily pass with good guesses and get credit for taking the course without ever seeing it. Or just keep taking it until they get enough right. So we've started included Quizmaker post tests inside the course. If they really want to, they can still skip to the end and take the test until they pass. So then if becomes part of a broader conversation about creating courses that are relevant to the employees. Then use good instructional design so the course is actually worth their time. That may be a topic for another thread though.
Here's a link to another community conversation about assessments. I don't think we got into the feedback part, but it still may be of interest to you.
I agree with Natalia's approach. My preference is to provide feedback for both correct and incorrect responses to questions since I see the quiz as an integral part of getting the message of the content across. If someone is going to take the effort to to figure out what the correct response is by reading through the feedback and use this to take the quiz again and get the items correct they are probably going to pick something up in the process.
Some of my courses are delivered through Blackboard and the quizzes have been pulled out of the Articulate module so that they are more accessible. One possible issue here is that I expect some learners will to jump to the quiz w/o taking the module. There may be a way to stop this, but even if this takes place, with feedback on the answers it still will be possible to cover content in the quiz.
I am both a developer and SME so I have a hard time being able to step back from a course and objectively decide how to best design things. It helps to hear from others.
It's very difficult to avoid cheating in e-learning... tech savvy learners can easily take screenshots of any slide, and maybe even do a screen recording of the entire module. The best way to avoid cheating is to design an engaging and interesting e-learning that is actually meaningful for the learner.... haha now there is something that is easier said than done!
My personal opinion is that i like to give/receive immediate feedback. I think its beneficial to have the learning reinforced/corrected as soon as a question is answered. I think it's easier to make the connection between where the learner went wrong and what the correct answer is when it's shown to them immediately, as opposed to giving them all the information at once at the end. Again, just a personal opinion!
Of course it depends on what your authoring software and LMS are, but an option could be to perhaps create a question pool. This way the questions will be slightly different every time the e-learning is done. Maybe even just randomizing the order of the questions can help prevent cheating.
Our team has found that providing question level feedback that makes the learner think about why a selected answer is incorrect, while giving a hint about what to consider for a better choice, adds to the learning process. This works well with experienced staff who "think they know the answer" and are sometimes surprised to learn that the way they've always done it isn't the best choice.
Often we allow enough "chances" to try out every possible answer. If the learner clicks on all of them they get feedback on why each of the wrong answers falls short - thereby learning more than they might have known if they just got the answer wrong and moved on.
Another way to look at this is to forget your existing material even exists and simply build assessment-based questions/interactions that match the desired learning outcomes. You then go back to your material and map it towards the assessment matter. Your choice as to how you handle the feedback of course.
The aim of this is to produce assessments that are more learner/goal/objective/outcome-centric instead of 'making your content look good and then bolting on some MCQs at the end'.
6 Replies
I think that assessments are often overlooked as part of the learning process. Most of the time I see them used just as a way to record that the course was taken. Once they reach 80%, SCORM or AICC reports to the LMS that the course has been completed. If all I really need is a record that someone viewed it or tested out of it, that's fine. They can be so much more than that, though.
For one, I think that if the goal is to actually measure a person's understanding of the material, unless you take the time to write a well-constructed assessment the only thing you're measuring is a person's ability to come to logical conclusions based on a process of elimination. (3 of the answers are ridiculous or obviously written as wrong answers so it must be choice C!).
Yes I want to track that a course has been completed and I can do that with the post test. However, I also see the final assessment as a continuation of the learning experience. I don't mean trick them or make it unnecessarily hard. I mean use thought-provoking questions with plausible options.
This is a great area to include frequently made mistakes. When I write questions, I will have a conversation with the SME specifically to determine realistic distractors (the wrong answer options). Then if I'm using feedback per question, instead of giving the correct answer I'll tell them WHY that answer was wrong and when applicable, what the consequence would be if they made that same choice on the job. And that's what I mean by continuing the learning experience into the assessment.
In our LMS, learners can go straight to the post test. Not because that's how we want it. That's just how it's set up. If the questions aren't well thought out, they can easily pass with good guesses and get credit for taking the course without ever seeing it. Or just keep taking it until they get enough right. So we've started included Quizmaker post tests inside the course. If they really want to, they can still skip to the end and take the test until they pass. So then if becomes part of a broader conversation about creating courses that are relevant to the employees. Then use good instructional design so the course is actually worth their time. That may be a topic for another thread though.
Here's a link to another community conversation about assessments. I don't think we got into the feedback part, but it still may be of interest to you.
I agree with Natalia's approach. My preference is to provide feedback for both correct and incorrect responses to questions since I see the quiz as an integral part of getting the message of the content across. If someone is going to take the effort to to figure out what the correct response is by reading through the feedback and use this to take the quiz again and get the items correct they are probably going to pick something up in the process.
Some of my courses are delivered through Blackboard and the quizzes have been pulled out of the Articulate module so that they are more accessible. One possible issue here is that I expect some learners will to jump to the quiz w/o taking the module. There may be a way to stop this, but even if this takes place, with feedback on the answers it still will be possible to cover content in the quiz.
I am both a developer and SME so I have a hard time being able to step back from a course and objectively decide how to best design things. It helps to hear from others.
Tony
It's very difficult to avoid cheating in e-learning... tech savvy learners can easily take screenshots of any slide, and maybe even do a screen recording of the entire module. The best way to avoid cheating is to design an engaging and interesting e-learning that is actually meaningful for the learner.... haha now there is something that is easier said than done!
My personal opinion is that i like to give/receive immediate feedback. I think its beneficial to have the learning reinforced/corrected as soon as a question is answered. I think it's easier to make the connection between where the learner went wrong and what the correct answer is when it's shown to them immediately, as opposed to giving them all the information at once at the end. Again, just a personal opinion!
Of course it depends on what your authoring software and LMS are, but an option could be to perhaps create a question pool. This way the questions will be slightly different every time the e-learning is done. Maybe even just randomizing the order of the questions can help prevent cheating.
Hope this helps!
Nicole
Our team has found that providing question level feedback that makes the learner think about why a selected answer is incorrect, while giving a hint about what to consider for a better choice, adds to the learning process. This works well with experienced staff who "think they know the answer" and are sometimes surprised to learn that the way they've always done it isn't the best choice.
Often we allow enough "chances" to try out every possible answer. If the learner clicks on all of them they get feedback on why each of the wrong answers falls short - thereby learning more than they might have known if they just got the answer wrong and moved on.
Another way to look at this is to forget your existing material even exists and simply build assessment-based questions/interactions that match the desired learning outcomes. You then go back to your material and map it towards the assessment matter. Your choice as to how you handle the feedback of course.
The aim of this is to produce assessments that are more learner/goal/objective/outcome-centric instead of 'making your content look good and then bolting on some MCQs at the end'.
This discussion is closed. You can start a new discussion or contact Articulate Support.