I was talking to a couple people the other day about whether or not they like to start their e-learning courses with a quiz. Since we were unable to come to a consensus, I thought I would open it up to the fine people of the E-learning Heroes Community.
What do you think? Do you like to begin your courses with a quiz? Why (or why not)?
I very much like this topic and I am interested to hear what other Heroes have to say about this.
I am of mixed minds on this issue because 1) I think pre-course quizzing can be valuable for many reasons including ... How else can you measure how much the students have learned, if you don't have anything to measure Post-Course results against? but at the same time 2) I'm hesitant to quiz learners on information or knowledge I haven't yet presented yet because I don't like "gotchas" and like to set up my learners for success.
That is my two Cents. I would like to hear others opinions as well.
Hi, Allison: It's common to start with a quiz or "pre-test," as it's sometimes called. Then you could scoot the learner to the appropriate section or bypass the training. Johnny would like that.
If you are not doing a pretest sort of thing, I probably wouldn't start with a quiz. How 'bout a story or a conversation between characters or a situation in which Johnny got in a whole heap of trouble because he didn't know the correct way to fill out TPS reports or whatever.
Course beginnings that are now cliche: executive videos saying how important the training is; newspaper headlines that scream how important the training is.
Sorry for the sarcasm. I've been in corporate elearning all day. --Daniel
I like this approach. I have just finished writing a dozen storyboards on financial topics for GenY. Each topic started with not a typical quiz...more questions that introduce some of the topic points which the learner answers Yes / No / Don't Know or True / False, maybe drag/drop or slider to answer a question and regardless of the response the user gets the same general feedback immediately...so it's not a graded quiz it's designed to get the learner thinking. Then those points are expanded on throughout the module.
@Allison: Yep, I do that from time to time as well. I like to call those 'trigger questions' as it (should) trigger some thoughts about the topic they are about to be presented with.
@Allison and David: I don't think a pretest is something I would want on every piece of training but I do feel there are many situations where a lot of people would benefit, especially in large corporations, if there where pretests for each learning objective so the learner could skip sections of the training they already know and get busy with the stuff that needs some work. In the end they should not be wasting their precious time. Less time spend on needless training increases the productivity of the employee.
Danika -- I understand why you are reluctant to ask learners questions about topics that they haven't seen yet, but I personally think that people are more likely to remember the correct answer after getting it wrong the first time. I also think they are more likely to pay close attention to the course if they believe that they will learn information that they don't already know, and a poor score on the pretest may be just what they need to realize that they maybe don't know as much as they may think about that particular topic.
Jeff -- I totally agree with you. I don't think that an entire quiz or pretest is always necessary, but starting out your course with a question or two to just get people thinking can be a good way to grab their attention.
Danika has a great point. People remember more when they make mistakes and learn from them, because it is based on experience.
Typically, when a quiz precedes the presentation, it is call an assessment. You allow the participant to gauge their skill level at the beginning, then compare their skill level at the end after they complete the quiz. The reinforcement value is much better with 'before and after' comparisons. When the participant experiences improved performance, they value the training more and will remember more as a result.
Call it an assessment and explain that it is not tracked, just a tool for the participant.
Allison and Jeff: I find the trigger question is very useful within the course also. A couple of open ended questions before a user watches a video or reviews a chunk of information assist the user to focus on key points. I think good questions point the user to process and apply the knowledge gained from the video or chunk to their performance goals.
The pre-test is a valuable assessment for the purpose of evaluating transfer of knowledge based on Kirkpatrick's model. As Jeff mentioned, context has significant influence on whether it should be used or not. B Jacobs explained the significant of reinforcement value splendidly here.
A strategy I used before was to conduct the pre-test to measure the learner's knowledge in a particular subject i.e. jargon definitions; if the learner received a grade greater than 90% then, I would program (or trigger) the module to skip that subject and not "waste" the learner's cognitive resources on stuff they already know.
What about using this technique as a reflective learning piece as well?
I agree with Cleon; that is the beauty of pre-tests and assessments - they help participants focus on important points or (as Alexander points out), the information that needs more attention.
Ahhh, reflective learning! You hit on my personal favorite, Alexander! Adding before and after surveys are perfect!
B Jacobs and Alex: great points to truly promote effective learner focused training.. I am creating a few trigger questions within a real-life scenario.. Concept: Think of adding a authority figure/ customer / important to the user as the "person" asking the question: Example: The Really Important Customer was asking just the other day about the installation techniques for the product.. he said his VP of Operation heard that the "5th" step is really important!!! Questions: What is that step?? Why is it important? What would you tell the Really Important Customer?" I think these scenario questions add that little extra attention grabber? Your thoughts, please and thanks?
This is a very effective approach as it's story based. When asking
questions is important to be able to align them back to learning objectives.
So, in your question "What is the step?" you can refer to a learning
objective dubbed "Identify each of the steps in the product installation
process".
I think your case scenario is a strong proposition to engage the learner
interactively. Therefore, I would also suggest illustrating the "cause and
effect" of actions chosen by the learner in the module i.e. What would you
tell the Really Important Customer? Option 1- The fifth step is not that
important as the others and can be skipped Option 2- The fifth step ensures
the product doesn't explode when you turn it on. So it's critical!!
Very helpful Alex, very.. I like your build on the discussion to add the cause and effect options concept. I am a little lost when it comes to effective feedback for users,, adn the ability to possibly branch them back to the course section for study.. Words of wisdom on this too please?
Cool Cleon, if you stay with the approach I wrote about, you could include a layer triggered to display a feedback caption possibly linked to a PDF or a section in the course.
Aside from that, you can also adjust the properties of your quiz interaction to allow for a review and them provide the feedback in that manner.
Totally vibing with Alexander's strategy. I'm finding that some form of a pre-assessment AND post-assessment helps learners see where they were at the start of the course so they can compare and get excited about how far they've come at the end of the course. Great reflection tool and win for all.
I totally agree with you, Cleon! Making questions scenario-based is a great way to encourage learner engagement.
I would even push it one step further and really immerse learner in the scenario.
For example, in your scenario you could say: "Your boss knocks on your office door and says, 'Hey [text variable], I've got the VP of Operations on the phone, he wants you to talk to someone about our installation techniques-. He keeps going on about the 5th step, and how important it is. Do you know what he is talking about?'" Then, you could have the learner choose how to respond to the customer.
Thanks a million Allison,, great immersion add to the scenario -- I will add your name to the contributors credit page.. really excellent add!
One more point for feedback, please.. In the scenario, how and when do you direct the learner to the a review section when they choose an incorrect options? Jump to? lightbox? attached doc? Depends on the answer,, degree of incorrectness? Reinforcing the right answer is good, but I find I may interrupt the flow of learning, redirecting sometimes? Your thoughts, please and thanks.
It is really up to you, there are a lot of different ways you could do it.
One way I've done this in the past is by creating a "learn more" or "solution" layer to your quiz, and then adding a button leading to that layer on the incorrect layer.
That way you can give your learner the option to either just continue on right away, or to learn a bit more and then continue.
I generally avoid quizzes and pre-/post-tests (in the traditional sense) like the plague. However.... if you think about it completely differently, they can be quite impactful.
One of the most important things with adult learners is to let them apply what the already know. Many eLearning offerings follow a "tell then test" design, and that rarely allows adults to apply what they already know.
My team has found it to be very effective to present a scenario (quiz question really) and allow the learner to choose a course of action (multiple choice answer) without making it look like a quiz.
The feedback is then consequential, meaning it's not just a right/wrong comment, but something that shows the real life impact.
For example, if a customer complains about the elevator in the building not working, let the learner choose an icon that represents calling the onsite maintenance team, calling the elevator vendor for an emergency response, or putting up an out of order sign.
If they select an out of order sign, the feedback is hearing an irate call or reading a negative survey response from the customer, and others. If they select calling the vendor for an emergency response, an invoice shows up on screen showing the actual cost and the learner's name approving the high expense and a note from the vendor saying there was a crayon in the door track. If they select calling the onsite maintenance team, the feedback will show that the maintenance member found the crayon, removed it, and the elevator worked, therefore the customer was happy.
In that approach, the learner gets to apply knowledge without being forced to listen to or read information they already know, and they don't feel like they're being tested. They simply respond and move on.
It depends. If the quiz serves as a benchmark or an adaptive engine then, yes, they are useful. But a quiz for the sake of a quiz could frustrate and bewilder learners...especially adults. What's the WIIFM for the quiz upfront? Adults will wonder if the quiz will serve a purpose and, if so, that should be stated in the directions. If it isn't tracked, then be explicit about that and reiterate the purpose of it since it is not for a grade.
For reflective learning, I would think questions with open-ended responses would be best...and maybe answers are stored to be reflected on at the conclusion of the course for comparison. "How do your original thoughts compare to what you think no that you have taken the course..." or something like that.
33 Replies
~~ Hello Community ~~
I very much like this topic and I am interested to hear what other Heroes have to say about this.
I am of mixed minds on this issue because 1) I think pre-course quizzing can be valuable for many reasons including ... How else can you measure how much the students have learned, if you don't have anything to measure Post-Course results against? but at the same time 2) I'm hesitant to quiz learners on information or knowledge I haven't yet presented yet because I don't like "gotchas" and like to set up my learners for success.
That is my two Cents. I would like to hear others opinions as well.
~ DD
Hi, Allison: It's common to start with a quiz or "pre-test," as it's sometimes called. Then you could scoot the learner to the appropriate section or bypass the training. Johnny would like that.
If you are not doing a pretest sort of thing, I probably wouldn't start with a quiz. How 'bout a story or a conversation between characters or a situation in which Johnny got in a whole heap of trouble because he didn't know the correct way to fill out TPS reports or whatever.
Course beginnings that are now cliche: executive videos saying how important the training is; newspaper headlines that scream how important the training is.
Sorry for the sarcasm. I've been in corporate elearning all day. --Daniel
Hi Allison
I like this approach. I have just finished writing a dozen storyboards on financial topics for GenY. Each topic started with not a typical quiz...more questions that introduce some of the topic points which the learner answers Yes / No / Don't Know or True / False, maybe drag/drop or slider to answer a question and regardless of the response the user gets the same general feedback immediately...so it's not a graded quiz it's designed to get the learner thinking. Then those points are expanded on throughout the module.
@Allison: Yep, I do that from time to time as well. I like to call those 'trigger questions' as it (should) trigger some thoughts about the topic they are about to be presented with.
@Allison and David: I don't think a pretest is something I would want on every piece of training but I do feel there are many situations where a lot of people would benefit, especially in large corporations, if there where pretests for each learning objective so the learner could skip sections of the training they already know and get busy with the stuff that needs some work. In the end they should not be wasting their precious time. Less time spend on needless training increases the productivity of the employee.
This post was removed by the author
Danika -- I understand why you are reluctant to ask learners questions about topics that they haven't seen yet, but I personally think that people are more likely to remember the correct answer after getting it wrong the first time. I also think they are more likely to pay close attention to the course if they believe that they will learn information that they don't already know, and a poor score on the pretest may be just what they need to realize that they maybe don't know as much as they may think about that particular topic.
Jeff -- I totally agree with you. I don't think that an entire quiz or pretest is always necessary, but starting out your course with a question or two to just get people thinking can be a good way to grab their attention.
Danika has a great point. People remember more when they make mistakes and learn from them, because it is based on experience.
Typically, when a quiz precedes the presentation, it is call an assessment. You allow the participant to gauge their skill level at the beginning, then compare their skill level at the end after they complete the quiz. The reinforcement value is much better with 'before and after' comparisons. When the participant experiences improved performance, they value the training more and will remember more as a result.
Call it an assessment and explain that it is not tracked, just a tool for the participant.
Allison and Jeff: I find the trigger question is very useful within the course also. A couple of open ended questions before a user watches a video or reviews a chunk of information assist the user to focus on key points. I think good questions point the user to process and apply the knowledge gained from the video or chunk to their performance goals.
The pre-test is a valuable assessment for the purpose of evaluating transfer of knowledge based on Kirkpatrick's model. As Jeff mentioned, context has significant influence on whether it should be used or not. B Jacobs explained the significant of reinforcement value splendidly here.
A strategy I used before was to conduct the pre-test to measure the learner's knowledge in a particular subject i.e. jargon definitions; if the learner received a grade greater than 90% then, I would program (or trigger) the module to skip that subject and not "waste" the learner's cognitive resources on stuff they already know.
What about using this technique as a reflective learning piece as well?
I agree with Cleon; that is the beauty of pre-tests and assessments - they help participants focus on important points or (as Alexander points out), the information that needs more attention.
Ahhh, reflective learning! You hit on my personal favorite, Alexander! Adding before and after surveys are perfect!
Yes B! Unfortunately, the reflective approach is often overlooked but so powerful for adult self-directed learners.
B Jacobs and Alex: great points to truly promote effective learner focused training.. I am creating a few trigger questions within a real-life scenario.. Concept: Think of adding a authority figure/ customer / important to the user as the "person" asking the question: Example: The Really Important Customer was asking just the other day about the installation techniques for the product.. he said his VP of Operation heard that the "5th" step is really important!!! Questions: What is that step?? Why is it important? What would you tell the Really Important Customer?" I think these scenario questions add that little extra attention grabber? Your thoughts, please and thanks?
Hi Cleon,
This is a very effective approach as it's story based. When asking
questions is important to be able to align them back to learning objectives.
So, in your question "What is the step?" you can refer to a learning
objective dubbed "Identify each of the steps in the product installation
process".
I think your case scenario is a strong proposition to engage the learner
interactively. Therefore, I would also suggest illustrating the "cause and
effect" of actions chosen by the learner in the module i.e. What would you
tell the Really Important Customer? Option 1- The fifth step is not that
important as the others and can be skipped Option 2- The fifth step ensures
the product doesn't explode when you turn it on. So it's critical!!
I hope this helps ; )
Alexander Salas
Learning Innovation Strategist
MSLPA, MAED, CompTIA A+, MTA Databases
636-474-9049
sl
Very helpful Alex, very.. I like your build on the discussion to add the cause and effect options concept. I am a little lost when it comes to effective feedback for users,, adn the ability to possibly branch them back to the course section for study.. Words of wisdom on this too please?
Cool Cleon, if you stay with the approach I wrote about, you could include a layer triggered to display a feedback caption possibly linked to a PDF or a section in the course.
Aside from that, you can also adjust the properties of your quiz interaction to allow for a review and them provide the feedback in that manner.
Totally vibing with Alexander's strategy. I'm finding that some form of a pre-assessment AND post-assessment helps learners see where they were at the start of the course so they can compare and get excited about how far they've come at the end of the course. Great reflection tool and win for all.
I totally agree with you, Cleon! Making questions scenario-based is a great way to encourage learner engagement.
I would even push it one step further and really immerse learner in the scenario.
For example, in your scenario you could say: "Your boss knocks on your office door and says, 'Hey [text variable], I've got the VP of Operations on the phone, he wants you to talk to someone about our installation techniques-. He keeps going on about the 5th step, and how important it is. Do you know what he is talking about?'" Then, you could have the learner choose how to respond to the customer.
Agreed! I think that this technique is a good way to motivate learners to "pull" the content instead of just always "pushing" the content on them.
Thanks a million Allison,, great immersion add to the scenario -- I will add your name to the contributors credit page.. really excellent add!
One more point for feedback, please.. In the scenario, how and when do you direct the learner to the a review section when they choose an incorrect options? Jump to? lightbox? attached doc? Depends on the answer,, degree of incorrectness? Reinforcing the right answer is good, but I find I may interrupt the flow of learning, redirecting sometimes? Your thoughts, please and thanks.
It is really up to you, there are a lot of different ways you could do it.
One way I've done this in the past is by creating a "learn more" or "solution" layer to your quiz, and then adding a button leading to that layer on the incorrect layer.
That way you can give your learner the option to either just continue on right away, or to learn a bit more and then continue.
Hope that helps!
Alex and Allison,
Thanks very much for your shared wisdom and engaging options.. Off to my course development, learners look out.. you may just like this training! :-D
I generally avoid quizzes and pre-/post-tests (in the traditional sense) like the plague. However.... if you think about it completely differently, they can be quite impactful.
One of the most important things with adult learners is to let them apply what the already know. Many eLearning offerings follow a "tell then test" design, and that rarely allows adults to apply what they already know.
My team has found it to be very effective to present a scenario (quiz question really) and allow the learner to choose a course of action (multiple choice answer) without making it look like a quiz.
The feedback is then consequential, meaning it's not just a right/wrong comment, but something that shows the real life impact.
For example, if a customer complains about the elevator in the building not working, let the learner choose an icon that represents calling the onsite maintenance team, calling the elevator vendor for an emergency response, or putting up an out of order sign.
If they select an out of order sign, the feedback is hearing an irate call or reading a negative survey response from the customer, and others. If they select calling the vendor for an emergency response, an invoice shows up on screen showing the actual cost and the learner's name approving the high expense and a note from the vendor saying there was a crayon in the door track. If they select calling the onsite maintenance team, the feedback will show that the maintenance member found the crayon, removed it, and the elevator worked, therefore the customer was happy.
In that approach, the learner gets to apply knowledge without being forced to listen to or read information they already know, and they don't feel like they're being tested. They simply respond and move on.
I totally agree! That is an excellent example of how to use quizzes as a learning tool and not just for evaluation purposes. Thanks for sharing!
It depends. If the quiz serves as a benchmark or an adaptive engine then, yes, they are useful. But a quiz for the sake of a quiz could frustrate and bewilder learners...especially adults. What's the WIIFM for the quiz upfront? Adults will wonder if the quiz will serve a purpose and, if so, that should be stated in the directions. If it isn't tracked, then be explicit about that and reiterate the purpose of it since it is not for a grade.
For reflective learning, I would think questions with open-ended responses would be best...and maybe answers are stored to be reflected on at the conclusion of the course for comparison. "How do your original thoughts compare to what you think no that you have taken the course..." or something like that.
I love your idea to store their original responses to have them reflect upon them later on!
This discussion is closed. You can start a new discussion or contact Articulate Support.