Off the bat - I think multiple choice and True/False are over-used and abused. I think it's a quick and easy way to go when you don't want to craft a more engaging or unique question. Not that MC and TF are bad question types, not at all. I just think it's important to spice it up with a variety of quiz types.
That being said, I'm a big believer in trying to build "scenario-based" quizzing as much as possible. It ends up really just being a simple multiple choice - your scenario set up/decision that needs to be made is the Question, and your multiple options are your different courses of action. I know I just said there are too much MC questions out there, but it doesn't FEEL like a multiple choice question when framed in a scenario-based context.
I've never really thought about the "too many" questions although I do think it's possible to exhaust your learner, and I think regardless of the amount of questions you have, you should always let your learners know up front before they start the quiz. This way they go in knowing whether its 30 questions or 3. I think it helps set them up for success and prevents guessing and wondering "when will this quiz be over!?"
I hope we hear from other community members on this topic!! It's a good one!
When I saw your question, two big Don'ts immediately popped up in my head: trick questions and unclear answers (I mean not clearly right or wrong). I understand that the latter also depends on the course itself - if you're trying to get your learner to think and reflect on a topic, it might help to give them controversial answers. But my personal preference is to have it clearly separated and not vague at all. Even worse is when none of the answers is right - then the learner feels stuck and frustrated. To sum it up in a positive Do: Please research some good answer possibilities! :)
There is a lot of research into evaluating learners. I designed a test that was accepted by the provincial government for evaluating a first aid course. Designing a good test that properly evaluates learners takes time and field testing to make sure it is accurate.
Here are a few of my thoughts:
As a rule of thumb I try and get one question for each learning objective.
Each question should have a correct answer and distractors that are incorrect.
Do not ask questions that were not covered in the course.
Do not ask negative questions. "Which one of these should you not do?"
Do not ask joke questions.
Often I will only use one correct and answer and two distractors (wrong answers). If is very hard to write three or more plausible distractors. Often people get one or two good distractors and the rest are obviously wrong. What point is there in wasting your time and the the learners time in writing obviously wrong distractors?
If you are asking a question with numbers as answers the numbers should be in order.
If you are asking a question where the answer is a range (1-3) then make sure the numbers don't overlap.
Since my e-learning courses are short. I usually write between 5 and 15 questions and I don't use a question bank.
Hi Cary, thanks for sharing! I especially like what you said about distractors -- why bother wasting your time and the learner's time with answers that are obviously wrong? That is a really great point.
I agree that with distractor responses, there's no point in coming up with answers that are definitely wrong. It's a much better strategy to come up with 1-2 answers that look like they could be correct, but aren't quite. This will make the learners slow down and think critically about their responses--I've noticed that approach with standardized testing like the AP tests or the SATs.
Also: co-sign on having at least one question per learning objective. I usually do 2 per LO, but I also like to present scenarios that weren't explicitly covered in the material, to see if they can apply the concepts we were trying to teach.
They say a good exam teaches while it tests; I think you can definitely use this idea when you create quiz questions.
When it comes to the design point of view, we have been making our quiz slides manually instead of choosing for the standard templates.
Simply because I am more in control that way. For example a quiz slide with multiple "fill in the blanks", where fields can hold the same correct question.
I.E.
Articulate StoryLine has two versions. Version ____ and _____
"1" and "2" being the correct answers can be typed in either input field and the user will get feedback whether it has 1, both or none correct. When the user gets only 1 out of 2 right, it will say which one that is.
Further i'm using the new build in slider to select options now, giving it a more interactive look and feel.
I like the question banks, making it a lot easier to manage all the questions, however not being able to edit the feedback layers on the quiz " review" is unfortunate.
To answer your questions on the amount of quizzes that should be presented, when i get briefed by our interaction designers we normally use about 8-10 quizzes throughout the course. A course is a bout 50 slides.
ps. As i'm downloading the latest articulate update(3), I had some time to kill, hence this post :)
Hey guys! Just thought I'd pop back in here and let you know that I posted an article inspired by this discussion. Check it out and let me know what you think.
The point in occasional detractor answers that are obviously wrong is that it helps keep a low achiever from being completely demoralized. They have the sense that they at least can say "I know #4 is wrong. I've learned something."
15 Replies
I love this topic Allison.
Off the bat - I think multiple choice and True/False are over-used and abused. I think it's a quick and easy way to go when you don't want to craft a more engaging or unique question. Not that MC and TF are bad question types, not at all. I just think it's important to spice it up with a variety of quiz types.
That being said, I'm a big believer in trying to build "scenario-based" quizzing as much as possible. It ends up really just being a simple multiple choice - your scenario set up/decision that needs to be made is the Question, and your multiple options are your different courses of action. I know I just said there are too much MC questions out there, but it doesn't FEEL like a multiple choice question when framed in a scenario-based context.
I've never really thought about the "too many" questions although I do think it's possible to exhaust your learner, and I think regardless of the amount of questions you have, you should always let your learners know up front before they start the quiz. This way they go in knowing whether its 30 questions or 3. I think it helps set them up for success and prevents guessing and wondering "when will this quiz be over!?"
I hope we hear from other community members on this topic!! It's a good one!
When I saw your question, two big Don'ts immediately popped up in my head: trick questions and unclear answers (I mean not clearly right or wrong). I understand that the latter also depends on the course itself - if you're trying to get your learner to think and reflect on a topic, it might help to give them controversial answers. But my personal preference is to have it clearly separated and not vague at all. Even worse is when none of the answers is right - then the learner feels stuck and frustrated. To sum it up in a positive Do: Please research some good answer possibilities! :)
There is a lot of research into evaluating learners. I designed a test that was accepted by the provincial government for evaluating a first aid course. Designing a good test that properly evaluates learners takes time and field testing to make sure it is accurate.
Here are a few of my thoughts:
Hi Cary, thanks for sharing! I especially like what you said about distractors -- why bother wasting your time and the learner's time with answers that are obviously wrong? That is a really great point.
I agree that with distractor responses, there's no point in coming up with answers that are definitely wrong. It's a much better strategy to come up with 1-2 answers that look like they could be correct, but aren't quite. This will make the learners slow down and think critically about their responses--I've noticed that approach with standardized testing like the AP tests or the SATs.
Also: co-sign on having at least one question per learning objective. I usually do 2 per LO, but I also like to present scenarios that weren't explicitly covered in the material, to see if they can apply the concepts we were trying to teach.
They say a good exam teaches while it tests; I think you can definitely use this idea when you create quiz questions.
I love that! Thanks for sharing :)
Hey Allison
When it comes to the design point of view, we have been making our quiz slides manually instead of choosing for the standard templates.
Simply because I am more in control that way. For example a quiz slide with multiple "fill in the blanks", where fields can hold the same correct question.
I.E.
Articulate StoryLine has two versions. Version ____ and _____
"1" and "2" being the correct answers can be typed in either input field and the user will get feedback whether it has 1, both or none correct. When the user gets only 1 out of 2 right, it will say which one that is.
Further i'm using the new build in slider to select options now, giving it a more interactive look and feel.
I like the question banks, making it a lot easier to manage all the questions, however not being able to edit the feedback layers on the quiz " review" is unfortunate.
To answer your questions on the amount of quizzes that should be presented, when i get briefed by our interaction designers we normally use about 8-10 quizzes throughout the course. A course is a bout 50 slides.
ps. As i'm downloading the latest articulate update(3), I had some time to kill, hence this post :)
Have a lovely day,
Bob
Wow that sounds really cool! When you're done building it, it would be great to see what it looks like if you don't mind sharing.
I'm not sure what you mean about not being able to edit the review feedback layers. You can edit the post-review quiz layer just like any other layer:
Hope that helps! :)
Allison
Hey Allison, i'll give you a private message once it's done.
Refer to this topic for the editing part. It's mainly the non-removable green and red bars.
https://community.articulate.com/discussions/articulate-storyline/how-to-hide-green-red-feedback-bar-at-the-bottom-of-quiz-while-review
How do i send private messages?
Hi Bob!
Did you see Steve Flowers' answer on how to do it?
Unfortunately, it is no longer possible to send private messages on E-learning Heroes. Is it something you can't share publicly?
Thanks,
Allison
Won't be able to share it in public i'm afraid, and we can't put bars on top as we use manual navigation at the bottom of the interface.
A request would be for this feedback layer to have more customization options.
Oh ok, too bad! I understand.
If you want, you can go ahead and submit a feature request.
:)
Hey guys! Just thought I'd pop back in here and let you know that I posted an article inspired by this discussion. Check it out and let me know what you think.
The point in occasional detractor answers that are obviously wrong is that it helps keep a low achiever from being completely demoralized. They have the sense that they at least can say "I know #4 is wrong. I've learned something."
This discussion is closed. You can start a new discussion or contact Articulate Support.