Hi, Heroes! I had a conversation with a client about the number of attempts allowed in a quiz. My client wants the learner to try again (unlimited) until they get it right. Personally, I think if they get it wrong the first time, it is a better approach to provide the correct answer right away. What is the point of having them guess more times? Perhaps it makes them think more deeply...or do they just click until they get it right?
Please weigh in on what you think is the best practice. Thanks!
Will Thalheimer has collected some research on Feedback practices. I think that's what's at stake in this situation. Not necessarily the number of attempts.
And yes, I'm a developer and that accurately reflects my thought process for things that bore me or have no meaning. I will game a quiz or assessment if I don't see the value in it. Especially if it's less effort than providing "thoughtful" multiple choice decisions.
According to Will's research aggregate, it's better for learning to delay feedback. That means one attempt and not letting someone know they got it wrong until after a period of time, the longer the better. Feels unintuitive and uncomfortable but I trust the research
In the end, I think it depends. But I do think there's something to providing a challenge. We've seen similar behavior to mine in our user tests. Folks respond well to challenges but will power down if it's just a slog.
Steve, I appreciate your thoughts, and agree with you. What a great paper by Will Thalheimer!
Judith, thanks for chiming in. As to your first bullet, in a multiple choice, the learner would keep getting the same Try Again until they get it right - regardless of the number of attempts you allow other than 1. When they get it right, and get the Correct feedback, you can provide the feedback describing why it is correct. I don't know how we can set up another feedback layer if the question is set to give Feedback by Question.
If you want each of the answers to give a different feedback, you can select Feedback by Choice from the menu provided in the quiz question.
For your third bullet, are you referring to a Multiple Reponse type question? If so, I don't know of a way to provide feedback for every possible combination of responses. (Variables can no doubt be created for anything we are talking about here!)
You can vary feedback but you have to do it through custom triggers. One way to do this is by adding triggers for each combination you want to vary by looking at the states of each button (When States of [All of, Any of] X, Y, Z... Are). I modify a variable based on these conditions and display that variable in my feedback box %feedbackTxt%.
Take a look at the attached story. Couple of things to make this work. Add the %feedbackTxt% variable to your variable manager and to your try again and incorrect boxes. Then it's just a matter of setting up your triggers. Trigger order could matter for some configurations. Might take some experimentation.
Thanks so much, Jill. I'm going to try your suggestions. There's still so much to learn!
For multiple responses, I decided to keep the responses to 4 with 1 incorrect answer. That way, my "Incorrect" feedback can relate directly to the incorrect answer.
Steve, I'm going to look at your example and see if I can make some of my questions work more elegantly with variables.
Hi again. I tried to change my feedback selection to Feedback by Choice, but that option does not appear in the menu. Thinking that it may be locked in, I tried creating a new questions, but still can't see Feedback by Choice.
Hi, Judith. The Feedback by Choice is not an option with a Multiple Response question, but it is an option with the Multiple Choice question. WIth a Multiple Response question, there are numerous possibilities of how the learner might answer - so that is why Steve has the variables to vary feedback with triggers with and states.
I wish I had more time to learn and play with variables...they can do so much. -Jill
That makes sense -- Not sure I understand the difference beween Multiple Choice and Multiple Response, but I tried changing to Mulltiple Choice and was able to select Feedback by Choice.
I used triggers with conditionals to delay feedback for Drag and Drop quizzes where I wanted to give the user more than one try, but didn't want to give feedback until the end. Using the states Drop Correct and Drop Incorrect can sometimes make it too easy to get the answer right on the second try but I do want the user to see what they had wrong. By adding a new state called Wrong that has the object highlighted and then in the Incorrect Layer using a trigger to change the state to Wrong if the state was Drop Incorrect allows feedback at the end. This sounds a little confusing, but see the attached story.
16 Replies
Will Thalheimer has collected some research on Feedback practices. I think that's what's at stake in this situation. Not necessarily the number of attempts.
http://www.willatworklearning.com/2008/05/free-research-r.html
The link is broken to the report but you can find it here:
http://willthalheimer.typepad.com/files/providing_learners_with_feedback_part1_may2008.pdf
http://willthalheimer.typepad.com/files/providing_learners_with_feedback_part2_may2008.pdf
In my view, an unlimited number of attempts is a signal that indicates:
1) I can game this assessment. There are no consequences.
2) It's OK to be lazy and not think about how I answer because it's just a tedious exercise anyway. It doesn't mean anything and there's no challenge.
3) It might actually be less energy on my part to "guess" and eliminate until I win than put effort into thinking about it.
And yes, I'm a developer and that accurately reflects my thought process for things that bore me or have no meaning. I will game a quiz or assessment if I don't see the value in it. Especially if it's less effort than providing "thoughtful" multiple choice decisions.
Well crafted challenges, on the other hand....
Hi Jill,
In Storyline, sometimes I find it difficult to make this choice:
Currently, I give the learner one try, but include a Retry Quiz on the results page.
What do the experts do?
According to Will's research aggregate, it's better for learning to delay feedback. That means one attempt and not letting someone know they got it wrong until after a period of time, the longer the better. Feels unintuitive and uncomfortable but I trust the research
In the end, I think it depends. But I do think there's something to providing a challenge. We've seen similar behavior to mine in our user tests. Folks respond well to challenges but will power down if it's just a slog.
Steve, I appreciate your thoughts, and agree with you. What a great paper by Will Thalheimer!
Judith, thanks for chiming in. As to your first bullet, in a multiple choice, the learner would keep getting the same Try Again until they get it right - regardless of the number of attempts you allow other than 1. When they get it right, and get the Correct feedback, you can provide the feedback describing why it is correct. I don't know how we can set up another feedback layer if the question is set to give Feedback by Question.
If you want each of the answers to give a different feedback, you can select Feedback by Choice from the menu provided in the quiz question.
For your third bullet, are you referring to a Multiple Reponse type question? If so, I don't know of a way to provide feedback for every possible combination of responses. (Variables can no doubt be created for anything we are talking about here!)
Thanks!
You can vary feedback but you have to do it through custom triggers. One way to do this is by adding triggers for each combination you want to vary by looking at the states of each button (When States of [All of, Any of] X, Y, Z... Are). I modify a variable based on these conditions and display that variable in my feedback box %feedbackTxt%.
Oh, cool! Will you share your variable?
Take a look at the attached story. Couple of things to make this work. Add the %feedbackTxt% variable to your variable manager and to your try again and incorrect boxes. Then it's just a matter of setting up your triggers. Trigger order could matter for some configurations. Might take some experimentation.
Thanks so much, Jill. I'm going to try your suggestions. There's still so much to learn!
For multiple responses, I decided to keep the responses to 4 with 1 incorrect answer. That way, my "Incorrect" feedback can relate directly to the incorrect answer.
Steve, I'm going to look at your example and see if I can make some of my questions work more elegantly with variables.
Hi again. I tried to change my feedback selection to Feedback by Choice, but that option does not appear in the menu. Thinking that it may be locked in, I tried creating a new questions, but still can't see Feedback by Choice.
Any ideas on why this happens?
Hi, Judith. The Feedback by Choice is not an option with a Multiple Response question, but it is an option with the Multiple Choice question. WIth a Multiple Response question, there are numerous possibilities of how the learner might answer - so that is why Steve has the variables to vary feedback with triggers with and states.
I wish I had more time to learn and play with variables...they can do so much. -Jill
That makes sense -- Not sure I understand the difference beween Multiple Choice and Multiple Response, but I tried changing to Mulltiple Choice and was able to select Feedback by Choice.
Thanks!.
Multiple choice are radio buttons (select one) where multiple select are checkboxes (select many).
Of course! Thanks, Steve. Don't know why I didn't get that right away.
Aging fast, I guess!
Judith
I used triggers with conditionals to delay feedback for Drag and Drop quizzes where I wanted to give the user more than one try, but didn't want to give feedback until the end. Using the states Drop Correct and Drop Incorrect can sometimes make it too easy to get the answer right on the second try but I do want the user to see what they had wrong. By adding a new state called Wrong that has the object highlighted and then in the Incorrect Layer using a trigger to change the state to Wrong if the state was Drop Incorrect allows feedback at the end. This sounds a little confusing, but see the attached story.
Thanks for sharing, Kate. Good solution!
This discussion is closed. You can start a new discussion or contact Articulate Support.