How many attempts to allow learner to answer correctly?

Jan 16, 2013

Hi, Heroes! I had a conversation with a client about the number of attempts allowed in a quiz. My client wants the learner to try again (unlimited) until they get it right. Personally, I think if they get it wrong the first time, it is a better approach to provide the correct answer right away.  What is the point of having them guess more times? Perhaps it makes them think more deeply...or do they just click until they get it right?

Please weigh in on what you think is the best practice. Thanks!

16 Replies
Steve Flowers

Will Thalheimer has collected some research on Feedback practices. I think that's what's at stake in this situation. Not necessarily the number of attempts. 

http://www.willatworklearning.com/2008/05/free-research-r.html

The link is broken to the report but you can find it here:

http://willthalheimer.typepad.com/files/providing_learners_with_feedback_part1_may2008.pdf

http://willthalheimer.typepad.com/files/providing_learners_with_feedback_part2_may2008.pdf

In my view, an unlimited number of attempts is a signal that indicates:

1) I can game this assessment. There are no consequences.

2) It's OK to be lazy and not think about how I answer because it's just a tedious exercise anyway. It doesn't mean anything and there's no challenge.

3) It might actually be less energy on my part to "guess" and eliminate until I win than put effort into thinking about it.

Judith Blackbourn

Hi Jill,

In Storyline, sometimes I find it difficult to make this choice:

  • Allow the learner to try twice, and have a Try Again and an Incorrect feedback (how would you create additional feedback layers for more tries?)
  • Give the learner only one chance, and respond with question-specific feedback.
  • Ask multiple choice questions, but have to use generic feedback  because Storyline is not set up to reply to different choice combinations.

Currently, I give the learner one try, but include a Retry Quiz on the results page.

What do the experts do?

Steve Flowers

According to Will's research aggregate, it's better for learning to delay feedback. That means one attempt and not letting someone know they got it wrong until after a period of time, the longer the better. Feels unintuitive and uncomfortable but I trust the research

In the end, I think it depends. But I do think there's something to providing a challenge. We've seen similar behavior to mine in our user tests. Folks respond well to challenges but will power down if it's just a slog.

Jill Freeman

Steve, I appreciate your thoughts, and agree with you. What a great paper by Will Thalheimer!  

Judith, thanks for chiming in. As to your first bullet, in a multiple choice, the learner would keep getting the same Try Again until they get it right  - regardless of the number of attempts you allow other than 1. When they get it right, and get the Correct feedback, you can provide the feedback describing why it is correct. I don't know how we can set up another feedback layer if the question is set to give Feedback by Question.

If you want each of the answers to give a different feedback, you can select Feedback by Choice from the menu provided in the quiz question.

For your third bullet, are you referring to a Multiple Reponse type question? If so, I don't know of a way to provide feedback for every possible combination of responses. (Variables can no doubt be created for anything we are talking about here!)

Thanks!

Steve Flowers

You can vary feedback but you have to do it through custom triggers. One way to do this is by adding triggers for each combination you want to vary by looking at the states of each button (When States of [All of, Any of] X, Y, Z... Are). I modify a variable based on these conditions and display that variable in my feedback box %feedbackTxt%.

Steve Flowers

Take a look at the attached story. Couple of things to make this work. Add the %feedbackTxt% variable to your variable manager and to your try again and incorrect boxes. Then it's just a matter of setting up your triggers. Trigger order could matter for some configurations. Might take some experimentation.

Judith Blackbourn

Thanks so much, Jill. I'm going to try your suggestions. There's still so much to learn!

For multiple responses, I decided to keep the responses to 4 with 1 incorrect answer. That way, my "Incorrect" feedback can relate directly to the incorrect answer.

Steve, I'm going to look at your example and see if I can make some of my questions work more elegantly with variables.

Jill Freeman

Hi, Judith. The Feedback by Choice is not an option with a Multiple Response question, but it is an option with the Multiple Choice question. WIth a Multiple Response question, there are numerous possibilities of how the learner might answer - so that is why Steve has the variables to vary feedback with triggers with and states.

I wish I had more time to learn and play with variables...they can do so much. -Jill

Kate Thompson

I used triggers with conditionals to delay feedback for Drag and Drop quizzes where I wanted to give the user more than one try, but didn't want to give feedback until the end.  Using the states Drop Correct and Drop Incorrect can sometimes make it too easy to get the answer right on the second try but I do want the user to see what they had wrong.  By adding a new state called Wrong that has the object highlighted and then in the Incorrect Layer using a trigger to change the state to Wrong if the state was Drop Incorrect allows feedback at the end.  This sounds a little confusing, but see the attached story.

This discussion is closed. You can start a new discussion or contact Articulate Support.