How many attempts to allow learner to answer correctly?

Jill Freeman

170 posts

Posted Wednesday, January 16, 2013 at 11:53 AM  

Hi, Heroes! I had a conversation with a client about the number of attempts allowed in a quiz. My client wants the learner to try again (unlimited) until they get it right. Personally, I think if they get it wrong the first time, it is a better approach to provide the correct answer right away.  What is the point of having them guess more times? Perhaps it makes them think more deeply...or do they just click until they get it right?

 

Please weigh in on what you think is the best practice. Thanks!

 


This post has 0 verified answers

All Replies

User Rank Steve Flowers

3,515 posts

Posted Wednesday, January 16, 2013 at 12:31 PM  

Will Thalheimer has collected some research on Feedback practices. I think that's what's at stake in this situation. Not necessarily the number of attempts. 

 

http://www.willatworklearning.com/2008/05/free-research-r.html

 

The link is broken to the report but you can find it here:

http://willthalheimer.typepad.com/files/providing_learners_with_feedback_part1_may2008.pdf

http://willthalheimer.typepad.com/files/providing_learners_with_feedback_part2_may2008.pdf

 

In my view, an unlimited number of attempts is a signal that indicates:

 

1) I can game this assessment. There are no consequences.

2) It's OK to be lazy and not think about how I answer because it's just a tedious exercise anyway. It doesn't mean anything and there's no challenge.

3) It might actually be less energy on my part to "guess" and eliminate until I win than put effort into thinking about it.


User Rank Steve Flowers

3,515 posts

Posted Wednesday, January 16, 2013 at 12:33 PM  

And yes, I'm a developer and that accurately reflects my thought process for things that bore me or have no meaning. I will game a quiz or assessment if I don't see the value in it. Especially if it's less effort than providing "thoughtful" multiple choice decisions.

 

Well crafted challenges, on the other hand.... 

 


Posted Wednesday, January 16, 2013 at 1:05 PM  

Hi Jill,

 

In Storyline, sometimes I find it difficult to make this choice:

  • Allow the learner to try twice, and have a Try Again and an Incorrect feedback (how would you create additional feedback layers for more tries?)
  • Give the learner only one chance, and respond with question-specific feedback.
  • Ask multiple choice questions, but have to use generic feedback  because Storyline is not set up to reply to different choice combinations.

Currently, I give the learner one try, but include a Retry Quiz on the results page.

 

What do the experts do?

 


User Rank Steve Flowers

3,515 posts

Posted Wednesday, January 16, 2013 at 1:13 PM  

According to Will's research aggregate, it's better for learning to delay feedback. That means one attempt and not letting someone know they got it wrong until after a period of time, the longer the better. Feels unintuitive and uncomfortable but I trust the research

 

In the end, I think it depends. But I do think there's something to providing a challenge. We've seen similar behavior to mine in our user tests. Folks respond well to challenges but will power down if it's just a slog.


Jill Freeman

170 posts

Posted Wednesday, January 16, 2013 at 3:28 PM  

Steve, I appreciate your thoughts, and agree with you. What a great paper by Will Thalheimer!  

 

Judith, thanks for chiming in. As to your first bullet, in a multiple choice, the learner would keep getting the same Try Again until they get it right  - regardless of the number of attempts you allow other than 1. When they get it right, and get the Correct feedback, you can provide the feedback describing why it is correct. I don't know how we can set up another feedback layer if the question is set to give Feedback by Question.

 

If you want each of the answers to give a different feedback, you can select Feedback by Choice from the menu provided in the quiz question.

 

For your third bullet, are you referring to a Multiple Reponse type question? If so, I don't know of a way to provide feedback for every possible combination of responses. (Variables can no doubt be created for anything we are talking about here!)

 

Thanks!


User Rank Steve Flowers

3,515 posts

Posted Wednesday, January 16, 2013 at 3:32 PM  

You can vary feedback but you have to do it through custom triggers. One way to do this is by adding triggers for each combination you want to vary by looking at the states of each button (When States of [All of, Any of] X, Y, Z... Are). I modify a variable based on these conditions and display that variable in my feedback box %feedbackTxt%.


User Rank Steve Flowers

3,515 posts

Posted Wednesday, January 16, 2013 at 5:45 PM  

Take a look at the attached story. Couple of things to make this work. Add the %feedbackTxt% variable to your variable manager and to your try again and incorrect boxes. Then it's just a matter of setting up your triggers. Trigger order could matter for some configurations. Might take some experimentation.


customColorFeedback.story

Posted Thursday, January 17, 2013 at 7:18 AM  

Thanks so much, Jill. I'm going to try your suggestions. There's still so much to learn!

For multiple responses, I decided to keep the responses to 4 with 1 incorrect answer. That way, my "Incorrect" feedback can relate directly to the incorrect answer.

 

Steve, I'm going to look at your example and see if I can make some of my questions work more elegantly with variables.

 


Posted Thursday, January 17, 2013 at 8:18 AM  

Hi again. I tried to change my feedback selection to Feedback by Choice, but that option does not appear in the menu. Thinking that it may be locked in, I tried creating a new questions, but still can't see Feedback by Choice.

Any ideas on why this happens?


Jill Freeman

170 posts

Posted Thursday, January 17, 2013 at 10:30 AM  

Hi, Judith. The Feedback by Choice is not an option with a Multiple Response question, but it is an option with the Multiple Choice question. WIth a Multiple Response question, there are numerous possibilities of how the learner might answer - so that is why Steve has the variables to vary feedback with triggers with and states.

 

I wish I had more time to learn and play with variables...they can do so much. -Jill


Posted Thursday, January 17, 2013 at 11:49 AM  

That makes sense -- Not sure I understand the difference beween Multiple Choice and Multiple Response, but I tried changing to Mulltiple Choice and was able to select Feedback by Choice.

 

Thanks!.


User Rank Steve Flowers

3,515 posts

Posted Thursday, January 17, 2013 at 12:02 PM  

Multiple choice are radio buttons (select one) where multiple select are checkboxes (select many). 


Posted Friday, January 18, 2013 at 6:51 AM  

Of course! Thanks, Steve. Don't know why I didn't get that right away.

Aging fast, I guess!

Judith


Posted Thursday, February 07, 2013 at 12:32 PM  

I used triggers with conditionals to delay feedback for Drag and Drop quizzes where I wanted to give the user more than one try, but didn't want to give feedback until the end.  Using the states Drop Correct and Drop Incorrect can sometimes make it too easy to get the answer right on the second try but I do want the user to see what they had wrong.  By adding a new state called Wrong that has the object highlighted and then in the Incorrect Layer using a trigger to change the state to Wrong if the state was Drop Incorrect allows feedback at the end.  This sounds a little confusing, but see the attached story.

 


DragAndDropDelayedFeedback.story