Only Require Some Questions on Test Retake

Jul 31, 2012

A client group is requesting that we set our Assessments so that learners only have to retake questions that they answered incorrectly.  Can't see that this is possible in the software, but wanted to check the forum.  Anybody know how this might be done?

153 Replies
Jeanette Brooks

Hi Judy! I'm sorry, right now that's not possible in Quizmaker '09 - when you allow a retake, there isn't a way to filter out questions that were answered correctly the first time through. That would make a great feature request though! What you might propose to your client as an alternative is to allow multiple attempts on the individual questions themselves as the learner goes through the quiz.

Jeanette Brooks

Hi Andrew! Right now the Retry Quiz feature in Storyline works a lot like it does in Quizmaker - it basically refreshes the quiz and the learner is allowed to retake the entire thing. There's not really a feature that allows you to customize the retry such that it displays only the questions the user missed the first time through. Definitely worthy of a feature request

I know this isn't exactly what you're looking for, but one thing you can do in Storyline (which isn't currently possible in Quizmaker) is display the Retry Quiz button only to learners who fail. Here's a bit more about that

Jacqueline Hutchinson

I have a client with a similar request.

The course I'm working on was already completed before the client decided to "play" with the quiz, or I would have used a different tool. I need to ensure that there are three attempts for the quiz. That after the first attempt the learner sees a "retry" page, after the second failed attempt the learner sees a "try one more time" page and after the third attempt they want the course to close and the LMS to record the course as a fail and force the learner to restart the course.

They also wanted ONLY the incorrectly answered questions displayed in the second and third attempts. As I say, if I had known they wanted this I would have completed the course in another tool. I don't want to have to redevelop the course (a lot of hours) but I need to try to deal with this so any input or ideas would be much appreciated.

Jeanette Brooks

Hi Jacqueline (also Andrew, you might be interested in the following info too)

The not-so-good news is, unfortunately the items you mention aren't thing things that can be accomplished in Quizmaker '09.

The good news is, Articulate Storyline however, offers some new features that make it possible to at least fulfill the Retry Quiz requirements you mention. You could set up a simple number variable which counts how many times the learner has failed, and then adjust the content and options on the Fail screen accordingly.

I also played around with an idea for displaying only the incorrect questions on the retries, and discovered that it is possible in Storyline, but the down side is, it's somewhat labor-intensive because it involves creating a true/false variable and 3 triggers for every question slide. (The true/false variable monitors whether the learner got the question right on a previous attempt. The triggers cause Storyline to jump past the question and count it correct each time the learner retries the quiz.) In addition, if you want to include a Review Quiz option on your result slide, it adds more complexity (and more triggers/variables), because if the user clicks Review Quiz, you DON'T want the slides to automatically skip ahead for questions that were answered correctly. So you need to set up the logic that tells Storyline to NOT skip past correctly answered questions if the user is reviewing their quiz, rather than retrying.

Anyway, here's a published sample. There are 3 true/false questions, and the correct answer for each is true. I think it gives you the behavior you want: 3 tries total on the quiz, and on retries you only see questions that you previously answered incorrectly. For anyone who's interested and has Storyline, here's the source file if you'd like to take a look under the hood.

Jeanette Brooks

Oh cool, then maybe Storyline will give you what you need! Disclaimer: the triggers in that sample are multitudinous! It took me a little bit of time to noodle out the logic (remember, I am a course developer, definitely not a programmer, lol) and there might be better ways of accomplishing it with fewer triggers/variables. But at least it's a start, and it's kind of cool that at least we know that it can be done.

That said, it would be awesome if you had a minute to submit a feature request for the feature of displaying only the incorrect questions on retries. That would make a great enhancement to Storyline. Thanks! :)

Jeanette Brooks

Hi all. Jacqueline asked offline for a little more detail about the logic behind the Storyline sample I shared above. So I put together a fairly long-winded screencast which I hope sheds some light. Here it is:

https://jeanette.viewscreencasts.com/807d214c9c711d530054a13b12d510be

As I mention in the video, the behavior Jacqueline's looking for is quite complex, so there are quite a few variables & triggers used in this project. If you're a Storyline user, feel free to dig into the sample file I posted above in an earlier post if it helps to take a look under the hood.

R Jordan

Hi Jeanette

For a testprep I am looking for a solution which I think is might require a similar setupas in this example.

I wouldlike to offer the learner the simple possibility to mark (and if possible to un-mark after review) a question so that he can review these questions latter.

At the moment I have on the slide a multiple choice quiz for the questions

Any help would be appreciated

Thanks

Jeanette Brooks

Can you say a little more about the functionality you're wanting?

  • When do you want learners to be able to flag the questions for review? Before they answer the question? Or after? 
  • Do you only need them to be able to flag questions which they got wrong? Or do you want them to be able to flag any questions, regardless of whether they got the right or wrong answer?
  • When you say you want them to review the questions they've flagged, what specifically do you want them to be able to do when they review? Just see the question (with their original answer and the appropriate feedback)? 
R Jordan

Hi Jeanette,

In response to your questions:

  • I want them to be able to do it before they answer the question.  However, after is also acceptable if there is no alternative.
  • I want them to be able to flag any questions, right or wrong because if they have guessed and got it right they might still want to flag it for review.
  • I would like them to just be able to see the original question in the original state without answer and feedback

I also have another question.  There are 400 questions in my multiple choice quiz. I want to be able to use this on the ipad and I was wondering if these flagged questions will be lost when the player app is closed.  If so, is there a way to save the flagged questions.

I really appreciate your help and advice.  Thanks

R Jordan

Hi Jeanette

I would like them to retake the question.  The multiple choice quiz that I am preparing is to help study for an aviation ground school theory test.   The actual test itself is only 40 questions.  These questions, however, are randomly chosen from the 400 questions that I am preparing my multiple choice quiz with.  It is a lot of material for a learner to cover and that is why it is important for learners to be able to flag the questions they don't know or didn't get right in order to focus on the material they have trouble with and not on the material that they were able to answer the questions for.  If they could retake the question that would be best.

Regards

Jeanette Brooks

Hi again R.  I asked David Fair of our support team to help out with this one, as I've not been able to create the kind of behavior you're looking for. Unfortunately his conclusion was the same... the ability to retry only the missed questions on subsequent quiz attempts is something you could do with "regular" quiz questions, but it's not possible to do it with a question bank where you draw just a subset of questions. The reason being, each retry of the quiz is going to display a different set of questions when the learner goes back a subsequent time.

However, you might be interested in taking a look at the attached file which David came up with. It comes close to the behavior you want, by providing a "flagging" feature and the ability to retry the missed and flagged questions...the only shortfall is, the questions aren't part of a question bank (they're just regular questions within the quiz).

Jeanette Brooks

Hi Sudheer, there isn't a built-in feature for doing what you have in mind, so you would need to create the logic manually, with triggers. Can you post your file (or a shortened example version of it) and describe a little more about what you have in mind? For example, do you want to REQUIRE learners to review the incorrect questions, or just give them the option? Also, do you want to prevent the retake button from appearing until after they complete their review? Also, is it important that the user only retake the incorrect questions when they do the retake? Or is it fine to have them take the whole quiz over again?

SUDHEER KUMAR datla

Hi Jeanette,

Thanks for the quick response, Here is an example. In this there are two slides about an article on  lion and an elephant, After those two slides user will go through the quiz and When user fails in the quiz he/she should retake the quiz but only the incorrect  answers along with the slide which belongs to them.

For Example:

If the user answered incorrectly about lions, the retry quiz should show the slide belonging to lions and also incorrect questions.

I am able to do the incorrect questions part but cant jump to the specific slide.

So wondering if you could help me. Thank You

Jeanette Brooks

Hi Sudheer - to be honest, there's not a seamless way to do this, but here's how I would handle something like that:

  • Create a true/false variable for each question to monitor whether the question has been answered correctly. You'll also use this to determine whether a question can be skipped if the user wants to retry the quiz but previously got some of the questions right.
  • On the result slide, provide a list of the questions and indicate which were answered correctly and which were not. You can do this by using the true/false variables as conditions.
  • On the incorrectly answered questions, provide a button that allows learners to review the related content. Add a trigger to the button which lightboxes the appropriate slide.

Check out the simple version of this approach in the attached file.

Jeanette Brooks

Hi Sudheer,

In that case, one way to handle it would be to do this:

  • Modify the Retry Quiz button to redirect learners to the first content slide (rather than to the first quiz question).
  • On each content slide, create a conditional trigger to "Jump to next slide when the timeline starts" if ALL of the quiz questions associated with that slide have been answered correctly. (This will be easy to do once you set up a true/false variable for each question to monitor whether the question was answered correctly.) 

A revised sample file is attached.

Max Perry

Hi Jeanette,

The responses and examples you have given above are very helpful and I have used the same variable and triggers system in my course to retake questions answered incorrectly. However, I am now trying to use the same system with a course which uses a question bank that randomly draws 4 questions out of 7. Therefore, when the retry button is hit the course randomly selects a different 4 questions from the bank, which usually results in the user taking new questions not just the ones answered incorrectly.

I was wondering if the exact same selection of questions can be drawn from the question bank on a retry? which in conjunction with the triggers would result in only the questions answered incorrectly being displayed.

I will be very grateful for any help you can provide.

Many thanks,

Max

This discussion is closed. You can start a new discussion or contact Articulate Support.