Prevent a question from be randomly selected twice from a question

Jun 25, 2013

Hello!

I want to randomly draw upon questions from a question bank. My bank consists of 5 questions. My module has 3 ‘blank’ question slides, whereby 1 question from the bank should be randomly selected to appear on each blank slide.

 The problem is this: when I publish the course, question “ABC”  is randomly drawn upon from the question bank more than once.

How do I signal to Storyline that, once a question has been randomly selected and presented to a user,  it cannot be drawn upon again.

?

31 Replies
Christine Hendrickson

Hi Alex,

In short, you cannot. If you have a small pool of questions to pull from, and you pull a small number randomly, you're definitely going to see questions repeat. However, if you have a larger pool to pull from, you're less likely to see the same question multiple times.

For example, if you have a question bank with 10 questions and you pull only 2, you may not see the same 2 questions more than once. However, if you have question bank of 5, like you're using, and pull 3, you're definitely going to see questions repeat. 

If you can, you might want to try adding some additional questions, pull less questions, or "lock" questions together. Locking would pretty much defeat the purpose of randomizing, though, especially with such a small amount of questions.

I hope this helps!

Phil Mayor

Christine Hendrickson said:

Hi Alex,

In short, you cannot. If you have a small pool of questions to pull from, and you pull a small number randomly, you're definitely going to see questions repeat. However, if you have a larger pool to pull from, you're less likely to see the same question multiple times.

For example, if you have a question bank with 10 questions and you pull only 2, you may not see the same 2 questions more than once. However, if you have question bank of 5, like you're using, and pull 3, you're definitely going to see questions repeat. 

If you can, you might want to try adding some additional questions, pull less questions, or "lock" questions together. Locking would pretty much defeat the purpose of randomizing, though, especially with such a small amount of questions.

I hope this helps!


Christine this sounds like a bug or if not it should be, Storyline should not pull the same question twice.

Christine Hendrickson

Hi Phil,

This isn't a bug, I'm afraid. I'm sure you've seen the other discussions on this :)

I do agree, however, that it would be better if the same question was not displayed again. Right now, however, I'm not aware of a way to place a restriction on a question bank question so that it doesn't show up more than once when they're pulled randomly. 

This is actually something I've submitted feedback on in the past, because I do believe that it's something that could be improved. 

If either of you are interested in sharing your feedback with our development team, I'd appreciate it!

Thanks!

Terry Bell

Well, this certainly sounds like a bug to me.  I can't say I've ever had an occasion where I WANTED the same question to be asked more than once.  

I don't think my boss is going to buy the "not a bug" claim. I will use your link to "share my feedback", and keep my fingers crossed that this doesn't come up in testing.

Pat Simon

Hi Alex and Terry,

I ran into the same issue and came across this message board hoping for an answer.  I figured out a way on my own, though it's a bit complicated.  I'll explain what I did, hoping this might help answer your question.  Please feel free to reach out to me with any other questions.

I'm making a Jeopardy-like game.  Each category has its own question bank.  I originally set it up so that each box links to its own slide that randomly pulls from its corresponding question bank.  But the problem I'm having is similar to Alex's.  Separate slides pulling from the same bank can result in repeating the same question.

From what I can tell, the only way to not have repeating questions is to have one slide for each category/question bank that randomly picks 4 questions from that bank.  Here is the game board below.  Pardon the subpar graphics.  They are placeholders at the moment.  

I changed the branching from each box having its own question slide to every box in a category go to the same question slide.  The problem now is when I click on a question, I have to answer all 4 questions in the category before I go back to the board.  I want to go back to the board after answering one question.  I tried setting the branching within the questions to all jump back to the game board slide.  This works for the first question in that category, but when I click on the next question in that category it takes me back to the first question and I have to click next to go to the next question.  I want Storyline to click next automatically.  (This is where it gets complicated)

To do that, I need each question tied to its own variable.  With variables, you can create If/Then statements that can trigger different actions when you return to the same slide.  For this instance, when you go to a question slide it first checks to see if the variable is true or false.  The initial state of the variable is true.  If it's true, nothing happens.  If it's false, it auto-jumps to the next question.  When you answer the question and click continue, the continue button toggles the variable to false so that when you return to the slide it will auto-jump to the next slide.  You need to repeat the following process for EVERY question.

To set up a variable, go to the response layers of the question and click the new trigger button and select adjust variable.  Click the "..." button to create a new variable.

Type in any name, select the True/False type, and set the initial value at true.

Now put in the settings seen above so that Storyline knows to change the variable to false when continue is clicked.

Next, make sure you place the variable trigger before the slide jump.  Copy and paste this in all other response layers.

Lastly, we need to have the question slide check for this variable's value.  Go to the base layer and add a new trigger to jump to next slide when timeline starts, then click "Show Conditions" and press the green plus button.

Then type in these conditions and click OK:

Tada!  You should now have a working randomly selected, non-repeatable Jeopardy-like quiz game.  Again, you need to set up a new variable for every question.  I know it's a pain in the buns but at least now you know how to have a little more control over customizing the functionality of your courses.  Let me know if you run into any snags.  

Helene Caura

Hi Pat

Thank you for this post. This exactly what I needed. I was forgetting the "jump to next slide" trigger and of course it was just blank. One thing you did not mention and I had to figure out: you need to select "all" in the question bank, not just one question out of all your questions. Otherwise it doesn't work.

I have also added a disabled state for my box that triggers the questions once they have all been answered (if variable 1 + 2 +3 + x = false) or if they score 20 points (their aim).

Cheers!

Alex Turcotte

Hi everyone.

Here's a follow-up question to Pat's solution:

I have a bank of 6 questions (A,B,C,D,E & F).

If the user has already seen "Question A", the "A" slide jumps to the "B" slide (which contains Question B)

If the user has already seen "Question B", the "B" slide jumps to the "C" slide (which contains Question C)

etc.

What trigger did you create for the *last* slide in your question bank?

 

Alex Turcotte

Hi Ashley,

My apologies if I’m being unclear!  I think I over-simplified my problem too much in my previous post. I’m re-opening this thread since there may be a problem with Pat’s proposed solution (although I may be misreading Pat’s proposed solution too!) …but the problem is this: I have an Exam for students. The Exam consists of 3 quiz questions that are randomly pulled from a bank of 12 possible questions (“A” to “L”).  

Here are the rules for the Exam:  

Rule 1 - The student is allowed to take the exam twice (at most).

Rule 2: the student is never allowed to be presented with a question s/he has seen on their first attempt.

Let’s say that on the first try, the student is presented with Questions “A” “I” & “L” (selected at random by Storyline). Let’s then suppose that the user fails. He/She is then allowed to re-take the Exam one last time.

Let’s then say that on the second attempt, the user is presented with “A”, “E” and “L”. The solution for “A”? Have Storyline re-direct a student to the next question ( “B” ) automatically. This works well for all questions …except for “L” (the last quiz page in the question bank). On the student’s second attempt, I can’t re-direct the student from “L” to the next question (since it doesn’t exist). How should I handle quiz "L"? Am I missing something when I read Pat’s solution?

Ashley Terwilliger-Pollard

Hi Alex,

No worries - it's a long solution and an involved one! So what happens if the user sees A and then B on the first attempt, on the second attempt they'll automatically jump to C? But then they'd only have one question vs. the two expected? It's definitely a custom design set up (I'm on the support side of the house) so I may defer to the community for other recommendations. Also, if you've got a working sample for everything except "L" can you share that here and that way we could play with what you've got set up? 

Matt Garratt

After days of tearing my hair out, I wish I never embarked on using the branching option in Storyline 2. There appears to be no way to guarantee questions will not be repeated. If the order is locked, Storyline takes the first question from the locked selection and plays it over and over. The only way to stop it is by not randomising the questions in the bank and put a different question at the top of each draw slide...to which I wonder why not just have the actual question slide rather than having banks at all?

This feature causes more frustration than value. I suggest removing this feature until it is functional. What quiz taker expects to reattempt a quiz question several times in one sitting?!

Ashley Terwilliger-Pollard

Hi Steven and Matt,

If you're revisiting the question bank slide and it's going to redraw them from the bank and there isn't a way to stop the questions the user has already seen from reappearing on a second attempt. There currently isnt a way to change this, and if you've locked a question to the top of the bank that one will reappear each time. 

Matt Garratt

Appreciate the quick response Ashley. I find that revisiting a question bank slide will show the same question that was shown on that slide originally (not a new question). If you create additional question bank slides drawing from the same bank of questions, there is a high chance that there will be repeat questions.

I have tried locking the order on the question bank slide (not locking a slide to top), but that shows the question at the top over and over. I'm open to suggestion if I'm using the question bank function incorrectly, but I've tried so many different things and read everything I can find on question banks in this and other forums and haven't been able to come up with a solution to the repeating question issue other than the workaround that defeats the purpose of question banks that I mentioned in the post prior. Appreciate any guidance you can provide.

The real mystery to me is how locking the slides anywhere other than the to the top works. It certainly hasn't behaved as I expected it would. I can't work out what the purpose of them are at all!

Ashley Terwilliger-Pollard

Hi Matt,

Are you only seeing this in the HTML5 output perhaps from Storyline 2? I know our team has investigated a few different issues where the HTML5 output (viewed in Mobile Safari or Chrome desktop for example) isn't showing new questions on a retry from the results slide. That is an element they're continuing to investigate. 

You mentioned additional question bank slides drawing from the same bank, and the repeat questions...that is expected. Each slide draw is being treated as a separate element so it's not able to read from the previous one to know that the user already saw questions 1 and 3 out of 5, and it could re-show question 3 and a new one of 4 at the second draw. 

If you've got a sample course here, sometimes it's easier to show that based on what you're working with so you can upload your project file here using the "add attachment" button and we can take a look. 

Matt Garratt

Cheers Ashley, I can certainly share a file but perhaps I need to define what I am trying to achieve better, and you can tell me if it is possible using banks or not. If you would still like a file after this I can definitely attach one.

Issues are being experienced when publishing to all formats (not just HTML5).

Challenge 1: I have a bank of 5 questions and I want students to attempt all of them in a random order with no questions repeated.

Challenge 2: Same bank of 5 questions and students attempting all 5 but this time I want Question 1 to appear first, then I want a random selection but Question 3 must appear after Question 2. (in this case an attempt order of Q1, Q4, Q5, Q2, Q3 is fine, as is Q1, Q2, Q3, Q5, Q4 etc).

If you could provide some guidance on how to achieve these two challenges I'd be very appreciative.

Matt Garratt

ahhh thank you Wendy for showing me how question banks are supposed to work.

I realise now where my issue lies.

My project quiz actually involves leaving the question bank slide (branching option in the quiz question that jumps to a different slide) before returning to the question bank.

If you leave the question bank slide and return you see the last question you answered and your response, with no submit option.

I have this set up as I wanted to design an adaptive test...one that sends the test taker to harder or easier questions depending on if you get the question prior correct or incorrect.

Am I correct in thinking that SL does not allow this? 

Ashley Terwilliger-Pollard

Hi Matt,

Thanks for sharing your set up and I'm glad Wendy was able to give you a sample file and walk you through it.

As far as returning to the question slide if you've already answered it, and the slides are set to "resume saved state" you'll return to it and not be able to answer again. If you were to use the "reset to initial state" slide property than you'd be able to retry the question on a revisit. Similarly you could include a retry button on your results slide and that'll automatically reset the slide elements.