Random draw from quiz bank

This is an oldie going back to Storyline 1... I recall posting about it then, but let me have another go at it.

The random draw from a quiz bank is not so random at all. This course presents the learner with ten quiz banks, one bank for each topic, with four questions per bank.  Testing shows that on average two to three questions are repeated if the learner fails and takes the quiz over.

The likelihood of drawing the same question from a bank of four is 1 in 4 or 25%. Ten banks of four will display the same question as the first pass about 2 or 3 times in our tests. This is not random at all from the learner's point of view. There are forty questions in total, displaying 10 for a test, there is no good reason to display a previously drawn question when the quiz is repeated after failing.

Random draw from a bank could be easily improved.  Tag a question that has been displayed and (in this example of four questions per bank), draw from 3, then 2, then 1, when you hit zero, reset them all and start over.

Sam

16 Replies
Tracy Parish

The probability that question 1 will come up every time from 4 random questions is 25%.  That's really good odds.  I would bet on those odds.

If you want to "randomly" force the draw then you might want to set up the questions into 4 banks of 10 questions.  Have them answer the first set.  Pass - done course, fail - trigger set to move them to the next bank of 10 questions.  Pass - done.  Fail....move them to the 3rd bank of questions.  

It could be done.  It's not really random, but may appear so to a learner.  Also would not be very random learner to learner.  Each learner would get the same set of first bank of 10 questions, even if in random order.

Sam Carter

Hi Tracy,

Thanks for your feedback.  I thought taking your suggestion to the extreme (1 bank) would be easier to understand.  Here goes.

Let's put all 40 questions in a single bank and serve up 10 randomly.  When a learner re-takes the quiz, the best implementation would randomly draw from the remaining 30, but SL will draw from the complete set of 40. The result is that there is a 25% chance of drawing a previously drawn quiz question on each draw, delivering 2.5 questions (on average) that have previously displayed.

This suggests reducing the number of banks isn't a workaround... it has no net improvement.

Tech andLearn.com

Thank you for the reply. I'm not really concerned about the same slide draw
when revisted, but, rather overall. For example, on the first attempt, I
have had several instances where, out of 20 questions, I get two draws of
the same question. So, in other words, if I ask for 10 slides drawn from a
20 question set, I may get two of the same questions (even in a row).

I suspect the software draws 10 at random from the 20 at the beginning of
the quiz but doesn't check the sample to make sure it hasn't drawn two of
the same question. This should be fairly easy to do as I've done it when
creating javascript-based question banks online. It would be great to have
this feature as there isn't really a time I can think of where anyone
should see duplicate questions in a first quiz draw.

For example, if the system pulls 10 at random from an array of 20
[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19]
>>>>[0,3,5,7,9,10,13,15,18,20] I would hope that I wouldn't see
[0,3,5,7,9,10,10,10,18,20] but according to the rules of the system, it
seems this can happen even though this is truly random.

Thanks,

Tom

Tech andLearn.com

Leslie, thanks for your help. I will get one to you tonight or tomorrow
morning. It is unremarkable - simply quiz draw with the randomize questions
box checked. I have only experienced the problem twice out of the many
times I have used it so it may not show up but once in a while as would be
expected with randomization. I got a call from my client about it the first
time it happened.

I will appreciate your help to see if there is something obvious that is
happening. I will be able to get that tonight or tomorrow morning for you.

Thank you,

Tom

Tech andLearn.com

Hello Leslie,

I was able to get some time this morning to do this. I have attached the
quiz. The bank has 17 questions in it.

I have not been able to replicate the issue again so far but did once after
the person sent me a concern. It was very apparent as it was a repeat of
question 2 in the bank (for the person reporting it and I believe he was
sure - very detail oriented). I got it to happen as a repeat of one of the
last 4 slides in the question bank which are all the same images, but
different questions. So the same question came up twice in a row.

I have not been able to replicate this again yet but will be watching
closely as I test.

I would welcome any insight you have on the setup.

Tom