Random draw from quiz bank
This is an oldie going back to Storyline 1... I recall posting about it then, but let me have another go at it.
The random draw from a quiz bank is not so random at all. This course presents the learner with ten quiz banks, one bank for each topic, with four questions per bank. Testing shows that on average two to three questions are repeated if the learner fails and takes the quiz over.
The likelihood of drawing the same question from a bank of four is 1 in 4 or 25%. Ten banks of four will display the same question as the first pass about 2 or 3 times in our tests. This is not random at all from the learner's point of view. There are forty questions in total, displaying 10 for a test, there is no good reason to display a previously drawn question when the quiz is repeated after failing.
Random draw from a bank could be easily improved. Tag a question that has been displayed and (in this example of four questions per bank), draw from 3, then 2, then 1, when you hit zero, reset them all and start over.