Strange pattern in group randomization

Apr 06, 2012

I have been using Randomize group feature for a while. I have recently counted the number of times each question from a group appeared in real quizzes and I observed a strange pattern: in all groups in all quizzes (over 100 in total), the first and last question in a group appear much less frequently than the rest of the questions in the group.

For instance, I have a group of 3 questions with 1 set to be included. The quiz was run 113 times. The middle question appeared in 59 of them, while the first and the last question in the group appeared in 27 each. As I said it is consistent across all groups an quizzes, regardless of the number of questions in the group and the number included.

Is there a flaw in randomization algorithm? Is there something I can do about it?

Thanks,
Arkady

3 Replies
Joseph Francis

I know this reply is quite late, but I only recently joined this forum.

As a Macromedia Authorware user for better than 15 years, I was very familiar with the Decision Icon (which is used by the course to determine what path to follow at that point). Two of the path options for the Decision Icon were Random and Random without Replacement.. Both used a standard randomization algorithm to determine the path, but the latter LOGGED what the randomly-selected path was, and did not allow that path to be selected again until ALL of the remaining paths had been selected at least once. It was useful if you needed the student to see every path, but you wanted to "mix up" the order of presentation. With the former option, the same path would and did often come up more than once when passing through the icon, with the explanation being that randomness indicated the same number might appear more than once.

With computers, most of the random-number generators used are really pseudorandom number generators, which means the numbers are generated in a predictable fashion, using a mathematical formula. You may have exposed a flaw in the "random" number generator that Quizmaker uses, but without access to the underlying source code, it's hard to determine if the algorithm is flawed, the application of the algorithm is flawed, or if it just so happened that the randomization had the middle question appearing in 59 out of 133 tries while the first and last questions appearing in 27 out of 113 tries each.

Arkady Maydanchik

Well, I can tell you for sure that it was NOT a random result. I have since analyzed hundreds of groups in my tests and consistently find that the first and last question in the group is shown about half as often as the other questions. It is quite statustically significant. So, it is either a flaw in the algorithm or a bug in the software implementation of the algorithm. The workaround I use whenever possible is to make groups with "select 1 question only" (so if I need 3 questions out of a pool of 15, I break them into 3 reasonably logical groups of 5. Then I duplicate one question and put it at both ends. If that is not acceptable, I simply put the questions that we find "less desirable" to the ends.

This discussion is closed. You can start a new discussion or contact Articulate Support.