Storyline 360 - Skipped Questions

Hello, friends. This is a very specific and strange issue. So far, we have very few clues as to how or why it happens, but I'll explain as much as I can.

We've identified, in 2 instances so far, issues where a question was "skipped" by Storyline 360 - the student wasn't given the opportunity answer it.

In one of the cases (in July), the number of required questions is 25 correct (100% pass). When the student finished the course, the result was a "Fail." The student was given the opportunity to review their missed questions, and indicated that the 1 question missed was never presented to them. Upon checking the data in our LMS, the results indicated that there were 24 correctly answered questions, and 0 incorrectly answered questions, with a "Fail" result. Again, there were supposed to be 25 total questions asked, but 1 question was never presented. This wasn't something we could replicate, and the course had previously been taken (without incident) several thousand times during the year.

The second case happened today - the same scenario. A course with a total of 8 questions was failed by a student who was only presented 7 questions. Like the first case, this is a high volume course that hadn't previously (to our knowledge) had any issues. In this instance, however, we received a little more information about how the issue may have happened. The slide before the question bank containing the question that was skipped had a 'Next' button. Apparently, the student clicked it once, and when nothing immediately happened (possibly due to latency), he clicked it once or twice more. When the question bank loaded, he said the question briefly appeared but "flashed" to the next question before he had an opportunity to answer it.

Based on that (little) information, this seems like it may be a "perfect storm" scenario wherein a student is clicking a 'Next' button multiple times  when there also happens to be latency with the network connection. I'm not sure that there's a solution, but I wanted to get this out there to see if there were any thoughts. If nothing else, perhaps this will provide a little help to anyone else that may come across the issue.

6 Replies
Katie Riggio

Greetings, Robert!

I appreciate you sharing this helpful context, and would love to help unlock both mysteries. Let's see if we can find a trend with the affected students!

Are they using the same browser and internet connection? 

Also, I agree with your thought that they lost their internet connection and may not have known it right away. Did you know Storyline lets you notify learners when they lose their internet connection? That way, they can re-establish their connection so they won't lose any progress and hopefully not run into the skipped question oddity. Here's how to enable this feature!

Robert Reynolds

Thanks for responding,  Katie.

Yes, both students in this case were in our company computer lab, using the same connection and the same browser/version (Chrome).

Question: It appears that, at least as of last year, the feature can only be enabled in the Flash output. Currently, we publish HTML5 with Flash fallback. Has this changed by chance?

Katie Riggio

Thanks for those extra clues, Robert.

If this happens again, we'd love to investigate further if you're comfortable with sharing the .story file. You can send it privately to us by using this link, and we'll delete the file after testing!

You're right: We're tracking requests for the same functionality for HTML5, so I'll add our conversation to the report so that we can update you on any new developments!