Tracking Interaction Data (Quizzes) Leading to Problems

Normal 0 false false false ontGrowAutofit /> MicrosoftInternetExplorer4

Hi.

 

I'm having a problem with reporting quiz results.  I have a final exam module with about 65 questions.  In testing it, I get intermittent errors in grading results.  Sometimes Storeyline correctly reports that I've passed, but sometimes it incorrectly reports that I've failed (it marks correctly answered questions as being incorrect).  I reported this problem in the Storyline forum previously and was advised to test the module at SCORM Cloud, which I did.  It passed, so (per your advise) I began working with my LMS provided to find the problem.  They looked into it and said that it appeared the problem was associated with a lot of API calls coming to the LMS from the Storyline results slide interaction (around 600 API calls) and that it appeared that the browser connection with the LMS was hanging.  I did not have my individual questions configured to submit answers, and relied on all the submission happening when the results slide was invoked.  I changed that to "Submit Interaction" each time the user clicked "Next" to go to the next question.  The LMS people (LearnUpon) tested the module again and said that, although it appeared that the API calls were more orderly as far as sequencing through the answer submissions, it was still generating 600+ API calls and therefore was taking nearly 2 minutes to "save" when the exam module was closed.  He suggested that it might be a better configuration to "remove the interaction data being tracked" and that I'd still be able to score the exam without tracking the interaction data.

 

I don't know what it means to "remove the interaction data being tracked."  Can you give me some guidance on this?  Moreover, do you have any insight into what I've described that might give you a suggestion as to what I should do?

 

Thank you.

 

Rich

 

 

5 Replies
Justin Grenier

Good Afternoon, Richard.

If you are tracking completion using the results of a quiz, there is no way to stop quiz data from being sent to the LMS.  I would recommend enabling LMS Debug Mode to demonstrate for your LMS Vendor precisely what data Storyline is sending to them.  If your content is sending over correct data and the LMS is tripping over the volume of said data, I'd recommend asking them to consult with Rustici Software for help optimizing their data handling.  We don't often hear of LMSes that are unable to handle the volume of standard SCORM interaction data.

A possible workaround might be to eliminate your Result slide and track completion using the number of slides viewed, which would minimize the interaction data that is sent to your LMS.  Implicitly, this would also eliminate your ability to evaluate learners based on their responses to quiz questions.

Good luck with this, Richard!

Richard  Crouse

Hi Justin.  Thank you for your response.  I followed up with LearnUpon and they explained the issue in a bit more detail.  They clarified that it wasn’t that their LMS was “tripping over the volume of data.”  Instead, they said that the volume of data interaction appears to be resulting in the browser attempting to establish more connections than it can handle and it is this that may be leading to lost data to SCORM.  A second problem they found (maybe related) is that Storyline occasionally reports a zero score (and therefore failed) in results calculations when in fact the score should be non-zero.  The team there is being very helpful to try to find what is going on, and they are now trying to reach out to other Storyline users among their clients to see if they have had similar issues and what work-around they may have come up with.  In the meantime, they’ve suggested that I ask if there is some way to configure my final exam quiz so that it submits the question results to SCORM one at a time after each question is answered.  If so, this might help me discover a work-around.  I actually have each quiz question configured to submit interactions when the student clicks the “Next” button to advance to each successive question.  So it would seem that there should be some method in Storyline to use the already submitted answers at SCORM in the LMS when the results slide is executed, and therefore not need to resubmit all the answers at one time at that slide.  

So to net this out:  Does Storyline allow you to submit question interaction data one at a time on each "next" click button in a quiz rather than submitting all of the data at the very end of the quiz for all 60+ questions?  If so, how do I configure the "next" button triggers to work this way?  (Note: The purpose of this is to debug why a score of 0 and a status of failed is being passed back in some cases. While the course works fine on some browsers, it does not on others.)

Many thanks for your help.

Regards.

Rich  

Justin Grenier

Good Morning, Rich.

Thanks for clarifying LearnUpon's hypothesis that it is the browser (and not the LMS) that is having trouble with the volume of data interaction between your content and the LMS.  However, I think it's important to note that if this behavior is irreproducible in SCORM Cloud, the problem would seem to be LMS-specific.

Having said that, if you change the navigation controls to Submit instead of Prev/Next on each slide (the opposite of this screenshot), I would expect to see less LMS communication happening on the Result slide.  As a small example of this:

Good luck to you, and do let us know what you find!

Richard  Crouse

Hi Justin.  That might have fixed it.  I made the change to using the Submit button and sent the quiz module over to LearnUpon again.  They retested and said that it is clearly a significant improvement and the browser issue appears to have been cleared up.  I'm letting our students access it now and will let you know if any problems are still found.  With fingers crossed, I'm hopeful that I'm out of the woods.

Thank you for your help.

Rich