Wrong quiz scores in Moodle

One of our trainers noticed that the score on the Storyline Results Slide is often recorded incorrectly in Moodle’s Gradebook. The wrong grades were always lower than what the learner earned, sometimes a zero. (Fortunately, the quiz results have no effect on workers' performance appraisals.) 

Exporting the Moodle Interactions Report revealed that, when a learner reaches some seemingly random question, the system stops reporting to Moodle and reports none of the responses afterward. The spreadsheet cells for these responses contain “ ”. The Storyline SCORM package continues as normal when this happens, with the correct grade appearing in the Results Slide.

For one quiz 14% were recorded incorrectly. Another quiz, had 10% that were wrong.

I have spent many hours analyzing the problem.  I haven’t been able to identify any common factor that connects the learners who have incorrect scores.  I attached some data. Here are the relevant settings (I think):

Storyline Publish Settings:

  • SCORM 1.2
  • Report status to LMS as: Passed/Incomplete
  • Track using quiz result
  • Result slide to report: 1.15 Results Slide

Moodle SCORM Compatibility Settings (Version 2.7.7)

  • Force completed: Yes
  • Auto-continue: No

Moodle SCORM Attempts Management Settings

  • Number of attempts: Unlimited attempts (Note: Once Moodle stops recording grades, subsequent attempts are not recorded)
  • Attempts grading: Highest attempt
  • Force new attempts: No
  • Lock after final attempt:

I am completely stuck. Any suggestions will be welcome. Moodlers, I encourage you to check an Interactions Report and see if you have a similar problem.

Thanks!

12 Replies
Christie Pollick

Hi, John -- May I ask if your test scores are being calculated incorrectly when you test your published output in the SCORM Cloud? If all is well when testing in the SCORM Cloud, it may be a good idea to reach out to Moodle directly for additional assistance. I do know that we also have a fair number of Moodle users here in the community, so hopefully someone is able to chime in to assist you further. :)

John Black

I suppose that is the next step, Christie. The drawback is the problem only happens perhaps 10% to 15% of the time. That means we'll have to buy a subscription to SCORM Cloud and take the quiz ourselves dozens of times. Of course, that won't be a truly representational sample because it will be people in our department taking the quiz. If the results are negative, maybe it's Moodle causing the problem or maybe the learners are doing something we didn't anticipate.

Christie Pollick

Hi, John -- Thanks for your response and I did want to mention in case you weren't aware that you would be able to use a Free Trial account indefinitely on the SCORM Cloud, assuming that the zipped content you want to test does not exceed 100 MB. Please take a look at this link for further info. 

I would be happy to test your course a few times for you as well to see if I encounter the same behavior you have described. Please use the ADD ATTACHMENT button in the bottom left corner of the reply box and browse for your file from there. :)

John Black

The free trial only includes 10 users. That's not a big enough sample, since we only have the problem intermittently. I think I'll have to spend the $75 for a 50-person license.

I can't believe no one else has this problem. Our Moodle installation has no customizations and the Storyline quiz is completely vanilla.

Dan Marsden

I haven't seen the issues John reported on our systems with newer versions of Moodle and the latest Articulate updates applied. I have seen similar issues with older versions - particularly when users are on dodgy wifi/or the moodle server is overloaded. Newer versions of moodle will now notify the user if they are on slow/dodgy connections with a pop-up message.

David - make sure you're not having the issue related to masteryscore in your system - Moodle handles the masteryscore node in the articulate package the way that the Rustici cloud used to. We finally added a setting in Moodle 3.1 to let you override the way masteryscore works in Moodle and disable it. More info on that setting is here: https://tracker.moodle.org/browse/MDL-50620