Wrong quiz scores in Moodle
Dec 09, 2015
One of our trainers noticed that the score on the Storyline Results Slide is often recorded incorrectly in Moodle’s Gradebook. The wrong grades were always lower than what the learner earned, sometimes a zero. (Fortunately, the quiz results have no effect on workers' performance appraisals.)
Exporting the Moodle Interactions Report revealed that, when a learner reaches some seemingly random question, the system stops reporting to Moodle and reports none of the responses afterward. The spreadsheet cells for these responses contain “ ”. The Storyline SCORM package continues as normal when this happens, with the correct grade appearing in the Results Slide.
For one quiz 14% were recorded incorrectly. Another quiz, had 10% that were wrong.
I have spent many hours analyzing the problem. I haven’t been able to identify any common factor that connects the learners who have incorrect scores. I attached some data. Here are the relevant settings (I think):
Storyline Publish Settings:
- SCORM 1.2
- Report status to LMS as: Passed/Incomplete
- Track using quiz result
- Result slide to report: 1.15 Results Slide
Moodle SCORM Compatibility Settings (Version 2.7.7)
- Force completed: Yes
- Auto-continue: No
Moodle SCORM Attempts Management Settings
- Number of attempts: Unlimited attempts (Note: Once Moodle stops recording grades, subsequent attempts are not recorded)
- Attempts grading: Highest attempt
- Force new attempts: No
- Lock after final attempt:
I am completely stuck. Any suggestions will be welcome. Moodlers, I encourage you to check an Interactions Report and see if you have a similar problem.
Thanks!
12 Replies
Hi, John -- May I ask if your test scores are being calculated incorrectly when you test your published output in the SCORM Cloud? If all is well when testing in the SCORM Cloud, it may be a good idea to reach out to Moodle directly for additional assistance. I do know that we also have a fair number of Moodle users here in the community, so hopefully someone is able to chime in to assist you further. :)
I suppose that is the next step, Christie. The drawback is the problem only happens perhaps 10% to 15% of the time. That means we'll have to buy a subscription to SCORM Cloud and take the quiz ourselves dozens of times. Of course, that won't be a truly representational sample because it will be people in our department taking the quiz. If the results are negative, maybe it's Moodle causing the problem or maybe the learners are doing something we didn't anticipate.
Hi, John -- Thanks for your response and I did want to mention in case you weren't aware that you would be able to use a Free Trial account indefinitely on the SCORM Cloud, assuming that the zipped content you want to test does not exceed 100 MB. Please take a look at this link for further info.
I would be happy to test your course a few times for you as well to see if I encounter the same behavior you have described. Please use the ADD ATTACHMENT button in the bottom left corner of the reply box and browse for your file from there. :)
Hi again, John -- I did want to mention that if you should decide to use the Free SCORM Cloud Trial that there is a 'cap' of 10 registered users. Beyond that, you get an email telling you you’ve got too many registered and then you clear them out and keep going.
The free trial only includes 10 users. That's not a big enough sample, since we only have the problem intermittently. I think I'll have to spend the $75 for a 50-person license.
I can't believe no one else has this problem. Our Moodle installation has no customizations and the Storyline quiz is completely vanilla.
I am having the same issue and it is quite embarrassing as I have been informing people they have not passed a module, much to their surprise as they were shown they passed in the SCORM. Do we have a solution/cause of this?
Hi David,
I don't see that we have additional information from John here, and as he wasn't able to attempt it within SCORM Cloud I don't know that the behavior was replicated outside of Moodle. Are you able to attempt it there? Have you reached out to Moodle about it?
I never figured this out. We just work around it by observing scores live and correcting them in the system. Of course, that would not work for distance learning applications.
Ashley - I posted this on the Moodle forums back when I originally posted here. I didn't get any responses.
David - It would be good to know your version and settings.
Thanks for that update John - and sorry you didn't hear anything back from the Moodle side of things. :(
I haven't seen the issues John reported on our systems with newer versions of Moodle and the latest Articulate updates applied. I have seen similar issues with older versions - particularly when users are on dodgy wifi/or the moodle server is overloaded. Newer versions of moodle will now notify the user if they are on slow/dodgy connections with a pop-up message.
David - make sure you're not having the issue related to masteryscore in your system - Moodle handles the masteryscore node in the articulate package the way that the Rustici cloud used to. We finally added a setting in Moodle 3.1 to let you override the way masteryscore works in Moodle and disable it. More info on that setting is here: https://tracker.moodle.org/browse/MDL-50620
Thanks Dan for popping in here - always appreciate your Moodle expertise.
This discussion is closed. You can start a new discussion or contact Articulate Support.