Issue with SCORM randomly stopping communicating with LMS (Moodle)

Jan 27, 2022

We're having an issue where the SCORM quiz randomly stops communicating with the LMS (Moodle 3.11.3). Issue appears to occur only on the 2nd attempt at the quiz. Quiz is setup in Storyline to allow 2 attempts but in the LMS to allow unlimited attempts.

What we see from Moodle logs is the learner takes the assessment, fails to pass the required grade and so takes it again. Often that's fine and they pass 2nd time. But on occasion the SCORM stops communicating with the LMS although the quiz progresses as normal with no indication to the learner that there's an issue.  This could be at the start of the 2nd attempt or part way through. We have logs indicating a learner was progressing through 2nd attempt then score stopped. Learner provided screenshot confirming quiz was completed with a pass.

Have setup a test quiz and can get the same to happen - quiz looks to be running fine but nothing logged in Moodle. However, can only simulate by closing course, refreshing browser and going back in. LMS shows SCORM launch but nothing further even though quiz is being taken.

Anyone encountered this and more importantly have a solution?

Screenshot shows test scenario: on question 6 of 10 yet no scored logged in Moodle (yes, refreshed logs!)

12 Replies
Joseph Francis

From Moodle's docs.

SCORM 2004 is not supported in Moodle. Parts of the API have been implemented, but others such as Navigation and Sequencing have not. Development on native SCORM 2004 support in Moodle has stopped. If you require a fully certified SCORM 2004 Player in Moodle, Rustici Software have a Moodle plugin which connects to their commercial SCORM Cloud service turning Moodle into a fully compliant SCORM 2004 LMS.

SCORM FAQ > Supported Versions

Adam Lavery

Thanks. We only recently switched to 2004 to resolve another issue. However, switched the test assessment back to 2.1 and see the same behaviour. Everything fine for 1st attempt but then sporadic recoding of results for subsequent attempts. Same whether using the 2nd attempt in the SCORM or exiting the activity and going back in. Here's log for a subsequent attempt i.e. LMS attempt 2 rather than SCORM retake (which allows 2 goes):

Log attempt 2

Notice just 2 of the 10 scores logged (at 50 & 100 i.e. questions 5 & 10). However, here's an extract of the interactions report:

Interactions

These are questions 4-6, all of which are correct and should have recorded a score. I answered all 10 correctly so there should be 10 scores logged, but only 2 were, It's seeing all the interactions but not logging them all. In this case it logged the passed status hence course shows has passed. In real-world cases, there's no log of status hence remains incomplete even though the learner received a pass.

Seems to me that somewhere between receiving an interaction and recording it something is going wrong.

Anyhow, I think this confirms that the issue is in Moodle and not Storyline so will switch focus to the Moodle question raised. Here if anyone wants to follow: https://moodle.org/mod/forum/discuss.php?d=431273

 

Adam Lavery

The resolution has been to change the course assessment to force a path that ensures the scores are properly recorded. We've disabled the exit option, added a warning on the fail/success layers that they must click a button (to progress to next slide) to correctly record scores. "Exit" slide has actions to forcefully record the completion/passed status and close.

The reason for the missing raw score logs is that the SCORM only periodically reports the interactions back to the LMS (every 30 seconds or so). If multiple questions have been answered between report backs, only the last score is logged. However, the full set of SCORM interactions is passed back and recorded. The missing raw score logs is confusing but a non-issue.

We're assuming this reporting delay introduces a window of opportunity for the learner to close the browser before the final scores and completion status have been reported back to the LMS. Adding a positive action final slide to exit the course, that ensures the status is reported back, seems to do the trick.

Learner may ignore correct path and simply close the browser window before final scores/status is reported back, but then that's on them! 

Paula Thrall

Thanks for your reply, Adam!  Would you be willing to share an example of that positive action final slide with the instructions to forcefully record the completion/passed status and close? Did you use javascript or a built-in storyline trigger to do that? 

I understand that in moodle there is an auto-save delay option that can be set (see attached). Do you or does anyone know if that is for moodle's own quiz settings or does it work for SCORM quiz results, too? Is this the reporting delay you are talking about, Adam?

I've thought of resetting that to only a couple seconds or so (the default is 1 minute), but since I haven't been able to recreate the problem a handful of our users are encountering, will it be a waste of time? Maybe someone else would chime in or tried this?

I really appreciate any help because I'm the only one here at my organization working with all these programs and these forums are the only help I have. Thank you!

Adam Lavery

That auto-save setting looks to be for Moodle quiz activities rather than SCORM. The reporting delay we monitored between the SCORM and LMS was around 30 seconds. 

The way we have tackled this is belt, braces and safety pins! Submit the results twice and explicitly mark course completed/passed on exit after success.

Results slide:

Results Slide triggers


Success layer: add button "Save score and continue" with 

Exit slide: add button "Complete course and exit" with

 

The theory is that on success the player must still be communicating with the LMS to move to the next slide. We save the results on entering the results slide. Then again on moving to next slide and finally explicitly mark as completed/passed when clicking the exit button.

Just fyi, I'm an IT man/developer and have only recently got involved with eLearning in the last few months. Steep learning curve! 

Paula Thrall

Thank you so much! Your screen shots are very helpful.  I have similar object triggers at the end of my storyline courses, but I do not have a specific exit slide with the "Complete course as Complete/Passed" 

How did you determine it was 30 second delay? Did you do some testing to see if adding that extra step was the key? Is is possible that the extra time/steps involved with the extra exit slide helps ensure the retry results autosave? 

Here is my result layer:

Result layer

You didn't show your failure layer, but here is mine.  Maybe there needs to be an autosave delay in the retries?   Or am I overthinking this?

Failure layer

And here is my Success layer:  You can see that in this layer the results must be 100% before they can exit.  If not, they can't even reach this layer.  I will try your method with the save score and continue (to an exit slide) like you suggested and see if that does the trick. 

Success layer

I'll report back here to later if this method works. So far, I've only encountered issues with about 4 people who cannot retry a quiz multiple times in the same attempt--but to me that is 4 too many. So annoying!  I hope to hear from you, too, how your method is working! 

Agree...steep learning curve! I'm currently my own team--I do both the course design and the LMS admin. Our IT people have no idea what I'm talking about. So thanks for being on my extended team!  If you ever need someone to reach out to...feel free! 

Paula Thrall

Adam, I'm following up to my earlier post. To fix the issue in our course with multiple retries not recording, I changed up our quizzes to retry immediately after getting a wrong answer. That way only the correct answers are pushing to the results slide. This has basically rendered the Retry/Review buttons on the Results slide useless.  Since switching over to this method, I've had very few results tracking issues since - Yay! Tracking issues now are as a result of cookies not enabled or out of date browsers. I don't believe my solution is the most effective way to administer a quiz, but for our compliance trainings this is the best option for now.