Stroyline 360 results screen shows score stored on LMS when course is resumed

Hello heroes,

I'd like some advice on a situation we are facing with a course being tested 0n SCORM Cloud. The following is the background:

Course Setup: Approximately 30 slides with a mix of static and interactive content. All slides are set to "Resume from Saved State".

Quiz:

  • A 28 item question bank at the end of slides, set to present 10 random questions in a quiz.
  • All questions have a custom layer that displays correct and incorrect icons in addition to specific feedback text.
  • Each question is allowed 2 attempts.
  • Each question has "Resume from Saved State" to allow reviewing a question when in a session.
  • Results slide has "Resume from Saved State"

Publish settings: Screenshots attached.

SCORM version: 1.2

LMS: Moodle

The following is the scenario where we are facing issues:

  • First attempt: The user completes the course, gets 80%, and exits the course from the results slide using an 'Exit' button, or by closing the window. Status and score is updated correctly on SCORM Cloud.
  • Second attempt: The user opens the course and 'resumes' from prompted. Is shown the results screen. The score displayed is 80%. User exits the course. Status and score remains unchanged on SCORM Cloud.
  • Third attempt: The user opens the course and 'Retakes' the course when prompted. Completes all slides and quiz and get 60%. User exits the course. Status and score remains unchanged on SCORM Cloud.
  • Fourth attempt: The user opens the course and 'resumes' from prompted. Is shown the results screen. However, the score displayed on the results screen now is 80%. 
  • Fifth Attempt: The user opens the course and 'Retakes' the course when prompted. Completes all slides and quiz and get 100%. User exits the course. Status and score remains unchanged on SCORM Cloud.

Questions:

Is the behavior correct?

  1. Shouldn't the results screen in the course (when resumed on the fourth attempt) be showing 60% , as that was the last score that the user had achieved? Are we missing anything here? 
  2. Shouldn't the score get updated to 100% on SCORM Cloud on the 5th attempt?

Any help is much appreciated.

14 Replies
Crystal Horn

Hi there, Sandip. Thanks for sharing that fantastic analysis! I was able to set up a similar project, though it only has 2 out of 3 questions being pulled from the question bank. I hosted in SCORM Cloud here, and I wasn't able to reproduce this behavior using Chrome 73 in Windows.

So that I can isolate some causes for your user, it would be helpful to know the following:

  • What version of Storyline 360 was used to publish this course? Go to Help > About Storyline. We are currently on update 26.
  • What browser is being used to test the course in SCORM Cloud?

Thank you for working through it with us!

Sandip Ubale

Hi Crystal,

Thank you for your response.

We are also on v3.26.18364.0, and are testing on Chrome Version 73.0.3683.103 (Official Build) (64-bit). on SCORM Cloud.

One aspect that has repeatedly cropped up when searching for similar issues in the forum is the 'Suspend Data' size for SCORM 1.2. We are saving the user responses, along with custom feedback layers, for each question. We feel that the suspend data size may have been exceeded due to this, leading to the abnormal behavior. We aren't sure though.

Please let me know if I can provide any additional information.

Looking forward to your comments.

 

Leslie McKerchie

Hello Sandip,

Thanks for the additional information. I do see that we have a similar issue reported:

Resuming a Quiz with question bank more than once does not save the correct status of the quiz. 

This is affecting both where the course is resuming as well as the score.

With your permission, I'd like you to share your project file with our support engineers to further test and confirm. You can share it privately by uploading it here. It will be deleted when troubleshooting is complete.

Sandip Ubale

Hi Leslie,

We can surely share the file for testing, I had raised a support ticket earlier (because I was at my wits' end with an impending delivery). The support case # is 01762981. I'm unsure whether the link you have mentioned will open another case. Please let me know how you would like to go about it.

Sandip Ubale

Hi Phil,

Thank you for your response. This is helpful information. We would like to be completely sure before we revert to our client, and as such would wait for the support team to chime in. 

@Leslie: We have uploaded the file to Jon, from the support team and are awaiting response.

@Phil and @Leslie: Could you care to share your thoughts about the following issue that we are facing now in addition to the SCORM related issue:

SCENARIO: The user attempts the course the first time, fails and exits the course. Then relaunches the course and 'resumes'. Is taken to the results screen (which shows the 'failed' layer). Clicks the 'Retake Course' button on the 'failed' layer, goes through the course again and reaches the Quiz. Answers a few of questions (which are displayed in a randomized manner from a question bank), and clicks the back button on the question slide (a custom back button we have added). Issue: At this point the course freezes and is stuck on the loading screen with no way out but to close the window. This issue happens randomly on any question (so not question specific), and happens randomly with any number of questions (so not at an identifiable instance).

We have never faced such issues earlier, but as every course is unique in its own way, we are unsure what we have done wrong.

Please note that this additional issue has also been highlighted to support. If this requires me to raise a separate ticket, happy to do so; anything to solve this.

Your continued help on this is much appreciated!

Sandip Ubale

So, I've heard from the support team and the following is what they had to say:

"After taking more time to look into the issue in regards to scoring in your course, we have found that this is a bug that currently does not have a workaround.

Until this bug is addressed, once a learner passes the threshold of the minimum required score to pass the course and closes the course window, that score will be sent to the LMS and cannot be changed, either up or down.

The best way around this is that if your learners get a passing score but they are not satisfied with that score, to have them ensure that they retake the quiz before closing the course window."

As of now, I'm guessing I'll need to await a workaround, and in the meantime will discuss alternatives with our customer.

With reference to the other issue regarding the page loading: we're still searching for help on that, and will open another discussion thread if needed.

Thank you everyone for the help with this issue.

Marty D

Folks,

A directly related question - but likely much easier to answer (I just can't figure it out.)

1. Working with a course/quiz/results screen someone else created. Loading course to Docebo.

2. On LMS: Course displays the Results slide upon revisiting course - but Results slide always resets to 0% - even if the learner's score was 100% on previous completion

Am I being thick here by asking - but is there a way/setting you'd use to ensure quiz scores are "remembered" by the SCORM object (SL course)? (The original score is being sent to the LMS, yes.)

Marty

Vincent Scoma

Hi Marty,

Happy to jump in to help! 

Without seeing the file, the results slide resetting when revisiting could potentially be a resume setting for that slide or a potential course trigger. A great place to start is to check the resume behavior for that slide. This resource provides more details on this here: 

If you need our help to take a look at your file, we're always more than happy to help! You can share files in this thread, or you're always welcome to share them privately here. We'll let you know our findings and will delete it after having a closer look!

Marty D

Hi Vincent,

Thanks for the thoughts. That "slide state on revisit" was my first guess as well.

The results slide was set to 'reset to initial state' by the original author ... however, even after I changed that setting to 'resume at saved' the quiz results still reset to 0% (but the 'Retry' button is appearing - as though the Results screen recognize the quiz was taken but isn't capturing the result.)

Project file is attached.

Marty

Leslie McKerchie

Hi Marty,

Thanks so much for the additional details and testing as well as sharing your .story file for us to take a look.

I was able to publish your project file to SCORM Cloud for comparison testing, but I am not experiencing the same results. My score is being retained on the Result Slide when resuming the content. I recorded a quick Peek 360 video here.

Do you experience the same issue when taking a look at my published output here?