Storyline review mode not being evaluated properly

Good Afternoon Kristy, and welcome to Heroes!

Most learning management systems (LMSs) consider a course to be in review mode when attempting it again after the completion requirement has been met, and Storyline supports this consideration by default.  If you'd like to change this default behavior, it's best to change it in the LMS.

The best way for you determine if this is a problem with your Storyline content or with your LMS is to test your course in SCORM Cloud, which is an industry-standard tool for testing SCORM output.  If the problem is reproducible in SCORM Cloud, we want to know more about it so that we can take a deeper look at the problem.  However, if the problem is isolated to Desire2Learn, you'll want to continue to pursue a solution with their Support Team.

Please let us know if you need anything else.  Thanks!

Thanks, Justin. That's helpful to know.

Testing in SCORM Cloud, I was able to recreate the same issue I experienced in D2L. That is, after I took the course my completion status was set to Complete, success was Passed, score was 100%. However, I relaunched the course and viewed only the first slide, then closed the course. After closing it, status was set back to Incomplete, success was Failed, and my score was lost.

There's nothing on the first slide that would reset the score/progress, and I don't know of any publish settings that would cause this behavior. Any insights about where I can start troubleshooting this?

~Kristy

Thanks for taking the time to test this out in SCORM Cloud, Kristy.  The fact that this is reproducible there is a really important find!

Could you please send us a copy of your .story project file so that we can take a deeper look at the problem?  Thanks!

Hi Justin,

I am experiencing the same issue with latest and updated version of Storyline + Moodle 2.8

Moodle seems to send the "review" mode to the SCORM package, but it is not displayed in read-only mode but in some kind of "continue last action" mode. And if I choose NO, it reset the last completed attempt.

Any ideas?

Hi Nadav,

It sounds like the course has the "resume" prompt set up to allow the user to chose whether or not they want to resume? You can force them to resume by choosing "Always" instead of "prompt to resume" as described here. 

Hi Nadav!

Did you test your content in SCORM Cloud as well as Justin shared above? This would help us identify if there is an issue with the course or in your LMS.

Hi Ashley, Thank you for your feedback. I actually like the resume feature behavior to function as it is. I only mentioned it since it seems similar to a review mode which I was looking for. but did not know how to activate. Moodle seems to send that parameter to the SCORM package but I do not see anything acknowledging it on the debug window of the communication between the Package and the LMS. What should I look for?

Hi Leslie, I uploaded the package to Rustici's SCORM Cloud and launched it several times. But I can not find the "review" button in SCORM Cloud's feature full  UI. Please advise.

Greetings, Nadav.

In this scenario, each of the cogs in the machine is making decisions:

1. The Learning Management System (LMS) makes a decision on whether or not to consider a course to be in review mode when attempting it again after the completion requirement has been met.  This decision should be exposed as part of the communication that you can examine via LMS Debug Mode, in the form of cmi.mode=normal or cmi.mode=review.
2. The LMS makes a decision on whether or not to supply the course with suspend_data that will allow the course to pick up where the learner left off on the previous attempt.  If the LMS has this suspend_data available, it will generally decide to supply the suspend_data, and this communication can also be observed via LMS Debug Mode.
3. If prompted, the Learner makes a decision on whether or not to resume where he/she left off.  This decision cannot be directly observed via LMS Debug Mode, but it can be inferred based on the course's ensuing behavior.
4. The course makes a decision on whether or not to use the supplied suspend_data to resume the course where the learner left off.  If the course receives this suspend_data from the LMS and the learner makes a decision to resume, the course will generally decide to resume, and this decision can be observed via LMS Debug Mode.

So, Moodle should tell the course about its review mode decision, and it should also send the suspend_data that the course needs in order to resume.

It sounds like you are saying that after a learner meets the completion requirement, attempts the course again, and chooses not to resume where he/she left off, Moodle is modifying the learner's original attempt.  If I've understood you correctly, it sounds like Moodle is probably sending cmi.mode=normal back to the course, and is overwriting the original attempt with the new attempt.  You can confirm or deny this by enabling LMS Debug Mode and observing the resulting LMS Debug Log.

A quick test shows that in contrast, SCORM Cloud seems to opt against exposing its decision on whether or not to consider a course to be in review mode when attempting it again after the completion requirement has been met.  Instead of changing cmi.mode to review on subsequent attempts, SCORM Cloud appears to keep the value at normal and to simply discard any new completion/scoring data that is sent by the course's next attempt.

If you need any help analyzing your LMS Debug Log or testing your .story project file, please feel free to send those items to us via a Support Case, and we'll be happy to take a deeper look.

Have a great day!

Hello Justin,

Thank you for your feedback.

I agree with all the steps you described. This is what I expect from the LMS and the SCORM package. Though I am getting something else...

Here is a link to the package:

https://docs.google.com/uc?authuser=0&id=0BzEUW_tmkNcvWFVxSGRMbUp6Qnc&export=download

Also, "It sounds like you are saying that after a learner meets the completion requirement, attempts the course again, and chooses not to resume where he/she left off, Moodle is modifying the learner's original attempt." - Yes, this is what happens. And looking at the logs (attached at the end) Moodle is sending cmi.mode=review and the package display "Resume dialogue", which seems like a bug on the Package side. correct me if I am wrong.

By looking at the logs, It seems Moodle is sending the correct review mode and the suspend data.

Moodle SCORM 1.2 API Loaded, Activity: לומדת SCORM בקישור מגוגל מסמכים, SCO: example_SCO
Thu, 23 Apr 2015 07:46:46 GMT: LMSInitialize("", "") => 0 scoid = 207
Thu, 23 Apr 2015 07:46:46 GMT: LMSGetValue("cmi.core.lesson_mode") - review => 0 scoid = 207
Thu, 23 Apr 2015 07:46:46 GMT: Commit("", "") => 0 scoid = 207
Thu, 23 Apr 2015 07:46:46 GMT: LMSCommit("AJAXResult", "true") => 0 scoid = 207
Thu, 23 Apr 2015 07:46:46 GMT: LMSCommit("result", "true") => 0 scoid = 207
Thu, 23 Apr 2015 07:46:46 GMT: LMSCommit("errorCode", "0") => 0 scoid = 207
Thu, 23 Apr 2015 07:46:46 GMT: LMSGetValue("cmi.suspend_data") - 2A5890807060ts1001311k01011011110121101311A101^1^1^n6jGAmRC35TY.5kdR9YC31t41^1^0~2f4~2G1cb101001a1a103Bu6Oi_8040$1g1171U4a5_$3480i_8040$0g117165t5_$34803400yw2fiq340034003400340097w020111^000000~2i1cb101001a1a103dT0q3400978030111^348034803400yw2eiq340034003400340097w020111^000000~261cb101001a1a1039F0e978000111^3480yw2chq340034003400340097w020111^00000002100 => 0 scoid = 207
Thu, 23 Apr 2015 07:46:46 GMT: LMSGetValue("cmi.core.lesson_status") - passed => 0 scoid = 207

I have tried to test this package on SCORM Cloud, but I can not figure out how to open the package in review mode after I finished a successful attempt. Please advise.

Thank you for all your support!

Nadav

Thanks for the additional details, Nadav.

Resume and review are two different functions, so I would not expect Moodle's sending of cmi.core.lesson_mode=review on subsequent attempts to change the course's resume behavior.  Since Moodle still seems to send the cmi.suspend_data necessary to let the learner resume, I would expect the course to continue to give the learner the opportunity to pick up where he/she left off.

However, what I would not expect is for Moodle to send cmi.core.lesson_mode=review to the course, and for the course to still send updated cmi.core.lesson_status or cmi.core.score.raw data back to Moodle.  When Storyline sees cmi.core.lesson_mode=review, that should be our signal to be careful to not change the learner's status or reset the score within the LMS.  I don't think you've included the entire LMS Debug Log above, and I would be very interested in seeing more if you find that a Storyline course is sending status/score information back to the LMS when in review mode.

I have been unable to observe SCORM Cloud sending cmi.core.lesson_mode=review to a course on subsequent attempts, and yet SCORM Cloud still does not modify the original attempt.  I wonder if SCORM Cloud is not using cmi.core.lesson_mode in the same way, and is instead simply discarding any new status/score data that is sent by the course's next attempt.

I think we should focus on your course's performance in Moodle, since Moodle does seem to be using cmi.core.lesson_mode as prescribed by the SCORM 1.2 specification.  Can you supply a complete LMS Debug Log that shows Storyline sending status/score information back to the LMS when in review mode?

Thanks!

Hi Justin,

I have setup a public Moodle development environment where you (and I) can explore this issue.

Here is a link to the Moodle SCORM activity:

http://104.155.72.74/mod/scorm/view.php?id=60

You can login as Student: username: student, password: student

Or as a Teacher (to reset attempts, and turn different settings on or off as you need)

username: teacher , password: teacher

I have setup an automatic debug mode, where a new tab is open in the browser when the activity is started, in which, all the debug log is displayed.

To enter the activity in "review" mode after it is "complete"ed, you have to click the SCO title under the "Content" section, on the launch page of the activity. (same link as above).

As a teacher, you can update activity's settings:

http://104.155.72.74/course/modedit.php?update=60&return=1

Or view user's reports (after they finish any attempt)

http://104.155.72.74/mod/scorm/report.php?id=60 

As a teacher, Feel free to upload any SCORM 1.2 activity that suppose to work properly (with functional review mode) to the course and test it too.

To upload a SCORM activity: (1) login as teacher. (2) Turn editing ON. (3) Drag a SCORM zip file onto the course front page (4) Choose "unzip as SCORM activity".

Feel free to play around with all the settings. The entire server is ephemeral. 

If you have any question, I will be glad to help.

Thank you for your time, helping me (us) on this.

Nadav

Hello again, Nadav.

I published this .story project file for SCORM 1.2, enabled LMS Debug Mode, and uploaded the resulting zipped, published output to Moodle, which you can access here.  I took the course twice (once in normal mode and once in review mode), and found that:

1. During review mode, Storyline successfully resumed the course where the learner left off, as expected.
2. During review mode, Storyline was careful to not change the learner's status or reset the score within the LMS, as expected.

If you'd like to watch this happening, you can open the resulting LMS Debug Log and observe the following:

• On Line 52, Moodle informs Storyline that we are in review mode.
• On Lines 118-119, Storyline declines to change the learner's status.
• On Lines 385-390, Storyline declines to changed the learner's score.

If Moodle is resetting the learner's attempt based on this scenario, you'll want to take that up with their support team.  All indications are that Storyline is doing the right thing here.

Good luck with your project!

Hello Justin,

Thank you for your detailed explanation, throughout the thread!

You are perfectly right. Choosing the "resume" and then "review", no student data is overwritten.

I guess I was confused with the "resume where you left" dialogue.  I expected the Package to display "review" mode directly and not enable the student to choose to resume the course after the attempt was "complete"ed.

Again, thank you!

Nadav

Hello,

I work on an e-learning development company. We build 1.2 SCORM packages and deploy them to our costumer's LMS.

I'm running through a problem similar to the mentioned above but the scenario is: Our customer's LMS send us lesson_mode=review after we completed the course. But it resets all the other SCORM variables to default values like lesson_status=not attempted and suspend_data=''. Chek the debug provided by Storyline package, below:

35:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - SCORM_GrabAPI, returning
36:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - Calling LMSInit
37:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - In SCORM_CallLMSInitialize
38:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - In SCORM_GrabAPI
39:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - SCORM_GrabAPI, returning
40:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - Calling LMSInitialize
41:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - strResult=true
42:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - Returning true
43:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - In SCORM_GetLessonMode
44:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - In SCORM_ClearErrorInfo
45:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - In SCORM_CallLMSGetValue strElement=cmi.core.lesson_mode
46:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - In SCORM_GrabAPI
47:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - SCORM_GrabAPI, returning
48:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - Call LMSGetValue
49:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - strResult=review
50:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - intSCORMError=0
51:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - Returning review
52:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - strLessonMode=review
53:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - Returning Review
54:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - Calling InitializeExecuted with parameter-true
55:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - In InitializeExecuted, blnSuccess=true, strErrorMessage=
56:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - In IsLoaded, returning -true
57:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - In GetDataChunk
58:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - In ClearErrorInfo
59:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - In IsLoaded, returning -true
60:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - In SCORM_GetDataChunk
61:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - In SCORM_ClearErrorInfo
62:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - In SCORM_CallLMSGetValue strElement=cmi.suspend_data
63:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - In SCORM_GrabAPI
64:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - SCORM_GrabAPI, returning
65:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - Call LMSGetValue
66:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - strResult=
67:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - intSCORMError=0
68:Thu Oct 05 2017 11:56:59 GMT-0300 (Hora oficial do Brasil) - Returning

We block the user progress, only allowing them to proceed if they watched the complete slide animation or viewed all of slide's items. We do this by using many Storyline variables. But if there's no suspend_data, there's no tracking variables for we to unblock the already passed slides.

Is there a way for we to check the lesson_mode inside the slides and unblock them for the users who have completed the course, already?

Thanks in advance. Best regards.

Hi Matheus, 

There isn't a way in Storyline to modify this - so I hope that someone in the community is able to share ideas with you!