Storyline Suspend Data Compression

Nov 09, 2018

Good day!

As some of us know SCORM 1.2 only limits its suspend data for up to 4096 characters. Storyline (360) compresses its data (e.g. SL variables and such) in order to fit the limitations of the suspend data. There must be an underlying decompression algorithm or its own unique reader on SL to read the suspend data. 

My question is when this compressed suspend data becomes decompressed, would there be a possibility of it hitting the 4096 limit?

60 Replies
Ashley Terwilliger-Pollard

Hi Christian, 

The suspend data is compressed and not human readable, but it's still something that your  Learning Management System (LMS) would be able to read and decipher. I haven't seen anyone crack the algorithm or determine a way around it though. 

If you can share a bit more about what you're hoping to accomplish or any trouble you've run into - I or others in the ELH community may be able to point you in the right direction. 

Christian Ompad

Hello Ashley,

I am trying to figure out a way to resolve an unwanted behavior in a course I'm working on. Even when its completed, it would always resume to an exam question. I would like it to resume on last page that the user was in when they completed the course. So far every topics in ELH and help from support have lead me to a conclusion that I am facing a suspend_data problem.

These are the suggested solutions that I have drawn from the discussion:

1. Publish to SCORM 2004 3rd/4th ed.

2. Minimize/delete slides.

3. Only set important slides' settings to resume to saved state and set others to reset to initial state to minimize sending data to suspend_data.

1 and 2 are not an option since the client's LMS only supports SCORM 1.2 and everything in the course is based on their specs. Therefore, it leaves me with option 3 but I have made little progress as to how to make this random behavior not be so random. 

Leslie McKerchie

Hi Christian,

If you have a large course that exceeds suspend data limits, here are some suggestions for correcting it:

  • Disable the resume feature in Storyline.
  • Reduce the number of slides until the resume feature works as expected. The limit will vary, depending on a variety of factors. You'll need to test your content in your LMS to verify.
  • Republish your course for SCORM 2004 3rd Edition or 4th Edition, both of which support much longer suspend data.

There are some community ideas shared here that may be helpful to you as you research what may work best for your situation and hopefully others in the community will chime in to help you out.

Christian Ompad

Hello Zoe,

Since our LMS only uses SCORM 1.2, I had to make do with the 4096 character limit (unless you republish to SCORM 2004 3rd or 4th edition). While the solution in #3 worked for me, there is no actual workaround with the limit. So basically what happened is, I found out that Storyline's Slide Setting "Resume to saved state" takes up additional suspend data and all I did was to limit its usage.

I highly discourage using it on slides that are "media-rich"; that is, slides that have a lot of video, graphics or sounds in them. Instead, for these kind of slides I used the "Reset to initial state" option this helped me save some of those precious suspend data.

 

David Hansen

If you are willing to open and manipulate files in the SCORM package, you can apply an easy patch that will add data compression to the suspend data sent to the LMS.  Note: Articulate suspend data is NOT compressed.  It is, however, not human readable.  But, it is VERY compressable - typically achieving a 10:1 ratio using a typical zlib compression method.

Note: the following suggestion is only for people comfortable with unzipping/zipping their SCORM package and doing basic edits to text-based xml, html and javascript files.  If this is not you, then you should not consider doing this or find someone that can help you with it.  A good xml/html/js savvy editor is also beneficial.  The brackets editor is a good example.

What you need to do:

1) Obtain the pako.min.js package. This is an open-source, well-developed and reviewed, time-tested zlib compression library written purely in Javascript.  You can google it and download just that file or download the latest version right from the repository using this link: pako.min.js.   You are now going to add this file into your SCORM package (zip archive). 

2) Unzip your SCORM course into a directory and change your working directory there.

3) Put a copy of the pako.min.js file into the lms/ subdirectoy.

4) Next edit index_lms_html5.html and search for "lms/API.js".  You should find something that looks like this:

<script src="lms/API.js" charset="utf-8"></script>

Then add this new line after that line:

<script src="lms/pako.min.js"></script>

Save the changes.

5) Next edit imsmanifest.xml, and go to the end of the file.  Just before the line </resource>, add a new line with:

<file href="lms/pako.min.js" />

Save the changes.

You have now successfully added the zlib compression library into your SCORM package.   All you need to do now is modify the routines that are used to send and receive the suspend data with the LMS.  To do that:

6) Edit the file lms/API.js

Search for "function GetDataChunk".  Replace the line containing return objLMS.GetDataChunk(); with the following lines:

try {
var strDataC=objLMS.GetDataChunk();
WriteToDebug("GetDataChunk strDataCompressed="+strDataC);
var strData=window.pako.inflate(atob(strDataC), {to: 'string'});
WriteToDebug("GetDataChunk strData="+strData);
return strData;
} catch (err) {
SetErrorInfo(ERROR_INVALID_RESPONSE, "DataChunk Inflate error");
return "";
}

Then scroll down a bit to the next function which should be "function SetDataChunk".  Replace the line containing return objLMS.SetDataChunk(strData); with the following lines:

try {
var strDataC=btoa(window.pako.deflate(strData, {to: 'string'}));
WriteToDebug("SetDataChunk strDataCompressed="+strDataC);
return objLMS.SetDataChunk(strDataC);
} catch (err) {
SetErrorInfo(ERROR_INVALID_RESPONSE, "DataChunk Deflate error");
return "";
}

Save your work.

At this point you are now done with the modification to add compression to your suspend data, so you can now:

7) Zip the contents of your SCORM package back up.

Some caveats about this modification: the format of the Articulate suspend data is a sequence of information that matches the sequence of slides in your course.  If the data happens to get truncated, it is possible for Articulate to still process the data up to the point of the truncation and resume up to the last slide of data.  This means resuming will still "kind of" work, just not exactly.  However, if the compressed data gets truncated, the compression algorithm will fail completely and ALL the resume data will be thrown out (and you'll resume at the first slide).  For me, this is a more than worthwhile trade-off, especially since in my experience, I typically have been seeing a 10:1 reduction in suspend data size - this means 30KB of suspend_data size will now be just 3KB and less than the default SCORM 1.2 size limit for suspend_data (4096 bytes).

Good Luck.

As to why Articulate development hasn't just added something like this to their course exporter?  Anybody's guess, I suppose...

David Hansen

Uh, yes.  We have been using this patch in production for about 6 months now on all of our Articulate-authored courses (both for our courses that are hosted on Cloud Scorm as well as for a number of customer-hosted LMS systems.  The customer-hosted LMS systems were the primary instigation for creating this patch since we seem to encounter quite a few of them that either do not support SCORM 2004 (where the suspend_data default size was increased to 32kb) and/or they do not have the ability to override the default size for suspend_data. FWIW, the Rustici SCORM driver does allow configuring/overriding the suspend_data size.  However, we have a couple of courses that were resulting in suspend_data sizes exceeding 50kb. After compression. these courses were then saving suspend_data that was just barely under 4k!

Nathan Hartwick

Thanks for info David. Was this patch created in part to help solve freezing or forever loading screens upon resume?  Have you tested this in a version of Moodle? Something I run into occasionally when using SCORM 1.2 in Moodle is situation where the activity won't load (loading dots) when trying to pull the resume data back in so then the attempt in Moodle needs to be reset to unlock the activity. I can't really recreate the issue consistently enough to troubleshoot and it only happens to maybe 1 in 100 users. Still very frustrating.

David Hansen

No, not exactly...   However, I have definitely encountered problems with some content authoring tools and certain LMS's where certain characters in the suspend_data cause problems with the LMS.

I first started working with the lower-level SCORM Javascript interface code (ADLnet's opensource version) when I needed to address a similar issue with a large client of ours.  The client wasn't going to change or upgrade their LMS system and we couldn't change the content authoring tool since it we were just a distributor.   What I discovered in that particular situation was that specific LMS imlementation of the SCORM interface was doing a Javascript eval() to save the suspend_data.  So any Javascript escape characters were then causing problems.  To address this, I simply wrapped the suspend_data with a btoa(), atob() as it was being sent to the LMS.  Those are native Javascript functions to convert from binary to base64 and vice versa).   This addressed that problem and the client was ecstatic to have the course working with their LMS.

The story then progresses when we had another large client that had another older LMS.  The LMS only supported SCORM 1.2 and could not override the size of the suspend_data from the 4096 bytes specified in SCORM 1.2 for suspend data.   We had authored this particular course in Articulate and the suspend_data size towards the end of the course was averaging 8-12 kbytes (the course has since grown and now averages around 24 kbytes).   The behavior an Articulate course typically exhibits when this limit is reached is to just resume at whatever previous location is represented by that first 4 kbytes.  Since we were invested in Articulate for this course, and the client couldn't just upgrade their LMS, we needed to figure something out.  When I looked at the LMS api within Articulate and recognized that it was pretty much based on the ADLnet opensource, I knew exactly how to handle it.

Now, I don't have any experience with Moodle.  We either support customer's LMS systems or our own that is based on cloud.scorm.com.  And I haven't had to deal with any customer's that are using Moodle.  I don't know if that's just because we haven't had any customer's using Moodle or whether Moodle generally works just fine and thus we haven't had any reported issues.   What I do know is that the SCORM 1.2 specifications states suspend_data must be a "A set of ASCII characters with a maximum length of 4096 characters".   Articulate does use some uncommon characters like  ^ and ~.  Those particular characters should not cause any Javascript problems, but then again it's hard to say how Moodle is implemented and what their interpretation of "ASCII characters" really is.

Lastly, I will say, that this patch does encode the compressed data in base64, so it should be very safe with any LMS.  It might even address your issue with Moodle.  Base64 is a very common standard for encoding to a minimum set of "safe" characters (that is characters that are common to most encodings and printable).   You can read more about that here: https://wikipedia.org/wiki/Base64

David Hansen

Oh, and I can say that I have run into issues like you describe with the stuck loading prompt in the past.  Usually it has turned out to be some odd event that failed upon resuming.  One of the best troubleshooting things in this situation is to have the user who experienced the problem (loading stuck) open their browser developer console (usually Ctrl-Shift-I on most browsers or right click then inspect).   All the browser errors are logged to that console, and though there are often lots of warnings and other errors going on, there may be a tell-tale error message that could help you down the path to a solution (or at least with what's going on that's preventing it from completing the resume).

Nathan Hartwick

Thanks for the in-depth answer. I'm definitely going to test the compression patch for suspend data in Moodle using SCORM 1.2. As for the odd loading event, I was able to inspect a frozen activity and in console I was shown an error that was not thrown in the same activity that's loading for another student. I'm currently chasing this down and it seems promising. Might fix a bug that has been an annoyance for some time.

Nathan Hartwick

To provide more information about my freeze error for SCORM 1.2 in Moodle, I found that if a user (myself for testing) answers one of the essay questions in the activity and the user's response has an apostrophe in it (conjugation or used as a single quotes) the activity freezes on the loading dots when resumed without the resume button loading (modern player/html5). For some reason the single quote/apostrophe is being read as code and not as punctuation because the student_response is closed with single quotes:

cmi.interactions_0.student_response = 'I'm gonna try with apostrophe';

Not even sure where/how to tackle this issue. It could be how it's pushed from Storyline to Moodle via SCORMfunctions.js or it could be how the suspend data is formatted and pushed to back to the activity.

David Hansen

Yes, that is an excellent example of the issue I encountered with one of our customer's LMS systems but was dealing with the suspend_data specifically.  The interface between the SCORM layer and that LMS system was doing a Javascript eval.  So, when you do something like eval("cmi.interactions_0.student_response = 'I'm gonna try with apostrophe';") that will then lead to a Javascript error because there is extraneous characters following the variable assignment (in this case: just the 'I').    However, I wouldn't think that should effect reloading, but maybe it does.  I haven't exactly studied whether Articulate resends interaction data from the LMS as it resumes - but I could see that if it did, it might present a problem upon reload.  Anyway, good luck!

David Hansen

Well, I would think one of two possibilities is going on: either the LMS is truncating or corrupting your suspend_data; or you may have made a very small typo or error in your implementation of the above. 

The first thing to check would be to open the browser developer console and then check for any errors.  If you accidentally made any mistakes when patching the above files, those errors would be reported in the developer console.

The other thing you can do to help debug, is to look at the LMS interface debug window.   To see the LMS debug window, you simply have to make a couple of changes in the file lms/Configuration.js.  First, make sure the first line has blnDebug = true;  If it's false, then just change it to true.   Then scroll down to or search for SHOW_DEBUG_ON_LAUNCH = false; and change that to true.  You will then need to re-zip the package and re-upload it to the LMS.   This simple change will then cause an additional window to open up when you launch the course that has all the debug logs for the LMS interface.

Note: if blnDebug is already set to true, then you can also open the debug window from the browser developer console of an already launched course.  To do that, go to the console section of the developer console, and type: window.SearchParentsForContentRoot().ShowDebugWindow();

Within the debug window, what you need to do is to find and compare the value of the suspend_data when SetDataChunk() is called versus the value when resuming from GetDataChunk().   If these values are different then your LMS is likely truncating or corrupting the data.

I hope that helps with your debugging.

s amer

Thank you David for your help and advice!

I have now successfully implemented the code and tested it on SCORM CLOUD and on our LMS (Webanywhere) Moodle version 1.6.

However this is to be implemented on  my clients system who use SAP success factors.

I am slightly apprehensive to give a 100% guarantee that the problem won't occur on their system but am struggling to find out the exact specification of the LMS but I believe it is a custom based solution (if anyone reading this knows?)

I have suggested that they complete a test but my team want to just offer a solution.

I have explained that although we have tested, it still needs to be tested on the Clients LMS as the

LMS can still truncate the data

Would this be a fair analysis? I appreciate your advice, although i'm confident with editing the package, when it comes to API's I am reaching my depths of understanding.

I'm still unclear as to whether this is specific to Articulate Storyline or this happens with any software of this type?

Thanks in advance

regards 

 

 

Thanks in advance

David Hansen

I do have a bit of experience dealing with customers using SAP Success Factors.  It has definitely been one of the more pain-in-the-derrière LMS systems.   And yes, SuccessFactors seems to be fixed in time on SCORM 1.2 (and doesn't even support recording interaction data).   Anyway, this fix should still help and work fine. But like you, and with my previous SuccessFactors experiences, I would be hesitant to give a 100% guarantee, at least without someone first testing your course on your client's specific version of SuccessFactors.   Though, I do know that by using base64 encoding, the suspend_data itself should be fine with SuccessFactors as long as it's not truncated.  What would cause me reservation is just the broad number of issues I've encountered with SuccessFactors.

As to whether the suspend_data actually gets truncated or not, that should depend solely on whether your course creates more than 4096 bytes of compressed data.  What we usually do is test our courses on cloud.scorm through to the last slide and then exit (causing a suspend).  We then look at the registration data on cloud.scorm and examine the size of the suspend_data (typically, you can just copy the result into a web-based byte/string counter and see how long it is).  If it's close to 4096, then we have concerns.  If it's smaller by at least a 10% margin, then we feel more confident that everything will be fine.  Otherwise, we have to figure out how to trim out some of the items that cause saved state to be larger (typically we pair down the number of interactions).

I hope that helps!   And good luck!

s amer

Thanks David for your help and advice. It has been tested on SuccessFactors and has no issues so far!

I have also tested on SCORM cloud.

However we have just uploaded a replacement module on Webanywhere (moodle 1.6) and got the attached error?

The first test we did worked ok. Have you seen this before?

regards

sharon