An Assessment Journey #1: What we do in the shadows

I've just finished coding a fairly sophisticated assessment product using Storyline 360 and thought I would take the opportunity to share through a series of posts some of the lessons learnt, tips and tricks and bits of code I have shamelessly stolen from other posters on the forum.

This first tip is about using a black screen that appears to the user as nothing more than a "fade through black" transition from one page to another within the published application. 

If you want to learn more about just that tip just scroll down to the "shadows" sub-heading - but it you want to learn more about how sophisticated this assessment is read on...

The Assessment

The requirement was to build and publish an online assessment within a SCORM 1.2  LMS.  So far so boring.  The next requirement was that whilst the first LMS would be SCORM 1.2 there were plans to move to an xAPI LMS in the future - so I needed a tool that could publish in SCORM and subsequently re-publish in xAPI - hence I chose Storyline 360. 

Republishing from SCORM to xAPI is never going to be as simple just choosing the alternative protocol from the publish drop-down menu - but it sure makes it possible to consider it.  I'll be happy to share more on this when I get to it.

The next requirement was that the assessment was to be a situational judgement assessment. 

This means that the user is posed a situation (i.e. "You are flying a plane and the fuel gauge alerts you that you are low on fuel - what do you do?")  A number of responses are then proposed (i.e. "Ignore it", "Land immediately", "Jump out of plane"....you get the idea).  For each possible response the learner indicates either AGREE ( i.e. it is an appropriate response), DISAGREE (i.e. it's never going to be appropriate to do that) or DON'T KNOW (i.e. I literally don't know if that is a right or wrong response).

The assessment in question has about 40 situations, and each situation has five possible responses.  So you the learner will be required to make about 200 responses (i.e. 40 x 5 = 200).  OK - so far nothing too difficult to create.

The next requirement was that for each response in addition to indicating whether they AGREED,DISAGREED or DID NOT KNOW learners also had to submit a score on how confident they were that their answer was correct (i.e. 0% = not at all confident, 100% totally confident). 

I'll talk more about this confidence rating in a subsequent post.  The addition of the confidence rating requirement meant that the standard quiz engine in Storyline 360 couldn't be used; for each of the 200 responses I needed to track two values - whether the learner was correct or incorrect AND what value did they submit as their confidence score.

The next requirement was that the 200 questions were spread across six topics.  Some topics had five situations - some had more.  So within my assessment I needed to link each question with its topic (i.e. T1, T2,...T6). 

To pass a topic the learner had to score at least 80% correct for all questions linked to that topic. 

To pass the assessment ALL topics had to be passed (i.e. to pass a user had to score at least 80% across ALL six topics).

AND - if they failed a topic (or any number of topics) the learner could re-attempt the assessment BUT they only had to answer questions related to topics that they had previously failed. 

So if I failed TOPIC 1 and TOPIC 5 - but passed all the others (T2, T3, T4 & T6) when I redo the assessment I only see questions related to Topic 1 and Topic 6. 

The assessment "remembers" which topics I have passed previously and doesn't present any questions linked to topics that I have already passed on previous attempts.  I only see questions related to topics that I have not passed.

Final requirement (well for this post) was that the learners only get three attempts to pass the assessment.  If they fail on the third attempt the assessment get locks automatically and cannot be unlocked unless a secret 16 digit code is entered in an unlock screen by the user.

Happy to report that all of this has been achieved using Storyline360 and published successfully within SCORM 1.2.  To date (July 2019)  there have been over 5000 registrations globally.

In subsequent posts I'll talk through the various tricks and tips I used to create this mammoth assessment - and happy to answer any questions or (even better) learn how I could have made it even easier to create).

What we do in the shadows

This first tip is about using a black screen that appears to the user as nothing more than a "fade through black" transition from one page to another within the published application.

This is often handy when you need to crunch some code to calculate a variable value that you might subsequently need on the next page for display or for other calculations.

The learner sees a black screen, but if you set it up to automatically transition to next screen within less than a second (say 0.50) then learners don't even realise there was a black page being displayed. 

It literally looks like a natural fade from page 1 to page 3 - page 2 (the black page) appears and disappears in the blink of an eye.

OK - so why would you want to do this?  In my assessment I wanted to display the user's name in the landing page; but to do this I had to first capture the user's name from the LMS using some javascript.  To enable this scrip to run I created a black page as the first screen, it displays for 0.50 of a second to enable the script to run and "grab" the user's name and store it as a storyline variable.

This is the script I run comes from here: 

https://community.articulate.com/discussions/articulate-storyline/retrieve-lms-user-name-as-variable

By showing my black page I can run the script without the user knowing and then show page 2 as if it is the very first page with a personalised welcome message using their name.

I also use this trick within a module if I need to calculate scores or other variables before deciding which page they should branch to next.

A simple trick but very handy.