Question Effectiveness Analysis

Aug 24, 2015

Hello All,

I've spent a couple of hours trying to figure out the data Storyline interactions write to our LMS and I can't make heads or tails out of it, I'm wondering if someone could help.  We're trying to analyze question and training effectiveness for a few of our courses by analyzing which questions our users get wrong the most often. When using question banks I have no idea how to get back to the question that's being asked.

We get a value that looks like this: "Scene7_QuestionDraw21_Slide6_MultiChoice_0_0" and then the results of that interaction. 

Parsing this string I know we're in scene 7 and are looking at "Slide 6" which is a multiple choice question.  The problem is that due to the "QuestionDraw", "Slide 6" can mean anything... 

Looking at the body of the data, "Slide 6" is sometimes a T/F question or a Multiple Response question... 

Is there any way to unwrap this data?  Why isn't Storyline's actual slide number not indicated anywhere?  Without it the data is useless. What am I missing?


9 Replies
Ashley Terwilliger-Pollard

Hi Peter,

Storyline doesn't include slide numbers on the slides or as a part of the output, and the data we send to an LMS is detailed here, but it'll be up to an LMS to decipher and output the data in a way that is readable to you as an admin. You may also want to look at testing the content at SCORM Cloud as it's an industry standard for testing LMS content. 

Peter Zaidel

Thanks again Ashley,

Let me try to rephrase my question.  Based on your article on what is sent to the LMS, it sends the Question Draw and the Slide #.  Is the Slide # being sent in the order it is in the story or is it the order that it is presented to the user?  If it's in the order it's in the story, that's ok, however if it's in the order it's presented to the user - there is no way to get back to the question since there is no other identifying information being sent about the question itself. 


Ashley Terwilliger-Pollard

Hi Peter,

I did a quick test with LMS debug mode on so I could see what it was sending, and it listed them in the order that you've set - but I also saw the interaction type listed too such as: 

Scene1_QuestionDraw11_Slide2_MultiChoice_ so you may also want to look into if your report is showing the question type as well as that should be reported. For my test purposes I used Scorm 1.2 and uploaded to SCORM Cloud. While viewing the debug log, I also had to scroll far to the right in Notepad and I could see the question text too. 

This discussion is closed. You can start a new discussion or contact Articulate Support.